IMPROVED CODING IN ADAPTIVE DEVIATION BY CROSS-COMPONENT SAMPLING

Abstract

An electronic device that executes a video decoding method, which comprises: receiving, from the video signal, an image frame that includes a first component and a second component; determining a classifier for the first component based on a first set of one or more samples of the second component associated with a respective sample of the first component and a second set of one or more samples of the first component associated with the respective sample of the first component; determining a per-sample deviation for the respective sample of the first component according to the classifier; and modifying a value of the respective sample of the first component based on the determined per-sample deviation.In some modalities, the first component is a luminance component and the second component is a chrominance component, and the classifier is determined by a combination of band deviation classifier and edge deviation classifier.

Description

IMPROVED CODING IN ADAPTIVE DEVIATION BY CROSS-COMPONENT SAMPLE FIELD OF INVENTION

[0002] This application generally relates to video encoding and compression and, more specifically, to methods and apparatus for improving the encoding efficiency of both luminance and chrominance. BACKGROUND OF THE INVENTION

[0003] Digital video is compatible with a variety of electronic devices, such as digital televisions, laptops and desktops, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing devices, and video streaming devices, among others. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video coding standards include Versatile Video Coding (WC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which were developed by a joint team comprised of ISO / IEC MPEG and ITU-T VCEG.AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as the successor to its earlier VP9 standard. Audio Video Coding (AVS), which refers to the digital audio and video compression standard, is another series of video compression standards developed by the Chinese Audio Video Coding Standards Working Group.

[0004] Video compression typically includes spatial (intra-frame) and / or temporal (inter-frame) prediction to reduce or eliminate redundancy inherent in video data. For block-based video coding, a video frame is divided into one or more segments, and each segment contains multiple video blocks, which can also be referred to as tree coding units (CTUs). Each CTU can contain one coding unit (CU) or be recursively split into smaller coding units until the predefined minimum coding unit size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs) and one or more prediction units (PUs). Each CU can be encoded in intra-, inter-, or IBC modes.Video blocks in an intracoded (I) segment of a video frame are encoded using spatial prediction with respect to reference samples in adjacent blocks within the same video frame. Video blocks in an intercoded (P or B) segment of a video frame can use either spatial prediction with respect to reference samples in adjacent blocks within the same video frame or temporal prediction with respect to reference samples in other previous and / or future reference video frames.

[0005] Spatial or temporal prediction based on a previously encoded reference block, such as a contiguous block, results in a predictive block for encoding the current video block. The process of finding the reference block can be performed using a block-matching algorithm. Residual data representing pixel differences between the current block to be encoded and the predictive block are referred to as the residual block or prediction errors. An intercoded block is encoded according to a motion vector pointing toward a reference block in a reference frame that forms the predictive block and the residual block. The process of determining the motion vector is typically referred to as motion estimation. An intracoded block is encoded according to an intra-prediction mode and the residual block.For greater compression, the residual block is transformed from the pixel domain to a transform domain, for example, a frequency domain, resulting in residual transform coefficients that can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be explored to produce a one-dimensional vector of transform coefficients, which can then be encoded by entropy into a video bitstream to achieve more uniform compression.

[0006] Subsequently, the encoded video bitstream is stored on a computer-readable storage medium (e.g., flash memory) that can be accessed by another electronic device with digital video capability or transmitted directly to the electronic device wirelessly or via cable. The electronic device then performs video decompression (which is the opposite of the video compression described above) by, for example, parsing the encoded video bitstream to extract syntax elements from the bitstream and reconstructing the digital video data to its original format from the encoded video bitstream based, at least in part, on the syntax elements extracted from the bitstream, and displays the reconstructed digital video data on a screen of the electronic device.

[0007] With digital video quality ranging from high definition to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded increases exponentially. This presents a constant challenge in terms of how to encode / decode video data more efficiently while preserving the image quality of the decoded video data. SUMMARY OF THE INVENTION

[0008] This application describes implementations related to the encoding and decoding of video data and, more particularly, methods and apparatus aimed at improving the encoding efficiency of the luminance and chrominance components, including improving encoding efficiency by exploring the cross-component relationship between the luminance component and the chrominance component.

[0009] According to a first aspect of this application, a video signal decoding method comprises: receiving, from the video signal, an image frame that includes a first component and a second component; determining a classifier for the first component based on a first set of one or more samples of the second component associated with a respective sample of the first component and a second set of one or more samples of the first component associated with the respective sample of the first component; determining a per-sample deviation for the respective sample of the first component according to the classifier; and modifying a value of the respective sample of the first component based on the determined per-sample deviation. In some embodiments, the first component is a luminance component and the second component is a first chrominance component.In some modalities, the classifier is determined by a first subclassifier and a second subclassifier, wherein the first subclassifier is determined by dividing a first dynamic interval of values ​​from the first set of one or more samples of the second component into a first number of bands and by selecting a band based on an intensity value from the first set of one or more samples of the second component, and the second subclassifier is determined based on the direction and strength of the boundary information from a first subgroup of the second set of one or more samples of the first component.

[0010] According to a second aspect of this application, an electronic apparatus includes one or more processing units, memory, and a plurality of programs stored in memory. The programs, when executed through one or more processing units, cause the electronic apparatus to perform the video signal encoding method described above.

[0011] According to a third aspect of this application, a non-transient, computer-readable storage medium stores a plurality of programs for execution through an electronic apparatus having one or more processing units. The programs, when executed through one or more processing units, cause the electronic apparatus to perform the video signal encoding method described above.

[0012] According to a fourth aspect of the present application, a computer-readable storage medium stores therein a bit stream comprising video information generated through the video decoding method, as described above.

[0013] It shall be understood that both the preceding general description and the following detailed description are examples only and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are included to enable a better understanding of the implementations and are incorporated into this document and constitute a part of the specification, illustrate the implementations described and, together with the description, serve to explain the underlying principles. Similar reference numbers refer to the corresponding parts.

[0015] Figure 1 is a block diagram illustrating an example of a video encoding and decoding system according to some implementations of the present description.

[0016] Figure 2 is a block diagram illustrating an example of a video encoder according to some implementations of the present description.

[0017] Figure 3 is a block diagram illustrating an example of a video decoder according to some implementations of the present description.

[0018] Figures 4A to 4E are block diagrams that illustrate how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some implementations of the present description.

[0019] Figure 5 is a block diagram representing the four gradient patterns used in Adaptive Sample Deviation (ASD) according to some implementations of the present description.

[0020] Figure 6A is a block diagram illustrating the CCSAO system and process being applied to chrominance samples, as well as the use of DBF Y as the input, according to some implementations of the present description.

[0021] Figure 6B is a block diagram illustrating the CCSAO system and process being applied to luminance and chrominance samples, as well as the use of DBF Y / Cb / Cr as the input, according to some implementations of the present description.

[0022] Figure 6C is a block diagram illustrating the CCSAO system and process that can operate independently according to some implementations of the present description.

[0023] Figure 6D is a block diagram illustrating the CCSAO system and process that can be applied recurrently (2 or N times), with equal or different deviations, according to some implementations of the present description.

[0024] Figure 6E is a block diagram illustrating the CCSAO system and process that are applied to the AVS standard, along with Enhanced Sample Adaptive Deviation (ESAO), according to some implementations of the present description.

[0025] Figure 6F is a block diagram illustrating the CCSAO system and process that are applied after SAO according to some implementations of the present description.

[0026] Figure 6G is a block diagram that illustrates that the CCSAO system and process can operate independently without CCALF, according to some implementations of the present description.

[0027] Figure 6H is a block diagram illustrating the CCSAO system and process being applied together with the Cross-Component Adaptive Loop Filter (CCALF), according to some implementations of the present description.

[0028] Figure 7 is a block diagram illustrating a sampling process that uses CCSAO according to some implementations of the present description.

[0029] Figure 8 is a block diagram that illustrates that the CCSAO process is interleaved with the vertical and horizontal unblocking filter (DBF) according to some implementations of the present description.

[0030] Figure 9 is a flowchart illustrating an example of a video signal decoding process that uses cross-component correlation according to some implementations of the present description.

[0031] Figure 10A is a block diagram showing a classifier that uses different positions of luminance (or chrominance) samples for C0 classification according to some implementations of the present description.

[0032] Figure 10B illustrates some examples of different shapes for luminance candidates, according to some implementations of the present description.

[0033] Figure 11 is a block diagram of a sample process that illustrates that all co-located and contiguous luminance / chrominance samples can be fed into CCSAO classification according to some implementations of the present description.

[0034] Figure 12 illustrates examples of classifiers by replacing the co-localized luminance sample value with a value obtained by weighting the colocalized and contiguous luminance samples according to some implementations of the present description.

[0035] Figure 13A is a block diagram illustrating that CCSAO does not apply to the current chrominance (luminance) sample if any of the colocalized and contiguous luminance (chrominance) samples used for classification are outside the current picture according to some implementations of the present description.

[0036] Figure 13B is a block diagram illustrating that CCSAO does not apply to the current luminance or chrominance sample if any of the colocalized and contiguous luminance or chrominance samples used for classification are outside the current picture according to some implementations of the present description.

[0037] Figure 14 is a block diagram illustrating that CCSAO does not apply to the current chrominance sample if a corresponding selected co-located or contiguous luminance sample used for classification is outside a virtual space defined by a virtual boundary (VB) according to some implementations of the present description.

[0038] Figure 15 shows that repetitive filling or mirroring is applied to luminance samples that are outside the virtual boundary according to some implementations of the present description.

[0039] Figure 16 shows that 1 additional luminance line buffer is required if the 9 co-located and contiguous luminance samples are used for classification according to some implementations of the present description.

[0040] Figure 17 shows an illustration in AVS where the CCSAO of 9 luminance candidates crossing the VB can increase 2 additional luminance line buffers according to some implementations of the present description.

[0041] Figure 18A shows an illustration in WC where the CCSAO of 9 luminance candidates crossing the VB can increase 1 additional luminance line buffer according to some implementations of the present description.

[0042] Figure 18B shows an illustration in which, when colocalized or contiguous chrominance samples are used to classify current luminance samples, the selected chrominance candidate may be on the other side of the VB and require additional chrominance line buffer according to some implementations of the present description.

[0043] Figures 19A to 19C show in AVS and VVC that CCSAO is disabled for a chrominance sample if any of the luminance candidates of the chrominance sample are on the other side of the VB (outside the VB of the current chrominance sample) according to some implementations of the present description.

[0044] Figures 20A to 20C show in AVS and WC, that CCSAO is enabled by the use of repetitive fill for a chrominance sample if any of the luminance candidates of the chrominance sample are on the other side of the VB (outside the VB of the current chrominance sample) according to some implementations of the present description.

[0045] Figures 21A to 21C show in AVS and WC, that CCSAO is enabled by the use of mirror fill for a chrominance sample if any of the luminance candidates of the chrominance sample are on the other side of the VB (outside the VB of the current chrominance sample) according to some implementations of the present description.

[0046] Figures 22A to 22B show that CCSAO is enabled by the use of two-sided symmetric fill for different CCSAO sample shapes according to some implementations of the present description.

[0047] Figure 23 shows the restrictions of using a limited number of luminance candidates for classification according to some implementations of the present description.

[0048] Figure 24 shows that the applied region of CCSAO does not align with the encoding tree block (CTB) / encoding tree unit (CTU) boundary according to some implementations of the present description.

[0049] Figure 25 shows that the frame splitting in the applied CCSAO region can be set with CCSAO parameters according to some implementations of the present description.

[0050] Figure 26 shows that the applied region of CCSAO can exhibit binary tree (BT) / quaternary tree (QT) / ternary tree (TT) separation at the frame / segment / CTB level according to some implementations of the present description.

[0051] Figure 27 is a block diagram illustrating a plurality of classifiers used and switched at different levels within an image frame according to some implementations of the present description.

[0052] Figure 28 is a block diagram illustrating that the CCSAO applied region division can be dynamic and changed at the image level, according to some implementations of the present description.

[0053] Figure 29 is a block diagram illustrating that the classification methods of SAOs that are disclosed in this description serve as a post-prediction filter according to some implementations of this description.

[0054] Figure 30 is a block diagram illustrating that for the SAO post-prediction filter, each component can use current and contiguous samples for classification according to some implementations of the present description.

[0055] Figure 31 is a flowchart illustrating an example of a video signal decoding process that uses cross-component correlation according to some implementations of the present description.

[0056] Figure 32 is a diagram illustrating a computing environment along with a user interface, according to some implementations of the present description. DETAILED DESCRIPTION OF THE INVENTION

[0057] Specific implementations, examples of which are illustrated in the accompanying drawings, will be discussed in detail below. The following detailed description sets forth a number of non-limiting specific details intended to aid in understanding the subject matter presented in this application. However, it will be evident to a person skilled in the art that various alternatives may be used without departing from the scope of the claims and that the subject matter may be practiced without these specific details. For example, it will be evident to a person skilled in the art that the subject matter presented herein may be implemented in various types of electronic devices with digital video capabilities.

[0058] The first-generation AVS standard includes the Chinese national standard “Information Technology, Advanced Audio-Video Coding, Part 2: Video” (known as AVS1) and “Information Technology, Advanced Audio-Video Coding, Part 16: Radio and Television Video” (known as AVS+). This standard can offer bitrate savings of around 50%, with the same perceptual quality compared to the MPEG-2 standard. The second-generation AVS standard includes the Chinese national standard series “Information Technology, Efficient Multimedia Coding” (known as AVS2), which is primarily aimed at the transmission of ultra-high-definition TV programs. The encoding efficiency of AVS2 is twice that of AVS+. Meanwhile, the Institute of Electrical and Electronics Engineers (IEEE) submitted the video portion of the AVS2 standard as an international standard for applications.The AVS3 standard is a next-generation video coding standard for ultra-high-definition video applications that aims to surpass the coding efficiency of the previous international standard, HEVC, by offering approximately 30% bitrate savings compared to HEVC. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was finalized, providing approximately 30% bitrate savings compared to HEVC. The AVS group currently maintains a reference software, called the High-Performance Model (HPM), to demonstrate a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is based on a block-based hybrid video coding framework.

[0059] Figure 1 is a block diagram illustrating an example of system 10 for parallel encoding and decoding of video blocks according to some implementations of the present description. As shown in Figure 1, system 10 includes a source device 12 that generates and encodes video data for subsequent decoding by a destination device 14. The source device 12 and the destination device 14 can comprise any of a wide variety of electronic devices, including desktop or laptop computers, tablets, smartphones, TV signal converters, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some implementations, the source device 12 and the destination device 14 are equipped with wireless communication capabilities.

[0060] In some implementations, the destination device 14 may receive the encoded video data, which will be decoded via a link 16. Link 16 may include any type of communication medium or device capable of transferring the encoded video data from the source device 12 to the destination device 14. In one example, link 16 may comprise a communication medium to enable the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines.The communication medium may be part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device 12 to the destination device 14.

[0061] In other implementations, the encoded video data can be transmitted from output interface 22 to a storage device 32. Subsequently, the destination device 14 can access the encoded video data from storage device 32 through input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other digital storage medium suitable for storing encoded video data. In another example, storage device 32 could be a file server or other intermediate storage device that can hold the encoded video data generated by source device 12.Destination device 14 can access the video data stored on storage device 32 by streaming or downloading. The file server can be any type of computer capable of storing encoded video data and transmitting that encoded video data to destination device 14. Examples of file servers include a web server (e.g., for a website), an FTP server, network-attached storage (NAS) devices, or a local disk drive. Destination device 14 can access the encoded video data through any standard data connection, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing the encoded video data stored on a file server.The transmission of encoded video data from storage device 32 can be a streaming transmission, a download transmission, or a combination of both.

[0062] As shown in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include a source, such as a video capture device, for example, a video camera, a video file containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as the source video, or a combination of such sources. For example, if the video source 18 is a video camera in a security surveillance system, the source device 12 and the destination device 14 may be camera phones or video phones. However, the implementations described in this application may be applicable to video encoding in general and may be applied to wireless and / or wired applications.

[0063] Captured, pre-captured, or computer-generated video can be encoded via video encoder 20. The encoded video data can be transmitted directly to the destination device 14 via the output interface 22 of the source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by the destination device 14 or other devices for decoding and / or playback. The output interface 22 can also include a modem and / or a transmitter.

[0064] The destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 may include a receiver and / or a modem and receive the encoded video data via link 16. The encoded video data, communicated via link 16 or provided on the storage device 32, may include a variety of syntax elements generated by the video encoder 20 for use by the video decoder 30 in decoding the video data. Such syntax elements may be included in the encoded video data, whether transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0065] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device and an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may comprise any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0066] The video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards, such as VVC, HEVC, MPEG-4 Part 10, Advanced Video Coding (AVC), AVS, or extensions of such standards. It is understood that this application is not limited to any specific video encoding / decoding standard and may apply to other video encoding / decoding standards. It is generally envisaged that the video encoder 20 of the source device 12 may be configured to encode video data in accordance with any of these current or future standards. Similarly, it is generally envisaged that the video decoder 30 of the destination device 14 may also be configured to decode video data in accordance with any of these current or future standards.

[0067] Each of the video encoder 20 and video decoder 30 may be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, an electronic device may store instructions for the software on a suitable non-transient, computer-readable medium and execute the instructions in the hardware through one or more processors to perform the video encoding / decoding operations disclosed herein.Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0068] Figure 2 is a block diagram illustrating an example of a video encoder 20 according to some implementations described in this application. The video encoder 20 can perform intra-coding and predictive inter-coding of video blocks within video frames. Predictive intra-coding is based on spatial prediction to reduce or eliminate spatial redundancy in video data within a given video image or frame. Predictive inter-coding is based on temporal prediction to reduce or eliminate temporal redundancy in video data within adjacent video images or frames of a video sequence. p7i«nn / C7n7 / e / Yi

[0069] As shown in Figure 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded image buffer (DPB) 64, a summing unit 50, a transform processing unit 52, a quantization unit 54, and an entropy encoding unit 56. The prediction processing unit 41 also includes a motion estimation unit 42, a motion compensation unit 44, a splitting unit 45, an intra-prediction processing unit 46, and an intra-block copy (BC) unit 48. In some implementations, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and a summing unit 62 for video block reconstruction.A loop filter 63, such as an unblocking filter, can be placed between the adder 62 and the DPB 64 to filter block boundaries and remove blocking artifacts from the reconstructed video. Another loop filter 63, in addition to the unblocking filter, can also be used to filter the output of the adder 62. Additional loop filtering 63, such as an adaptive per-sample deviation (APD) and / or an adaptive loop filter (ALF), can be applied to the reconstructed CU before it is placed in the reference image storage and used as a reference for encoding future video blocks. The video encoder 20 can take the form of a single fixed or programmable hardware unit or can be divided among one or more of the fixed or programmable hardware units illustrated.

[0070] Video data memory 40 can store video data for encoding by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for use in encoding video data through video encoder 20 (for example, in intra-coding or predictive inter-coding modes). Video data memory 40 and DPB 64 can be comprised of any of a variety of memory devices. In various instances, video data memory 40 may be on-chip with other components of video encoder 20, or off-chip relative to those components.

[0071] As shown in Figure 2, after receiving the video data, the splitting unit 45 within the prediction processing unit 41 splits the video data into video blocks. This splitting can also include dividing a video frame into segments, tiles, or other larger encoding units (CUs) according to predefined splitting structures, such as a quaternary tree structure associated with the video data. The video frame can be split into multiple video blocks (or sets of video blocks referred to as tiles).The prediction processing unit 41 can select one of a plurality of possible predictive coding modes, such as one of a plurality of predictive intra-coding modes or one of a plurality of predictive inter-coding modes, for the current video block based on error results (e.g., coding rate and distortion level). The prediction processing unit 41 can provide the resulting intra-coded or inter-coded prediction block to the adder 50 to generate a residual block and to the adder 62 to reconstruct the coded block for later use as part of a reference frame. The prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode flags, split information, and other syntax information, to the entropy coding unit 56.

[0072] To select a suitable predictive intra-coding mode for the current video block, the intra-prediction processing unit 46 within the prediction processing unit 41 can perform predictive intra-coding of the current video block relative to one or more contiguous blocks in the same frame as the current block to be encoded, thus providing spatial prediction. The motion estimation unit 42 and the motion compensation unit 44 within the prediction processing unit 41 perform predictive inter-coding of the current video block relative to one or more predictive blocks in one or more reference frames, thus providing temporal prediction. The video encoder 20 can perform multiple encoding passes, for example, to select a suitable encoding mode for each block of video data.

[0073] In some implementations, motion estimation unit 42 determines the Interprediction mode for a current video frame by generating a motion vector, which indicates the displacement of a prediction unit (PU) of a video block within the current video frame relative to a predictive block within a reference video frame, according to a predetermined pattern within a sequence of video frames. Motion estimation, performed by motion estimation unit 42, is the process of generating motion vectors, which estimate the movement of video blocks.A motion vector, for example, can indicate the displacement of a PU of a video block within a current video or image frame relative to a predictive block within a reference frame (or other encoded unit) relative to the current block being encoded within the current frame (or other encoded unit). The default pattern can designate video frames in the sequence as P-frames or B-frames. The intra-block copy (BC) unit 48 can determine vectors, for example, block vectors, for intra-BC encoding, similarly to the determination of motion vectors through the motion estimation unit 42 for inter-prediction, or it can use the motion estimation unit 42 to determine the block vector.

[0074] A predictive block is a block of a reference frame that is considered to closely match the PU of the video block to be encoded in terms of pixel difference, which can be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for subinteger pixel positions of reference frames stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 can perform a motion search relative to whole pixel positions and fractional pixel positions and generate a motion vector with fractional-pixel accuracy.

[0075] The motion estimation unit 42 calculates a motion vector for a PU of a video block in an intercoded prediction frame by comparing the position of the PU with the position of a predictive block of a reference frame selected from a first list of reference frames (List 0) or a second list of reference frames (List 1), each of which identifies one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44 and then to the entropy encoding unit 56.

[0076] Motion compensation, performed via motion compensation unit 44, may involve collecting or generating the predictive block based on the motion vector determined by motion estimation unit 42. Once it receives the motion vector for the current video block's PU, motion compensation unit 44 can locate a predictive block that the motion vector points to in one of the reference frame lists, retrieve the predictive block from DPB 64, and forward the predictive block to adder 50. Adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the predictive block provided by motion compensation unit 44 from the pixel values ​​of the current video block being encoded.The pixel difference values ​​that make up the residual video block can include luminance or chrominance difference components, or both. The motion compensation unit 44 can also generate syntax elements associated with the video blocks of a video frame for use by the video decoder 30 in decoding the video blocks of the video frame. Syntax elements can include, for example, syntax elements that define the motion vector used to identify the predictive block, any indicator that indicates the prediction mode, or any other syntax information described in this document. It is important to note that the motion estimation unit 42 and the motion compensation unit 44 can be fully integrated, but they are illustrated separately for conceptual purposes.

[0077] In some implementations, the intra-block copy (BC) unit 48 can generate vectors and collect predictive blocks similarly to that described above in relation to the motion estimation unit 42 and the motion compensation unit 44, but the predictive blocks are in the same frame as the current block being encoded, and the vectors are referred to as block vectors rather than motion vectors. In particular, the intra-block copy (BC) unit 48 can determine an intra-prediction mode to be used to encode a current block. In some examples, the intra-block copy (BC) unit 48 can encode a current block using different intra-prediction modes, for example, during separate encoding passes, and test its performance by means of a rate distortion analysis.The intrablock copy (BC) 48 unit can then select, from among the various tested intra-prediction modes, a suitable intra-prediction mode to use and, consequently, generate an intra-mode flag. For example, the intrablock copy (BC) 48 unit can calculate rate distortion values ​​by performing a rate distortion analysis for the various tested intra-prediction modes and select the intra-prediction mode with the best rate distortion characteristics as the appropriate intra-prediction mode to use. The rate distortion analysis typically determines a degree of distortion (or error) between a coded block and an original uncoded block that was encoded to produce the coded block, as well as a bit rate (i.e., a number of bits) used to produce the coded block.The intra-block copy (BC) unit 48 can calculate ratios from distortions and speeds for the various encoded blocks in order to determine which intra-prediction mode shows the best speed distortion value for the block.

[0078] In other examples, the intra-block copy (BC) unit 48 may use the motion estimation unit 42 and the motion compensation unit 44, in whole or in part, to perform such functions for BC intra-prediction in accordance with the implementations described herein. In any case, for intra-block copying, a predictive block may be a block that is considered to closely match the block to be encoded, in terms of pixel difference, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and the identification of the predictive block may include calculating values ​​for subinteger pixel positions.

[0079] Whether the predictive block originates from the same frame according to intraprediction, or from a different frame according to interprediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predictive block from the pixel values ​​of the current video block being encoded, forming pixel difference values. The pixel difference values ​​that form the residual video block can include differences in both luminance and chrominance components.

[0080] The intra-prediction processing unit 46 can intra-predict a current video block, as an alternative to the inter-prediction performed via the motion estimation unit 42 and the motion compensation unit 44, or the intra-block copy prediction performed via the intra-block copy (BC) unit 48, as described above. In particular, the intra-prediction processing unit 46 can determine an intra-prediction mode to be used for encoding a current block. To do this, the intra-prediction processing unit 46 can encode a current block using different intra-prediction modes, for example, during separate encoding passes, and the intra-prediction processing unit 46 (or a mode selection unit, in some examples) can select a suitable intra-prediction mode to use from the tested intra-prediction modes.The intraprediction processing unit 46 can provide information indicative of the intraprediction mode selected for the block to the entropy encoding unit 56. The entropy encoding unit 56 can encode the information indicating the selected intraprediction mode into the bit stream.

[0081] Once the prediction processing unit 41 determines the predictive block for the current video block by inter-prediction or intra-prediction, the adder 50 forms a residual video block by subtracting the predictive block from the current video block. The residual video data from the residual block can be included in one or more transform units (TUs) and is provided to the transform processing unit 52. The transform processing unit 52 transforms the residual video data into residual transform coefficients by means of a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0082] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan of an array that includes the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0083] Following quantization, the entropy coding unit 56 encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval division entropy coding (PIPE), or another entropy coding technique or methodology. The encoded bitstream can then be transmitted to the video decoder 30 or archived to the storage device 32 for later transmission or retrieval by the video decoder 30.The entropy encoding unit 56 can also entropy-encode motion vectors and other syntax elements for the current video frame being encoded.

[0084] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate a reference block for predicting other video blocks. As noted above, the motion compensation unit 44 can generate a motion-compensated predictive block from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the predictive block to calculate subinteger pixel values ​​for use in motion estimation.

[0085] Adder 62 adds the reconstructed residual block to the motion-compensated predictive block produced by the motion-compensation unit 44 to produce a reference block for storage in DPB 64. Then, the intra-block copy (BC) unit 48, motion estimation unit 42, and motion-compensation unit 44 can use the reference block as a predictive block to interpredict another video block in a later video frame.

[0086] Figure 3 is a block diagram illustrating an example of a video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summing unit 90, and a DPB 92. The prediction processing unit 81 also includes a motion compensation unit 82, an intra-prediction processing unit 84, and an intra-block copy (BC) unit 85. The video decoder 30 can perform a decoding process that is generally the reciprocal of the encoding process described above with respect to the video encoder 20 in relation to Figure 2.For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-prediction unit 84 can generate prediction data based on intra-prediction mode signals received from the entropy decoding unit 80.

[0087] In some examples, a video decoder unit 30 may be assigned the task of performing the implementations of this application. Also, in some examples, the implementations of this description may be divided among one or more of the units of the video decoder 30. For example, the intra-block copy (BC) unit 85 may perform the implementations of this application, alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra-prediction processing unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra-block copy (BC) unit 85, and the functionality of the intra-block copy (BC) unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0088] Video data memory 79 can store video data, such as an encoded video bitstream, for decoding by the other components of the video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source such as a camera, by wireless or wired network video data communication, or by accessing physical data storage media (for example, a flash drive or hard disk). Video data memory 79 can include an encoded picture buffer (CPB) that stores encoded video data from an encoded video bitstream.The decoded picture buffer (DPB) 92 of the video decoder 30 stores reference video data for use in decoding video data through the video encoder 30 (for example, in intra-coding or predictive intercoding modes). The video data buffer 79 and DPB 92 can be comprised of any of a variety of memory devices, such as dynamic random-access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data buffer 79 and DPB 92 are represented as two separate components of the video decoder 30 in Figure 3. However, it will be evident to a person skilled in the art that the video data buffer 79 and DPB 92 can be provided in the same memory device or in separate memory devices.In some examples, the video data memory 79 may be on-chip with other components of the video decoder 30, or off-chip relative to those components.

[0089] During the decoding process, video decoder 30 receives a stream of encoded video bits representing the video blocks of an encoded video frame and associated syntax elements. Video decoder 30 can receive the syntax elements at the video frame level and / or at the video block level. The entropy decoding unit 80 of video decoder 30 entropy-decodes the bit stream to generate quantized coefficients, motion vectors or intra-prediction mode flags, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0090] When the video frame is encoded as an intra-coded predictive frame (I) or for intra-coded predictive blocks in other frame types, the intra-prediction processing unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on a signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0091] When the video frame is encoded as an intercoded predictive frame (i.e., B or P), the motion compensation unit 82 of the prediction processing unit 81 produces one or more predictive blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the predictive blocks can be produced from a reference frame within one of the reference frame lists. The video decoder 30 can construct the reference frame lists, List 0 and List 1, using predefined reference frame-based construction techniques stored in the DPB 92.

[0092] In some examples, when the video block is encoded according to the pzi«nn / cznz / e / Yi intra-block copy mode described herein, the intra-block copy (BC) unit 85 of the prediction processing unit 81 produces predictive blocks for the current video block based on block vectors and other syntax elements received from the entropy decoding unit 80. The predictive blocks may be within a reconstructed region of the same image as the current video block defined by the video encoder 20.

[0093] The motion compensation unit 82 and / or the intra-block copy (BC) unit 85 determines the prediction information for a video block from the current video frame by syntactically analyzing the motion vectors and other syntax elements, and then uses the prediction information to produce the predictive blocks for the current video block being decoded.For example, the motion compensation unit 82 uses some of the received syntax elements to determine a prediction mode (e.g., intra-prediction or inter-prediction) used to encode the video blocks in the video frame, an inter-prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, motion vectors for each inter-coded predictive video block in the frame, inter-prediction status for each inter-coded predictive video block in the frame, as well as other information to decode the video blocks in the current video frame.

[0094] Similarly, the intra-block copy (BC) unit 85 can use some of the received syntax elements, e.g., a flag, to determine that the current video block was predicted by intra-block copy mode, construction information of which video blocks in the frame are within the reconstructed region and should be stored in DPB 92, block vectors for each video block predicted by intra-block copy of the frame, intra-block copy prediction status for each video block predicted by intra-block copy of the frame, and other information to decode the video blocks in the current video frame.

[0095] Motion compensation unit 82 can also perform interpolation using the interpolation filters employed by video encoder 20 during video block encoding to calculate interpolated values ​​for subinteger pixels in reference blocks. In this case, motion compensation unit 82 can determine the interpolation filters used by video encoder 20 from the received syntax elements and use the interpolation filters to produce predictive blocks.

[0096] The inverse quantization unit 86 inversely quantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameter calculated by the video encoder 20 for each video block in the video frame to determine a degree of quantization. The inverse transform processing unit 88 applies an inverse transform, for example, an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to reconstruct the remaining blocks in the pixel domain.

[0097] Once the motion compensation unit 82 or the intra-block copy (BC) unit 85 generates the predictive block for the current video block based on the vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by summing the residual block from the inverse transform processing unit 88 and a corresponding predictive block generated by the motion compensation unit 82 and the intra-block copy (BC) unit 85. A loop filter 91 can be placed between the adder 90 and the DPB 92 for further processing of the decoded video block. Loop filtering 91, such as an unblocking filter, an adaptive per-sample deviation (APD), and an adaptive loop filter (ALF), can be applied to the reconstructed CU before placing it in the reference image storage.The video blocks decoded in a given frame are then stored in the DPB 92, which stores the reference frames used for subsequent motion compensation of the following video blocks. The DPB 92, or a memory device separate from the DPB 92, can also store decoded video for later display on a display device, such as display device 34 in Figure 1.

[0098] In a typical video encoding process, a video sequence normally includes an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other cases, a frame may be monochromatic and thus include only a two-dimensional array of luminance samples.

[0099] Like HEVC, the AVS3 standard takes as its starting point the block-based hybrid video coding framework. The input video signal is processed block by block (called coding units (CUs)). Unlike HEVC, which divides blocks only based on quaternary trees, in AVS3, a tree coding unit (CTU) is split into several CUs to accommodate various local characteristics based on quaternary / binary / extended quaternary trees. Furthermore, HEVC eliminates the concept of multiple split unit types; that is, the separation of the CU, prediction unit (PU), and transform unit (TU) does not exist in AVS3. Instead, each CU is always used as the basic unit for both prediction and transform without further partitioning. In the AVS3 tree-splitting structure, a CTU is first split based on a quaternary tree structure.Afterwards, each leaf node of a quaternary tree can also be divided based on an extended binary and quaternary tree structure.

[00100] As shown in Figure 4A, the video encoder 20 (or, more specifically, the splitting unit 45) generates an encoded representation of a frame by first splitting the frame into a set of tree-like coding units (CTUs). A video frame can include any number of CTUs arranged sequentially in a left-to-right, top-to-bottom frame scan order. Each CTU is a larger logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a set of sequence parameters, such that all CTUs in a video sequence are the same size, for example, 128 x 128, 64 x 64, 32 x 32, and 16 x 16. However, it should be noted that the present application is not necessarily limited to a specific size.As shown in Figure 4B, each CTU can comprise a luminance sample tree encoding block (CTB), two corresponding chrominance sample tree encoding blocks, and syntax elements used to encode the samples in the tree encoding blocks. The syntax elements describe the properties of the various unit types in a pixel encoded block and how the video sequence can be reconstructed in the video decoder 30, including inter-prediction or intra-prediction, the intra-prediction mode, motion vectors, and other parameters. In monochrome images or images with three independent color planes, a CTU can comprise a single tree encoding block and syntax elements used to encode the samples in the tree encoding block. A tree encoding block can be an NxN block of samples.

[00101] To achieve improved performance, the video encoder 20 can recursively perform tree splits, such as binary tree splits, ternary tree splits, quaternary tree splits, or a combination of both, on the CTU's tree encoding blocks and split the CTU into smaller encoding units (CUs). As depicted in Figure 4C, the 64x64 CTU 400 is first split into four smaller CUs, each with a block size of 32x32. Among these four smaller CUs, CU 410 and CU 420 are each split into four 16x16 CUs by block size. Each of the two 16x16 CUs 430 and 440 is then split into four 8x8 CUs by block size.Figure 4D represents a quaternary tree data structure illustrating the final result of the CTU 400 splitting process depicted in Figure 4C. Each leaf node of the quaternary tree corresponds to a CU of a respective size ranging from 32x32 to 8x8. Similar to the CTU depicted in Figure 4B, each CU may comprise one encoding block (CB) of luminance samples and two corresponding encoding blocks of chrominance samples from a frame of the same size, along with syntax elements used to encode the samples within the encoding blocks. In monochrome images or images with three independent color planes, a CU may comprise a single encoding block and the syntax structures used to encode the samples within that block.It should be noted that the quaternary tree split depicted in Figures 4C and 4D is for illustrative purposes only, and that a CTU can be split into several CUs to accommodate diverse local characteristics based on quaternary / ternary / binary tree splits. In the multi-tree structure, a CTU is split into a quaternary tree structure, and each leaf CU of a quaternary tree can also be split into binary and ternary tree structures. As shown in Figure 4E, there are five types of splitting / dividing in AVS3: quaternary tree split, horizontal binary tree split, vertical binary tree split, extended horizontal quaternary tree split, and extended vertical quaternary tree split.

[00102] In some implementations, the Video Encoder 20 can also divide a CU's encoding block into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same inter-prediction or intra-prediction applies. A prediction unit (PU) in a CU can comprise a luminance sample prediction block, two corresponding chrominance sample prediction blocks, and syntax elements used to predict the prediction blocks. In monochrome images or images with three independent color planes, a PU can comprise a single prediction block and syntax structures used to predict the prediction block. The Video Encoder 20 can generate luminance, Cb, and Cr prediction blocks for the luminance, Cb, and Cr prediction blocks of each PU in the CU.

[00103] Video Encoder 20 can use intra-prediction or inter-prediction to generate predictive blocks for a PU. If Video Encoder 20 uses intra-prediction to generate predictive blocks for a PU, it can generate the PU's predictive blocks based on decoded samples from the frame associated with the PU. If Video Encoder 20 uses inter-prediction to generate predictive blocks for a PU, it can generate the PU's predictive blocks based on decoded samples from one or more frames other than the frame associated with the PU.

[00104] Once the video encoder 20 generates the luminance predictive blocks, Cb and Cr, for one or more PUs of a CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the CU's luminance predictive blocks from its original luminance encoding block, such that each sample in the CU's luminance residual block indicates a difference between a luminance sample in one of the CU's luminance predictive blocks and a corresponding sample in the CU's original luminance encoding block.Similarly, the video encoder 20 can generate a Cb residual block and a Cr residual block, respectively, for the CU, such that each sample in the CU's Cb residual block indicates a difference between a Cb sample in one of the CU's Cb predictive blocks and a corresponding sample in the CU's original Cb encoding block, and each sample in the CU's Cr residual block can indicate a difference between a Cr sample in one of the CU's Cr predictive blocks and a corresponding sample in the CU's original Cr encoding block.

[00105] Furthermore, as illustrated in Figure 4C, the video encoder 20 can use quaternary tree splitting to decompose the residual luminance blocks, Cb and Cr, of a CU pzi«nn / cznz / e / Yi into one or more luminance transform blocks, Cb and Cr. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU can comprise one luminance sample transform block, two corresponding chrominance sample transform blocks, and syntax elements used to transform the transform block samples. Therefore, each TU of a CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with the TU may be a sub-block of the CU's residual luminance block.The Cb transform block can be a sub-block of the Cb residual block of the CU. The Cr transform block can be a sub-block of the Cr residual block of the CU. In monochrome images or images with three independent color planes, a TU can comprise a single transform block and syntax structures used to transform the samples within the transform block.

[00106] Video Encoder 20 can apply one or more transforms to a luminance transform block of a TU to generate a luminance coefficient block for the TU. A coefficient block can be a two-dimensional array of transform coefficients. A transform coefficient can be a scalar quantity. Video Encoder 20 can apply one or more transforms to a Cb transform block of a TU to generate a Cb coefficient block for the TU. Video Encoder 20 can apply one or more transforms to a Cr transform block of a TU to generate a Cr coefficient block for the TU.

[00107] After generating a block of coefficients (for example, a block of luminance coefficients, a block of Cb coefficients, or a block of Cr coefficients), the Video Encoder 20 can quantize the block of coefficients. Quantization generally refers to a process in which the transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, providing greater compression. Once the Video Encoder 20 quantizes a block of coefficients, it can entropy-encode the syntax elements that indicate the quantized transform coefficients. For example, the Video Encoder 20 can perform context-adaptive binary arithmetic (CABAC) encoding on the syntax elements that indicate the quantized transform coefficients. Finally, the video encoder 20 can generate a bitstream that includes a bit sequence that forms a representation of the encoded frames and associated data, which are either stored on the storage device 32 or transmitted to the destination device 14.

[00108] After receiving a bitstream generated by video encoder 20, video decoder 30 can parse the bitstream to obtain syntax elements. Video decoder 30 can reconstruct the video data frames based, at least in part, on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the reciprocal of the encoding process performed by video encoder 20. For example, video decoder 30 can perform inverse transforms on the coefficient blocks associated with the TUs of a current CU to reconstruct the residual blocks associated with the TUs of the current CU.The video decoder 30 also reconstructs the encoding blocks of the current CU by adding the predictive block samples for the current CU's PUs to the corresponding transform block samples of the current CU's TUs. After reconstructing the encoding blocks for each CU in a frame, the video decoder 30 can reconstruct the frame.

[00109] SAO is a process that modifies decoded samples by conditionally adding a bias value to each sample after the unblocking filter has been applied, based on values ​​in lookup tables transmitted by the encoder. SAO filtering is performed for each region based on a filter type selected by each CTB using the syntax element sao-type-idx. A value of 0 for sao-type-idx indicates that the SAO filter is not applied to the CTB, and values ​​of 1 and 2 signal the use of band-bias and edge-bias filter types, respectively. In the band-bias mode specified by sao-type-idx equal to 1, the selected bias value depends directly on the sample amplitude.In this mode, the total sample amplitude range is evenly divided into 32 segments called bands, and sample values ​​belonging to four of these bands (which are consecutive within the 32 bands) are modified by adding transmitted values ​​denoted as band offsets, which can be positive or negative. The primary reason for using four consecutive bands is that in smoothed areas where banding artifacts may appear, sample amplitudes in a CTB tend to be concentrated in only a few of these bands. Furthermore, the design choice to use four offsets aligns with the edge offset mode of operation, which also uses four offset values.In the edge deviation mode specified by sao-type-idx equal to 2, a sao-eo-class syntax element with values ​​from 0 to 3 signals whether a horizontal, vertical, or one of two diagonal gradient directions is used for edge deviation classification in the CTB.

[00110] Figure 5 is a block diagram representing the four gradient patterns used in the SAO according to some implementations of the present description. The four gradient patterns 502, 504, 506, and 508 are for the respective sao-eo-class pattern in edge deviation mode. The sample labeled “p” indicates a central sample to be considered. Two samples labeled “nO” and “n1” specify two contiguous samples along the gradient patterns (a) horizontal (sao-eo-class = 0), (b) vertical (sao-eo-class = 1), (c) 135° diagonal (sao-eo-class = 2), and (d) 45° (sao-eo-class = 3). Each sample in the CTB is classified into one of five Edgeldx categories by comparing the sample p value located at some position with the nO and n1 values ​​of two samples located at adjacent positions, as shown in Figure 5.This classification is done for each sample based on the decoded sample values, so no additional signaling is required for Edgeldx classification. Depending on the Edgeldx category at the sample position, a deviation value from a transmitted lookup table is added to the sample value for Edgeldx categories 1 through 4. Deviation values ​​are always positive for categories 1 and 2 and negative for categories 3 and 4. Thus, the filter generally has a smoothing effect on edge deviation mode. Table 1 illustrates an Edgeldx category of samples in the SAO edge classes. Edgeldx Condition Meaning 0 Cases not included in the list Monotonic area 1 p < no yp < ni Local min. 2 p <noyp = mop<niyp = no Borde 3 p> noyp = mop>niyp = no Edge 4 p > no yp > m Max. local Table 1: Edgeldx categories of samples in SAO edge classes.

[00111] For SAO types 1 and 2, a total of four amplitude deviation values ​​are passed to the decoder for each CTB. For type 1, the sign is also encoded. The deviation values ​​and related syntax elements, such as sao-type-idx and sao-eo-class, are determined by the encoder, usually using criteria that optimize distortion rate performance. SAO parameters can be indicated to be inherited from the left or top CTB using a merge indicator to make signaling efficient. In summary, SAO is a nonlinear filtering operation that allows for further refinement of the reconstructed signal and can improve signal representation in both smoothed areas and surrounding edges.

[00112] In some modalities, methods and systems are described in this document to improve coding efficiency or to reduce the complexity of Sample Adaptive Deviation (SAO) by introducing cross-component information. SAO is used in the HEVC, WC, AVS2, and AVS3 standards. Although the existing SAO design in the HEVC, WC, AVS2, and AVS3 standards is used as the basic SAO method in the following descriptions, for a person skilled in video coding techniques, the cross-component methods disclosed herein may be applicable to other loop filter designs or other coding tools with a similar design nature. For example, in the AVS3 standard, SAO is replaced by a coding tool called Enhanced Sample Adaptive Deviation (ESAO). However, the CCSAO described herein may also be applied concurrently with ESAO.In another example, CCSAO can be applied to the AV1 standard at the same time as the Restricted Directional Enhancement Filter (CDEF).

[00113] For existing AOS designs in HEVC, WC, AVS2, and AVS3 standards, the deviation values ​​for luminance (Y), chrominance (Cb), and chrominance (Cr) samples are determined independently. That is, for example, the deviation for current chrominance samples is determined solely by the values ​​of current and adjacent chrominance samples, without considering co-located or adjacent luminance samples. However, luminance samples retain more detailed information about the original image than chrominance samples and can benefit the decision regarding the deviation for current chrominance samples.Furthermore, since chrominance samples typically lose high-frequency detail after RGB-to-YCbCr conversion, or after quantization and unblocking filtering, introducing luminance samples with preserved high-frequency detail for chrominance deviation decision-making can benefit chrominance sample reconstruction. Therefore, greater gain can be expected when exploring cross-component correlation, for example, when using Cross-Component Sample Adaptive Deviation (CCSAO) methods and systems. Another example is that for CCSAO, luminance sample deviations are initially determined solely by the luminance samples themselves. However, for instance, a luminance sample with the same band deviation (BO) classification can also be classified using its co-located and contiguous chrominance samples, which can lead to a more effective classification.SAO classification can be seen as a shortcut to compensate for the difference in samples between the original and reconstructed images. Therefore, effective classification is desirable.

[00114] Figure 6A is a block diagram illustrating the CCSAO system and process applied to chrominance samples, as well as the use of DBY Y as the input, according to some implementations of the present description. Luminance samples after the luminance unblocking filter (DBY Y) are used to determine additional deviations for chrominance Cb and Cr after SAO Cb and SAO Cr. For example, the current chrominance sample 602 is first classified using co-located luminance samples 604 and contiguous (white) 606, and the corresponding CCSAO deviation value for the corresponding class is added to the value of the current chrominance sample. Figure 6B is a block diagram illustrating the CCSAO system and process applied to luminance and chrominance samples, as well as the use of DBF Y / Cb / Cr as the input, according to some implementations of the present description.Figure 6C is a block diagram illustrating the CCSAO system and process that can operate independently according to some implementations of this description. Figure 6D is a block diagram illustrating the CCSAO system and process that can be applied recurrently (2 or N times), with equal or different deviations, according to some implementations of this description. In summary, in some modes, to classify the current luminance sample, current and contiguous luminance samples, co-located and contiguous chrominance samples (Cb and Cr) can be used. In some modes, to classify the current chrominance sample (Cb or Cr), co-located and contiguous luminance samples, cross-chrominance samples (both co-located and contiguous), and current and contiguous chrominance samples can be used.In some modalities, CCSAO can be cascaded after (1) DBF Y / Cb / Cr, (2) Y / Cb / Cr of reconstructed image before DBF, (3) after SAO Y / Cb / Cr or (4) before ALF Y / Cb / Cr.

[00115] In some modalities, CCSAO can be applied in conjunction with other coding tools, for example, ESAO in the AVS standard, or CDEF in the AV1 standard. Figure 6E is a block diagram illustrating the CCSAO system and process applied in conjunction with ESAO to the AVS standard according to some implementations of this description.

[00116] Figure 6F is a block diagram illustrating the CCSAO system and process applied after SAO according to some implementations of this description. In some modes, Figure 6F shows that the CCSAO location can be after SAO, i.e., the location of the Cross-Component Adaptive Loop Filter (CCALF) in the VVC standard. Figure 6G is a block diagram illustrating that the CCSAO system and process can operate independently without CCALF, according to some implementations of this description. In some modes, SAO Y / Cb / Cr can be replaced by ESAO, for example, in the AVS3 standard.

[00117] Figure 6H is a block diagram illustrating the CCSAO system and process applied in conjunction with CCALF, according to some implementations of this description. In some modes, Figure 6H shows that CCSAO can be applied in conjunction with CCALF. In some modes, the locations of CCALF and CCSAO in Figure 6H may be changed. In some modes, in Figures 6A through 6H, or throughout this description, the SAO Y / Cb / Cr blocks may be replaced by ESAO Y / Cb / Cr (in AVS3) or CDEF (in AV1). Note that Y / Cb / Cr may also be denoted as Y / UA / in the video encoding area.

[00118] In some modalities, the classification of current chrominance samples involves reusing the type (edge ​​deviation (EO) or BO), class, and SAO category of the co-localized luminance sample. The corresponding CCSAO deviation can be signaled or derived from the decoder itself. For example, h_Y is the SAO deviation of co-localized luminance, h_Cb and h_Cr are the CCSAO deviations of Cb and Cr, respectively, h_Cb (or h_Cr) = w * h_Y, where w can be selected from a limited table. For example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4...etc., where |w| only includes power values ​​of two.

[00119] In some modalities, the comparison score [-8, 8] of the co-localized luminance samples (YO) and the 8 contiguous luminance samples is used, giving a total of 17 classes. Initial class = 0 Loop on the 8 contiguous luminance samples (Yi, i=1 to 8) if YO > Yi Class += 1 otherwise, if YO < Yi Class -= 1

[00120] In some modes, the aforementioned classification methods can be combined. For example, the comparison score combined with SAO's BO (32-band classification) is used to increase diversity, resulting in a total of 17 * 32 classes. In some modes, Cb and Cr can use the same class to reduce complexity or to save bits.

[00121] Figure 7 is a block diagram illustrating a sampling process that uses the CCSAO according to some implementations of the present description. Specifically, Figure 7 shows that the CCSAO input can include the vertical and horizontal DBF inputs to simplify class determination or to increase flexibility. For example, Y0_DBF_V, Y0_DBF_H, and YO are co-located luminance samples at the DBF_V, DBF_H, and SAO inputs, respectively. Yi_DBF_V, Yi_DBF_H, and Yi are eight contiguous luminance samples at the DBF_V, DBF_H, and SAO inputs, respectively, where i = 1 to 8. Max. ME = max. (Y0_DBF_V, Y0_DBF_H, Y0_DBF) Max Yi = max. (Yi_DBF_V, Yi_DBF_H, Yi_DBF) And they feed max. YO and max. Yi into the CCSAO classification.

[00122] Figure 8 is a block diagram illustrating how the CCSAO process is interleaved with the vertical and horizontal DBF according to some implementations of this description. In some modes, the CCSAO blocks in Figures 6, 7, and 8 can be selective. For example, using Y0_DBF_V and Yi_DBF_V for the first CCSAO_V, which applies the same sampling process as in Figure 6, while simultaneously using the DBF_V luminance sample input as the CCSAO input.

[00123] In some modalities, the implemented CCSAO syntax is shown in Table 2. Level Syntax Element Meaning SPS cc_sao_enabled_flag whether CCSAO is enabled or not in the SH sequence slice_cc_sao_cb_flag slice_cc_sao_cr_flag whether CCSAO is enabled or not for Cb or Cr CTU cc sao m erg el eft f I ag cc_sao_m erg e_u p_f I ag whether CCSAO deviation is merged from the left or top CTU CTU cc_sao_class_idx CCSAO class index of this CTU CTU cc_sao_of f set_s ig n_f I ag ccsaooffsetabs CCSAO deviation values ​​for Cb and Cr of this CTU class Table 2: Example of CCSAO syntax

[00124] In some modes, to signal the CCSAO deviation values ​​of Cb and Cr, if an additional chrominance deviation is signaled, the other chrominance component deviation can be derived by a plus or minus sign, or weighted to avoid bit overprocessing. For example, h_Cb and h_Cr are the CCSAO deviations of Cb and Cr, respectively. With explicit signaling w, where w = ±1 w | with limited candidates | w |, h_Cr can be derived from h_Cb without explicitly signaling h_Cr itself. h_Cr = w *h_Cb

[00125] Figure 9 is a flowchart illustrating an example of a 900 process for decoding video signals that uses cross-component correlation according to some implementations of the present description.

[00126] The video decoder 30 receives the video signal which includes a first component and a second component (910). In some modes, the first component is a luminance component, and the second component is a chrominance component of the video signal.

[00127] Video decoder 30 also receives a plurality of deviations associated with the second component (920).

[00128] Next, the video decoder 30 uses a characteristic measurement of the first component to obtain a classification category associated with the second component (930). For example, in Figure 6, the current chrominance sample 602 is first classified using co-located luminance samples 604 and contiguous (white) 606, and the corresponding CCSAO deviation value is added to the current chrominance sample.

[00129] Video decoder 30 also selects a first deviation from the plurality of deviations for the second component according to the classification category (940).

[00130] Video decoder 30 further modifies the second component based on the first selected deviation (950).

[00131] In some modalities, using the characteristic measurement of the first component to obtain the classification category associated with the second component (930) includes: using a respective sample of the first component to obtain a respective classification category for each respective sample of the second component, wherein the respective sample of the first component is a co-located respective sample of the first component for each respective sample of the second component. For example, the classification of current chrominance samples consists of reusing the type (EO or BO), class, and SAO category of the co-located luminance sample.

[00132] In some modes, using the characteristic measurement of the first component to obtain the classification category associated with the second component (930) includes: using a respective sample of the first component to obtain a respective classification category for each respective sample of the second component, wherein the respective sample of the first component is reconstructed before being unlocked or reconstructed after being unlocked. In some modes, the first component is unlocked in an unlock filter (DBF). In some modes, the first component is unlocked in a luminance unlock filter (DBY Y). For example, as an alternative to Figures 6 or 7, the CCSAO input may also be before the DBY Y.

[00133] In some modes, the characteristic measurement is derived by dividing the sample value range of the first component into several bands and selecting a band based on the intensity value of a sample in the first component. In some modes, the characteristic measurement is derived from the Band Deviation (BO).

[00134] In some modalities, the characteristic measurement is derived based on the direction and strength of the edge information of a sample in the first component. In some modalities, the characteristic measurement is derived from the Edge Deviation (ED).

[00135] In some modes, modifying the second component (950) involves directly adding the first selected deviation to the second component. For example, the corresponding CCSAO deviation value is added to the current chrominance component sample.

[00136] In some modes, modifying the second component (950) involves mapping the first selected deviation to a second deviation and adding the mapped second deviation to the second component. For example, to signal CCSAO deviation values ​​for Cb and Cr, if an additional chrominance deviation is signaled, the other chrominance component deviation can be derived by using a plus or minus sign or weighting to avoid bit overprocessing.

[00137] In some modes, receiving the video signal (910) involves receiving a syntax element that indicates whether the video signal decoding method used by CCSAO is enabled for the video signal in the Sequence Parameter Set (SPS). In some modes, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[00138] In some modes, receiving the video signal (910) involves receiving a syntax element that indicates whether the video signal decoding method used by CCSAO is enabled or disabled for the second component at the segment level. In some modes, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled or disabled in the respective segment for Cb or Cr.

[00139] In some modes, receiving the plurality of offsets associated with the second component (920) comprises receiving different offsets for different Coding Tree Units (CTUs). In some modes, for a CTU, cc_sao_offset_sign_flag indicates a sign for an offset, and cc_sao_offset_abs indicates the offset values ​​per CCSAO of Cb and Cr of the current CTU.

[00140] In some modes, receiving the plurality of deviations associated with the second component (920) comprises receiving a syntax element that indicates whether the deviations received from a CTU are the same as those from a contiguous CTU of the CTU, where the contiguous CTU is a left contiguous CTU or a CTU above. For example, cc_sao_merge_up_flag indicates whether the deviation by CCSAO merges or does not merge from the left or top CTU.

[00141] In some modes, the video signal also includes a third component, and the video signal decoding method used by CCSAO also includes: receiving a second plurality of deviations associated with a third component; using the characteristic measurement of the first component to obtain a second classification category associated with the third component; selecting a third deviation from the second plurality of deviations for the third component according to the second classification category; and modifying the third component based on the selected third deviation.

[00142] Figure 11 is a block diagram of a sample process illustrating that all co-localized and contiguous (white) luminance / chrominance samples can be fed into CCSAO classification according to some implementations of the present description. Figures 6A, 6B, and Figure 11 show the CCSAO classification input. In Figure 11, the current chrominance sample is 1104, the cross-component co-localized chrominance sample is 1102, and the co-localized luminance sample is 1106.

[00143] In some modes, an example classifier (C0) uses the co-localized luminance or chrominance sample value (YO) in Figure 12 (Y4 / U4A / 4 in Figure 6B and Figure 6C) for classification. Where band_num is the number of evenly divided bands of the dynamic range of luminance or chrominance, and bit_depth is the sequence bit depth, an example of the class index for the current chrominance sample is: Class (C0) = (I * band_num) » bit_depth

[00144] In some categories, the classification takes rounding into account, for example: Class (CO) = ((I * band_num) + (1 « bit_depth)) » bit_depth

[00145] Some examples of band_num and bit_depth are provided in Table 3. Table 3 shows three classification examples when the number of bands is different for each of the classification examples. band_num 16 bit_depth 10 Class ME 0 0 63 1 64 127 2 128 191 3 192 255 4 256 319 5 320 383 6 384 447 7 448 511 8 512 575 9 576 639 10 640 703 11 704 767 12 768 831 13 832 895 14 896 959 15 960 1023 band_num 7 blt_depth 10 Class ME 0 0 145 1 146 292 2 293 438 3 439 584 4 585 730 5 731 877 6 878 1023 band_num 7 blt_depth 8 Class ME 0 0 36 1 37 72 2 73 109 3 110 145 4 146 182 5 183 218 6 219 255 Table 3: Example of band_num and bit_depth for each class index.

[00146] In some embodiments, a classifier uses different positions of luminance samples for C0 classification. Figure 10A is a block diagram showing a classifier that uses different positions of luminance (or chrominance) samples for C0 classification according to some implementations of the present description; for example, contiguous Y7 is used, but not YO, for C0 classification.

[00147] In some modalities, different classifiers can be changed at the Sequence Parameter Set (SPS) / Adaptation Parameter Set (APS) / Image Parameter Set (PPS) / Image Header (PH) / Segment Header (SH) / Tree Coding Unit (CTU) / Coding Unit (CU) level. For example, in Figure 10, YO is used for POCO, but Y7 is used for POC1, as shown in Table 4. POC Classifier C0 band_num Total classes 0 C0 using position YO 8 8 1 C0 using position Y7 8 8 pzi«nn / cznz / e / Yi Table 4: Different classifiers are applied to different images. In some modes, Figure 10B illustrates examples of different shapes for luminance candidates, according to some implementations of this description. For example, a restriction can be applied to the shapes. In some examples, the total number of luminance candidates must be the power of 2, as shown in Figure 10B (b), (c), and (d). In some examples, the number of luminance candidates must be symmetric both horizontally and vertically with respect to the chrominance sample (at the center), as shown in Figure 10B (a), (c), (d), and (e). In some modes, the power-of-2 restriction and the symmetry restriction can also be applied to the chrominance candidates. The U / V portion of Figure 6B and Figure 6C show an example of a symmetry restriction. In some modes, the format with a different color may have different classifier “restrictions.”For example, color format 420 uses luminance / chrominance candidate selection (one candidate selected in the 3x3 way), as shown in Figure 6B and Figure 6C, but color format 444 uses Figure 10B(f) for luminance and chrominance candidate selection, color format 422 uses Figure 10B(g) for luminance (2 chrominance samples share 4 luminance candidates), Figure 10B(f) for chrominance candidates.

[00148] In some modes, the C0 position and band_num of C0 can be combined and changed at the SPS / APS / PPS / PH / SH / CTU / CU level. Different combinations may result in different classifiers, as shown in Table 5. POC Classifier C0 band_num Total classes 0 C0 using position YO 16 16 1 C0 using position Y7 8 8 Table 5: Different combinations of classifier and number of bands are applied to different images.

[00149] In some embodiments, the co-localized luminance sample value (YO) is replaced by a value (Yp) obtained by weighting the co-localized and contiguous luminance samples. Figure 12 illustrates examples of classifiers by replacing the co-localized luminance sample value with a value obtained by weighting the co-localized and contiguous luminance samples according to some implementations of the present description. The colocalized luminance sample value (YO) may be replaced by a phase-corrected value (Yp) obtained by weighting the contiguous luminance samples. A different Yp may result in a different classifier.

[00150] In some modes, a different Yp is applied to different chrominance formats. For example, in Figure 12, the Yp of (a) is used for chrominance format 420, the Yp of (b) is used for chrominance format 422, and YO is used for chrominance format 444.

[00151] In some modalities, another classifier (C1) consists of the comparison score [8, 8] of the co-located luminance samples (YO) and the 8 contiguous luminance samples, giving a total of 17 classes, as shown below. Initial class (C1) = 0, Loop over the 8 contiguous luminance samples (Yi, i=1 to 8) if YO > Y¡ Class += 1 otherwise, if YO < Y¡ Class -= 1

[00152] In some modes, a variation (C1') only counts the comparison score [0, 8] and this gives a total of 8 classes. (C1, C1') is a group of classifiers and a PH / SH level indicator may be signaled for the change between C1 and C1'.

[00153] Initial class (C1 ') = 0, Loop over the 8 contiguous luminance samples (Yi, i=1 to 8) if YO > Yi Class += 1

[00154] In some modes, a variation (C1s) selectively uses N of M contiguous samples to count the comparison score. An M-bit bitmask can be signaled at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level to indicate which contiguous samples are selected for counting the comparison score. Figure 6B is used as an example of a luminance classifier: 8 contiguous luminance samples are candidates, and an 8-bit bitmask (01111110) is signaled at PH, indicating that samples Y1 through Y6 are selected, so the comparison score is in [-6, 6], yielding 13 deviations. The selective C1s classifier provides the decoder with more options for switching between deviation signaling overhead and classification granularity.

[00155] As with C1s, a variation (C1's) only counts the comparison score [0, +N], the above example bitmask 01111110 gives a comparison score of [0, 6], which yields 7 deviations.

[00156] In some modes, different classifiers are combined to produce a general classifier. For example, different classifiers are applied to different images (different POC values), as shown in Table 6-1. POC Classifier C0 band_num Total classes 0 combine C0 and C1 16 16*17 1 combine C0 and C1 16 16*9 2 combine C0 and C1 7 7*17 Table 6-1: Different general classifiers apply to different images.

[00157] In some modes, another example classifier (C3) uses a bitmask for classification, as shown in Table 6-2. A 10-bit bitmask is signaled at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level to indicate the classifier. For example, the bitmask 11 1100 0000 means that, for a given 10-bit luminance sample value, only the most significant bits (MSBs)—4 bits—are used for classification, resulting in a total of 16 classes. Another example bitmask, 10 0100 0001, means that only 3 bits are used for classification, resulting in a total of 8 classes.

[00158] In some modes, the bitmask length (N) can be set or changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. For example, for a 10-bit sequence, a 4-bit bitmask of 1110 is signaled in PH, in an image, and the 3 bits b9, b8, b7 MSB are used for classification. Another example is a 4-bit bitmask of 0011 in LSB, and b0, b1 are used for classification. The bitmask classifier can be applied to luminance or chrominance classification. Whether or not MBS or LSB is used for the bitmask N can be set or changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level.

[00159] In some modes, the luminance position and C3 bitmask can be combined and changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. Different combinations may result in different classifiers.

[00160] In some modes, a “maximum number of 1s” bitmask restriction can be applied to limit the corresponding number of deviations. For example, restricting the “maximum number of 1s” bitmask to 4 in SPS makes the maximum deviations in the sequence 16. The bitmask in different POCs may be different, but the “maximum number of 1s” must not exceed 4 (the total number of classes must not exceed 16). The “maximum number of 1s” value can be signaled and changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. POC Classifier 10-bit bitmask C3 Total classes 0 C3 using position YO 111100 0000 16 Luminance sample values ​​class index 00 0000 1111 0 (0000) 101011 0011 9 (1010) 111100 1001 15(1111) POC Classifier 10-bit bitmask C3 Total classes 1 C3 using position Y4 10 0100 0001 8 Luminance sample value class index 00 0000 1111 1 (001) 10 1011 0011 5(101) 11 1W0 1001 7(111) Table 6-2: The example of a classifier that uses a bitmask for classification (the position of the bitmask is underlined).

[00161] In some modes, as shown in Figure 11, other cross-component chrominance samples, for example, chrominance sample 1102 and its adjacent samples, can also be fed into CCSAO classification, for example, for current chrominance sample 1104. For example, Cr chrominance samples can be fed into CCSAO Cb classification. Cb chrominance samples can be fed into classification CCSAO Cr. The cross-component chrominance sample classifier may be the same as the cross-component luminance classifier or may have its own classifier, as disclosed in this description. The two classifiers may be combined to form a joint classifier for classifying the current chrominance sample. For example, a joint classifier combining cross-component luminance and chrominance samples produces a total of 16 classes, as shown in Table 6-3. pzi«nn / cznz / e / Yi POC Classifier classes Total classes 0 Combine C3 using position Y4 Bitmask: woioooooi 8 C0 using co-localized cross-chrominance position C0 band_num:2 2 16 Table 6-3: Example of a classifier using a conjoint classifier that combines cross-component luminance and chrominance samples (mask position is underlined)

[00162] All the above classifiers (C0, C1, C1', C2, C3) can be combined. For example, see Table 6-4. POC Classifier Total classes 0 Combine C0, C1 and C2 C0 band_num: 4 C2 band_num: 4 4*17*4 1 Combine C0, C1' and C2 C0 band_num: 6 C2 band_num: 4 6*9*4 2 Combine C1 and C3 The number C3 of 1s: 4 16*17 Table 6-4: Different classifiers are combined.

[00163] In some modes, an example classifier (C2) uses the difference (Yn) of co-located and contiguous luminance samples. Figure 12(c) shows an example of Yn, which has a dynamic range of [-1024, 1023] when the bit depth is 10. Where band_num C2 is the number of evenly divided bands of the dynamic range Yn, Class (C2) = (Yn + (1 « bit_depth) * band_num) » (bit_depth + 1).

[00164] In some modes, C0 and C2 are combined to produce a general classifier. For example, different classifiers are applied to different images (different POCs), as shown in Table 7. POC Classifier C0 band_num C2 band_num Total classes 0 combine CO and C2 16 16 16*17 1 combine CO and C2 8 7 8*7 Table 7: Different general classifiers apply to different images.

[00165] In some modes, all the aforementioned classifiers (C0, C1, C1', C2) are combined. For example, different classifiers are applied to different images (different POCs), as shown in Table 8-1. POC Classifier C0 band_num C2 band_num Total classes 0 combine CO, C1 and C2 4 4 4*17*4 1 combine C0, C1' and C2 6 4 6*9*4 Table 8: Different general classifiers apply to different images.

[00166] In some modes, multiple classifiers are used in the same POC. The current frame is divided into many regions, and each region uses the same classifier. For example, 3 different classifiers are used in POCO, and which classifier (0, 1, or 2) is used and signaled at the CTU level, as shown in Table 9. POC Classifier C0 band_num Region 0 C0 using position YO 16 0 0 C0 using position YO 8 1 0 C0 using position Y1 8 2 Table 9: Different general classifiers are applied to different regions in the same image.

[00167] In some modalities, the maximum number of plural classifiers (plural classifiers may also be called alternate deviation sets) may be fixed or signaled at the SPS / APS / PPS / PH / SH / CTU / CU level. In one example, the fixed (predefined) maximum number of plural classifiers is 4. In that case, 4 different classifiers are used in POCO, and which classifier (0, 1, or 2) is used and signaled at the CTU level. The truncated unary code (TU) may be used to indicate the classifier used for each luminance or chrominance CTB. For example, as shown in Table 10, when the TU code is 0: CCSAO does not apply; when the TU code is 10: set 0 applies; when the TU code is 110, set 1 applies; when the TU code is 1110: set 2 applies; When the TU code is 1111: set 3 applies.The fixed-length code, the Golomb-Rice code, and the exponential Golomb code can also be used to indicate the classifier (deviation set index) for CTB. Three different classifiers are used in POC1. POC Classifier C0 band_num Region Code TU 0 C0 using position Y3 6 0 10 0 C0 using position Y3 7 1 110 0 C0 using position Y1 3 2 1110 0 C0 using position Y6 6 3 1111 1 C0 using position YO 16 0 10 1 C0 using position YO 8 1 110 1 C0 using position Y1 8 2 1110 Table 10: The truncated unary code (TU) is used to indicate the classifier used for each chrominance CTB.

[00168] An example of Cb and Cr CTB offset set indices is provided for the 1280x720 POCO sequence (the number of CTUs in a frame is 10x6 if the CTU size is 128x128). Cb POCO uses 4 offset sets and Cr uses 1 offset set. As shown in Table 11-1, when the offset set index is 0: CCSAO is not applied; when the offset set index is 1: set 0 is applied; when the offset set index is 2: set 1 is applied; when the offset set index is 3: set 2 is applied; when the offset set index is 4: set 3 is applied. Type means the position of the chosen co-localized luminance sample (Yi). Different sets of deviations can have different types, band_num, and corresponding deviations. q?1 «nn / czn^ / e / γ Table 11-1: An example of CTB deviation set indices for Cb and Cr is provided for the 1280x720 POCO sequence (the number of CTUs in a frame is 10x6 if the size of the CTU is 128x128).

[00169] In some modes, an example of the joint use of co-located / current and contiguous Y / U / V samples for classification (joint 3-component bandNum classification for each Y / U / V component) is provided in Table 11-2. In POCO, offset sets {2,4,1} are used for {Y, U, V}, respectively. Each offset set can be adaptedally changed at the SPS / APS / PPS / PH / SH / CTU / CU / Sub-block level. Different offset sets can have different classifiers. For example, as a candidate position (candPos) indicated in Figures 6B and 6C for classifying the current Y4 luminance sample, Y seto selects {current Y4, co-located U4, co-located V4} as candidates, with different bandNum {Y, U, V} = {16,1,2}, respectively. With {candY, candU, candV} as the sample values ​​of the selected candidates {Y, U, V}, the total number of classes is 32, and the derivation of the class index can be shown as: bandY = (candY * bandNumY) » BitDepth; bandU = (candU * bandNumU) » BitDepth; bandV = (candV * bandNumV) » BitDepth; classldx = bandY * bandNumU * bandNumV + bandU * bandNumV + bandV;

[00170] Another example is in the set1 V classification of component POC1. In that example, candPos = {contiguous Y8, contiguous U3, contiguous V0} is used with bandNum = {4,1,2}, which produces 8 classes. POC Comp current set of Classifier: candPos(Y,U,V) with bandNum (Y,U,V) deviations i Total classes (number of deviations) 0 Y 0[(Y4, U4, V4), (16, 1,2) 16*1*2=32 1 l(Y4, U0, V2), (15, 4, 1) 15*4*1=60 u 0|(Y8, U3, V0), (1,1,2) 2 1 ¡(Y4. U1, V0), (15, 2, 2) 60 2¡(Y6, U6, V6), (4,4, 1) 16 3¡(Y2, U0, V5), (1,1,1) 1 V 0¡(Y2, U0, V5), (1,1,1) 1 Y 0|(Y4, U1, V0), (15, 2, 2) 60 u 0j(Y6, U2, V1), (7, 1,2) 14 V oj(Y8. U3, V0), (1,1,2) 2 1 |(Y8, U3, V0), (4, 1,2) 8 Table 11-2: Example of joint use of co-located / current and contiguous Y / U / V samples for classification.

[00171] In some modes, an example is provided of the joint use of co-located and contiguous Y / U / V samples for the classification of the current Y / U / V sample (joint classification of 3-component edgeNum (C1s) and bandNum for each Y / U / V component), as shown in Table 11-3. Edge CandPos is the centered position used for the C1s classifier, edge bitMask is the C1s contiguous sample activation indicator, and edgeNum is the corresponding number of C1s classes. In this example, C1s is applied only to the Y classifier (so edgeNum equals edgeNumY), where edge candPos is always Y4 (the current / co-located sample position). However, C1s can be applied to the Y / U / V classifiers, where edge candPos is the contiguous sample position.

[00172] If diff denotes the comparison score of Y C1s, the derivation of classldx can be bandY = (candY * bandNumY) » BitDepth; bandU = (candU * bandNumU) » BitDepth; bandV = (candV * bandNumV) » BitDepth; edgeldx = diff + (edgeNum » 1); bandldx = bandY * bandNumU * bandNumV + bandU * bandNumV + bandV; classldx = bandldx * edgeNum + edgeldx; P71 «nn / P7n7 / e / YiAi POC 0 Current component YU Set 0 1 2 0 edge candPos(y) (Y4) (Y4) (Y4) (Y4) edge bitmask(Y) 10000001 00010000 01111110 10000000 edgeNum 5, [-2,2] 3, [-1,1] 13, [-6,6] 3, [-1,1] band candPos(Y,U,V) (Y4, U4, V4) (Y4, U0, V2) (Y4, U1, V2) (Y8, U3, V0) bandNum(Y,U,V) (16, 1, 2) (15, 4, 1) (2,1,1) (1, 1,2) Total classes 5*16*1*2=160 3*15*4*1=180 13*2*1*1*=26 3*rrr2=6 Signaled deviations 160 deviation values: (3, 3, 2, 1 ...) 180 deviation values ​​26 deviation values ​​6 deviation values ​​idx of the classified set 0 1 2 0 Component YYYU Table 11-3 (part 1): Example of joint use of co-located / current and contiguous Y / U / V samples for classification. POC 1 Current Component VY Set 1 2 3 0 0 edge candPos(y) (Y4) (Y4) (Y4) (Y4) reuse edge bitmask(Y) 00000000 00000000 10000001 00000000 reuse edgeNum 1, [0] 1, [0] 5, [-2,2] 1,[0] reuse band candPos(Y,U,V) (Y4, L)1, V0) (Y6, U6, V6) (Y2, U0, V5) (Y2, U0, V5) reuse bandNum(Y,U,V) (15,2,2) (4,1,1) (1,1, 1) (1,1,1) reuse Total classes 60 4 1 1 160 Signaled deviations 60 deviation values ​​4 deviation values: (1,2, 0, 1) 1 deviation value 1 deviation value signal idxY = 0, reuse parameters and deviations (3, 3, 2, -1...) Idx of the classified set 1 2 3 0 UUUVY component Table 11-3 (part 2): Example of joint use of co-located / current and contiguous Y / U / V samples for classification. POC Current Component UV Set 1 0 0 1 edge candPos(y) (Y4) reuse (Y4) (Y4) edge bitmask(Y) 11111111 reuse 00000000 00000000 edgeNum 17, [-8,8] reuse 1, [0] 1,[0] band candPos(Y,U,V) (Y4, U1, V2) reuse (Y8, U3, V0) (Y8, U3, V0) bandNum(Y,U,V) (4, 1,1) reuse (1,1,2) (4,1,2) Total classes 17*4*1*1=68 4 2 8 Signaled deviations 68 deviation values ​​signal idxU = 2, reuse parameters and deviations (1, 2, 0, 1) 2 deviation values ​​8 deviation values ​​Idx of the classified set 3 YUVV component Table 11-3 (part 3): Example of joint use of co-located / current and contiguous Y / U / V samples for classification.

[00173] In some modes, the maximum band_num (bandNumY, bandNumU, or bandNumV) can be set or signaled at the SPS / APS / PPS / PH / SH / CTU / CU level. For example, setting the maximum band_num=16 in the decoder means that for each frame, 4 bits are signaled to indicate the C0 band_num in that frame. Some other examples of maximum band_num are provided in Table 12. Band_num_min Band_num_max Band_num bit 1 1 0 1 2 1 1 4 2 1 8 3 1 16 4 1 32 5 1 64 6 1 128 7 1 256 8 Table 12: Examples of band_num and maximum band_num bit.

[00174] In some modes, the maximum number of classes or deviations (combinations of multiple classifiers using the same classifier, for example, C1s edgeNum * C1 bandNumY * bandNumU * bandNumV) for each set (or the entire aggregated set) may be fixed or signaled at the SPS / APS / PPS / PH / SH / Region / CTU / CU level. For example, max. is fixed for all aggregated sets of class_num=256*4, and an encoder compliance check or a decoder normative check may be used to verify the restriction.

[00175] In some modes, a restriction can be applied to the C0 classification, for example, restricting band_num (bandNumY, bandNumU, or bandNumV) to only be a power of 2 values. Instead of explicitly signaling band_num, a band_num_shift syntax is signaled. The decoder can use the offset operation to avoid multiplication. A band_num_shift can be used for different components. Class (CO) = (ME » band_num_shift) » bit_depth

[00176] Another example of operation involves taking rounding into consideration to reduce error. Class (CO) = ((I + (1 « (band_num_shift -1))) » band_num_shift) » bit_depth

[00177] For example, if band_num_max (Y, U or V) is 16, the possible band_num_shift candidates are 0, 1,2, 3, 4, which correspond to band_num = 1,2, 4, 8, 16, as shown in Table 13. POC Classifier C0 band_num_shift C0 band_num Total classes 0 C0 using position YO 4 16 16 1 C0 using position Y7 3 8 8 pzi«nn / cznz / e / Yi Band_num_max valid band_num Candidates Band_num_shift 1 1 0 2 1, 2 0, 1 4 1,2, 4 0, 1,2 8 1, 2, 4, 8 0, 1, 2, 3 16 1,2, 4, 8, 16 0, 1, 2, 3, 4 32 1,2,4, 8, 16, 32 0, 1,2, 3, 4, 5 64 1,2,4, 8,16, 32, 64 0, 1, 2, 3, 4, 5, 6 128 1,2, 4, 8, 16, 32, 64, 128 0, 1,2, 3, 4, 5, 6, 7 256 1, 2, 4, 8, 16, 32, 64, 128, 256 0, 1, 2, 3, 4, 5, 6, 7, 8 Table 13: Candidates band_num and corresponding band_num_shift.

[00178] In some modes, the classifiers applied to Cb and Cr are different. The Cb and Cr deviations for all classes can be signaled separately. For example, different signaled deviations apply to different chrominance components, as shown in Table 14. POC Component Classifier C0 band_num Total classes Signaled deviations 0 Cb CO 16 16 16 0 Cr CO 5 5 5 Table 14: The Cb and Cr deviations for all classes can be signaled separately.

[00179] In some modes, the maximum deviation value is fixed or signaled at the level of Sequence Parameter Set (SPS)ZAdaptation Parameter Set (APS)ZImage Parameter Set (PPS)ZImage Header (PH)ZSegment Header (SH). For example, the maximum deviation is between [-15, 15]. Different components may have different maximum deviation values.

[00180] In some modes, deviation signaling can use differential pulse-code modulation (DPCM). For example, deviations {3, 3, 2, 1, -1} can be signaled as {3, 0, -1, -1, -2}.

[00181] In some modes, deviations can be stored in APS or in a buffer for subsequent reuse of the Zsegment image. An index can be signaled to indicate which previous frame deviations are used for the current image.

[00182] In some modalities, the Cb and Cr classifiers are the same. The Cb and Cr deviations for all classes can be signaled together, for example, as shown in Table 15. czipnnzczn^zezYi POC Component Classifier C0 band_num Total classes Signaled deviations 0 Cb and Cr CO 8 8 8 Table 15: The Cb and Cr deviations for all classes can be signaled together.

[00183] In some modes, the Cb and Cr classifier may be the same. The Cb and Cr deviations for all classes may be signaled together, with a sign indicator difference, for example, as shown in Table 16. According to Table 16, when the Cb deviations are (3, 3, 2, -1), the derived Cr deviations are (-3, -3, -2, 1). POC Component Classifier CO band_num Total classes Signaled deviations Sign indicator 0 Cb and Cr C0 4 4 4: (3, 3, 2, -1) 1:(-) Table 16: The Cb and Cr deviations for all classes can be signaled together with a sign indicator difference.

[00184] In some modalities, the sign indicator may be signaled for each class, for example, as shown in Table 17. According to Table 17, when the Cb deviations are (3, 3,2, -1), the derived Cr deviations are (-3, 3, 2,1) according to the respective signaled indicator. POC Component Classifier CO band_num Total classes Signaled deviations Sign indicator 0 Cb and Cr CO 4 4 4: (3, 3, 2,-1) 1:(-,+,+,-) Table 17: The Cb and Cr deviations for all classes can be signaled together with a sign indicator signaled for each class.

[00185] In some modes, the Cb and Cr classifiers may be the same. The Cb and Cr deviations for all classes may be signaled together, with a weight difference, for example, as shown in Table 18. The weight (w) may be selected from a limited table, for example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4...etc., where |w| only includes power values ​​of 2. According to Table 18, when the Cb deviations are (3, 3, 2, -1), the derived Cr deviations are (6, -6, -4, 2) according to the respective signaled indicator. POC Component Classifier CO band_num Total classes Signaled deviations Signaled weight 0 Cb and Cr CO 4 4 4: (3, 3, 2, -1) -2 Table 18: The Cb and Cr deviations for all classes can be signaled together with a weight difference.

[00186] In some modalities, the weight can be signaled for each class, for example, as shown below in Table 19. According to Table 19, when the Cb deviations are (3, 3, 2, -1), the derived Cr deviations are (-6, 12, 0, -1) according to the respective signaled indicator. POC Component Classifier CO bandnum Total classes Signaled deviations Signaled weight 0 Cby Cr CO 4 4 4: (3, 3, 2, -1) 4: (-2, 4, 0, 1) Table 19: The Cb and Cr deviations for all classes can be signaled together with a signaled weight for each class.

[00187] In some modalities, if plural classifiers are used in the same POC, different sets of deviations are signaled separately or jointly.

[00188] In some modes, previously decoded deviations can be stored for use in future frames. An index can be signaled to indicate which set of previously decoded deviations is used for the current frame, to reduce deviation signaling overprocessing. For example, POCO deviations can be reused through POC2 with the signaling deviation set idx = 0, as shown in Table 20. POC Component Classifier CO band_num Total classes Signaled deviations Stored deviation set idx 0 Cb CO 4 4 4: (3, 3, 2,-1) 0 0 Cr CO 4 4 4: (-2, 1,0, 1) 0 1 Cb CO 4 4 4:(0, 0, 1,-1) 1 1 Cr CO 4 4 4: (1,2, 0, 1) 1 2 Cb CO 4 4 Reuse deviations (3, 3, 2, 1) signal idx=0 2 Cr CO 4 4 Reuse deviations (-2, 1, 0, 1) signal idx=0 Table 20: An index can be signaled to indicate which set of previously decoded deviations is used for the current frame.

[00189] In some modes, the set of idx reuse deviations for Cb and Cr may be different, for example, as shown in Table 21. POC Component Classifier CO band_num Total classes Signaled deviations Stored deviation set idx 0 Cb CO 4 4 4: (3, 3, 2, -1) 0 0 Cr CO 4 4 4: (-2, 1,0, 1) 0 1 Cb CO 4 4 4: (0, 0, 1, -1) 1 1 Cr CO 4 4 4: (1,2, 0, 1) 1 2 Cb CO 4 4 Reuse deviations (3, 3, 2,-1) signal idx=0 2 Cr CO 4 4 Reuse deviations (1, 2, 0, 1) signal idx=1 Table 21: An index can be signaled to indicate which set of previously decoded deviations is used for the current frame, and the index can be different for the Cb and Cr components.

[00190] In some modes, deviation signaling can use additional syntax that includes start and length to reduce signaling overprocessing. For example, when band_num=256, only deviations from band_idx=37~44 are signaled. In the example shown in Table 22-1, both the start and length syntaxes are encoded with a fixed length of 8 bits, which must match the bits of band_num. bandjdx deviation 1 0 2 0 3 0 37 start=37 deviation[O] 38 deviation[1] band_num_max bits, start, length of band_num 39 deviation[2] 1 0 40 deviation[3] 2 1 41 deviation[4] 4 2 42 deviation[5] 8 3 43 deviation[6] 16 4 44 length=8 deviation[7] 32 5 64 6 255 0 128 1 256 0 256 8 pzi«nn / cznz / e / Yi Table 22-1: Deviation signaling uses additional syntax that includes start and length.

[00191] In some modalities, if CCSAO is applied to all three YUV components, co-located and contiguous YUV samples can be used together for classification, and all the aforementioned deviation signaling methods for Cb / Cr can be extended to Y / Cb / Cr. In some modalities, different sets of component deviations can be stored and used separately (each component has its own stored sets) or jointly (each component shares / reuses the same stored set). Table 22-2 shows an example of a separate set. POC Current Component Deviation Set Classifier: candPos(Y, U, V) with bandNum (Y, U, V) Total classes (number of deviations) Signaled deviations Classified deviation set idx 0 Y 0 (Y4, U4, V4), (16, 1,2) 16*1*2=32 32 deviation values ​​(3, 3, 2, 1, ...) 0 Y 1 (Y4, U0, V2), (14, 4,1) 15*4*1=60 60 deviation values ​​0 YU 0 (Y8, U3, V0), (1,1,2) 2 2 deviation values ​​0 U 1 (Y4, U1, V0), (15,2,2) 60 60 deviation values ​​1 U 2 (Y6, U6, V6), (4,1,1) 4 4 deviation values ​​2 U 3 (Y2, U0, V5), (1,1,1) 1 1 deviation value 3 UV 0 (Y2, UO, V5), (1,1,1) 1 1 deviation value 0 V 1 Y 0 Reuse stored deviation set Y idx 0 32 Signal idx = 0 and reuse deviations (3, 3, 2, -1, ...) YU 0 Reuse stored deviation set U idx 2 4 Signal idx = 2 and reuse deviations (1, 2, 0, 1) UV 0 (Y8, U3, VO), (1,1,2) 2 2 deviation values ​​V 1 (Y8, U3, VO), (4,1,2) 8 8 deviation values ​​V. Table 22-2: An example showing that different sets of deviations can be stored and used separately (each component has its own stored sets) or jointly (each component shares / reuses the same stored set).

[00192] In some modes, if a sequence bit depth is greater than 10 (or a certain bit depth), the deviation can be quantized before it is signaled. On the decoder side, the decoded deviation is dequantized before being applied, as shown in Table 23. For example, for a 12-bit sequence, the decoded deviations are shifted (dequantized) by 2. Signaled deviation Dequantified and applied deviation 0 0 1 4 2 8 3 12 14 56 15 60 Table 23: The decoded deviation is dequantified before applying it.

[00193] In some modes, the deviation can be calculated as CcSaoOffsetVal=( 1 - 2 * ccsao_offset_sign_flag ) * (ccsao_offset_abs « (BitDepth - Min( 10, BitDepth)))

[00194] In some modalities, sample processing is described below. Where R(x, y) is the sample luminance or chrominance value before CCSAO; R'(x, y) is the output sample luminance or chrominance value after CCSAO: offset = ccsao_offset [classjndex of R(x, y)] R'(x, y) = Clip3( 0, (1 « bit_depth) - 1, R(x, y) + offset)

[00195] According to the equations above, each sample luminance or chrominance value R(x, y) is classified using the indicated classifier of the current image and / or the idx of the current set of deviations. The corresponding deviation of the derived class index is added to each sample luminance or chrominance value R(x, y). A Clip 3 clipping function is applied to (R(x, y) + offset) to make the output sample luminance or chrominance value R'(x, y) within the dynamic bit-depth range, e.g., range 0 to (1 « bit_depth) - 1.

[00196] In some modalities, boundary processing is described. If any of the co-located and contiguous luminance (chrominance) samples used for classification are outside the current image, CCSAO is not applied to the current chrominance (luminance) sample. Figure 13A is a block diagram illustrating that CCSAO is not applied to the current chrominance (luminance) sample if any of the co-located and contiguous luminance (chrominance) samples used for classification are outside the current image, according to some implementations of this description. For example, in Figure 13A(a), if a classifier is used, CCSAO is not applied to the chrominance components in the leftmost column 1 of the current image. For example, if CT is used, CCSAO does not apply to the left column 1 and the chrominance components in the top row 1 of the current image, as shown in Figure 13A(b).

[00197] Figure 13B is a block diagram illustrating that CCSAO is applied to the current luminance or chrominance sample if any of the co-located and contiguous luminance or chrominance samples used for classification are outside the current image, according to some implementations of this description. In some embodiments, if any of the co-located and contiguous luminance or chrominance samples used for classification are outside the current image, one variation involves using the missing samples repeatedly, as shown in Figure 13B(a), or applying a mirror fill to the missing samples to create samples for classification, as shown in Figure 13B(b), and CCSAO can then be applied to the current luminance or chrominance samples.

[00198] Figure 14 is a block diagram illustrating that CCSAO does not apply to the current chrominance sample if a corresponding selected co-located or contiguous luminance sample used for classification lies outside a virtual space defined by a virtual boundary, according to some implementations of this description. In some modalities, a virtual boundary (VB) is a virtual line that separates the space within an image frame. In some modalities, if a virtual boundary (VB) is applied to the current frame, CCSAO does not apply to chrominance samples that have a corresponding selected luminance position outside a virtual space defined by the virtual boundary. Figure 14 shows an example with a virtual boundary for the C0 classifier with 9 luminance position candidates.For each CTU, CCSAO does not apply to chrominance samples for which the corresponding selected luminance position lies outside a virtual space enclosed by the virtual boundary. For example, in Figure 14(a), CCSAO does not apply to chrominance sample 1402 when the position of the selected luminance sample Y7 lies beyond the horizontal virtual boundary 1406, which is located 4 pixel lines from the bottom of the frame. For example, in Figure 14(b), CCSAO does not apply to chrominance sample 1404 when the position of the selected luminance sample Y5 lies beyond the vertical virtual boundary 1408, which is located n pixel lines from the right side of the frame.

[00199] Figure 15 shows that repetitive or mirror filling is applied to luminance samples that lie outside the virtual boundary according to some implementations of this description. Figure 15(a) shows an example of repetitive filling. If the original Y7 is selected as the classifier located on the bottom side of VB 1502, the value of luminance sample Y4 is used for classification (copied to position Y7), instead of the value of the original luminance sample Y7. Figure 15(b) shows an example of mirror filling. If Y7 is selected as the classifier located on the bottom side of VB 1504, the value of luminance sample Y1, which is symmetric to the value of Y7 with respect to the luminance sample, is used for classification, instead of the value of the original luminance sample Y7.Filling methods provide a greater possibility that chrominance samples will apply CCSAO, so that a greater coding gain can be achieved.

[00200]

[00215] In some modalities, a restriction may be applied to reduce the required CCSAO line buffer and to simplify condition verification processing. Figure 16 shows that one additional luminance line buffer—that is, the full-line luminance samples from line -5 above the current VB 1602—may be required if the nine co-located and contiguous luminance samples are used for classification according to some implementations of this description. Figure 10B(a) shows an example that uses only six luminance candidates for classification, which reduces the line buffer and does not require any additional boundary verification in Figure 13A and Figure 13B.

[00201] In some modes, using luminance samples for CCSAO classification can increase the luminance line buffer and thus increase the hardware implementation cost of the decoder. Figure 17 shows an illustration in AVS where the CCSAO of 9 luminance candidates crossing VB 1702 can add 2 additional luminance line buffers according to some implementations of this description. For luminance and chrominance samples above Virtual Limit (VB) 1702, DBF / SAO / ALF are processed in the current CTU row. For luminance and chrominance samples below VB 1702, DBF / SAO / ALF are processed in the next CTU row. In the AVS decoder hardware design, luminance samples from line -4 to -1 before DBF, samples from line -5 before SAO, and chrominance samples from line -3 to -1 before DBF, as well as samples from line -4 before SAO, are stored as line buffers for processing by DBF / SAO / ALF in the next CTU row. When the next row of CTUs is processed, luminance and chrominance samples that are not in the line buffer are not available.However, for example, at the chrominance line position -3(b), the chrominance sample is processed in the next CTU row, but CCSAO needs the luminance sample lines -7, -6, and -5 before SAO for classification. The luminance sample lines -7 and -6 before SAO are not in the line buffer, so they are unavailable. Furthermore, adding the luminance sample lines -7 and -6 before SAO to the line buffer would increase the decoder hardware implementation cost. In some examples, the luminance VB (line -4) and the chrominance VB (line -3) may be different (not aligned).

[00202] Similar to Figure 17, Figure 18A shows an illustration in WC where the CCSAO of 9 luminance candidates crossing VB 1802 can increase by 1 additional luminance line buffer according to some implementations of this description. The VB may differ in different standards. In VVC, the luminance VB is line -4 and the chrominance VB is line -2, so the CCSAO of 9 candidates can increase by 1 luminance line buffer.

[00203] In some embodiments, in a first solution, CCSAO is disabled for a chrominance sample if any of the luminance candidates of the chrominance sample are on the other side of the VB (outside the VB of the current chrominance sample). Figures 19A to 19C show in AVS and WC that CCSAO is disabled for a chrominance sample if any of the luminance candidates of the chrominance sample are on the other side of VB 1902 (outside the VB of the current chrominance sample) according to some implementations of the present description. Figure 14 also shows some examples of this implementation.

[00204] In some modes, in a second solution, repetitive fill is used for CCSAO starting from a luminance line near the VB and on the other side of the VB, for example, the luminance line -4, for luminance candidates that “cross the VB.” In some modes, repetitive fill starting from the lower VB of the nearest contiguous luminance element is implemented for chrominance candidates that “cross the VB.” Figures 20A to 20C show in AVS and VVC that CCSAO is enabled by the use of repetitive fill for a chrominance sample if any of the luminance candidates in the chrominance sample are on the other side of VB 2002 (outside the VB of the current chrominance sample) according to some implementations of this description. Figure 14(a) also shows some examples of this implementation.

[00205] In some embodiments, in a third solution, mirror fill is used for CCSAO starting from the lower luminance VB for luminance candidates that “cross the VB.” Figures 21A to 21C show in AVS and VVC that CCSAO is enabled by using mirror fill for a chrominance sample if any of the luminance candidates in the chrominance sample are on the other side of VB 2102 (outside the VB of the current chrominance sample) according to some implementations of the present description. Figures 14(b) and 13B(b) also show some examples of this implementation. In some embodiments, in a fourth solution, “two-sided symmetric fill” is used to apply CCSAO.Figures 22A to 22B show that CCSAO is enabled by using two-sided symmetric fill for some examples of different CCSAO shapes (e.g., 9 luminance candidates (Figure 22A) and 8 luminance candidates (22B)) according to some implementations of the present description. For a luminance sample set with a co-located luminance sample centered on a chrominance sample, if one side of the luminance sample set lies outside VB 2202, a two-sided symmetric fill is applied to both sides of the luminance sample set. For example, in Figure 22A, luminance samples Y1, Y1, and Y2 lie outside VB 2202, so both Y1, Y1, Y2 and Y6, Y7, Y8 are filled using Y3, Y4, Y5. For example, in Figure 22B, the luminance sample YO is outside of VB 2202, so YO is filled using Y2, while Y7 is filled using Y5.

[00206] Figure 18B shows an illustration in which, when colocalized or contiguous chrominance samples are used to classify current luminance samples, the selected chrominance candidate may be on the other side of the VB and require additional chrominance line buffering according to some implementations of this description. Solutions similar to those described 1 through 4 above can be applied to address this problem.

[00207] Solution 1 involves disabling the CCSAO for a luminance sample when any of its chrominance candidates might be on the other side of the VB.

[00208] Solution 2 consists of using repetitive padding from the lower VB of the nearest contiguous chrominance element for chrominance candidates that “cross the VB”.

[00209] Solution 3 consists of using mirror fill from the lower chrominance VB for chrominance candidates that “cross the VB”.

[00210] Solution 4 involves using “two-sided symmetric filling.” For a candidate set centered on the CCSAO co-localized chrominance sample, if one side of the candidate set is outside the VB, two-sided symmetric filling is applied to both sides.

[00211] Fill methods allow more luminance or chrominance samples to have the possibility of applying CCSAO so that a greater encoding gain can be achieved.

[00212] In some modes, in the bottom CTU row of the image boundary (or segment, tile, brick), samples below the VB are processed in the current CTU row, so the special handling mentioned above (Solutions 1, 2, 3, 4) does not apply to the bottom CTU row of the image boundary (or segment, tile, brick). For example, a 1920x1080 frame is divided among 128x128 CTUs. A frame contains several 15x9 CTUs (rounded up). The bottom CTU row is the 15th CTU row. The decoding process goes from CTU row to CTU row, and from CTU to CTU for each CTU row. Unblocking must be applied along the horizontal CTU boundaries, between the current CTU row and the next CTU row.The CTB VB applies to each CTU row because within a CTU, in the lower luminance / chrominance line of 4 / 2, the DBF samples (in the case of VVC) are processed in the next CTU row and are not available to CCSAO in the current CTU row. However, in the lower CTU row of the image frame, the DBF samples from the lower luminance / chrominance line of 4 / 2 are available in the current CTU row, since there is no next CTU row and they are processed into DBF in the current CTU row.

[00213] In some modes, a restriction may be applied to reduce the line buffer required by CCSAO and to simplify the boundary processing condition check, as explained in Figure 16. Figure 23 shows the restrictions on using a limited number of luminance candidates for classification according to some implementations of this description. Figure 23(a) shows the restriction of using only 6 luminance candidates for classification. Figure 23(b) shows the restriction of using only 4 luminance candidates for classification.

[00214] In some modes, the applied region is implemented. The CCSAO applied region unit can be based on CTBs. That is, the on / off control, the CCSAO parameters (deviations, luminance candidate positions, band_num, bitmask, etc., used for classification, the deviance set index) are the same in a CTB.

[00215] In some modes, the applied region may not align with the CTB boundary. For example, the applied region does not align with the chrominance CTB boundary but is offset. The syntax (on / off control, CCSAO parameters) is still signaled for each CTB, but the actually applied region does not align with the CTB boundary. Figure 24 shows that the CCSAO applied region does not align with the CTB / CTU boundary 2406 according to some implementations of this description. For example, the applied region does not align with the chrominance CTB / CTU boundary 2406 but with samples (4, 4) offset to the upper left, toward VB 2408. This non-aligned CTB boundary design benefits the unlocking process, as the same unlocking parameters are used for each region of the 8x8 unlocking process.

[00216] In some modes, the CCSAO applied region unit (mask size) may be variable (larger or smaller than the CTB size), as shown in Table 24. The mask size may differ for different components. The mask size can be changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. For example, in PH, a series of mask on / off indicators and deviation set indices are signaled to indicate information for each CCSAO region. pzi«nn / cznz / e / Yi POC Component CTB Size Mask Size 0 Cb 64x64 128x128 0 Cr 64x64 32x32 1 Cb 64x64 16x16 1 Cr 64x64 256x256 Table 24: The CCSAO applied region unit (mask size) may be variable.

[00217] In some modes, the frame splitting in the applied CCSAO region can be fixed. For example, splitting the frame into N regions. Figure 25 shows that the frame splitting in the applied CCSAO region can be fixed with CCSAO parameters according to some implementations of this description.

[00218] In some modes, each region may have its own region on / off control flag and CCSAO parameters. Additionally, if the region size is larger than the CTB size, it may have both CTB on / off control flags and region on / off control flags. Figures 25(a) and (b) show some examples of splitting the frame into N regions. Figure 25(a) shows the vertical split into 4 regions. Figure 25(b) shows the square split into 4 regions. In some modes, as with the image-level CTB on / off control flag (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag), if the region on / off control flag is disabled, the CTB on / off flags may also be signaled.Otherwise, the CCSAO applies to all CTBs in that region without subsequent signaling of CTB indicators.

[00219] In some modes, different CCSAO applied regions can share the same region on / off control and CCSAO parameters. For example, in Figure 25(c), region 0-2 shares the same parameters and region 3-15 shares the same parameters. Figure 25(c) also shows that the region on / off control indicator and CCSAO parameters can be signaled in a Hilbert scan order.

[00220] In some modes, the CCSAO applied region unit may exhibit quaternary tree / binary tree / ternary tree separation at the image / segment / CTB level. As with CTB separation, a series of separation indicators are signaled to indicate the division of the CCSAO applied region. Figure 26 shows that the CCSAO applied region may exhibit binary tree (BT) / quaternary tree (QT) / ternary tree (TT) separation at the frame / segment / CTB level according to some implementations of this description.

[00221] Figure 27 is a block diagram illustrating a plurality of classifiers used and switched at different levels within an image frame according to some implementations of the present description. In some modes, if a plurality of classifiers is used in a frame, the method of applying the classifier set index can be changed at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block level. For example, four classifier sets are used in a frame, switched at PH, as shown in Table 25. Figures 27(a) and (c) show a region classifier set by default. Figure 27(b) shows that the classifier set index is signaled at the mask / CTB level, where 0 means CCSAO disabled for this CTB, and 1–4 means set index. POC 0 Square division into 4 regions (same as QT frame separation to a maximum depth of 1) (a) 1 CTB level, change classifier (b) 2 Vertical division into 4 regions (c) 3 QT frame separation to a maximum depth of 2 Table 25: Four sets of classifiers are used on a frame with changed PH.

[00222] In some modes, in the case of a default region, a region level indicator may be displayed if the CTBs in that region do not use the default set index (for example, the region level indicator is 0), but instead use a different set of classifiers in that frame. For example, if the default set index is used, the region level indicator is 1. For example, in a square split into 4 regions, the following sets of classifiers are used, as shown in Table 26, POC Region Indicator Uses default set index 0 1 1 Uses default sets: 1 2 1 Uses default sets: 2 3 1 Uses default sets: 3 4 0 The CTB changes the set from 1 to 4 Table 26: A region level indicator can be signaled to show if the CTBs in that region do not use the default set index.

[00223] Figure 28 is a block diagram illustrating that the CCSAO applied region division can be dynamic and changed at the image level, according to some implementations of this description. For example, Figure 28(a) shows that 3 sets of CCSAO offsets are used in this POC (set_num = 3), so the image frame is divided vertically into 3 regions. Figure 28(b) shows that 4 sets of CCSAO offsets are used in this POC (set_num = 4), so the image frame is divided horizontally into 4 regions. Figure 28(c) shows that 3 sets of CCSAO offsets are used in this POC (set_num = 3), so the image frame is divided in frames into 3 regions. Each region can have its own region enable flag to store for each CTB enable / disable control bit. The number of regions depends on the signaled image set_num.

[00224] In some modes, the implemented CCSAO syntax is shown in Table 27. In AVS3, the term patch is similar to segment, while patch header is similar to segment header. FLC means fixed-length code. TU means truncated unary code. EGk means Golomb exponential code of order k, where k can be fixed. Level Syntax Element Binarization Meaning SPS cc_sao_enabled_flag FLC If CCSAO is enabled in the sequence, it can be inferred that it is disabled (disabled state) when chromaFormat is CHROMA 400 PH / SH ph_cc_sao_y_flag ph_cc_sao_cb_flag Ph_cc_sao_cr_flag FLC If CCSAO is enabled in this image / segment for Y / Cb / Cr, it can be inferred that it is disabled (disabled state) (disabled) when chromaFormat is CHROMA400 PH / SH P h_cc_sao_sto red_off sets_s etjdx FLC which set of previously decoded deviations is used, the Y / U / V deviation set can be independent or shared PH / SH ph_cc_sao_y_ctb_control_fla g ph_cc_sao_cb_ctb_control_fl ag ph_cc_sao_cr_ctb_controljl ag FLC whether or not Y / Cb / Cr on / off control is enabled at the CTB level PH / SH P h_cc_sao_y_set_n u m_m inu s1 P h_cc_sao_cb_set_n u m_m in us1 P h_cc_sao_cb_set_n u m_m in us1 UVLC the number of alternate sets used in the image / segment.SPS / APS / PPS / PH / SH / CTU P h_cc_sao_y_c I ass_y_e nab I edjlag ph_cc_sao_y_class_u_enabl edjlag P h_cc_sao_y_c I ass_v_e nab I edjlag P h_cc_sao_cb_cl ass_y_e na bledjlag P hccsaocbcl assue na bledjlag P h_cc_sao_cb_cl ass_v_e na bledjlag ph_cc_sao_cr_class_y_enab ledjlag ph_cc_sao_cr_class_u_enab ledjlag ph_cc_sao_cr_class_v_enab ledjlag FLC whether or not the current component can use other components for classification, for example, if ph_cc_sao_y_class_u_enabledflag = 0 The Y component cannot use the Cb sample for classification, and it is not necessary to signal a classification parameter, such as bandNumLJ. Otherwise, if the indicator is 1, Cb can be used to classify the current Y. SPS / APS / ph_cc_sao_y_band_num_y_FLC band numbers modified in shape. PPS / PH / SH / CTU minusl ph_cc_sao_y_band_num_u_ minusl ph_cc_sao_y_band_num_v_ minusl ph ccsao cb band num y _minus1 ph_cc_sao_cb_band_num_u _minus1 ph_cc_sao_cb_band_num_v _minus1 ph_cc_sao_cr_band_num_y _minus1 ph cc_sao_cr band_num u _minus1 ph_cc_sao_cr_band_n u m_v _minus1 adaptive for classification, for example, ph_cc_sao_cb_band_nu m_y_m in us 1 ph_cc_sao_cb_band_num_u_minus1 ph_cc_sao_cb_band_nu m_v_m in us 1 indicates, for the classification of component Cb, Y / U / V bandNum used for joint classification of 3-component bandNum SPS / APS / PPS / PH / SH / CTU P h_cc_sao_y_cand_pos_y ph_cc_sao_y_cand_pos_u P h_cc_sao_y_cand_pos_v ph_cc_sao_cb_cand_pos_y ph_cc_sao_cb_cand_pos_u ph_cc_sao_cb_cand_pos_v ph_cc_sao_cr_cand_pos_y ph_cc_sao_cr_cand_pos_u ph_cc_sao_cr_cand_pos_v FLC Indication of the classifier candidate position, e.g., ph_cc_sao_y_cand_pos_y, ph_cc_sao_y_cand_pos_u, ph_cc_sao_y_cand_pos_v indicates, for the classification of component Y,that the candidate positions Y / U / V are selected as a joint classification of 3-component bandNum SPS / APS / PPS / PH / SH / CTU cc_sao_y_offset_sign_flag cc_sao_y_offset_abs cc_sao_cb_offset_sig n_f lag cc_sao_cb_offset_abs cc_sao_cr_offset_sign_flag cc_sao_cr_offset_abs FLC TU or EGk FLC TU or EGk CCSAO deviation values ​​of Y, Cb and Cr of each CTU class ctb_cc_sao_y_flag ctb_cc_sao_cb_flag CABAC, contexts 1 or whether CCSAO is enabled or not for the current CTB of Y, Cb or Cr, Q71 «nn / czn^ / e / Yi ctb_cc_sao_cr_f I ag 2 (top and left) CTU ctb_cc_sao_y_set_idx ctb_cc_sao_cb_set_idx ctb_cc_sao_cr_set_idx TU or EGK which CCSAO offset set is used for the current Y, Cb or Cr CTB (if CCSAO is enabled) CTU cc_sao_y_m e rg e_l eft_f lag cc_sao_y_m e rg e_u p_f lag cc_sao_cb_m e rg e_l eft_f lag cc_sao_cb_m e rg e_u p_f lag cc_sao_cr_merge_left_flag cc_sao_cr_m e rg e_u p_f I ag CABAC if the deviation by CCSAO merges or not from the left or upper CTU Table 27: Example of CCSAO syntax

[00225] If a higher-level flag is disabled, lower-level flags can be inferred from the flag's disabled state and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this image, ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag are not present and are inferred to be false.

[00226] In some modes, the SPS ccsao_enabled_flag is conditional on the SPS SAO enabled indicator, as shown in Table 28. sps_sao_enabled_flag u(1) if( sps_sao_enabled_flag && ChromaArrayType != 0) sps_ccsao_enabled_flag u(1) sps_alf_enabled_flag u(1) if( sps_alf_enabled_flag && ChromaArrayType != 0) sps_ccalf_enabled_flag u(1) Table 28: The SPS ccsao enabled flag is conditional on the SPS SAO enabled indicator.

[00227] In some modes, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag indicate whether or not the granular control of Cb / Cr CTB activation / deactivation is enabled. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag can be subsequently signaled. Otherwise, whether or not CCSAO is applied in the current image will depend on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag, without subsequently signaling ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag at the CTB level.

[00228] In some modes, for ph cc sao cb type and ph cc sao cr type, an indicator can be subsequently signaled to distinguish whether the centered luminance position (position YO in Figure 10) is used for classification, for a chrominance sample, to reduce bit overprocessing. Similarly, if cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, an indicator can be subsequently signaled using the same mechanism. For example, if the number of CO luminance position candidates is 9, cc_sao_cb_TypeO_flag is subsequently signaled to distinguish whether the central co-localized luminance position is used as shown in Table 29. If the central co-localized luminance position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining 8 contiguous luminance positions is used. ctb_cc_sao_cb_flag u(1) if( ctb_cc_sao_cb_flag) cc_sao_cb_typeO_flag u(1), can be context-coded if( !cc_sao_cb_typeO_flag) cc_sao_cb_type_idc u(3), can be context-coded Table 29: cc_sao_cb_typeO_flag is signaled to distinguish whether the central co-located luminance position is used.

[00229] Table 30 below shows an example in AVS where single classifiers (set_num = 1) or a plurality of classifiers (set_num > 1) are used in the frame. Note that the syntax notation can be mapped to the notation used previously. ccsao_parameter_picture_header_set() { for (compldx=0;compldx<2;compldx++) { picture_ccsao_enable_flag[compldx] u(1) if (PictureCcSaoEnableFlag[compldx]) { pictu re_ccsao_lcu_co ntro l_f I ag [compl dx] u(1) if (PictureCcSaoLcuControlFlag[compldx]) { p¡ ctu re_ccsa o_set_n um_minus1[compldx] u(2)} for (setldx=0; setldx<PictureCcSaoSetNum[compldx]; setldx++) { picture_ccsao_type[compldx][setldx] u(4) picture_ccsao_band_num_m¡nus1[compldx][setldx] u(4)}}}} ccsao_parameter_set() { for (compldx=0;compldx<2;compldx++) { if (PictureCcSaoEnableFlag[compldx]) { if (PictureCcSaoLcuControlFlag[compldx]) { for (Lculndex=0; Lculndex<PictureW¡dthlnLcu*P¡ctureHeightlnLcu) { ccsao_lcu_enable_flag[compldx][Lculndex] ae(v) if (CcSaoLcuEnableFlag[compldx][Lculndex] && PictureCcSaoSetNum[comp] > 1) { ccsao_lcu_set_idx[compldx][Lculndex] ae(v)}}} for (setldx=0; setldx<PictureCcSaoSetNum[comp]; setldx++) { for (i=0; ¡<PictureCcSaoBandNum[compldx][setldx]; i++){ ccsao_offset_abs[compldx][setldx][i] ae(v) if (CcSaoOffsetAbs[compldx][setldx][i]) { ccsao_offset_sign[compldx][setldx][¡] u(1)}}}}}} Table 30: Example in AVS where simple classifiers (set_num = 1) or a plurality of classifiers (set_num > 1) are used in the frame.

[00230] When combined with Figure 25 or Figure 27, where each region has its own set, the example syntax may include the region on / off control flag (picture_ccsao_lcu_control_flag[compldx][setldx]), as shown in Table 31. ccsao_parameter_picture_header_set() { for (compldx=0;compldx<2;compldx++) { picture_ccsao_enable_flag[compldx] u(1) if (PictureCcSaoEnableFlag[compldx]) { picture_ccsao_set_num_minus1 [compldx] u(2) for (setldx=0; setldx <pictureccsaosetnum[compldx]; setldx++) { picture_ccsao_lcu_control_flag[compldx][setldx] u(1) picture_ccsao_type[compldx][setldx] u(4) picture_ccsao_band_num_minus1[compldx][setldx]}}Table 31: Each region has its own set and the example syntax may include the region on / off control flag (picture_ccsao_lcu_control_flag[compldx][setldx]).

[00231] In some modalities, an extension to the SAO filter for intra- and inter-post-prediction is illustrated later. In some modalities, the SAO classification methods disclosed herein can serve as a post-prediction filter, and the prediction can be of the intra-, inter-, or other prediction type, such as Intra Block Copy. Figure 29 is a block diagram illustrating that the SAO classification methods disclosed herein serve as a post-prediction filter according to some implementations of this description.

[00232] In some modalities, a corresponding classifier is chosen for each component Y, U, and V. Furthermore, for each component prediction sample, it is first classified, and then a corresponding deviation is added. For example, each component can use the current and adjacent samples for classification. Y uses the current Y sample and the adjacent Y sample, while U / V uses the current U / V samples for classification, as shown in Table 32. Figure 30 is a block diagram illustrating that, for the post-prediction SAO filter, each component can use the current and adjacent samples for classification according to some implementations of the present description. POC Component Classifier CO band_num Total classes Deviation derived from current component 0 Y combines C0 and C1 16 16*17 h_Y[i] 0 U C0 uses position UO 8 8 h_U[i] 0 V C0 uses position VO 32 32 h_V[i] pzi«nn / cznz / e / Yi Table 32: A corresponding classifier is chosen for each component Y, U, and V.

[00233] In some modalities, the refined prediction samples (Ypred', Upred', Vpred') are updated by adding the corresponding class deviation and are subsequently used for intra, inter, or other prediction.

[00234] Ypred' = clip3(0, (1 « bit_depth)-1, Ypred + h_Y[i])

[00235] Upred' = clip3(0, (1 « bit_depth)-1, Upred + h_U[i])

[00236] Vpred' = clip3(0, (1 « bit_depth)-1, Vpred + h_V[i])

[00237] In some modes, for the U and V chrominance components, in addition to the current chrominance component, the cross component (Y) can be used for an additional deviation classification. The additional cross component deviation (h'_U, h'_V) can be added to the current component deviation (h_U, h_V), for example, as shown in Table 33. POC Component Classifier CO band_num Total classes Deviation derived from current component 0 U CO uses position Y4 16 16 h'_U[i] 0 V CO uses position Y1 7 7 h'_V[i] Table 33: For the U and V chrominance components, in addition to the current chrominance component, the cross component (Y) can be used for further deviation classification.

[00238] In some modalities, the refined prediction samples (“Upred”, “Vpred”) are updated by adding the corresponding class deviation and are subsequently used for intra, inter, or other prediction.

[00239] Upred” = clip3(0, (1 « bit_depth)-1, Upred' + h'_U[i])

[00240] Vpred” = clip3(0, (1 « bit_depth)-1, Vpred' + h'_V[i])

[00241] In some modalities, intra- and inter-prediction may use different SAO filter deviations.

[00242] Figure 31 is a flowchart illustrating an example of a 3100 process for decoding video signals that uses cross-component correlation according to some implementations of the present description.

[00243] The video decoder 30 (as shown in Figure 3) receives, from the video signal, an image frame that includes a first component and a second component. In some modes, the first component is a luminance component and the second component is a first chrominance component (3110).

[00244] The video decoder 30 determines a classifier for the first component based on a first set of one or more samples of the second component associated with a respective sample of the first component and a second set of one or more samples of the first component associated with the respective sample of the first component (3120). In some modalities, the classifier is determined by a first subclassifier and a second subclassifier, wherein the first subclassifier is determined by dividing a first dynamic range of values ​​from the first set of one or more samples of the second component into a first number of bands and by selecting a band based on an intensity value from the first set of one or more samples of the second component, and the second subclassifier is determined based on the direction and strength of the edge information from a first subgroup of the second set of one or more samples of the first component (3120-1).For example, the combined luminance edge deviation (EO) and chrominance band deviation (BO) classifier is used to classify a luminance sample.

[00245] Video decoder 30 determines a per-sample deviation for the respective sample of the first component according to classifier (3130).

[00246] Video decoder 30 modifies a sample value of the first component based on the determined sample deviation (3140).

[00247] In some modalities, the image frame also includes a third component, and wherein the classifier for the first component is further based on a third set of one or more samples of the third component associated with the respective sample of the first component, wherein the third component is a second chrominance component.

[00248] In some modalities, determining the per-sample deviation for the respective sample of the first component according to classifier (3130) includes: selecting the per-sample deviation from a plurality of deviations for the first component according to the selected band and boundary information.

[00249] In some modalities, before determining the classifier for the first component (3120), the respective sample of the first component is reconstructed by means of a loop filter and the first set of one or more samples of the second component is reconstructed by means of a loop filter, wherein the loop filter is the Unblocking Filter (DBF) or the Adaptive Deviation per Sample (SAO).

[00250] In some modalities, the first set of one or more samples of the second component associated with the respective sample of the first component is selected from one or more of the co-located and contiguous samples of the second component in relation to the respective sample of the first component.

[00251] In some modalities, the second set of one or more samples of the first component associated with the respective sample of the first component is selected from one or more of the current and contiguous samples of the first component in relation to the respective sample of the first component.

[00252] In some forms, the classifier for the respective sample of the first component is also determined by: determining a third subclassifier of the classifier by dividing a second dynamic interval of values ​​from a second subgroup of the second set of one or more samples of the first component into a second number of bands; determining a fourth subclassifier of the classifier by dividing a third dynamic interval of values ​​from a third set of one or more samples of the third component into a third number of bands; and combining the first subclassifier, the second subclassifier, the third subclassifier, and the fourth subclassifier.

[00253] In some modalities, modifying the respective sample value of the first component based on the determined sample deviation (3140) is applied recurrently.

[00254] In some modalities, the total number of classifier classes is based on a combination of the first and second subclassifiers. The total number of classes falls within a maximum range, and this maximum range is fixed or signaled at one or more of the Sequence Parameter Set (SPS), Adaptation Parameter Set (APS), Image Parameter Set (PPS), Image Header (PH), and Segment Header (SH) levels. For example, subclassifier 1 has 4 classes, subclassifier 2 has 16 classes, the combined classifier has 4x16 classes, and this 4x16 number of classes falls within a maximum range.

[00255] In some modes, modify the value of the respective sample of the first component based on the determined per-sample deviation (3140) if the first set of one or more samples of the second component associated with the respective sample of the first component is located on the same side of a virtual boundary of a Coding Tree Unit (CTU) relative to the respective sample of the first component. For example, chrominance candidates are used to classify a luminance sample. If any of the chrominance candidates of the luminance sample are not on the same side as the VB of the luminance sample, the CCSA disqualifies it for the luminance sample.

[00256] In some embodiments, if a first subset of the first set of one or more samples of the second component associated with the respective sample of the first component lies on a different side of a virtual boundary with respect to the respective sample of the first component, and a remaining subset of the first set of one or more samples of the second component associated with the respective sample of the first component lies on the same side of the virtual boundary with respect to the respective sample of the first component, the first subset of the first set of one or more samples of the second component is replaced by copying a second subset of the remaining subset of the first set of one or more samples of the second component,If the first set of one or more samples of the second component associated with the respective sample of the first component is located on a different side of the virtual boundary relative to the respective sample of the first component, the first set of one or more samples of the second component is replaced by copying a fourth set of one or more samples of the second component on the same side of the virtual boundary relative to the respective sample of the first component to replace the first set of one or more samples of the second component.

[00257] For example, chrominance candidates are used to classify a luminance sample. If any of the chrominance candidates in the luminance sample are not on the same side as the VB of the luminance sample, repetitive filling and / or mirroring is used to derive chrominance candidates on the other side of the VB of the luminance sample based on the chrominance candidates that are on the same side of the VB of the luminance sample. CCSAO is then applied to the luminance sample based on the repetitive filling and / or mirroring chrominance candidates.

[00258] In some modalities, the second subset of the remaining subset of the set of one or more samples of the second component is from a row closer to the first subset, or the second subset is located in a symmetric location of the virtual boundary or the respective sample of the first component in relation to the first subset.

[00259] In some modalities, the fourth set of one or more samples on the same side of the virtual boundary in relation to the respective sample of the first component is one row closer to the first set of one or more samples of the second component, or the fourth set of one or more samples is located in a symmetrical location of the virtual boundary or the respective sample of the first component in relation to the first set of one or more samples of the second component.

[00260] In some embodiments, if a first subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on a different side of the virtual boundary with respect to the respective sample of the first component, and a remaining subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on the same side of the virtual boundary with respect to the respective sample of the first component, the first subset in a first boundary position is replaced by copying a second subset of one or more central subsets of the remaining subset of the first set of one or more samples of the second component,and a third subset in the remaining subset in a second boundary position is replaced by copying the second subset or a fourth subset from one or more central subsets of the remaining subset of the first set of p7i«nn / C7n7 / e / Yi one or more samples of the second component.

[00261] For example, chrominance candidates are used to classify a luminance sample. If any of the chrominance candidates of the luminance sample are not on the same side as the VB of the luminance sample, two-sided symmetric filling is used to derive chrominance candidates on the other side of the VB of the luminance sample based on the chrominance candidates that are on the same side of the VB of the luminance sample. CCSAO is then applied to the luminance sample based on the two-sided symmetric filling chrominance candidates.

[00262] In some modalities, the third subset and the first subset are symmetrically placed within the first set of one or more samples of the second component.

[00263] Figure 32 shows a computing environment 3210 along with a user interface 3250. The computing environment 3210 can be part of a data processing server. The computing environment 3210 includes a processor 3220, memory 3230, and an input / output interface 3240.

[00264] The 3220 processor typically controls the general operations of the 3210 computing environment, such as operations associated with the display, data acquisition, data communications, and image processing. The 3220 processor may include one or more processors to execute instructions in order to perform all or some of the steps of the methods described above. In addition, the 3220 processor may include one or more modules that facilitate interaction between the 3220 processor and other components. The processor may be a Central Processing Unit (CPU), a microprocessor, a single-chip machine, a Graphics Processing Unit (GPU), or similar.

[00265] The 3230 memory is configured to store various types of data to support the operation of the 3210 computing environment. The 3230 memory may include default software 3232. Examples of such data include instructions for any application or methods operated in a 3210 computing environment, video data sets, image data, and so forth. The 3230 memory may be implemented using any type of volatile or non-volatile memory device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[00266] The 3240 I / O interface provides an interface between the 3220 processor and peripheral interface modules, such as a keyboard, click wheel, buttons, and the like. Buttons may include, but are not limited to, a start button, a start scan button, and a stop scan button. The 3240 I / O interface can be coupled to an encoder and a decoder.

[00267] In one embodiment, a non-transient, computer-readable storage medium is also provided comprising a plurality of programs, for example, contained in memory 3230, executable through processor 3220 in computing environment 3210, for performing the methods described above. Alternatively, the non-transient, computer-readable storage medium may have stored therein a bitstream or data stream comprising encoded video information (for example, video information comprising one or more syntax elements) generated by an encoder (for example, the video encoder 20 shown in Figure 2) using, for example, the encoding method described above, for use by a decoder (for example, the video decoder 30 shown in Figure 3) when decoding video data.The non-transient, computer-readable storage medium can be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or similar.

[00268] In one embodiment, a computing device comprising one or more processors (for example, the 3220 processor) is also provided; and the non-transient, computer-readable storage medium or memory 3230 having stored therein a plurality of programs executable through one or more processors, wherein the processor(s), once the plurality of programs is executed, are configured to perform the methods described above.

[00269] In one embodiment, a computer program product is also provided comprising a plurality of programs, for example, contained in memory 3230, executable through processor 3220 in computing environment 3210, for performing the methods described above. For example, the computer program product may include non-transient, computer-readable storage media. pzi«nn / cznz / e / Yi

[00270] In one mode, the 3210 computing environment can be implemented with one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[00271] Other modalities also include different subsets of the above modalities, combined or otherwise rearranged into other modalities.

[00272] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted on a computer-readable medium, such as one or more instructions or code, and executed through a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium, such as data storage media, or communication media, including any medium that facilitates the transfer of a computer program from one location to another, for example, in accordance with a communication protocol.Thus, computer-readable media can generally refer to (1) tangible, computer-readable, non-transient storage media or (2) a communication medium, such as a signal or carrier wave. Data storage media can be any available medium that one or more computers or one or more processors can access to retrieve instructions, code, and / or data structures to implement the applications described herein. A computer program product may include a computer-readable medium.

[00273] The terminology used in describing the implementations contained herein is for the purpose of describing specific implementations only and not to limit the scope of the claims. As used in the description of the implementations and in the appended claims, the singular forms "a," "one," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The term "and / or," as used herein, shall also be understood to refer to and encompass all possible combinations of one or more of the associated enumerated elements.It shall also be understood that the terms "includes" and / or "that includes", when used in this specification, specify the presence of the indicated attributes, elements and / or components, but do not exclude the presence or addition of one or more attributes, elements, components and / or groups thereof.

[00274] It shall also be understood that, although the terms first, second, etc., may be used herein to describe different elements, those elements shall not be limited by such terms. These terms are used only to distinguish one element from another. For example, a first electrode could be called a second electrode and, similarly, a second electrode could be called a first electrode, without departing from the scope of the implementations. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[00275] References throughout this specification to "an example" or similar phrases, whether singular or plural, mean that one or more specific attributes, structures, or features described in relation to an example are included in at least one example in this specification. Thus, the occurrence of phrases like "in an example" or similar phrases, whether singular or plural, in various parts of this specification does not necessarily refer to the same example. Furthermore, the specific attributes, structures, or features in one or more examples may be combined in any appropriate manner.

[00276] The description in this application is provided for illustrative and descriptive purposes and is not intended to be exhaustive or limited to the invention as described. Various modifications, variations, and alternative implementations will be apparent to those skilled in the art who benefit from the teachings presented in the foregoing descriptions and the corresponding drawings. The embodiment was chosen and described to explain in the least amount of terms the principles of the invention and its practical application, and to enable others skilled in the art to understand the invention for different implementations and to make the best use of the underlying principles and the various modifications in a manner appropriate for the intended use.Therefore, it should be understood that the scope of the claims should not be limited to the specific examples of the implementations described and that it is intended to include modifications and other implementations that fall within the scope of the appended claims.

Claims

1. A method for decoding a video signal, comprising: receiving, from the video signal, an image frame that includes a first component and a second component; determining a classifier for the first component based on a first set of one or more samples of the second component associated with a respective sample of the first component and a second set of one or more samples of the first component associated with the respective sample of the first component; determining a per-sample deviation for the respective sample of the first component according to the classifier;and modify a respective sample value of the first component based on the determined per-sample deviation, wherein: the first component is a luminance component and the second component is a first chrominance component, the classifier is determined by a first subclassifier and a second subclassifier, wherein the first subclassifier is determined by dividing a first dynamic range of values ​​from the first set of one or more samples of the second component into a first number of bands and by selecting a band based on an intensity value from the first set of one or more samples of the second component, and the second subclassifier is determined based on the direction and solidity of the edge information from a first subgroup of the second set of one or more samples of the first component.

2. The method according to claim 1, wherein the image frame also includes a third component, and wherein the classifier for the first component is further based on a third set of one or more samples of the third component associated with the respective sample of the first component, wherein the third component is a second chrominance component.

3. The method according to claim 1, wherein determining the per-sample deviation for the respective sample of the first component according to the classifier includes: selecting the per-sample deviation from a plurality of deviations for the first component according to the selected band and boundary information.

4. The method according to claim 1, wherein, prior to determining the classifier for the first component, the respective sample of the first component is reconstructed by means of a loop filter and the first set of one or more samples of the second component is reconstructed by means of a loop filter, wherein the loop filter is the Unblocking Filter (DBF) or the Sample Adaptive Deviation (SAO).

5. The method according to claim 1, wherein the first set of one or more samples of the second component associated with the respective sample of the first component is selected from one or more of the co-located and contiguous samples of the second component in relation to the respective sample of the first component.

6. The method according to claim 1, wherein the second set of one or more samples of the first component associated with the respective sample of the first component is selected from one or more of the current and contiguous samples of the first component in relation to the respective sample of the first component.

7. The method according to claim 2, wherein the classifier for the respective sample of the first component is also determined by: determining a third subclassifier of the classifier by dividing a second dynamic interval of values ​​from a second subgroup of the second set of one or more samples of the first component into a second number of bands; determining a fourth subclassifier of the classifier by dividing a third dynamic interval of values ​​from a third set of one or more samples of the third component into a third number of bands; and combining the first subclassifier, the second subclassifier, the third subclassifier, and the fourth subclassifier.

8. The method according to claim 1, wherein modifying the respective sample value of the first component based on the determined sample deviation is applied recurrently.

9. The method according to claim 1, wherein a total number of classifier classes is based on a combination of the first subclassifier and the second subclassifier, the total number of classes is within a maximum range, and the maximum range is fixed or signaled at one or more of the Sequence Parameter Set (SPS), Adaptation Parameter Set (APS), Image Parameter Set (PPS), Image Header (PH), and Segment Header (SH) levels.

10. The method according to claim 1, wherein modifying the respective sample value of the first component based on the per-sample deviation determined if the first set of one or more samples of the second component associated with the respective sample of the first component is located on the same side of a virtual boundary of a Tree Coding Unit relative to the respective sample of the first component.

11. The method according to claim 1, wherein: if a first subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on a different side of a virtual boundary relative to the respective sample of the first component, and a remaining subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on the same side of the virtual boundary relative to the respective sample of the first component, the first subset of the first set of one or more samples of the second component is replaced by copying a second subset of the remaining subset of the first set of one or more samples of the second component;If the first set of one or more samples of the second component associated with the respective sample of the first component is located on a different side of the virtual boundary in relation to the respective sample of the first component, the first set of one or more samples of the second component is replaced by copying a fourth set of one or more samples of the second component on the same side of the virtual boundary in relation to the respective sample of the first component to replace the first set of one or more examples of the second component.

12. The method according to claim 11, wherein the second subset of the remaining subset of the first set of one or more samples of the second component is from a row closer to the first subset, or the second subset is located in a symmetrical location of the virtual boundary or the respective sample of the first component relative to the first subset.

13. The method according to claim 11, wherein the fourth set of one or more samples on the same side of the virtual boundary relative to the respective sample of the first component is in a row closer to the first set of one or more samples of the second component, or the fourth set of one or more samples is located in a symmetrical location of the virtual boundary or the respective sample of the first component relative to the first set of one or more samples of the second component.

14. The method according to claim 1, wherein: if a first subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on a different side of a virtual boundary relative to the respective sample of the first component, and a remaining subset of the first set of one or more samples of the second component associated with the respective sample of the first component is located on the same side of the virtual boundary relative to the respective sample of the first component, the first subset in a first boundary position is replaced by copying a second subset from one or more central subsets of the remaining subset of the first set of one or more samples of the second component,and a third subset in the remaining subset in a second boundary position is replaced by copying the second subset or a fourth subset from one or more central subsets of the remaining subset of the first set of one or more samples of the second component.

15. The method according to claim 14, wherein the third subset and the first subset are positioned symmetrically within the first set of one or more samples of the second component.

16. An electronic apparatus comprising: one or more processing units; and memory coupled to the one or more processing units; wherein the one or more processing units are configured to perform the method in accordance with any of claims 1 to 15.

17. A computer-readable storage medium for decoding a video signal, the computer-readable storage medium having stored therein a bit stream comprising video information to be decoded by the method for decoding the video signal in accordance with any of claims 1 to 15.

18. A computer-readable storage medium for decoding a video signal, the computer-readable storage medium storing data, which causes an electronic apparatus having one or more processing units to perform the method in accordance with any of claims 1 to 15.

Images