Cross component sample adaptive offset

By determining classifiers and applying band offsets based on cross-component relationships, the encoding efficiency of luma and chroma components is improved, addressing inefficiencies in existing video encoding technologies and reducing bit rate requirements.

JP2024541528AActive Publication Date: 2024-11-08BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024531398
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-11-25
Filing Date
2022-11-25
Publication Date
2024-11-08
Estimated Expiration
2042-11-25

AI Technical Summary

Technical Problem

Existing video encoding technologies do not effectively utilize cross-component relationships between luma and chroma components, leading to inefficiencies in encoding efficiency.

Method used

A method and apparatus that determine a classifier for each sample of the second component based on collocated samples of the first component, using band classifiers to modify samples with band offsets, and signal indicators in the bitstream to improve encoding efficiency by leveraging cross-component relationships.

Benefits of technology

Enhances encoding efficiency by optimizing the encoding process through the use of band classifiers and band offsets, reducing the bit rate required for video data transmission while maintaining video quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024541528000001_ABST
    Figure 2024541528000001_ABST
Patent Text Reader

Abstract

A method and apparatus for video encoding are provided, in which a decoder may receive a picture frame including one or more components including a first component and a second component. The decoder determines a classifier for each sample of the second component according to a co-located sample of the first component, and in response to determining that the classifier is a band classifier or a band classifier including a combined classifier, obtains an indicator indicating one or more bandNum segments of the one or more components, determines one or more bandNum segments according to the indicator and from a mapping table, determines a band offset according to the one or more bandNum segments, and modifies each sample of the second component according to a sample offset including the band offset.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 283,266, entitled “CROSS-COMPONENT SAMPLE ADAPTIVE OFFSET,” filed on November 25, 2021, the contents of which are incorporated herein by reference in their entirety for all purposes. This disclosure relates generally to video encoding and compression, and more specifically to methods and apparatus for improving the efficiency of both luma (luminance, Luma) encoding and chroma (color difference, Chroma) encoding. [Background technology]

[0002] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, video streaming devices, and the like. The electronic devices transmit and receive, or otherwise communicate, digital video data over a communications network and / or store the digital video data in a storage device. Due to the limited bandwidth capacity of a communications network and the limited memory resources of a storage device, the video data may be compressed prior to its communication or storage in accordance with one or more video coding standards through the use of video coding. Video coding standards include, for example, Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), Moving Picture Expert Group (MPEG) coding, and the like. AOMedia Video 1 (AV1) was developed as a successor to the predecessor standard VP9. Audio Video Coding (AVS), referring to the compression standard for digital audio and digital video, is another series of video compression standards. Video coding typically employs prediction techniques (such as inter-prediction or intra-prediction) that exploit the redundancy inherent in the video data, with the goal of compressing the video data into a format that uses a lower bitrate while avoiding or minimizing degradation of the video quality. Summary of the Invention [Problem to be solved by the invention]

[0003] This disclosure describes example implementations of methods and apparatus for encoding and decoding video data, and more specifically, for improving the coding efficiency of both luma and chroma components, including improving coding efficiency by exploring cross-component relationships between the luma (luminance) and chroma (color difference) components. [Means for solving the problem]

[0004] According to a first aspect of the present application, a video decoding method is provided. The method may include a decoder receiving a picture frame including one or more components, where the one or more components may include a first component and a second component. The method may further include determining a classifier for each sample of the second component according to collocated samples of the first component, and in response to determining whether the classifier is a band classifier or a joint classifier including the band classifier, obtaining an indicator indicative of one or more bandNum segments of the one or more components, and determining the one or more bandNum segments according to the indicator. The method may further include the decoder determining a band offset according to the one or more bandNum segments, and modifying each sample of the second component according to a sample offset including the band offset.

[0005] According to a second aspect of the present application, there is provided a video encoding method, the method including an encoder being capable of determining a classifier for each sample of a second component according to a co-located sample of a first component, the picture frame may include one or more components including the first component and the second component, the method may further include the encoder predefining or signaling an indicator in the bitstream in response to determining whether the classifier is a band classifier or a combined classifier including the band classifier, the indicator indicating one or more bandNum segments of the one or more components.

[0006] According to a third aspect of the present application, there is provided a video decoding apparatus, the apparatus may include one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, the one or more processors configured to perform the method according to the first aspect upon execution of the instructions.

[0007] According to a fourth aspect of the present application, there is provided a video encoding apparatus, the apparatus may include one or more processors and a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors, the one or more processors configured to perform the method according to the second aspect upon execution of the instructions.

[0008] According to a fifth aspect of the present invention, there is provided a non-transitory computer readable storage medium storing computer executable instructions which, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to the first aspect based on the bitstream.

[0009] According to a sixth aspect of the present invention there is provided a non-transitory computer readable storage medium which, when executed by one or more computer processors, causes the one or more computer processors to perform a method according to the second aspect to transmit a bitstream.

[0010] It should be understood that both the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present disclosure.

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example system for encoding and decoding video blocks in accordance with some implementations of this disclosure. [Diagram 2] 1 is a block diagram illustrating an example video encoder in accordance with some implementations of the present disclosure. [Diagram 3] 1 is a block diagram illustrating an example video decoder in accordance with some implementations of the present disclosure. [Figure 4A] A block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4B] A block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4C] A block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4D] A block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4E] A block diagram illustrating how a frame is recursively partitioned into multiple video blocks of different sizes and shapes in accordance with some implementations of this disclosure. [Figure 4F] FIG. 2 is a block diagram showing an intra mode defined in VVC. [Figure 4G] FIG. 2 is a block diagram illustrating multiple reference lines for intra prediction. [Figure 5A] FIG. 1 is a block diagram illustrating four gradient patterns used in sample adaptive offset (SAO), according to some implementations of the present disclosure. [Figure 5B] A block diagram showing a decoder for a deblocking filter (DBF) combined with the proposed SAO filtering SAOV and SAOH in accordance with some implementations of the present disclosure. [Figure 6] FIG. 13 is a block diagram illustrating that both the proposed bilateral filter (BIF) and the SAO use samples from the deblocking stage as input, in accordance with some implementations of the present disclosure. [Figure 7] A block diagram showing a naming convention for samples surrounding a central sample, according to some implementations of the present disclosure. [Figure 8A]FIG. 1 is a block diagram illustrating a 5×5 diamond-shaped ALF filter applied to chroma components in accordance with some implementations of the present disclosure. [Figure 8B] FIG. 2 is a block diagram illustrating a 7×7 diamond-shaped ALF filter applied to the luma component in accordance with some implementations of the present disclosure. [Figure 9A] FIG. 1 illustrates a Laplacian calculation by partial samples according to some implementations of the present disclosure. [Figure 9B] FIG. 1 illustrates a Laplacian calculation by partial samples according to some implementations of the present disclosure. [Figure 9C] FIG. 1 illustrates a Laplacian calculation by partial samples according to some implementations of the present disclosure. [Figure 9D] FIG. 1 illustrates a Laplacian calculation by partial samples according to some implementations of the present disclosure. [Figure 10A] FIG. 1 is a block diagram illustrating a system level diagram of a CC-ALF process for SAO, luma ALF, and chroma ALF processes according to some implementations of the present disclosure. [Figure 10B] A diagram showing that filtering in CC-ALF is achieved by applying a linear diamond-shaped filter to the luma channel, according to some implementations of the present disclosure. [Figure 11] FIG. 13 illustrates a modified block classification at a virtual boundary, according to some implementations of the present disclosure. [Figure 12] A diagram illustrating modified ALF filtering for luma components at virtual boundaries in accordance with some implementations of the present disclosure. [Figure 13A] A diagram illustrating a CCSAO applied to chroma samples and using DBF Y as input, in accordance with some implementations of the present disclosure. [Figure 13B] A diagram showing CCSAO applied to luma and chroma samples and using DBF Y / Cb / Cr as input, in accordance with some implementations of the present disclosure. [Figure 13C] FIG. 1 illustrates an independently functioning CCSAO according to some implementations of the present disclosure. [Figure 13D] A diagram illustrating a recursively applied CCSAO in accordance with some implementations of the present disclosure. [Figure 13E] A diagram illustrating the parallel application of SAO and BIF in accordance with some implementations of the present disclosure. [Figure 13F] A diagram illustrating the replacement and parallel application of SAO to BIF in accordance with some implementations of the present disclosure. [Figure 14] A diagram illustrating CCSAO being applied in parallel with other encoding tools in accordance with some implementations of the present disclosure. [Figure 15A] A diagram showing the position of a CCSAO after an SAO in accordance with some implementations of the present disclosure. [Figure 15B] FIG. 1 illustrates a CCSAO functioning independently without a CCALF, according to some implementations of the present disclosure. [Figure 15C] FIG. 13 illustrates a CCSAO acting as a post-reconstruction filter, according to some implementations of the present disclosure. [Figure 16] FIG. 1 illustrates a CCSAO applied in parallel with CCALF according to some implementations of the present disclosure. [Figure 17] A diagram showing the use of another luma sample position instead of C0 classification as another classifier, according to some implementations of the present disclosure. [Figure 18A] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18B] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18C] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18D] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18E]1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18F] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 18G] 1A-1C are diagrams illustrating different candidate shapes for which constraints may be applied in accordance with some implementations of the present disclosure. [Figure 19] A diagram showing that co-located and adjacent chroma samples of other cross components can also be provided to CCSAO classification in addition to luma, according to some implementations of the present disclosure. [Figure 20A] A diagram showing that co-located luma sample values ​​can be replaced with phase correction values ​​by weighting adjacent luma samples, according to some implementations of the present disclosure. [Figure 20B] A diagram showing that co-located luma sample values ​​can be replaced with phase correction values ​​by weighting adjacent luma samples, according to some implementations of the present disclosure. [Figure 21A] A diagram showing that co-located luma sample values ​​can be replaced with phase correction values ​​by weighting adjacent luma samples, according to some implementations of the present disclosure. [Figure 21B] A diagram showing that co-located luma sample values ​​can be replaced with phase correction values ​​by weighting adjacent luma samples, according to some implementations of the present disclosure. [Figure 22A] FIG. 13 illustrates an example of classifying c using edge strength, according to some implementations of the present disclosure. [Figure 22B] FIG. 13 illustrates an example of classifying c using edge strength, according to some implementations of the present disclosure. [Figure 23A] A diagram showing that CCSAO is not applied to the current chroma sample if any of the co-located and adjacent luma samples used for classification are outside the current picture, according to some implementations of the present disclosure. [Figure 23B]A diagram showing that CCSAO is not applied to the current chroma sample if any of the co-located and adjacent luma samples used for classification are outside the current picture, according to some implementations of the present disclosure. [Figure 24A] FIG. 13 illustrates, in accordance with some implementations of the present disclosure, whether to repeat the missing samples or to mirror pad to create samples for classification when any of the co-located and adjacent luma samples used for classification are outside the current picture. [Figure 24B] FIG. 13 illustrates, in accordance with some implementations of the present disclosure, whether to repeat the missing samples or to mirror pad to create samples for classification when any of the co-located and adjacent luma samples used for classification are outside the current picture. [Diagram 25] FIG. 13 illustrates that a 9 luma candidate CCSAO can increase two additional luma line buffers in an AVS according to some implementations of the present disclosure. [Figure 26A] FIG. 13 is a diagram showing that 9 luma candidate CCSAOs can increase one additional luma line buffer in VVC according to some implementations of the present disclosure. [Figure 26B] A diagram showing that a selected chroma candidate may span VB and require an additional chroma line buffer when co-located and adjacent luma samples are used to classify the current luma sample, according to some implementation examples of the present disclosure. [Figure 27A] A diagram showing that in AVS and VVC according to some implementation examples of the present disclosure, if any of the luma candidates of a chroma sample straddles VB (is outside the current chroma sample VB), CCSAO is disabled for the chroma sample. [Figure 27B] A diagram showing that in AVS and VVC according to some implementation examples of the present disclosure, if any of the luma candidates of a chroma sample straddles VB (is outside the current chroma sample VB), CCSAO is disabled for the chroma sample. [Figure 27C]A diagram showing that in AVS and VVC according to some implementation examples of the present disclosure, if any of the luma candidates of a chroma sample straddles VB (is outside the current chroma sample VB), CCSAO is disabled for the chroma sample. [Figure 28A] A diagram showing example hypothetical boundaries for C0 with 9 luma position candidates in accordance with some implementations of the present disclosure. [Figure 28B] A diagram showing example hypothetical boundaries for C0 with 9 luma position candidates in accordance with some implementations of the present disclosure. [Figure 29A] A diagram showing that in AVS and VVC according to some implementation examples of the present disclosure, if any of the luma candidates of a chroma sample straddles the VB (is outside the current chroma sample VB), CCSAO is enabled for the chroma sample using repetition padding. [Figure 29B] A diagram showing that in AVS and VVC, if any of the luma candidates for a chroma sample spans the VB (is outside the current chroma sample VB), CCSAO is enabled using repetition padding for the chroma sample, according to some implementation examples of the present disclosure. [Figure 29C] A diagram showing that in AVS and VVC, if any of the luma candidates for a chroma sample spans the VB (is outside the current chroma sample VB), CCSAO is enabled using repetition padding for the chroma sample, according to some implementation examples of the present disclosure. [Figure 30A] A diagram showing that in AVS and VVC, CCSAO is enabled using mirror padding for a chroma sample when any of the luma candidates for the chroma sample straddles the VB (is outside the current chroma sample VB) in accordance with some implementation examples of the present disclosure. [Figure 30B] A diagram showing that in AVS and VVC, CCSAO is enabled using mirror padding for a chroma sample when any of the luma candidates for the chroma sample straddles the VB (is outside the current chroma sample VB) in accordance with some implementation examples of the present disclosure. [Figure 30C] A diagram showing that in AVS and VVC, CCSAO is enabled using mirror padding for a chroma sample when any of the luma candidates for the chroma sample straddles the VB (is outside the current chroma sample VB) in accordance with some implementation examples of the present disclosure. [Figure 31A] A diagram showing that in AVS and VVC, CCSAO is enabled with double-sided symmetric padding when one side is outside the VB, according to some implementation examples of the present disclosure. [Figure 31B] A diagram showing that in AVS and VVC, CCSAO is enabled with double-sided symmetric padding when one side is outside the VB, according to some implementation examples of the present disclosure. [Figure 32A] A diagram showing that repeat padding or mirror padding can be applied to luma samples outside the virtual boundary, in accordance with some implementations of the present disclosure. [Figure 32B] A diagram showing that repeat padding or mirror padding can be applied to luma samples outside the virtual boundary, in accordance with some implementations of the present disclosure. [Figure 33A] A diagram illustrating restrictions applied to reduce the line buffer required by the CCSAO and simplify boundary processing condition checks in accordance with some implementations of the present disclosure. [Figure 33B] A diagram illustrating restrictions applied to reduce the line buffer required by the CCSAO and simplify boundary processing condition checks in accordance with some implementations of the present disclosure. [Diagram 34] A diagram showing that the CCSAO application area is not aligned to the CTB boundary in accordance with some implementations of the present disclosure. [Diagram 35] A diagram showing that the CCSAO application area frame partition may be fixed in accordance with some implementations of the present disclosure. [Diagram 36] A diagram showing that CCSAO application area partitions are dynamic and can switch at the picture level, according to some implementations of the present disclosure. [Figure 37] FIG. 13 illustrates that, according to some implementation examples of the present disclosure, when multiple classifiers are used in one frame, the method of applying the classifier set index can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block level. [Figure 38] A diagram showing that the CCSAO application area can be BT / QT / TT divided from the frame / slice / CTB level according to some implementation examples of the present disclosure. [Figure 39] A diagram illustrating a CCSAO classifier that takes into account the coding information of the current component or cross component, in accordance with some implementations of the present disclosure. [Figure 40A] FIG. 1 is a block diagram illustrating the SAO classification method presented in this disclosure acting as a post-prediction filter, in accordance with some implementations of the present disclosure. [Figure 40B] FIG. 13 is a block diagram illustrating a post-prediction SAO filter in which each component can use the current sample and neighboring samples for classification, in accordance with some implementations of the present disclosure. [Figure 40C] FIG. 13 is a block diagram illustrating a post-prediction SAO filter in which each component can use the current sample and neighboring samples for classification, in accordance with some implementations of the present disclosure. [Figure 40D] FIG. 13 is a block diagram illustrating a post-prediction SAO filter in which each component can use the current sample and neighboring samples for classification, in accordance with some implementations of the present disclosure. [Diagram 41] FIG. 1 illustrates a computing environment coupled to a user interface in accordance with some implementations of the present disclosure. [Diagram 42] 1 is a flow diagram illustrating a video decoding method according to some implementations of the present disclosure. [Diagram 43] A block diagram illustrating video encoding, in accordance with some implementations of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Reference will now be made in detail to specific implementation examples, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are presented to aid in the understanding of the subject matter presented herein. However, it will be apparent to one skilled in the art that various alternatives can be used without departing from the scope of the claims, and that the subject matter can be practiced without these specific details. For example, it will be apparent to one skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices having digital video capabilities.

[0014] The terms used in this disclosure are only employed for the purpose of describing certain embodiments and are not intended to limit the disclosure. The singular forms "A / an", "said" and "the" in this disclosure and the appended claims are intended to include the plural forms unless otherwise clearly indicated throughout this disclosure. Also, the term "and / or" as used in this disclosure should be understood to refer to and include one or any or all possible combinations of the associated items listed.

[0015] Throughout this specification, references to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language means that a particular feature, structure, or characteristic described is included in at least one embodiment or example. A feature, structure, element, or characteristic described in connection with one or more embodiments may be applicable to other embodiments, unless expressly stated otherwise.

[0016] Throughout this disclosure, terms such as "first," "second," "third," etc. are used solely to refer to associated elements, e.g., devices, components, compositions, steps, etc., and do not imply any spatial or temporal order unless otherwise specified. For example, "first device" and "second device" may refer to two separately formed devices or alternatively two parts, components, or operational states of the same device, which may be arbitrarily named.

[0017] The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," and "sub-unit" may include memory (shared memory, dedicated memory, or collection of memories) that stores code or instructions executable by one or more processors. A module may include one or more circuits that may or may not contain stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached to or located adjacent to each other.

[0018] As used herein, the terms "if" and "when" may be understood to mean "when" or "in response to," depending on the context. These terms, when they appear in the claims, may not indicate that the associated limitation or feature is conditional or optional. For example, a method may include the steps: i) if or if condition X exists, function or operation X' is performed; and ii) if or if condition Y exists, function or operation Y' is performed. The method may be implemented with both the ability to perform function or operation X' and the ability to perform function or operation Y'. Thus, functions X' and Y' may both be performed at different times during multiple executions of the method.

[0019] A unit or module may be implemented solely by software, solely by hardware, or by a combination of hardware and software. In a solely software implementation, for example, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked to each other to perform a specific function.

[0020] The first generation of AVS standards include the Chinese national standards "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). Compared with the MPEG-2 standard, it can save about 50% bitrate with the same perceptual quality. The second generation of AVS standards includes the Chinese national standards "Information Technology, Efficient Multimedia Coding" (known as AVS2) series, which is mainly targeted at the transmission of extra HD television programs. The coding efficiency of AVS2 is twice that of AVS+. Meanwhile, the video part of the AVS2 standard has been submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard for applications. The AVS3 standard is a new generation video coding standard for UHD video applications that aims to surpass the coding efficiency of the latest international standard HEVC, providing approximately 30% bitrate savings over the HEVC standard. In March 2019, at the 68th AVS Conference, the AVS3-P2 baseline was completed, which provides approximately 30% bitrate savings over the HEVC standard. Currently, one reference software, called the High Performance Model (HPM), is maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0021] 1 is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel, according to some implementations of the present disclosure. As shown in FIG. 1, system 10 includes a source device 12, which generates and encodes video data that is subsequently decoded by a destination device 14. Source device 12 and destination device 14 may comprise any of a variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0022] In some implementations, the destination device 14 may receive the encoded video data to be decoded via a link 16. The link 16 may comprise any type of communication medium or equipment capable of moving the encoded video data from the source device 12 to the destination device 14. In one example, the link 16 may include a communication medium that allows the source device 12 to transmit the encoded video data directly to the destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device 14. The communication medium may include a wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or other equipment useful in facilitating communication from the source device 12 to the destination device 14.

[0023] In some other implementations, the encoded video data can be transmitted from the output interface 22 to the storage device 32. The encoded video data in the storage device 32 can then be accessed by the destination device 14 via the input interface 28. The storage device 32 can include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a read-only compact disc memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing the encoded video data. In a further example, the storage device 32 can communicate with a file server or another intermediate storage device capable of holding the encoded video data generated by the source device 12. The destination device 14 can access the video data stored in the storage device 32 by streaming or downloading. The file server can be any type of computer capable of storing the encoded video data and transmitting the encoded video data to the destination device 14. Exemplary file servers include a web server (e.g., for websites), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The destination device 14 may access the encoded video data over any standard data connection suitable for accessing the encoded video data stored on the file server, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a digital subscriber line (DSL), cable modem, etc.), or a combination of both. The transmission of the encoded video data from the storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0024] As shown in Figure 1, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include a video capture device such as a video camera, a video archive containing previously captured video, a video supply interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphic data as the source video or a combination of such sources. As an example, if video source 18 is a video camera of a security surveillance system, source device 12 and destination device 14 may take the form of a camera phone or video phone. However, the implementations described in this application are applicable to video encoding in general and to wireless and / or wired applications.

[0025] The captured, pre-captured, or computer-generated video may be encoded in a video encoder 20. The encoded video data may be transmitted directly to the destination device 14 via the source device output interface 22. The encoded video data may also (or alternatively) be stored on a storage device 32 for later access by the destination device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or a transmitter.

[0026] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or modem and may receive encoded video data over link 16. The encoded video data communicated over link 16 or provided to storage device 32 may include various syntax elements generated by video encoder 20 for use by the video decoder in decoding the video data. Such syntax elements may be included within the encoded video data transmitted over a communications medium and stored on a storage medium or on a file server.

[0027] In some implementations, destination device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0028] Video encoder 20 and video decoder 30 may operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4, Part 10, AVC, AVS, or extensions of such standards. It should be understood that the present application is not limited to a particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. In general, it is contemplated that video encoder 20 of source device 12 may be configured to encode video data according to any current or future standard. Similarly, it is also contemplated that video decoder 30 of destination device 14 may decode video data according to any of these current and future standards.

[0029] Each of the video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. If implemented partially in software, an electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations described in this disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) within the respective device.

[0030] 2 is a block diagram illustrating an example video encoder 20 according to some implementations described in this application. The video encoder 20 can perform intra-predictive and inter-predictive coding of video blocks in video frames. Intra-predictive coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-predictive coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence. It should be noted that the term "frame" may be used as a synonym for "image" or "picture" in the field of video coding.

[0031] As shown in FIG. 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partition unit 45, an intra prediction processing unit 46, and an intra block duplication (BC) unit 48. In some implementations, the video encoder 20 also includes an inverse quantization unit 58 for video block reconstruction, an inverse transform processing unit 60, and an adder 62. An in-loop filter 63, such as a deblocking filter, may be disposed between the adder 62 and the DPB 64 to filter block boundaries and remove blockiness artifacts from the reconstructed video. Another in-loop filter, such as a sample adaptive offset (SAO) filter and / or an adaptive in-loop filter (ALF), may also be used in addition to the deblocking filter to filter the output of the adder 62. In some examples, the in-loop filter may be omitted and the decoded video blocks may be provided directly to DPB 64 by summer 62. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0032] Video data memory 40 may store video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18 shown in FIG. 1. DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use in encoding the video data by video encoder 20 (e.g., in intra- or inter-predictive encoding modes). Video data memory 40 and DPB 64 may be formed by any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components of video encoder 20 or may be off-chip with respect to those components.

[0033] As shown in FIG. 2, after receiving the video data, a partition unit 45 in the prediction processing unit 41 partitions the video data into video blocks. This partitioning may also include partitioning the video frame into slices, tiles (e.g., a set of video blocks), or other larger coding units (CUs) according to a predefined partitioning structure, such as a Quad-Tree (QT) structure, associated with the video data. A video frame is or can be considered as a two-dimensional array or matrix of samples having sample values. The samples in the array may also be referred to as pixels or pels. The number of samples in the horizontal or vertical directions (or axes) of the array or picture defines the size and / or resolution of the video frame. A video frame can be divided into multiple video blocks, for example, using QT partitions. A video block is also or can be considered as a two-dimensional array or matrix of samples having sample values, but has smaller dimensions than a video frame. The number of samples in the horizontal and vertical directions (or axes) of a video block defines the size of the video block. A video block may be further partitioned into one or more block partitions or sub-blocks (which may again form blocks), for example using QT partitions, Binary-Tree (BT) partitions, or Triple-Tree (TT) partitions, or any combination thereof repeatedly. Note that the term "block" or "video block" as used herein refers to a portion of a frame or picture, specifically a rectangular (square or non-square) portion.For example, according to HEVC and VVC, a block or video block may be or correspond to a Coding Tree Unit (CTU), a CU, a Prediction Unit (PU), or a Transform Unit (TU), and / or may be or correspond to a corresponding block, such as a Coding Tree Block (CTB), a Coding Block (CB), a Prediction Block (PB), or a Transform Block (TB), and / or may correspond to a sub-block.

[0034] Prediction processing unit 41 may select one of a number of possible predictive coding modes, such as one of a number of intra-predictive coding modes or one of a number of inter-predictive coding modes, for the current video block based on the error results (e.g., code rate and distortion level). Prediction processing unit 41 may provide the resulting intra- or inter-predictive coded block to summer 50 to generate a residual block, and to summer 62 to reconstruct a coded block for later use as part of a reference frame. Prediction processing unit 41 also provides syntax elements such as motion vectors, intra-mode indicators, partition information, and other such syntax information to entropy coding unit 56.

[0035] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block relative to one or more neighboring blocks in the same frame as the current block being coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-prediction coding of the current video block relative to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple coding passes, e.g., to select an appropriate coding mode for each block of video data.

[0036] In some implementations, motion estimation unit 42 determines the inter prediction mode for the current video frame by generating a motion vector indicating the displacement of a video block in the current video frame relative to a predictive block in a reference video frame according to a predetermined pattern in the sequence of video frames. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, the motion vector may indicate the displacement of a video block in a current video frame or picture relative to a predictive block in a reference frame related to a current block being coded in the current frame. The predetermined pattern may designate a video frame in the sequence as a P frame or a B frame. Intra BC unit 48 may determine vectors, e.g., block vectors, for intra BC coding in a manner similar to how motion estimation unit 42 determines motion vectors for inter prediction, or may utilize motion estimation unit 42 to determine block vectors.

[0037] A prediction block for a video block may be or correspond to a block or reference block of a reference frame that is deemed to closely match the video block being coded in terms of pixel difference, where pixel difference may be determined by Sum of Absolute Difference (SAD), Sum of Square Difference (SSD), or other difference metric. In some implementations, video encoder 20 may calculate values ​​for fractional pixel locations of the reference frame stored in DPB64. For example, video encoder 20 may interpolate values ​​for quarter-pixel locations, eighth-pixel locations, or other fractional pixel locations of the reference frame. Thus, motion estimation unit 42 may perform motion searches for full and fractional pixel locations and output motion vectors with fractional pixel accuracy.

[0038] Motion estimation unit 42 calculates a motion vector for a video block in an inter-predictively coded frame by comparing the position of the video block to the position of a predictive block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy coding unit 56.

[0039] The motion compensation performed by motion compensation unit 44 may include fetching or generating a predictive block based on the motion vector determined by motion estimation unit 42. Upon receiving the motion vector for the current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points in one of the reference frame lists, retrieve the predictive block from DPB 64, and forward the predictive block to summer 50. Summer 50 then subtracts pixel values ​​of the predictive block provided by motion compensation unit 44 from pixel values ​​of the current video block being encoded to form a residual video block of pixel difference values. The pixel difference values ​​forming the residual video block may include either luma (luminance) or chroma (color difference) difference components, or both. Motion compensation unit 44 may also generate associated syntax elements of the video block of the video frame for use by video decoder 30 in decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining the motion vector used to identify the predictive block, any flags indicating a prediction mode, or any other syntax information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are shown separately for conceptual purposes.

[0040] In some implementations, the intra BC unit 48 can generate a vector and fetch a predictive block in a manner similar to that described above in connection with the motion estimation unit 42 and the motion compensation unit 44, but the predictive block is in the same frame as the current block to be coded, and the vector is referred to as a block vector as opposed to a motion vector. Specifically, the intra BC unit 48 can determine an intra prediction mode to use for coding the current block. In some examples, the intra BC unit 48 can code the current block using various intra prediction modes, e.g., during separate coding passes, and test their performance by rate-distortion analysis. The intra BC unit 48 can then select an appropriate intra prediction mode to use from among the various tested intra prediction modes, and generate an intra mode indicator accordingly. For example, the intra BC unit 48 can calculate rate-distortion values ​​using a rate-distortion analysis for the various tested intra prediction modes, and can select the intra prediction mode with the best rate-distortion characteristics among the tested modes as the appropriate intra prediction mode to use. The rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original, yet-to-be-coded block that was coded to produce the coded block, as well as the bitrate (i.e., number of bits) used to generate the coded block. Intra BC unit 48 may calculate ratios from the distortion and rate for various coded blocks to determine which intra prediction mode exhibits the best rate-distortion value for that block.

[0041] In other examples, intra BC unit 48 may use all or a portion of motion estimation unit 42 and motion compensation unit 44 to perform such functions for intra BC prediction according to implementations described herein. In any case, with respect to intra block duplication, the predictive block may be a block that is deemed to closely match the block to be coded in terms of pixel difference, which may be determined by SAD, SSD, or other difference metric, and identification of the predictive block may include calculation of fractional pixel location values.

[0042] Regardless of whether the predictive block is from the same frame via intra prediction or from another frame via inter prediction, video encoder 20 may form a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being encoded to form pixel difference values. The pixel difference values ​​that form the residual video block may include both luma and chroma component differences.

[0043] Intra-prediction processing unit 46 may intra-predict the current video block as an alternative to inter-prediction performed by motion estimation unit 42 and motion compensation unit 44 or intra-block duplication prediction performed by intra BC unit 48, as described above. In particular, intra-prediction processing unit 46 may determine an intra-prediction mode to use for encoding the current block. To that end, intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, e.g., during separate encoding passes, and intra-prediction processing unit 46 (or a mode selection unit in some examples) may select and use an appropriate intra-prediction mode from the tested intra-prediction modes. Intra-prediction processing unit 46 may provide information to entropy coding unit 56 indicating the selected intra-prediction mode for the block. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode into the bitstream.

[0044] After prediction processing unit 41 determines a predictive block for a current video block, via inter- or intra-prediction, summer 50 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block, which may be included in one or more TUs, is provided to transform processing unit 52. Transform processing unit 52 converts the residual video data into residual transform coefficients using a transform, such as a Discrete Cosine Transform (DCT) or a conceptually similar transform.

[0045] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0046] Following quantization, entropy coding unit 56 entropy codes the quantized transform coefficients into a video bitstream, using, for example, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), Syntax-based context-adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding, or another entropy coding method or technique. The coded bitstream may then be transmitted to video decoder 30 as shown in FIG. 1 or stored in storage 32 as shown in FIG. 1 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 may also entropy code motion vectors and other syntax elements for the current video frame being coded.

[0047] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate reference blocks for predicting other video blocks. As mentioned above, motion compensation unit 44 may generate motion compensated prediction blocks from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate fractional pixel values ​​for use in motion estimation.

[0048] Summer 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0049] 3 is a block diagram illustrating an exemplary video decoder 30 according to some implementations of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 can perform a decoding process that is generally the reverse of the encoding process described above for the video encoder 20 in relation to FIG. 2. For example, the motion compensation unit 82 can generate prediction data based on a motion vector received from the entropy decoding unit 80, and the intra prediction unit 84 can generate prediction data based on an intra prediction mode indicator received from the entropy decoding unit 80.

[0050] In some examples, units of the video decoder 30 may be tasked with performing implementations of the present disclosure. Also, in some examples, implementations of the present disclosure may be distributed among one or more units of the video decoder 30. For example, the intra BC unit 85 may perform implementations of the present disclosure alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include an intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0051] Video data memory 79 may store video data, such as an encoded video bitstream, that is decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include a Coded Picture Buffer (CPB) that stores encoded video data from an encoded video bitstream. DPB 92 of video decoder 30 stores reference video data for use by video decoder 30 in decoding video data (e.g., in intra- or inter-prediction coding modes). Video data memory 79 and DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. 3, for purposes of illustration, video data memory 79 and DPB 92 are depicted as two separate components of video decoder 30. However, it will be apparent to one skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or by separate memory devices. In some examples, video data memory 79 may be on-chip with other components of video decoder 30 or off-chip with respect to those components.

[0052] During the decoding process, video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. Video decoder 30 may receive the syntax elements at the video frame level and / or the video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors or intra-prediction mode indicators, and other syntax elements. Entropy decoding unit 80 then forwards the motion vectors or intra-prediction mode indicators, and other syntax elements to prediction processing unit 81.

[0053] When a video frame is coded as an intra-predictive coded (I) frame, or coded for intra-coded predictive blocks in other types of frames, intra-prediction processing unit 84 of prediction processing unit 81 may generate predictive data for video blocks of the current video frame based on the signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0054] When a video frame is coded as an inter-predictive (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 generates one or more prediction blocks for video blocks of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each prediction block may be generated from a reference frame in one of the reference frame lists. Video decoder 30 may construct the reference frame lists, List0 and List1, using a default construction technique based on the reference frames stored in DPB 92.

[0055] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a predictive block for the current video block based on the block vectors and other syntax elements received from entropy decoding unit 80. The predictive block may be within the same reconstruction region of the picture as the current video block defined by video encoder 20.

[0056] Motion compensation unit 82 and / or intra BC unit 85 determine prediction information for video blocks of the current video frame by analyzing the motion vectors and other syntax elements, and then use the prediction information to generate a predictive block for the current video block to be decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine a prediction mode (e.g., intra prediction or inter prediction) to use to encode the video blocks of the video frame, an inter prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, a motion vector for each inter predictive coded video block of the frame, an inter prediction state for each inter predictive coded video block of the frame, and other information for decoding video blocks in the current video frame.

[0057] Similarly, intra BC unit 85 may use some of the received syntax elements, such as flags, to determine that the current video block was predicted using intra BC mode, construction information which video blocks of the frame are within the reconstruction domain and should be stored in DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction states for each intra BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0058] Motion compensation unit 82 may also perform interpolation using an interpolation filter, such as that used by video encoder 20 during encoding of the video block, to calculate interpolated values ​​for fractional pixels of the reference block. In this case, motion compensation unit 82 may determine from the received syntax element the interpolation filter used by video encoder 20 and use the interpolation filter to generate the predictive block.

[0059] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80 using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct residual blocks in the pixel domain.

[0060] After motion compensation unit 82 or intra BC unit 85 generates a predictive block for the current video block based on the vectors and other syntax elements, summer 90 reconstructs a decoded video block for the current video block by adding a residual block from inverse transform processing unit 88 and a corresponding predictive block generated by motion compensation unit 82 and intra BC unit 85. An in-loop filter 91, such as a deblocking filter, an SAO filter, and / or an ALF, may be placed between summer 90 and DPB 92 to further process the decoded video block. The in-loop filter 91 may be applied to the reconstructed CU before the reconstructed CU is stored in a reference picture store. In some examples, the in-loop filter 91 may be omitted and the decoded video block may be provided directly to DPB 92 by summer 90. The decoded video blocks in a given frame are then stored in DPB 92. DPB 92 stores reference frames used for subsequent motion compensation of the next video block. DPB 92, or a memory device separate from DPB 92, may also store the decoded video for later display on a display device, such as display device 34 of FIG.

[0061] In a typical video encoding process, a video sequence usually includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In another example, a frame may be monochromatic and thus include only one two-dimensional array of luma samples.

[0062] Similar to HEVC, the AVS3 standard is built on a block-based hybrid video coding framework. The input video signal is processed by blocks (called coding units (CUs)). Unlike HEVC, which partitions blocks based only on quadtrees, in AVS3, one coding tree unit (CTU) is partitioned into CUs based on quadtrees / binary trees / extended quadtrees to adapt to varying local characteristics. Furthermore, the concept of multiple partition unit types in HEVC is removed. That is, there is no separation of CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is always used as a basic unit for both prediction and transformation without further partitioning. In the tree partition structure of AVS3, first, one CTU is partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary tree and extended quadtree structures.

[0063] As shown in FIG. 4A, the video encoder 20 (or more specifically, the partition unit 45) generates a coded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs ordered consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTUs are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in a video sequence have the same size, which is one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a particular size. As shown in FIG. 4B, each CTU may comprise one CTB of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe the properties of different types of units of coding blocks of pixels and how a video sequence may be reconstructed at video decoder 30, including inter or intra prediction, intra prediction modes, motion vectors, and other parameters. For a monochrome picture or a picture with three separate color planes, a CTU may comprise a single coding tree block and syntax elements used to code samples of this coding tree block. A coding tree block may be an N×N block of samples.

[0064] To achieve better performance, the video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination thereof, on the coding tree blocks of the CTU to partition the CTU into smaller CUs. As shown in FIG. 4C, the 64×64 CTU 400 is first partitioned into four small CUs, each with a block size of 32×32. Among the four small CUs, the CU 410 and the CU 420 are each partitioned into four CUs with a block size of 16×16. The two 16×16 CUs 430, 440 are each further partitioned into four CUs with a block size of 8×8. FIG. 4D shows a quad tree data structure illustrating the final result of the partitioning process of the CTU 400 shown in FIG. 4C, where each leaf node of the quad tree corresponds to one CU, each of which ranges in size from 32×32 to 8×8. Similar to the CTU shown in FIG. 4B, each CU may comprise a CB of luma samples, two corresponding coding blocks of chroma samples of the same size frame, and syntax elements used to code the samples of the coding block. For monochromatic pictures or pictures with three separate color planes, a CU may comprise a single coding block and syntax structures used to code the samples of this coding block. It should be noted that the quadtree partitions shown in FIG. 4C and FIG. 4D are for illustrative purposes only, and one CTU may be divided into multiple CUs to adapt to varying local characteristics based on quadtree / ternary tree / binary tree partitions. In multiple types of tree structures, one CTU may be partitioned by a quadtree structure, and each quadtree leaf CU may be further partitioned by binary tree and ternary tree structures. As shown in FIG. 4E, there are five possible partition types for a coding block with width W and height H: 4 partitions, 2 partitions horizontally, 2 partitions vertically, 3 partitions horizontally, and 3 partitions vertically. There are five possible partition types in AVS3: 4 partitions, 2 partitions horizontal, 2 partitions vertical, horizontally extended quadtree partition, and vertically extended quadtree partition.

[0065] In some implementations, video encoder 20 may further partition the coding blocks of a CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same inter or intra prediction is applied. A PU of a CU may comprise a PB of luma samples and two corresponding PBs of chroma samples and syntax elements used to predict these PBs. For a monochromatic picture or a picture with three separate color planes, a PU may comprise a single PB and syntax structures used to predict this PB. Video encoder 20 may generate luma, Cb, Cr prediction blocks for the luma, Cb, Cr PBs of each PU of a CU.

[0066] Video encoder 20 may generate predictive blocks for a PU using intra prediction or inter prediction. If video encoder 20 generates predictive blocks for a PU using intra prediction, video encoder 20 may generate predictive blocks for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 generates predictive blocks for a PU using inter prediction, video encoder 20 may generate predictive blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0067] After video encoder 20 generates predictive luma, Cb, and Cr blocks for one or more PUs of a CU, video encoder 20 may generate a luma residual block for the CU by subtracting the predictive luma block of the CU from its original luma coding block, where each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, where each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0068] Further, as shown in FIG. 4C, video encoder 20 may use a quadtree partition to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A TU of a CU may comprise a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochromatic picture or a picture with three separate color planes, a TU may comprise a single transform block and syntax structures used to transform samples of the transform block.

[0069] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0070] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients and achieve further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits forming a representation of the encoded frame and associated data. The bitstream may be stored in a storage device 32 or transmitted to a destination device 14.

[0071] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reverse to the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with the TUs of the current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coding blocks of the current CU by adding samples of the predictive blocks for the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. After reconstructing the coding blocks for each CU of the frame, video decoder 30 may reconstruct the frame.

[0072] As mentioned above, video coding mainly uses two modes, intra-frame prediction (or intra prediction) and inter-frame prediction (or inter prediction), to achieve video compression. It is noted that IBC can be considered as intra-frame prediction or a third mode. Of the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it utilizes motion vectors from a reference video block to predict a current video block.

[0073] However, as video data capture technology improves and video block sizes become more refined to preserve details in the video data, the amount of data required to represent the motion vectors of the current frame also increases significantly. One way to overcome this challenge is to take advantage of the fact that neighboring CUs in both spatial and temporal domains not only have similar video data for prediction purposes, but also the motion vectors between these neighboring CUs are similar. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU by exploring their spatial and temporal correlations. This is also called the "Motion Vector Predictor (MVP)" of the current CU.

[0074] Instead of encoding into the video bitstream the actual motion vector for the current CU, as determined by motion estimation unit 42 described above with respect to Figure 2, the motion vector predictor for the current CU is subtracted from the actual motion vector of the current CU to generate a Motion Vector Difference (MVD) for the current CU. In doing so, the motion vector for each CU of a frame, as determined by motion estimation unit 42, does not need to be encoded into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0075] Similar to the process of selecting a prediction block in a reference frame during inter-frame prediction of a coding block, it is necessary for both the video encoder 20 and the video decoder 30 to adopt a set of rules to build a motion vector candidate list (also known as a "merge list") for a current CU using potential motion vector candidates for spatially neighboring CUs and / or temporally co-located CUs of the current CU, and then select one member from the motion vector candidate list as one motion vector for the current CU. In this way, the motion vector candidate list itself does not need to be transmitted from the video encoder 20 to the video decoder 30, and the index of the selected motion vector predictor in the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to use the same motion vector predictor in the motion vector candidate list for encoding and decoding the current CU.

[0076] In general, the basic intra prediction scheme applied in VVC remains almost the same as in HEVC, except that some prediction tools have been further extended, added and / or improved, such as extended intra prediction in wide-angle intra modes, Multiple Reference Line (MRL) intra prediction, Position-Dependent intra Prediction Combination (PDPC), Intra Sub-Partition (ISP) prediction, Cross-Component Linear Model (CCLM) prediction, and Matrix weighted Intra Prediction (MIP), etc.

[0077] Similar to HEVC, VVC predicts samples of the current CU using a set of reference samples adjacent to the current CU (i.e., above the current CU or to the left of the current CU). However, to capture finer edge orientations present in natural video (especially high-resolution video content such as 4K), the number of angular intra modes is extended from 33 in HEVC to 93 in VVC. FIG. 4F is a block diagram showing the intra modes defined in VVC. As shown in FIG. 4F, among the 93 angular intra modes, modes 2 to 66 are conventional angular intra modes, and modes -1 to -14 and modes 67 to 80 are wide-angle intra modes. In addition to the angular intra modes, the planar mode (mode 0 in FIG. 1) and direct current (DC) mode (mode 1 in FIG. 1) of HEVC are also applied to VVC.

[0078] As shown in FIG. 4E, since a quadtree / binary / ternary tree partition structure is applied in VVC, besides square video blocks, rectangular video blocks also exist in intra prediction in VVC. Due to the uneven width and height of one given video block, various sets of angular intra modes can be selected from 93 angular intra modes for different block shapes. More specifically, for both square and rectangular video blocks, besides planar and DC modes, 65 angular intra modes out of 93 angular intra modes are supported for each block shape. If the rectangular block shape of a video block satisfies a certain condition, the index of the wide-angle intra mode of the video block can be adaptively determined by video decoder 30 according to the index of the conventional angular intra mode received from video encoder 20 using a mapping relationship as shown in Table 1-0 below. That is, for a non-square block, the wide-angle intra mode is signaled by video encoder 20 using the index of the conventional angular intra mode. This is parsed and then mapped to wide-angle intra-mode indices by video decoder 30, so that the total number of intra-modes (i.e., 67) (i.e., planar mode, DC mode, and 65 angular intra-modes out of 93 angular intra-modes) remains unchanged, and the intra-prediction mode coding scheme remains unchanged. As a result, efficient signaling of intra-prediction modes is achieved while still providing a consistent design across different block sizes.

[0079] Table 1-0 shows the mapping relationship between the indexes of conventional angle intra modes and the indexes of wide angle intra modes for intra prediction of different block shapes in VCC, where W represents the width of the video block and H represents the height of the video block. [Table 1]

[0080] Similar to HEVC intra prediction, all intra modes (i.e., planar, DC, and angular intra modes) in VVC utilize a set of reference samples above and to the left of the current video block for intra prediction. However, unlike HEVC where only the nearest row / column of reference samples (i.e., the 0th line 201 in FIG. 4G) is used, MRL intra prediction is introduced in VVC. In addition to the nearest row / column of reference samples, two additional rows / columns of reference samples (i.e., the 1st line 203 and the 3rd line 205 in FIG. 4G) are available for intra prediction. The index of the selected row / column of reference samples is signaled from video encoder 20 to video decoder 30. When a non-nearest row / column of reference samples (i.e., the 1st line 203 and the 3rd line 205 in FIG. 4G) is selected, planar modes are excluded from the set of intra modes that may be used to predict the current video block. MRL intra prediction is disabled for the first column / row of a video block within the current CTU to prevent the use of extended reference samples outside the current CTU.

[0081] Sample Adaptive Offset (SAO)

[0082] Sample Adaptive Offset (SAO) is a process that modifies decoded samples after application of a deblocking filter by conditionally adding an offset value to each sample based on the values ​​of a look-up table transmitted by the encoder. SAO filtering is performed on a per-region basis based on the filtering type selected per CTB by the syntax element sao-type-idx. A value of 0 for sao-type-idx indicates that no SAO filter is applied to the CTB, while values ​​1 and 2 signal the use of band-offset and edge-offset filtering types, respectively. In band-offset mode, specified by sao-type-idx being 1, the offset value selected depends directly on the sample amplitude. In this mode, the full amplitude range of the samples is uniformly divided into 32 segments called bands, and sample values ​​belonging to four of these bands (that are consecutive within the 32 bands) are modified by adding a transmitted value, denoted as band-offset. This value can be positive or negative. The main reason for using four consecutive bands is that in smooth regions where banding artifacts may appear, the sample amplitudes in the CTB tend to be concentrated in only a few of these bands. Furthermore, the design choice of using four offsets is unified with the edge offset operation mode, which also uses four offset values. In the edge offset mode, specified by sao-type-idx being 2, the syntax element sao-eo-class, with a value between 0 and 3, signals whether the horizontal, vertical, or one of the two diagonal gradient directions is used for edge offset classification in the CTB.

[0083] FIG. 5A is a block diagram illustrating four gradient patterns used in SAO according to some implementations of the present disclosure. The four gradient patterns 502, 504, 506, 508 are for each sao-eo-class in edge offset mode. The sample labeled "p" indicates the center sample to be considered. The two samples labeled "n0" and "n1" designate two adjacent samples along the gradient patterns: (a) horizontal (sao-eo-class=0), (b) vertical (sao-eo-class=1), (c) 135° diagonal (sao-eo-class=2), and (d) 45° (sao-eo-class=3). Each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p located at a certain position with the values ​​n0 and n1 of two samples located at adjacent positions, as shown in FIG. 5A. This classification is performed for each sample based on the decoded sample value, so no additional signaling is required for EdgeIdx classification. Depending on the EdgeIdx category at the sample location, an offset value from the transmitted lookup table is added to the sample value for EdgeIdx categories 1 through 4. The offset value is always positive for categories 1 and 2, and always negative for categories 3 and 4. Thus, the filter generally has a smoothing effect in edge offset mode. Table 1-1 below shows sample EdgeIdx categories for SAO edge classes. [Table 2]

[0084] For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder for each CTB. For type 1, the sign is also coded. The offset values ​​and related syntax elements such as sao-type-idx and sao-eo-class are determined by the encoder, typically using a criterion that optimizes rate-distortion performance. SAO parameters can be indicated as inherited from the left or top CTB using a merge flag for efficient signaling. In summary, SAO is a nonlinear filtering operation that allows further refinement of the reconstructed signal, enhancing the signal representation both in smooth areas and around edges.

[0085] Pre-Sample Adaptive Offset (Pre-SAO)

[0086] In some examples, a pre-sample adaptive offset (Pre-SAO) is implemented. The low-complexity coding performance of Pre-SAO is promising for the development of future video coding standards. In some examples, Pre-SAO is applied only to luma component samples, using luma samples for classification. Pre-SAO works by applying two SAO-like filtering operations, called SAOV and SAOH, which are applied together with a deblocking filter (DBF) before applying the existing (legacy) SAO. The first SAO-like filter, SAOV, operates to apply SAO to the input picture Y2 after the deblocking filter for vertical edges (DBFV) has been applied.

number

[0087] where T is a given positive constant, and d1 and d2 are f(i) = Y1(i) - Y2(i) where Y1(i) is the offset coefficient associated with the two classes based on the sample difference between Y1(i) and Y2(i) given by

[0088] The first class for d1 is given by making f(i)>T for all sample positions i, and the second class for d2 is given by f(i)<-T. The offset coefficients d1 and d2 are calculated in the encoder to minimize the mean square error between the output picture Y3 of SAOV and the original picture X, similar to conventional SAO processing. After SAOV is applied, a second SAO-like filter SAOH operates to apply SAO to Y4 after SAOV application, where the classification is based on the sample difference between Y3(i) and Y4(i), which are the output pictures of the deblocking filter for horizontal edges (DBFH), as shown in FIG. 5B. A procedure similar to SAOV is applied for SAOH, using Y3(i)-Y4(i) instead of Y1(i)-Y2(i) for classification. The two offset coefficients, a predefined threshold T, and an enable flag for each of SAOH and SAOV are signaled at the slice level. SAOH and SAOV are applied independently to the luma and the two chroma components.

[0089] In some cases, SAOH and SAOV operate only on picture samples affected by the respective deblocking (DBFV or DBFH). Thus, unlike existing SAO processes, only a subset of all samples in a given spatial domain (picture, or CTU in case of legacy SAO) is processed in Pre-SAO, so the resulting increase in the average amount of decoder-side computation per picture sample is kept low (preliminary estimates suggest 2-3 comparisons and 2 additions per sample in the worst case scenario). Pre-SAO requires only the samples used in the deblocking filter, and no additional samples need to be stored in the decoder.

[0090] Bilateral Filter (BIF)

[0091] In some embodiments, a bilateral filter (BIF) is implemented to explore compression efficiency beyond VVC. The BIF is performed in a loop filter stage of sample adaptive offset (SAO). Both the bilateral filter (BIF) and SAO use samples from deblocking as input. Each filter generates an offset for each sample, which is added to the input sample and clipped before going to ALF.

[0092] For details, see the output sample I OUT is obtained as follows: I OUT =clip3(I C +ΔI BIF +ΔI SAO ) Here, I C is the input sample from deblocking, and ΔI BIF is the offset from the bilateral filter, and ΔI SAO is the offset from SAO.

[0093] In some embodiments, this implementation provides the possibility for the encoder to enable or disable filtering at the CTU and slice level. The encoder evaluates the rate-distortion optimization (RDO) cost to make the decision.

[0094] In the PPS of Table 1-2 showing the picture parameter set RBSP syntax, the following syntax elements are introduced: [Table 3]

[0095] When pps_bilateral_filter_enabled_flag is 0, the bilateral loop filter is disabled for slices that reference a PPS. When pps_bilateral_filter_flag is 1, the bilateral loop filter is enabled for slices that reference a PPS.

[0096] bilateral_filter_strength specifies the strength value of the bilateral loop filter used in the bilateral transform block filtering process. The value of bilateral_filter_strength must be in the range from 0 to 2, inclusive.

[0097] bilateral_filter_qp_offset specifies the offset used in deriving the bilateral filter lookup table LUT(x) for the slice that references the PPS. The value of bilateral_filter_qp_offset must be in the range of -12 to +12, inclusive.

[0098] The following syntax elements are introduced in Table 1-3, which shows the slice header syntax, and in Table 1-4, which shows the coding tree unit syntax. [Table 4]

[0099] The meaning is as follows: slice_bilateral_filter_all_ctb_enabled_flag, when equal to 1, indicates that the bilateral filter is enabled and applied to all CTBs in the current slice. If slice_bilateral_filter_all_ctb_enabled_flag is not present, it is inferred to be equal to 0.

[0100] slice_bilateral_filter_enabled_flag equal to 1 indicates that the bilateral filter is enabled and can be applied to the CTB of the current slice. If slice_bilateral_filter_enabled_flag is not present, it is inferred to be equal to slice_bilateral_filter_all_ctb_enabled_flag.

[0101] When bilateral_filter_ctb_flag[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is equal to 1, it specifies that the bilateral filter is applied to the luma coding tree block of the coding tree unit at luma position (xCtb, yCtb). When bilateral_filter_ctb_flag[cIdx][xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is equal to 0, it specifies that the bilateral filter is not applied to the luma coding tree block of the coding tree unit at luma position (xCtb, yCtb). If bilateral_filter_ctb_flag is not present, it is inferred to be equal to (slice_bilateral_filter_all_ctb_enabled_flag & slice_bilateral_filter_enabled_flag).

[0102] In some examples, the filtering process for a filtered CTU proceeds as follows: At picture boundaries where samples are unavailable, the bilateral filter uses an expansion function (sample repetition) to fill in the unavailable samples. For virtual boundaries, the operation is the same as for SAO and no filtering occurs. When crossing horizontal CTU boundaries, the bilateral filter has access to the same samples that the SAO has access to. Figure 7 is a block diagram showing the naming convention for samples surrounding a central sample, according to some implementation examples of the present disclosure. As an example, the central sample I C If is on the top line of the CTU, then I NW , I A , I NE As with SAO, it is read from the CTU above, but I AA is padded so no extra line buffer is needed. C The samples surrounding are labeled according to Figure 7, where A, B, L, R stand for up, down, left, right, NW, NE, SW, SE for north-west, etc. Similarly, AA stands for up-up, BB for down-down, etc. This diamond shape isAA , I BB , I LL , or I RR This is different from the alternative method which does not use square filter support.

[0103] Each surrounding sample I A , I R etc. are the corresponding modifier values ​​μ ΔIA , μΔ IR These are calculated as follows. R Starting with the contribution from , the difference is calculated as follows: ΔI R =(|I R -I C |+4)>>3 where || denotes the absolute value. For non-10-bit data, ΔI R =(|I R -I C |+2 n-6 )>>(n-7) is used instead. For 8-bit data, n=8, etc. The resulting value is then clipped to be less than 16. sI R =min(15,ΔI R )

[0104] The modifier value is then calculated as follows:

number

[0105] These values ​​can be stored using 6 bits per entry, resulting in 26*16*6 / 8=312 bytes, or 300 bytes excluding the first column which is all zeros. ΔIL , μ ΔIA , and μ ΔIB The modifier value for I L , I A , and I B The diagonal sample I is calculated in the same way. NW , I NE , I SE , I SW , and I, two steps away AA , I BB , I RR , I LL For, the calculation follows Equation 2 and Equation 3, but with values ​​shifted by 1. For example, the diagonal sample I SE Using

number

[0106] The modifier values ​​are added together. m sum = μ ΔIA +μ ΔIB +μ ΔIL +μ ΔIR +μ ΔINW +μ ΔINE +μ ΔISW +μ ΔISE +μ ΔIAA +μ ΔIBB +μ ΔILL +μ ΔIRR

[0107] In some instances, μ ΔIR Ga-μ ΔIA Similarly, for the above sample, μ ΔIA Ga-μ ΔIB and similar symmetry is seen for the diagonal modifier values ​​and for modifier values ​​two rows apart. This is because in a hardware implementation, six values ​​μ ΔIR , μ ΔIB , μΔI SW , μ ΔISE , μ ΔIRR , and μ ΔIBB This means that it is sufficient to calculate ,the remaining six values ​​can be obtained from the previous calculations.

[0108] where the value m sum is multiplied by c=1, 2, or 3. This can be done using a single adder and a logical AND gate as follows: c v =k1&(m sum <<1)+k2&m sum Here, & denotes logical and, k1 is the most significant bit of the multiplier c, and k2 is the least significant bit. The value to be multiplied is obtained using the minimum block dimension D=min(width,height) in Table 1-5, which shows how to obtain the c parameter from the minimum size of a block D=min(width,height). [Table 5]

[0109] Finally, the bilateral filter offset ΔI BIF is calculated. For full strength filtering, the following formula is used: ΔI BIF =(c v +16)>>5 On the other hand, for half-strength filtering, the following equation is used: ΔI BIF =(c v +32)>>6

[0110] The general form for n-bit data is

number

[0111] Adaptive Loop Filter (ALF)

[0112] In VVC, an adaptive loop filter (ALF) is applied to the block-based filters: for the luma component, one of 25 filters is selected for each 4x4 block based on local gradient orientation and activity.

[0113] Two diamond filter shapes (as shown in Figures 8A-8B) are used: a 7x7 diamond shape is applied to the luma component and a 5x5 diamond shape is applied to the chroma components.

[0114] For the luma component, each 4x4 block is classified into one of 25 classes. The classification index C is determined by its directionality D and the quantized value of the activity. JPEG2024541528000011.jpg43 Based on this, it can be deduced that:

number

[0115] JPEG2024541528000013.jpg9162

number

[0116] To reduce the complexity of block classification, a subsampled 1D Laplacian computation is applied. The same subsampled positions are used for gradient computation in all directions, as shown in Figures 9A-9D.

[0117] Next, the maximum and minimum values ​​of the horizontal and vertical gradient D are set as follows:

number

number

[0118] These values ​​are compared with each other and with two thresholds t1, t2 to obtain the value of the directionality D: JPEG2024541528000017.jpg28160 The activity value A is calculated as follows:

number

[0119] For the chroma components of the picture, no classification method is applied.

[0120] Geometric transformation of filter coefficients and clipping values

[0121] Before filtering each 4x4 luma block, we apply a geometric transformation such as rotation, diagonal or vertical flip to the filter coefficients f(k,l) and the corresponding filter clipping values ​​c(k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the area supported by the filter. The idea is to make different blocks that we apply ALF to more similar by aligning their orientation.

[0122] Three geometric transformations are introduced, including diagonal, vertical flip and rotation: JPEG2024541528000020.jpg15112

[0123] where K is the size of the filter, 0≦k,l≦K-1 are the coefficient coordinates, with position (0,0) being the top-left corner and position (K-1,K-1) being the bottom-right corner. The transformation is applied to the filter coefficients f(k,l) and clipping values ​​c(k,l) according to the gradient value calculated for that block. The relationship between the transformation and the four gradients in the four directions is summarized in Table 1-6 below, which shows the mapping between the gradients calculated for one block and the transformation. [Table 6]

[0124] Filtering Process

[0125] On the decoder side, when ALF is enabled for the CTB, each sample R(i,j) in the CU is filtered to obtain the sample value R'(i,j) as shown below:

number

[0126] Cross component adaptive loop filter (CC-ALF)

[0127] CC-ALF uses the luma sample values ​​to refine each chroma component by applying an adaptive linear filter to the luma channel and then using the output of this filtering operation for chroma refinement. Figure 10A provides a system level diagram of the CC-ALF process for SAO, luma ALF and chroma ALF processes.

[0128] Filtering in CC-ALF is achieved by applying a linear diamond filter (Figure 10B) to the luma channel. One filter is used for each chroma channel, and the operation can be expressed as:

number

[0129] As shown in FIG. 10B, the luma filter support is the area collocated with the current chroma sample after taking into account the spatial scaling factors between the luma and chroma planes.

[0130] In the VVC reference software, the CC-ALF filter coefficients are calculated by minimizing the mean squared error of each chroma channel relative to the original chroma content. To achieve this, the VTM algorithm uses a coefficient derivation process similar to that used for chroma ALF. Specifically, a correlation matrix is ​​derived and coefficients are calculated using a Cholesky decomposition solver to minimize the mean squared error metric. In designing the filters, up to eight CC-ALF filters can be designed and transmitted per picture. The resulting filters are then ordered in CTUs for each of the two chroma channels.

[0131] Further features of CC-ALF include: The design uses a 3x4 diamond shape with 8 taps. Seven filter coefficients are transmitted in the APS. The transmitted coefficients each have a dynamic range of 6 bits and are restricted to values ​​that are powers of 2. The eighth filter coefficient is calculated by the decoder so that the sum of the filter coefficients is zero. · APS can be referenced in the slice header. ·CC-ALF filter selection is controlled at the CTU level for each chroma component. Border padding for horizontal virtual borders uses the same memory access pattern as Luma ALF.

[0132] As an additional feature, the reference encoder can be configured via a configuration file to allow some basic subjective tuning. When enabled, the VTM will attenuate the application of CC-ALF in regions that are coded with a high QP and are close to mid-gray or contain a large amount of high luma frequencies. Algorithmically, this is achieved by disabling the application of CC-ALF in CTUs where any of the following conditions are true: · The slice QP value minus 1 is less than or equal to the base QP value; · The number of chroma samples whose local contrast is greater than (1<<(bitDepth-2))-1 exceeds the CTU height, where local contrast is the difference between the maximum and minimum luma sample values ​​within the filter support region; More than a quarter of the chroma samples are between (1<<(bitDepth-1))-16 and (1<<(bitDepth-1))+16.

[0133] The aim of this feature is to provide some guarantee that CC-ALF does not amplify artifacts introduced early in the decoding path (due to the fact that VTM does not currently explicitly optimize for chroma subjective quality). Alternative encoder implementations are expected to either not use this feature or incorporate alternative strategies suited to their encoding characteristics.

[0134] Filter Parameter Signaling

[0135] The ALF filter parameters are signaled in an adaptive parameter set (APS). One APS can signal up to 25 pairs of luma filter coefficients and clipping value indexes, and up to 8 pairs of chroma filter coefficients and clipping value indexes. To reduce bit overhead, filter coefficients of different classifications for the luma component can be merged. In the slice header, the APS index used for the current slice is signaled.

[0136] The clipping value index decoded from the APS allows to determine the clipping values ​​using a table of clipping values ​​for both luma and chroma components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are given by the following formula:

number

[0137] In the slice header, up to seven APS indices can be signaled to specify the luma filter set to be used for the current slice. The filtering process is further controllable at the CTB level. A flag is always signaled to indicate whether an ALF is applied to the luma CTB. The luma CTB can select a filter set among 16 fixed filter sets and a filter set from the APS. A filter set index is signaled to the luma CTB to indicate which filter set is applied. The 16 fixed filter sets are predefined and hardcoded in both the encoder and the decoder.

[0138] For chroma components, an APS index is signaled in the slice header indicating the chroma filter set used for the current slice. At the CTB level, if an APS has more than one chroma filter set, a filter index is signaled for each chroma CTB.

[0139] The filter coefficients are quantized with norm 128. To limit the multiplication complexity, coefficient values ​​at non-central locations are quantized by -2 7 ~2 7 Bitstream adaptation is applied such that the range is between -1 and -2 (inclusive). The coefficients in the center positions are not signaled in the bitstream and are assumed to be equal to 128.

[0140] Virtual border filtering for line buffer reduction

[0141] In VVC, to reduce the line buffer requirements of ALF, a modified block classification and filtering is adopted for samples near the horizontal CTU boundary. For this purpose, as shown in Fig. 11, a virtual boundary is defined as a line shifted by “N” samples from the horizontal CTU boundary, where N is equal to 4 for the luma component and 2 for the chroma component.

[0142] A modified block classification is applied to the luma component as shown in Figure 11. For the 1D Laplacian gradient computation of 4x4 blocks above the virtual boundary, only samples above the virtual boundary are used. Similarly, for the 1D Laplacian gradient computation of 4x4 blocks below the virtual boundary, only samples below the virtual boundary are used. The quantization of the activity value A is scaled accordingly to account for the reduced number of samples used for the 1D Laplacian gradient computation.

[0143] The filtering process uses a symmetric padding operation on the virtual boundary for both luma and chroma components. As shown in Figure 12, if the sample being filtered is located below the virtual boundary, the adjacent samples located above the virtual boundary are padded, while the corresponding samples on the opposite side are also padded symmetrically.

[0144] Unlike the symmetric padding method used for horizontal CTU boundaries, a simple padding process is applied to slice, tile and subpicture boundaries when boundary-spanning filters are disabled. A simple padding process is also applied to picture boundaries. The padded samples are used for both classification and filtering processes. To compensate for the extreme padding when filtering samples just above or below a virtual boundary, we reduce the filtering strength in these cases, for both luma and chroma, by increasing the right shift by 3 in the formula to obtain the sample value R'(i,j).

[0145] For the existing SAO design of HEVC, VVC, AVS2 and AVS3 standards, the luma Y, chroma Cb and chroma Cr sample offset values ​​are determined independently. That is, for example, the offset of a current chroma sample is determined only by the value of the current chroma sample and the value of the adjacent chroma sample, and the co-located luma sample and the adjacent luma sample are not considered. However, the luma sample holds more detailed information of the original picture than the chroma sample, which may help to determine the offset of the current chroma sample. Furthermore, since the chroma sample usually loses high frequency details after the color conversion from RGB to YCbCr or after the quantization and deblocking filter, the introduction of the luma sample that holds high frequency details for the chroma offset determination is helpful for the chroma sample reconstruction. Therefore, further gains can be expected by exploring the correlation of the cross components, for example, by using the method and system of Cross Component Sample Adaptive Offset (CCSAO). In some embodiments, the correlation here includes not only cross-component sample values, but also picture / coding information such as prediction / residual coding mode, transform type, and quantization / deblocking / SAO / ALF parameters from the cross-component.

[0146] In another example, for SAO, the luma sample offset is determined only by the luma sample. However, for example, luma samples with the same band offset (BO) classification can be further classified by their co-position and neighboring chroma samples, which may result in more efficient classification. SAO classification can be seen as a shortcut to compensate for sample differences between the original picture and the reconstructed picture. Therefore, an effective classification is desired.

[0147] Cross-component Sample Adaptive Offset (CCSAO)

[0148] The existing SAO design in HEVC, VVC, AVS2, and AVS3 standards is used as the basic SAO method in the following description. For those skilled in the video coding field, the proposed cross-component method described in this disclosure can be applied to other loop filter designs and other coding tools with similar design principles. For example, in the AVS3 standard, SAO is replaced by a coding tool called Enhanced Sample Adaptive Offset (ESAO), but the proposed CCSAO can also be applied in parallel with ESAO. Another example where CCSAO can be applied in parallel is the Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0149] 13A-13F are diagrams of the proposed method. In FIG. 13A, the luma sample after the luma deblocking filter (DBF Y) is used to determine the additional offsets of chroma Cb and Cr after SAO Cb and SAO Cr. For example, the current chroma sample (1302) is first classified using the co-located luma sample (1304) and the neighboring luma sample (1306), and the CCSAO offset of the corresponding class is added to the current chroma sample. In FIG. 13B, the CCSAO is applied to the luma sample and the chroma sample, using the DBF Y / Cb / Cr as input. In FIG. 13C, the CCSAO can operate independently. In FIG. 13D, the CCSAO can be applied recursively (2 times or N times) with the same or different offsets in the same codec stage, or repeatedly in different stages. In FIG. 13E, the CCSAO is applied in parallel with the SAO and BIF. In Figure 13F, CCSAO replaces SAO and is applied in parallel with BIF.

[0150] Therefore, the classification of the current luma sample may use the current and adjacent luma samples, the co-located and adjacent chroma samples (Cb and Cr), and further, the classification of the current chroma sample (Cb or Cr) may use the information of the co-located and adjacent luma samples, the co-located and adjacent cross chroma samples, and the current and adjacent chroma samples.

[0151] FIG. 14 shows that CCSAO may be applied in parallel with other coding tools, such as ESAO in AVS standard and CDE in AV1 standard. FIG. 15A shows that the position of CCSAO may be after SAO, i.e., the position of CCALF in VVC standard. In FIG. 15B, CCSAO can operate independently without CCALF. In FIG. 15C, CCSAO can act as a post-reconstruction filter, i.e., it uses reconstructed samples as input to classification and corrects luma / chroma samples before entering adjacent intra prediction. FIG. 16 shows that CCSAO may also be applied in parallel with CCALF. In FIG. 16, the positions of CCALF and CCSAO can be swapped. Please note that in FIG. 13A-FIG. 16, or in other paragraphs of this disclosure, SAO Y / Cb / Cr block can be replaced by ESAO Y / Cb / Cr (in AVS3) or CDEF (in AV1). Please note that Y / Cb / Cr may be denoted as Y / U / V in the video coding field.

[0152] In some examples, if the video is in RGB format, the proposed CCSAO can also be applied by simply mapping the YUV representation in the following paragraphs to GBR.

[0153] It should be noted that the figures of this disclosure are combinable with all examples mentioned in this disclosure.

[0154] classification

[0155] 13A-13F and 19 show the input of CCSAO classification. 13A-13F and 19 also show that all co-located and adjacent luma / chroma samples can be fed to CCSAO classification. Note that the classifiers proposed in this disclosure can also serve the original SAO classification, so that the classifiers described in this disclosure can not only serve cross-component classification (e.g., classifying chroma using luma or vice versa), but also single-component classification (e.g., classifying luma using luma or classifying chroma using chroma).

[0156] An example classifier (C0) uses the value of the co-located luma or chroma sample (Y0 in FIG. 13A) (Y4 / U4 / V4 in FIG. 13B to FIG. 13C) for classification. If band_num is the number of bands into which the luma or chroma dynamic range is evenly divided, and bit_depth is the bit depth of the sequence, an example of the class index of the current chroma sample is: Class(C0)=(Y0*band_num)>>bit_depth It becomes.

[0157] Some examples of band_num and bit_depth are shown in Table 2-2 below. Table 2-2 shows three classification examples where the number of bands is different in each classification example. The classification can take rounding into account. Class (C0)=((Y0 * band_num)+(1<<bit_depth))> >bit_depth Some examples of band_num and bit_depth are listed in Table 2-1 below. [Table 7]

[0158] In some examples, the classifier uses a different luma (or chroma) sample location for C0 classification. For example, use neighboring Y7 instead of Y0 for C0 classification as shown in Figure 17. It is possible to switch between different classifiers at SPS / APS / PPS / PH / SH / Region / CTU / CU / Sub-block / Sample level. For example, in Figure 17, use Y0 for POC0 but Y7 for POC1 as shown in Table 2-2 below. [Table 8]

[0159] Figures 18A-18G show some examples of luma candidates with different shapes. As shown in Figures 18B-18D, a constraint can be applied to the shapes that the total number of candidates must be a power of two. As shown in Figures 18A, 18C-18E, a constraint can be applied to the shapes that the number of luma candidates must be horizontally and vertically symmetric with the chroma samples. The power of two constraint and symmetry constraint can also be applied to the chroma candidates. In Figures 13B-13C, the U / V section shows an example of a symmetry constraint.

[0160] In some examples, different color formats can have different classifier "constraints." For example, 420 uses the luma / chroma candidate selection of Figures 13B-13C (select one candidate from a 3x3 shape), while 444 uses Figure 18F for luma and chroma candidate selection, and 422 uses Figure 18G for luma (2 chroma samples share 4 luma candidates) and Figure 18F for chroma candidates.

[0161] C0 position and C0band_num can be combined and switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different combinations can result in different classifiers as shown in Table 2-3 below. [Table 9]

[0162] In some examples, the co-located luma sample value (Y0) is replaced with a value (Yp) by weighting the co-located and neighboring luma samples. Figures 20A-20B show two examples. Different Yp may have different classifiers. Different Yp may be applied to different chroma formats. For example, Yp in Figure 20A is used for 420, Yp in Figure 20B is used for 422, and Y0 is used for 444.

[0163] In some examples, another example classifier (C1) compares scores of the co-located luma sample (Y0) with eight adjacent luma samples [-8, 8], resulting in a total of 17 classes. Initial class (C1) = 0, loop over 8 adjacent luma samples (Yi,i = 1 to 8) if Y0>Yi Class+=1 else if Y0 <Yi Class-=1

[0164] In some examples, an example of C1 is equal to the following function with a threshold th of 0: ClassIdx=Index2ClassTable(f(C,P1)+f(C,P2)+…+f(C,P8)) f(x,y)=1, if xy>th; f(x,y)=0, if xy=th; f(x,y)=-1, if xy <th

[0165] In some examples, similar to the C4 classifier, one or more thresholds can be predefined (e.g., stored in a LUT) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level to aid in classification (quantization) of the differences.

[0166] In some examples, the variant (C1') only counts the comparison scores [0,8], which results in 8 classes. (C1,C1') is a classifier group, and the PH / SH level flags can be signaled to switch between C1 and C1'. Initial class (C1') = 0, loop over 8 adjacent luma samples (Yi,i = 1 to 8) if Y0>Yi Class+=1

[0167] In some examples, the variant (C1s) selectively uses the neighboring N out of M neighboring samples to count the comparison score. An M-bit bitmask can be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock level to indicate which neighboring samples are selected to count the comparison score. Using FIG. 13B as an example of a luma classifier: 8 neighboring luma samples are candidates, and an 8-bit bitmask (01111110) is signaled at PH, which indicates that 6 samples Y1 to Y6 are selected, and the comparison score is within [-6,6], resulting in an offset of 13. The selective classifier C1s gives the encoder more options for the tradeoff between offset signaling overhead and classification granularity.

[0168] In some examples, similar to C1s, the variant (C1's) counts only comparison scores [0,+N], giving that in the previous example of bitmask 01111110 the comparison score is within [0,6], resulting in an offset of 7.

[0169] Different classifiers can be combined to obtain a general-purpose classifier, for example for different pictures (different POC values) different classifiers are applied as shown in Table 2-4 below. [Table 10]

[0170] In some examples, another classifier example (C2) uses the difference between the co-located luma sample and the adjacent luma sample (Yn). Figures 21A-21B show an example of Yn, whose dynamic range is [-1024, 1023] when the bit depth is 10. Let C2 band_num be the number of bands into which the Yn dynamic range is equally divided. Class(C2)=(Yn+(1<<bit_depth)*band_num)> >(bit_depth+1) We can combine C0 and C2 to get a general-purpose classifier, for example, as shown in Table 2-5 below. [Table 11]

[0171] In some examples, another classifier example (C3) uses a bit mask for classification as shown in Table 2-6. Table 2-6 shows a classifier example that uses a bit mask for classification (bit mask position is underlined). A 10-bit bit mask is signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level to indicate the classifier. For example, a bit mask of 11 1100 0000 means that for a given 10-bit luma sample value, only the most significant four bits are used for classification, which gives a total of 16 classes. Another example bit mask of 10 0100 0001 means that only three bits are used for classification, which gives a total of 8 classes. The bit mask length (N) is fixed or switchable at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, for a 10-bit sequence, in a 4-bit bitmask 1110 signaled in the PH in the picture, the three MSBs b9, b8, b7 are used for classification. Another example is a 4-bit bitmask 0011, with the LSBs b0, b1 being used for classification. The bitmask classifier can be applied to a luma classifier or a chroma classifier. The use of MSB or LSB for the bitmask N can be fixed or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0172] In some examples, the luma position and C3 bitmask can be combined and switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different combinations can result in different classifiers.

[0173] In some cases, a bitmask restriction "max number of 1s" can be applied to limit the number of corresponding offsets. For example, limiting the bitmask "max number of 1s" to 4 in an SPS results in a maximum offset of 16 for the sequence. Bitmasks for different POCs may be different, but the "max number of 1s" shall not exceed 4 (all classes shall not exceed 16). The "max number of 1s" value can be signaled and switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. [Table 12]

[0174] As shown in FIG. 19, other cross-component chroma samples may also be fed into the CCSAO classification. The classifier for the cross-component chroma samples may be the same as the luma cross-component classifier or may have its own classifier as described in this disclosure. The two classifiers can be combined to form a combined classifier for classifying the current chroma sample. For example, a combined classifier that combines cross-component luma and chroma samples may result in a total of 16 classes as shown in Table 2-7 below. Table 2-7 shows an example of a classifier that uses a combined classifier that combines cross-component luma and chroma samples (bitmask positions are underlined). [Table 13]

[0175] All the classifiers mentioned above (C0, C1, C1', C2, C3) can be combined (combined). See Table 2-8 below for an example. Table 2-8 shows the combination of different classifiers. [Table 14]

[0176] In some examples, another classifier example (C4) uses the difference between the CCSAO input and the sample value to be compensated for classification. For example, if CCSAO is applied at the ALF stage, the difference between the pre-ALF and post-ALF sample values ​​of the current component is used for classification. As with the classifiers, one or more thresholds can be predefined (e.g., kept in a LUT) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level to help classify (quantize) the difference. The C4 classifier can be combined with C0 Y / U / V bandNum to form a joint classifier (e.g., the POC1 example shown in Table 2-9). Table 2-9 shows that the classifier example uses the difference between the CCSAO input value and the sample value to be compensated for classification. [Table 15]

[0177] In some embodiments, the example classifier (C5) uses "coding information" to aid in sub-block classification because different coding modes may introduce different distortion statistics into the reconstructed image. CCSAO samples are classified by their previous coding information, and the combination of coding information can form a classifier, for example, as shown in Table 2-10 below. Figure 39 shows another example of different stages of coding information for C5. Table 2-10 shows that CCSAO samples are classified by their previous coding information, and the combination of coding information can form a classifier. [Table 16]

[0178] In some examples, the classifier example (C6) uses YUV color transform values ​​for classification. For example, to classify the current Y component, select 1 / 1 / 1 co-located or adjacent Y / U / V samples, color transform them to RGB, and quantize the R value using C3 bandNum as the current Y component classifier.

[0179] In some examples, the example classifier (C7) can be considered a generalized version of C0 / C3 and C6. To determine the C0 / C3 bandNum classification of the current component, the co-located / current and adjacent samples of all three color components are used. For example, as in FIG. 13B, the current U sample, the co-located and adjacent Y / V, the current and adjacent U samples are used. This is formulated as follows:

number

[0180] In some embodiments, one special subset example of C7 can use only 1 / 1 / 1 co-located or adjacent Y / U / V samples to derive intermediate sample S. This can be considered a special case of C6 (color transformation using 3 components). S can be further fed into a C0 / C3 bandNum classifier. classIdx=bandS=(S*bandNumS)>>BitDepth;

[0181] In some embodiments, similar to the C0 / C3 bandNum classifiers, C7 can also be combined with other classifiers to create a combined classifier. In some examples, C7 may not be the same as the later example, which jointly uses co-located and adjacent Y / U / V samples for classification (ternary combined bandNum classification for each Y / U / V component).

[0182] In some embodiments, one constraint: ijThe sum of = 1, is c ij It is sometimes applied to reduce the signaling overhead of and to constrain the values ​​of S to be within the range of the bit depth. For example, c00=(1-sum of other c ij ) is enforced. ij Whether (c00 in this example) is forced (or derived by other coefficients) can be predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0183] In some embodiments, another classifier example (C8) uses the spatial activity information of the cross component / current component as the classifier. Similar to the block activity classifier above, one sample located at (k,l) can obtain the sample activity as follows: JPEG2024541528000036.jpg14126

[0184] JPEG2024541528000037.jpg26161 Here, the (BD-6) or B notation is a predefined normalization term related to bit depth.

[0185] In some embodiments, A is then further mapped to the range [0,4]:

number

[0186] In some embodiments, another classifier example (C9) can use the spatial gradient information of the cross component / current component as a classifier. Similar to the block gradient classifier above, one sample located at (k,l) can obtain the sample gradient class as follows: (1) Calculate the gradient (Laplacian or forward / backward) in N directions. (2) Calculate the maximum and minimum gradients of M grouped directions (M<=N). (3) Compare the N values ​​with each other and set m thresholds t1 to t m Calculate the directionality D. (4) Apply a geometric transformation according to the relative gradient magnitude (optional).

[0187] For example, similar to the ALF block classifier, but applying the following at the sample level for sample classification: (1) Calculate the gradient in four directions (Laplacian). (2) Calculate the maximum and minimum values ​​of the gradients in the two grouped directions (H / V and D / A). (3) Compare the N values ​​and set two thresholds t1 to t m Calculate the directionality D. (4) Apply a geometric transformation according to the relative gradient magnitude as shown in Table 1-6.

[0188] In some examples, C8 and C9 can be combined to create a combined classifier.

[0189] In some embodiments, another classifier example (C10) can use the edge information of the cross component / current component to classify the current component. By extending the original SAO classifier, C10 can more effectively extract the edge information of the cross / current component as follows: (1) Select one direction and calculate two edge strengths, where one direction is formed by the current sample and two adjacent samples, and one edge strength is calculated by subtracting the current sample from one adjacent sample. (2) Quantize each edge strength into M segments using M-1 thresholds Ti. (3) Classify the current component sample using M*M classes.

[0190] 22A-22B are diagrams illustrating an example of using edge information of the cross / current component to classify the current component, according to some implementations of the present disclosure. The current sample is represented by c, and the two adjacent samples of the current / cross component are represented by a and b. In this example, (1) Select one diagonal direction from the four candidate directions, and the differences (ca) and (cb) are two edge strengths in the range of -1023 to 1023 (e.g., for a 10b sequence). (2) Quantize each edge strength into four segments using a common threshold [-T, 0, T]. (3) Classify the current component samples using 16 classes.

[0191] As shown in Figures 22A-22B, one diagonal direction is selected, and the differences (ca) and (cb) are quantized into 4x4 segments with thresholds [-T,0,T] to form 16 edge segments. The location of (a,b) can be indicated by signaling two constructs: edgeDir and edgeStep.

[0192] In some examples, the direction pattern can extend to 0 degrees, 45 degrees, 90 degrees, 135 degrees (45 degrees between directions), or 22.5 degrees between directions, or be a predefined set of directions, or be signaled at the SPS / APS / PPS / PH / SH / region (set) / CTU / CU / sub-block / sample level.

[0193] In some instances, the edge strength may be defined as (ba), which simplifies the calculation but at the expense of accuracy.

[0194] In some examples, the M-1 thresholds may be predefined or signaled at the SPS / APS / PPS / PH / SH / region (set) / CTU / CU / subblock / sample level.

[0195] In some examples, the M-1 thresholds may be different sets for edge strength calculations, e.g., different sets for (ca), (cb). If different sets are used, the sum of the classes may be different. For example, if [-T,0,T] is used to calculate (ca) and [-T,T] is used to calculate (cb), the sum of the classes is 4*3.

[0196] In some cases, the M-1 thresholds can use the "symmetric" property to reduce signaling overhead. For example, one can use the predefined pattern [-T,0,T] and not [T0,T1,T2], which requires signaling three thresholds. Another example is [-T,T].

[0197] In some instances, the threshold may include only power-of-two values, which not only effectively captures the edge strength distribution but also reduces the complexity of the comparison (only the most significant N bits need to be compared).

[0198] In some examples, the positions of a, b can be indicated by signaling two constructs: (1) edgeDir, which indicates the selected direction, and (2) edgeStep, which indicates the sample distance used to calculate the edge strength, as in Figures 22A-22B.

[0199] In some examples, edgeDir / edgeStep may be predefined or signaled at the SPS / APS / PPS / PH / SH / region (set) / CTU / CU / subblock / sample level.

[0200] In some examples, edgeDir / edgeStep may be encoded in other ways, such as a fixed-length code (FLC), a truncated unary (TU) code, an exponential-golomb code of degree k (EGk), a signed EG0 (SVLC), or an unsigned EG0 (UVLC).

[0201] In some examples, C10 can be combined with bandNumY / U / V or other classifiers to form a combined classifier, for example combining 16 edge strengths with up to 4 bandNumY bands to yield 64 classes.

[0202] In some embodiments, other classifier examples that use only the current component information to classify the current component can be used as cross-component classifiers. For example, as shown in FIG. 5A and Table 1-1, the luma sample information and eo-class are used to determine EdgeIdx to classify the current chroma sample. Other "non-cross component" classifiers that can also be used as cross-component classifiers include edge direction, pixel intensity, pixel variation, pixel variance, pixel Laplacian sum, Sobel operator, Compass operator, high pass filtered value, low pass filtered value, etc.

[0203] In some embodiments, multiple classifiers are used in the same POC. The current frame is divided into multiple regions, and each region uses the same classifier. For example, in POC0, three different classifiers are used, and which classifier (0, 1, or 2) is used is signaled at the CTU level, as shown in Table 2-11 below. This indicates that different general classifiers are applied to different regions of the same picture. [Table 17]

[0204] In some embodiments, the maximum number of classifiers (also called alternative offset sets) is fixed or can be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. In one example, the fixed (predefined) maximum number of classifiers is 4. In that case, 4 different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level. Truncated unary (TU) codes can be used to indicate the classifier used for each luma or chroma CTB. For example, as shown in Table 2-12 below, if the TU code is 0, CCSAO is not applied. If the TU code is 10, set 0 is applied, if the TU code is 110, set 1 is applied, if the TU code is 1110, set 2 is applied, and if the TU code is 1111, set 3 is applied. Fixed length codes, golom-rice codes, and exponential-Golomb codes can also be used to indicate the classifier (offset set index) for the CTB. In POC1, three different classifiers are used. [Table 18]

[0205] Examples of CTB offset set indexes for Cb and Cr are shown for 1280x720 sequence POC0 (when CTU size is 128x128, the number of CTUs in a frame is 10x6). POC0 Cb uses 4 offset sets and Cr uses 1 offset set. For example, as shown in Table 2-13 below, if the offset set index is 0, CCSAO is not applied. If the offset set index is 1, set 0 is applied. If the offset set index is 2, set 1 is applied. If the offset set index is 3, set 2 is applied. If the offset set index is 4, set 3 is applied. type means the position of the selected co-located luma sample (Yi). Different offset sets may have different types, band_num, and corresponding offsets. Table 2-13 shows an example of CTB offset set indexes for Cb and Cr for 1280x720 sequence POC0 (where CTU size is 128x128, the number of CTUs in the frame is 10x6). [Table 19]

[0206] In some embodiments, an example of jointly applying co-located / current and adjacent Y / U / V samples for classification (three-component joint bandNum classification for each Y / U / V component) is shown in Table 2-14 below. Table 2-14 shows an example of jointly using co-located / current and adjacent Y / U / V samples for classification. In POC0, {24,1} offset set is applied to {Y,U,V} respectively. Each offset set can be adaptively switched at SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different offset sets can have different classifiers. For example, to classify the current Y4 luma sample as the candidate position (candPos) shown in Figure 13B and Figure 13C, Y set0 selects {current Y4, co-located U4, co-located V4} as candidates, and has different bandNum{Y,U,V}={16,1,2} respectively. Using {candY, candU, candV} as the sample values ​​of the selected {Y, U, V} candidates, the total number of classes is 32, and the classification index derivation is expressed as follows: bandY = (candY * bandNumY) >> BitDepth; bandU = (candU * bandNumU) >> BitDepth; bandV = (candV * bandNumV) >> BitDepth; classIdx = bandY * bandNumU * bandNumV + bandU * bandNumV + band V.

[0207] In some embodiments, the classIdx derivation of the combined classifier can be expressed as an "or-shift" form to simplify the derivation process. For example, for max bandNum={16,4,4}, classIdx=(bandY<<4)|(bandU<<2)|bandV It is.

[0208] Another example is the classification of component V set1 in POC1. In this example, we use candPos={neighbor Y8, neighbor U3, neighbor V0} with bandNum={4,1,2} to generate 8 classes. [Table 20]

[0209] In some embodiments, examples of joint use of co-located and adjacent Y / U / V samples for current Y / U / V sample classification are enumerated (three-component joint edgeNum(C1) and bandNum classification for each Y / U / V component), for example as shown in Table 2-15 below. edge CandPos is the center position used in C1 classifier, edge bitMask is the activation indicator of C1 adjacent samples, and edgeNum is the corresponding C1 class number. In this example, C1 is only applied to Y classifier (hence edgeNum is equal to edgeNumY), and edge candPos is always Y4 (current / co-located sample position). However, C1 is applicable to Y / U / V classifiers that have edge candPos as the adjacent sample position.

[0210] If diff denotes the comparison score of Y C1, then the derivation of classIdx is as follows: bandY = (candY * bandNumY) >> BitDepth; bandU = (candU * bandNumU) >> BitDepth; bandV = (candV * bandNumV) >> BitDepth; edgeIdx = diff + (edgeNum >> 1); bandIdx = bandY * bandNumU * bandNumV + bandU * bandNumV + band V; classIdx = bandIdx * edgeNum + edgeIdx; [Table 21] [Table 22] [Table 23]

[0211] In some embodiments, as described above, for a single component, multiple C0 classifiers can be combined (different position or weight combinations, bandNum) to form a combined classifier. This combined classifier can be combined with other components to classify one U sample (Y / V can have the same concept), for example, using two Y samples (candY / candX, and bandNumY / bandNumX), one U sample (candU and bandNumU), and one V sample (candV and bandNumV). The derivation of the class index is as follows: bandY = (candY * bandNumY) >> BitDepth; bandX = (candX * bandNumX) >> BitDepth; bandU = (candU * bandNumU) >> BitDepth; bandV = (candV * bandNumV) >> BitDepth; classIdx = bandY * bandNumX * bandNumU * bandNumV + bandX * bandNumU * bandNumV + bandU * bandNumV + band V;

[0212] In some embodiments, when using multiple C0s in one single component, some decoder norm constraints or encoder compatibility constraints may be applied. The constraints include: (1) the selected C0 candidates must be different from each other (e.g., candX!=candY), and / or (2) the newly added bandNum must be smaller than other bandNum (e.g., bandNumX<=bandNumY). By applying intuitive constraints within one single component (Y), redundant cases can be removed to save bit cost and complexity.

[0213] In some embodiments, max band_num (bandNumY, bandNumU, or bandNumV) can be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, fix max band_num=16 at the decoder and signal 4 bits in each frame to indicate C0 band_num in the frame. Some other max band_num examples are shown in Table 2-16 below. [Table 24]

[0214] In some embodiments, the maximum number of classes or offsets (combinations of using multiple classifiers jointly, e.g., C1s edgeNum*C1 bandNumY*bandNumU*bandNumV) for each set (or all added sets) can be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, the maximum could be fixed at class_num=256*4 for all added sets, and an encoder conformance check or a decoder norm check could be used to verify the constraint.

[0215] In some embodiments, restrictions can be applied to the C0 classification, for example restricting band_num (bandNumY, bandNumU, or bandNumV) to only power-of-two values. Instead of explicitly signaling band_num, the syntax band_num_shift is signaled. The decoder can use shift operations to avoid multiplications. Different band_num_shifts can be used for different components. Class(C0)=(Y0>>band_num_shift)>>bit_depth

[0216] Another example operation considers rounding to reduce error. Class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth

[0217] For example, if band_num_max (Y, U, or V) is 16, then the possible band_num_shift candidates are 0, 1, 2, 3, 4, corresponding to band_num=1, 2, 4, 8, 16, as shown in Table 2-17. [Table 25]

[0218] Offset Signaling

[0219] In some embodiments, the classifiers applied to Cb and Cr are different. The Cb and Cr offsets for all classes can be signaled separately. For example, different signaling offsets are applied for each chroma component, as shown in Table 2-18 below. [Table 26]

[0220] In some embodiments, the max offset value is fixed or signaled at the sequence parameter set (SPS) / adaptation parameter set (APS) / picture parameter set (PPS) / picture header (PH) / slice header (SH) / region / CTU / CU / sub-block / sample level. For example, the max offset is between [-15,15]. Different components can have different max offset values.

[0221] In some embodiments, the offset signaling can use Differential Pulse-code Modulation (DPCM), for example, the offset {3, 3, 2, 1, -1} can be signaled as {3, 0, -1, -1, -2}.

[0222] In an embodiment, the offsets may be stored in an APS or memory buffer for next picture / slice reuse, and an index may be signaled to indicate which of the stored previous frame offsets is to be used for the current picture.

[0223] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes can be signaled jointly, for example as shown in Table 2-19 below. [Table 27]

[0224] In some embodiments, the classifiers for Cb and Cr can be the same. The Cb and Cr offsets for all classes can be jointly signaled by different sign flags, for example as shown in Table 16 below. According to Table 2-20, if the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,-3,-2,1). [Table 28]

[0225] In some embodiments, a sign flag can be signaled for each class, for example as shown in Table 2-21 below: According to Table 2-21, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,3,2,1) according to the respective sign flags. [Table 29]

[0226] In some embodiments, the classifiers for Cb and Cr may be the same. The Cb and Cr offsets for all classes can be jointly signaled with weight differences, for example, as shown in Table 2-22 below, which shows that the Cb and Cr offsets for all classes can be jointly signaled with weight differences. The weights (w) can be selected within a limited table, such as ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., where |w| includes only power-of-two values. According to Table 18, if the Cb offset is (3,3,2,-1), the Cr offset derived based on the respective sign flags is (-6,-6,-4,2). [Table 30]

[0227] In some embodiments, weights can be signaled for each class as shown in Table 2-23 below. This indicates that the Cb offset and Cr offset of all classes can be signaled together with weights signaled for each class. According to Table 2-22, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-6,12,0,-1) according to the respective sign flags. [Table 31]

[0228] In some embodiments, when multiple classifiers are used for the same POC, different offset sets are signaled individually or jointly.

[0229] In some embodiments, previously decoded offsets can be saved for use in future frames. To reduce the overhead of offset signaling, an index can be signaled to indicate which previously decoded offset set to use for the current frame. For example, the POC0 offset can be reused by POC2 by signaling offset set idx=0, as shown in Table 2-23 below. Table 2-23 shows that an index can be signaled to indicate which previously decoded offset set to use for the current frame. [Table 32]

[0230] In some embodiments, the reuse offset set idx for Cb and Cr may be different, for example as shown in Table 2-24 below. Table 2-24 shows that an index can be signaled to indicate which previously decoded offset set is used for the current frame. The index may be different for the Cb and Cr components. [Table 33]

[0231] In some embodiments, offset signaling can use additional syntax including start and length to reduce signaling overhead. For example, when band_num=256, only offsets from band_idx=37 to 44 are signaled. In the example in Table 2-25 below, both start and length syntax are coded with a fixed length of 8 bits that aligns with the band_num bits. [Table 34]

[0232] In some embodiments, when CCSAO is applied to all YUV3 components, co-located and adjacent YUV samples can be jointly used for classification, and all the above-mentioned offset signaling methods for Cb / Cr can be extended to Y / Cb / Cr. In some embodiments, offset sets for different components are saved and can be used individually (each component has its own saved set) or jointly (each component shares / reuses the same saved set). An example of an individual set is shown in Table 2-26 below, which shows that offset sets for different components can be saved and can be used individually (each component has its own saved set) or jointly (each component shares / reuses the same saved set). [Table 35]

[0233] In some embodiments, if the sequence bit depth is greater than 10 (or a specific bit depth), the offset may be quantized before signaling. At the decoder side, the decoded offset is dequantized before applying it, as shown in Table 2-27 below. For example, for a 12-bit sequence, the decoded offset is shifted left (dequantized) by 2. [Table 36]

[0234] In some embodiments, the offset can be calculated as CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<(BitDepth-Min(10,BitDepth))).

[0235] In some embodiments, offset quantization is selectable (programmable) at the encoder. Whether offset quantization is enabled (on / off control) and the indicated quantization step size can be predefined or signaled at the SPS / APS / PPS / PH / SH / Region (set) / CTU / CU / Sub-block / Sample level. For example, the quantization step size can be predefined depending on the bit depth or resolution and switched at PH. The on / off control flag and the quantization step size can be stored in the APS for future frame reuse. The range of step sizes supported in this sequence can be predefined or signaled at the SPS / APS / PPS / PH / SH / Region (set) / CTU / CU / Sub-block / Sample level. The offset quantization mechanism allows the encoder to trade off bit cost and picture quality improvement for offset accuracy.

[0236] In some embodiments, the offset binarization method may depend on the quantization step size. The offset binarization method can be predefined or signaled at each SPS / APS / PPS / PH / SH / Region(Set) / CTU / CU / Sub-Block / Sample level. Different components can have different {on / off control, quantization step size, offset binarization method} or share the same ones. For example, U / V use the same one and Y uses a different one. Different bit depths of sequences can have different predefined quantization step size / offset binarization methods. For example, use different EGk orders for different quantization step sizes.

[0237] For example, for a sequence of {8b, 10b, 12b, 14b, 16b}, we predefine step size={0, 0, 2, 4, 6} and switch step size / binarization method at different levels.

[0238] For example, for an 8b sequence, 1 SPS flag to enable offset quantization, with predefined step size = 0 (offset=0,±1,±2…) One PH syntax that adaptively changes the step size to 2 (offset=0, ±4, ±8…) One domain (set) level syntax that adaptively changes the step size of each set (Set0=0,Set1=1…) · One region (set) level syntax to switch between predefined binarization methods: Set0:TU, Set1:EG1, Set2:FLC…

[0239] For example, for a 10b sequence, 1 SPS flag to enable offset quantization, with predefined step size = 1 (offset=0, ±2, ±4…) Predefined quantization step sizes for binarization mapping: 0->EG0,1->EG1,2->EG2 One APS syntax that preserves the previously used quantization step size (q) / corresponding EGk order. New pictures can add new APS indexes. Index0: Set0: q=0, Set1: q=2, Set2: q=1, Set3: q=0 Index1: Set0: q=1, Set1: q=0, Set2: q=0, Set3: q=2 … Each region (set) in one picture can reuse the quantization step size (q) / corresponding EGk order in the stored APS.

[0240] For example, for sequences of {<480p, 720p, 1080p, 4K, >= 8K}, we predefine step size = {0, 0, 2, 4, 6} and switch step size / binarization method at different levels.

[0241] In some embodiments, the concept of filter strength is further introduced. For example, the classifier offset can be further weighted before applying to the samples. The weights (w) can be selected in a table of power-of-two values, e.g., ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., etc., where |w| contains only power-of-two values. The weight index can be signaled at the level of SPS / APS / PPS / PH / SH / Region (Set) / CTU / CU / Sub-Block / Sample. The quantization offset signal can be considered as a subset of this weight application. When a recursive CCSAO is applied as shown in FIG. 13D, a similar weight index mechanism can be applied between the first and second stages.

[0242] In some examples, weightings for different classifiers, i.e. offsets for multiple classifiers, can be combined and applied to the same sample. A similar weight index mechanism can be signaled as described above. For example, offset_final = w * offset_1 + (1-w) * offset_2, or offset_final = w1 * offset_1 + w2 * offset_2 + …

[0243] Adaptation Parameter Set (APS)

[0244] In some embodiments, instead of directly signaling the CCSAO parameters to the PH / SH, previously used parameters / offsets can be stored in an adaptive parameter set (APS) or memory buffer for reuse in the next picture / slice. An index can be signaled to the PH / SH to indicate which previous stored frame offset is used for the current picture / slice. A new APS ID can be created to maintain the CCSAO history offsets. The table below shows an example using FIG. 13E, candPos, bandNum{Y,U,V}={16,4,4}. In some examples, the candPos, bandNum, offset signaling method can be fixed length code (FLC), or other methods such as truncated unary (TU) code, exponential-Golomb code of degree k (EGk), signed EG0 (SVLC), or unsigned EG0 (UVLC). In this case sao_cc_y_class_num (or cb,cr) is equal to sao_cc_y_band_num_y*sao_cc_y_band_num_u*sao_cc_y_band_num_v (or cb,cr). ph_sao_cc_y_aps_id is the index of the parameters used in this picture / slice. The cb and cr components can follow the same signaling logic. [Table 37] JPEG2024541528000060.jpg238148

[0245] aps_adaptation_parameter_set_id provides an identifier for the APS for reference by other syntax elements. If aps_params_type is equal to CCSAO_APS, the value of aps_adaptation_parameter_set_id shall be (for example) in the range from 0 to 7, inclusive.

[0246] ph_sao_cc_y_aps_id specifies the aps_adaptation_parameter_set_id of the CCSAO APS to which the Y color component of the slice of the current picture refers. If ph_sao_cc_y_aps_id is present, the following applies: the value of sao_cc_y_set_signal_flag of an APS NAL unit with aps_params_type equal to CCSAO_APS and aps_adaptation_parameter_set_id equal to ph_sao_cc_y_aps_id shall be equal to 1. The TemporalId of an APS Network Abstraction Layer (NAL) unit with aps_params_type equal to CCSAO_APS and aps_adaptation_parameter_set_id equal to ph_sao_cc_y_aps_id shall be less than or equal to the TemporalId of the current picture.

[0247] In some embodiments, an APS update mechanism is described herein. The maximum number of APS offset sets can be predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different components may have different maximum number limitations. If the APS offset set is full, the newly added offset set may replace one of the existing stored offsets using a first-in-first-out (FIFO), last-in-first-out (LIFO), or least-recently-used (LRU) scheme, or receive an index value indicating which APS offset set should be replaced. In some examples, if the selected classifier is configured with candPos / edge info / coding info..., etc., all classifier information may be captured as part of the APS offset set and may also be stored in the APS offset set along with its offset value. In some cases, the above update mechanism may be predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0248] In some embodiments, a constraint called "pruning" can be applied, e.g., newly received classifier information and offsets must not be the same as any of the stored APS offset sets (of the same component or across different components).

[0249] In some examples, when the C0 candPos / bandNum classifier is used, the maximum number of APS offset sets is 4 per Y / U / V, and FIFO updates are used for Y / V, with idx indicating the update used for U. Table 2-29 shows the updates of the CCSAO offset sets using FIFO. [Table 38]

[0250] In some embodiments, the pruning criteria can be relaxed to provide a more flexible method for encoder tradeoffs, such as allowing N offsets to vary when applying the pruning operation (e.g., N=4), or in another example, allowing a difference in the value of each offset (denoted by "thr") when applying the pruning operation (e.g., ±2), etc.

[0251] In some embodiments, the two criteria may be applied simultaneously or separately, and the application of each criterion may be predefined or toggled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0252] In some embodiments, N / thr can be predefined or switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0253] In some embodiments, the FIFO updates can be (1) cyclically updating from the previously left set idx as in the example above (when all are updated, start again from set 0), or (2) updating every time from set 0. In some examples, updates can be done at the PH (as in the example) or SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level when a new offset set is received.

[0254] For LRU updates, the decoder maintains a count table, which counts the "total number of offset sets used" and can be updated at SPS / APS / Group of Pictures (GOP) / PPS / PH / SH / Region / CTU / CU / Sub-block / Sample level. A newly received offset set replaces the least recently used offset set in an APS. If two stored offset sets have the same count, FIFO / LIFO can be used. See for example component Y in Table 2-30 below. [Table 39]

[0255] In some embodiments, different components may have different update mechanisms.

[0256] In some embodiments, different components (e.g. U / V) can share the same classifier (same candPos / edge info / coding info / offsets can further have weights in modifiers).

[0257] In some embodiments, a "patch" implementation may be used in the offset replacement mechanism since offset sets used in different pictures / slices may have only small offset value differences. In some embodiments, the "patch" implementation is differential pulse code modulation (DPCM). For example, when signaling a new offset set (OffsetNew), the offset value may be placed on top of an existing APS stored offset set (OffsetOld). The encoder signals only the difference value to update the old offset set (DPCM: OffsetNew=OffsetOld+delta). As shown in Table 2-31, options other than FIFO update may also be used in the following example (LRU, LIFO, or signaling an index indicating which set to update). The YUV components may have the same update mechanism or may use different update mechanisms. In the example of Table 2-31, the classifier candPos / bandNum does not change, but overwriting of the set classifier can be indicated by signaling an additional flag (flag=0: update set offset only, flag=1: update both set classifier and set offset). [Table 40]

[0258] In some embodiments, the DPCM differential offset value may be signaled with FLC / TU / EGk (order=0,1,...) code. One flag may be signaled for each offset set, indicating whether to enable DPCM signal. The DPCM differential offset value, or the newly added offset value (signaled directly without DPCM when APS DPCM=0 is enabled) (ccsao_offset_abs) may be dequantized / mapped before applying to the target offset (CcSaoOffsetVal). The offset quantization step may be predefined or signaled at each SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, one method is: Quantization step=2: CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<1) The solution is to directly signal the offset with

[0259] Another way is with quantization step=2: CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<1) The other is to directly use the DPCM signaling offset in the 1 / 2 kHz band.

[0260] In some embodiments, a constraint may be applied to reduce the overhead of direct offset signaling, e.g., the updated offset value must have the same sign as the old offset value. With an inferred offset sign in this way, the newly updated offset does not need to send the sign flag again (ccsao_offset_sign_flag is inferred to be the same as the old offset).

[0261] In some embodiments, the sample processing is described below. Let R(x,y) be the input luma or chroma sample value before CCSAO, and R'(x,y) be the output luma or chroma sample value after CCSAO: offset = ccsao_offset [class_index of R(x, y)] R'(x, y) = Clip3( 0, (1 << bit_depth) - 1, R(x, y) + offset )

[0262] Sample processing

[0263] According to the above formula, each luma or chroma sample value R(x,y) is classified using the classifier indicated by the current picture and / or the current offset set idx. The corresponding offset of the derived class index is added to each luma or chroma sample value R(x,y). The clip function Clip3 is applied to (R(x,y)+offset) to bring the luma or chroma output sample value R’(x,y) within the bit-depth dynamic range, for example, within the range from 0 to (1<<bit_depth)-1.

[0264] For each luma sample or chroma sample, first, classify using the classifier indicated by the current picture / current offset set idx, second, add the corresponding offset of the derived class index, and third, clip to the bit-depth dynamic range.

[0265] FIG. 6 is a block diagram showing that in some implementations of the present disclosure, both the proposed bilateral filter (BIF) and SAO use samples from the deblocking stage as inputs.

[0266] In some embodiments, when CCSAO operates with other loop filters, the clip operation may be as follows. (1) Clipping after addition: The following formulas show examples of (a) when CCSAO operates with SAO and BIF, or (b) when CCSAO replaces SAO and still operates with BIF. (a) I OUT =clip1(I C +ΔI SAO +ΔI BIF ++ΔI CCSAO ) (b) I OUT =clip1(I C +ΔI CCSAO +ΔI BIF ) (2) Clipping before addition when operating with BIF: In some embodiments, the order of clipping is switchable. (a) IOUT =clip1(I C +ΔI SAO ) I' OUT =clip1(I OUT +ΔI BIF ) I'' OUT =clip1(I'' OUT +ΔI CCSAO ) (b)I OUT =clip1(I C +ΔI BIF ) I' OUT =clip1(I' OUT +ΔI CCSAO ) (3) Clipping after partial addition (a)I OUT =clip1(I C +ΔI SAO +ΔI BIF ) I' OUT =clip1(I OUT +ΔI CCSAO )

[0267] In some embodiments, different clipping combinations provide different tradeoffs between correction accuracy and hardware temporary buffer size (register or SRAM bit width).

[0268] FIG. 6 shows SAO / BIF offset clipping. More specifically, for example, FIG. 6 shows the design of the current BIF when interacting with SAO. The offsets from SAO and BIF are added to the input sample, followed by one-bit-depth clipping. However, if the CCSAO is also added at the SAO stage, two possible clipping designs can be selected: (1) add one bit-depth clipping to the CCSAO, or (2) one harmonic design that performs joint clipping after adding the SAO / BIF / CCSAO offsets to the input sample. In some embodiments, the above clipping designs differ only in the luma samples, since the BIF is applied only to the luma samples.

[0269] Boundary Processing

[0270] In some embodiments, the boundary process is described below. If any of the co-located and adjacent luma (chroma) samples used for classification are located outside the current picture, the CCSAO is not applied to the current chroma (luma) sample. Figures 23A-23B are block diagrams illustrating that if any of the co-located and adjacent luma (chroma) samples used for classification are outside the current picture, the CCSAO is not applied to the current chroma (luma) sample according to some implementation examples of the present disclosure. For example, in Figure 23A, if a classifier is used, the CCSAO is not applied to the chroma components of the left one column of the current picture. For example, if C1' is used, the CCSAO is not applied to the chroma components of the left one column and the top one row of the current picture, as shown in Figure 23B.

[0271] 24A-24B are diagrams illustrating the application of CCSAO to a current luma or chroma sample when any of the co-located and adjacent luma or chroma samples used for classification are outside the current picture, according to some implementations of the present disclosure. In some embodiments, in one variation, when any of the co-located and adjacent luma or chroma samples for classification are outside the current picture, the CCSAO can be applied to the current luma or chroma sample by repeating the missing sample as shown in FIG. 24A or by mirror padding the missing sample to create a sample for classification as shown in FIG. 24B. In some embodiments, when any of the co-located and adjacent luma (chroma) samples used for classification are outside the current sub-picture / slice / tile / patch / CTU / 360 virtual boundary, the invalidation / repeat / mirror picture boundary processing methods disclosed herein can also be applied to the sub-picture / slice / tile / CTU / 36 degree virtual boundary.

[0272] For example, a picture may be divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs that covers a rectangular area of ​​the picture.

[0273] A slice consists of an integer number of complete tiles or an integer number of consecutive complete CTU rows within a picture tile.

[0274] A subpicture contains one or more slices that collectively cover a rectangular area of ​​the picture.

[0275] In some embodiments, 360-degree video is captured on a sphere and does not have inherently "borders". Thus, reference samples outside the boundaries of a reference picture in the projection domain can always be derived from neighboring samples in the spherical domain. In a projection format consisting of multiple planes, no matter how dense the frame packing arrangement is used, discontinuities appear between two or more adjacent planes in a frame-packed picture. In VVC, vertical and / or horizontal virtual boundaries are introduced where in-loop filtering operations are disabled, and the location of those boundaries is signaled in either the SPS or the picture header. Compared to using two tiles, one for each set of consecutive planes, the use of 360-degree virtual boundaries is more flexible because the size of the planes does not need to be a multiple of the CTU size. In some embodiments, the maximum number of vertical 360-degree virtual boundaries is three, and the maximum number of horizontal 360 virtual boundaries is also three. In some embodiments, the distance between two virtual boundaries is equal to or greater than the CTU size, and the granularity of the virtual boundaries is 8 luma samples, e.g., an 8x8 sample grid.

[0276] 28A-28B are block diagrams illustrating that the CCSAO is not applied to a current chroma sample if the corresponding selected co-located or adjacent luma sample used for classification is outside the virtual space defined by the virtual boundary, according to some implementations of the present application. In some embodiments, the virtual boundary (VB) is a virtual line that separates a space in a picture frame. In some embodiments, once the virtual boundary (VB) is applied in the current frame, the CCSAO is not applied to chroma samples that have a selected corresponding luma position outside the virtual space defined by the virtual boundary. FIG. 28A-28B are examples with virtual boundaries for a C0 classifier with nine luma position candidates. For each CTU, the CCSAO is not applied to chroma samples whose corresponding selected luma position is outside the virtual space bounded by the virtual boundary. For example, in FIG. 28A, if the selected Y7 luma sample position is on the other side of the horizontal virtual boundary 2806 located four pixel lines from the bottom of the frame, the CCSAO is not applied to chroma sample 2802. For example, in FIG. 28B, if the selected Y5 luma sample position is located on the opposite side of the vertical virtual boundary 2808 that is located on y pixel lines from the right side of the frame, then CCSAO is not applied to the chroma sample 2804.

[0277] 32A-32B show that repetitive padding or mirror padding can be applied to luma samples outside the virtual boundary according to some implementations of the present disclosure. FIG. 32A shows an example of repetitive padding. If the original Y7 is selected as the classifier located below VB3202, the luma sample value of Y4 is used for classification instead of the original Y7 luma sample value (copied to the Y7 position). FIG. 32B shows an example of mirror padding. If Y7 is selected as the classifier located below VB3204, the Y1 luma sample value, which is symmetrical to the Y7 value with respect to the Y0 luma sample value, is used for classification instead of the original Y7 luma sample value. The padding scheme gives more chroma samples applicability to CCSAO, so more coding gain can be achieved.

[0278] In some embodiments, restrictions can be applied to reduce the line buffer required for CCSAO and simplify boundary processing condition checks. Figure 26A shows that, according to some implementations of the present disclosure, if all 9 co-located adjacent luma samples are used for classification, an additional 1 luma line buffer may be required, i.e., all line luma samples of line -5 above the current VB1602. Figures 18A-18G show an example of using only 6 luma candidates for classification. This reduces the line buffer and does not require the additional boundary checks in Figures 23A-23B and 24A-24B.

[0279] In some embodiments, using luma samples for CCSAO classification may increase the luma line buffer and thus increase the hardware implementation cost of the decoder. FIG. 25 illustrates that nine luma candidate CCSAOs crossing VB 1702 may increase two additional luma line buffers according to some embodiments of the present disclosure. For luma and chroma samples above the virtual boundary (VB) 1702, DBF / SAO / ALF are processed in the current CTU row. For luma and chroma samples below VB 1702, DBF / SAO / ALF are processed in the next CTU row. In the hardware design of the AVS decoder, pre-DBF samples of luma lines -4 to -1, pre-SAO samples of line -5, and pre-DBF samples of chroma lines -3 to -1, pre-SAO samples of line -4 are saved as line buffers for the next CTU row DBF / SAO / ALF processing. When processing the next CTU row, luma and chroma samples that are not in the line buffers are unavailable. However, for example, at the location of chroma line -3(b), the chroma sample is processed in the next CTU row, but CCSAO needs pre-SAO luma sample lines -7, -6, and -5 for classification. Pre-SAO luma sample lines -7 and -6 cannot be used because they are not in the line buffer. Also, adding pre-SAO luma sample lines -7 and -6 to the line buffer would increase the hardware implementation cost of the decoder. In some examples, luma VB (line -4) and chroma VB (line -3) may be different (misaligned).

[0280] Similar to Figure 25, Figure 26A is an illustration showing in VVC that nine luma candidate CCSAOs intersecting VB1802 may increase one additional luma line buffer according to some embodiments of the present disclosure. Different standards may have different VBs. In VVC, luma VB is line -4 and chroma VB is line -2, so nine CCSAO candidates will increase one luma line buffer.

[0281] In some embodiments, in the first solution, if any of the luma candidates of a chroma sample straddles a VB (outside the VB of the current chroma sample), the CCSAO is disabled for the chroma sample. Figures 27A-27C show that in AVS and VVC, according to some embodiments of the present disclosure, if any of the luma candidates of a chroma sample straddles a VB2702 (outside the VB of the current chroma sample), the CCSAO is disabled for the chroma sample. Figures 28A-28B also show some examples of this implementation.

[0282] In some embodiments, in the second solution, for "cross-VB" luma candidates, repetition padding from luma lines close to and opposite VB, e.g., luma line -4, is used for CCSAO. In some embodiments, repetition padding from the nearest luma below VB is implemented for "cross-VB" chroma candidates. Figures 29A-29C are diagrams illustrating that CCSAO is enabled using repetition padding for chroma samples if any of the luma candidates of the chroma samples straddles VB (outside the current chroma sample VB) in AVS and VVC according to an exemplary implementation of the present application. Figure 28A also shows some examples of this implementation.

[0283] In some embodiments, as a third solution, mirror padding from below the luma VB is used for the CCSAO for "cross-VB" luma candidates. Figures 30A-30C illustrate enabling CCSAO using mirror padding for a chroma sample when any of the luma candidates of the chroma sample straddles VB3002 (outside the current chroma sample VB) in AVS and VVC according to some implementations of the present disclosure. Figures 28B and 24B also illustrate some examples of this implementation. In some embodiments, as a fourth solution, "two-sided symmetric padding" is used for applying the CCSAO. Figures 31A-31B illustrate enabling CCSAO using two-sided symmetric padding for some examples of different CCSAO shapes (e.g., 9 luma candidates (Figure 31A) and 8 luma candidates (Figure 31B)) according to some implementations of the present disclosure. For luma sample sets with a central luma sample co-located with the chroma samples, if one side of the luma sample set is outside VB3 102, then two-sided symmetric padding is applied to both sides of the luma sample set. For example, in Figure 31A, luma samples Y0, Y1, Y2 are outside VB3 102, so both Y0, Y1, Y2 and Y6, Y7, Y8 are padded using Y3, Y4, Y5. For example, in Figure 31B, luma sample Y0 is outside VB3 102, so Y0 is padded with Y2 and Y7 is padded with Y5.

[0284] 26B illustrates that when co-located or adjacent chroma samples are used to classify the current luma sample, the selected chroma candidate may span VB and require an additional chroma line buffer according to some embodiments of the present disclosure. Solutions 1-4 similar to those described above can be applied to address the problem.

[0285] Solution 1 is to disable CCSAO for a luma sample if any of its chroma candidates spans VB.

[0286] Solution 2 is to use iterative matching from the nearest chromas under VB for the "cross VB" chroma candidates.

[0287] Solution 3 is to use mirror padding from below chroma VB for the "cross VB" chroma candidate.

[0288] Solution 4 is to use "double-sided symmetric padding". For a candidate set centered on a CCSAO co-located chroma sample, if one side of the candidate set is outside the VB, double-sided symmetric padding is applied to both sides.

[0289] The padding method offers the possibility to apply CCSAO to more luma or chroma samples, and therefore more coding gain can be realized.

[0290] In some embodiments, in the bottom picture (or slice, tile, brick) boundary CTU row, the samples below VB are processed in the current CTU row, and therefore the above special handling (solutions 1, 2, 3, 4) is not applied to this bottom picture (or slice, tile, brick) boundary CTU row. For example, a 1920x1080 frame is divided into 128x128 CTUs. The frame contains 15x9 (rounded up) CTUs. The bottom CTU row is the 15th CTU row. The decoding process is done CTU row by CTU, and CTU by CTU for each CTU row. Deblocking needs to be applied along the horizontal CTU boundary between the current CTU row and the next CTU row. Inside one CTU, VB in CTB is applied to each CTU row because in the bottom 4 / 2 luma / chroma line, the DBF samples (in case of VVC) are processed in the next CTU row and are not available for CCSAO in the current CTU row. However, in the bottom CTU row of the picture frame, there is no next CTU row left, so the DBF samples of the bottom 4 / 2 luma / chroma line are available in the current CTU row and are DBF processed in the current CTU row.

[0291] In some embodiments, the VBs shown in Figures 13-22 may be replaced with subpicture / slice / tile / patch / CTU / 360 virtual boundary boundaries. In some embodiments, the positions of the chroma and luma samples in Figures 13-22 may be swapped. In some embodiments, the positions of the chroma and luma samples in Figures 6, 23A-32B may be replaced with the positions of the first and second chroma samples. In some embodiments, the ALF VBs inside the CTU may be generally horizontal. In some embodiments, the subpicture / slice / tile / patch / CTU / 360 virtual boundary boundaries may be horizontal or vertical.

[0292] In some embodiments, restrictions can be applied to reduce the line buffers required for CCSAO and simplify boundary processing condition checks. Figure 26A shows that an additional 1 luma line buffer (full line luma samples for line:-5) may be required if all 9 co-located neighboring luma samples are used for classification. Figures 33A-B show restrictions of using a limited number of luma candidates for classification in accordance with some embodiments of the present disclosure. Figure 33A shows a restriction of using only 6 luma candidates for classification. Figure 33B shows a restriction of using only 4 luma candidates for classification.

[0293] Application area

[0294] In some embodiments, an application region is implemented. The CCSAO application region unit can be based on a CTB, i.e., the on / off control, CCSAO parameters (classification, offset used for offset set index, luma candidate position, band_num, bitmask, etc.) are the same within one CTB.

[0295] In some embodiments, the application region may not be aligned to the CTB boundary. For example, the application region is not aligned to the chroma CTB boundary, but is shifted. The syntax (on / off control, CCSAO parameters) is still signaled for each CTB, but the true application region is not aligned to the CTB boundary. FIG. 34 shows that the CCSAO application region is not aligned to the CTB / CTU boundary 3406 by some implementations of the present disclosure. For example, the application region is not aligned to the chroma CTB / CTU boundary 3406, but is sample shifted (4,4) up-left relative to the VB 3408. This non-aligned CTB boundary design benefits the deblocking process because the same deblocking parameters are used for each 8×8 deblocking process region.

[0296] In some embodiments, the CCSAO application region unit (mask size) may be variable (larger or smaller than CTB size), as shown in Table 2-32. The mask size may be different for different components. The mask size can be switched at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, in PH, a series of mask on / off flags and offset set index are signaled to indicate each CCSAO region information. [Table 41]

[0297] In some embodiments, the CCSAO applicability region frame partition may be fixed, for example, partitioning the frame into N regions. Figure 35 illustrates that, according to some implementations of the present disclosure, the CCSAO applicability region frame partition may be fixed by a CCSAO parameter.

[0298] In some embodiments, each region can have its own region on / off control flag and CCSAO parameters. Also, if the region size is larger than the CTB size, it can have both CTB on / off control flag and region on / off control flag. Figure 35(a) and Figure 35(b) show some examples of partitioning a frame into N regions. Figure 35(a) shows vertical partition into four regions. Figure 35(b) shows square partition into four regions. In some embodiments, similar to the picture level CTB all on control flag (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag), if the region on / off control flag is off, the CTB on / off flag can be further signaled. Otherwise, the CCSAO is applied to all CTBs in this region without further signaling the CTB flag.

[0299] In some embodiments, different CCSAO application regions can share the same region on / off control and CCSAO parameters. For example, in Figure 35(c), regions 0-2 share the same parameters, and regions 3-15 share the same parameters. Figure 35(c) also shows that the region on / off control flags and CCSAO parameters can be signaled in Hilbert scan order.

[0300] In some embodiments, the CCSAO application area unit can be a quadtree / binarytree / ternarytree partition from the picture / slice / CTB level. Similar to the CTB partition, a series of partitioned flags are signaled to indicate the CCSAO application area partition. Figure 36 shows that according to some implementations of the present disclosure, the CCSAO application area can be a binary tree (BT) / quadtree (QT) / ternary tree (TT) partition from the frame / slice / CTB level.

[0301] FIG. 37 is a block diagram illustrating multiple classifiers used and switched at different levels within a picture frame, according to some implementations of the present disclosure. In some embodiments, when multiple classifiers are used in a frame, the method of applying the classifier set index can be switched at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. For example, four classifier sets are used in a frame and switched at PH, as shown in Table 2-33 below. FIG. 37(a) and FIG. 37(c) show the default fixed region classifier. FIG. 37(b) shows the classifier set index is signaled at the mask / CTB level, where 0 means CCSAO off for this CTB, and 1-4 means set index. [Table 42]

[0302] In some embodiments, for a default region, a region level flag may be signaled if the CTB in this region does not use the default set index (e.g., the region level flag is 0) but uses other classifier sets in this frame. For example, if the default set index is used, the region level flag is 1. For example, in four regions of a square partition, the following classifier sets are used, as shown in Table 2-34 below, which indicates that a region level flag may be signaled to indicate whether the CTB in this region does not use the default set index. [Table 43]

[0303] FIG. 38 is a block diagram illustrating that according to some implementations of the present disclosure, the CCSAO application region partition is dynamic and can be switched at the picture level. For example, FIG. 38(a) shows that three CCSAO offset sets are used in this POC (set_num=3), and the picture frame is partitioned vertically into three regions. FIG. 38(b) shows that four CCSAO offset sets are used in this POC (set_num=4), and the picture frame is partitioned horizontally into four regions. FIG. 38(c) shows that three CCSAO offset sets are used in this POC (set_num=3), and the picture frame is partitioned by raster into three regions. Each region can have its own region all-on flag to store the CTB on / off control bits. The number of regions depends on the picture set_num being signaled.

[0304] The CCSAO application region may be a specific area according to the coding information within the block (such as sample positions, sample coding modes, loop filter parameters, etc.). For example, 1) the CCSAO application region is only applicable if the sample is skip mode coded, or 2) the CCSAO application region includes only N samples along a CTU boundary, or 3) the CCSAO application region includes only samples on an 8x8 grid in a frame, or 4) the CCSAO application region includes only DBF filtered samples, or 5) the CCSAO application region includes only the top M rows and left N rows in a CU, or (6) the CCSAO application region includes only intra-coded samples, or (7) the CCSAO application region includes only samples in cbf=0 blocks, or (8) the CCSAO application region is only on blocks with block QP of [N,M], where (N,M) can be pre-defined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level. Cross-component coding information may also be taken into account, where (9) the CCSAO application region is on chroma samples whose co-located luma samples are within a cbf=0 block.

[0305] In some embodiments, whether to introduce coding information application area restriction can be predefined, or one control flag can be signaled at SPS / APS / PPS / PH / SH / region (per alternative set) / CTU / CU / sub-block / sample level to indicate whether specific coding information is included / not included in CCSAO application. According to the predefined condition or control flag, the decoder skips CCSAO processing to those regions. For example, YUV uses another predefined condition / flag-controlled condition switched at region (set) level. CCSAO application decision can be at CU / TU / PU level or sample level. Table 2-35 shows that YUV uses another predefined condition / flag-controlled condition switched at region (set) level. [Table 44]

[0306] Another example is the reuse of all or part of the bilateral validation constraints (predefined). bool isInter = (currCU.predMode == MODE_INTER ) ? true : false; if (ccSaoParams.ctuOn [ctuRsAddr] && ((TU::getCbf (currTU, COMPONENT_Y) || isInter == false ) && (currTU.cu ->qp > 17 )) && (128 > std::max(currTU.lumaSize().width, currTU.lumaSize().height)) && ((isInter == false ) || (32 > std::min(currTU.lumaSize().width, currTU.lumaSize().height))))

[0307] In some embodiments, excluding certain areas may be beneficial for CCSAO statistics collection. For areas that really need correction, offset derivation may be more accurate or appropriate. For example, a block with cbf=0 usually means that the block is perfectly predicted and does not need further correction. Excluding such blocks may be beneficial for offset derivation in other areas.

[0308] Different application regions can use different classifiers. For example, in a CTU, skip mode uses C1, 8x8 grid uses C2, and skip mode and 8x8 grid uses C3. For example, in a CTU, samples coded in skip mode use C1, samples in the center of a CU use C2, and samples coded in skip mode at the center of a CU use C3. Figure 39 illustrates that a CCSAO classifier can take into account coding information of a current component or cross-component, according to some implementations of the present disclosure. For example, different coding modes / parameters / sample positions can form different classifiers. Different coding information can also be combined to form a joint classifier. Different regions can use different classifiers. Figure 29 illustrates another example of application regions.

[0309] In some embodiments, a predefined or flag-controlled "encoded information exclusion area" mechanism may be used in the DBF / Pre-SAO / SAO / BIF / CCSAO / ALF / CCALF / NN Loop Filter (NNLF), or other loop filters.

[0310] syntax

[0311] The CCSAO syntax implemented in some embodiments is shown in Table 2-36 below. In some examples, the binarization of each syntax element can be changed. In AVS3, the term patch is similar to slice, and the patch header is similar to the slice header. FLC stands for fixed length code. TU stands for truncated unary code. EGk stands for exponential-Golomb code with degree k, where k can be fixed. SVLC stands for signed EG0. UVLC stands for unsigned EG0. [Table 45] JPEG2024541528000069.jpg214164

[0312] If a higher level flag is off, the lower level flags can be inferred from their off state and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this picture, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag are not present and are inferred to be false.

[0313] In some embodiments, the SPS ccsao_enabled_flag is conditional on the SPS SAO enabled flag, as shown in Table 2-37 below. [Table 46]

[0314] In some embodiments, ph_cc_sao_cb_ctb_control_flag, ph_cc_sao_cr_ctb_control_flag indicate whether to enable the granularity of CTB on / off control of Cb / Cr. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag may be further signaled. Otherwise, whether CCSAO is applied to the current picture depends on ph_cc_sao_cb_flag, ph_cc_sao_cr_flag, and ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag are not further signaled at the CTB level.

[0315] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, in order to reduce bit overhead, a flag may be further signaled to distinguish whether the co-located central luma position (Y0 position in Figures 18A-18G) is used for classification on chroma samples. Similarly, if cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, a flag may be further signaled by the same mechanism. For example, as shown in Table 2-38 below, if the number of C0 luma position candidates is 9, cc_sao_cb_type0_flag is further signaled to distinguish whether the co-located central luma position is used. If the co-located central luma position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining 8 adjacent luma positions is used. [Table 47]

[0316] Table 2-39 below shows an example in which a single (set_num=1) or multiple (set_num>1) classifiers are used in a frame in AVS. The syntax notation can be mapped to the notation used above. [Table 48]

[0317] When combined with FIG. 35 or FIG. 37 where each region has its own set, an example syntax may include region on / off control flags (picture_ccsao_lcu_control_flag[compIdx][setIdx]) as shown in Table 2-40 below. [Table 49]

[0318] In some embodiments, pps_ccsao_info_in_ph_flag and gci_no_sao_constraint_flag may be added to the high level syntax.

[0319] In some embodiments, pps_ccsao_info_in_ph_flag equal to 1 specifies that CCSAO filter information may be present in a PH syntax structure and is not present in slice headers that reference a PPS that does not contain a PH syntax structure. pps_ccsao_info_in_ph_flag equal to 0 indicates that CCSAO filter information is not present in a PH syntax structure and may be present in slice headers that reference a PPS. If pps_ccsao_info_in_ph_flag is not present, its value is inferred to be equal to 0.

[0320] In some embodiments, gci_no_ccsao_constraint_flag equal to 1 specifies that sps_ccsao_enabled_flag of all pictures in OlsInScope is equal to 0. When gci_no_ccsao_constraint_flag is equal to 0, no such constraint is imposed. In some embodiments, a video bitstream is composed of one or more output layer sets (OLS) according to a rule. In the examples herein, OlsInScope refers to one or more OLSs that are in scope. In some examples, a profile_tier_level() syntax structure provides level information to which OlsInScope conforms, and optionally profile, tier, subprofile, and general constraint information. If the profile_tier_level() syntax structure is included in the VPS, OlsInScope is one or more OLSs specified in the VPS. If the profile_tier_level() syntax is included in the SPS, OlsInScope is an OLS that includes only the lowest layer among the layers that reference the SPS, and this lowest layer is an independent layer.

[0321] In some embodiments, the separate signaling of band_num_y_minus1, band_num_u_minus1, band_num_v_minus1 may result in syntactic redundancy. For example, as shown in Table 2-41, U1 / V1 is the same as Y1 because no segment applies to U / V (it is redundant). [Table 50]

[0322] In some embodiments, if the classifier is a band classifier or a combined classifier that includes a band classifier, an indicator is predefined or signaled by the encoder. In some examples, the band classifier can be determined by utilizing sample values ​​based on one or more samples from the same location and / or adjacent samples of the Y component and the current and adjacent samples of the U / V components for each sample of the U / V components, dividing the range of sample values ​​into multiple bands, and selecting a band from the multiple bands.

[0323] In some examples, if a classifier, or a composite classifier (e.g., C0+C10), is composed of bandNum segments of one or more components, the indicator may be a bandNum indicator (bandIdc, bandNum mapping table) predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block level to indicate the bandNum segments of one or more components. Different components can have different bandNum indicators. Different components can share the same bandNum indicator. For example, U / V share the same bandNum mapping table.

[0324] For example, if the bandIdc of U / V contains bandNum segments of two or more components, the bandNum indicator can be predefined for different components. Table 2-42 shows an example of a bandNum mapping table. As shown in Table 2-42 below, if the current sample is a U component, a bandNum indicator signaled as 2 may indicate the use of the Y3 segment, i.e., the bandNum segment of the Y component. In another example, if the current sample is a V component, a bandNum indicator signaled as 6 indicates the use of the U2 and V2 segments, and a bandNum indicator signaled as 7 indicates the use of the Y2, U2, and V2 segments. [Table 51]

[0325] In some embodiments, C (current chroma) and CT (transposed chroma, using other chroma component) can be used to represent bandIdc. Table 2-43 shows an example of a bandNum mapping table. For example, as shown in Table 2-43, when C represents the current chroma component, CT represents the other chroma component. For example, when C represents the U component, CT represents the V component, and when C represents the V component, CT represents the U component. [Table 52]

[0326] In some embodiments, the bandNum mapping table may be predefined or adjusted / changed by the encoder at SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block level according to different granularity requirements.

[0327] Extensions to intra and interpost prediction SAO filters

[0328] In some embodiments, the extension to intra and inter post-prediction SAO filters is further described below. In some embodiments, the SAO classification method disclosed in this disclosure (including cross-component sample / coding information classification) can function as a post prediction filter, where the prediction can be intra, inter, or other prediction tools such as Intra Block Copy. Figure 40A is a block diagram illustrating the SAO classification method disclosed in this disclosure acting as a post prediction filter according to some implementations of the present disclosure.

[0329] In some embodiments, for each Y, U, and V component, a corresponding classifier is selected. For each component prediction sample, classification is first performed and a corresponding offset is added. For example, each component can use the current sample and neighboring samples for classification. As shown in Table 2-44 below, Y uses the current Y sample and neighboring Y samples, and U / V uses the current U / V sample for classification. Figures 40B-40D are block diagrams illustrating that each component can use the current sample and neighboring samples for classification for a post-prediction SAO filter, according to some implementations of the present disclosure. [Table 53]

[0330] In some embodiments, the refined prediction samples (Ypred', Upred', Vpred') are updated by adding the corresponding class offsets and then used for intra, inter, or other prediction. Ypred' = clip3(0, (1 << bit_depth)-1, Ypred + h_Y[i]) Upred' = clip3(0, (1 << bit_depth)-1, Upred + h_U[i]) Vpred' = clip3(0, (1 << bit_depth)-1, Vpred + h_V[i])

[0331] In some embodiments, in addition to the current chroma component, for the chroma U and V components, the cross component (Y) can be used for further offset classification. For example, as shown in Table 2-45 below, an additional cross component offset (h'_U,h'_V) can be added to the current component offset (h_U,h_V). [Table 54]

[0332] In some embodiments, the improved prediction samples (Upred'', Vpred'') are updated by adding the corresponding class offset and then used for intra, inter or other prediction. Upred'' = clip3(0, (1 << bit_depth)-1, Upred' + h'_U[i]) Vpred'' = clip3(0, (1 << bit_depth)-1, Vpred' + h'_V[i])

[0333] In some embodiments, intra and inter prediction may use different SAO filter offsets.

[0334] Extension to post reconstruction filters

[0335] FIG. 15C is a block diagram illustrating the SAO classification method disclosed in this disclosure acting as a post-reconstruction filter, according to some implementations of the present disclosure.

[0336] In some embodiments, the SAO / CCSAO classification method disclosed herein (including cross-component sample / coded information classification) may function as a filter applied to the reconstructed samples of a tree unit (TU). As shown in FIG. 15C, the CCSAO may function as a post-reconstruction filter. That is, it uses the reconstructed samples (after prediction / residual sample addition, before deblocking) as input to the classification and compensates the luma / chroma samples before entering the neighboring intra prediction. The CCSAO post-reconstruction filter may reduce the distortion of the current TU sample and provide a better prediction for the neighboring intra / inter blocks. With more accurate prediction, better compression efficiency can be expected.

[0337] Encoding Algorithm

[0338] In some embodiments, to efficiently determine the optimal CCSAO parameters for a picture, a hierarchical rate-distortion (RD) optimization algorithm is designed, which includes: 1) an incremental scheme to search for an optimal single classifier; 2) a training process to fine-tune the offset value of one classifier; and 3) a robust algorithm to effectively assign appropriate classifiers to different local regions. A typical CCSAO classifier is as follows: band Y =(Y col N Y )>>BD band U =(U col N U )>>BD band V =(V col N V )>>BD i=band Y ·(N U N V )+band U N V +band V C' rec =Clip1(C rec +σ_ CCSAO [i])) Here, {Y col ,U col ,V col} are the three co-located samples used to classify the current sample, and {N Y ,N U ,N V} are the number of bands applied to the Y, U, and V components, respectively, BD is the coding bit depth, and C rec and C' rec are the reconstructed samples before and after CCSAO is applied, and σ_ CCSAO [i] is the value of the CCSAO offset that applies to the i-th category, and Clip1() clips the input to the range of bit depth, i.e. [0,2 BD,-1], and >> represents a right-shift operation. In the proposed CCSAO, the co-located luma samples can be selected from nine candidate positions, but the co-located chroma samples are fixed.

[0339] Progressive Search Scheme

[0340] In some embodiments, N categories (N Y ,N U ,N V A multi-stage early termination method is applied to search for the best classifier consisting of N categories. If the classifier with fewer categories does not improve the RD cost, the classifier with more categories is skipped. Multiple breakpoints for N-category early stopping are set based on different configurations. For example, AI: every 4 categories (N Y ,N U ,N V <4,8,12…), RA / LB: 16 categories (N Y ,N U ,N V <16,32,48,64...).

[0341] In addition, if Ny is smaller than Nu or Nv, or if all categories N are larger than the threshold, the classifier is skipped. The progressive approach not only adjusts the overall bit cost, but also significantly reduces the coding time. COL The process is repeated for positions to determine the best single classifier.

[0342] Offset value refinement

[0343] In some embodiments, for a given classifier, the reconstructed samples in a picture are first classified according to equation (1). The SAO fast distortion estimation is used to derive an initial offset for each category. Then, it is iteratively estimated by the RD cost of smaller offset values ​​until the value is zero. Then, the CTBs without RD cost improvement are disabled, and the remaining CTBs are retrained to obtain improved offset values. The CTB on-off procedure is repeated until the RD cost of the picture does not improve or a threshold count is reached.

number

[0344] In some embodiments, for one category, k, s(k), x(k) are the sample position, the original sample, and the sample before CCSAO, E is the sum of the differences between s(k) and x(k), N is the number of samples, ΔD is the differential distortion estimated by applying the offset h, ΔJ is the RD cost, λ is the Lagrange multiplier, and R is the bit cost.

[0345] In some embodiments, the original samples can be true original samples (raw image samples without pre-processing) or Motion Compensated Temporal Filter (MCTF) original samples (a classical coding algorithm that pre-processes the original samples before coding). λ is the same as SAO / ALF or weighted by a factor (depending on the configuration / resolution).

[0346] In some embodiments, the encoder optimized the CCSAO by trading off the total RD cost of all categories.

[0347] In some embodiments, the statistical data E and N for each category are stored for each CTB for further determining multiple region classifiers.

[0348] Robust Multiclassifier Assignment

[0349] In some embodiments, to see if the second classifier provides benefits to overall image quality, the CTBs with CCSAO enabled are sorted in ascending order according to distortion (or according to RD cost, including bit cost).

[0350] In some embodiments, the half (or predefined / dependent ratio, e.g. (setNum-1) / setNum-1) of CTBs with smaller distortion keep the same classifier, and the other half CTBs are trained with a new second classifier. Meanwhile, during offset refinement of turning CTBs on and off, each CTB can select its best classifier, so that the good classifiers can be propagated to more CTBs. In the spirit of shuffle and spread, this strategy gives both randomness and robustness to the parameter decision. If there is no further improvement in RD cost with the current number of classifiers, more classifiers are skipped.

[0351] 41 illustrates a computing environment 4110 coupled to a user interface 4150. The computing environment 4110 may be part of a data processing server. The computing environment 4110 includes a processor 4120, a memory 4130, and an input / output (I / O) interface 4140.

[0352] The processor 4120 typically controls the overall operation of the computing environment 4110, such as operations related to display, data acquisition, data communication, and image processing. The processor 4120 may include one or more processors to execute instructions to perform all or a portion of the steps in the methods described above. Additionally, the processor 4120 may include one or more modules that facilitate interaction between the processor 4120 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single chip machine, a graphics processing unit (GPU), or the like.

[0353] The memory 4130 is configured to store various types of data to support the operation of the computing environment 4110. The memory 4130 may include predefined software 4132. Examples of such data include instructions for any application or method operated by the computing environment 4110, video data sets, image data, etc. The memory 4130 may be implemented using any type of volatile or non-volatile memory device, such as, for example, a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk, or a combination thereof.

[0354] The I / O interface 4140 provides an interface between the processor 4120 and a peripheral interface module, such as a keyboard, a click wheel, buttons, including but not limited to a home button, a start scan button, and a stop scan button. The I / O interface 4140 can be coupled to an encoder and a decoder.

[0355] FIG. 42 is a flow diagram illustrating a video decoding method according to an example of this disclosure.

[0356] In step 4201, the processor 4120 may receive a picture frame including one or more components from a video decoder side, where the one or more components may include a first component and a second component.

[0357] In some examples, the first component can include one of a luma component, a first chroma component, or a second chroma component, and the second component can include one of a luma component, a first chroma component, or a second chroma component. The luma component can be a Y component, the first chroma component can be a U component, and the second chroma component can be a V component. The first chroma component and the second chroma component are interchangeable.

[0358] In step 4202, the processor 4120 may determine a classifier for each sample of the second component according to the co-located samples of the first component.

[0359] In some examples, the co-located sample of the first component can be obtained by linear weighting from at least one of a plurality of co-located and adjacent luma samples, or a plurality of linearly weighted samples of a triplet of a luma component, a first chroma component, and a second chroma component.

[0360] In some examples, the co-located sample of the first component can be obtained by linearly weighting samples from multiple co-located and adjacent luma samples, as shown in Figures 20A-20B and 21A-21B. For example, the co-located luma sample value (Y0) can be replaced with a value (Yp) by weighting the co-located and adjacent luma samples, as shown in Figures 20A-20B.

[0361] In some examples, the co-located sample of the first component can be obtained by linearly weighting multiple linearly weighted samples for three components including a luma component, a first chroma component, and a second chroma component. For example, in classifier C7, the co-located / current and adjacent samples of all three color components are used to derive the classification of the current component C0 / C3 bandNum. For example, to classify the current U sample, the co-located and adjacent Y / V, current and adjacent U samples are used as in FIG. 13B.

[0362] In step 4203, the processor 4120 may obtain an indicator indicative of one or more bandNum segments of the one or more components in response to determining that the classifier is a band classifier or a combined classifier that includes a band classifier.

[0363] In some examples, the indicator may be predefined or signaled by the encoder at at least one level. For example, the at least one level may include at least one of a sequence parameter set (SPS) level, an adaptation parameter set (APS) level, a picture parameter set (PPS) level, a picture header (PH) level, a sequence header (SH) level, a region level, a coding tree unit (CTU) level, a sub-block level, or a sample level. If the indicator is predefined, the encoder does not need to design or signal, reducing complexity.

[0364] In some examples, the luma component, the first chroma component, and the second chroma component may use different indicators to indicate their respective bandNum segments, as shown in Table 2-42.

[0365] In some examples, at least two of the luma component, the first chroma component, or the second chroma component use the same indicator to indicate their respective bandNum segments.

[0366] In some examples, a chroma transpose component may be used in the bandNum mapping table, and the chroma transpose component is the transpose component of the first chroma component or the second chroma component. For example, as shown in Table 2-43, C represents the current chroma component, which may be the first chroma component or the second chroma component, and CT represents the chroma transpose component that is the transpose of C, i.e., the other chroma component. For example, when C represents the U component, CT represents the V component, and when C represents the V component, CT represents the U component.

[0367] In some examples, the indicator may indicate one or more bandNum segments of one or more components including at least one of a luma component, a first chroma component, a second chroma component, or a chroma transpose component related to the first chroma component or the second chroma component, as shown in Table 2-42.

[0368] In step 4204, the processor 4120 may determine one or more bandNum segments according to the indicator and then determine a band offset according to the one or more bandNum segments.

[0369] In some examples, the one or more bandNum segments may be determined according to an indicator determined from a mapping table.

[0370] In some examples, the mapping table may be predefined or adjusted / changed by the encoder. For example, the mapping table may be a bandNum mapping table as shown in Table 2-42 and Table 2-43.

[0371] In some instances, where the mapping table is predefined, the mapping table is designed offline and cannot be designed or modified by the encoder, both the encoder and the decoder can have the mapping table, and there is no need to signal the mapping table in the bitstream sent to the decoder. In such a situation, it is possible to reduce complexity by eliminating the need for the encoder / decoder to support different variables, since both the encoder and the decoder do not need to modify the mapping table.

[0372] In one example, the mapping table can be predefined offline, as shown in Table 2-42 or Table 2-43. Both the encoder and the decoder know the mapping table without signaling. By predefining it, no design or signaling is required at the encoder.

[0373] In some other instances, where the mapping table can be changed or adjusted, the encoder can change the mapping table at different levels.

[0374] In step 4205, the processor 4120 can modify each sample of the second component based on a sample offset that includes a band offset.

[0375] In some examples, the sample offset may include a band offset and an edge offset, as previously described.

[0376] FIG. 43 is a flowchart illustrating a video encoding method according to an example of this disclosure.

[0377] In step 4301, the processor 4120 can determine a classifier for each sample of the second component according to the co-located samples of the first component from the video encoder side, and the picture frame can include one or more components including the first component and the second component.

[0378] In some examples, the first component can include one of a luma component, a first chroma component, or a second chroma component, and the second component can include one of a luma component, a first chroma component, or a second chroma component. The luma component can be a Y component, the first chroma component can be a U component, and the second chroma component can be a V component. The first chroma component and the second chroma component are interchangeable.

[0379] In some examples, the co-located sample of the first component can be obtained by linear weighting from at least one of a plurality of co-located and adjacent luma samples, or a plurality of linearly weighted samples of a triplet of a luma component, a first chroma component, and a second chroma component.

[0380] In some examples, the co-located sample of the first component can be obtained by linearly weighting samples from multiple co-located and adjacent luma samples, as shown in Figures 20A-20B and 21A-21B. For example, the co-located luma sample value (Y0) can be replaced with a value (Yp) by weighting the co-located and adjacent luma samples, as shown in Figures 20A-20B.

[0381] In some examples, the co-located sample of the first component can be obtained by linearly weighting multiple linearly weighted samples for the three components consisting of the luma component, the first chroma component, and the second chroma component. For example, in classifier C7, the co-located / current and adjacent samples of all three color components are used to derive the classification of the current component C0 / C3 bandNum. For example, to classify the current U sample, the co-located and adjacent Y / V, current and adjacent U samples are used as in FIG. 13B.

[0382] In step 4302, the processor 4120 may predefine or signal an indicator in the bitstream, and in response to determining that the classifier is a band classifier or a combined classifier that includes a classifier, the indicator indicates one or more bandNum segments of one or more components.

[0383] In some examples, the indicator may be predefined or signaled by the encoder at at least one level, for example, the at least one level may include at least one of a sequence parameter set (SPS) level, an adaptation parameter set (APS) level, a picture parameter set (PPS) level, a picture header (PH) level, a sequence header (SH) level, a region level, a coding tree unit (CTU) level, a sub-block level, or a sample level.

[0384] In some examples, the one or more bandNum segments can be determined according to an indicator determined from a mapping table. In some examples, the processor 4120 can predefine or adjust / change the mapping table. For example, the mapping table can be a bandNum mapping table as shown in Table 2-42 and Table 2-43.

[0385] In some cases, if the mapping table is predefined, both the encoder and the decoder can have the mapping table and there is no need to signal the mapping table in the bitstream sent to the decoder, in which case neither the encoder nor the decoder need to change the mapping table.

[0386] In some other instances, where the mapping table can be changed or adjusted, the encoder can change the mapping table at different levels.

[0387] In some examples, the luma component, the first chroma component, and the second chroma component may use different indicators to indicate their respective bandNum segments, as shown in Table 2-42.

[0388] In some examples, at least two of the luma component, the first chroma component, or the second chroma component use the same indicator to indicate their respective bandNum segments.

[0389] In some examples, a chroma transpose component may be used in the mapping table, and the chroma transpose component is the transpose component of the first chroma component or the second chroma component. For example, as shown in Table 2-43, C represents the current chroma component, which may be the first chroma component or the second chroma component, and CT represents the chroma transpose component that is the transpose of C, i.e., the other chroma component. For example, when C represents the U component, CT represents the V component, and when C represents the V component, CT represents the U component.

[0390] In some examples, the indicator may indicate one or more bandNum segments of one or more components including at least one of a luma component, a first chroma component, a second chroma component, or a chroma transpose component related to the first chroma component or the second chroma component, as shown in Table 2-42.

[0391] In some examples, after receiving a bitstream encoded by an encoder, the video decoder may determine one or more bandNum segments according to the indicator, and then determine a band offset according to the one or more bandNum segments.

[0392] In some examples, the one or more bandNum segments may be determined according to an indicator determined from a mapping table.

[0393] In some examples, the mapping table may be predefined or adjusted / changed by the encoder. For example, the mapping table may be a bandNum mapping table as shown in Table 2-42 and Table 2-43.

[0394] In some examples, the decoder can modify each sample of the second component based on a sample offset that includes a band offset.

[0395] In some embodiments, a non-transitory computer-readable storage medium is also provided that includes, for example in memory 4130, a number of programs executable by the processor 4120 in the computing environment 4110 to perform the above-described methods. In one example, the programs may be executed by the processor 4120 in the computing environment 4110 to receive (e.g., from the video encoder 20 of FIG. 2) a bitstream or data stream including encoded video information (e.g., video blocks representing encoded video frames, and / or one or more associated syntax elements, etc.) and to perform the above-described decoding method according to the received bitstream or data stream. In another example, the programs may be executed by the processor 4120 in the computing environment 4110 to perform the above-described encoding method to encode the video information (e.g., video blocks representing video frames, and / or one or more syntax elements, etc.) into a bitstream or data stream and may be executed by the processor 4120 in the computing environment 4110 to transmit the bitstream or data stream (e.g., to the video decoder 30 of FIG. 3). Alternatively, the non-transitory computer-readable storage medium may store therein a bitstream or data stream including encoded video information (e.g., video information including one or more syntax elements) generated by an encoder (e.g., video encoder 20 of FIG. 2) using the encoding methods described above for use by a decoder (e.g., video decoder 30 of FIG. 3) in decoding video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0396] In one embodiment, a computing device is also provided that includes one or more processors (e.g., processor 4120) and a non-transitory computer readable storage medium or memory 4130 having stored thereon a number of programs executable by the one or more processors, where the one or more processors are configured to execute the number of programs to perform the methods described above.

[0397] In one embodiment, a computer program product is also provided that comprises a number of programs executable by the processor 4120 in the computing environment 4110 to perform the aforementioned methods, e.g., in the memory 4130. For example, the computer program product may include a non-transitory computer-readable storage medium.

[0398] In one embodiment, the computing environment 4110 can be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0399] Further embodiments also include various subsets of the above embodiments combined or otherwise rearranged into various other embodiments.

[0400] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code for execution by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates the transmission of a computer program from one place to another, for example according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the implementations described in this application. A computer program product may include a computer-readable medium.

[0401] The description of the present disclosure has been presented for purposes of illustration and is not intended to be exhaustive or to be limiting of the disclosure. Many modifications, variations, and alternative implementations will be apparent to one skilled in the art having the benefit of the teachings presented in the above descriptions and the associated drawings.

[0402] Unless otherwise specified, the order of steps of the method according to the present disclosure is only intended as an example, and the steps of the method according to the present disclosure are not limited to the order specifically described above, and can be changed according to actual conditions. Moreover, at least one of the steps of the method according to the present disclosure can be adjusted, combined, or deleted according to practical requirements.

[0403] The examples have been chosen and described to explain the principles of the disclosure and to enable others skilled in the art to understand the disclosure in its various embodiments and to best utilize the underlying principles and various embodiments with various modifications as appropriate for the particular applications contemplated. Therefore, it is to be understood that the scope of the disclosure is not limited to the specific examples disclosed, and that modifications and other implementations are intended to be included within the scope of the disclosure.

Claims

1. receiving, by a decoder, a picture frame comprising one or more components including a first component and a second component; determining, by the decoder, a classifier for each sample of the second component according to co-located samples of the first component; obtaining, by the decoder, an indicator indicative of one or more bandNum segments of the one or more components in response to determining whether the classifier is a band classifier or a combined classifier including the band classifier; determining the one or more bandNum segments according to the indicator; determining a band offset according to the one or more bandNum segments; modifying, by the decoder, each sample of the second component according to a sample offset that includes the band offset; A method for decoding a video, comprising:

2. 2. The method of claim 1, further comprising obtaining the one or more bandNum segments according to the indicator determined from a mapping table, the one or more bandNum segments being obtained by dividing a range of sample values ​​of the one or more components.

3. the indicator is predefined or signaled at at least one level by an encoder; 3. The method of claim 2, wherein the at least one level comprises at least one of a sequence parameter set (SPS) level, an adaptive parameter set (APS) level, a picture parameter set (PPS) level, a picture header (PH) level, a sequence header (SH) level, a region level, a coding tree unit (CTU) level, a sub-block level, or a sample level.

4. The method of claim 3 , wherein the mapping table is predefined or adjusted at the at least one level by the encoder.

5. the first component includes one of a luma component, a first chroma component, or a second chroma component; The method of claim 1 , wherein the second component comprises one of the luma component, the first chroma component, or the second chroma component.

6. The method of claim 5 , wherein the luma component, the first chroma component, and the second chroma component use different indicators to indicate their respective bandNum segments.

7. The method of claim 5 , wherein at least two of the luma component, the first chroma component, or the second chroma component use the same indicator to indicate their respective bandNum segments.

8. The method of claim 5 , wherein a chroma transpose component is used in the mapping table, the chroma transpose component being a transpose component of the first chroma component or the second chroma component.

9. in response to determining that the first chroma component is the current component, the chroma transpose component is the second chroma component; The method of claim 8 , wherein in response to determining that the second chroma component is the current component, the chroma transpose component is the first chroma component.

10. 10. The method of claim 9, wherein the indicator indicates the one or more bandNum segments of the one or more components including at least one of the luma component, the first chroma component, the second chroma component, or the chroma transpose component related to the first chroma component or the second chroma component.

11. The co-located samples of the first component are Multiple co-located and adjacent luma samples, or a plurality of linearly weighted samples for three components consisting of a luma component, a first chroma component, and a second chroma component; The method of claim 1 , wherein the signal is obtained by linearly weighting samples from at least one of:

12. determining, by an encoder, a classifier for each sample of a second component according to a co-located sample of a first component, wherein a picture frame is composed of one or more components including the first component and the second component; predefining or signaling, by the encoder, an indicator in the bitstream in response to determining whether the classifier is a band classifier or a combined classifier that includes the band classifier, the indicator indicating one or more bandNum segments of the one or more components; 23. A method for encoding a video comprising:

13. determining the one or more bandNum segments according to the indicator determined from a mapping table; The method of claim 12 , wherein the one or more bandNum segments are obtained by dividing a range of sample values ​​of the one or more components.

14. the indicator is predefined or signaled at at least one level by an encoder; 14. The method of claim 13, wherein the at least one level comprises at least one of a sequence parameter set (SPS) level, an adaptive parameter set (APS) level, a picture parameter set (PPS) level, a picture header (PH) level, a sequence header (SH) level, a region level, a coding tree unit (CTU) level, a sub-block level, or a sample level.

15. The method of claim 14 , further comprising predefining the mapping table or adjusting the mapping table at the at least one level by the encoder.

16. the first component includes one of a luma component, a first chroma component, or a second chroma component; The method of claim 12 , wherein the second component comprises one of the luma component, the first chroma component, or the second chroma component.

17. The method of claim 16 , wherein the luma component, the first chroma component, and the second chroma component use different indicators to indicate their respective bandNum segments.

18. 17. The method of claim 16, wherein at least two of the luma component, the first chroma component, or the second chroma component use a same indicator to indicate a respective bandNum segment.

19. The method of claim 16 , wherein a chroma transpose component is used in the mapping table, the chroma transpose component being a transpose of the first chroma component or the second chroma component.

20. in response to determining that the first chroma component is a current component, the chroma transpose component is the second chroma component; 20. The method of claim 19, wherein in response to determining that the second chroma component is a current component, the chroma transpose component is the first chroma component.

21. 21. The method of claim 20, wherein the indicator indicates the one or more bandNum segments of the one or more components including at least one of the luma component, the first chroma component, the second chroma component, or the chroma transpose component related to the first chroma component or the second chroma component.

22. The co-located samples of the first component are A number of co-located and adjacent luma samples, or a number of linearly weighted samples from three components: a luma component, a first chroma component, and a second chroma component; The method of claim 12 , wherein the signal is obtained by linearly weighting samples from at least one of:

23. one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; Equipped with The one or more processors are configured to, upon execution of the instructions, perform the method of any one of claims 1 to 11. Video Decoding Device.

24. one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; Equipped with The one or more processors are configured to perform the method according to any one of claims 12 to 22 upon execution of instructions. Video encoding device.

25. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to any one of claims 1 to 11 based on the bitstream.

26. A non-transitory computer-readable storage medium for storing computer-executable instructions which, when executed by one or more computer processors, cause the one or more computer processors to perform the method of any one of claims 12 to 22 and to transmit the bitstream.