Enhanced coding in inter-component sample adaptation offsets

Abstract

An electronic device performs a method for decoding a video signal, the method including: receiving from the video signal a picture frame including a first component and a second component; selecting an edge direction in an array sample of the first component or a current sample of the second component for each sample of the second component; calculating a first edge strength by determining a difference in values ​​of two samples from a group consisting of the arrayed or current sample, a first adjacent sample, and a second adjacent sample; quantizing the first edge strength into M segments according to M-1 thresholds; determining a classifier for each sample of the second component based on the M segments; determining a sample offset for each sample of the second component according to the classifier; and modifying each sample of the second component based on the determined sample offset.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to U.S. Provisional Patent Application No. 63 / 241,917, filed on September 8, 2021, entitled “CROSS-COMPONENT SAMPLE ADAPTIVE OFFSET”, and U.S. Provisional Patent Application No. 63 / 248,895, filed on September 27, 2021, entitled “CROSS-COMPONENT SAMPLE ADAPTIVE OFFSET”, all of which are incorporated herein by reference in their entirety. This application relates in general to video coding and compression, and more particularly to methods and apparatus for improving the coding efficiency of both luminous and chroma. [Background technology]

[0002] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video gaming consoles, smartphones, video teleconferencing devices, video streaming devices, etc. The electronic devices transmit and receive digital video data via a communication network or communicate in other ways, and / or store digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited memory resources of the storage device, video coding may be used to compress the video data according to one or more video coding standards before the video data is communicated or stored. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), Moving Picture Expert Group (MPEG) coding, etc. AOMedia Video 1 (AV1) was developed as a successor to the previous standard VP9. Audio Video Coding (AVS) refers to digital audio and digital video compression standards and is another series of video compression standards. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit the redundancy inherent in video data. The purpose of video coding is to compress the video data in a form that avoids or minimizes the degradation of video quality while using a lower bit rate. Summary of the Invention Problems to be Solved by the Invention

[0003] This application relates to an implementation example of a method and apparatus for improving the encoding efficiency of both a luma component and a chroma component, including encoding and decoding video data, and more particularly, improving the encoding efficiency by exploring the inter-component relationship between the luma component and the chroma component. **Means for Solving the Problem**

[0004] According to a first aspect of the present application, a method for decoding a video signal includes receiving, from the video signal, a picture frame including a first component and a second component; determining a classifier for each sample of the second component according to the array samples of the first component or the edge direction and one or more edge strengths of the current sample of the second component for each sample of the second component; determining a sample offset for each sample of the second component according to the classifier; and modifying each sample of the second component based on the determined sample offset. In some embodiments, determining a classifier for each sample of the second component according to the edge direction and one or more edge strengths includes selecting an edge direction in the arrayed or current sample; identifying the first adjacent sample and the second adjacent sample of the arrayed or current sample; calculating a first edge strength by determining the difference in the values of two samples from a group consisting of the arrayed or current sample, the first adjacent sample, and the second adjacent sample; quantizing the first edge strength into M segments according to M-1 threshold values T(1),...T(M-1), where M is a positive integer; and determining a classifier based on the M segments.

[0005] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. The programs cause the electronic device to execute the method for encoding the video signal described above when executed by the one or more processing units.

[0006] According to a third aspect of this application, a non-temporary computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. When executed by one or more processing units, the programs cause the electronic device to perform the method of encoding the video signals described above.

[0007] According to a fourth aspect of this application, a computer-readable storage medium stores a bitstream containing instructions, and when executed, the instructions cause a decoding device to perform the method of decoding the video signal described above.

[0008] Please understand that both the above general description and the following detailed description are merely examples and do not limit this disclosure.

[0009] The accompanying drawings are incorporated herein and constitute part of this specification, providing consistent examples and illustrating the principles of this disclosure together with this description. [Brief explanation of the drawing]

[0010] [Figure 1] This block diagram shows an exemplary system for video block coding and decoding relating to several implementation examples of this disclosure. [Figure 2] A block diagram showing exemplary video encoders relating to some implementation examples of this disclosure. [Figure 3] A block diagram showing exemplary video decoders relating to some implementation examples of this disclosure. [Figure 4A] The block diagram shows how a frame is recursively divided into multiple video blocks of different sizes and shapes by some implementation examples of this disclosure. [Figure 4B] The block diagram shows how a frame is recursively divided into multiple video blocks of different sizes and shapes by some implementation examples of this disclosure. [Figure 4C]The block diagram shows how a frame is recursively divided into multiple video blocks of different sizes and shapes by some implementation examples of this disclosure. [Figure 4D] The block diagram shows how a frame is recursively divided into multiple video blocks of different sizes and shapes by some implementation examples of this disclosure. [Figure 4E] The block diagram shows how a frame is recursively divided into multiple video blocks of different sizes and shapes by some implementation examples of this disclosure. [Figure 4F] An intra-mode block defined in VVC. [Figure 4G] This block contains multiple reference lines for intra-prediction. [Figure 5A] This is a block diagram illustrating the four gradient patterns used in sample adaptive offsets (SAOs) in some implementation examples of this disclosure. [Figure 5B] This is a block diagram illustrating the naming conventions for samples surrounding a central sample in several implementation examples of this disclosure. [Figure 6A] This block diagram shows a CCSAO system and process applied to a chroma sample by several implementation examples of this disclosure, using DBF Y as input. [Figure 6B] This block diagram shows a CCSAO system and process that can be applied to luma and chroma samples by some implementation examples of this disclosure, and which uses DBF Y / Cb / Cr as input. [Figure 6C] This block diagram shows a CCSAO system and process that can function independently through several implementation examples of this disclosure. [Figure 6D] This block diagram shows a CCSAO system and process that can be recursively applied (2 or N times) at the same or different offsets by several implementation examples of the present disclosure. [Figure 6E]This block diagram shows CCSAO systems and processes applied in parallel with the enhanced sample adaptive offset (ESAO) in the AVS standard, as demonstrated by several implementation examples of this disclosure. [Figure 6F] This block diagram shows CCSAO systems and processes applied after SAO through several implementation examples of this disclosure. [Figure 6G] This block diagram shows that several implementation examples of this disclosure demonstrate that CCSAO systems and processes can function independently without CCALF. [Figure 6H] This block diagram shows CCSAO systems and processes applied in parallel to intercomponent adaptive loop filters (CCALF) by several implementation examples of the present disclosure. [Figure 6I] This block diagram shows the CCSAO system and processes applied in parallel to SAO and BIF by some implementation examples of this disclosure. [Figure 6J] This block diagram shows a CCSAO system and process applied in parallel to BIF by replacing SAO with some implementation examples of this disclosure. [Figure 7] This block diagram shows a sample process using CCSAO related to some implementation examples of this disclosure. [Figure 8] This block diagram shows that some implementation examples of this disclosure interleave the CCSAO process with vertical and horizontal deblocking filters (DBFs). [Figure 9] This flowchart illustrates an exemplary process for decoding a video signal using inter-component correlations relating to several implementation examples of this disclosure. [Figure 10A] This is a block diagram showing a classifier that uses different luma (or chroma) sample positions for C0 classification, relating to some implementation examples of this disclosure. [Figure 10B] This figure shows some examples of different shapes for Luma candidates relating to some implementation examples of this disclosure. [Figure 11]This is a block diagram of a sample process demonstrating that several implementation examples of this disclosure can supply all sequence and adjacent luma / chroma samples for CCSAO classification. [Figure 12A] The figure shows an exemplary classifier obtained by replacing the sequence luma sample values ​​with values ​​obtained by weighting the sequence and adjacent luma samples, as shown in some implementation examples of this disclosure. [Figure 12B] This figure shows subsample Laplacian calculations relating to several implementation examples of this disclosure. [Figure 12C] This figure shows an example of using inter-component / current-component edge information for the classification of current components in some implementation examples of this disclosure. [Figure 13] The block diagram shows that some implementation examples of this disclosure apply CCSAO and other in-loop filters have different clipping combinations. [Figure 14A] The block diagram shows that CCSAO is not applied to the current chroma (luma) sample if either the sequence used for classification or the adjacent luma (chroma) sample is outside the current picture, as demonstrated by some implementation examples of this disclosure. [Figure 14B] Block diagrams show that some implementation examples of this disclosure demonstrate that CCSAO is applied to the current luma or chroma sample when either the sequence used for classification or the adjacent luma or chroma sample is outside the current picture. [Figure 14C] A block diagram shows that CCSAO is not applied to the current chroma sample when the corresponding selected sequence or adjacent chroma sample used for classification lies outside the virtual space defined by the virtual boundary (VB), as illustrated by some implementation examples of this disclosure. [Figure 15] The figures show that some implementation examples of this disclosure apply iteration or mirror padding to luma samples outside the virtual boundary. [Figure 16]The figure shows that some implementation examples of this disclosure require an additional luma line buffer when all nine sequences and adjacent luma samples are used for classification. [Figure 17] This figure shows that in AVS, the nine luma candidate CCSAOs intersecting VB can be increased by only two additional luma line buffers, based on some implementation examples of this disclosure. [Figure 18A] This figure shows that in some implementation examples of this disclosure, the nine luma candidate CCSAOs intersecting VB in VVC can be increased by only one additional luma line buffer. [Figure 18B] Some implementation examples of this disclosure show that when an array or adjacent chroma sample is used to classify the current chroma sample, the selected chroma candidate may cross the VB and require an additional chroma line buffer. [Figure 19A] The figure shows that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the chroma sample candidates cross the VB (are outside the current chroma sample VB) as shown in some implementation examples of this disclosure. [Figure 19B] The figure shows that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the chroma sample candidates cross the VB (are outside the current chroma sample VB) as shown in some implementation examples of this disclosure. [Figure 19C] The figure shows that in AVS and VVC, CCSAO is disabled for a chroma sample if any of the chroma sample candidates cross the VB (are outside the current chroma sample VB) as shown in some implementation examples of this disclosure. [Figure 20A] The diagram shows that in AVS and VVC, CCSAO is enabled using iterative padding on the chroma sample if any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 20B] The diagram shows that in AVS and VVC, CCSAO is enabled using iterative padding on the chroma sample if any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 20C] The diagram shows that in AVS and VVC, CCSAO is enabled using iterative padding on the chroma sample if any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 21A] The diagram shows that in AVS and VVC, CCSAO is enabled using mirror padding on the chroma sample when any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 21B] The diagram shows that in AVS and VVC, CCSAO is enabled using mirror padding on the chroma sample when any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 21C] The diagram shows that in AVS and VVC, CCSAO is enabled using mirror padding on the chroma sample when any of the chroma candidate chroma samples cross the VB (is outside the current chroma sample VB). [Figure 22A] The figures show that some implementation examples of this disclosure enable CCSAO using double-sided symmetrical padding on different CCSAO sample shapes. [Figure 22B] The figures show that some implementation examples of this disclosure enable CCSAO using double-sided symmetrical padding on different CCSAO sample shapes. [Figure 23] This figure shows the limitation of using a limited number of Luma candidates for classification in some implementation examples of this disclosure. [Figure 24] This figure shows that some implementation examples of this disclosure demonstrate that the CCSAO application region is not aligned with the coding tree block (CTB) / coding tree unit (CTU) boundary. [Figure 25] This figure shows that the CCSAO application domain frame division can be fixed by CCSAO parameters, based on several implementation examples of this disclosure. [Figure 26] This figure shows that the CCSAO application domain may be a binary tree (BT), quadrant (QT), or ternary tree (TT) partitioned from the frame / slice / CTB level, as illustrated by several implementation examples of this disclosure. [Figure 27] This is a block diagram showing multiple classifiers used and switched at different levels within a picture frame, relating to some implementation examples of this disclosure. [Figure 28] This block diagram shows that some implementation examples of this disclosure demonstrate that the CCSAO application area divisions are dynamic and can be switched at the picture level. [Figure 29] This figure shows that some implementation examples of this disclosure allow the CCSAO classifier to take into account currently or intercomponent coding information. [Figure 30] This block diagram shows that some implementation examples of this disclosure demonstrate that the SAO classification method disclosed in this disclosure acts as a post-prediction filter. [Figure 31] The block diagram shows that, in some implementation examples of this disclosure, each component can be used to classify the current sample and adjacent samples for post-predictive SAO filtering. [Figure 32] This block diagram shows that some implementation examples of this disclosure demonstrate that the SAO classification method disclosed in this disclosure acts as a reconstructed filter. [Figure 33] This flowchart illustrates an exemplary process for decoding a video signal using inter-component correlations relating to several implementation examples of this disclosure. [Figure 34] This figure shows a computing environment coupled to a user interface relating to several implementation examples of this disclosure. [Modes for carrying out the invention]

[0011] Next, we will refer in detail to specific implementation examples, which are illustrated in the attached drawings. In the following detailed description, numerous non-limiting specific details will be mentioned to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternative means may be used without departing from the claims, and that the subject matter may be implemented without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented in many types of electronic devices having digital video capabilities.

[0012] It should be noted that terms such as “first,” “second,” etc., used in this description, the claims of this disclosure, and the accompanying drawings are used to distinguish subjects and are not used to describe any particular order or sequence. The data used in this manner may be replaced under appropriate conditions, and it should be understood that the embodiments of this disclosure described herein may be implemented in an order other than that illustrated in the accompanying drawings or described herein.

[0013] The first generation of AVS standards includes the Republic of China national standards "Information Technology, Advanced Audio Video Coding, Part 2: Video" (known as AVS1) and "Information Technology, Advanced Audio Video Coding Part 16: Radio Television Video" (known as AVS+). Compared to the MPEG-2 standard, it can offer approximately 50% bitrate savings at the same perceived quality. The second generation of AVS standards includes a series of Republic of China national standards "Information Technology, Efficient Multimedia Coding" (known as AVS2), which primarily targets the transmission of additional HD TV programs. The encoding efficiency of AVS2 is twice that of AVS+. Meanwhile, the video portion of the AVS2 standard was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as a single international standard for application. The AVS3 standard is a new generation video coding standard for UHD video that aims to surpass the coding efficiency of the latest international standard, HEVC, offering approximately 30% bit rate savings compared to the HEVC standard. In March 2019, at the 68th AVS Conference, the AVS3-P2 baseline was completed, which also offers approximately 30% bit rate savings compared to the HEVC standard. Currently, one standard software called the High Performance Model (HPM) is maintained by the AVS Group to demonstrate the standard implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework.

[0014] Figure 1 is a block diagram showing an exemplary system 10 for parallel encoding and decoding of video blocks according to some implementations of the present disclosure. As shown in Figure 1, the system 10 includes a source device 12, which generates video data and encodes it to be later decoded by a destination device 14. The source device 12 and destination device 14 can constitute any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video gaming consoles, and video streaming devices. In some implementations, the source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0015] In some implementations, the destination device 14 can receive encoded video data to be decoded via link 16. Link 16 can comprise any type of communication medium or device capable of moving the encoded video data from source device 12 to destination device 14. In one example, link 16 may comprise a communication medium to enable source device 12 to transmit the encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium can include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, wide area network, or global network, such as the Internet. The communication medium can include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 12 to destination device 14.

[0016] In some other implementation examples, encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 can then be accessed by destination device 14 via input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media for storing the encoded video data, such as a hard drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium. In further examples, storage device 32 may correspond to a file server or another intermediate storage device capable of holding the encoded video data generated by source device 12. Destination device 14 can access the stored video data from storage device 32 via streaming or download. The file server can be any type of computer capable of storing the encoded video data and transmitting the encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), File Transfer Protocol (FTP) servers, network-attached storage (NAS) devices, or local disk drives. The destination device 14 can access the encoded video data stored on the file server via any standard data connection, including a wireless channel suitable for accessing the encoded video data (e.g., Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., Digital Subscriber Line (DSL), cable modem, etc.), or a combination of both. Transmission of the encoded video data from the storage device 32 can be via streaming transmission, download transmission, or a combination of both.

[0017] As shown in Figure 1, the source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include a source such as a video capture device, e.g., a video camera, a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. For example, if the video source 18 is a video camera in a security surveillance system, the source device 12 and the destination device 14 can form a camera phone or videophone. However, the implementation examples described in this application may apply to general video coding and may be applicable to wireless and / or wired applications.

[0018] Captured, pre-captured, or computer-generated video can be encoded by the video encoder 20. The encoded video data can be transmitted directly to the destination device 14 via the output interface 22 of the source device 12. The encoded video data can also (or otherwise) be stored on the storage device 32 for later access by the destination device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or transmitter.

[0019] The destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. The input interface 28 may include a receiver and / or modem that can receive encoded video data via link 16. The encoded video data communicated via link 16 or provided on the storage device 32 may include various syntactic elements generated by the video encoder 20 for use by the video decoder 30 when decoding the video data. Such syntactic elements may be contained within the encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0020] In some implementations, the destination device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the destination device 14. The display device 34 displays the decoded video data to the user and can consist of any of a variety of display devices, such as a liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

[0021] The video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards such as VVC, HEVC, MPEG-4, Part 10, AVC, AVS, or extensions of such standards. It should be understood that this application is not limited to specific video encoding / decoding standards, but may apply to other video encoding / decoding standards. Generally, the video encoder 20 of the source device 12 is intended to be configured to encode video data in accordance with any of these current or future standards. Similarly, generally, the video decoder 30 of the destination device 14 is intended to be configured to decode video data in accordance with any of these current or future standards.

[0022] The video encoder 20 and video decoder 30 can each be implemented as one or more suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device can store instructions for the software in a suitable non-temporary computer-readable medium and use one or more processors to execute these instructions in hardware to perform the video encoding / decoding operations disclosed herein. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, which can each be integrated as part of a composite encoder / decoder (CODEC) in their respective devices.

[0023] Figure 2 is a block diagram showing an exemplary video encoder 20 relating to some implementation examples described in this application. The video encoder 20 can perform intra and interpredictive coding of video blocks within a video frame. Intrapredictive coding relies on spatial prediction to reduce or eliminate spatial redundancy of video data within a given video frame or picture. Interpredictive coding relies on temporal prediction to reduce or eliminate temporal redundancy of video data within adjacent video frames or pictures in a video sequence. Note that the term “frame” may be used as a synonym for the terms “image” or “picture” in the field of video coding.

[0024] As shown in Figure 2, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-prediction processing unit 46, and an intra-block copy (BC) unit 48. In some implementation examples, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for reconstructing video blocks. An in-loop filter 63, such as a deblocking filter, may be placed between the adder 62 and the DPB 64 to filter block boundaries and remove block rate artifacts from the reconstructed video. In addition to the deblocking filter for filtering the output of the adder 62, another in-loop filter, such as a sample-adaptive offset (SAO) filter and / or an adaptive in-loop filter (ALF), may be used. In some examples, the in-loop filter may be omitted, and the decoded video block may be provided directly to the DPB64 by the adder 62. The video encoder 20 may take the form of a fixed or programmable hardware unit, or may be divided among one or more of the fixed or programmable hardware units shown.

[0025] The video data memory 40 can store video data to be encoded by the components of the video encoder 20. As shown in Figure 1, the video data in the video data memory 40 may be obtained from, for example, the video source 18. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use by the video encoder 20 (e.g., intra or interpredictive coding mode) when encoding video data. The video data memory 40 and DPB 64 may be formed by any of various memory devices. In various examples, the video data memory 40 may be on-chip with the other components of the video encoder 20 or off-chip with respect to those components.

[0026] As shown in Figure 2, after receiving video data, the partitioning unit 45 within the prediction processing unit 41 divides the video data into video blocks. This partitioning may also involve dividing the video frame into slices, tiles (e.g., several sets of video blocks), or other larger coding units (CUs) according to a predefined partitioning structure, such as a quadtree (QT) structure associated with the video data. A video frame is, or may be considered as, a two-dimensional array or matrix of samples having sample values. Samples in the array may also be called pixels or pels. The number of samples in the horizontal and vertical (or axis) directions of the array or picture defines the size and / or resolution of the video frame. A video frame may be divided into multiple video blocks, for example by using a QT partition. In this case as well, a video block is, or may be considered as, a two-dimensional array or matrix of samples having sample values, but with smaller dimensions than a video frame. The number of samples in the horizontal and vertical (or axis) directions of the video block defines the size of the video block. A video block may be further divided into one or more block segments or subblocks (which may form blocks again) by iteratively using, for example, QT partitioning, binary tree (BT) partitioning, or ternary tree (TT) partitioning, or any combination thereof. Note that in this specification, the terms “block” or “video block” can refer to a portion of a frame or picture, in particular a rectangular (square or non-square) portion. Referring to HEVC and VVC, for example, a block or video block may be or correspond to an Encoded Tree Unit (CTU), CU, Prediction Unit (PU), or Translating Unit (TU), and / or corresponding blocks, for example an Encoded Tree Block (CTB), Coding Block (CB), Prediction Block (PB), or Translating Block (TB), and / or corresponding to a subblock.

[0027] The prediction processing unit 41 can select one of several possible prediction coding modes for the current video block, such as one of several intra-predictive coding modes or one of several inter-predictive coding modes, based on the error results (e.g., coding rate and distortion level). The prediction processing unit 41 can provide the resulting intra- or inter-predictive coded block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coded block for later use as part of a reference frame. The prediction processing unit 41 also provides syntactic elements such as motion vectors, intra-mode indicators, piecewise information, and other such syntactic information to the entropy coding unit 56.

[0028] To select an appropriate intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 may perform intra-predictive coding of the current video block for one or more adjacent blocks in the same frame as the current block to be coded, in order to provide spatial predictions. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 may perform inter-predictive coding of the current video block for one or more prediction blocks in one or more reference frames, in order to provide temporal predictions. The video encoder 20 may perform multiple coding passes to select an appropriate coding mode for each block of video data, for example.

[0029] In some implementations, the motion estimation unit 42 determines the inter-prediction mode for the current video frame by generating motion vectors that show the displacement of the video block in the current video frame relative to the predicted block in the reference video frame, according to a predetermined pattern in the sequence of video frames. Motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors and estimating motion relative to the video block. The motion vectors can show, for example, the displacement of the video block in the current video frame or picture relative to the predicted block in the reference frame relative to the current block encoded in the current frame. The predetermined pattern can specify the video frames in the sequence as P-frames or B-frames. The intra-BC unit 48 can determine vectors, such as block vectors, for intra-BC coding, similar to the determination of motion vectors by the motion estimation unit 42 for inter-prediction, or can utilize the motion estimation unit 42 to determine block vectors.

[0030] The prediction block for a video block can be, or correspond to, a block in the reference frame that is considered to closely match the video block to be encoded in terms of pixel difference, and this can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can calculate values ​​for pixel positions of the reference frame stored in the DPB64 that are less than an integer. For example, the video encoder 20 can interpolate values ​​for pixel positions of one-quarter, one-eighth, or other fractional values ​​in the reference frame. Thus, the motion estimation unit 42 can perform motion searches for complete and fractional pixel positions and output motion vectors with fractional pixel precision.

[0031] The motion estimation unit 42 calculates motion vectors for video blocks in the interpredictive coded frame by comparing the position of a video block with the position of a predicted block in a reference frame selected from a first reference frame list (List0) or a second reference frame list (List1), each of which identifies one or more reference frames stored in the DPB64. The motion estimation unit 42 transmits the calculated motion vectors to the motion compensation unit 44, and then to the entropy coding unit 56.

[0032] Motion compensation performed by the motion compensation unit 44 may involve fetching or generating a prediction block based on the motion vector determined by the motion estimation unit 42. Upon receiving the motion vector for the current video block, the motion compensation unit 44 may locate the position of the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve this prediction block from the DPB 64, and transfer this prediction block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include luma or chroma difference components or both. The motion compensation unit 44 may also generate syntactic elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. Syntactic elements may include, for example, syntactic elements defining the motion vector used to identify the prediction block, any flags indicating the prediction mode, or any other syntactic information described herein. Note that the motion estimation unit 42 and the motion compensation unit 44 can be highly integrated, but are shown separately for conceptual purposes.

[0033] In some implementations, the intraBC unit 48 can generate vectors and fetch predicted blocks, similar to those described above in relation to the motion estimation unit 42 and the motion compensation unit 44, but the predicted blocks are in the same frame as the currently encoded block, and the vectors are called block vectors rather than motion vectors. In particular, the intraBC unit 48 can determine the intraprediction mode to use to encode the current block. In some examples, the intraBC unit 48 can encode the current block using various intraprediction modes, for example, during separate encoding passes, and their performance can be tested by velocity distortion analysis. The intraBC unit 48 can then select an appropriate intraprediction mode from among the various tested intraprediction modes and use and generate an intramode indicator accordingly. For example, the intraBC unit 48 can calculate velocity distortion values ​​using velocity distortion analysis for various tested intraprediction modes, and select and use the intraprediction mode with the best velocity distortion characteristics from among the tested modes as the appropriate intraprediction mode. Velocity distortion analysis generally determines the amount of distortion (or error) between an encoded block and the original unencoded block that was encoded to produce the encoded block, as well as the bit velocity (i.e., the number of bits) used to produce the encoded block. The intraBC unit 48 can calculate ratios of distortion and velocity for various encoded blocks to determine which intraprediction mode exhibits the best velocity distortion value for that block.

[0034] In other examples, the intra-BC unit 48 may perform such functions for intra-BC prediction by using the motion estimation unit 42 and the motion compensation unit 44 in whole or in part, as described in the implementation examples herein. In any case, for intra-block copying, the predicted block may be a block that is considered to be closely aligned with the block to be encoded with respect to the pixel difference, which may be determined by SAD, SSD, or other difference metrics, and the identification of the predicted block may include the calculation of values ​​for pixel positions less than an integer.

[0035] Regardless of whether the predicted block is from the same frame according to intra-prediction or from different frames according to inter-prediction, the video encoder 20 can form a residual video block by forming a pixel difference value by subtracting the pixel value of the predicted block from the pixel value of the currently encoded video block. The pixel difference value that forms the residual video block may include differences in both luma and chroma components.

[0036] As described above, the intra-prediction processing unit 46 can intra-predict the current video block as an alternative to inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, or to intra-block copy prediction performed by the intra-BC unit 48. In particular, the intra-prediction processing unit 46 can determine the intra-prediction mode to use to encode the current block. In this case, the intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, in separate encoding passes, and the intra-prediction processing unit 46 (or, in some examples, the mode selection unit) can select and use an appropriate intra-prediction mode from the tested intra-prediction modes. The intra-prediction processing unit 46 can provide the entropy encoding unit 56 with information indicating the selected intra-prediction mode for that block. The entropy encoding unit 56 can encode information indicating the selected intra-prediction mode in the bitstream.

[0037] After the prediction processing unit 41 determines the predicted block for the current video block via interpretation or intraprediction, the adder 50 forms the residual video block by subtracting the predicted block from the current video block. The residual video data in the residual block may be contained in one or more TUs and is provided to the transformation processing unit 52. The transformation processing unit 52 transforms the residual video data into residual transformation coefficients using a transformation such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0038] The conversion processing unit 52 can transmit the resulting conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0039] Following quantization, the entropy coding unit 56 entropy-codes the quantized transformation coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), stochastic interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The coded bitstream may then be transmitted to the video decoder 30, as shown in Figure 1, or stored in the storage device 32, as shown in Figure 1, for later transmission to or retrieval by the video decoder 30. The entropy coding unit 56 can also entropy-code motion vectors and other syntactic elements for the current video frame being coded.

[0040] The inverse quantization unit 58 and the inverse transformation processing unit 60 apply inverse quantization and inverse transformation, respectively, to reconstruct residual video blocks within the pixel domain and generate reference blocks for predicting other video blocks. As described above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to calculate pixel values ​​less than integers for use in motion estimation.

[0041] The adder 62 adds the reconstructed residual block to the motion compensation prediction block provided by the motion compensation unit 44 to produce a reference block for storage in the DPB 64. The reference block can then be used by the intraBC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block for interpreting another video block in a later video frame.

[0042] Figure 3 is a block diagram showing an exemplary video decoder 30 relating to some implementation examples of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transformation processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-prediction unit 84, and an intra-BC unit 85. The video decoder 30 can perform a decoding process, which is generally in the reverse direction of the encoding process described above with respect to the video encoder 20 in relation to Figure 2. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, and the intra-prediction unit 84 can generate prediction data based on an intra-prediction mode indicator received from the entropy decoding unit 80.

[0043] In some examples, a unit of the video decoder 30 may be given the task of performing the implementation of the present invention. Also, in some examples, the implementation of the present disclosure may be divided among one or more units of the video decoder 30. For example, the intraBC unit 85 can perform the implementation of the present invention alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intraprediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intraBC unit 85, and the functions of the intraBC unit 85 may be performed by other components of the predictive processing unit 81, such as the motion compensation unit 82.

[0044] The video data memory 79 can store video data, such as an encoded video bitstream, so that it can be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 can be retrieved, for example, from a storage device 32, from a local video source such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include an encoded picture buffer (CPB) that stores the encoded video data from the encoded video bitstream. The DPB 92 of the video decoder 30 stores reference video data for use by the video decoder 30 (e.g., in intra or interpredictive coding mode) when decoding the video data. The video data memory 79 and DPB 92 may be formed by various memory devices such as dynamic random-access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data memory 79 and DPB92 are depicted in Figure 3 as two separate components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB92 may be provided by the same memory device or by separate memory devices. In some examples, the video data memory 79 may be on-chip with the other components of the video decoder 30 or off-chip with respect to those components.

[0045] During the decoding process, the video decoder 30 receives an encoded video bitstream representing the video blocks and associated syntactic elements of the encoded video frames. The video decoder 30 can receive syntactic elements at the video frame level and / or video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra-predictive mode indicators, and other syntactic elements. The entropy decoding unit 80 then transfers the motion vectors or intra-predictive mode indicators and other syntactic elements to the prediction processing unit 81.

[0046] When a video frame is encoded as an intra-predictive coding (I) frame, or encoded for an intra-coded prediction block in another type of frame, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0047] When a video frame is encoded as an interpredictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 yields one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntactic elements received from the entropy decoding unit 80. Each prediction block may be derived from one of the reference frames in the reference frame list. The video decoder 30 may construct reference frame lists List0 and List1 using a default construction technique based on the reference frames stored in the DPB92.

[0048] In some examples, when a video block is encoded according to the intraBC mode described herein, the intraBC unit 85 of the prediction processing unit 81 yields a predicted block for the current video block based on the block vector and other syntactic elements received from the entropy decoding unit 80. The predicted block may be located within the reconstructed region of the same picture as the current video block defined by the video encoder 20.

[0049] The motion compensation unit 82 and / or intra-BC unit 85 determine prediction information for the video block of the current video frame by parsing motion vectors and other syntactic elements, and then use the prediction information to obtain a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntactic elements to determine the prediction mode used to encode the video block of the video frame (e.g., intra or inter-predict), the inter-predict frame type (e.g., B or P), construction information for one or more of the frame's reference frame lists, motion vectors for each inter-predict coded video block of the frame, the inter-predict state for each inter-predict coded video block of the frame, and other information for decoding the video block in the current video frame.

[0050] Similarly, the intraBC unit 85 can use some of the received syntactic elements, such as flags, to determine that the current video block was predicted using intraBC mode, construction information of the video blocks of the frame located within the reconstructed region and to be stored in the DPB 92, block vectors for each intraBC predicted video block of the frame, intraBC prediction state for each intraBC predicted video block of the frame, and other information for decoding the video blocks in the current video frame.

[0051] The motion compensation unit 82 can also perform interpolation using interpolation filters used by the video encoder 20 during video block encoding to calculate interpolation values ​​for pixels of a reference block that are less than an integer. In this case, the motion compensation unit 82 can determine from the received syntactic elements which interpolation filters are used by the video encoder 20 and use those interpolation filters to produce a predicted block.

[0052] The inverse quantization unit 86 inverse quantizes the quantized transformation coefficients entropy-decoded by the entropy decoding unit 80 provided in the bitstream, using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transformation processing unit 88 reconstructs the residual blocks in the pixel domain by applying an inverse transformation, such as an inverse DCT, inverse integer transformation, or a conceptually similar inverse transformation process, to the transformation coefficients.

[0053] After the motion compensation unit 82 or intraBC unit 85 generates a prediction block for the current video block based on vectors and other syntactic elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse processing unit 88 with the corresponding prediction block generated by the motion compensation unit 82 and intraBC unit 85. For further processing of the decoded video block, an in-loop filter 91 such as a deblocking filter, SAO filter, and / or ALF may be placed between the adder 90 and the DPB 92. The in-loop filter 91 may be applied on the reconstructed CU before being placed in the reference picture store. In some examples, the in-loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video block in a given frame is then stored in the DPB 92, which stores a reference frame to be used for motion compensation after the next video block. DPB92, or a separate memory device from DPB92, may also store the decoded video for later presentation on a display device such as the display device 34 in Figure 1.

[0054] In a typical video coding process, a video sequence typically contains an ordered set of frames or pictures. Each frame can contain three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, a frame may be monochromatic and therefore contain only one two-dimensional array of luma samples.

[0055] Like HEVC, the AVS3 standard is built on a block-based hybrid video coding framework. The input video signal is processed block by block (called coding units (CUs)). Unlike HEVC, which partitions blocks based solely on quadtrees, in AVS3, a single coding tree unit (CTU) is partitioned into CUs based on quadtrees / binary trees / extended quadtrees to adapt to varying local characteristics. In addition, the concept of multiple partitioned unit types in HEVC is eliminated; that is, there is no separation of CUs, predictive units (PUs), and transform units (TUs) in AVS3. Instead, each CU is always used as the base unit for both predictive and transformive, without further partitioning. In the AVS3 tree partitioning structure, a single CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary and extended quadtree structures.

[0056] As shown in Figure 4A, the video encoder 20 (or more specifically, the partitioning unit 45) generates an encoded representation of a frame by first dividing the frame into a set of CTUs. A video frame can contain integer CTUs that are sequentially ordered from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set, so that all CTUs in a video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to a specific size. As shown in Figure 4B, each CTU may comprise one CTB of chroma samples, two corresponding coding tree blocks of chroma samples, and syntactic elements used to encode the samples of these coding tree blocks. The syntactic elements describe the characteristics of different types of units in the pixel coding block and how the video sequence can be reconstructed in the video decoder 30, which include inter- or intra-prediction, intra-prediction mode, motion vector, and other parameters. For a monochrome picture, or a picture with three distinct color planes, the CTU may comprise a single coding tree block and syntactic elements used to code samples of this coding tree block. The coding tree block may be an N×N block of samples.

[0057] To achieve better performance, the video encoder 20 can recursively perform tree partitioning, such as binary, ternary, quadtree partitioning, or a combination thereof, on the encoding tree block of the CTU to divide the CTU into smaller CUs. As shown in Figure 4C, the 64x64 CTU400 is first divided into four smaller CUs, each with a block size of 32x32. Of the four smaller CUs, CU410 and CU420 are each divided into four CUs with a block size of 16x16. The two 16x16 CUs, CU430 and CU440, are further divided into four CUs with a block size of 8x8. Figure 4D depicts a quadtree data structure showing the result of the partitioning process of the CTU400 as depicted in Figure 4C, where each leaf node of the quadtree corresponds to one CU of each size in the range of 32x32 to 8x8. Similar to the CTU depicted in Figure 4B, each CU may comprise a CB of lumens sample, two corresponding coding blocks of chromens sample in frames of the same size, and syntactic elements used to code the samples in these coding blocks. For a monochrome picture, or a picture with three distinct color planes, a CU may comprise a single coding block and syntactic structures used to code the samples in this coding block. Note that the quadtree partitions depicted in Figures 4C and 4D are for illustrative purposes only, and a single CTU may be partitioned into multiple CUs to accommodate local characteristics that vary based on the quadtree / ternary / binary partition. In multiple types of tree structures, a single CTU may be partitioned by a quadtree structure, and each quadtree leaf CU may be further partitioned by binary and ternary structures. As shown in Figure 4E, coding blocks have width W and height H, and there are five possible partition types: quad-partition, horizontal 2-partition, vertical 2-partition, horizontal 3-partition, and vertical 3-partition. AVS3 offers five possible partitioning types: quadripartition, horizontal bipartition, vertical bipartition, horizontally extended quadtree partitioning (not shown in Figure 4E), and vertically extended quadtree partitioning (not shown in Figure 4E).

[0058] In some implementations, the video encoder 20 can further divide the coded block of the CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same inter or intra prediction is applied. The PU of the CU may comprise a PB for lumar samples, two corresponding PBs for chromar samples, and syntactic elements used to predict these PBs. For a monochrome picture, or a picture with three distinct color planes, the PU may comprise a single PB and a syntactic structure used to predict this PB. For each lumar, Cb, and Cr PB of the CU's PU, the video encoder 20 can generate predicted lumar, Cb, and Cr blocks.

[0059] The video encoder 20 can generate prediction blocks for the PU using intra-prediction or inter-prediction. If the video encoder 20 uses intra-prediction to generate prediction blocks for the PU, the video encoder 20 can generate prediction blocks for the PU based on decoded samples of frames associated with this PU. If the video encoder 20 uses inter-prediction to generate prediction blocks for the PU, the video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frames associated with this PU.

[0060] After the video encoder 20 generates predicted Luma, Cb, and Cr blocks for one or more PUs of the CU, the video encoder 20 can generate a Luma residual block for the CU by subtracting the predicted Luma block of the CU from its original Luma-encoded block, where each sample in the Luma residual block of the CU represents the difference between a Luma sample in one of the predicted Luma blocks of the CU and the corresponding sample in the original Luma-encoded block of the CU. Similarly, the video encoder 20 can generate a Cb residual block and a Cr residual block for the CU, where each sample in the Cb residual block of the CU represents the difference between a Cb sample in one of the predicted Cb blocks of the CU and the corresponding sample in the original Cb-encoded block of the CU, and each sample in the Cr residual block of the CU represents the difference between a Cr sample in one of the predicted Cr blocks of the CU and the corresponding sample in the original Cr-encoded block of the CU.

[0061] Furthermore, as shown in Figure 4C, the video encoder 20 can use a quadtree partition to decompose the luma, Cb, and Cr residual blocks of the CU into one or more luma, Cb, and Cr transformation blocks, respectively. A transformation block is a rectangular (square or non-square) block of samples to which the same transformation is applied. The TU of the CU may comprise a transformation block for luma samples, two corresponding transformation blocks for chroma samples, and syntactic elements used to transform the transformation block samples. Thus, each TU of the CU can be associated with a luma transformation block, a Cb transformation block, and a Cr transformation block. In some examples, a luma transformation block associated with a TU may be a subblock of the luma residual block of the CU. A Cb transformation block may be a subblock of the Cb residual block of the CU. A Cr transformation block may be a subblock of the Cr residual block of the CU. For a monochrome picture, or a picture with three distinct color planes, the TU may comprise a single transformation block and syntactic structures used to transform the samples of the transformation block.

[0062] The video encoder 20 can generate a Luma coefficient block for a TU by applying one or more transformations to the Luma transformation block of the TU. The coefficient block can be a two-dimensional array of transformation coefficients. The transformation coefficients can be scalar quantities. The video encoder 20 can generate a Cb coefficient block for a TU by applying one or more transformations to the Cb transformation block of the TU. The video encoder 20 can generate a Cr coefficient block for a TU by applying one or more transformations to the Cr transformation block of the TU.

[0063] After generating a coefficient block (e.g., a Luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 can quantize this coefficient block. Quantization generally refers to the process by which the transformation coefficients are quantized to provide further compression by reducing the amount of data used to represent the transformation coefficients. After the video encoder 20 has quantized the coefficient block, the video encoder 20 can entropy encode the syntactic elements representing the quantized transformation coefficients. For example, the video encoder 20 can perform CABAC on the syntactic elements representing the quantized transformation coefficients. Finally, the video encoder 20 can output a bitstream containing the sequence of bits and associated data that form a representation of the encoded frame, which is stored in the storage device 32 or transmitted to the destination device 14.

[0064] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntactic elements from it. The video decoder 30 can reconstruct frames of video data based at least partially on the syntactic elements obtained from the bitstream. The process of reconstructing video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can reconstruct the residual blocks associated with the TU of the current CU by performing an inverse transform on the coefficient blocks associated with the TU of the current CU. The video decoder 30 also reconstructs the encoded blocks of the current CU by adding samples of the prediction blocks for the PU of the current CU to the corresponding samples of the transform blocks for the TU of the current CU. After reconstructing the encoded blocks for each CU of the frame, the video decoder 30 can reconstruct this frame.

[0065] As mentioned above, video coding primarily uses two modes to achieve video compression: intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction). Note that IBC can be considered either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes to coding efficiency more than intra-frame prediction because it uses motion vectors to predict the current video block from a reference video block.

[0066] However, as video data capture techniques continue to evolve to preserve details within video data, and video block sizes are further improved, the amount of data required to represent the motion vector for the current frame is also substantially increasing. One way to overcome this challenge is to utilize the fact that a group of adjacent CUs in both the spatial and temporal domains not only have similar video data for predictive purposes, but the motion vectors between these adjacent CUs are also similar. Thus, by exploring spatial and temporal correlations, also known as "motion vector predictors (MVPs)" of the current CU, it is possible to use motion information of spatially adjacent CUs and / or temporally arranged CUs as an approximation of the motion information (e.g., motion vector) of the current CU.

[0067] Instead of encoding the actual motion vector of the current CU, determined by the motion estimation unit 42 as described above (as shown in Figure 2), into the video bitstream, the motion vector predictor of the current CU is subtracted from the actual motion vector of the current CU to obtain the motion vector difference (MVD) for the current CU. By doing so, it is no longer necessary to encode the motion vector determined by the motion estimation unit 42 for each CU in the frame into the video bitstream, which can significantly reduce the amount of data used to represent motion information within the video bitstream.

[0068] Similar to the process of selecting a predicted block in a reference frame during interframe prediction of a code block, both the video encoder 20 and the video decoder 30 must employ a set of rules to construct a list of motion vector candidates (also known as a "merge list") for the current CU using those potential motion vector candidates associated with spatially adjacent CUs and / or temporally arranged CUs of the current CU, and then select one member from the motion vector candidate list as the motion vector predictor for the current CU. By doing so, it is not necessary to transmit the motion vector candidate list itself from the video encoder 20 to the video decoder 30, and it is sufficient for the video encoder 20 and the video decoder 30 to encode and decode the current CU using the same motion vector predictor in the motion vector candidate list.

[0069] Generally, the basic intra-prediction schemes applied in VVC remain largely the same as those in HEVC, with the exception of several prediction tools that have been further extended, added, and / or improved, such as extended intra-prediction with wide-angle intra-mode, multiple reference line (MRL) intra-prediction, composite position-dependent intra-prediction (PDPC), intra-subsegmentation (ISP) prediction, inter-component linear model (CCLM) prediction, and matrix-weighted intra-prediction (MIP).

[0070] Similar to HEVC, VVC uses a set of reference samples adjacent to the current CU (i.e., above or to the left of the current CU) to predict the sample of the current CU. However, to capture the finer edge orientation present in raw video (especially high resolution, e.g., 4K video content), the number of angular intra-modes has been expanded from 33 in HEVC to 93 in VVC. Figure 4F is a block diagram showing the intra-modes defined in VVC. As shown in Figure 4F, of the 93 angular intra-modes, modes 2-66 are conventional angular intra-modes, while modes -1-14 and modes 67-80 are wide-angle intra-modes. In addition to angular intra-modes, the planar mode (mode 0 in Figure 1) and DC mode (mode 1 in Figure 1) from HEVC are also applied in VVC.

[0071] As shown in Figure 4E, since a quadtree / binary / ternary tree partitioning structure is applied in VVC, rectangular video blocks exist for intra-prediction in VVC, in addition to square video blocks. Since the width and height of a given video block are not equal, various sets of angular intra-modes may be selected from the 93 angular intra-modes for different block shapes. More specifically, for both square and rectangular video blocks, in addition to the planar and DC modes, 65 of the 93 angular intra-modes are also corresponding to each block shape. When the rectangular block shape of a video block satisfies certain conditions, the video decoder 30 may adaptively determine the index of the wide-angle intra-mode of the video block according to the index of the conventional angular intra-modes received from the video encoder 20, using the mapping relationships shown in Table 1 below. In other words, for non-square blocks, the video encoder 20 signals wide-angle intra-modes using the conventional angular intra-mode index, and the video decoder 30 maps them to the wide-angle intra-mode index after analysis, thus ensuring that the total number of intra-modes (i.e., 67 out of 93 angular intra-modes, including planar mode, DC mode, and 65 angular intra-modes) does not change, and the intra-predictive mode coding method also does not change. As a result, good signaling efficiency of intra-predictive modes is achieved while providing a consistent design across different block sizes.

[0072] Table 1-0 shows the mapping relationship between the index for the conventional angular intra-mode and the index for the wide-angle intra-mode for intra-prediction of different block shapes in VCC, where W represents the width of the video block and H represents the height of the video block. [Table 1]

[0073] Similar to intra-prediction in HEVC, all intra-modes in VVC (i.e., planar, DC, and angular intra-modes) utilize a set of reference samples above and to the left of the current video block for intra-prediction. However, unlike HEVC, where only the nearest row / column of the reference sample (i.e., the 0th line 201 in Figure 4G) is used, VVC introduces MRL intra-prediction, where two additional rows / columns of the reference sample (i.e., the 1st line 203 and the 3rd line 205 in Figure 4G) may also be used for intra-prediction, in addition to the nearest row / column of the reference sample. The index of the selected row / column of the reference sample is signaled and sent from the video encoder 20 to the video decoder 30. When a row / column that is not the nearest row / column of the reference sample (i.e., the 1st line 203 or the 3rd line 205 in Figure 4G) is selected, the planar mode is excluded from the set of intra-modes that can be used to predict the current video block. To prevent the use of extended reference samples outside the current CTU, MRL intra-prediction is disabled for the first row / column of the video block within the current CTU.

[0074] Sample Adaptive Offset (SAO) is a process that modifies decoded samples by conditionally adding an offset value to each sample based on the values ​​in a lookup table transmitted by the encoder after the application of a deblocking filter. SAO filtering is performed region by region based on the filtering type selected for each CTB by the syntactic element sao-type-idx. A value of 0 for sao-type-idx indicates that no SAO filter is applied to the CTB, while values ​​1 and 2 signal the use of the band offset and edge offset filtering types, respectively. In the band offset mode specified by sao-type-idx equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the full sample amplitude range is uniformly divided into 32 segments called bands, and sample values ​​belonging to four of these bands (consecutive within the 32 bands) are modified by adding a transmitted value, indicated as the band offset, which can be positive or negative. The main reason for using four consecutive bands is that within smooth regions where banding artifacts can appear, the sample amplitude of the CTB tends to be concentrated in only some of these bands. In addition, the design choice to use four offsets is unified into an edge offset operating mode that also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the syntactic element sao-eo-class, which has values ​​from 0 to 3, signals whether horizontal, vertical, or one of two oblique directions is used for edge offset classification in CTB.

[0075] Figure 5A is a block diagram illustrating four gradient patterns used in SAO in some implementation examples of this disclosure. The four gradient patterns 502, 504, 506, and 508 are for their respective sao-eo-class in edge offset mode. The sample labeled "p" indicates the central sample to be considered. The two samples labeled "n0" and "n1" specify two adjacent samples along the gradient patterns of (a) horizontal (sao-eo-class=0), (b) vertical (sao-eo-class=1), (c) 135° diagonal (sao-eo-class=2), and (d) 45° (sao-eo-class=3). Each sample in CTB is classified into one of five EdgeIdx categories by comparing the sample value p located at a certain position with the values ​​n0 and n1 of two adjacent samples located at adjacent positions, as shown in Figure 5A. This classification is performed for each sample based on the decoded sample value, and therefore no additional signaling is required for EdgeIdx classification. For EdgeIdx categories 1-4, an offset value from the transmitted lookup table is added to the sample value, depending on the EdgeIdx category at the sample location. The offset value is always positive for categories 1 and 2, and negative for categories 3 and 4. Therefore, the filter generally has a smoothing effect in edge offset mode. Table 1-1 below shows the sample EdgeIdx categories in the SAO edge class. [Table 2]

[0076] For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder for each CTB. In type 1, the code is also encoded. The offset values, as well as related syntactic elements such as sao-type-idx and sao-eo-class, are determined by the encoder, typically using criteria that optimize velocity distortion performance. SAO parameters may be indicated to be inherited from the left or upper CTB using merge flags to make signaling more efficient. In summary, SAO is a nonlinear filtering operation that allows for additional refinement of the reconstructed signal, enhancing the signal representation both in smooth regions and around edges.

[0077] In some embodiments, pre-sample adaptive offset (SAO) is implemented. The low-complexity coding performance of pre-SAO is promising for future video coding standard development. In some examples, pre-SAO is applied only to luma component samples using luma samples for classification. Pre-SAO works by applying two SAO-like filtering operations called SAOV and SAOH, which are applied in conjunction with a deblocking filter (DBF) before applying the existing (legacy) SAO. The first SAO-like filter, SAOV, works to apply SAO to the input picture Y2 after a deblocking filter (DBFV) has been applied to the vertical edges. Y3(i)=Clip1(Y2(i)+d1·(f(i)>T?1:0)-d2·(f(i)<-T?1:0))

[0078] Here, T is a predetermined positive constant, and d1 and d2 are, f(i) = Y1(i) - Y2(i) This is an offset coefficient associated with the two classes, based on the sample-by-sample difference between Y1(i) and Y2(i) given by [the formula].

[0079] The first class for d1 is given such that all sample locations i are obtained such that f(i)>T, and the second class for d2 is given by f(i)<-T. The offset coefficients d1 and d2 are calculated by the encoder so as to minimize the mean squared error between the output picture Y3 of SAOV and the original picture X, similar to existing SAO processes. After SAOV is applied, a second SAO-like filter SAOH operates to apply SAO to Y4 after SAOV is applied by classification based on the sample-by-sample difference between Y3(i) and Y4(i), which is the output picture of the deblocking filter (DBFH) for horizontal edges. The same procedure as for SAOV is applied to SAOH, with Y3(i)-Y4(i) used for its classification instead of Y1(i)-Y2(i). Two offset coefficients for each of SAOH and SAOV, a given threshold T, and an enable flag are signaled at the slice level. SAOH and SAOV are applied independently to the luma and the two chroma components.

[0080] In some cases, both SAOV and SAOH operate only on picture samples affected by their respective deblocking (DBFV or DBFH). Therefore, unlike existing SAO processes, only a portion of all samples within a given spatial domain (picture, or CTU in the case of legacy SAO) are processed by SAO pre, resulting in a low average increase in decoder-side operation per picture sample (two or three comparisons and two additions per sample in the worst-case scenario by preliminary estimation). SAO pre only requires samples used by the deblocking filter and does not store additional samples in the decoder.

[0081] In some embodiments, a bidirectional filter (BIF) is implemented to explore compression efficiencies beyond VVC. The BIF is carried out in a sample-adaptive offset (SAO) loop filter stage. Both the bidirectional filter (BIF) and the SAO use samples from deblocking as input. Each filter creates a sample-by-sample offset, which is added to the input sample, then clipped, and then proceeds to the ALF.

[0082] For details, see Output Sample I OUT teeth, I OUT =clip3(I C +ΔI BIF +ΔI SAO ) It is obtained as, and here I C This is the input sample from deblocking, and ΔI BIF ΔI is the offset from the two-way filter. SAO This is the offset from SAO.

[0083] In some embodiments, the implementation example provides the possibility that the encoder enables or disables filtering at the CTU and slice levels. The encoder makes this decision by evaluating the velocity-distortion optimization (RDO) cost.

[0084] The following syntactic elements are introduced in PPS. [Table 3]

[0085] A pps_bilateral_filter_enabled_flag equal to 0 specifies that the bidirectional loop filter is disabled for slices referencing the PPS. A pps_bilateral_filter_flag equal to 1 specifies that the bidirectional loop filter is enabled for slices referencing the PPS.

[0086] `bilateral_filter_strength` specifies the bilateral loop filter strength value used in the bilateral transform block filter process. The value of `bilateral_filter_strength` should be within the range of 0 to 2.

[0087] `bilateral_filter_qp_offset` specifies the offset used in deriving the bidirectional filter lookup table LUT(x) for slices referencing PPS. `bilateral_filter_qp_offset` should be inclusively within the range of -12 to +12.

[0088] The following syntactic elements will be introduced. [Table 4] [Table 5]

[0089] The meaning is as follows: A slice_bilateral_filter_all_ctb_enabled_flag equal to 1 indicates that bilateral filtering is enabled and applies to all CTBs in the current slice. When slice_bilateral_filter_all_ctb_enabled_flag does not exist, it is inferred to be equal to 0.

[0090] A value of 1 for slice_bilateral_filter_enabled_flag indicates that bidirectional filtering is enabled and may be applied to the current slice's CTB. If slice_bilateral_filter_enabled_flag does not exist, it is inferred to be equal to slice_bilateral_filter_all_ctb_enabled_flag.

[0091] The bilateral_filter_ctb_flag[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equal to 1 specifies that the bilateral filter is applied to the luma coding tree block of the coding tree unit at the luma location (xCtb, yCtb). The bilateral_filter_ctb_flag[cIdx][xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] equal to 0 specifies that the bilateral filter is not applied to the luma coding tree block of the coding tree unit at the luma location (xCtb, yCtb). When bilateral_filter_ctb_flag does not exist, it is inferred to be equal to (slice_bilateral_filter_all_ctb_enabled_flag & slice_bilateral_filter_enabled_flag).

[0092] In some examples, for a filtered CTU, the filtering process proceeds as follows. At a picture boundary where samples are not available, the bilateral filter uses extension (sample repetition) to fill in the unavailable samples. For a virtual boundary, the behavior is the same as for SAO, i.e., no filtering is performed. When crossing a horizontal CTU boundary, the bilateral filter can access the same samples that SAO accesses to. FIG. 5B is a block diagram depicting the naming convention for samples surrounding a central sample according to some implementations of the present disclosure. As an example, if the central sample I C is located on the topmost line of the CTU, I NW , I A , and I NE are read from the upper CTU in the same manner as SAO, but I AA is padded, and thus no extra line buffer is required. The central sample I CThe surrounding sample is shown according to Figure 5B, where A, B, L, and R represent up, down, left, and right, and NW, NE, SW, SE represent northwest, etc. Similarly, AA represents up-up, BB represents down-down, etc. This diamond shape is I AA , I BB , I LL , or I RR This differs from another method that uses square filter support without using [a specific method].

[0093] JPEG0007840400000006.jpg52170

[0094] Next, the modified value is,

number

[0095] JPEG0007840400000008.jpg41170

number

[0096] Both modified values ​​are summed up.

number

[0097] JPEG0007840400000012.jpg69170 [Table 6]

[0098] Finally, the two-way filter offset ΔI BIF The following is calculated. For complete intensity filtering, the following is used: ΔI BIF =(c v +16)>>5 For filtering at half intensity, the following is used: ΔI BIF =(c v +32)>>6

[0099] The general formula for n-bit data is: r add =2 14-n-bilateral_filter_strength r shift =15-n-bilateal_filter_strength ΔI BIF =(c v +r add )>>r shift This uses the `bilateral_filter_strength` parameter, where `bilateral_filter_strength` can be 0 or 1, and the signal is processed in pps.

[0100] In some embodiments, methods and systems for improving coding efficiency or reducing the complexity of sample-adaptive offsets (SAOs) by introducing inter-component information are disclosed herein. SAOs are used in the HEVC, VVC, AVS2, and AVS3 standards. In the following description, existing SAO designs in the HEVC, VVC, AVS2, and AVS3 standards are used as basic SAO methods for those skilled in the art of video coding, but the inter-component methods described herein may also be applied to other loop filter designs or other coding tools with a similar design spirit. For example, in the AVS3 standard, SAOs are replaced by a coding tool called enhanced sample-adaptive offsets (ESAOs). However, the CCSAOs disclosed herein may be applied in parallel with ESAOs. In another example, a CCSAO may be applied in parallel with a Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0101] In existing SAO designs for HEVC, VVC, AVS2, and AVS3 standards, the sample offset values ​​for lumens (Y), chromens (Cb), and chromens (Cr) are determined independently. That is, for example, the current chroma sample offset is determined only by the current chroma sample value and adjacent chroma sample values, without considering the sequence or adjacent lumens samples. However, lumens samples retain more original picture detail than chromens and can benefit from the determination of the current chroma sample offset. Furthermore, since chromens typically lose high-frequency detail after color conversion from RGB to YCbCr, or after quantization and deblocking filtering, introducing lumens samples with preserved high-frequency detail for chroma offset determination can benefit from the reconstruction of the chroma sample. Therefore, further gains can be expected by exploring inter-component correlations, for example by using inter-component sample adaptive offset (CCSAO) methods and systems. In some embodiments, this correlation includes not only inter-component sample values ​​but also picture / encoded information such as the predicted / residual coding mode, transformation type, and quantization / deblocking / SAO / ALF parameters derived from the inter-component sample values.

[0102] Another example is that, in the case of SAO, the luma sample offset is determined solely by the luma sample. However, luma samples having the same band offset (BO) classification may be further classified by their sequence and adjacent chromatic samples, thereby obtaining a more effective classification. SAO classification can be obtained as a shortcut to compensate for sample differences between the original picture and the reconstructed picture. Therefore, an effective classification is desired.

[0103] Figure 6A is a block diagram showing a CCSAO system and process applied on a chroma sample using DBF Y as input by several implementations of this disclosure. The luma sample after the luma deblocking filter (DBF Y) is used to determine additional offsets to the chroma Cb and Cr after SAO Cb and SAO Cr. For example, the current chroma sample 602 is first classified using the sequence luma sample 604 and the adjacent (white) luma sample 606, and the corresponding CCSAO offset value for the corresponding class is added to the current chroma sample value. Figure 6B is a block diagram showing a CCSAO system and process applied on a luma sample and chroma sample using DBF Y / Cb / Cr as input by several implementations of this disclosure. Figure 6C is a block diagram showing a CCSAO system and process that can function independently by several implementations of this disclosure. Figure 6D is a block diagram showing a CCSAO system and process that can be applied recursively (2 or N times) with the same or different offsets at the same codec stage, or repeated at different stages, by several implementations of this disclosure. In summary, in some embodiments, information on the current luma sample and adjacent luma samples, sequence and adjacent chroma samples (Cb and Cr) may be used to classify the current luma sample. In some embodiments, sequence and adjacent luma samples, sequence and adjacent cross-chroma samples, as well as the current chroma sample and adjacent chroma samples may be used to classify the current chroma sample (Cb or Cr). In some embodiments, CCSAO may be cascaded after (1) DBF Y / Cb / Cr, (2) reconstructed image Y / Cb / Cr before DBF, or (3) SAO Y / Cb / Cr, or (4) ALF Y / Cb / Cr.

[0104] In some embodiments, CCSAO can also be applied in parallel with other coding tools, such as ESAO in the AVS standard, CDEF in the AV1 standard, or neural network loop filters (NNLF). Figure 6E is a block diagram showing a CCSAO system and process applied in parallel with ESAO in the AVS standard by some implementation examples of this disclosure.

[0105] Figure 6F is a block diagram showing a CCSAO system and process applied after SAO by some implementations of the present disclosure. In some embodiments, Figure 6F shows that the location of the CCSAO may be after SAO, i.e., at the location of the intercomponent adaptive loop filter (CCALF) in the VVC standard. Figure 6G is a block diagram showing that the CCSAO system and process may function independently without CCALF by some implementations of the present disclosure. In some embodiments, SAO Y / Cb / Cr may be replaced with ESAO, for example in the AVS3 standard.

[0106] Figure 6H is a block diagram showing a CCSAO system and process applied in parallel to CCALF by some implementations of this disclosure. In some embodiments, CCSAO may be applied in parallel to CCALF. In some embodiments, the locations of CCALF and CCSAO may be switched, as shown in Figure 6H. In some embodiments, as shown in Figures 6A–6H, or throughout this disclosure, SAO Y / Cb / Cr blocks may be replaced with ESAO Y / Cb / Cr (AVS3) or CDEF (AV1). Note that Y / Cb / Cr may also be represented as Y / U / V in the video coding region. In some embodiments, if the video is in RGB format, CCSAO can also be applied by simply mapping the YUV notation to GBR, respectively, in this disclosure.

[0107] Figure 6I is a block diagram showing a CCSAO system and process applied in parallel to SAO and BIF by some implementations of the present disclosure. Figure 6J is a block diagram showing a CCSAO system and process applied in parallel to BIF by swapping SAOs by some implementations of the present disclosure. In some embodiments, the current chroma sample classification reuses the SAO type (edge ​​offset (EO) or BO), class, and category of the sequence chroma sample. The corresponding CCSAO offsets can be signaled or derived from the decoder itself. For example, let h_Y be the sequence chroma SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a restricted table, e.g., ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., where |w| contains only powers of 2 values.

[0108] In some embodiments, a sequence luma sample (Y0) and a comparison score [-8,8] of eight adjacent luma samples are used, resulting in a total of 17 classes. Initial class = 0 A loop (Yi, i=1~8) through eight adjacent luma samples. if Y0 > Yi Class += 1 else if Y0 <Yi Class-=1

[0109] In some embodiments, the classification methods described above can be combined. For example, to increase diversity, a comparison score combined with SAO BO (32-band classification) is used, resulting in a total of 17*32 classes. In some embodiments, Cb and Cr can use the same class to reduce complexity or to save bits.

[0110] Figure 7 is a block diagram showing a sample process using CCSAO in relation to some implementation examples of the present disclosure. Specifically, Figure 7 shows that the inputs to the CCSAO can be vertical and horizontal DBF inputs in order to simplify class determination or to increase flexibility. For example, Y0_DBF_V, Y0_DBF_H, and Y0 are array luma samples in the inputs of DBF_V, DBF_H, and SAO, respectively. Yi_DBF_V, Yi_DBF_H, and Yi are eight adjacent luma samples in the inputs of DBF_V, DBF_H, and SAO, respectively, where i=1 to 8. Max Y0=max(Y0_DBF_V,Y0_DBF_H,Y0_DBF) Max Yi=max(Yi_DBF_V,Yi_DBF_H,Yi_DBF) max Y0 and max Yi are used for the CCSAO classification.

[0111] Figure 8 is a block diagram showing that the CCSAO process is interleaved into vertical and horizontal DBFs in some implementations of the present disclosure. In some embodiments, the CCSAO blocks in Figures 6, 7, and 8 may be selective. For example, the DBF_V sample input may be used as the CCSAO input while using Y0_DBF_V and Yi_DBF_V for a first CCSAO_V to which the same sample processing as in Figure 6 is applied.

[0112] In some embodiments, the implemented CCSAO syntax is shown in Table 2 below. [Table 7]

[0113] In some embodiments, if one additional chroma offset is signaled to signal the CCSAO Cb and Cr offset values, the other chroma component offset may be derived by a positive or negative sign, or by weighting to reduce bit overhead. For example, let h_Cb and h_Cr be the offsets for CCSAO Cb and Cr, respectively. When w is explicitly signaled such that w = ±|w| and the |w| candidates are limited, h_Cr may be derived from h_Cb without explicitly signaling h_Cr itself. h_Cr=w*h_Cb

[0114] Figure 7 is a block diagram showing a sample process using CCSAO in some implementations of the present disclosure. Figure 8 is a block diagram showing that the CCSAO process is interleaved with vertical and horizontal deblocking filters (DBFs) in some implementations of the present disclosure.

[0115] Figure 9 is a flowchart illustrating an exemplary process 900 for decoding a video signal using inter-component correlations relating to some implementations of the present disclosure.

[0116] The video decoder 30 receives a video signal containing a first component and a second component (910). In some embodiments, the first component is the luma component of the video signal, and the second component is the chroma component.

[0117] The video decoder 30 also receives multiple offsets associated with the second component (920).

[0118] The video decoder 30 then uses the characterization of the first component to obtain the classification category associated with the second component (930). For example, in Figure 6, the current chroma sample 602 is first classified using the sequence chroma sample 604 and the adjacent (white) chroma sample 606, and the corresponding CCSAO offset value is added to the current chroma sample.

[0119] The video decoder 30 further selects a first offset from a plurality of offsets for the second component according to the classification category (940).

[0120] The video decoder 30 also modifies the second component based on the selected first offset (950).

[0121] In some embodiments, obtaining a classification category associated with a second component by utilizing the characterization of a first component (930) includes obtaining a classification category for each of each of the second component samples by utilizing each of the first component samples, where each of the first component samples is the respective sequence sample of the first component for each of the second component samples. For example, current chroma sample classification reuses the SAO type (EO or BO), class, and category of the sequence chroma samples.

[0122] In some embodiments, obtaining a classification category associated with a second component using characterization of a first component (930) includes obtaining a classification category for each sample of the second component using each sample of the first component, where each sample of the first component is reconstructed before or after deblocking. In some embodiments, the first component is deblocked with a deblocking filter (DBF). In some embodiments, the first component is deblocked with a Luma deblocking filter (DBF Y). For example, as an alternative to Figure 6 or Figure 7, the CCSAO input may precede the DBF Y.

[0123] In some embodiments, the characterization is derived by dividing the range of sample values ​​of the first component into several bands and selecting a band based on the intensity value of the sample within the first component. In some embodiments, the characterization is derived from the band offset (BO).

[0124] In some embodiments, the characterization is derived based on the direction and intensity of the edge information of the sample within the first component. In some embodiments, the characterization is derived from the edge offset (EO).

[0125] In some embodiments, modifying the second component (950) includes directly adding a selected first offset to the second component. For example, the corresponding CCSAO offset value is added to the current chromatic component sample.

[0126] In some embodiments, modifying the second component (950) includes mapping a selected first offset to a second offset and adding the mapped second offset to the second component. For example, if one additional chroma offset is signaled in order to signal the CCSAO Cb and Cr offset values, the other chroma component offset may be derived by using a positive or negative sign, or by weighting to reduce bit overhead.

[0127] In some embodiments, receiving a video signal (910) includes receiving a syntactic element indicating whether a method of decoding the video signal using CCSAO is enabled for the video signal in the sequence parameter set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0128] In some embodiments, receiving a video signal (910) includes receiving a syntactic element indicating whether a method of decoding the video signal using CCSAO is enabled for a second component at the slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled within the respective slice for Cb or Cr.

[0129] In some embodiments, receiving a plurality of offsets associated with a second component (920) includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign for the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​for the current CTU.

[0130] In some embodiments, receiving a plurality of offsets associated with a second component (920) includes receiving a syntactic element indicating whether the received offset of a CTU is the same as one of the adjacent CTUs of the CTU, where adjacent CTUs are adjacent CTUs to the left or above. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from a CTU to the left or above.

[0131] In some embodiments, the video signal further includes a third component, and a method of decoding the video signal using a CCSAO further includes receiving a second plurality of offsets associated with the third component, obtaining a second classification category associated with the third component by utilizing a characterization of the first component, selecting a third offset from the second plurality of offsets for the third component according to the second classification category, and modifying the third component based on the selected third offset.

[0132] Figure 11 is a block diagram of the sampling process showing that all sequence and adjacent (white) luma / chroma samples can be fed into CCSAO classification by some implementations of the present disclosure. In some embodiments, the classifiers described in the present disclosure can function not only as intercomponent classification (e.g., classifying chroma using luma, or vice versa) but also as single-component classification (e.g., classifying luma using luma, or classifying chroma using chroma). In some embodiments, the classifiers disclosed in the present disclosure can also benefit from the original SAO classification and may be applied to the original SAO classification, for example. Figures 6A, 6B, and 11 show the inputs to CCSAO classification. In Figure 11, the current chroma sample is 1104, the intercomponent sequence chroma sample is 1102, and the sequence luma sample is 1106.

[0133] In some embodiments, an exemplary classifier (C0) uses the sequence luma or chroma sample values ​​(Y0) (Y4 / U4 / V4 in Figures 6B and 6C) in Figure 12A below for classification. Let band_num be the number of equally divided bands of the luma or chroma dynamic range, and bit_depth be the sequence bit depth. An example of a class index for the current chroma sample is as follows: Class (C0) = (Y0 * band_num) >> bit_depth

[0134] In some embodiments, this classification takes rounding into account, for example, as follows: Class (C0) = ((Y0*band_num) + (1<<bit_depth))> >bit_depth

[0135] Several examples of band_num and bit_depth are given in Table 3 below. Table 3 shows three classification examples where the number of bands differs for each classification example. [Table 8]

[0136] In some embodiments, the classifier uses a different luma sample position for the C0 classification. Figure 10A is a block diagram showing a classifier that uses a different luma (or chroma) sample position for the C0 classification according to some implementation examples of the present disclosure, for example, using the adjacent Y7 instead of Y0 for the C0 classification.

[0137] In some embodiments, different classifiers can be switched at the Sequence Parameter Set (SPS) / Adaptive Parameter Set (APS) / Picture Parameter Set (PPS) / Picture Header (PH) / Slice Header (SH) / Region / Coding Tree Unit (CTU) / Coding Unit (CU) / Subblock / Sample Level. For example, in Figure 10, Y0 is used for POC0 but Y7 is used for POC1, as shown in Table 4 below. [Table 9]

[0138] In some embodiments, Figure 10B shows some examples of different shapes for luma candidates relating to some implementations of the present disclosure. For example, constraints may be applied to the shape. In some cases, the total number of luma candidates must be a power of 2, as shown in Figures 10B(b),(c), and(d). In some cases, the number of luma candidates must be horizontally and vertically symmetric with respect to the chroma sample (center), as shown in Figures 10B(a),(c),(d), and(e). In some embodiments, the power of 2 constraint and the symmetry constraint may also apply to the chroma candidates. The U / V portions of Figures 6B and 6C show an example of a symmetry constraint. In some embodiments, different color formats may have different classifier "constraints". For example, the 420-color format uses the luma / chroma candidate selection shown in Figures 6B and 6C (one candidate is selected from a 3x3 shape), while the 444-color format uses Figure 10B(f) for luma and chroma candidate selection, and the 422-color format uses Figure 10B(g) for luma (two chroma samples share four luma candidates) and Figure 10B(f) for chroma candidates.

[0139] In some embodiments, the C0 position and the C0 band_num may be combined and can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different combinations may be different classifiers, as shown in Table 5 below. [Table 10]

[0140] In some embodiments, the sequence lumasample value (Y0) is replaced with a value (Yp) obtained by weighting the sequence and adjacent lumasamples. Figure 12A shows exemplary classifiers by replacing the sequence lumasample value with a value obtained by weighting the sequence and adjacent lumasamples, as demonstrated by some implementations of the present disclosure. The sequence lumasample value (Y0) may be replaced with a phase correction value (Yp) obtained by weighting adjacent lumasamples. Different Yp may be different classifiers.

[0141] In some embodiments, different Yp values ​​are applied to different chroma formats. For example, as shown in Figure 12A, Yp in Figure 12A(a) is used for the 420 chroma format, Yp in Figure 12A(b) is used for the 422 chroma format, and Y0 is used for the 444 chroma format.

[0142] In some embodiments, another classifier (C1) is the comparison score [-8,8] between a sequence luma sample (Y0) and eight adjacent luma samples, resulting in a total of 17 classes, as shown below. Initial class (C1) = 0, loop through 8 adjacent Luma samples (Yi, i = 1 to 8) if Y0 > Yi Class += 1 else if Y0 <Yi Class-=1

[0143] In some embodiments, the example of C1 is equivalent to the following function, where the threshold th is 0. ClassIdx=Index2ClassTable(f(C,P1)+f(C,P2)+...+f(C,P8)) if xy>th,f(x,y)=1;if xy=th,f(x,y)=0;if xy <th,f(x,y)=-1 In the above formula, Index2ClassTable is a lookup table (LUT), C is the current sample or sequence sample, and P1 to P8 are adjacent samples.

[0144] In some embodiments, similar to C4 classifiers, one or more thresholds may be predefined (e.g., maintained in a LUT) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level to assist in difference classification (quantization).

[0145] In some embodiments, the variation (C1') counts only the comparison score [0,8], which results in eight classes. (C1,C1') are classifier groups, and a PH / SH level flag may be signaled to switch between C1 and C1'.

[0146] Initial class (C1') = 0, loop through 8 adjacent Luma samples (Yi, i = 1 to 8) if Y0 > Yi Class += 1

[0147] In some embodiments, the variation (C1) selectively uses adjacent N samples from M adjacent samples to count the comparison score. To indicate which adjacent samples were selected to count the comparison score, an M-bit bitmask may be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Using Figure 6B as an example for a Luma classifier, eight adjacent Luma samples are candidates, and an 8-bit bitmask (01111110) is signaled at PH, which indicates that six samples Y1-Y6 were selected, and therefore the comparison score is [-6,6], resulting in an offset of 13. The selective classifier C1 gives the encoder more choices due to the trade-off between the offset signaling the overhead and the classification granularity.

[0148] Similar to C1, the variation (C1') counts only the comparison score [0,+N], and the previous example of bitmask 01111110 gives a comparison score of [0,6], resulting in an offset of 7.

[0149] In some embodiments, different classifiers are combined to produce a general-purpose classifier. For example, different classifiers are applied to different pictures (different POC values), as shown in Table 6-1 below. [Table 11]

[0150] In some embodiments, as shown in Table 6-2, another exemplary classifier (C3) uses a bitmask for classification. To illustrate this classifier, a 10-bit bitmask is signaled at the SPS / APS / PPS / PH / SH / Region / CTU / CU / Subblock / Sample Level. For example, bitmask 11 1100 0000 means that for a given 10-bit luma sample value, only the most significant bits (MSB): 4 bits are used for classification, resulting in a total of 16 classes. Another exemplary bitmask 10 0100 0001 means that only 3 bits are used for classification, resulting in a total of 8 classes.

[0151] In some embodiments, the bitmask length (N) may be fixed or switchable at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, for a 10-bit sequence, a 4-bit bitmask 1110 is signaled in PH within the picture, and the MSB 3 bits b9, b8, and b7 are used for classification. Another example is a 4-bit bitmask 0011 on the LSB, with b0 and b1 used for classification. The bitmask classifier can be luma or chroma classification. Whether the MSB or LSB is used for bitmask N may be fixed or switchable at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0152] In some embodiments, the luma position and the C3 bitmask may be combined and can be switched between SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample levels. Different combinations may result in different classifiers.

[0153] In some embodiments, the "maximum number of ones" bitmask limit may be applied to limit the number of corresponding offsets. For example, limiting the "maximum number of ones" bitmask to 4 in SPS results in a maximum offset of 16 for the sequence. Bitmasks for different POCs may be different, but the "maximum number of ones" shall not exceed 4 (and shall not exceed 16 across all classes). The "maximum number of ones" value may be signaled and can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. [Table 12]

[0154] In some embodiments, as shown in Figure 11, other intercomponent chromat samples, such as chromat sample 1102 and its adjacent samples, may also be fed into the CCSAO classification, for example, for the current chromat sample 1104. For example, a Cr chromat sample may be fed into the CCSAO Cb classification. A Cb chromat sample may be fed into the CCSAO Cr classification. The classifier of the intercomponent chromat sample may be the same as the luma intercomponent classifier, or it may have its own classifier, as described in this disclosure. Two classifiers can be combined to form a joint classifier for classifying the current chromat sample. For example, a joint classifier combining an intercomponent luma sample and a chromat sample results in a total of 16 classes, as shown in Table 6-3 below. [Table 13]

[0155] All of the above-mentioned classifiers (C0, C1, C1', C2, C3) can be combined. See, for example, Table 6-4 below. [Table 14]

[0156] In some embodiments, an exemplary classifier (C2) uses the difference (Yn) between the sequence and adjacent luma samples. Figure 12A(c) shows an example of Yn, which has a dynamic range of [-1024, 1023] when the bit depth is 10. The band_num of C2 is the number of equally divided bands of the Yn dynamic range. Class (C2) = (Yn + (1 <<bit_depth)*band_num)> >(bit_depth+1).

[0157] In some embodiments, C0 and C2 are combined to produce a general-purpose classifier. For example, different classifiers are applied to different pictures (different POCs), as shown in Table 7 below. [Table 15]

[0158] In some embodiments, all of the above-described classifiers (C0, C1, C1', C2) are combined. For example, different classifiers are applied to different pictures (different POCs), as shown in Table 8-1 below. [Table 16]

[0159] In some embodiments, as shown in Table 8-2 below, an exemplary classifier (C4) uses the difference between the CCSAO input value and the sample value to be compensated for for classification. For example, when CCSAO is applied at the ALF stage, the difference between the sample values ​​of the current component before and after ALF is used for classification. To assist in the classification (quantization) of the difference, one or more thresholds may be predefined (e.g., maintained in a lookup table (LUT)) or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. The C4 classifier can be combined with the Y / U / VbandNum of C0 to form a co-classifier (e.g., exemplary POC1 as shown in Table 8-2). [Table 17]

[0160] In some embodiments, different encoding modes may introduce different distortion statistics into the reconstructed image, so the exemplary classifier (C5) uses “encoded information” to aid in subblock classification. For example, as shown in Table 8-3 below, a CCSAO sample is classified by the previous encoded information of that sample, and a combination of encoded information can form a classifier. Figure 30 below shows another example of different stages of encoded information for C5. [Table 18]

[0161] In some embodiments, an exemplary classifier (C6) uses YUV color conversion values ​​for classification. For example, to classify the current Y component, a 1 / 1 / 1 array or adjacent Y / U / V samples are selected to be color-converted to RGB, and the R values ​​are quantized using the bandNum of C3 to become the current Y component classifier.

[0162] In some embodiments, exemplary classifiers (C7) may be obtained as simplified variations of C0 / C3 and C6. To derive the bandNum classification of the current component C0 / C3, sequence / current and adjacent samples of all three color components are used. For example, to classify the current U sample, sequence, and adjacent Y / V, the current and adjacent U samples are used, as in Figure 6B, which is...

number

[0163] In some embodiments, one special case of C7 may derive an intermediate sample S using only a 1 / 1 / 1 sequence or adjacent Y / U / V samples, which may also be obtained as a special case of C6 (color conversion by using three components). S may be further fed into a bandNum classifier of C0 / C3. classIdx=bandS=(S*bandNumS)>>BitDepth

[0164] In some embodiments, C7, like the bandNum classifiers C0 / C3, may also be combined with other classifiers to form a co-classifier. In some examples, C7 may not be the same as in later examples (three-component co-bandNum classification for each Y / U / V component) which jointly use sequences and adjacent Y / U / V samples for classification.

[0165] In some embodiments, in order to reduce the cij signaling overhead and limit the value of S within the bit depth range, one constraint of sum of cij = 1 may be applied. For example, force c00 = (1 - sum of other cij). Which cij (c00 in this example) is forced (derived by other coefficients) may be pre-defined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / sub-block / sample level.

[0166] JPEG0007840400000027.jpg16168

Number

[0167] JPEG0007840400000029.jpg11170

Number

[0168] In some embodiments, in order to reduce the complexity of block classification, subsampled 1-D Laplacian calculation is applied. FIG. 12B shows subsampled Laplacian calculation according to some implementations of the present disclosure. As shown in FIG. 12B, the same subsample positions are used for gradient calculations in all directions.

[0169] Then, the maximum and minimum values of D for the horizontal and vertical gradients are

Number

[0170] In some embodiments, the maximum and minimum values of the gradients in two diagonal directions are

Number

[0171] JPEG0007840400000034.jpg77170

[0172] In some embodiments, the activity value A is [Number] calculated as

[0173] JPEG0007840400000036.jpg13170

[0174] In some embodiments, the classification method is not applied to the chroma components within the picture.

[0175] In some embodiments, before filtering each 4×4 luma block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k, l) and the corresponding filter clipping values c(k, l) according to the gradient values calculated for that block. This is equivalent to applying these transformations to the samples within the filter corresponding region. This concept is to make the different blocks to which the ALF is applied more similar by aligning the directions.

[0176] In some embodiments, three geometric transformations including diagonal, vertical flipping and rotation are introduced. Diagonal: f D (k, l) = f(l, k), c D (k, l) = c(l, k) Vertical flipping: f V (k, l) = f(k, K - l - 1), c V (k, l) = c(k, K - l - 1) Rotation: f [[ID=四十二]] R (k, l) = f(K - l - 1, k), c R (k, l) = c(K - l - 1, k) Here, K is the size of the filter, and 0 ≤ k, l ≤ K-1 are the coefficient coordinates, so that (0,0) is the upper left corner and (K-1,K-1) is the lower right corner. These transformations are applied to the filter coefficients f(k,l) and clipping value c(k,l) depending on the gradient values ​​calculated for that block. The relationship between the transformations and the four gradients in the four directions is summarized in Table 8-4 below. [Table 19]

[0177] JPEG0007840400000038.jpg66170

[0178] JPEG0007840400000039.jpg33170

[0179] JPEG0007840400000040.jpg39170

[0180] In some embodiments, A may then be further mapped to the range [0,4].

number

[0181] In some embodiments, another exemplary classifier (C9) can use inter-component / current-component spatial gradient information as a classifier. Similar to the block gradient classifier described above, a single sample located at (k,l) can obtain a sample gradient class by the following: (1) Calculate N directional gradients (Laplacian or forward / reverse). (2) Calculate the maximum and minimum values ​​of the M grouped directions of gradients (M ≤ N). (3) Compare N values ​​with each other and set m thresholds t1~t mThe direction D is calculated by comparing with. (4) Apply a geometric transformation according to the magnitude of the relative gradient (optional).

[0182] In some embodiments, an exemplary classifier (C9) can be applied at the sample level for sample classification as follows. (1) Calculate four directional gradients (Laplacian). (2) Calculate the maximum and minimum values of the gradients of two grouped directions (H / V and D / A). (3) Compare N values with each other, and two thresholds t1~t m The direction D is calculated by comparing with. (4) Apply a geometric transformation according to the magnitude of the relative gradient as shown in Table 8-4.

[0183] In some embodiments, C8 and C9 can be combined to form a joint classifier.

[0184] In some embodiments, another exemplary classifier (C10) can use inter-component / current component edge information for the classification of the current component. By extending the original SAO classifier, C10 can extract inter-component / current component edge information more effectively as follows. (1) Select one direction to calculate two edge strengths, one direction is formed by the current sample and two adjacent samples, and one edge strength is calculated by subtracting one adjacent sample from the current sample (or determining the difference between the value of the current sample and the value of one adjacent sample). (2) Quantize each edge strength into M segments by M-1 thresholds Ti. (3) Use an M*M class to classify the current component sample.

[0185] Figure 12C shows an example of using inter-component / current-component edge information for current-component classification by several implementations of this disclosure. The current sample is represented by c, and the two adjacent samples between the current-component / inter-component are represented by a and b. In this example, (1) one diagonal direction is selected from four direction candidates. The difference (ca) and (cb) are two edge intensities in the range of -1023 to 1023 (for example, for a sequence of 10b). (2) Each edge intensity is quantized into four segments by a common threshold [-T, 0, T]. (3) Classify the current component sample using 16 classes.

[0186] As shown in Figure 12C, one diagonal direction is selected, and the difference (ca) and (cb) are quantized into four and four segments by the threshold [-T, 0, T], which form 16 edge segments. The position of (a, b) can be indicated by signaling two syntaxes edgeDir and edgeStep.

[0187] In some embodiments, the directional patterns can be 0, 45, 90, 135 degrees (with 45 degrees between directions), or extended to 22.5 degrees between directions, or can be predefined directions set or signaled at the SPS / APS / PPS / PH / SH / region(set) / CTU / CU / subblock / sample level.

[0188] In some embodiments, edge strength may also be defined as (ba), which simplifies the calculation but at the expense of precision.

[0189] In some embodiments, M-1 thresholds may be predefined or signaled at the SPS / APS / PPS / PH / SH / region(set) / CTU / CU / subblock / sample level.

[0190] In some embodiments, the M-1 thresholds can be different sets for edge strength calculations, e.g., different sets for (ca) and (cb). If different sets are used, all classes may be different. For example, if [-T, 0, T] is used to calculate (ca) but [-T, T] is used for (cb), then the total number of classes is 4*3.

[0191] In some embodiments, the M-1 thresholds can use a "symmetrical" property to reduce signaling overhead. For example, instead of [T0, T1, T2] which requires signaling three thresholds, a predefined pattern [-T, 0, T] may be used. Another example is [-T, T].

[0192] In some embodiments, the threshold only needs to include a value that is a power of 2, which not only effectively captures the edge intensity distribution but also reduces the complexity of the comparison (only the MSB N bits need to be compared).

[0193] In some embodiments, the positions of a and b may be indicated by signaling two syntax elements, as shown in Figure 12C: (1) edgeDir indicating the selected direction, and (2) edgeStep indicating the sample distance used to calculate the edge strength.

[0194] In some embodiments, edgeDir / edgeStep may be predefined or signaled at the SPS / APS / PPS / PH / SH / region(set) / CTU / CU / subblock / sample level.

[0195] In some embodiments, edgeDir / edgeStep may be encoded in other ways, such as a fixed-length code (FLC), a shortened unary (TU) code, an exponential Golomb code of degree k (EGk), a signed EG0 (SVLC), or an unsigned EG0 (UVLC).

[0196] In some embodiments, C10 can be combined with bandNumY / U / V or other classifiers to form a co-classifier. For example, combining 16 edge intensities with up to 4 bandNumY bands yields 64 classes.

[0197] In some embodiments, other exemplary classifiers that use only the information of the current component for the classification of the current component may be used as inter-component classifiers. For example, as shown in Figure 5A and Table 1, chroma sample information and eo-class are used to derive EdgeIdx and classify the current chroma sample. Other “non-inter-component” classifiers that can also be used as inter-component classifiers include edge direction, pixel intensity, pixel variation, pixel dispersion, sum of pixel Laplacians, Sobel operator, compass operator, high-pass filter value, low-pass filter value, etc.

[0198] In some embodiments, multiple classifiers are used in the same POC. The current frame is divided into several regions, each region using the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level, as shown in Table 9 below. [Table 20]

[0199] In some embodiments, the maximum number of classifiers (which may also be called alternative offset sets) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. In one example, the fixed (predefined) maximum number of classifiers is 4. In this case, four different classifiers are used in POC0, and which classifier (0, 1, or 2) is used is signaled at the CTU level. Shortened unary (TU) codes may be used to indicate the classifier used for each luma or chroma CTB. For example, as shown in Table 10 below, when the TU code is 0, CCSAO is not applied; when the TU code is 10, set 0 is applied; when the TU code is 110, set 1 is applied; when the TU code is 1110, set 2 is applied; and when the TU code is 1111, set 3 is applied. To indicate classifiers for CTB (offset set index), fixed-length codes, Golomb-Rice codes, and exponential Golomb codes may be used. Three different classifiers are used in POC1. [Table 21]

[0200] An example of CTB offset set indices for Cb and Cr is given for the 1280×720 sequence POC0 (the number of CTUs in the frame is 10×6 if the CTU size is 128×128). Cb in POC0 uses four offset sets, and Cr uses one offset set. As shown in Table 11-1 below, when the offset set index is 0, CCSAO is not applied; when the offset set index is 1, set 0 is applied; when the offset set index is 2, set 1 is applied; when the offset set index is 3, set 2 is applied; and when the offset set index is 4, set 3 is applied. The type refers to the position of the selected sequence luma sample (Yi). Different offset sets can have different types, band_num, and corresponding offsets. [Table 22]

[0201] In some embodiments, an example of jointly using sequence / current and adjacent Y / U / V samples for classification is given in Table 11-2 below (three-component joint bandNum classification for each Y / U / V component). In POC0, the {2,4,1} offset set is used for {Y,U,V} respectively. Each offset set can be adapted and switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different offset sets can have different classifiers. For example, to classify the current Y4 luma sample as candidate positions (candPos) shown in Figures 6B and 6C, Y set0 selects {current Y4, sequence U4, sequence V4} as candidates, each with different bandNum {Y,U,V}={16,1,2}. When {candY,candU,candV} are used as sample values ​​for the selected {Y,U,V} candidates, the total number of classes is 32, and the class index can be derived as follows: bandY=(candY*bandNumY)>>BitDepth; bandU=(candU*bandNumU)>>BitDepth; bandV=(candV*bandNumV)>>BitDepth; classIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV;

[0202] In some embodiments, the derivation of the classIdx of a coclassifier can be expressed in "or-shift" form to simplify the derivation process. For example, max bandNum={16,4,4}. classIdx=(bandY<<4)|(bandU<<2)|bandV

[0203] Another example is component V set1 classification in POC1. In that example, bandNum={4,1,2} and candPos={adjacent Y8,adjacent U3,adjacent V0} are used, resulting in 8 classes. [Table 23]

[0204] In some embodiments, for example, as shown in Table 11-3 below, one example is the joint use of sequence and adjacent Y / U / V samples in the current Y / U / V sample classification (three-component joint edgeNum(C1) and bandNum classification for each Y / U / V component). EdgeCandPos is the central position used for the C1 classifier, edgebitMask is the C1 adjacent sample activation indicator, and edgeNum is the number of the corresponding C1 class. In this example, C1 is applied only to the Y classifier (thus edgeNum is equal to edgeNumY), and edgecandPos is always Y4 (current / sequence sample position). However, C1 may be applied to Y / U / V classifiers that have edgecandPos as an adjacent sample position.

[0205] If diff represents the C1 comparison score of Y, the derivation of classIdx could be as follows: bandY=(candY*bandNumY)>>BitDepth; bandU=(candU*bandNumU)>>BitDepth; bandV=(candV*bandNumV)>>BitDepth; edgeIdx = diff + (edgeNum >> 1); bandIdx=bandY*bandNumU*bandNumV +bandU*bandNumV +bandV; classIdx=bandIdx*edgeNum+edgeIdx; [Table 24] [Table 25] [Table 26]

[0206] In some embodiments, as discussed above, multiple C0 classifiers can be combined for a single component (combinations of different positions or weights, bandNum) to form a co-classifier. This co-classifier can be combined with other components to form another co-classifier, for example, two Y samples (candY / candX and bandNumY / bandNumX), one U sample (candU and bandNumU), and one V sample (candV and bandNumV) can be used to classify one U sample (Y / V may have the same concept). The class index derivation can be shown as follows: bandY=(candY*bandNumY)>>BitDepth bandX=(candX*bandNumX)>>BitDepth bandU=(candU*bandNumU)>>BitDepth bandV=(candV*bandNumV)>>BitDepth classIdx=bandY*bandNumX*bandNumU*bandNumV +bandX*bandNumU*bandNumV +bandU*bandNumV +gv

[0207] In some embodiments, when using multiple C0s for a single component, several decoder norms or encoder compatibility constraints may be applied. These constraints include (1) selected C0 candidates must be distinct from each other (e.g., candX != candY), and / or (2) newly added bandNums must be smaller than other bandNums (e.g., bandNumX ≤ bandNumY). By applying intuitive constraints within a single component (Y), redundant cases can be eliminated, saving bit cost and complexity.

[0208] In some embodiments, the maximum band_num (bandNumY, bandNumU, or bandNumV) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, if the decoder fixes max band_num=16 for each frame, 4 bits will be signaled to indicate the band_num of C0 in the frame. Some other examples of maximum band_num are listed in Table 12 below. [Table 27]

[0209] In some embodiments, the maximum number of classes or offsets (combinations using multiple classifiers together, e.g., C1 edgeNum*C1 bandNumY*bandNumU*bandNumV) for each set (or all added sets) may be fixed or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, max may be fixed at class_num=256*4 for all added sets, and encoder compatibility checks or decoder normative checks may be used to verify the constraints.

[0210] In some embodiments, the restriction may be applied to the C0 classification, for example, restricting band_num(bandNumY, bandNumU, or bandNumV) to only powers of 2. Instead of explicitly signaling band_num, the syntax band_num_shift is signaled. The decoder can use a shift operation to avoid multiplication. Different band_num_shift may be used for different components. Class (C0) = (Y0 >> band_num_shift) >> bit_depth

[0211] Another example of calculation considers rounding to reduce errors. Class(C0)=((Y0+(1<<(band_num_shift-1)))>>band_num_shift)>>bit_depth

[0212] For example, if band_num_max(Y, U, or V) is 16, the possible band_num_shift candidates are 0, 1, 2, 3, and 4, corresponding to band_num=1, 2, 4, 8, and 16, respectively, as shown in Table 13. [Table 28] [Table 29]

[0213] In some embodiments, different classifiers are applied to Cb and Cr. Cb and Cr offsets for all classes may be signaled separately. For example, different signaled offsets are applied to different chroma components, as shown in Table 14 below. [Table 30]

[0214] In some embodiments, the maximum offset value is fixed or signaled at the Sequence Parameter Set (SPS) / Adaptive Parameter Set (APS) / Picture Parameter Set (PPS) / Picture Header (PH) / Slice Header (SH) / Region / CTU / CU / Subblock / Sample Level. For example, the maximum offset is [-15, 15]. Different components may have different maximum offset values.

[0215] In some embodiments, the offset signaling can be performed using differential pulse code modulation (DPCM). For example, an offset {3,3,2,1,-1} may be signaled as {3,0,-1,-1,-2}.

[0216] In some embodiments, the offset may be stored in an APS or memory buffer for reuse in the next picture / slice. An index may be signaled to indicate which stored previous frame offset is used for the current picture.

[0217] In some embodiments, the Cb and Cr classifiers are the same. For example, as shown in Table 15 below, the Cb and Cr offsets for all classes may be signaled together. [Table 31]

[0218] In some embodiments, the classifiers for Cb and Cr may be the same. For example, as shown in Table 16 below, the Cb and Cr offsets for all classes may be jointly signaled by the sign flag difference. According to Table 16, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,-3,-2,1). [Table 32]

[0219] In some embodiments, the code flags may be signaled for each class, for example, as shown in Table 17 below. According to Table 17, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-3,3,2,1) according to the respective code flag. [Table 33]

[0220] In some embodiments, the classifiers for Cb and Cr may be the same. For example, as shown in Table 18 below, the Cb and Cr offsets for all classes may be jointly signaled by a weight difference. The weights (w) may be selected from a restricted table, for example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4..., where |w| contains only powers of 2. According to Table 18, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-6,-6,-4,2) depending on the respective sign flags. [Table 34]

[0221] In some embodiments, the weights can be signaled for each class, for example, as shown in Table 19 below. According to Table 19, when the Cb offset is (3,3,2,-1), the derived Cr offset is (-6,12,0,-1) according to the respective sign flags. [Table 35]

[0222] In some embodiments, when multiple classifiers are used in the same POC, different offset sets are signaled separately or jointly.

[0223] In some embodiments, previously decoded offsets may be stored for use in future frames. To reduce the overhead of signaling offsets, an index may be signaled to indicate which previously decoded offset set is used for the current frame. For example, as shown in Table 20 below, the POC0 offset may be reused by POC2 by signaling offset set idx=0. [Table 36]

[0224] In some embodiments, the reuse offset set idx for Cb and Cr may be different, as shown in Table 21 below, for example. [Table 37]

[0225] In some embodiments, signaling of offsets may use additional syntax including start and length to reduce signaling overhead. For example, when band_num=256, only offsets between band_idx=37 and 44 are signaled. In the examples in Table 22-1 below, both start and length syntax are 8-bit fixed-length codes that should be aligned with the band_num bit. [Table 38]

[0226] In some embodiments, when CCSAO is applied to all YUV3 components, the sequence and adjacent YUV samples may be used jointly for classification, and the signaling methods for all the above-described offsets for Cb / Cr may be extended to Y / Cb / Cr. In some embodiments, different component offset sets may be stored and used separately (each component has its own stored set), or they may be stored and used jointly (each component shares / reuses the same stored set). Examples of separate sets are shown in Table 22-2 below. [Table 39] JPEG0007840400000062.jpg47170

[0227] In some embodiments, if the sequence bit depth is greater than 10 (or a specific bit depth), the offset may be quantized before signaling. On the decoder side, the decoded offset is dequantized before application, as shown in Table 23-1 below. For example, for a 12-bit sequence, the decoded offset is left-shifted (dequantized) by 2. [Table 40]

[0228] In some embodiments, the offset can be calculated as CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<(BitDepth-Min(10,BitDepth))).

[0229] In some embodiments, the concept of filter strength is further introduced herein. For example, the classifier offset can be further weighted before being applied to the sample. The weight (w) can be selected from a table of values ​​that are powers of 2, e.g., ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| contains only values ​​that are powers of 2. The weight index can be signaled at the SPS / APS / PPS / PH / SH / region (set) / CTU / CU / subblock / sample level. Signaling of the quantized offset can be obtained as part of this weight application. When a recursive CCSAO is applied, as shown in Figure 6D, a similar weight index mechanism can be applied between the first and second stages.

[0230] In some examples, by weighting different classifiers, the offsets of multiple classifiers can be applied to the same sample having a combination of weights. As mentioned above, similar weight indexing mechanisms can be signaled. For example, offset_final = w * offset_1 + (1 - w) * offset_2, or offset_final = w1 * offset_1 + w2 * offset_2 + ...

[0231] In some embodiments, instead of directly signaling the CCSAO parameters within PH / SH, previously used parameters / offsets can be stored in an Adaptive Parameter Set (APS) or memory buffer for reuse in the next picture / slice. An index can be signaled within PH / SH to indicate which stored previous frame offset is used for the current picture / slice. A new APS ID can be created to maintain the CCSAO history offset. The table below shows an example using Figure 6I, candPos, and bandNum{Y,U,V}={16,4,4}. In some examples, candPos, bandNum, and the offset signaling method can be a fixed-length code (FLC), or other methods such as a shortened unary (TU) code, an exponential Golomb code with order k (EGk), signed EG0 (SVLC), or unsigned EG0 (UVLC). In this case, sao_cc_y_class_num (or cb, cr) is equal to sao_cc_y_band_num_y * sao_cc_y_band_num_u * sao_cc_y_band_num_v (or cb, cr). ph_sao_cc_y_aps_id is the parameter index used within this picture / slice. Note that the cb and cr components can follow the same signaling logic. [Table 41] JPEG0007840400000065.jpg185170

[0232] aps_adaptation_parameter_set_id provides an identifier for the APS for reference by other syntactic elements. When aps_params_type is equal to CCSAO_APS, the value of aps_adaptation_parameter_set_id shall be (for example) inclusively within the range of 0 to 7.

[0233] ph_sao_cc_y_aps_id specifies the aps_adaptation_parameter_set_id of the CCSAO APS referenced by the Y color component of the slice in the current picture. When ph_sao_cc_y_aps_id exists, the following applies: The value of sao_cc_y_set_signal_flag for an APS NAL unit having an aps_params_type equal to CCSAO_APS and an aps_adaptation_parameter_set_id equal to ph_sao_cc_y_aps_id shall be equal to 1, and the TemporalId of an APS Network Abstraction Layer (NAL) unit having an aps_params_type equal to CCSAO_APS and an aps_adaptation_parameter_set_id equal to ph_sao_cc_y_aps_id shall be less than or equal to the TemporalId of the current picture.

[0234] In some embodiments, APS update mechanisms are described herein. The maximum number of APS offset sets can be predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Different components may have different maximum number limits. If an APS offset set is full, a newly added offset set can replace one existing stored offset using a first-in, first-out (FIFO), last-in, first-out (LIFO), or most recently unused (LRU) mechanism, or an index value indicating which APS offset set should be replaced is received. In some examples, if the selected classifiers consist of candPos / edge info / coding info, etc., all classifier information can be obtained as part of the APS offset set and can also be stored within the APS offset set having its offset values. In some cases, the update mechanisms described above may be predefined or signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0235] In some embodiments, a constraint called "prunening" can be applied. For example, newly received classifier information and offsets cannot be the same as any of the stored APS offset sets (same or different components).

[0236] In some examples, when the C0 candPos / bandNum classifier is used, the maximum number of APS offset sets is 4 per Y / U / V, with FIFO updates used for Y / V and the idx indicating the update used for U. [Table 42] JPEG0007840400000067.jpg128170

[0237] In some embodiments, the pruning criterion may be relaxed to give a more flexible way for encoder trade-offs, for example, allowing N offsets to be different when applying the pruning operation (e.g., N=4), and in another example, allowing a difference (expressed as "thr") for each offset value when applying the pruning operation (e.g., ±2).

[0238] In some embodiments, two criteria may be applied simultaneously or individually. Whether each criterion is applied is predefined or can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0239] In some embodiments, N / thr can be predefined or can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level.

[0240] In some embodiments, FIFO updates can (1) update previously remaining set idx in a circular fashion, as in the example above (starting again from set 0 if all have been updated), or (2) update from set 0 each time. In some examples, when a new offset set is received, updates can be performed at the PH level (as in the example), or at the SPS / APS / PPS / PH / SH / region / CTU / CU / S subblock / sample level.

[0241] During LRU updates, the decoder maintains a count table that counts the "total number of offset sets used," which can be refreshed at the SPS / APS / Picture Group (GOP) Structure / PPS / PH / SH / Region / CTU / CU / Subblock / Sample level. Newly received offset sets replace recently unused offset sets in the APS. If two stored offset sets have the same number, FIFO / LIFO can be used. For example, see component Y in Table 23-4 below. [Table 43]

[0242] In some embodiments, different components can have different renewal mechanisms.

[0243] In some embodiments, different components (e.g., U / V) can share the same classifier (the same candPos / edge info / coding info / offset may have further weights with modifiers).

[0244] In some embodiments, offset sets using different pictures / slices may have only slight differences in offset values, and therefore, a “patch” implementation may be used in the offset exchange mechanism. In some embodiments, a “patch” implementation is differential pulse code modulation (DPCM). For example, when signaling a new offset set (OffsetNew), the offset value may be located above an existing APS memory offset set (OffsetOld). The encoder signals only the delta value to update the old offset set (DPCM: OffsetNew = OffsetOld + delta). In the following examples shown in Table 23-5, an alternative to FIFO updating (LRU, LIFO, or signaling an index indicating which set should be updated) may be used. The YUV component may have the same update mechanism or use a different update mechanism. In the examples in Table 23-5, the classifier candPos / bandNum does not change, but the override of the set classifier may be indicated by signaling an additional flag (flag=0: update only the set offset, flag=1: update both the set classifier and the set offset). [Table 44] JPEG0007840400000070.jpg29170

[0245] In some embodiments, DPCM delta offset values ​​may be signaled with FLC / TU / EGk (order = 0, 1, ...) codes. For each offset set, one flag may be signaled indicating whether DPCM signaling is enabled. The DPCM delta offset values, or newly added offset values ​​(which are signaled directly without DPCM when APS DPCM=0 is enabled) (ccsao_offset_abs), may be dequantized / mapped before being applied to the target offset (CcSaoOffsetVal). The offset quantization step can be predefined or can be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, one method is to signal the offset directly by quantization step = 2. CcSaoOffsetVal=(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<1) Another approach is to use a DPCM signaling offset by quantization step = 2. CcSaoOffsetVal=CcSaoOffsetVal+(1-2*ccsao_offset_sign_flag)*(ccsao_offset_abs<<1)

[0246] In some embodiments, one constraint may be applied to reduce the overhead of direct offset signaling, for example, the updated offset value must have the same sign as the old offset value. By using such an inferred offset code, the newly updated offset does not need to transmit the code flag again (ccsao_offset_sign_flag is inferred to be the same as the old offset).

[0247] In some embodiments, the sample processing is described below. R(x,y) is the input luma or chroma sample value before CCSAO, and R'(x,y) is the output luma or chroma sample value after CCSAO. offset = ccsao_offset[class_index of R(x, y)] R'(x, y) = Clip3(0, (1 << bit_depth) - 1, R(x, y) + offset)

[0248] According to the above equation, each luma or chroma sample value R(x, y) is classified using the classifier indicated by the current picture and / or the current offset set idx. The corresponding offset of the derived class index is added to each luma or chroma sample value R(x, y). The Clip3 function is applied to (R(x, y) + offset) to create an output luma or chroma sample value R'(x, y) within the bit depth dynamic range, for example, in the range from 0 to (1 << bit_depth) - 1.

[0249] Figure is a block diagram showing that CCSAO is applied and other in-loop filters have different combinations of clipping according to some implementation examples of the present disclosure.

[0250] In some embodiments, when CCSAO is operated by other loop filters, the clipping operation can include the following. (1) Clipping after addition The following equations show (a) an example where CCSAO is operated by SAO and BIF, or (b) an example where CCSAO replaces SAO but is still operated by BIF. (a) I OUT = clip1(I C + ΔI SAO + ΔI BIF ++ ΔI CCSAO ) (b) I OUT = clip1(I C + ΔI CCSAO + ΔI BIF ) (2) Clipping before addition when operated by BIF In some embodiments, the clipping order can be switched. (a) IOUT =clip1(I C +ΔI SAO ) I' OUT =clip1(I OUT +ΔI BIF ) I" OUT =clip1(I” OUT +ΔI CCSAO ) (b)I OUT =clip1(I C +ΔI BIF ) I' OUT =clip1(I' OUT +ΔI CCSAO ) (3) Clipping after partial addition (a)I OUT =clip1(I C +ΔI SAO +ΔI BIF ) I' OUT =clip1(I OUT +ΔI CCSAO )

[0251] In some embodiments, different clipping combinations result in different trade-offs between correction accuracy and hardware temporary buffer size (register or SRAM bit width).

[0252] Figure 13(a) shows SAO / BIF offset clipping. Figure 13(b) shows one additional bit-depth clipping for CCSAO. Figure 13(c) shows co-clipping after adding SAO / BIF / CCSAO offsets to the input samples. More specifically, for example, Figure 13(a) shows the current BIF design when interacting with SAO. Offsets from SAO and BIF are added to the input samples, followed by one bit-depth clipping. However, when CCSAO is also added at the SAO stage, as shown in Figures 13(b) and 13(c), two possible clipping designs can be selected: (1) the addition of one additional bit-depth clipping for CCSAO, and (2) a harmonic design that performs co-clipping after adding SAO / BIF / CCSAO offsets to the input samples. In some embodiments, the clipping designs described above differ only in the luma samples, as BIF is applied only to luma samples.

[0253] In some embodiments, boundary processing is described below. If either the sequence used for classification or the adjacent luma (chroma) sample is outside the current picture, CCSAO is not applied to the current chroma (chroma) sample. Figure 14A is a block diagram showing that, in some implementations of this disclosure, CCSAO is not applied to the current chroma (chroma) sample if either the sequence used for classification or the adjacent luma (chroma) sample is outside the current picture. For example, in Figure 14A(a), when the classifier is used, CCSAO is not applied to the chroma component in the leftmost column of the current picture. For example, when C1' is used, as shown in Figure 14A(b), CCSAO is not applied to the chroma component in the leftmost column and the top row of the current picture.

[0254] Figure 14B is a block diagram showing that, in some implementations of this disclosure, CCSAO is applied to the current luma or chroma sample when either the sequence and adjacent luma or chroma sample used for classification are located outside the current picture. In some embodiments, variation may be applied to the current luma or chroma sample when either the sequence and adjacent luma or chroma sample used for classification are located outside the current picture, as shown in Figure 14B(b), by using a lost sample iteratively or by mirror-padding the lost sample to create a sample for classification, as shown in Figure 14B(a), and CCSAO may be applied to the current luma or chroma sample. In some embodiments, the invalidation / iteration / mirroring picture boundary processing methods disclosed herein may also be applied to a subpicture / slice / tile / CTU / 360 virtual boundary when either the sequence and adjacent luma (chroma) sample used for classification are located outside the current subpicture / slice / tile / CTU / 360 virtual boundary.

[0255] For example, a picture is divided into one or more tile rows and one or more tile columns. A tile is a series of CTUs that cover a rectangular area of ​​the picture.

[0256] A slice consists of a complete tile of integers or a complete row of consecutive integers within a picture tile.

[0257] A subpicture contains one or more slices that collectively cover the rectangular area of ​​the picture.

[0258] In some embodiments, 360-degree video is captured on a sphere, which essentially has no "boundaries," and a reference sample located outside the boundary of a reference picture within the projected domain can always be obtained from an adjacent sample within the spherical domain. In the case of projection formats consisting of multiple faces, discontinuities appear between two or more adjacent faces in the picture packed in the frame, regardless of the type of small frame packing arrangement used. In VVC, vertical and / or horizontal virtual boundaries are introduced, which disable in-loop filtering, and the position of these boundaries is signaled in the SPS or picture header. The use of 360 virtual boundaries is more flexible than using two tiles, one for each pair of consecutive faces, because it does not require the face size to be a multiple of the CTU size. In some embodiments, the maximum number of vertical 360 virtual boundaries is 3, and the maximum number of horizontal 360 virtual boundaries is also 3. In some embodiments, the distance between two virtual boundaries is greater than or equal to the CTU size, and the granularity of the virtual boundaries is 8 luma samples, e.g., an 8x8 sample grid.

[0259] Figure 14C is a block diagram illustrating that, in some implementations of this disclosure, CCSAO does not apply to the current chroma sample if the corresponding selected sequence or adjacent luma sample used for classification is outside the virtual space defined by the virtual boundary. In some embodiments, the virtual boundary (VB) is a virtual line that separates space within a picture frame. In some embodiments, if the virtual boundary (VB) is applied within the current frame, CCSAO does not apply to the selected chroma sample if the corresponding luma position is outside the virtual space defined by the virtual boundary. Figure 14C shows an example of a virtual boundary for a C0 classifier with 9 luma position candidates. For each CTU, CCSAO does not apply to chroma samples where the corresponding selected luma position is outside the virtual space enclosed by the virtual boundary. For example, in Figure 14C(a), when the selected Y7 luma sample position is on the opposite side of the horizontal virtual boundary 1406 located 4 pixels from the bottom edge of the frame, CCSAO does not apply to chroma sample 1402. For example, in Figure 14C(b), when the selected Y5 chroma sample position is located on the opposite side of the vertical virtual boundary 1408, which is located on the line of the y pixels from the right edge of the frame, CCSAO is not applied to the chroma sample 1404.

[0260] Figure 15 shows, through several implementations of this disclosure, that iterative or mirror padding can be applied to chroma samples outside the virtual boundary. Figure 15(a) shows an example of iterative padding. If the original Y7 is selected to be a classifier located at the bottom of VB1502, the Y4 chroma sample value is used for classification (copied to the Y7 position) rather than the original Y7 chroma sample value. Figure 15(b) shows an example of mirror padding. If Y7 is selected to be a classifier located at the bottom of VB1504, the Y1 chroma sample value, which is symmetric to the Y7 value with respect to the Y0 chroma sample, is used for classification rather than the original Y7 chroma sample value. The padding method gives the possibility of applying CCSAO to more chroma samples, and therefore more coding gain can be achieved.

[0261] In some embodiments, limitations may be applied to reduce the line buffer required by CCSAO and to simplify boundary processing condition checks. Figure 16 shows that in some implementations of this disclosure, when all nine array and adjacent luma samples are used for classification, an additional luma line buffer, i.e., all line luma samples of line-5 above VB1602, may be required. Figure 10B(a) shows an example where only six luma candidates are used for classification, which reduces the line buffer and does not require any additional boundary checks as shown in Figures 14A and 14B.

[0262] In some embodiments, using luma samples for CCSAO classification can increase the luma line buffer and therefore increase the decoder hardware implementation cost. Figure 17 shows how, in some implementations of this disclosure, nine luma candidate CCSAOs crossing VB1702 can increase the luma line buffer by two additional values ​​in AVS. For luma and chroma samples above the virtual boundary (VB) 1702, the DBF / SAO / ALF is processed in the current CTU line. For luma and chroma samples below VB1702, the DBF / SAO / ALF is processed in the next CTU line. In the AVS decoder hardware design, the pre-DBF samples from luma lines -4 to -1, the pre-SAO sample from line -5, and the pre-DBF samples from chroma lines -3 to -1, and the pre-SAO sample from line -4 are stored as line buffers for DBF / SAO / ALF processing in the next CTU line. When processing the next CTU line, luma and chroma samples that are not in the line buffer are unavailable. However, for example, at the position of chroma line -3(b), the chroma sample is processed in the next CTU line, but CCSAO requires pre-SAO luma sample lines -7, -6, and -5 for classification. Pre-SAO luma sample lines -7 and -6 are not in the line buffer and are therefore unavailable. Adding pre-SAO luma sample lines -7 and -6 to the line buffer increases the decoder hardware implementation cost. In some examples, luma VB (line -4) and chroma VB (line -3) may be different (not aligned).

[0263] Similar to Figure 17, Figure 18A shows how, in a VVC, nine candidate luma CCSAOs intersecting VB1802 can increase the luma line buffer by one additional line. VB may differ in different standards. In a VVC, luma VB is line-4 and chroma VB is line-2, and therefore nine candidate CCSAOs can increase the luma line buffer by one line.

[0264] In some embodiments, in the first solution, if any of the luma candidates of the chroma sample cross VB (are outside the current chroma sample VB), the CCSAO is disabled for the chroma sample. Figures 19A–19C show that, with some implementation examples of this disclosure, the CCSAO is disabled for the chroma sample in AVS and VVC if any of the luma candidates of the chroma sample cross VB1902 (are outside the current chroma sample VB). Figure 14C also shows some examples of this implementation.

[0265] In some embodiments, in the second solution, iterative padding is used for CCSAO for "crossing VB" chroma candidates from a chroma line that is close to VB and on the opposite side of VB, e.g., chroma line -4. In some embodiments, iterative padding is implemented for "crossing VB" chroma candidates from the chroma line closest to the adjacent chroma below VB. Figures 20A–20C show that in AVS and VVC, CCSAO is enabled using iterative padding for chroma samples if any of the chroma sample chroma candidates cross VB2002 (are outside the current chroma sample VB), by some implementation examples of this disclosure. Figure 14C(a) also shows some examples of this implementation.

[0266] In some embodiments, in the third solution, mirror padding is used for CCSAO from below the luma VB for “crossing VB” luma candidates. Figures 21A–21C show that in some implementations of this disclosure, CCSAO is enabled in AVS and VVC by using mirror padding for the chroma sample when any of the luma candidates of the chroma sample crosses VB2102 (is outside the current chroma sample VB). Figures 14C(b) and 14B(b) also show some examples of this implementation. In some embodiments, in the fourth solution, “double-sided symmetric padding” is used to apply CCSAO. Figures 22A–22B show that in some implementations of this disclosure, CCSAO is enabled by using double-sided symmetric padding for some examples of different CCSAO shapes (e.g., nine luma candidates (Figure 22A) and eight luma candidates (Figure 22B)). For a luma sample set having a central luma sample arranged with chromatic samples, if one side of the luma sample set is outside VB2202, then double-sided symmetrical padding is applied to both sides of the luma sample set. For example, in Figure 22A, luma samples Y0, Y1, and Y2 are outside VB2202, and therefore Y0, Y1, Y2 and Y6, Y7, Y8 are both padded using Y3, Y4, Y5. For example, in Figure 22B, luma sample Y0 is outside VB2202, and therefore Y0 is padded using Y2 and Y7 is padded using Y5.

[0267] Figure 18B illustrates how, in some implementations of this disclosure, sequence or adjacent chroma samples are used to classify the current chroma samples, and the selected chroma candidates may require an additional chroma line buffer as they cross the VB. Solutions 1–4, similar to those described above, may be applied to address this problem.

[0268] Solution 1 is to disable CCSAO for the luma sample if any of the chroma candidates can cross VB.

[0269] Solution 2 involves using iterative padding from the chroma closest to the adjacent chroma below VB for the "crossing VB" chroma candidate.

[0270] Solution 3 involves using mirror padding from below the chroma VB for the "crossing VB" chroma candidate.

[0271] Solution 4 is to use "double-sided symmetric padding". For a candidate set located at the center of a CCSAO sequence chromatic sample, if one side of the candidate set is outside the VB, double-sided symmetric padding is applied to both sides.

[0272] The padding method allows for the application of CCSAO to more luma or chromatic samples, and therefore greater coding gain can be achieved.

[0273] In some embodiments, at the bottom picture (or slice, tile, brick) boundary CTU row, samples below VB are processed in the current CTU row, and therefore the special handling described above (Solutions 1, 2, 3, 4) does not apply to this bottom picture (or slice, tile, brick) boundary CTU row. For example, a 1920×1080 frame is divided into 128×128 CTUs. The frame contains 15×9 CTUs (rounded up). The bottom CTU row is the 15th CTU row. The decoding process is performed row by row, and for each CTU row, it is performed CTU by CTU. Deblocking needs to be applied along the horizontal CTU boundary between the current CTU row and the next CTU row. Inside one CTU, at the bottom 4 / 2 luma / chroma line, DBF samples (in the case of VVC) are processed in the next CTU row and are not available for the CCSAO of the current CTU row, so the VB of the CTB is applied to each CTU row. However, in the bottom CTU row of the picture frame, since there are no more CTU rows remaining, the bottom 4 / 2 luma / chroma line DBF sample is available in the current CTU row and will be DBF processed in the current CTU row.

[0274] In some embodiments, the VB shown in Figures 13 to 22 may be replaced with the boundary of a subpicture / slice / tile / patch / CTU / 360 virtual boundary. In some embodiments, the positions of the chroma and luma samples in Figures 13 to 22 may be switched. In some embodiments, the positions of the chroma and luma samples in Figures 13 to 22 may be replaced with the positions of a first chroma sample and a second chroma sample. In some embodiments, the VB of the ALF within the CTU may generally be horizontal. In some embodiments, the boundary of a subpicture / slice / tile / patch / CTU / 360 virtual boundary may be horizontal or vertical.

[0275] In some embodiments, as illustrated in Figure 16, restrictions may be applied to reduce the line buffer required by the CCSAO and to simplify boundary processing condition checks. Figure 23 illustrates restrictions used by some implementations of this disclosure to classify a limited number of luma candidates. Figure 23(a) shows a restriction to classify only six luma candidates. Figure 23(b) shows a restriction to classify only four luma candidates.

[0276] In some embodiments, application regions are implemented. The CCSAO application region unit can be based on a CTB; that is, the on / off control and CCSAO parameters (classification offset, offset used for set indexing, luma candidate position, band_num, bitmask, etc.) are the same within a single CTB.

[0277] In some embodiments, the application region cannot be aligned to the CTB boundary. For example, the application region is not aligned to the chroma CTB boundary but is shifted. The syntax (on / off control, CCSAO parameters) is still shown for each CTB, but the actual application region is not aligned to the CTB boundary. Figure 24 shows that in some implementation examples of this disclosure, the CCSAO application region is not aligned to the CTB / CTU boundary 2406. For example, the application region is not aligned to the chroma CTB / CTU boundary 2406 but is shifted to the upper left (4,4) relative to VB2408. This unaligned CTB boundary design benefits the deblocking process because the same deblocking parameters are used for each 8x8 deblocking process region.

[0278] In some embodiments, the CCSAO application area unit (mask size) may be variable (larger or smaller than the CTB size), as shown in Table 24. The mask size may differ for different components. The mask size can be switched at the SPS / APS / PPS / PH / SH / area / CTU / CU / subblock / sample level. For example, in PH, a series of mask on / off flags and offset set indices are shown to indicate each CCSAO area information. [Table 45]

[0279] In some embodiments, the CCSAO application region frame divisions can be fixed. For example, the frame is divided into N regions. Figure 25 shows that, in some implementation examples of this disclosure, the CCSAO application region frame divisions can be fixed by CCSAO parameters.

[0280] In some embodiments, each region may have its own region on / off control flag and CCSAO parameters. Also, if the region size is larger than the CTB size, it may have both a CTB on / off control flag and a region on / off control flag. Figures 25(a) and (b) show some examples of dividing a frame into N regions. Figure 25(a) shows a vertical division consisting of four regions. Figure 25(b) shows a square division consisting of four regions. In some embodiments, if the region on / off control flag is off, the CTB on / off flag may be further signaled, similar to the picture-level CTB on the all-on control flag (ph_cc_sao_cb_ctb_control_flag / ph_cc_sao_cr_ctb_control_flag). Otherwise, CCSAO is applied to all CTBs within this region without further signaling of the CTB flag.

[0281] In some embodiments, different CCSAO application regions can share the same region on / off control and CCSAO parameters. For example, in Figure 25(c), regions 0-2 share the same parameters, and regions 3-15 share the same parameters. Figure 25(c) also shows that region on / off control flags and CCSAO parameters can be signaled in Hilbert scan order.

[0282] In some embodiments, a CCSAO application area unit can be a quadtree / binary / ternary tree partitioned from the picture / slice / CTB level. Similar to CTB partitioning, a set of partitioned flags are signaled to indicate the CCSAO application area partition. Figure 26 shows that, with some implementation examples of this disclosure, a CCSAO application area can be a binary (BT) / quadtree (QT) / ternary tree (TT) partitioned from the frame / slice / CTB level.

[0283] Figure 27 is a block diagram showing multiple classifiers used and switched at different levels within a picture frame by several implementations of this disclosure. In some embodiments, when multiple classifiers are used in a single frame, the method of applying the classifier set index can be switched at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. For example, four classifier sets are used in a frame and switched at PH, as shown in Table 25 below. Figures 27(a) and 27(c) show the default fixed region classifier. Figure 27(b) shows that the classifier set index is signaled at the mask / CTB level, where 0 means CCSAO off for this CTB and 1-4 means set index. [Table 46]

[0284] In some embodiments, for a default region, the CTB within this region does not use the default set index (e.g., the region-level flag is 0), but if other classifier sets are used within this frame, the region-level flag may be signaled. For example, if the default set index is used, the region-level flag is 1. For example, in four regions of a square division, the following classifier sets are used, as shown in Table 26-1 below. [Table 47]

[0285] Figure 28 is a block diagram illustrating that, in some implementations of this disclosure, the CCSAO application area divisions may be dynamic and can be switched at the picture level. For example, Figure 28(a) shows that three CCSAO offset sets are used in this POC (set_num=3), and therefore the picture frame is divided vertically into three regions. Figure 28(b) shows that four CCSAO offset sets are used in this POC (set_num=4), and therefore the picture frame is divided horizontally into four regions. Figure 28(c) shows that three CCSAO offset sets are used in this POC (set_num=3), and therefore the picture frame is raster-divided into three regions. Each region may have its own all-on flag to conserve CTB on / off control bits. The number of regions depends on the signaled picture set_num. The CCSAO application area can be a specific region according to the encoding information within the block (sample position, sample encoding mode, loop filter parameters, etc.). For example, (1) the CCSAO application area can be applied only when the sample is encoded in skip mode, or (2) the CCSAO application area includes only N samples along the CTU boundary, or (3) the CCSAO application area includes only samples on an 8x8 grid within the frame, or (4) the CCSAO application area includes only filtered samples in DBF, or (5) the CCSAO application area includes only the top M and left N rows within the CU, or (6) the CCSAO application area includes only intra-encoded samples, or (7) the CCSAO application area includes only samples within a cbf=0 block, or (8) the CCSAO application area is located only on a block having a block QP of [N,M], where (N,M) can be predefined, or it can be signaled at the SPS / APS / PPS / PH / SH / region / CTU / CU / subblock / sample level. Intercomponent coding information may be taken into consideration, and (9) the CCSAO application region is on the chromatic sample where the sequence luma sample is in the cbf=0 block.

[0286] In some embodiments, whether or not to introduce coding information application area restrictions can be predefined, or signaled in a single control flag at the SPS / APS / PPS / PH / SH / area (per alternate set) / CTU / CU / subblock / sample level to indicate whether specified coding information is included / excluded in CCSAO application. The decoder omits CCSAO processing for those areas according to the predefined conditions or control flags. For example, YUV uses different predefined / flag control conditions that switch at the area (set) level. CCSAO application decisions can be made at the CU / TU / PU or sample level. [Table 48]

[0287] Another example is reusing all or part of a two-way enablement constraint (which is predefined). bool isInter=(currCU.predMode==MODE_INTER)?true:false; if(ccSaoParams.ctuOn[ctuRsAddr] &&((TU::getCbf(currTU,COMPONENT_Y)||isInter==false)&&(currTU.cu->qp>17)) &&(128>std::max(currTU.lumaSize().width,currTU.lumaSize().height)) &&((isInter==false)||(32>std::min(currTU.lumaSize().width,currTU.lumaSize().height))))

[0288] In some embodiments, excluding specific regions may be advantageous for CCSAO statistical collection. The offset derivation may be more precise or preferable for those regions that truly need correction. For example, blocks with cbf=0 usually mean that blocks that do not need further correction are fully predicted. Excluding those blocks may be advantageous for the offset derivation of other regions.

[0289] Different application areas can use different classifiers. For example, in a CTU, skip mode uses C1, 8x8 grid uses C2, and skip mode and 8x8 grid use C3. For example, in a CTU, skip mode encoded samples use C1, CU-centered samples use C2, and samples encoded in skip mode at the CU center use C3. Figure 29 shows how, by some implementation examples of this disclosure, the CCSAO classifier can take into account currently or between components encoding information. For example, different encoding modes / parameters / sample locations can form different classifiers. Different encoding information can be combined to form a co-classifier. Different regions can use different classifiers. Figure 29 also shows another example of an application area.

[0290] In some embodiments, a predefined or flag-controlled "coding information exclusion zone" mechanism can be used in DBF / SAO pre / SAO / BIF / CCSAO / ALF / CCALF / NN loop filter (NNLF) or other loop filters.

[0291] In some embodiments, the implemented CCSAO syntax is shown in Table 27 below. In some examples, the binarization of each syntactic element may be changed. In AVS3, the term patch is analogous to slice, and patch headers are analogous to slice headers. FLC represents a fixed-length code. TU represents a shortened unary code. EGk represents an exponential Golomb code of degree k, where k may be fixed. SVLC represents a signed EG0. UVLC represents an unsigned EG0. [Table 49] JPEG0007840400000076.jpg213170

[0292] If a higher-order flag is off, lower-order flags can be inferred from the off state of the flags and do not need to be signaled. For example, if ph_cc_sao_cb_flag is false in this picture, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag do not exist and are inferred to be false.

[0293] In some embodiments, the ccsao_enabled_flag of the SPS is adjusted with the SAO enablement flag of the SPS, as shown in Table 28 below. [Table 50]

[0294] In some embodiments, ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag indicate whether to enable granularity of Cb / Cr CTB on / off control. If ph_cc_sao_cb_ctb_control_flag and ph_cc_sao_cr_ctb_control_flag are enabled, ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag may be further signaled. Otherwise, whether CCSAO is applied to the current picture depends on ph_cc_sao_cb_flag and ph_cc_sao_cr_flag, and ctb_cc_sao_cb_flag and ctb_cc_sao_cr_flag are not further signaled at the CTB level.

[0295] In some embodiments, for ph_cc_sao_cb_type and ph_cc_sao_cr_type, flags may be further signaled to distinguish whether the central array luma position (Y0 position in Figure 10) is used for classification to the chroma sample, in order to reduce bit overhead. Similarly, if cc_sao_cb_type and cc_sao_cr_type are signaled at the CTB level, flags may be further signaled by the same mechanism. For example, if there are 9 candidate C0 luma positions, cc_sao_cb_type0_flag is further signaled to distinguish whether the central array luma position is used, as shown in Table 29 below. If the central array luma position is not used, cc_sao_cb_type_idc is used to indicate which of the remaining 8 adjacent luma positions is used. [Table 51]

[0296] Table 30 below shows an example of how a single (set_num=1) or multiple (set_num>1) classifier is used within a frame in AVS. The syntactic notation can be mapped to the notation used above. [Table 52] [Table 53]

[0297] When combined with Figure 25 or Figure 27, where each region has its own set, an example syntax may include a region on / off control flag (picture_ccsao_lcu_control_flag[compIdx][setIdx]), as shown in Table 31 below. [Table 54]

[0298] In some embodiments, for high-level syntax, pps_ccsao_info_in_ph_flag and gci_no_sao_constraint_flag can be added.

[0299] In some embodiments, a pps_ccsao_info_in_ph_flag equal to 1 indicates that CCSAO filter information may be present within the PH syntax structure but not within the slice header referencing the PPS, which does not contain the PH syntax structure. A pps_ccsao_info_in_ph_flag equal to 0 indicates that CCSAO filter information is not present within the PH syntax structure but may be present within the slice header referencing the PPS. When it is not present, the value of pps_ccsao_info_in_ph_flag is inferred to be equal to 0.

[0300] In some embodiments, a gci_no_ccsao_constraint_flag equal to 1 specifies that sps_ccsao_enabled_flag equals 0 for all pictures in OlsInScope. A gci_no_ccsao_constraint_flag equal to 0 imposes no such constraint. In some embodiments, a video bitstream contains one or more output layer sets (OLS) according to rules. In the examples herein, OlsInScope refers to one or more OLS in scope. In some examples, the profile_tier_level() syntax structure provides level information and optionally provides profile, tier, subprofile, and general constraint information that OlsInScope adheres to. When the profile_tier_level() syntax structure is contained within a VPS, OlsInScope is one or more OLS specified by the VPS. When the profile_tier_level() syntax structure is contained within an SPS, OlsInScope is an OLS containing only the lowest layer among the layers referencing the SPS, and this lowest layer is an independent layer.

[0301] In some embodiments, extensions to intra and interpredictive post-SAO filters are further shown below. In some embodiments, the SAO classification method disclosed herein (including the classification of intercomponent sample / coding information) can function as a postpredictive filter, and the prediction can be intra, inter, or other prediction tools, such as intrablock copy. Figure 30 is a block diagram showing the SAO classification method disclosed herein functioning as a postpredictive filter in some implementation examples of the disclosure.

[0302] In some embodiments, a corresponding classifier is selected for each Y, U, and V component. For each component predicted sample, the corresponding classifier is first classified, and a corresponding offset is added. For example, each component can use the current sample and adjacent samples for classification. As shown in Table 32 below, Y uses the current Y sample and adjacent Y samples, and U / V uses the current U / V sample for classification. Figure 31 is a block diagram showing that, in some implementations of this disclosure, each component can use the current sample and adjacent samples for classification for the post-predicted SAO filter. [Table 55]

[0303] In some embodiments, improved prediction samples ('Ypred', 'Upred', 'Vpred') are updated by adding the corresponding class offset and then used for intra, inter, or other predictions.

[0304] Ypred'=clip3(0,(1< <bit_depth)-1,Ypred+h_Y[i])

[0305] Upred'=clip3(0,(1< <bit_depth)-1,Upred+h_U[i])

[0306] Vpred'=clip3(0,(1< <bit_depth)-1,Vpred+h_V[i])

[0307] In some embodiments, in addition to the current chromatic components, an inter-component (Y) may be used for further offset classification of the U and V components of the chromatic. For example, an additional inter-component offset (h'_U, h'_V) may be added to the current component offset (h_U, h_V), as shown in Table 33 below. [Table 56]

[0308] In some embodiments, improved prediction samples (Upred, Vpred) are updated by adding a corresponding class offset and then used for intra, inter, or other predictions.

[0309] Upred"=clip3(0,(1< <bit_depth)-1,Upred’+h’_U[i])

[0310] Vpred”=clip3(0,(1< <bit_depth)-1,Vpred’+h’_V[i])

[0311] In some embodiments, intra and inter predictions can use different SAO filter offsets.

[0312] Figure 32 is a block diagram showing how the SAO classification method disclosed herein acts as a reconstructed filter in some implementation examples of this disclosure.

[0313] In some embodiments, the SAO / CCSAO classification methods disclosed herein (including the classification of inter-component sample / coding information) can act as filters applied to reconstructed samples of tree units (TUs). As shown in Figure 32, the CCSAO can act as a post-reconstruction filter, i.e., the reconstructed sample (after the addition of predicted / residual samples, before deblocking) can be used as input for classification, compensating for luma / chroma samples before proceeding to adjacent intra / inter prediction. The CCSAO post-reconstruction filter can reduce the distortion of the current TU sample and provide better predictions for adjacent intra / inter blocks. With more precise predictions, better compression efficiency can be expected.

[0314] Figure 33 is a flowchart illustrating an exemplary process 3300 for decoding a video signal using inter-component correlations relating to some implementations of the present disclosure.

[0315] In one embodiment, a video decoder 30 (shown in Figure 3) receives a picture frame containing a first component and a second component from a video signal (3310).

[0316] The video decoder 30 determines a classifier for each sample of the second component according to the edge direction and one or more edge strengths of the sequence sample of the first component or the current sample of the second component for each sample of the second component (3320). In some embodiments, determining a classifier for each sample of a second component according to the edge direction and one or more edge intensities (3320) includes selecting an edge direction in the arranged or current sample (3320-1), identifying a first adjacent sample and a second adjacent sample of the arranged or current sample (3320-2), calculating a first edge intensity by determining the difference in values ​​of two samples from a group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample (3320-3), quantizing the first edge intensity into M segments according to M-1 thresholds T(1),...T(M-1), where M is a positive integer (3320-4), and determining a classifier based on the M segments (3320-5).

[0317] The video decoder 30 determines the sample offset for each sample of the second component according to the classifier (3330).

[0318] The video decoder 30 corrects each sample of the second component (3340) based on the determined sample offset.

[0319] In some embodiments, the first component is a component selected from the group consisting of a luma component, a first chromatic component, and a second chromatic component, and the second component is a component selected from the group consisting of a luma component, a first chromatic component, and a second chromatic component.

[0320] In some embodiments, determining a classifier based on M segments (3320-5) includes determining a classifier using M classes corresponding to the M segments.

[0321] In some embodiments, quantizing the first edge intensity into M segments according to M-1 thresholds T(1),...T(M-1) (3320-4) is the first edge intensity into segments [-2 b +1,T(1)], [T(1),T(2)], ...[T(M-1),2 b This includes quantizing to -1], and the first edge strength is [-2 b +1,2 b The range is -1], where b is the sequence bit depth of the video signal. "[" or "]" indicates a boundary value, for example, T(1) is contained within a particular segment, e.g., [T(1), T(2)], and "(" or ")" indicates a boundary value, for example, T(2) is not contained within a particular segment, e.g., [T(1), T(2)].

[0322] In some embodiments, two samples from a group consisting of an arranged or current sample, a first neighboring sample, and a second neighboring sample are the arranged or current sample and the first neighboring sample. In some embodiments, determining a classifier for each sample of the second component according to the edge direction and one or more edge intensities (3320) further includes calculating a second edge intensity by determining the difference between the value of the arranged or current sample and the value of the second neighboring sample, quantizing the second edge intensity into N segments according to N-1 thresholds T(1),...T(N-1), where N is a positive integer, and determining a classifier based on the N segments.

[0323] In some embodiments, determining a classifier based on an additional N segments includes determining a classifier using M × N classes corresponding to the combined M × N segments.

[0324] In some embodiments, M is not equal to N, or M is equal to N.

[0325] In some embodiments, the thresholds T(1),...T(M-1) are symmetrically distributed with 0 as the center of symmetry.

[0326] In some embodiments, two samples from a group consisting of an arranged or current sample, a first adjacent sample, and a second adjacent sample are the first adjacent sample and the second adjacent sample.

[0327] In some embodiments, selecting an edge direction in an arranged or current sample (3320-1) includes selecting an edge direction from a group consisting of edge directions of 0, 22.5, 45, 67.5, 90, 112.5, and 135 degrees.

[0328] In some embodiments, the edge direction, M-1 thresholds, sample distance from the sequenced or current sample to the first adjacent sample, and sample distance from the sequenced or current sample to the second adjacent sample are predefined or signaled at one or more levels of the sequence parameter set (SPS), adaptive parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, coding tree unit (CTU), coding unit (CU), subblock, and sample level.

[0329] In some embodiments, determining a classifier for each sample of the second component (3320) is further done according to a band classifier, which is determined by utilizing sample values ​​based on the sequence and adjacent samples of the first component for each sample of the second component, as well as one or more samples from current and adjacent samples of the second component, dividing the range of sample values ​​into several bands, and selecting bands based on the intensity values ​​of the sample values.

[0330] Figure 34 shows a computing environment 3410 coupled to a user interface 3450. The computing environment 3410 can be part of a data processing server. The computing environment 3410 includes a processor 3420, memory 3430, and an input / output (I / O) interface 3440.

[0331] The processor 3420 typically controls the overall operation of the computing environment 3410, including operations associated with display, data acquisition, data communication, and image processing. The processor 3420 may include one or more processors to execute instructions for performing all or some of the steps in the aforementioned method. Furthermore, the processor 3420 may include one or more modules that facilitate interaction between the processor 3420 and other components. The processor can be a central processing unit (CPU), a microprocessor, a single-chip machine, a graphical processing unit (GPU), and the like.

[0332] Memory 3430 is configured to store various types of data to support the operation of the computing environment 3410. Memory 3430 may contain certain software 3432. Examples of such data include instructions for any application or method to be operated on the computing environment 3410, video datasets, image data, etc. Memory 3430 can be implemented using any type of volatile or non-volatile memory device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

[0333] The I / O interface 3440 provides an interface between the processor 3420 and peripheral interface modules such as a keyboard, click wheel, and buttons. The buttons may include, but are not limited to, a home button, a scan start button, and a scan stop button. The I / O interface 3440 can be coupled to an encoder and decoder.

[0334] In one embodiment, a non-temporary computer-readable storage medium is also provided, which contains, for example, a memory 3430, a plurality of programs executable by a processor 3420 within a computing environment 3410 to perform the method described above. Alternatively, the non-temporary computer-readable storage medium may store a bitstream or datastream containing encoded video information (e.g., video information including one or more syntactic elements), which is generated by an encoder (e.g., a video encoder 20 in Figure 2) using the above-described encoding method, which is used, for example, by a decoder (e.g., a video decoder 30 in Figure 3) when decoding video data. The non-temporary computer-readable storage medium can be, for example, a ROM, random-access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage device.

[0335] In one embodiment, a computing device is also provided comprising one or more processors (e.g., processor 3420) and a non-temporary computer-readable storage medium or memory 3430 storing a plurality of programs executable by the one or more processors, wherein the one or more processors are configured to perform the method described above when executing the plurality of programs.

[0336] In one embodiment, a computer program product is also provided that includes, for example, a plurality of programs executable by a processor 3420 within a computing environment 3410 to perform the method described above, contained in memory 3430. For example, the computer program product may include a non-temporary computer-readable storage medium.

[0337] In one embodiment, the computing environment 3410 may be implemented by one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs, GPUs, controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0338] Further embodiments also include various subsets of the above embodiments combined or otherwise rearranged in various other embodiments.

[0339] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium, or transmitted through a computer-readable medium, and executed by a hardware-based processing unit. The computer-readable medium may include computer-readable storage media corresponding to tangible media such as data storage media, or communication media including any medium that facilitates the transmission of computer programs from one place to another according to a communication protocol, for example. Thus, the computer-readable medium can generally correspond to (1) non-transient tangible computer-readable storage media, or (2) communication media such as signals or carrier waves. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementation of the implementation examples described in this application. Computer program products may include computer-readable media.

[0340] The descriptions in this disclosure are provided for illustrative purposes only and are not intended to be exhaustive or limitful. Many modifications, variations, and alternative implementations will be apparent to those skilled in the art who benefit from the teachings presented in the above description and the accompanying drawings.

[0341] Unless otherwise specifically stated, the order of the steps of the methods provided herein is for illustrative purposes only, and the steps of the methods provided herein are not specifically limited to the order described above and may be modified in accordance with practical conditions. In addition, at least one of the steps of the methods provided herein may be adjusted, combined, or deleted in accordance with practical requirements.

[0342] The examples are selected and described to illustrate the principles of this disclosure and to enable those skilled in the art to understand this disclosure in relation to various implementations and to make the best use of the underlying principles and various implementations, along with various modifications suitable for specific intended applications. Therefore, it should be understood that the scope of this disclosure is not limited to the specific examples disclosed, and that modifications and other implementations are also intended to be included within the scope of this disclosure.

Claims

1. A method for decoding a video signal, Receiving a picture frame containing a first component and a second component from the aforementioned video signal, Determining the classifier of the second component for each of the samples according to the edge direction and one or more edge strengths of the sequence sample of the first component or the current sample of the second component for each of the samples of the second component, The sample offset of the second component for each of the samples is determined according to the classifier, This includes correcting each of the samples of the second component based on the determined sample offset, The classifier for each of the samples of the second component is determined according to the edge direction and the one or more edge strengths. Selecting the edge direction in the arranged or current sample, Identifying the first and second adjacent samples of the arranged or current sample, The first edge strength is calculated by determining the difference between the values ​​of two samples from a group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample. The first edge intensity is quantized into M segments according to M-1 thresholds T(1), . . T(M-1), wherein M is an integer of 2 or more. A method comprising determining the classifier based on the M segments.

2. The first component is a component selected from the group consisting of a luma component, a first chroma component, and a second chroma component. The second component is a component selected from the group consisting of the luma component, the first chroma component, and the second chroma component. The method according to claim 1.

3. The method according to claim 1, wherein determining the classifier based on the M segments includes determining the classifier using M classes corresponding to the M segments.

4. Quantizing the first edge intensity into M segments according to M-1 thresholds T(1), . . . T(M-1) is such that the first edge intensity is segmented [-2 ｂ +1, T(1)], [T(1), T(2)], . .. .. [T(M-1), 2 ｂ This includes quantizing to -1, wherein the first edge strength is [-2 ｂ +1, 2 ｂ The method according to claim 1, wherein b has a range of -1, and b is the sequence bit depth of the video signal.

5. The two samples from the group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample are the arranged or current sample and the first adjacent sample, The classifier for each of the samples of the second component is determined according to the edge direction and the one or more edge strengths. The second edge strength is calculated by determining the difference between the values ​​of the arranged or current samples and the values ​​of the second adjacent samples. The second edge intensity is quantized into N segments according to N-1 thresholds T(1), ... T(N-1), wherein N is an integer of 2 or more. The method according to claim 1, further comprising determining the classifier based on the N segments.

6. The method according to claim 5, further comprising determining the classifier based on the N segments by using M × N classes corresponding to the combined M × N segments.

7. The method according to claim 5, wherein M is not equal to N, or M is equal to N.

8. The method according to claim 1, wherein the threshold values ​​T(1), . . T(M-1) are symmetrically distributed with 0 as the center of symmetry.

9. The method according to claim 1, wherein the two samples from the group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample are the first adjacent sample and the second adjacent sample.

10. The method according to claim 1, wherein selecting the edge direction in the arranged or current sample includes selecting the edge direction from a group consisting of edge directions of 0, 22.5, 45, 67.5, 90, 112.5, and 135 degrees.

11. The method according to claim 1, wherein the edge direction, the M-1 thresholds, the sample distance from the arranged or current sample to the first adjacent sample, and the sample distance from the arranged or current sample to the second adjacent sample are predefined or signaled at one or more of the levels of sequence parameter set (SPS), adaptive parameter set (APS), picture parameter set (PPS), picture header (PH), slice header (SH), region, coding tree unit (CTU), coding unit (CU), subblock, and sample.

12. The method according to claim 1 or 5, wherein determining the classifier for each of the samples of the second component is further performed according to a band classifier, the band classifier is determined by utilizing sample values ​​based on the sequence and adjacent samples of the first component for each of the samples of the second component and one or more samples from the current and adjacent samples of the second component, dividing the range of sample values ​​into several bands, and selecting bands based on the intensity values ​​of the sample values.

13. One or more processing units, A memory connected to one or more processing units, An electronic device comprising a plurality of programs stored in the memory, wherein when the plurality of programs are executed by one or more processing units, the electronic device is made to perform the method according to any one of claims 1 to 11. electronic equipment.

14. Non-temporary computer-readable storage medium and A processor configured to perform an encoding method to generate and transmit a bitstream, Equipped with, The aforementioned encoding method is Obtain a picture frame containing the first and second components, Determining the classifier of the second component for each of the samples according to the edge direction and one or more edge strengths of the sequence sample of the first component or the current sample of the second component for each of the samples of the second component, The sample offset of the second component for each of the samples is determined according to the classifier, This includes correcting each of the samples of the second component based on the determined sample offset, The classifier for each of the samples of the second component is determined according to the edge direction and the one or more edge strengths. Selecting the edge direction in the arranged or current sample, Identifying the first and second adjacent samples of the arranged or current sample, The first edge strength is calculated by determining the difference between the values ​​of two samples from a group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample. The first edge intensity is quantized into M segments according to M-1 thresholds T(1), . . T(M-1), wherein M is an integer of 2 or more. This includes determining the classifier based on the M segments, An electronic device wherein the bitstream is decoded by the decoding method described in any one of claims 1 to 11.

15. A non-temporary computer-readable storage medium for storing a plurality of programs for execution by an electronic device having one or more processing units, wherein when the plurality of programs are executed by the one or more processing units, the non-temporary computer-readable storage medium causes the electronic device to execute the method according to any one of claims 1 to 11.

16. A computer program product for video coding, which, when executed by one or more computer processors, stores computer-executable instructions that cause the one or more computer processors to perform the method according to any one of claims 1 to 11.

17. A method for storing a bitstream, This involves executing an encoding method to generate a bitstream, The aforementioned encoding method is Obtain a picture frame containing the first and second components, Determining the classifier of the second component for each of the samples according to the edge direction and one or more edge strengths of the sequence sample of the first component or the current sample of the second component for each of the samples of the second component, The sample offset of the second component for each of the samples is determined according to the classifier, This includes correcting each of the samples of the second component based on the determined sample offset, The classifier for each of the samples of the second component is determined according to the edge direction and the one or more edge strengths. Selecting the edge direction in the arranged or current sample, Identifying the first and second adjacent samples of the arranged or current sample, The first edge strength is calculated by determining the difference between the values ​​of two samples from a group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample. The first edge intensity is quantized into M segments according to M-1 thresholds T(1), . . T(M-1), wherein M is an integer of 2 or more. This includes determining the classifier based on the M segments, Includes storing the bitstream, A method wherein the bitstream is decoded by the decoding method described in any one of claims 1 to 11.

18. A method for transmitting a bitstream, This involves executing an encoding method to generate a bitstream, The aforementioned encoding method is Obtain a picture frame containing the first and second components, Determining the classifier of the second component for each of the samples according to the edge direction and one or more edge strengths of the sequence sample of the first component or the current sample of the second component for each of the samples of the second component, The sample offset of the second component for each of the samples is determined according to the classifier, This includes correcting each of the samples of the second component based on the determined sample offset, The classifier for each of the samples of the second component is determined according to the edge direction and the one or more edge strengths. Selecting the edge direction in the arranged or current sample, Identifying the first and second adjacent samples of the arranged or current sample, The first edge strength is calculated by determining the difference between the values ​​of two samples from a group consisting of the arranged or current sample, the first adjacent sample, and the second adjacent sample. The first edge intensity is quantized into M segments according to M-1 thresholds T(1), . . T(M-1), wherein M is an integer of 2 or more. This includes determining the classifier based on the M segments, This includes transmitting the bitstream, A method wherein the bitstream is decoded by the decoding method described in any one of claims 1 to 11.

Images