Methods, apparatuses, and media for video encoding

Abstract

The present disclosure relates to a method, apparatus and medium for video coding. The method comprises: obtaining a first image frame comprising a first component and a second component, wherein the first component is a luma component and the second component is a chroma component; determining a plurality of sample offsets associated with the second component in the first image frame; deriving a first type of index for the second component from a first set of one or more samples of the first component, wherein the first set of one or more samples of the first component corresponds to each respective sample of the second component; selecting a first sample offset from the plurality of sample offsets for the second component according to the first type of index; and obtaining a cross-component offset post-sample value of the second component based on the first sample offset.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202110926995.4, which claims priority to U.S. Patent Application No. 63 / 065,351, filed on August 13, 2020. Technical Field

[0002] This application generally relates to video encoding and decoding and compression, and more specifically, to methods and apparatus for improving the efficiency of chroma encoding and decoding. Background Technology

[0003] Digital video is supported by a wide variety of electronic devices, including digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video codec standards include Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), which were jointly developed by ISO / IEC MPEG and ITU-T VCEG. AOMedia Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its previous standard, VP9. Audio Video Coding (AVS) (which refers to digital audio and digital video compression standards) is another series of video compression standards developed by the Audio and Video Coding Standard Workgroup of China.

[0004] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or eliminate redundancy inherent in video data. For block-based video coding and decoding, video frames are divided into one or more stripes, each strip containing multiple video blocks, which may also be referred to as Coding Tree Units (CTUs). Each CTU may contain one Coding Unit (CU) or be recursively divided into smaller CUs until a preset minimum CU size is reached. Each CU (also called a leaf CU) contains one or more Transform Units (TUs), and each CU also contains one or more Prediction Units (PUs). Each CU can be encoded and decoded in intra-frame, inter-frame, or IBC modes. Spatial prediction is used to encode video blocks in the intra-frame coding and decoding (I) stripes of a video frame relative to reference samples in adjacent blocks within the same video frame. Video blocks in the inter-frame encoding and decoding (P or B) stripe of a video frame can use spatial prediction relative to reference samples in adjacent blocks within the same video frame or temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0005] A prediction block for the current video block to be encoded is generated based on spatial or temporal predictions of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the motion vector pointing to the reference block in the reference frame that forms the prediction block, and the residual block. The process of determining the motion vector is typically called motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain, such as the frequency domain, to generate residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged as a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, and then entropy-encoded into a video bitstream for further compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities, or transmitted directly to the electronic device via wired or wireless means. The electronic device then performs video decompression (the reverse of the video compression process described above) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and renders the reconstructed digital video data on the display of the electronic device.

[0007] As digital video quality progresses from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while encoding / decoding video data more efficiently remains a challenge. Summary of the Invention

[0008] This application describes implementations of methods and apparatus relating to video data encoding and decoding, and more particularly to improving the encoding and decoding efficiency of chroma encoding and decoding (including improving encoding and decoding efficiency by exploring cross-component relationships between luminance and chroma components).

[0009] According to a first aspect of this application, a method for decoding a video signal includes: receiving a video signal comprising a first component and a second component in a first image frame; receiving from the video signal a plurality of sample offsets associated with the second component in the first image frame; obtaining a first classifier for the second component from a first set of one or more samples of the first component relative to each sample of the second component; selecting a first sample offset from the plurality of sample offsets for the second component according to the first classifier; and modifying the second component based on the first sample offset in the first image frame.

[0010] In some embodiments, the method of decoding a video signal further includes: obtaining a second classifier for the second component in a second region of a first image frame from a second set of one or more samples of the first component relative to each sample of the second component; selecting a second sample offset from a plurality of sample offsets for the second component according to the second classifier; and modifying the second component based on the second sample offset in the second region of the first image frame; wherein the first classifier for the second component in the first image frame is obtained in a first region of the first image frame.

[0011] In some embodiments, the first image frame is divided into multiple regions, and a different classifier is used for each of the multiple regions.

[0012] According to a second aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by the one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above.

[0013] According to a third aspect of this application, a non-transitory computer-readable storage medium stores multiple programs for execution by an electronic device having one or more processing units. When executed by the one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above. Attached Figure Description

[0014] The accompanying drawings, included to provide a further understanding of the embodiments and incorporated herein by reference and forming part of the specification, illustrate the described embodiments and, together with the specification, serve to explain the underlying principles. Similar reference numerals denote corresponding parts.

[0015] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0016] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0017] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0018] Figures 4A to 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0019] Figure 5 This is a block diagram depicting four gradient modes used in Sample Offset Compensation (SAO) according to some embodiments of the present disclosure.

[0020] Figure 6A This is a block diagram illustrating a system and process for cross-component sample offset compensation (CCSAO) according to some embodiments of the present disclosure.

[0021] Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with ESAO in the AVS standard, according to some embodiments of this disclosure.

[0022] Figure 7 This is a block diagram illustrating a sample process using CCSAO according to some embodiments of the present disclosure.

[0023] Figure 8 This is a block diagram illustrating a CCSAO process according to some embodiments of the present disclosure, interleaved with vertical and horizontal deblocking filters (DBF).

[0024] Figure 9 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0025] Figure 10 This is a block diagram illustrating a classifier that classifies samples using different brightness locations according to some embodiments of the present disclosure.

[0026] Figure 11This is a block diagram illustrating a sample process according to some embodiments of the present disclosure, in which, in addition to luminance, other cross-component isotopic and adjacent chromaticity samples are also fed into the CCSAO classification.

[0027] Figure 12 The illustration shows an exemplary classifier according to some embodiments of the present disclosure, which replaces the same brightness sample value with a value obtained by weighting the same brightness sample and the adjacent brightness sample.

[0028] Figure 13A This is a block diagram illustrating how, according to some embodiments of the present disclosure, CCSAO is not applied to the current chromaticity sample if either the co-position luminance sample or the adjacent luminance sample used for classification is outside the current image.

[0029] Figure 13B This is a block diagram illustrating the application of CCSAO to the current chromaticity sample if either the co-occurrence luminance sample or the adjacent luminance sample used for classification is outside the current image, according to some embodiments of the present disclosure.

[0030] Figure 14 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation in an image frame according to some embodiments of the present disclosure. Detailed Implementation

[0031] Reference will now be made in detail to the specific embodiments illustrated in the accompanying drawings. Numerous non-limiting specific details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0032] The first generation of AVS standards included the Chinese national standards "Information Technology Advanced Audio and Video Coding Part 2: Video" (referred to as AVS1) and "Information Technology Advanced Audio and Video Coding Part 16: Broadcast Television Video" (referred to as AVS+). Compared to the MPEG-2 standard, the first generation of AVS standards could provide approximately 50% bitrate savings while maintaining the same perceived quality. The second generation of AVS standards included the Chinese national standards "Information Technology High-Efficiency Multimedia Coding" (referred to as AVS2), primarily targeting the transmission of additional HD TV programs. AVS2's coding and decoding efficiency was twice that of AVS+. Simultaneously, the video portion of the AVS2 standard was submitted as an international application standard by the Institute of Electrical and Electronics Engineers (IEEE). The AVS3 standard is a new generation of video coding and decoding standards for UHD video applications, aiming to surpass the coding and decoding efficiency of the latest international standard HEVC. The AVS3 standard offers approximately 30% more bitrate savings than the HEVC standard. In March 2019, at the 68th AVS conference, the AVS3-P2 baseline was completed, offering approximately 30% bitrate savings compared to the HEVC standard. Currently, the AVS working group maintains reference software, known as the High Performance Model (HPM), to demonstrate a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework.

[0033] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data to be decoded by a destination device 14 at a later time. The source device 12 and destination device 14 can include any of a variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, the source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0034] In some implementations, destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated and transmitted to destination device 14 according to communication standards such as wireless communication protocols. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (such as a local area network, wide area network, or global network (such as the Internet)). The communication medium may include a router, switch, base station, or any other device that may be used to facilitate communication from source device 12 to destination device 14.

[0035] In some other embodiments, encoded video data can be transferred from output interface 22 to storage device 32. The encoded video data in storage device 32 can then be accessed by destination device 14 via input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 can correspond to a file server or another intermediate storage device capable of holding the encoded video data generated by source device 12. Destination device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing encoded video data and transferring it to destination device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 can access the encoded video data via any standard data connection, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on a file server. Transferring encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0036] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. The video source 18 may include sources such as video capture devices, such as cameras, video feed interfaces containing previously captured video files, video feed interfaces for receiving video from video content providers, and / or computer graphics systems for generating computer graphics data as source video, or combinations of such sources. As an example, if the video source 18 is a camera in a security monitoring system, then source device 12 and destination device 14 can form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0037] Captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0038] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for video decoder 30 to use in decoding the video data. Such syntax elements may be included within encoded video data transmitted over a communication medium, stored on a storage medium, or stored on a file server.

[0039] In some embodiments, destination device 14 may include display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0040] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards (such as VVC, HEVC, MPEG-4 Part 10, Advanced Video Coding (AVC), AVS, or extensions of such standards). It should be understood that this application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally contemplated that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally contemplated that the video decoder 30 of destination device 14 can be configured to decode video data according to any of these current or future standards.

[0041] The video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented in part in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any one of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0042] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and decoding and inter-frame predictive coding and decoding of video blocks within a video frame. Intra-frame predictive coding and decoding relies on spatial prediction to reduce or remove spatial redundancy of video data within a given video frame or image. Inter-frame predictive coding and decoding relies on temporal prediction to reduce or remove temporal redundancy of video data within adjacent video frames or images of a video sequence.

[0043] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter (such as a deblocking filter (not shown)) may be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (not shown) may be used to filter the output of the adder 62. Before placing the reconstructed CU in the reference image storage and using it as a reference for encoding and decoding future video blocks, further loop filtering, such as Sample Adaptive Offset (SAO) and Adaptive In-Loop Filter (ALF), can be applied to the reconstructed CU. The video encoder 20 can take the form of a fixed or programmable hardware unit, or can be partitioned into one or more of the fixed or programmable hardware units illustrated.

[0044] Video data memory 40 can store video data to be encoded by components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for encoding the video data by video encoder 20 (e.g., in intra-frame prediction codec mode or inter-frame prediction codec mode). Video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, video data memory 40 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0045] like Figure 2As shown, after receiving video data, the partitioning unit 45 within the prediction processing unit 41 partitions the video data into video blocks. This partitioning may also include dividing the video frame into stripes, tiles, or other larger coding units (CUs) according to a predefined partitioning structure (such as a quadtree structure associated with the video data). A video frame can be partitioned into multiple video blocks (or sets of video blocks referred to as tiles). The prediction processing unit 41 can select one of several possible prediction coding modes for the current video block based on error results (e.g., codec rate and distortion level), such as one of several intra-frame prediction coding modes or one of several inter-frame prediction coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame prediction coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements such as motion vectors, intra-frame mode indicators, partitioning information, and other such syntax information to entropy coding unit 56.

[0046] To select an appropriate intra-predictive coding / decoding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding / decoding on the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded / decoded, to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding / decoding on the current video block relative to one or more prediction blocks in one or more reference frames, to provide temporal prediction. The video encoder 20 can execute multiple encoding channels, for example, to select an appropriate coding / decoding mode for each block of video data.

[0047] In some implementations, motion estimation unit 42 determines the inter-frame prediction mode of the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. These motion vectors indicate the displacement of prediction units (PUs) of video blocks within the current video frame relative to prediction blocks in a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. These motion vectors may, for example, indicate the displacement of a PU of a video block within the current video frame or image relative to a prediction block (or other coding unit) in a reference frame, relative to a current block (or other coding unit) being encoded / decoded within the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors for intra-frame BC encoding / decoding, such as block vectors, in a manner similar to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or block vectors may be determined using motion estimation unit 42.

[0048] A prediction block is a block of reference frames that is considered to closely match the PU of the video block to be encoded or decoded in terms of pixel differences, which can be determined by the sum of absolute differences (SAD), the sum of square differences (SSD), or other difference metrics. In some embodiments, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional-pixel positions and output motion vectors with fractional-pixel precision.

[0049] The motion estimation unit 42 calculates the motion vector of the PU in the inter-frame prediction codec frame by comparing the position of the PU with the position of the prediction block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), each of which is identified by one or more reference frames stored in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0050] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. After receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, acquire the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block with pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the current video block being encoded and decoded. The pixel differences forming the residual video block may include a luminance difference component or a chrominance difference component, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for identifying prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. Note that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are illustrated separately for conceptual purposes.

[0051] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but wherein the prediction blocks are in the same frame as the current block being encoded / decoded, and wherein the vectors are referred to as block vectors relative to the motion vectors. Specifically, the intra-BC unit 48 can determine an intra-prediction mode for encoding the current block. In some examples, the intra-BC unit 48 can encode the current block using various intra-prediction modes, for example, during a separate encoding pass, and test its performance through rate-distortion analysis. Next, the intra-BC unit 48 can select an appropriate intra-prediction mode from the various tested intra-prediction modes to use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis for various tested intra-prediction modes to calculate rate-distortion values ​​and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the appropriate intra-prediction mode to use. Rate-distortion analysis typically determines the amount of distortion (or error) between a coded block and the original uncoded block (which is encoded to produce the coded block), as well as the bit rate (i.e., number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio based on the distortion and rate of each coded block to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0052] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform this function for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, the predicted block may be a block that is considered to closely match the block to be encoded or decoded in terms of pixel differences, which may be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and the identification of the predicted block may include calculating values ​​for sub-integer pixel positions.

[0053] Regardless of whether the prediction block originates from the same frame based on intra-frame prediction or from different frames based on inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded and decoded, thereby forming a pixel difference. The pixel difference forming the residual video block can include luminance component difference and chrominance component difference.

[0054] As described above, the intra-prediction processing unit 46 can perform intra-prediction on the current video block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, or the intra-block copy prediction performed by the intra-BC unit 48. Specifically, the intra-prediction processing unit 46 can determine an intra-prediction mode for encoding the current block. To this end, the intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during separate encoding channels, and the intra-prediction processing unit 46 (or, in some examples, a mode selection unit) can select an appropriate intra-prediction mode from the tested intra-prediction modes for use. The intra-prediction processing unit 46 can provide information indicating the selected intra-prediction mode of the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode in the bitstream.

[0055] After prediction processing unit 41 determines the prediction block of the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more Transform Units (TUs) and provided to transform processing unit 52. Transform processing unit 52 uses transformations such as Discrete Cosine Transform (DCT) or conceptually similar transformations to transform the residual video data into residual transform coefficients.

[0056] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0057] After quantization, the entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using methods or techniques such as Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), Syntax-based Context-Adaptive Binary Arithmetic Coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding, or other entropy coding methods or techniques. The encoded bitstream can then be transmitted to the video decoder 30 or archived in the storage device 32 for later transmission to or retrieval by the video decoder. The entropy coding unit 56 can also entropy-encode the motion vectors and other syntax elements of the current video frame being encoded / decoded.

[0058] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain, thereby generating a reference block for predicting other video blocks. As described above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of frames stored in the DPB64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to compute sub-integer pixel values ​​for motion estimation.

[0059] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to produce a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0060] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction processing unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform operations generally combined with the above. Figure 2The decoding process is the opposite of the encoding process described for video encoder 20. For example, motion compensation unit 82 can generate prediction data based on motion vectors received from entropy decoding unit 80, while intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from entropy decoding unit 80.

[0061] In some examples, tasks can be assigned to units of the video decoder 30 to perform embodiments of this application. Similarly, in some examples, embodiments of this disclosure can be divided among one or more units of the video decoder 30. For example, the intra-frame BC unit 85 can perform embodiments of this application alone or in combination with other units of the video decoder 30 (such as the motion compensation unit 82, the intra-frame prediction processing unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the function of the intra-frame BC unit 85 can be performed by other components of the prediction processing unit 81 (such as the motion compensation unit 82).

[0062] Video data memory 79 may store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. For example, video data stored in video data memory 79 may be obtained from storage device 32 or from a local video source (such as a camera) via wired or wireless network transmission of video data or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include a Coded Picture Buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded Picture Buffer (DPB) 92 of video decoder 30 stores reference video data for decoding by video decoder 30 (e.g., in intra-frame prediction codec mode or inter-frame prediction codec mode). The video data storage devices 79 and DPB 92 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data storage devices 79 and DPB 92 are shown in... Figure 3The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0063] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantized coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0064] When a video frame is encoded or decoded as an intra-predictive codec (I) frame or as an intra-predictive block in other types of frames, the intra-predictive processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode of the signal transmission and reference data from the previous decoded block of the current frame.

[0065] When a video frame is encoded / decoded into an inter-frame prediction codec (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks of the video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each prediction block can be generated from a reference frame within one of the reference frame lists. The video decoder 30 can construct the reference frame list using the default construction technique based on the reference frames stored in the DPB 92: list 0 and list 1.

[0066] In some examples, when a video block is encoded or decoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block may lie within the reconstructed region of the same image as the current video block defined by the video encoder 20.

[0067] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate prediction blocks for the decoded current video blocks. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) used to encode and decode video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), the construction information of one or more reference frame lists in the frame's reference frame list, the motion vectors of each inter-frame prediction encoded video block in the frame, the inter-frame prediction state of each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0068] Similarly, the intra-BC unit 85 can use some of the received syntax elements (e.g., flags) to determine whether the current video block was predicted using the following: the intra-BC mode, construction information about the video block of the frame being within the reconstructed region and to be stored in the DPB 92, the block vector of each intra-BC predicted video block of the frame, the intra-BC prediction state of each intra-BC predicted video block of the frame, and other information for decoding the video block in the current video frame.

[0069] The motion compensation unit 82 can also perform interpolation using an interpolation filter to calculate interpolated values ​​for sub-integer pixels of the reference block, as used by the video encoder 20 during encoding of video blocks. In this case, the motion compensation unit 82 can determine the interpolation filter used by the video encoder 20 from the received syntax elements and use the interpolation filter to generate the prediction block.

[0070] The inverse quantization unit 86 uses the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization, to inverse quantize the quantized transform coefficients provided in the bitstream and entropy decoded by the entropy decoding unit 80. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients in order to reconstruct the residual block in the pixel domain.

[0071] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block of the current video block by summing the residual block from the inverse transform processing unit 88 and the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be positioned between the adder 90 and the DPB 92 to further process the decoded video block. Loop filtering, such as a deblocking filter, sample offset compensation (SAO), and adaptive loop filter (ALF), may be applied to the reconstructed CU before placing it in the reference image storage. The decoded video block in a given frame is then stored in the DPB 92, which stores a reference frame for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store decoded video for later presentation in, for example, Figure 1 Display devices such as 34.

[0072] In a typical video encoding and decoding process, a video sequence typically consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other instances, a frame may be monochrome and therefore consist of only a two-dimensional array of luminance samples.

[0073] Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which partitions blocks solely based on quadtrees, in AVS3, a coding unit (CTU) is divided into multiple CUs to accommodate different local characteristics based on quadtrees / binary trees / extended quadtrees. Furthermore, the concept of multiple partitioning unit types in HEVC has been removed; that is, the splitting of CUs, prediction units (PUs), and transform units (TUs) does not exist in AVS3. Instead, each CU serves as the basic unit for both prediction and transform without further partitioning. In AVS3's tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary trees and extended quadtree structures.

[0074] like Figure 4AAs shown, the video encoder 20 (or more specifically, the partitioning unit 45) generates an encoded representation of the frame by first dividing the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in the sequence parameter set such that all CTUs in the video sequence have the same size, namely one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4B As shown, each CTU may include a Coding Tree Block (CTB) for luminance samples, two corresponding CTBs for chrominance samples, and syntax elements for encoding and decoding the samples of the CTBs. The syntax elements describe the attributes of different types of units within the pixel's CTB and how the video sequence can be reconstructed at the video decoder 30. These syntax elements include inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single CTB and syntax elements for encoding and decoding the samples of the CTB. The CTB may be an N×N sample block.

[0075] To achieve better performance, the video encoder 20 can recursively perform tree partitioning on the coding tree blocks of the CTU, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination of both, and divide the CTU into smaller coding units (CUs). For example... Figure 4C The diagram describes how the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Within these smaller CUs, CU 410 and CU 420 are each divided into four 16×16 CUs. The two 16×16 CUs, 430 and 440, are further divided into four 8×8 CUs. Figure 4D The illustration depicts, as shown in the diagram. Figure 4C The final result of the partitioning process of CTU 400 described in the figure is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of a size ranging from 32×32 to 8×8. Similar to... Figure 4B The depicted CTU, each CU may include a coded block (CB) of luminance samples and two corresponding coded blocks of chrominance samples of the same size frame, as well as syntax elements for encoding and decoding the samples of the coded blocks. In a monochrome image or an image with three separate color planes, the CU may include a single coded block and a syntax structure for encoding and decoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be partitioned into multiple CUs to accommodate different local characteristics based on quadtree / ternary / binary tree partitioning. In a multi-type tree structure, a CTU is partitioned by a quadtree structure, and each quadtree leaf CU can be further partitioned by a binary or ternary tree structure. Figure 4E As shown, AVS3 has five partition / division types: quad partition, horizontal binary partition, vertical binary partition, horizontal extended quadtree partition, and vertical extended quadtree partition.

[0076] In some implementations, the video encoder 20 may further divide the coded blocks of the CU into one or more M×N prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame or intra-frame) is applied. A prediction unit (PU) of the CU may include a prediction block for a luma sample, two corresponding prediction blocks for a chroma sample, and syntax elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and syntax structures for predicting the prediction blocks. The video encoder 20 may generate predicted luma, Cb, and Cr blocks for each PU of the CU.

[0077] Video encoder 20 can generate prediction blocks for the PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0078] After the video encoder 20 generates predicted luminance, Cb, and Cr blocks for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block of the CU by subtracting the predicted luminance block of the CU from its original luminance coding block, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks of the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0079] In addition, such as Figure 4CAs illustrated, the video encoder 20 can use quadtree partitioning to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transform blocks. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of the CU can include a transform block for a luminance sample, two corresponding transform blocks for a chrominance sample, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of the transform block.

[0080] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block of the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block of the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0081] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy-encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform Context-Adaptive Binary Arithmetic Coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a representation of bit sequences forming an encoded frame and associated data, which is stored in storage device 32 or transmitted to destination device 14.

[0082] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements from it. The video decoder 30 can reconstruct frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the prediction blocks of the PU of the current CU to corresponding samples of the transform blocks of the TU of the current CU. After reconstructing the coded blocks of each CU of the frame, the video decoder 30 can reconstruct the frame.

[0083] SAO is a process of modifying decoded samples by conditionally adding an offset value to each sample after applying a deblocking filter, based on values ​​in a lookup table transmitted by the encoder. SAO filtering is performed on a region-by-region basis, based on the filter type selected for each CTB by the syntax element `sao-type-idx`. A value of 0 for `sao-type-idx` indicates that the SAO filter is not applied to the CTB, while values ​​of 1 and 2 indicate that band offset and edge offset filter types are used, respectively. In the band offset mode specified by `sao-type-idx` equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the full sample amplitude range is uniformly divided into 32 segments called bands, and the samples belonging to four bands (which are consecutive within the 32 bands) are modified by adding a transmitted value represented as a band offset (which can be positive or negative). The main reason for using four consecutive bands is that in smooth regions where band artifacts may occur, the sample amplitude in the CTB tends to concentrate in only a few bands. Furthermore, the design choice of using four offsets is consistent with the edge offset operation mode, which also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the syntax element sao-eo-class with a value from 0 to 3 indicates whether to use one of the horizontal, vertical, or two diagonal gradient directions for edge offset classification in CTB.

[0084] Figure 5This is a block diagram depicting four gradient patterns used in SAO according to some embodiments of the present disclosure. The four gradient patterns 502, 504, 506, and 508 are used for corresponding sao-eo-classes in the edge offset patterns. A sample labeled “p” indicates the center sample to be considered. Two samples labeled “n0” and “n1” specify two adjacent samples along gradient patterns of (a) horizontal (sao-eo-class = 0), (b) vertical (sao-eo-class = 1), (c) 135° diagonal (sao-eo-class = 2), and (d) 45° (sao-eo-class = 3). Each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p at a certain location with the values ​​n0 and n1 of the two adjacent samples, such as... Figure 5 As shown. This classification is based on the decoded samples for each sample, therefore EdgeIdx classification does not require additional signal transmission. Depending on the EdgeIdx category at the sample location, for EdgeIdx categories 1 to 4, an offset value from the transmitted lookup table is added to the sample. The offset values ​​for categories 1 and 2 are always positive, while the offset values ​​for categories 3 and 4 are always negative. Therefore, the filter typically has a smoothing effect in edge offset mode.

[0085] Table 1 below illustrates the sample EdgeIdx categories in the SAO edge class.

[0086]

[0087] Table 1: Sample EdgeIdx categories in the SAO edge class.

[0088] For SAO types 1 and 2, a total of four amplitude offset values ​​are transmitted to the decoder for each CTB. For type 1, the symbol is also encoded. The offset values ​​and associated syntax elements (such as sao-type-idx and sao-eo-class) are determined by the encoder—typically using a standard optimized for rate-distortion performance. A merging flag can be used to indicate that the SAO parameters are inherited from the left or top of the CTB to make signal transmission valid. In summary, SAO is a non-linear filtering operation that allows for additional refinement of the reconstructed signal, and SAO can enhance the signal representation around smooth regions and edges.

[0089] In some embodiments, this document discloses methods and systems for improving encoding / decoding efficiency or reducing the complexity of Sample Offset Compensation (SAO) by introducing cross-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. Although existing SAO designs in the HEVC, VVC, AVS2, and AVS3 standards are used as the basic SAO approach in the following description, those skilled in the art of video encoding / decoding will find that the cross-component approach described herein can also be applied to other loop filter designs or other encoding / decoding tools with similar design principles. For example, in the AVS3 standard, SAO is replaced by an encoding / decoding tool known as Enhanced Sample Adaptive Offset (ESAO). However, the CCSAO disclosed herein can also be applied in parallel with ESAO. In another example, CCSAO can be applied in parallel with the Constrained Directional Enhancement Filter (CDEF) in the AV1 standard.

[0090] In existing SAO designs in HEVC, VVC, AVS2, and AVS3 standards, the luminance (Y), chrominance (Cb), and chrominance (Cr) sample offsets are determined independently. That is, for example, the current chrominance sample offset is determined only by the current and adjacent chrominance samples, without considering co-located or adjacent luminance samples. However, luminance samples retain more original image detail than chrominance samples, and luminance samples can be beneficial for determining the current chrominance sample offset. Furthermore, since chrominance samples typically lose high-frequency details after color conversion from RGB to YCbCr or after quantization and passing through a deblocking filter, introducing luminance samples with high-frequency details preserved for chrominance offset decisions may be beneficial for chrominance sample reconstruction. Therefore, further gains can be expected by exploring cross-component correlations, such as through methods and systems using Cross-Component Sample Adaptive Offset (CCSAO).

[0091] Figure 6A This is a block diagram illustrating a CCSAO system and process according to some embodiments of the present disclosure. Luminance samples after a luminance deblocking filter (DBF Y) are used to determine additional offsets of chromaticity Cb and Cr after SAOCb and SAO Cr. For example, the current chromaticity sample 602 is first classified using co-occurrence 604 and adjacent (white) luminance samples 606, and the corresponding CCSAO offset value for the corresponding class is added to the current chromaticity sample value.

[0092] In some embodiments, CCSAO can also be used in parallel with other codec tools, such as ESAO in the AVS standard or CDEF in the AV1 standard. Figure 6B This is a block diagram illustrating a system and process for CCSAO applied in parallel with ESAO in the AVS standard, according to some embodiments of this disclosure.

[0093] In some embodiments, the current chroma sample classification reuses the SAO type (EO or BO), class, and category of the isotope luminance sample. The corresponding CCSAO offset can be obtained from the decoder itself via signal transmission or retrieval. For example, let h_Y be the isotope luminance SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a finite table. For example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes powers of 2.

[0094] In some embodiments, the comparison scores [-8, 8] of the same luminance sample (Y0) with the eight adjacent luminance samples are used to generate a total of 17 classes.

[0095] Initial Class = 0

[0096] Cyclicly analyze 8 adjacent brightness samples (Yi, i = 1 to 8)

[0097] if Y0 > Yi Class += 1

[0098] else if Y0 < Yi Class-=1

[0099] In some embodiments, the above classification methods can be combined. For example, comparison scores combined with SAO BO (32-band classification) are used to increase diversity, resulting in a total of 17*32 classes. In some embodiments, Cb and Cr can use the same class to reduce complexity or save bits.

[0100] Figure 7 This is a block diagram illustrating a sample process using CCSAO according to some embodiments of the present disclosure. Specifically, Figure 7 The diagram illustrates that inputs to the CCSAO can incorporate both vertical and horizontal DBF inputs to simplify class determination or increase flexibility. For example, let Y0_DBF_V, Y0_DBF_H, and Y0 be co-position luminance samples at the inputs of DBF_V, DBF_H, and SAO, respectively. Let Yi_DBF_V, Yi_DBF_H, and Yi be the eight adjacent luminance samples at the inputs of DBF_V, DBF_H, and SAO, where i = 1 to 8.

[0101] MaxY0=max(Y0_DBF_V, Y0_DBF_H, Y0_DBF)

[0102] MaxYi=max(Yi_DBF_V, Yi_DBF_H, Yi_DBF)

[0103] Max Y0 and Max Yi are then fed into the CCSAO classification.

[0104] Figure 8 This is a block diagram illustrating the CCSAO process interleaved with vertical and horizontal DBF according to some embodiments of the present disclosure. In some embodiments, Figure 6, Figure 7 and Figure 8 The CCSAO blocks in the diagram can be selective. For example, Y0_DBF_V and Yi_DBF_V can be used for the first CCSAO_V, and the first CCSAO_V can be processed using the same sample as in Figure 6, while using the input of the DBF_V luminance sample as the CCSAO input.

[0105] In some embodiments, the implemented CCSAO syntax is shown in Table 2 below.

[0106]

[0107] Table 2: Examples of CCSAO syntax

[0108] In some embodiments, when signaling CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, another chroma component offset can be obtained by adding, subtracting, or weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. Using explicit signaling w, where w = +-|w| has a finite number of |w| candidates, h_Cr can be obtained from h_Cb without explicitly signaling h_Cr itself.

[0109] h_Cr=w*h_Cb

[0110] Figure 9 This is a flowchart illustrating an exemplary process 900 for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0111] The video decoder 30 receives a video signal (910) comprising a first component and a second component. In some embodiments, the first component is the luminance component of the video signal and the second component is the chrominance component of the video signal.

[0112] The video decoder 30 also receives multiple offsets (920) associated with the second component.

[0113] The video decoder 30 then uses the characteristic measurement results of the first component to obtain the classification category associated with the second component (930). For example, in Figure 6, the current chroma sample 602 is first classified using the co-occurrence 604 and the adjacent (white) luminance sample 606, and the corresponding CCSAO offset value is added to the current chroma sample.

[0114] The video decoder 30 further selects a first offset (940) for the second component from multiple offsets based on the classification category.

[0115] The video decoder 30 additionally modifies the second component (950) based on the selected first offset.

[0116] In some embodiments, obtaining the classification category associated with the second component using the characteristic measurement results of the first component (930) includes: obtaining the corresponding classification category for each corresponding sample of the second component using the corresponding samples of the first component, wherein the corresponding samples of the first component are the corresponding isotopes of the first component for each corresponding sample of the second component. For example, the current chromaticity sample classification is the SAO type (EO or BO), class, and category of the isotope luminance sample.

[0117] In some embodiments, obtaining the classification category associated with the second component using the characteristic measurement results of the first component (930) includes: obtaining the corresponding classification category of each corresponding sample of the second component using the corresponding samples of the first component, wherein the corresponding samples of the first component are reconstructed before or after deblocking. In some embodiments, the first component is deblocked at a deblocking filter (DBF). In some embodiments, the first component is deblocked at a luminance deblocking filter (DBF Y). For example, alternative to Figure 6 or Figure 7 The CCSAO input can also be placed before DBF Y.

[0118] In some embodiments, the characteristic measurement results are obtained by dividing the sample range of the first component into several bands and selecting a band based on the intensity value of the sample in the first component. In some embodiments, the characteristic measurement results are obtained from the band offset (BO).

[0119] In some embodiments, the characteristic measurement results are obtained based on the direction and intensity of the edge information of the samples in the first component. In some embodiments, the characteristic measurement results are obtained from the edge offset (EO).

[0120] In some embodiments, modifying the second component (950) includes directly adding the selected first offset to the second component. For example, adding the corresponding CCSAO offset value to the current chromaticity component sample.

[0121] In some embodiments, modifying the second component (950) includes mapping the selected first offset to the second offset and adding the mapped second offset to the second component. For example, when signaling CCSAO Cb and Cr offset values, if an additional chroma offset is signaled, another chroma component offset can be obtained by adding, subtracting, or weighting to save bit overhead.

[0122] In some embodiments, receiving a video signal (910) includes a receive syntax element indicating whether a method for decoding the video signal using CCSAO is enabled for the video signal in the Sequence Parameter Set (SPS). In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0123] In some embodiments, receiving a video signal (910) includes receiving a syntax element indicating whether a method for decoding the video signal using CCSAO is enabled for the second component at the slice level. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled for Cb or Cr in the corresponding slice.

[0124] In some embodiments, receiving multiple offsets (920) associated with the second component includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign of the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​for the current CTU.

[0125] In some embodiments, receiving multiple offsets (920) associated with the second component includes a receive syntax element indicating whether the received offset of the CTU is the same as the received offset of one of the CTU's neighboring CTUs, wherein the neighboring CTU is either the left-adjacent CTU or the top-adjacent CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from the left or above the CTU.

[0126] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using CCSAO further includes: receiving a second plurality of offsets associated with the third component; obtaining a second classification category associated with the third component using characteristic measurement results of the first component; selecting a third offset for the third component from the second plurality of offsets according to the second classification category; and modifying the third component based on the selected third offset.

[0127] Figure 11 This is a block diagram illustrating a sample process according to some embodiments of the present disclosure, in addition to luminance, other cross-component isotopic (1102) and adjacent (white) chromaticity samples are also fed into the CCSAO classification. Figure 6A , Figure 6B and Figure 11 The input for CCSAO classification is shown. Figure 11 In the current chromaticity sample, the cross-component co-position chromaticity sample is 1102, and the co-position luminance sample is 1106.

[0128] In some embodiments, the classifier example (C0) uses the same luminance sample (Y0) for classification. Let band_num be the number of equal bands in the luminance dynamic range, and bit_depth be the sequence bit depth. The class index of the current chroma sample is:

[0129] Class(C0)=(Y0*band_num)>>bit_depth

[0130] Table 3 below lists some examples of band_num and bit_depth. Table 3 shows three classification examples when the number of bands for each classification example is different.

[0131]

[0132] Table 3: Exemplary band_num and bit_depth for each class index.

[0133] In some embodiments, the classifier uses different brightness sample locations for C0 classification. Figure 10 This is a block diagram illustrating a classifier that performs C0 classification using sample locations with different brightness levels according to some embodiments of the present disclosure, for example, using adjacent Y7 instead of Y0 for C0 classification.

[0134] In some embodiments, different classifiers can be switched between the Sequence Parameter Set (SPS), Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), Coding Unit (CTU), and Coding Unit (CU) levels. For example, in Figure 10 In this example, Y0 is used for POC0, but Y7 is used for POC1, as shown in Table 4 below.

[0135] POC Classifier C0 band_num General Category 0 C0 uses position Y0 8 8 1 C0 uses position Y7 8 8

[0136] Table 4: Different classifiers applied to different images

[0137] In some embodiments, the C0 position and C0 band number (band_num) can be combined and switched at the SPS / APS / PPS / PH / SH / CTU / CU levels. Different combinations can be different classifiers, as shown in Table 5 below.

[0138] POC Classifier C0 band_num General Category 0 C0 uses position Y0 16 16 1 C0 uses position Y7 8 8

[0139] Table 5: Different classifier and band number combinations applied to different images

[0140] In some embodiments, the same luminance sample value (Y0) is replaced with a value (Yp) obtained by weighting the same luminance sample and the adjacent luminance sample. Figure 12 The illustration shows an exemplary classifier according to some embodiments of the present disclosure, which replaces the same luminance sample value with a value obtained by weighting the same luminance sample and adjacent luminance samples. The same luminance sample value (Y0) can be replaced with a phase correction value (Yp) obtained by weighting adjacent luminance samples. Different Yp values ​​can be different classifiers.

[0141] In some embodiments, different Yp values ​​are applied to different chroma formats. For example, in Figure 12 In (a), Yp is used for 420 chroma format, Yp is used for 422 chroma format in (b), and Y0 is used for 444 chroma format.

[0142] In some embodiments, another classifier (C1) is the comparison score [-8, 8] of the same brightness sample (Y0) with the 8 neighboring brightness samples, as shown below, generating a total of 17 classes.

[0143] Initially, Class(C1) = 0, then cycle through the 8 adjacent brightness samples (Yi, i = 1 to 8).

[0144] if Y0 > Yi Class += 1

[0145] else if Y0 < Yi Class-=1

[0146] In some embodiments, variant (C1′) only calculates the comparison score [0, 8], and this produces 8 classes. (C1, C1′) is a group of classifiers and can be switched between C1 and C1′ by signaling a PH / SH level flag.

[0147] Initially, Class(C1') = 0, then cycle through the 8 adjacent brightness samples (Yi, i = 1 to 8).

[0148] if Y0 > Yi Class += 1

[0149] In some embodiments, different classifiers are combined to produce a general classifier. For example, different classifiers are applied for different images (different POC values), as shown in Table 6 below.

[0150] POC Classifier C0 band_num General Category 0 Combining C0 and C1 16 16*17 l Combinations C0 and C1' 16 16*9 2 Combining C0 and C1 7 7*17

[0151] Table 6: Different general classifiers applied to different images

[0152] In some embodiments, the classifier example (C2) uses the difference (Yn) between co-located and adjacent brightness samples. Figure 12 (c) shows an example of Yn, where the dynamic range of Yn is [-1024, 1023] when the bit depth is 10. Let C2band_num be the number of equal bands in the dynamic range of Yn.

[0153] Class(C2)=(Yn+(1<<bit_depth)*band_num)>>(bit_depth+1).

[0154] In some embodiments, C0 and C2 are combined to produce a general classifier. For example, different classifiers are applied for different images (different POCs), as shown in Table 7 below.

[0155] POC Classifier C0 band_num C2 band_num General Category 0 Combining C0 and C2 16 16 16*17 1 Combining C0 and C2 8 7 8*7

[0156] Table 7: Different general classifiers applied to different images

[0157] In some embodiments, all of the above classifiers (C0, C1, C1′, C2) are combined. For example, different classifiers are applied for different images (different POCs), as shown in Table 8 below.

[0158] POC Classifier C0 band_num C2 band_num General Category 0 Combinations C0, C1, and C2 4 4 4*17*4 1 Combinations C0, C1′ and C2 6 4 6*9*4

[0159] Table 8: Different general classifiers applied to different images

[0160] In some embodiments, multiple classifiers are used within the same POC. The current frame is divided into several regions, and each region uses the same classifier. For example, three different classifiers are used in POC0, and which classifier (0, 1, or 2) is used for signal transmission at the CTU level is shown in Table 9 below.

[0161] POC Classifier C0 band_num area 0 C0 uses position Y0 16 0 0 C0 uses position Y0 8 1 0 C0 uses position Y1 8 2

[0162] Table 9: Different general classifiers applied to different regions of the same image

[0163] In some embodiments, the classifiers applied to Cb and Cr are different. The Cb and Cr offsets for all classes can be transmitted separately using signals. For example, different signal transmissions of offsets are applied to different chromaticity components, as shown in Table 10 below.

[0164] POC Quantity Classifier C0 band_num General Category Offset transmitted by signal 0 Cb C0 16 16 16 0 Cr C0 5 5 5

[0165] Table 10: Cb and Cr offsets of all classes can be transmitted separately using signals.

[0166] In some embodiments, the maximum offset value is fixed or transmitted as a signal in the form of a Sequence Parameter Set (SPS) / Adaptive Parameter Set (APS) / Image Parameter Set (PPS) / Image Header (PH) / Strip Header (SH). For example, the maximum offset is between [-15, 15].

[0167] In some embodiments, the offset signal transmission can use Differential Pulse-Code Modulation (DPCM). For example, the offset {3, 3, 2, 1, -1} can be transmitted as {3, 0, -1, -1, -2}.

[0168] In some embodiments, the offset can be stored in the APS or memory buffer for reuse in the next image / strip. An index can be signaled to indicate which stored previous frame offsets are used for the current image.

[0169] In some embodiments, the classifiers for Cb and Cr are the same. The Cb and Cr offsets for all classes can be transmitted jointly via signaling, for example, as shown in Table 11 below.

[0170] POC Quantity Classifier C0 band_num General Category Offset transmitted by signal 0 Cb and Cr C0 8 8 8

[0171] Table 11: Cb and Cr offsets of all classes can be transmitted jointly using signals.

[0172] In some embodiments, the classifiers for Cb and Cr can be the same. The Cb and Cr offsets and sign difference for all classes can be transmitted jointly by signaling, for example, as shown in Table 12 below. According to Table 12, when the Cb offset is (3, 3, 2, -1), the resulting Cr offset is (-3, -3, -2, 1).

[0173]

[0174] Table 12: Cb and Cr offsets and sign differences for all classes that can be jointly transmitted using signals.

[0175] In some embodiments, a symbol flag can be transmitted for each class using a signal. For example, as shown in Table 13 below. According to Table 13, when the Cb offset is (3, 3, 2, -1), the Cr offset obtained according to the corresponding symbol flag is (-3, -3, 2, 1).

[0176]

[0177] Table 13: The Cb and Cr offsets of all classes can be jointly transmitted using signals, and the symbol for each class can be transmitted using signals.

[0178] remember

[0179] In some embodiments, the classifiers for Cb and Cr can be the same. The Cb and Cr offsets and weight differences for all classes can be jointly transmitted using signals, for example, as shown in Table 14 below. The weights (w) can be selected from a finite table, for example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes powers of 2. According to Table 14, when the Cb offset is (3, 3, 2, -1), the Cr offset obtained according to the corresponding sign flag is (-6, -6, -4, 2).

[0180]

[0181] Table 14: Cb and Cr offsets and weight differences of all classes can be jointly transmitted using signals.

[0182] In some embodiments, weights can be transmitted for each class using signals. For example, as shown in Table 15 below. According to Table 15, when the Cb offset is (3, 3, 2, -1), the Cr offset obtained according to the corresponding symbol flag is (-6, 12, 0, -1).

[0183]

[0184] Table 15: The Cb and Cr offsets of all classes can be jointly transmitted using signals, and the weights can be transmitted using signals for each class.

[0185] In some embodiments, if multiple classifiers are used in the same POC, different offset sets are transmitted individually or jointly using signals.

[0186] In some embodiments, previously decoded offsets can be stored for use in future frames. Signaling transmission indexes can be used to indicate which previously decoded offset set is used in the current frame to reduce offset signaling transmission overhead. For example, the POC0 offset can be reused by POC2, where the signaling transmission offset set idx = 0, as shown in Table 16 below.

[0187]

[0188] Table 16: A signal transmission index can be used to indicate which previously decoded offset set is used for the current frame. In some embodiments, the reuse offset sets idx for Cb and Cr can be different. For example, as shown in Table 17 below.

[0189]

[0190] Table 17: A signal transmission index can be used to indicate which previously decoded offset set is used for the current frame, and the indices for the Cb and Cr

[0191] components can be different.

[0192] In some embodiments, sample processing is described below. Let R(x, y) be the input chrominance sample value before CCSAO, and R′(x, y) be the output chrominance sample value after CCSAO:

[0193] offset = ccsao_offset[class_index of R(x, y)]

[0194] R′(x, y) = Clip3(0, (1 << bit_depth) - 1, R(x, y) + offset)

[0195] According to the above equation, each chrominance sample value R(x, y) is classified using the indicated classifier of the current image. The corresponding offset of the obtained class index is added to each chrominance sample value R(x, y). The clipping function Clip 3 is applied to (R(x, y) + offset) to make the output chrominance sample value R'(x, y) within the bit-depth dynamic range, e.g., within the range from 0 to (1 << bit_depth) - 1.

[0196] In some embodiments, boundary processing is described below. If any co-located luma sample and adjacent luma sample used for classification are outside the current image, CCSAO is not applied to the current chrominance sample. Figure 13A is a block diagram illustrating that when any one of the co-located luma sample and adjacent luma sample used for classification is outside the current image, CCSAO is not applied to the current chrominance sample according to some embodiments of the present disclosure. For example, in Figure 13A (a), if the classifier is used, CCSAO is not applied to the left 1-column chrominance component of the current image. For example, if C1' is used, CCSAO is not applied to the left 1-column and top 1-row chrominance components of the current image, as Figure 13A (b) shows.

[0197] Figure 13BThis is a block diagram illustrating the application of CCSAO to the current chromaticity sample when either the co-occurrence luminance sample or the adjacent luminance sample used for classification is outside the current image, according to some embodiments of this disclosure. In some embodiments, a variation is that if any co-occurrence luminance sample or adjacent luminance sample used for classification is outside the current image, the missing sample is reused, such as... Figure 13B As shown in (a), or the missing samples are filled with mirror images to create samples for classification, such as... Figure 13A As shown in (b), CCSAO can be applied to the current chromaticity sample.

[0198] In some embodiments, the implemented CCSAO syntax is shown in Table 18 below. In AVS3, the term "patch" is similar to "strip," and the "patch header" is similar to "strip header." FLC represents fixed-length codec. TU represents truncated unary codec. EGk represents k-order exponential Columbus codec, where k can be fixed.

[0199]

[0200] Table 18: Exemplary CCSAO Syntax

[0201] If a higher-level flag is off, the lower-level flag can be inferred from its off state without signal transmission. For example, if ph_cc_sao_cb_flag is false in this image, then ph_cc_sao_cb_band_num_minus1, ph_cc_sao_cb_luma_type, cc_sao_cb_offset_sign_flag, cc_sao_cb_offset_abs, ctb_cc_sao_cb_flag, cc_sao_cb_merge_left_flag, and cc_sao_cb_merge_up_flag do not exist and are inferred to be false.

[0202] Figure 14 This is a flowchart illustrating an exemplary process 1400 for decoding a video signal using cross-component correlation in an image frame according to some embodiments of the present disclosure.

[0203] Video decoder 30 (e.g.) Figure 3 (As shown) receives a video signal (1410) comprising a first component and a second component from a first image frame. In some embodiments, the first component is a luminance component and the second component is a chrominance component.

[0204] The video decoder 30 also receives from the video signal a plurality of sample offsets (1420) associated with the second component in the first image frame.

[0205] The video decoder 30 obtains a first classifier (1430) for the second component from a first set of one or more samples relative to each sample of the second component.

[0206] The video decoder 30 then selects the first sample offset (1440) from multiple sample offsets for the second component according to the first classifier.

[0207] The video decoder 30 additionally modifies the second component (1450) based on the first component offset in the first image frame.

[0208] In some embodiments, the video decoder 30 further obtains a second classifier for the second component in a second region of a first image frame from a second set of one or more samples of the first component relative to each sample of the second component; selects a second sample offset from a plurality of sample offsets for the second component according to the second classifier; and modifies the second component based on the second sample offset in the second region of the first image frame. The video decoder 30 obtains a first classifier for the second component in the first image frame in a first region of the first image frame.

[0209] In some embodiments, the step of selecting a first sample offset (1440) from a plurality of sample offsets for a second component according to a first classifier includes: dividing the dynamic range of values ​​of a first group of one or more samples of the first component into a plurality of bands according to the first classifier; and selecting a first sample offset from a plurality of sample offsets for the second component, the first sample offset corresponding to a first selected band from the plurality of bands of the first component.

[0210] In some embodiments, the first classifier is a combination of one or more classifiers in a first group, and the second classifier is a combination of one or more classifiers in a second group.

[0211] In some embodiments, the first image frame is divided into multiple regions, and a different classifier is used for each of the multiple regions. In some embodiments, the video decoder 30 further receives CTU-level syntax elements to indicate which classifier is used for each of the multiple regions in the first image frame.

[0212] In some embodiments, the video decoder 30 further receives video signals comprising a first component and a second component from a second image frame; and the video decoder 30 modifies the second component in the second image frame based on an offset set index, the offset set index indicating which previously decoded offset set from the first image frame stored in memory is used for the second image frame. For example, if the first image frame uses four offset sets, the second image frame can reuse four offset sets of idx = 0 to 3. An offset set may include the offset of each band in the sample band.

[0213] In some embodiments, the video decoder 30 further receives a video signal including a first component and a second component in a second image frame; the video decoder 30 receives a plurality of sample offsets associated with the second component in the second image frame from the video signal; the video decoder 30 obtains a second classifier for the second component from a second set of one or more samples of the first component relative to each sample of the second component; the video decoder 30 selects a second sample offset from the plurality of sample offsets for the second component according to the second classifier; and the video decoder 30 modifies the second component based on the second sample offset in the second image frame. In some embodiments, the definition of the first set of one or more sample values ​​differs from the definition of the second set of one or more sample values.

[0214] In some embodiments, the definition of a first set of one or more samples for obtaining the first classifier, relative to each sample of the second component, is switched at one or more levels of SPS, APS, PPS, PH, SH, CTU, and CU.

[0215] In some embodiments, the definition of a first group of one or more samples for obtaining the first classifier, relative to each sample of the first component and the second component, is the number of bands dividing the dynamic range of the position of the first group of one or more samples and the value of the first group of one or more samples of the first component.

[0216] In some embodiments, the video decoder 30 further receives a video signal comprising a first component and a second component from a third image frame; and the video decoder modifies the second component in the third image frame based on an offset set index, the offset set index indicating which previously decoded offset set from other image frames is used for the third image frame. In some embodiments, the first offset set is stored in memory identified by a first offset set index, and the second offset set is stored in memory identified by a second offset set index for future use.

[0217] In some embodiments, the video decoder 30 further receives a video signal comprising a first component and a second component in a third image frame; and modifies the second component in the third image frame based on an offset set index, the offset set index indicating which previously decoded offset set from other image frames is used for the third image frame. In some embodiments, the second component comprises a Cb component and a Cr component, and the offset set indices used for the Cb component and the Cr component in the third image frame are different.

[0218] In some embodiments, a first group of one or more samples of the first component relative to each sample of the second component are co-sampling samples of the first component relative to each sample of the second component.

[0219] In some embodiments, a first group of one or more samples of the first component relative to each sample of the second component are adjacent samples of the first component relative to each sample of the second component.

[0220] In some embodiments, receiving a video signal (1410) includes receiving a syntax element indicating whether cross-component sample offset compensation (CCSAO) is enabled for the video signal.

[0221] Other embodiments also include various subsets of the above embodiments that are combined or otherwise rearranged in various other embodiments.

[0222] In one or more examples, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that includes any medium that facilitates, for example, transferring a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementing the embodiments described in this application. A computer program product can include a computer-readable medium.

[0223] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated enumerated items. It will be further understood that when the terms “comprises” and / or “comprising” are used in this specification, they specify the presence of stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0224] It should also be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but the first electrode and the second electrode are not the same electrode.

[0225] Throughout this specification, references to “an example,” “example,” “exemplary example,” etc., in singular or plural form, mean that one or more specific features, structures, or characteristics described in connection with the example are included in at least one example of this disclosure. Therefore, phrases such as “in an example,” “in the example,” “in the exemplary example,” etc., appearing in singular or plural form in various places throughout this specification do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples may include combinations in any suitable manner.

[0226] The description of this application has been presented for purposes of illustration and description, and is not intended to be exhaustive or to limit the invention to the form disclosed. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings. Embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention and to best utilize the basic principles and various embodiments with modifications suitable for the intended particular use. Therefore, it should be understood that the scope of the claims should not be limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A video encoding method, the method comprising: A first image frame is acquired, comprising a first component and a second component, wherein the first component is a luminance component and the second component is a chrominance component; Determine multiple sample offsets associated with the second component in the first image frame; A first class index for the second component is obtained from a first group of one or more samples of the first component, wherein the first group of one or more samples of the first component corresponds to each corresponding sample of the second component; Based on the first type of index, a first sample offset is selected from a plurality of sample offsets for the second component; and The cross-component offset sample value of the second component is obtained based on the first sample offset.

2. The method of claim 1, further comprising: A second class index for a second component in a second region of the first image frame is obtained from a second group of one or more samples of the first component, wherein the second group of one or more samples of the first component corresponds to each corresponding sample of the second component; Based on the second type of index, a second sample offset is selected from a plurality of sample offsets for the second component; and The cross-component offset sample value of the second component is obtained based on the second sample offset; Specifically, the first class index for the second component in the first image frame is obtained in the first region of the first image frame.

3. The method of claim 2, wherein, The step of obtaining a first class index for the second component from a first group of one or more samples of the first component includes: The first class index is obtained based on one or more classifiers from the first group of one or more samples used for the first component, and The step of obtaining a second-class index for the second component in the second region of the first image frame from a second set of one or more samples of the first component includes: The second class index is obtained based on one or more classifiers from the second group of one or more samples used for the first component.

4. The method of claim 1, wherein, The first image frame is divided into multiple regions, and a different class index is used for each of the multiple regions.

5. The method of claim 4, further comprising: A syntax element is determined to indicate which class index to use for each of the plurality of regions in the first image frame, wherein the syntax element is switched at the CTU level.

6. The method of claim 1, further comprising: Obtain a second image frame that includes the first component and the second component; as well as The offset set index is used to obtain the cross-component offset sample value of the second component in the second image frame, the offset set index indicating which previously encoded offset set in the first image frame stored in memory is used for the second image frame.

7. The method of claim 1, further comprising: Obtain a second image frame that includes the first component and the second component; Determine multiple sample offsets associated with the second component in the second image frame; A second class index for the second component is obtained from one or more samples in the second group of the first component, wherein the one or more samples in the second group of the first component correspond to each corresponding sample of the second component; Based on the second type of index, a second sample offset is selected from the plurality of sample offsets for the second component; and The cross-component offset sample value of the second component is obtained based on the second sample offset; The definitions of one or more samples in the first group are different from the definitions of one or more samples in the second group.

8. The method of claim 3, wherein, One of the classifiers in the first group of one or more classifiers is obtained based on a first parameter, which is used to select a first band corresponding to the first group of one or more samples from the first component from a first number of dividing bands of the dynamic range of the sample values ​​of the first component. One of the classifiers in the second group is obtained based on a second parameter, which is used to select a second stripe corresponding to the second group of one or more samples of the first component from a second number of dividing stripes of the dynamic range of the sample values ​​of the first component.

9. The method of claim 8, wherein, The step of selecting a first sample offset from a plurality of sample offsets for the second component according to the first type of index includes: Select the first sample offset from the plurality of sample offsets corresponding to the selected first band for the second component; and The step of selecting the second sample offset from a plurality of sample offsets for the second component according to the second type of index includes: Select the second sample offset from the plurality of sample offsets for the second component, corresponding to the selected second stripe.

10. The method of claim 8, wherein, Each of the plurality of first parameters is switched at one or more of the SPS, APS, PPS, PH, SH, CTU, and CU levels, and each of the plurality of second parameters is switched at one or more of the SPS, APS, PPS, PH, SH, CTU, and CU levels.

11. The method of claim 10, wherein, The first parameter includes the location of one or more samples in the first group and the first quantity, and the plurality of second parameters include the location of one or more samples in the second group and the second quantity.

12. The method of claim 7, further comprising: Obtain a third image frame that includes the first component and the second component; as well as The cross-component offset sample value of the second component in the third image frame is obtained based on the offset set index, which indicates which previously encoded offset set in other image frames is used for the third image frame, wherein the first offset set is stored in memory identified by the first offset set index, and the second offset set is stored in memory identified by the second offset set index for future use.

13. The method of claim 7, further comprising: Obtain a third image frame that includes the first component and the second component; as well as The cross-component offset sample value of the second component in the third image frame is obtained based on the offset set index, which indicates which previously encoded offset set in other image frames is used for the third image frame, wherein the second component includes a Cb component and a Cr component, and the offset set indices used for the Cb component and the Cr component in the third image frame are different.

14. The method of claim 1, wherein, The first group of one or more samples in the first component are co-samples of each corresponding sample.

15. The method of claim 1, wherein, The first group of one or more samples in the first component are the neighboring samples of the co-sampling samples of each corresponding sample.

16. The method of claim 1, further comprising: Determines whether to enable cross-component sample offset compensation (CCSAO) syntax elements.

17. An electronic device comprising: One or more processing units; A memory coupled to the one or more processing units; as well as A plurality of programs stored in the memory, which, when executed by the one or more processing units, cause the electronic device to perform the method as described in any one of claims 1 to 16.

18. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein, When the plurality of programs are executed by the one or more processing units, the electronic device performs the method as described in any one of claims 1 to 16.

19. A computer program product comprising instructions for execution by a computing device having one or more processors, wherein, when executed by the one or more processors, the computing device performs the method as claimed in any one of claims 1 to 16.

20. A non-transitory computer-readable medium storing a video bitstream and instructions, wherein, When the instructions are executed by the processor, they implement the method of any one of claims 1 to 16 to generate the video bitstream.

Images