Methods and apparatus for encoding video

Abstract

A method and apparatus for encoding video. The method includes: determining a plurality of offsets associated with a first chroma component of the video; obtaining a classification category associated with the first chroma component using a sample characteristic measure of the luminance component of the video; selecting a first offset from the plurality of offsets for the first chroma component according to the classification category; modifying the sample value of the first chroma component based on the selected first offset; transmitting a video signal including the luminance component and the first chroma component; and transmitting the plurality of offsets associated with the first chroma component.

Description

[0001] This application is a divisional application of the invention patent application filed on June 3, 2021, with application number "202180003309.4" and titled "Enhancement of Chroma Encoding and Decoding in Cross-Component Correlation". Technical Field

[0002] This application generally relates to video encoding and decoding and compression, and more specifically, to methods and apparatus for improving the efficiency of chroma encoding and decoding. Background Technology

[0003] Various electronic devices (such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards. Some well-known video codec standards include Universal Video Coding (VVC), High Efficiency Video Coding (HEVC, also known as H.265 or MPEG-H Part 2), and Advanced Video Coding (AVC, also known as H.264 or MPEG-4 Part 10), jointly developed by ISO / IEC MPEG and ITU-T TVCEG. AO Media Video 1 (AV1) was developed by the Alliance for Open Media (AOM) as a successor to its previous standard VP9. Audio and Video Coding (AVS) refers to digital audio and digital video compression standards, a series of video compression standards developed by the China Audio and Video Coding Standards Working Group.

[0004] Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video coding, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain one coding unit (CU) or be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be coded in intra-frame, inter-frame, or IBC modes. Video blocks in an intra-frame coded (I) strip of a video frame are coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame. Video blocks in an inter-frame coded (P or B) strip of a video frame can be coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame or temporal prediction with respect to reference samples in other previous and / or future reference video frames.

[0005] A prediction block for the current video block to be encoded is derived based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on the residual block and the motion vector pointing to the reference block forming the prediction block in the reference frame. The process of determining the motion vector is typically called motion estimation. Intra-frame coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain (e.g., the frequency domain) to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, which is then entropy-encoded into the video bitstream for even greater compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or transmitted directly to the electronic device via wired or wireless means. The electronic device then performs video decompression (a process opposite to the video compression described above), for example, by parsing the encoded video bitstream to obtain semantic elements from the bitstream, and reconstructs the digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and the electronic device presents the reconstructed digital video data on its display.

[0007] As digital video quality evolves from high definition to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while efficiently encoding / decoding video data remains a long-standing challenge. Summary of the Invention

[0008] This application describes implementations related to video data encoding and decoding, and more specifically, describes real-time methods related to methods and apparatus for improving the efficiency of chroma encoding and decoding, including improving encoding and decoding efficiency by exploring cross-component relationships between luminance and chroma components.

[0009] According to a first aspect of this application, a method for encoding video includes: determining a plurality of offsets associated with a first chroma component of the video; obtaining a classification category associated with the first chroma component using a sample characteristic measure of a luminance component of the video; selecting a first offset from the plurality of offsets for the first chroma component according to the classification category; modifying the sample value of the first chroma component based on the selected first offset; transmitting a video signal including the luminance component and the first chroma component; and transmitting the plurality of offsets associated with the first chroma component.

[0010] According to a second aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above.

[0011] According to a third aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processor units. When executed by one or more processing units, the programs cause the electronic device to perform the method of encoding and decoding video data as described above. Attached Figure Description

[0012] The accompanying drawings, included to provide a further understanding of the embodiments and incorporated herein as a part of the specification, illustrate the described embodiments and, together with the description, serve to explain the underlying principles. Similar reference numerals denote corresponding parts.

[0013] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0014] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0015] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0016] Figures 4A to 4E This is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0017] Figure 5 This is a block diagram depicting four gradient modes used in Sample Adaptive Offset (SAO) according to some embodiments of the present disclosure.

[0018] Figure 6This is a block diagram illustrating a system and process for cross-component sample adaptive offset (CCSAO) according to some embodiments of the present disclosure.

[0019] Figure 7 This is a block diagram illustrating sample processing using CCSAO according to some embodiments of the present disclosure.

[0020] Figure 8 This is a block diagram illustrating the CCSAO process interleaved with vertical and horizontal deblocking filters (DBFs) according to some embodiments of the present disclosure.

[0021] Figure 9 This is a flowchart illustrating an exemplary process for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure. Detailed Implementation

[0022] Referring now to the detailed description, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0023] The first generation of AVS standards included the Chinese national standards "Information Technology, Advanced Audio and Video Coding, Part 2: Video" (referred to as AVS1) and "Information Technology, Advanced Audio and Video Coding, Part 16: Broadcast Television Video" (referred to as AVS+). Compared to the MPEG-2 standard, the first generation of AVS standards offered approximately 50% bitrate savings while maintaining the same perceived quality. The second generation of AVS standards included the Chinese national standard series "Information Technology, High-Efficiency Multimedia Coding" (referred to as AVS2), primarily targeting the transmission of Ultra HD TV programs. AVS2's coding efficiency was twice that of AVS+. Simultaneously, the video portion of the AVS2 standard was submitted by the Institute of Electrical and Electronics Engineers (IEEE) as an international standard for application. The AVS3 standard is a new generation of video codec standards for UHD video applications, designed to surpass the coding efficiency of the latest international standard HEVC, which offers approximately 30% bitrate savings. In March 2019, at the 68th AVS meeting, the AVS3-P2 baseline was completed, offering approximately 30% bitrate savings relative to the HEVC standard. Currently, a reference software called the High Performance Model (HPM) is maintained by the AVS group to demonstrate a reference implementation of the AVS3 standard. Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework.

[0024] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel, according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes source device 12, which generates and encodes video data that will later be decoded by target device 14. Source device 12 and target device 14 can include any electronic device from a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, source device 12 and target device 14 are equipped with wireless communication capabilities.

[0025] In some implementations, target device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to target device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to target device 14 in real time. The encoded video data may be modulated according to communication standards, such as wireless communication protocols, and transmitted to target device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, switch, base station, or any other means that may facilitate communication from source device 12 to target device 14.

[0026] In some other embodiments, encoded video data may be sent from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by target device 14 via input interface 28. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, storage device 32 may correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Target device 14 may access the stored video data from storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing and sending encoded video data to target device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The target device 14 can access the encoded video data via any standard data connection suitable for accessing encoded video data stored on the file server, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or a combination of both. The transmission of encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0027] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources or combinations of such sources, such as: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera in a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding / decoding and can be applied to wireless and / or wired applications.

[0028] Video captured, pre-captured, or computer-generated video can be encoded by video encoder 20. Encoded video data can be directly transmitted to target device 14 via output interface 22 of source device 12. Encoded video data can also (or optionally) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or transmitter.

[0029] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various semantic elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such semantic elements may be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0030] In some embodiments, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays decoded video data to the user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0031] The video encoder 20 and video decoder 30 may operate according to proprietary or industry standards (e.g., VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), AVS, or extensions of such standards). It should be understood that this application is not limited to any particular video encoding / decoding standard and is applicable to other video encoding / decoding standards. It is generally understood that the video encoder 20 of the source device 12 may be configured to encode video data according to any of these current or future standards. Similarly, it is generally understood that the video decoder 30 of the target device 14 may be configured to decode video data according to any of these current or future standards.

[0032] The video encoder 20 and video decoder 30 can be implemented as any circuit of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When implemented in part in software, the electronic device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0033] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within adjacent video frames or pictures of a video sequence.

[0034] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter (such as a deblocking filter (not shown)) may be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, another loop filter (not shown) may be used to filter the output of the adder 62. Before placing the reconstructed CU into a reference image memory and using it as a reference for encoding and decoding future video blocks, loop filtering, such as Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF), can be further applied to the reconstructed CU. The video encoder 20 can take the form of a fixed or programmable hardware unit, or can be distributed among one or more of the fixed or programmable hardware units described.

[0035] Video data memory 40 stores video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for use by video encoder 20 (e.g., in intra-frame or inter-frame predictive coding modes) when encoding the video data. Video data memory 40 and DPB 64 may be formed from any of a variety of memory devices. In various examples, video data memory 40 may be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0036] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into stripes, tiles, or other larger coding units (CUs) according to a predefined splitting structure (such as a quadtree structure) associated with the video data. The video frame may be divided into multiple video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of several feasible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one or more inter-frame predictive coding modes among multiple intra-frame predictive coding modes. The prediction processing unit 41 may provide the resulting intra-frame predictive coding block or inter-frame predictive coding block to adder 50 to generate a residual block, and to adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides semantic elements (such as motion vectors, intra-frame mode indicators, segmentation information, and other such semantic information) to entropy coding unit 56.

[0037] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each block of video data.

[0038] In some implementations, motion estimation unit 42 determines an inter-frame prediction mode for the current video frame by generating motion vectors based on a predetermined pattern within the video frame sequence. The motion vectors indicate the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion for a video block. For example, the motion vectors may indicate the displacement of a PU of a video block within the current video frame or picture relative to a prediction block (or other coding unit) in a reference frame associated with the current block (or other coding unit) being encoded within the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors (e.g., block vectors) for intra-frame BC coding in a similar manner to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0039] Regarding pixel differences, the predicted block is a block of a reference frame considered to closely match the PU of the video block to be encoded. Pixel differences can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output a motion vector with fractional-pixel accuracy.

[0040] The motion estimation unit 42 calculates the motion vector of the PU for a video block in an inter-frame predictive coding frame by comparing the position of the PU with the position of the predicted block in a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), where each identifier in the first and second reference frame lists is stored in one or more reference frames in the DPB 64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0041] Motion compensation performed by motion compensation unit 44 may involve extracting or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in a reference frame list within a reference frame list, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the currently encoded video block. The pixel differences forming the residual video block may include a luminance difference component or a chrominance difference component, or both. Motion compensation unit 44 may also generate semantic elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Semantic elements may include, for example, semantic elements defining motion vectors for identifying prediction blocks, any markers indicating prediction modes, or any other semantic information described herein. It should be noted that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0042] In some implementations, the intra-BC unit 48 may generate vectors and extract prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44; however, these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 may determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 may, for example, use various intra-prediction modes to encode the current block during individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 may select a suitable intra-prediction mode from the various tested intra-prediction modes for use and generate an intra-mode indicator accordingly. For example, the intra-BC unit 48 may use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was encoded to generate the coded block, as well as the bit rate (i.e., the number of bits) used to generate the coded block. Intra-frame BC unit 48 can calculate the ratio from the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0043] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, in terms of pixel differences, the predicted block may be a block considered to closely match the block to be encoded, the pixel differences may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics, and the identification of the predicted block may include calculating values ​​for sub-integer pixel positions.

[0044] Regardless of whether the predicted block comes from the same frame predicted intra-frame or from different frames predicted inter-frame, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0045] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine an intra-frame prediction mode for encoding the current block. To this end, the intra-frame prediction processing unit 46 can, for example, use various intra-frame prediction modes to encode the current block during individual encoding passes, and the intra-frame prediction processing unit 46 (or, in some examples, a mode selection unit) can select a suitable intra-frame prediction mode from the tested intra-frame prediction modes for use. The intra-frame prediction processing unit 46 can provide information indicating the intra-frame prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0046] After prediction processing unit 41 determines a prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 uses a transform (such as discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients.

[0047] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Optionally, the entropy coding unit 56 can perform the scan.

[0048] After quantization, entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), semantically based context-adaptive binary arithmetic coding (SBAC), probabilistic interval segmented entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream can then be sent to video decoder 30 or archived in storage device 32 for later transmission to or retrieval by video decoder 30. Entropy coding unit 56 can also entropy-encode motion vectors and other semantic elements used for the current video frame being encoded.

[0049] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual video blocks in the pixel domain for generating reference blocks to predict other video blocks. As noted above, the motion compensation unit 44 can generate motion-compensated prediction blocks from one or more reference blocks of the frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction blocks to compute sub-integer pixel values ​​for use in motion estimation.

[0050] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block to be stored in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42 and motion compensation unit 44 to perform inter-frame prediction of another video block in subsequent video frames.

[0051] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 can perform the functions described above. Figure 2 The decoding process described for the video encoder 20 is essentially the inverse of the encoding process. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from the entropy decoding unit 80.

[0052] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be distributed across one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application individually or in combination with other units of the video decoder 30 (such as the motion compensation unit 82, the intra-frame prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81 (such as the motion compensation unit 82).

[0053] Video data memory 79 may store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (such as a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 (e.g., in intra-frame or inter-frame predictive coding modes) when decoding video data. Video data memory 79 and DPB 92 may be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are shown in... Figure 3 The two components are depicted as video decoder 30. However, it will be apparent to those skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, video data memory 79 may be on-chip along with other components of video decoder 30, or off-chip relative to those components.

[0054] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of encoded video frames and associated semantic elements. The video decoder 30 may receive semantic elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other semantic elements. The entropy decoding unit 80 then forwards the motion vectors and other semantic elements to the prediction processing unit 81.

[0055] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0056] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other semantic elements received from the entropy decoding unit 80. Each prediction block can be generated from a reference frame within a reference frame list. The video decoder 30 can construct the reference frame list, list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0057] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other semantic elements received from the entropy decoding unit 80. The prediction block can be located within the reconstructed region of the same image as the current video block, as defined by the video encoder 20.

[0058] Motion compensation unit 82 and / or intra-frame prediction (BC) unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other semantic elements, and then use this prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received semantic elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frames in the reference frame list for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0059] Similarly, the intra-BC unit 85 may use some of the received semantic elements, such as tags, to determine which video blocks in the frame were predicted using the intra-BC mode, which video blocks in the frame are within the reconstruction region and should be stored in the DPB 92, the block vector for each intra-BC predicted video block in the frame, the intra-BC prediction state for each intra-BC predicted video block in the frame, and other information for decoding video blocks in the current video frame.

[0060] The motion compensation unit 82 may also perform interpolation using interpolation filters, such as those used by the video encoder 20 during the encoding of video blocks, to compute interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 may determine the interpolation filters used by the video encoder 20 from the received semantic elements and use these interpolation filters to generate the prediction block.

[0061] The dequantization unit 86 dequantizes the quantized transform coefficients, which are provided in the bitstream and entropy decoded by the entropy decoding unit 80, using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0062] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other semantic elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. Loop filtering, such as a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF), may be applied to the reconstructed CU before it is placed in the reference picture memory. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later presentation on a display device (e.g., Figure 1 On the display device 34).

[0063] In a typical video coding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of chrominance samples (Cb). SCr is a two-dimensional array of chrominance samples (Cr). In other cases, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0064] Like HEVC, the AVS3 standard is built on a block-based hybrid video codec framework. The input video signal is processed block by block (called a coding unit (CU)). Unlike HEVC, which only partitions blocks based on quadtrees, in AVS3, a coding tree unit (CTU) is split into CUs based on quadtrees / binary trees / extended quadtrees to accommodate varying local characteristics. Furthermore, the concept of multiple partitioned unit types in HEVC is removed; that is, there is no distinction between CUs, prediction units (PUs), and transform units (TUs) in AVS3. Instead, each CU is used consistently as the basic unit for both prediction and transform without further partitioning. In AVS3's tree partitioning structure, a CTU is first partitioned based on a quadtree structure. Then, each quadtree leaf node can be further partitioned based on binary trees and extended quadtree structures.

[0065] like Figure 4A As shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates a coded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered sequentially from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 using a sequence parameter set, such that all CTUs in the video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to a specific size. Figure 4B As shown, each CTU may include a coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and semantic elements for encoding the samples of the coding tree blocks. The semantic elements describe the properties of different types of units within the coded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In monochrome images or images with three separate color planes, the CTU may include a single coding tree block and semantic elements for encoding the samples of that coding tree block. The coding tree block may be an N×N sample block.

[0066] To achieve better performance, the video encoder 20 can recursively perform tree partitioning on the coding tree blocks of the CTU, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination of both, and divide the CTU into smaller coding units (CUs). Figure 4C As depicted, the 64×64 CTU 400 is first divided into four smaller CUs, each with a block size of 32×32. Among these four smaller CUs, CU 410 and CU 420 are each divided into four CUs with a block size of 16×16. The two 16×16 CUs, 430 and CU 440, are further divided into four CUs with a block size of 8×8. Figure 4D Depicting as shown Figure 4C The final result of the CTU 400 partitioning process described in the paper is a quadtree data structure, where each leaf node of the quadtree corresponds to a CU of various sizes ranging from 32×32 to 8×8. Similar to... Figure 4B The CTU depicted in the image may include, for each CU, a coded block (CB) of luminance samples of the same size frame and two corresponding coded blocks of chrominance samples, and semantic elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, the CU may include a single coded block and a semantic structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4DThe quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, there are five split / segmentation types in AVS3: quadruple split, horizontal binary split, vertical binary split, horizontal extended quadruple split, and vertical extended quadruple split.

[0067] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more M×N prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter-frame or intra-frame) is applied. A prediction unit (PU) of the CU may include a prediction block for luma samples, two corresponding prediction blocks for chroma samples, and semantic elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and a semantic structure for predicting the prediction blocks. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for each PU of the CU.

[0068] Video encoder 20 can use intra-frame prediction or inter-frame prediction to generate prediction blocks for the PU. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0069] After the video encoder 20 generates a predicted luminance block, a predicted Cb block, and a predicted Cr block for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block for the CU by subtracting the predicted luminance block of the CU from the original luminance coding block of the CU, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0070] In addition, such as Figure 4C As shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A transform unit (TU) of the CU may include a transform block for luminance samples, two corresponding transform blocks for chrominance samples, and semantic elements for transforming the samples in the transform block. Therefore, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU may be a sub-block of the CU's luminance residual block. A Cb transform block may be a sub-block of the CU's Cb residual block. A Cr transform block may be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU may include a single transform block and a semantic structure for transforming the samples in that transform block.

[0071] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0072] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 may quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 may entropy encode the semantic elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the semantic elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream may be stored in storage device 32 or transmitted to target device 14.

[0073] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain semantic elements. The video decoder 30 can reconstruct frames of video data based at least in part on the semantic elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the prediction block of the PU for the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0074] SAO (Short-Area Offset) is a process of modifying decoded samples by conditionally adding an offset value to each sample based on a lookup table sent by the encoder after applying a deblocking filter. SAO filtering is performed on a region-by-region basis, based on the filter type selected by the semantic element `sao-type-idx` according to the CTB (Constant Band Offset). A value of 0 for `sao-type-idx` indicates that the SAO filter is not applied to the CTB, while values ​​of 1 and 2 respectively signal the use of the band offset filter type and the edge offset filter type. In the band offset mode specified by `sao-type-idx` equal to 1, the selected offset value depends directly on the sample amplitude. In this mode, the entire sample amplitude range is uniformly divided into 32 segments (called bands), and the sample values ​​belonging to four of these bands (which are consecutive within the 32 bands) are modified by adding to the sent value (represented as the band offset), which can be positive or negative. The main reason for using four consecutive bands is that in smooth regions where banding artifacts may occur, the sample amplitude in the CTB tends to concentrate in only a few bands. Furthermore, the design choice using four offsets is consistent with the operation of the edge offset mode, which also uses four offset values. In the edge offset mode specified by sao-type-idx equal to 2, the semantic element sao-eo-class with values ​​from 0 to 3 is used to signal whether it is horizontal, vertical, or one of the two diagonal gradient directions for edge offset classification in CTB.

[0075] Figure 5This is a block diagram depicting four gradient patterns used in SAO according to some embodiments of the present disclosure. Four gradient patterns 502, 504, 506, and 508 are used for the corresponding sao-eo-class in the edge offset pattern. A sample marked "p" indicates the center sample to be considered. Two samples marked "n0" and "n1" specify two adjacent samples along (a) the horizontal (sao-eo-class = 0) gradient pattern, (b) the vertical (sao-eo-class = 1) gradient pattern, (c) the 135° diagonal (sao-eo-class = 2) gradient pattern, and (d) the 45° diagonal (sao-eo-class = 3) gradient pattern. Figure 5 As shown, each sample in the CTB is classified into one of five EdgeIdx categories by comparing the sample value p at a certain location with the values ​​n0 and n1 of two samples at neighboring locations. This classification is based on the decoded sample value, therefore EdgeIdx classification does not require additional signaling. Depending on the EdgeIdx category at the sample location, for EdgeIdx categories 1 to 4, the offset value from the transmitted lookup table is added to the sample value. The offset value is always positive for categories 1 and 2, and always negative for categories 3 and 4. Therefore, the filter typically has a smoothing effect in edge offset mode. Table 1 below shows the sample EdgeIdx categories in SAO edge classification.

[0076]

[0077] Table 1: EdgeIdx Classification of Sample Points in SAO Edge Classification

[0078] For SAO Type 1 and SAO Type 2, a total of four amplitude offset values ​​are sent to the decoder for each CTB. For Type 1, the symbol is also encoded. The offset values ​​and associated semantic elements (such as sao-type-idx and sao-eo-class) are determined by the encoder—typically using a standard optimized for rate-distortion performance. A merging flag can be used to indicate whether the SAO parameters are inherited from the left CTB or the upper CTB, thus making the signaling valid. In summary, SAO is a nonlinear filtering operation that allows for additional corrections to the reconstructed signal, and it enhances the signal representation of smooth regions and surrounding edges.

[0079] In some embodiments, this document discloses methods and systems for improving coding efficiency or reducing the complexity of Sample Adaptive Offset (SAO) by introducing cross-component information. SAO is used in the HEVC, VVC, AVS2, and AVS3 standards. Although existing SAO designs in the HEVC or VVC standards are used as the basic SAO method in the following description, those skilled in the art of video coding and decoding will find that the methods described in this disclosure can also be applied to other loop filter designs or other coding and decoding tools with the same or similar design spirit, such as the Constrained Direction Enhancement Filter (CDEF) in the AV1 standard.

[0080] For existing SAO designs in HEVC, VVC, AVS2, or AVS3 standards, the luma Y-sample offset, chroma Cb-sample offset, and chroma Cr-sample offset values ​​are determined independently. That is, for example, the current chroma sample offset is determined solely by neighboring chroma sample values, without considering co-located or adjacent luma samples. However, because luma samples retain more original image detail than chroma samples, luma samples may be beneficial for the current chroma sample offset decision. Furthermore, chroma samples typically lose high-frequency details after color conversion from RGB to YCbCr or after deblocking filters. Therefore, introducing luma samples with high-frequency details preserved for chroma offset decisions may be beneficial for chroma sample reconstruction. Therefore, further gains can be expected by exploring the cross-component correlation of SAO, for example, through the use of cross-component sample adaptive offset (CCSAO) methods and systems.

[0081] Figure 6 This is a block diagram illustrating a system and process for CCSAO according to some embodiments of the present disclosure. Luminance samples (DBF Y) after passing through a luminance deblocking filter are used to determine additional offsets in chromaticity Cb and Cr after SAO Cb and SAO Cr. For example, the current chromaticity sample 602 is first classified using a co-occurring luminance sample 604 and a neighboring (white) luminance sample 606, and the corresponding CCSAO offset value is added to the current chromaticity sample.

[0082] In some embodiments, the current chroma sample classification reuses the SAO type (EO or BO), class, and category of the corresponding luminance sample. The corresponding CCSAO offset can be transmitted via signal or derived from the decoder itself. For example, let h_Y be the corresponding luminance SAO offset, and h_Cb and h_Cr be the CCSAO Cb and Cr offsets, respectively. h_Cb (or h_Cr) = w * h_Y, where w can be selected from a finite table. For example, ±1 / 4, ±1 / 2, 0, ±1, ±2, ±4, etc., where |w| only includes powers of 2.

[0083] In some embodiments, the comparison scores [-8, 8] of the co-located luminance sample (Y0) and the neighboring 8 luminance samples are used, which results in a total of 17 categories.

[0084] Initial class = 0

[0085] Loop through the neighboring 8 luminance samples (Yi, i = 1 to 8)

[0086] If Y0 > Yi, then classification += 1

[0087] Otherwise if Y0 < Yi, then classification -= 1

[0088] In some embodiments, the above classification methods can be combined. For example, the comparison scores combined with SAO BO (32-band classification) are used to increase diversity, which results in a total of 17×32 categories. In some embodiments, Cb and Cr can use the same classification to reduce complexity or save bits.

[0089] Figure 7 is a block diagram showing sample processing using CCSAO according to some implementations of the present disclosure. Specifically, Figure 7 it shows that the input of CCSAO can introduce the inputs of vertical DBF and horizontal DBF to simplify classification determination or increase flexibility. For example, let Y0_DBF_V, Y0_DBF_H, and Y0 be the co-located luminance samples at the inputs of DBF_V, DBF_H, and SAO respectively. Yi_DBF_V, Yi_DBF_H, and Yi are the neighboring 8 luminance samples at the inputs of DBF_V, DBF_H, and SAO respectively, where i = 1 to 8.

[0090] Max Y0 = max(Y0_DBF_V, Y0_DBF_H, Y0_DBF)

[0091] Max Yi = max(Yi_DBF_V, Yi_DBF_H, Yi_DBF)

[0092] And Max Y0 and Max Yi are fed into the CCSAO classification.

[0093] Figure 8 is a block diagram showing the interleaving of the CCSAO process with vertical DBF and horizontal DBF according to some embodiments of the present disclosure. In some embodiments, Figure 6 、 Figure 7 and Figure 8 the CCSAO blocks in can be selective. For example, Y0_DBF_V and Yi_DBF_V are used for the first CCSAO_V, which applies the same sample processing as in Figure 6 while using the input of the DBF_V luminance sample as the CCSA0 input.

[0094] In some embodiments, CCSAO semantics are implemented as shown in Table 2 below.

[0095]

[0096] Table 2: Examples of CCSAO semantics

[0097] In some embodiments, to transmit CCSAO Cb and Cr offset values ​​via signaling, if an additional chrominance offset is to be transmitted via signaling, the offset of another chrominance component can be derived by adding or subtracting a sign or by weighting to save bit overhead. For example, let h_Cb and h_Cr be the offsets of CCSAO Cb and Cr, respectively. Using explicit signaling w, where w = +-|w|, and with a finite number of |w| candidates, h_Cr can be derived from h_Cb without explicit signaling h_Cr itself.

[0098] h_Cr=w*h_Cb

[0099] Figure 9 This is a flowchart illustrating an exemplary process 900 for decoding a video signal using cross-component correlation according to some embodiments of the present disclosure.

[0100] The video decoder 301 receives a video signal (910) comprising a first component and a second component. In some embodiments, the first component is the luminance component of the video signal, and the second component is the chrominance component of the video signal.

[0101] The video decoder 30 also receives multiple offsets (920) associated with the second component.

[0102] The video decoder 30 then uses the characteristic measure of the first component to obtain the classification category associated with the second component (930). For example, in Figure 6 First, the current chromaticity sample 602 is classified using the same luminance sample 604 and the neighboring (white) luminance sample 606, and the corresponding CCSAO offset value is added to the current chromaticity sample.

[0103] The video decoder 30 also selects a first offset (940) from multiple offsets for the second component based on the classification category.

[0104] The video decoder 30 further modifies the second component (950) based on the selected first offset.

[0105] In some embodiments, obtaining a classification category associated with a second component using a characteristic metric of the first component (930) includes: obtaining a corresponding classification category for each corresponding sample of the second component using corresponding samples of the first component, wherein the corresponding sample of the first component is a corresponding isotope sample of the first component for each corresponding sample of the second component. For example, the current chromaticity sample classification reuses the SAO type (EO or BO), classification, and category of the isotope luminance sample.

[0106] In some embodiments, obtaining a classification category (930) associated with a second component using a characteristic metric of the first component includes: obtaining a corresponding classification category for each corresponding sample of the second component using corresponding samples of the first component, wherein the corresponding samples of the first component are reconstructed before or after deblocking. In some embodiments, the first component is deblocked at a deblocking filter (DBF). In some embodiments, the first component is deblocked at a luminance deblocking filter (DBFY). For example, alternatively Figure 6 or Figure 7 The CCSAO input can also be placed before DBF Y.

[0107] In some implementations, the characteristic metric is derived by dividing the range of sample values ​​of the first component into several bands and selecting bands based on the intensity values ​​of the samples in the first component. In some embodiments, the characteristic metric is derived from band offset (BO).

[0108] In some embodiments, the feature measure is derived based on the direction and intensity of the edge information of the samples in the first component. In some embodiments, the feature measure is derived from the edge offset (EO).

[0109] In some embodiments, modifying the second component (950) includes adding the selected first offset directly to the second component. For example, adding the corresponding CCSAO offset value to the current chromaticity component sample.

[0110] In some embodiments, modifying the second component (950) includes mapping the selected first offset to the second offset and adding the mapped second offset to the second component. For example, to signal CCSAO Cb and Cr offset values, if an additional chroma offset is to be signaled, another chroma component offset can be derived by using a plus or minus sign or weighting to save bit overhead.

[0111] In some embodiments, receiving a video signal (910) includes receiving a semantic element in a sequence parameter set (SPS), wherein the semantic element indicates whether a method for decoding the video signal using CCSAO is enabled for the video signal. In some embodiments, cc_sao_enabled_flag indicates whether CCSAO is enabled at the sequence level.

[0112] In some embodiments, receiving a video signal (910) includes receiving a semantic element at the slice level, wherein the semantic element indicates whether a method for decoding the video signal using CCSAO is enabled for a second component. In some embodiments, slice_cc_sao_cb_flag or slice_cc_sao_cr_flag indicates whether CCSAO is enabled in the corresponding slice for Cb or Cr.

[0113] In some embodiments, receiving multiple offsets (920) associated with the second component includes receiving different offsets for different coding tree units (CTUs). In some embodiments, for a CTU, cc_sao_offset_sign_flag indicates the sign of the offset, and cc_sao_offset_abs indicates the CCSAO Cb and Cr offset values ​​for the current CTU.

[0114] In some embodiments, receiving multiple offsets (920) associated with the second component includes receiving a semantic element, wherein the semantic element indicates whether the received offset of the CTU is the same as the offset of one of the CTU's neighboring CTUs, wherein the neighboring CTU is either the left neighboring CTU or the top neighboring CTU. For example, cc_sao_merge_up_flag indicates whether the CCSAO offset is merged from the left CTU or from the top CTU.

[0115] In some embodiments, the video signal further includes a third component, and the method of decoding the video signal using CCSAO further includes: receiving a second plurality of offsets associated with the third component; obtaining a second classification category associated with the third component using a characteristic metric of the first component; selecting a third offset from the second plurality of offsets for the third component according to the second classification category; and modifying the third component based on the selected third offset.

[0116] Other embodiments include various subsets of the above embodiments that are combined or otherwise rearranged in various other embodiments.

[0117] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored on or transmitted via a computer-readable medium as one or more instructions or code, and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used to implement the implementations described in this application. Computer program products may include computer-readable media.

[0118] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or,” as used herein, refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprising” and / or “including…”, when used in this specification, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0119] It will also be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0120] Throughout this specification, references to the singular or plural forms of "an example," "example," "exemplary example," etc., indicate that one or more specific features, structures, or characteristics described in connection with the example are included in at least one example of this disclosure. Therefore, phrases such as "in an example," "in the example," "in the exemplary example," etc., appearing in various places throughout this specification, do not necessarily refer to the same example. Furthermore, specific features, structures, or characteristics in one or more examples can be combined in any suitable manner.

[0121] The description in this application has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the invention in the forms disclosed. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the associated drawings. The embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention, and preferably utilize the basic principles and various embodiments with modifications suitable for the intended particular use. Therefore, it will be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for encoding video, comprising: Determine multiple offsets associated with the first chroma component of the video; The classification category associated with the first chromaticity component is obtained by using the sample characteristics of the luminance component of the video. A first offset is selected from the plurality of offsets for the first chromaticity component according to the classification category; Modify the sample value of the first chromaticity component based on the selected first offset; Transmit a video signal including the luminance component and the first chrominance component; Send multiple offsets associated with the first chromaticity component.

2. The method of claim 1, wherein obtaining the classification category associated with the first chroma component using the sample characteristic measure of the luminance component of the video comprises: The corresponding sample points of the luminance component are used to obtain the corresponding classification category of each sample point of the first chromaticity component, wherein the corresponding sample points of the luminance component are the corresponding isotopic sample points of the luminance component for each sample point of the first chromaticity component.

3. The method of claim 1, wherein obtaining the classification category associated with the first chroma component using the sample characteristic measure of the luminance component of the video comprises: The corresponding sample points of the luminance component are used to obtain the corresponding classification category of each sample point of the first chrominance component, wherein the corresponding sample points of the luminance component are reconstructed sample points after being deblocked.

4. The method of claim 1, wherein the sample characteristic metric is derived from a plurality of bands selected based on the values ​​of the samples in the luminance component, wherein each of the plurality of bands includes a range of sample values ​​of the luminance component.

5. The method according to claim 1, wherein the sample characteristic metric is derived based on the direction and intensity of the edge information of the sample points in the luminance component.

6. The method according to claim 1, wherein modifying the sample value of the first chromaticity component comprises: The selected first offset is directly added to the reconstructed sample value of the first chromaticity component after the deblocking and sample adaptive offset (SAO) processes have been performed.

7. The method according to claim 1, wherein modifying the sample value of the first chromaticity component comprises: The selected first offset is mapped to the second offset, and the mapped second offset is added to the sample value of the first chromaticity component.

8. The method of claim 1, wherein transmitting the video signal comprises: A semantic element is sent in the sequence parameter set, wherein the semantic element indicates whether cross-component sample adaptive offset CCSAO is enabled for the video.

9. The method of claim 1, wherein transmitting the video signal comprises: A semantic element is sent at the strip level, wherein the semantic element indicates whether cross-component sample adaptive offset CCSAO is enabled for the first chroma component.

10. The method of claim 1, wherein sending a plurality of offsets associated with the first chromaticity component comprises: Send different offsets for different coding tree units (CTUs).

11. The method of claim 1, wherein sending a plurality of offsets associated with the first chromaticity component comprises: Send a semantic element, wherein the semantic element indicates whether the offset sent by the CTU is the same as the offset of one of the CTU's neighboring CTUs, wherein the neighboring CTU is either the left neighboring CTU or the top neighboring CTU.

12. The method according to claim 1, further comprising: Determine a second plurality of offsets associated with the second chroma component of the video; The second classification category associated with the second chromaticity component is obtained by using the sample point characteristic measurement of the luminance component; A third offset is selected from the second plurality of offsets for the second chromaticity component according to the second classification category; The sample values ​​of the second chromaticity component are modified based on the selected third offset.

13. An electronic device comprising: One or more processing units; A memory coupled to one or more processing units; as well as Multiple programs are stored in the memory, and when executed by the one or more processing units, the multiple programs cause the electronic device to perform the method according to any one of claims 1 to 12.

14. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein, When executed by one or more processing units, the plurality of programs cause the electronic device to perform the method according to any one of claims 1 to 12 to generate a bit stream and store the bit stream.

Images