Chroma Residual Encoding and Decoding

Abstract

The present application relates to decoding video data. An electronic device reconstructs multiple residual blocks of a video frame from a video bitstream. Luminance residuals and chrominance residuals of multiple residual blocks of the video frame are clipped into a first dynamic range and a second dynamic range, respectively. For each residual block, a luminance residual for a luminance component and a chrominance residual for two chrominance components are determined based on the clipped luminance residuals and chrominance residuals of multiple residual blocks of the video frame. In some embodiments, the electronic device determines whether a joint coding and decoding (JCCR) mode of chrominance residuals is enabled. Based on determining that the JCCR mode is enabled, the chrominance residuals for the two chrominance components are jointly determined according to the JCCR scheme.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 955,319, filed on December 30, 2019, entitled “Joint Coding of Chrominance Residuals,” which is incorporated herein by reference in its entirety.

[0003] This application is related to a PCT application entitled “Methods and Apparatus of Joint Coding of Chroma Residuals” with application number PCT / US2020 / 030743 filed on April 30, 2020, which claims priority to U.S. provisional patent application No. 62 / 841,158 filed on April 30, 2019, entitled “Joint Coding of Chrominance Residuals”, all of which are hereby incorporated by reference in their entirety. Technical Field

[0004] The present application relates generally to video data encoding and compression, and in particular, to methods and apparatus for decoding chroma residuals. Background Art

[0005] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression standards such as those defined by MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC) and Versatile Video Codec (VVC) standards. Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video codecs, a video frame is partitioned into one or more slices, each with multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU), or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf-CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU can be encoded and decoded in intra mode, inter mode, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are encoded using spatial predictions of reference samples in neighboring blocks within the same video frame. Video blocks in an inter-coded (P or B) slice of a video frame can use spatial predictions of reference samples in neighboring blocks within the same video frame, or temporal predictions of reference samples in other previous and / or future reference video frames.

[0006] A prediction block for the current video block to be encoded is derived based on spatial prediction or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished by a block matching algorithm. The residual data representing the pixel differences between the current block to be encoded and the prediction block is called a residual block or prediction error. Inter-coded blocks are encoded according to a motion vector and a residual block, which points to a reference block in a reference frame that forms the prediction block. The process of determining a motion vector is generally referred to as motion estimation. Intra-coded blocks are encoded according to an intra-prediction mode and a residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain (e.g., frequency domain) to derive residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients and then entropy encoded into a video bitstream to achieve even greater compression.

[0007] The coded video bitstream is then stored in a computer-readable storage medium (e.g., a flash memory) to be accessed by another electronic device with digital video capabilities or directly sent to the electronic device by wire or wirelessly. The electronic device then performs video decompression (which is the reverse process of the video compression described above) by, for example, parsing the coded video bitstream to obtain syntax elements from the bitstream, and reconstructing digital video data from the coded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and presents the reconstructed digital video data on a display of the electronic device.

[0008] In order to maintain flexibility and scalability, video codec standards usually define options for the syntax of the encoded video bitstream, which details the parameters allowed by the syntax in the bitstream. In many cases, the options also provide details about the decoding operations that the decoder should perform to obtain the syntax parameters from the bitstream and obtain correct results in decoding. As the quality of digital video changes from HD to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. How to encode / decode video data more efficiently while maintaining the image quality of the decoded video data is a long-standing challenge. Summary of the invention

[0009] The present application describes embodiments related to encoding and decoding of video data, and more specifically, describes embodiments related to methods and devices for decoding luma residuals and chroma residuals. In one aspect of the present application, a method for decoding video data includes: reconstructing multiple residual blocks of a video frame from a video bitstream. The method also includes: clipping the luma residuals and chroma residuals of the multiple residual blocks of the video frame into a first dynamic range and a second dynamic range, respectively, and determining the luma residual for the luma component and the chroma residual for the two chroma components of each residual block of the video frame based on the clipped luma residuals and chroma residuals of the multiple residual blocks of the video frame. In some embodiments, the method also includes: determining whether the joint coding and decoding (JCCR) mode of chroma residuals is enabled. Based on determining that the JCCR mode is enabled, the chroma residuals for the two chroma components of the video frame are jointly determined according to the scheme of joint coding and decoding of the chroma residuals. In the example, one or both of the first dynamic range and the second dynamic range are defined as [-2 K-1 ,2 K-1 -1] or [-(2 K-1 -1),2 K-1 -1], where K is an integer (eg, 16). K may correspond to a predefined bit depth or an encoded bit depth of a video frame.

[0010] In another aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When the programs are executed by the one or more processing units, the electronic device performs the method for encoding video data as described above.

[0011] In another aspect of the present application, a non-volatile computer-readable storage medium stores a plurality of programs executed by an electronic device having one or more processing units. When the programs are executed by the one or more processing units, the electronic device executes the method for decoding video data as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of the specification, illustrate the described embodiments and together with the description serve to explain the basic principles. Like reference numerals designate corresponding parts.

[0013] Figure 1 is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0014] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0015] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0016] FIG. 4A to FIG. 4E is a block diagram illustrating how a frame may be recursively partitioned into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0017] Figure 5A and Figure 5B is a flowchart illustrating an exemplary process for a video encoder to implement a technique for encoding video data using a joint coding scheme of chroma residual according to some embodiments of the present invention.

[0018] FIG. 6A to FIG. 6C is a flowchart illustrating an exemplary process for a video decoder according to some embodiments of the present invention to implement a technique for decoding video data using a joint coding scheme of chroma residual.

[0019] Figure 7 is a block diagram of an exemplary video decoder configured for luma mapping and chroma scaling according to some embodiments.

[0020] Figure 8 is a block diagram of an exemplary video decoder configured to implement adaptive color transform (ACT) according to some embodiments.

[0021] Fig. 9A and Fig. 9B is a block diagram of two example residual decoding subsystems that may be applied in a video decoder according to some embodiments.

[0022] Fig. 10A and Fig. 10B are block diagrams of two further example residual decoding subsystems according to some embodiments.

[0023] Fig.11A and Fig. 11B is a flowchart of a video decoding method implemented by an electronic device according to some embodiments. DETAILED DESCRIPTION

[0024] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0025] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0026] In some embodiments, the target device 14 may receive the encoded video data to be decoded via the link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14. In one example, the link 16 may include a communication medium that enables the source device 12 to directly send the encoded video data to the target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and sent to the target device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that may be beneficial to facilitate communication from the source device 12 to the target device 14.

[0027] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. Subsequently, the encoded video data in the storage device 32 may be accessed by the target device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device 32 may correspond to a file server or another intermediate storage device that may hold the encoded video data generated by the source device 12. The target device 14 may access the stored video data from the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include a network server (e.g., for a website), an FTP server, a network attached storage (NAS) device, or a local disk drive. The target device 14 may access the encoded video data through any standard data connection suitable for accessing encoded video data stored on a file server, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both wireless channels and wired connections. The transmission of the encoded video data from the storage device 32 may be a streaming transmission, a download transmission, or a combination of both streaming transmission and download transmission.

[0028] like Figure 1As shown in , source device 12 includes video source 18, video encoder 20 and output interface 22. Video source 18 may include sources such as or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or a video phone. However, the embodiments described in this application may be generally applicable to video encoding and decoding, and may be applied to wireless and / or wired applications.

[0029] The captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video data may be sent directly to the target device 14 via the output interface 22 of the source device 12. The encoded video data may also (or alternatively) be stored on a storage device 32 for later access by the target device 14 or other device for decoding and / or playback. The output interface 22 may further include a modem and / or a transmitter.

[0030] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0031] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device and an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0032] The video encoder 20 and the video decoder 30 may operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, Advanced Video Codec (AVC)), or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 may be configured to encode video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 may be configured to decode video data according to any of these current standards or future standards.

[0033] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, respectively, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-volatile computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0034] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described herein. The video encoder 20 may perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0035] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. The loop filter 66 can be located between the adder 62 and the DPB 64, and includes a deblocking filter to filter the block boundary and remove the block effect from the reconstructed video. The loop filter 66 also includes a sample adaptive offset (SAO) filter and / or an adaptive loop filter (ALF) to filter the output of the adder 62 before the output of the adder 62 is input to the DPB 64 and used to encode and decode other video blocks. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0036] The video data memory 40 may store video data to be encoded by the components of the video encoder 20. The video data in the video data memory 40 may be obtained, for example, from the video source 18. The DPB 64 is a buffer that stores reference video data for use by the video encoder 20 (e.g., in an intra-frame or inter-frame prediction coding mode) when encoding the video data. The video data memory 40 and the DPB 64 may be formed by any of a variety of memory devices. In various examples, the video data memory 40 may be on-chip with other components of the video encoder 20, or off-chip relative to those components.

[0037] like Figure 2As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into slices, tiles, or other larger coding units (CUs) according to a predefined partitioning structure associated with the video data (such as a quadtree structure). The video frame may be divided into a plurality of video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of a plurality of possible prediction coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (such as motion vectors, intra-frame mode indicators, segmentation information, and other such semantic information) to the entropy coding unit 56.

[0038] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0039] In some embodiments, the motion estimation unit 42 determines the inter-prediction mode for the current video frame by generating a motion vector according to a predetermined pattern within the sequence of video frames, the motion vector indicating the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. The motion estimation performed by the motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, the motion vector may indicate the displacement of a PU of a video block within the current video frame or picture relative to a prediction block within a reference frame (or other coding unit) associated with the current block being encoded within the current frame (or other coding unit). The predetermined pattern may designate the video frames in the sequence as P frames or B frames. The intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by the motion estimation unit 42 for inter prediction, or may utilize the motion estimation unit 42 to determine the block vector.

[0040] A prediction block is a block of a reference frame that is considered to closely match a PU of a video block to be encoded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some embodiments, video encoder 20 may calculate values ​​for sub-integer pixel positions of reference frames stored in DPB 64. For example, video encoder 20 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of a reference frame. Thus, motion estimation unit 42 may perform motion searches relative to full pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0041] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-prediction coded frame by comparing the position of the PU to the position of a prediction block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy encoding unit 56.

[0042] The motion compensation performed by the motion compensation unit 44 may involve obtaining or generating a prediction block based on the motion vector determined by the motion estimation unit 42. After receiving the motion vector for the PU of the current video block, the motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from the DPB 64, and forward the prediction block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include a luminance difference component or a chrominance difference component or both. The motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining a motion vector for identifying a prediction block, any flag indicating a prediction mode, or any other syntax information described herein. It should be noted that the motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0043] In some embodiments, the intra BC unit 48 may generate vectors and obtain prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra BC unit 48 may determine the intra prediction mode to be used to encode the current block. In some examples, the intra BC unit 48 may encode the current block using various intra prediction modes, for example, during separate encoding passes, and test their performance through rate-distortion analysis. Next, the intra BC unit 48 may select a suitable intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​for the various tested intra prediction modes using rate-distortion analysis, and select an intra prediction mode with the best rate-distortion characteristic among the tested modes as a suitable intra prediction mode to use. The rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra BC unit 48 may calculate ratios based on the distortion and rate for the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0044] In other examples, intra BC unit 48 may use, in whole or in part, motion estimation unit 42 and motion compensation unit 44 to perform such functions for intra BC prediction in accordance with embodiments described herein. In either case, for intra block copying, the prediction block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and identifying the prediction block may include calculating values ​​for sub-integer pixel positions.

[0045] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0046] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 or the intra-frame block copy prediction performed by the intra BC unit 48 as described above, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine the intra-frame prediction mode for encoding the current block. To this end, the intra-frame prediction processing unit 46 can use various intra-frame prediction modes to encode the current block, for example, during separate encoding passes, and the intra-frame prediction processing unit 46 (or in some examples, the mode selection unit) can select a suitable intra-frame prediction mode from the tested intra-frame prediction modes to use. The intra-frame prediction processing unit 46 can provide information indicating the intra-frame prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0047] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0048] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0049] After quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to video decoder 30, or archived in storage device 32 for later sending to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0050] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0051] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0052] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0053] In some examples, units of the video decoder 30 may be tasked to perform embodiments of the present application. In addition, in some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30 (e.g., the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., the motion compensation unit 82).

[0054] The video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 may be obtained, for example, from the storage device 32, from a local video source (e.g., a camera), via a wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. A decoded picture buffer (DPB) 92 of the video decoder 30 stores reference video data for use by the video decoder 30 when decoding the video data (e.g., in intra-frame or inter-frame prediction coding modes). The video data memory 79 and the DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data memory 79 and the DPB 92 are shown in FIG. Figure 3 92 are depicted as two different components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and the DPB 92 can be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 can be on-chip with other components of the video decoder 30, or off-chip relative to those components.

[0055] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0056] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0057] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0058] In some examples, when a video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0059] The motion compensation unit 82 and / or the intra BC unit 85 determine prediction information for a video block of the current video frame by parsing the motion vector and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine a prediction mode (e.g., intra prediction or inter prediction) for encoding a video block of a video frame, an inter prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, motion vectors for each inter prediction encoded video block of the frame, inter prediction states for each inter prediction encoded video block of the frame, and other information for decoding a video block in the current video frame.

[0060] Similarly, the intra BC unit 85 may use some of the received syntax elements, such as flags, to determine whether the current video block is predicted using intra BC mode, construction information of which video blocks of the frame are within the reconstruction region and should be stored in the DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding the video block in the current video frame.

[0061] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0062] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0063] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. The loop filter 94 can be located between the adder 90 and the DPB 92 and includes a deblocking filter to filter the block boundaries and remove block effects from the decoded video block. The loop filter 94 also includes an SAO filter and an ALF to filter the decoded video block output by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 can also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0064] In a typical video encoding process, a video sequence usually includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore include only a two-dimensional array of luma samples.

[0065] like Figure 4A As shown in , the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in a raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set so that all CTUs in a video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. As Figure 4BAs shown in , each CTU may include one coding tree block (CTB) of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units of coding pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements for encoding the samples of the coding tree block. The coding tree block may be an N×N block of samples.

[0066] To achieve better performance, the video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination thereof, on the coding tree block of the CTU and divide the CTU into smaller coding units (CUs). Figure 4C As depicted in , a 64×64 CTU 400 is first divided into four smaller CUs, each CU having a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are respectively divided into four CUs with a block size of 16×16. Two 16×16 CUs 430 and CU 440 are further divided into four CUs with a block size of 8×8. Figure 4D Depicted is a diagram showing Figure 4C The quadtree data structure of the final result of the partitioning process of the CTU 400 depicted in FIG. 4 corresponds to a CU of a corresponding size ranging from 32×32 to 8×8. Figure 4B Each CU may include a coding block (CB) of luma samples and two corresponding coding blocks of chroma samples of the same size frame, and syntax elements for encoding the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures for encoding the samples of the coding block. It should be noted that Figure 4C and Figure 4D The quadtree partitioning depicted in FIG is for illustrative purposes only, and one CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In a multi-type tree structure, one CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary and ternary tree structure. Figure 4E As shown, there are five types of segmentation, namely, quadruple segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal ternary segmentation, and vertical ternary segmentation.

[0067] In some embodiments, the video encoder 20 may further partition the coding block of the CU into one or more (M×N) prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter or intra) is applied. The prediction unit (PU) of the CU may include a prediction block of luminance samples, two corresponding prediction blocks of chrominance samples, and syntax elements for predicting the prediction blocks. In a monochrome picture or a picture with three separate color planes, a PU may include a single prediction block and a syntax structure for predicting the prediction block. The video encoder 20 may generate a predicted luminance block, a predicted Cb block, and a predicted Cr block for the luminance prediction block, the Cb prediction block, and the Cr prediction block of each PU of the CU.

[0068] Video encoder 20 may use intra prediction or inter prediction to generate a prediction block for a PU. If video encoder 20 uses intra prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0069] After the video encoder 20 generates the predicted luma block, the predicted Cb block, and the predicted Cr block for one or more PUs of a CU, the video encoder 20 may generate a luma residual block for the CU by subtracting the predicted luma block of the CU from the original luma coding block of the CU, so that each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, so that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0070] In addition, if Figure 4CAs shown in , the video encoder 20 can use quadtree partitioning to decompose the luma residual block, Cb residual block and Cr residual block of the CU into one or more luma transform blocks, Cb transform blocks and Cr transform blocks. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. The transform block (TU) of the CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU may be associated with a luma transform block, a Cb transform block and a Cr transform block. In some examples, the luma transform block associated with the TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax structure for transforming the samples of the transform block.

[0071] The video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficient may be a scalar. The video encoder 20 may apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0072] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode a syntax element indicating the quantized transform coefficients. For example, the video encoder 20 may perform context-adaptive binary arithmetic coding and decoding (CABAC) on the syntax element indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits that form a representation of an encoded frame and associated data, and the bitstream is stored in a storage device 32 or sent to a target device 14.

[0073] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 can reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally inverse to the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coding block of the current CU by adding the samples of the prediction block for the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coding block for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0074] Recent studies have shown that there appears to be a correlation between the Cb residual and the Cr residual of a CU. In some cases, the two chroma residuals appear to be inversely correlated with each other. In this case, a mode for joint coding of chroma residuals is proposed to signal only one chroma residual block (e.g., Cb residual block) of the CU and a flag indicating the enablement of joint coding of the chroma residual, thereby improving coding efficiency. In some embodiments, the average of the positive Cb residual and the negative Cr residual is used as the joint residual of the two components, thereby improving accuracy when the two chroma residuals are not exactly inversely correlated, as follows:

[0075] resJoint=(resCb–resCr) / 2,

[0076] Where resCb represents the Cb residual block of the CU, and resCr represents the Cr residual block of the CU.

[0077] In some embodiments, the video codec computes both the average sum block and the average difference block between two chroma residuals as follows:

[0078] resJointCb=(resCb–resCr) / 2;

[0079] resJointCr=(resCb+resCr) / 2

[0080] Compared to the two residual blocks resCb and resCr, the values ​​in the average difference block resJointCr have smaller magnitudes and can be quantized with a small number of bits at the same or similar level of precision.

[0081] In some embodiments, different modes for joint coding of chroma residuals are proposed, where each mode targets a specific correlation relationship between Cb residual and Cr residual, as follows:

[0082] Mode 1: Cb is encoded and decoded and Cr is obtained according to Cr = CSign * Cb / 2;

[0083] Mode 2: Cb is encoded and decoded and Cr is obtained according to Cr = CSign * Cb;

[0084] Mode 3: Cr is encoded and decoded and Cb is obtained according to Cb=CSign*Cr / 2,

[0085] Wherein CSign represents a sign used to obtain the second chroma residual block from the first chroma residual block. CSign is signaled as a tile group header syntax element and has a value of -1 or 1.

[0086] In some embodiments, the mode of joint coding of chroma residual is signaled via a TU level flag (i.e., tu_cb_cr_joint_residual). If tu_cb_cr_joint_residual is equal to 1, one of the three modes described above is used. The specific mode used is derived from the signaled chroma coding block flag (CBF) according to the following table:

[0087]

[0088] Table 1: Derivation of joint chroma residual coding and decoding mode based on CBF

[0089] In some embodiments, if the joint chroma encoding mode is selected, the quantization parameter (QP) used to encode the joint chroma residual component is reduced by 1 (for modes 1 and 3) or 2 (for mode 2).

[0090] In summary, the video encoder 20 obtains a joint chroma residual by corresponding mixing operations of the Cb residual and the Cr residual, and selects one of the three supported chroma codec modes (including CSign) based on a distortion analysis (e.g., the distortion obtained by first mixing the Cb residual and the Cr residual into a joint chroma residual and then reconstructing the Cb residual and the Cr residual from the joint chroma residual without quantization). The selected mode is then tested in an additional mode decision process (i.e., using transform, quantization, and entropy coding). In some embodiments, a tile group header syntax element indicating a sign (CSign) for obtaining a second chroma component is determined by analyzing the correlation between the original Cb component and the high-pass filtered version of the Cr component of the tile group.

[0091] In some embodiments, the correlation between the first chroma residual and the second chroma residual indicates that the signaling of the tu_cb_cr_joint_residual flag depends on the signaling of one but not two chroma coding block flags. For example, if the chroma coding block flag that is signaled first has a value of 1, the tu_cb_cr_joint_residual flag will be signaled, and due to the correlation between the first chroma residual block and the second chroma residual block, the second chroma coding block flag does not need to be signaled. The second chroma coding block flag is signaled only when the tu_cb_cr_joint_residual flag has a value of 0 (i.e., there is no correlation between the first chroma residual block and the second chroma residual block).

[0092] In some embodiments, one or both contexts are used for CABAC encoding and decoding of the tu_cb_cr_joint_residual flag. For example, one of the two contexts is selected based on the value of the Cr coded block flag. When the Cr coded block flag is 1, one context is used; otherwise (i.e., the Cr coded block flag is equal to 0), the other context is used. If the Cb coded block flag is equal to 1, the TU level flag tu_cb_cr_joint_residual is signaled, and the two contexts are used to encode and decode the tu_cb_cr_joint_residual flag.

[0093] In some embodiments, the TU level flag tu_cb_cr_joint_residual is signaled only when both chroma CBFs are 1. When the tu_cb_cr_joint_residual flag has a value of 1, an additional syntax element is signaled to indicate which of the three modes is selected. For CABAC encoding and decoding of mode syntax, different codeword binarizations may be used. An exemplary codeword binarization may be a truncated unary codeword with a maximum codeword index of 2, as shown in Table 2 below.

[0094]

[0095] Table 2: Codeword binarization for joint chroma codec mode signaling

[0096] In some embodiments, additional syntax elements are proposed to control the syntax signaling of the mode of joint coding and decoding of chroma residual at different levels. For example, the syntax element can be signaled at the video sequence level, the picture level or the tile group level, the tile level or the slice level. When the syntax element is signaled with a value of 1 at a particular level, a TU-level control flag (i.e., tu_cb_cr_joint_residual) at or below that level is also signaled to indicate the use of joint coding and decoding of chroma residual. When the syntax is signaled with a value of 0, joint coding and decoding of chroma residual is disabled at that level, and when the CU is coded and decoded at the level at which the flag is signaled with a value of 0 or at a level below which the flag is signaled with a value of 0, the TU-level control flag will not be coded and decoded.

[0097] Figure 5A and Figure 5B is a flow chart illustrating an exemplary process 500 for implementing a technique for encoding video data using a joint coding scheme for chroma residuals by a video encoder 20 according to some embodiments of the present disclosure. The video encoder 20 obtains a first syntax element associated with a first layer of the hierarchical structure from a video bitstream having a hierarchical structure (510). As explained above, there are multiple options for the first layer, and therefore, the first syntax element may be in one of a sequence parameter set (SPS), a picture parameter set (PPS), a tile group header, a tile header, a slice header, etc. The multiple chroma components of each of the one or more blocks correspond to a transform unit of the video data, which is in turn associated with a particular coding unit.

[0098] Next, the video encoder 20 checks the value of the first syntax element (e.g., in the form of a one-bit flag) and determines whether the syntax element indicates that the joint coding and decoding mode of the chroma residual is enabled (530). For example, a value of 1 indicates that the joint coding and decoding mode of the chroma residual is enabled, and a value of 0 indicates that the joint coding and decoding mode of the chroma residual is disabled. If the joint coding and decoding mode of the chroma residual is enabled (530-yes), the video encoder 20 then jointly encodes the chroma residuals for multiple chroma components of one or more blocks under the first layer into the video bitstream according to a predefined scheme for joint coding and decoding of the chroma residual (550). As described above, at least three different schemes for joint coding and decoding of chroma residuals are proposed to process different types of video data. One of the multiple chroma components is processed using various syntax elements and CABAC contexts accordingly, and other chroma components are obtained from the processed chroma components according to the correlation relationship under the predefined scheme for joint coding and decoding of the chroma residual. If the joint coding mode of the chroma residual is disabled (530-No), the video encoder 20 then encodes the chroma residuals for the multiple chroma components of the one or more blocks under the first layer into the video bitstream separately (570). In other words, each of the multiple chroma components of the one or more blocks is encoded into the video bitstream, and for each CU, the TU level control flag tu_cb_cr_joint_residual is set to zero.

[0099] Finally, video encoder 20 outputs a video bitstream (590) including an encoded chroma residual of multiple chroma components of one or more blocks and a first syntax element.

[0100] In some embodiments, Figure 5BAs shown in , after the first syntax element indicates that the joint coding mode of the chroma residual is enabled, the video encoder 20 obtains a second syntax element associated with each of the one or more blocks (550-1), and determines whether the second syntax element indicates that the block-level joint coding mode of the chroma residual is enabled (550-3). If so (550-3, yes), the video encoder 20 jointly encodes the chroma residuals for multiple chroma components of the block into the video bitstream according to the predefined scheme of the joint coding of the chroma residual (550-5); otherwise (550-3, no), the video encoder 20 encodes the chroma residuals for multiple chroma components of the block into the video bitstream separately (550-7). In other words, a value of 0 at the first syntax element may disable the application of the joint coding of the chroma residuals of all blocks under the first layer, so that the second syntax element does not need to be signaled at the block level. However, a value of 1 at the first syntax element does not indicate that each block under the first layer must be encoded using one of the schemes of the joint coding of the chroma residual. By selecting the second syntax element, each block still has its own control, thereby enhancing the flexibility of implementing the video encoder.

[0101] In some embodiments, the video encoder 20 selects a mode from a plurality of modes (see, e.g., Table 1 above) according to the values ​​of the chroma coding flags of the plurality of chroma components of the block, which may require rate-distortion analysis. Then, the video encoder 20 encodes the chroma residual of one chroma component of the plurality of chroma components of the block and the values ​​of the chroma coding flags of the plurality of chroma components of the block into the video bitstream, respectively, according to the selected mode.

[0102] FIG. 6A to FIG. 6C 600 is a flowchart showing an exemplary process 600 of a technique for decoding video data using a joint coding scheme of chroma residuals implemented by a video decoder according to some embodiments of the present disclosure. The video decoder 30 receives a first syntax element associated with a first layer of the hierarchical structure from a video bitstream having a hierarchical structure (610), and then checks whether the first syntax element indicates that the joint coding mode of the chroma residual is enabled (630). If so (630-yes), the video decoder 30 jointly reconstructs the chroma residuals for multiple chroma components of one or more blocks under the first layer from the video bitstream according to a predefined scheme for joint coding of the chroma residuals (650). Otherwise (630-no), the video decoder 30 reconstructs the chroma residuals for multiple chroma components of one or more blocks under the first layer from the video bitstream, respectively (670). As explained above, there are multiple options for the first layer, and therefore, the first syntax element may be in one of a sequence parameter set (SPS), a picture parameter set (PPS), a tile group header, a tile header, a slice header, etc. The multiple chroma components of each of the one or more blocks correspond to a transform unit of the video data, which is in turn associated with a particular coding unit.

[0103] In some embodiments, Figure 6B As shown in , after the first syntax element indicates that the joint coding mode of the chroma residual is enabled, the video decoder 30 receives a second syntax element associated with each of one or more blocks from the video bitstream (650-1), and determines whether the second syntax element indicates that the block-level joint coding mode of the chroma residual is enabled (650-3). If so (650-3, yes), the video decoder 30 jointly reconstructs the chroma residuals for multiple chroma components of the block from the video bitstream according to the predefined scheme of the joint coding of the chroma residual (650-5); otherwise (650-3, no), the video decoder 30 reconstructs the chroma residuals for multiple chroma components of the block from the video bitstream separately (650-7). In other words, a value of 0 at the first syntax element may disable the application of the joint coding of the chroma residuals of all blocks under the first layer, so that the second syntax element does not need to be signaled at the block level. However, a value of 1 at the first syntax element does not indicate that each block under the first layer must be encoded using one of the schemes of the joint coding of the chroma residual. By selecting the second syntax element, each block still has its own control, thereby enhancing the flexibility of implementing a video decoder.

[0104] In some embodiments, Figure 6C As shown in and as described above in conjunction with Table 1, each of the multiple chroma components of the block has a chroma coding flag, and the predefined scheme for joint coding of the chroma residual has multiple modes (650-11). The video decoder 30 selects a mode (650-13) from multiple modes (see, for example, Table 1 above) according to the values ​​of the chroma coding flags of the multiple chroma components of the block, and then reconstructs the chroma residuals (650-15) for the multiple chroma components of the block from the video bitstream according to the selected mode. Assuming that the multiple chroma components of the block include a first chroma component (e.g., a Cb component) and a second chroma component (e.g., a Cr component) (650-15-1), the video decoder 30 reconstructs the chroma residuals (650-15-3) for the first chroma component of the block from the video bitstream, and as described above with respect to the multiple modes of joint coding of the chroma residual, the chroma residuals (650-15-5) for the second chroma component are directly obtained from the chroma residuals for the first chroma component of the block.

[0105] As shown in Table 1 above, when the TU level tu_cb_cr_joint_residual flag is 1, and Cb CBF is 1 and CrCBF is 0, mode 1 is selected. However, it is still possible that both chroma blocks (Cb block and Cr block) actually have non-zero residuals, resulting in a difference between the chroma block CBF value sent with the signal and the actual corresponding chroma block residual. When such chroma CBF value is subsequently used for other purposes (for example, as a context for encoding and decoding other syntax), this difference may affect the encoding and decoding performance. Figure 6C As shown in , video decoder 30 may reset chroma coding flags for multiple chroma components of a block to predefined values ​​(650-17). For example, in a scheme with joint chroma residual coding and decoding with multiple modes, when the TU level flag tu_cb_cr_joint_residual is signaled as 1, the Cb and Cr chroma coded block flag (CBF) syntax elements are reset to 1 after the current block is reconstructed, regardless of which of the three modes is used. For example, in mode 1, even if the Cr CBF signaled is 0, it will be reset to 1 after the current block is reconstructed.

[0106] Figure 7 is a block diagram of an exemplary video decoder 30 configured for luma mapping and chroma scaling (LMCS) according to some embodiments. In the video decoder 30, a first plurality of decoding modules (shadow) are implemented in a mapped domain and include an entropy decoding unit 80, an inverse quantization unit 86, an inverse transform processing unit 88, a luma intra prediction unit 84A, and a luma sample reconstruction unit 90A. Luma prediction samples and luma residual samples are added in the mapped domain. A second plurality of decoding modules (non-shadow) are implemented in a non-mapped domain (also referred to as an original domain) and include a motion compensated prediction unit 82, a chroma intra prediction unit 84B, a chroma sample reconstruction unit 90B, and a loop filter 94, which further includes one or more of a deblocking filter, an SAO filter, and an ALF filter. Chroma prediction samples and chroma residual samples are added in the non-mapped domain. The luma component and chroma component of the reference picture are output in the non-mapped domain and stored in a decoded picture buffer (DPB) 92. LMCS is applied before loop filters 94A and 94B configured to process the luma component and chroma component of the reconstructed video data, respectively. LMCS is implemented jointly by a loop mapping module 96 for the luma component and a luma-dependent chroma residual scaling unit 98. The loop mapping module 96 is implemented based on an adaptive piecewise linear model and further includes a forward mapping unit 96A and an inverse mapping unit 96B.

[0107] The loop mapping module 96 is configured to adjust the dynamic range of the input video data to improve the coding efficiency. The forward mapping unit 96A is based on the forward mapping function FwdMap, and the inverse mapping unit 96B is based on the corresponding inverse mapping function InvMap. The forward mapping function FwdMap is sent from the video encoder 20 to the video decoder 30 using a luminance mapping model 702 (e.g., a piecewise linear model with 16 equal-sized segments). In some embodiments, the inverse mapping function InvMap is directly derived from the forward mapping function FwdMap and does not need to be sent by signal.

[0108] In some embodiments, one or more parameters of the mapping unit 96 (e.g., parameters of the brightness mapping model 702) are signaled at the picture level. For example, a presence flag is signaled to indicate whether the brightness mapping model 702 is present in the current image strip. If the brightness mapping model 702 is present in the current image strip, a plurality of piecewise linear model parameters are further signaled to facilitate the implementation of the piecewise linear model. Based on the piecewise linear model, the dynamic range of the input video data is divided into 16 segments of equal size in the original domain, and each segment of the dynamic range of the input video data is mapped to a corresponding segment in the original domain. For a given segment in the original domain, its corresponding segment in the mapping domain may have the same size or a different size as the given segment in the original domain. The size of each segment in the mapping domain is indicated by the number of codewords (i.e., codewords storing mapping sample values) of the corresponding segment. For each segment in the original domain, the piecewise linear mapping parameters may be obtained based on the number of codewords in its corresponding segment in the mapping domain. In the example, the input video data has a resolution of 10 bits. If each of the segments in the mapped domain has 64 codewords assigned to the corresponding segment, then each of the 16 segments in the original domain has 64 pixel values ​​according to a one-to-one mapping (i.e., a mapping where each sample value is unchanged). The number of codewords signaled for each segment in the mapped domain is used to calculate the scaling factor and the forward mapping function or the inverse mapping function is adjusted accordingly for the segment. Furthermore, in some embodiments, another LMCS control flag is signaled to enable / disable LMCS for an image slice at a picture level.

[0109] In the example, for the i-th segment (i=0...15), the corresponding piecewise linear model applied by the forward mapping function FwdMap of the forward mapping unit 96A is defined by two input pivot points InputPivot[i] and InputPivot[i+1] and two output (mapped) pivot points MappedPivot[i] and MappedPivot[i+1]. Furthermore, if the input video data has a resolution of 10 bits, the values ​​of InputPivot[i] and MappedPivot[i] (i=0...15) are calculated as follows:

[0110] 1. Set the variable OrgCW = 64

[0111] 2. For i = 0:16, InputPivot[i] = i * OrgCW

[0112] 3. For i = 0:16, MappedPivot[i] is calculated as follows:

[0113] MappedPivot[0] = 0;

[0114] for(i=0;i<16;i++)

[0115] MappedPivot[i+1]=MappedPivot[i]+SignaledCW[i]

[0116] where SignaledCW[i] is the number of codewords signaled for the i-th segment.

[0117] A chroma residual scaling (CRS) unit 98 is connected to the inverse transform processing unit 88 and is configured to compensate for the mutual influence of quantization precision between the luma component and the corresponding chroma component when loop mapping is applied to the luma component. A first CRS flag is signaled in the picture header to indicate whether chroma residual scaling is enabled or disabled. If luma mapping is enabled and dual-tree segmentation of luma components and chroma components is disabled for the current picture, a second CRS flag is signaled to indicate whether luma-dependent chroma residual scaling is applied. If luma mapping is not used or dual-tree segmentation is enabled for the current picture, chroma residual scaling is disabled via the first CRS flag. In addition, in some embodiments, chroma residual scaling is disabled for a CU containing less than or equal to four chroma samples.

[0118] For both intra CUs and inter CUs, the scaling parameter used to scale the chroma residual is determined based on the average value of the corresponding mapped luma prediction samples. Assume avg' Y is the average value of the brightness prediction samples in the mapping domain. The scaling parameter C is determined according to the following stepsScaleInv :

[0119] 1. Find avg' in the mapping domain Y The segment index Y of the piecewise linear model to which it belongs 1dx Here, Y 1dx Has an integer value ranging from 0 to 15.

[0120] 2. C ScaleInv =cScaleInv[Y 1dx ], where cScaleInv[i] (i=0…15) is a pre-computed 16-segment lookup table (LUT).

[0121] Intra prediction is performed in the mapped domain in LMCS. When the CU is encoded or decoded in intra prediction mode, combined intra and inter prediction (CIIP) mode, or intra block copy (IBC) mode, avg' Y is calculated as the average of the brightness prediction samples. Otherwise, avg' Y is calculated as the average of the inter-predicted luma samples in the forward mapping. In addition, unlike the luma mapping performed on a sample-by-sample basis, the original chroma residual sample values ​​C ScaleInv It is fixed for the entire chroma CU. Given the original chroma residual sample value C ScaleInv , chroma residual scaling is applied as follows:

[0122] Encoder side: C ResScale =C Res / C ScaleInv

[0123] Decoder side: C Res =C ResScale ×C ScaleInv

[0124] Among them C Res Represents the scaled chroma residual sample value.

[0125] Figure 8 is a block diagram of an exemplary video decoder 30 configured to implement Inverse Adaptive Color Transform (ACT) according to some embodiments. In the HEVC SCC extension, ACT is applied to reduce redundancy between three color components in the 444 chroma format. ACT is also adopted into the VVC standard to enhance the codec efficiency of 444 chroma format codecs. For example, ACT performs an in-loop color space conversion in the prediction residual domain by adaptively converting the residual from the input color space to the YCgCo color space. Figure 8, in some embodiments, two color spaces are adaptively selected by signaling an ACT flag at the CU level. When the ACT flag is equal to a first value ("1"), the residual of the CU is encoded in the YCgCo color space, and the residual of the CU is decoded using the inverse ACT unit 802. When the ACT flag is equal to a second value ("0"), the residual of the CU is encoded in the original color space of the input video data, and the residual of the CU is decoded without using the inverse ACT unit 802. In addition, in some embodiments, when there is at least one non-zero coefficient in the CU, ACT is enabled for the CU only in inter-frame prediction or IBC mode, and the inverse ACT unit 802 is enabled to decode the residual of the CU in inter-frame prediction or IBC mode. In some embodiments, when the chroma component selects the same intra-frame prediction mode as the luminance component, ACT is enabled for the CU only in intra-frame prediction mode, and the inverse ACT unit 802 and the intra-frame prediction unit 84 are enabled to decode the residual of the CU in intra-frame prediction mode.

[0126] In some embodiments, the color space conversion for ACT is based on the YCgCo transformation matrix. Specifically, the forward YCgCo color transformation matrix and the inverse YCgCo color transformation matrix are applied to convert the GBR vector to the YCgCo vector and the YCgCo vector to the GBR vector, as follows:

[0127]

[0128] In some embodiments, the ACT transform matrices (e.g., the forward YCgCo color transform matrix and the inverse YCgCo color transform matrix in the above equation) are not normalized. A QP adjustment (-5, -5, -3) is applied to transform the residuals of the Y, Cg, and Co components to compensate for the dynamic range change of the residual signal before and after the color transform. The adjusted QP affects the quantization and inverse quantization of the residuals in the CU. For other encoding and decoding processes (such as deblocking), the original and unadjusted QP is still applied. In addition, in some embodiments, the forward color transform and the inverse color transform require access to the residuals of all three color components, and ACT is always disabled for separate tree partitions and ISP modes with different prediction block sizes for different color components.

[0129] Fig. 9A and Fig. 9B 9 are block diagrams of two example residual decoding subsystems 900 and 950 that may be applied in the video decoder 30 according to some embodiments. Each of the residual decoding subsystems 900 and 950 includes an inverse quantization unit 86, an inverse transform processing unit 88, an inverse JCCR unit 902, an inverse ACT unit 802, and an inverse CRS unit 98. The inverse JCCR unit 902 and the inverse ACT unit 802 may be selectively arranged in Figure 7The inverse transform processing unit 88 and the CRS unit 98 in the video bitstream are connected to each other. The quantized transform coefficients are provided to the video decoder 30 in the video bitstream and are entropy decoded by the entropy decoding unit 80. The inverse quantization unit 86 uses the same quantization parameter calculated by the video encoder 20 for each video block in the video frame to inverse quantize the decoded quantized transform coefficients to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain. The reconstructed residual block includes the luminance component and the chrominance component of multiple pixels in the video frame. The inverse JCCR unit 902 is enabled in the JCCR mode to jointly determine the chrominance residual for the chrominance component of the video frame according to the joint coding and decoding scheme of the chrominance residual. For example, in JCCR mode, only one chroma residual block (e.g., Cb residual block) of the CU and a flag indicating that joint coding of the chroma residual is enabled are signaled, and two chroma residuals (e.g., Cb residual and Cr residual) are jointly determined from the only chroma residual block. The inverse ACT unit 802 is connected to the inverse JCCR unit 902 to apply an adaptive color transform to the chroma residual (e.g., both the Cb residual and the Cr residual) of the CU. The inverse CRS unit 98 is connected to the inverse ACT unit 802 and is configured to compensate for the mutual influence of quantization precision between the luminance component of the CU and the corresponding chroma component when loop mapping is applied to the luminance component of the CU.

[0130] In some embodiments, the residual generated by each unit of the residual decoding subsystem 900 and 950 is controlled within one or more respective dynamic ranges (e.g., corresponding to a 16-bit resolution including a sign bit). In the example residual decoding subsystems 900 and 950, a first clipping operation is applied, i.e., a first clipping unit 904 is connected between the inverse transform processing unit 88 and the inverse JCCR unit 902 to maintain a first dynamic range DR for the luma residual and the chroma residual, respectively. A and the second dynamic range DR B In the example, the dynamic range DR A and DR B One or both of corresponds to a first bit depth equal to 16 bits and is defined as [-2 15 ,2 15 -1](including 2 15 and 2 15 -1). The first clipping unit 904 generates a clipping function according to Clip(-2 15 ,2 15 -1,M) clips the corresponding residuals of the luminance residual and the chrominance residual of the residual block M output by the inverse transform processing unit 88 to the corresponding dynamic range DR A and / or DR BIn yet another example, the dynamic range DR A and DR B One or both of corresponds to a first bit depth equal to 16 bits and is defined as [-2 15 -1,2 15 -1](including -2 15 -1 and 2 15 -1). The first clipping unit 904 performs another clipping function Clip(-(2 15 -1),2 15 -1,M) clips the corresponding residuals of the luminance residual and the chrominance residual of the residual block M output by the inverse transform processing unit 88 to the corresponding dynamic range DR A and / or DR B In addition, in another example, the dynamic range DR A and DR B One or both of the B ,2 B -1](including -2 B and 2 B -1). The first clipping unit 904 uses the clipping function Clip(-2 B ,2 B -1,M) or Clip(-(2 B -1),2 B -1,M) clips the corresponding residuals of the luminance residual and the chrominance residual of the residual block M output by the inverse transform processing unit 88 to the corresponding dynamic range DR A and / or DR B The integer B is the luma residual obtained from the video bitstream and applied by inverse transform processing unit 88 to reconstruct the residual block of the CU of the corresponding video frame. The integer B refers to the encoding bit depth of the input video data.

[0131] Furthermore, in some embodiments, a bitstream consistency constraint is applied to ensure that the residual (i.e., the residual fed to the inverse quantization unit 86) is within a 16-bit dynamic range (i.e., [-2 15 ,2 15 -1]). Fig. 9A , no clipping unit is applied after the inverse JCCR unit 902, and when JCCR mode 2 with a signaled negative sign is selected (i.e., Cr=-Cb) and the decoded Cb residual samples are equal to the minimum value of a 16-bit signed integer (i.e., -2 15 ), the decoded residual output by JCCR unit 902 exceeds the 16-bit dynamic range. In these cases, the Cr residual sample is equal to 2 15 , 2 15 In the 16-bit range [-2 15 ,215 -1].

[0132] In the example residual decoding subsystem 950, two different clipping operations are applied. In addition to the first clipping unit 904, a second clipping unit 906 is connected to the output of the inverse JCCR unit 902, for example, connected between the inverse JCCR unit 902 and the inverse ACT unit 802, to maintain the chrominance dynamic range DR CR . Chroma dynamic range DR CR Corresponding to the chroma bit depth, the chroma bit depth is selectively equal to or less than the second dynamic range DR B The corresponding second bit depth. The second dynamic range DR B Chroma Dynamic Range DR CR Same as or including chrominance dynamic range DR CR For example, the second bit depth and the chroma bit depth are both equal to K, where K is an integer (eg, 16, or the encoded bit depth received by the video bitstream), and the second dynamic range DR is B and chroma dynamic range DR CR are identical and are selected from [-(2 K -1),2 K -1], [-2 K ,2 K -1]、[-(2 K -1),2 K ] and [-2 K ,2 K ].

[0133] Fig. 10A and Fig. 10B 1000 and 1050, respectively, are block diagrams of two other example residual decoding subsystems 1000 and 1050 that may be applied in the video decoder 30 according to some embodiments. Each of the residual decoding subsystems 1000 and 1050 includes an inverse quantization unit 86, an inverse transform processing unit 88, an inverse JCCR unit 902, an inverse ACT unit 802, and an inverse CRS unit 98. In both of the example residual decoding subsystems 1000 and 1050, at least three clipping operations are applied. A first clipping unit 904 is connected between the inverse transform processing unit 88 and the inverse JCCR unit 902 to maintain a first dynamic range DR of a luma residual and a chroma residual corresponding to a first bit depth and a second bit depth A and the second dynamic range DR B The third cropping unit 1002 is connected between the inverse quantization unit 86 and the inverse transform processing unit 88 to maintain a third dynamic range DR of the residual information corresponding to the third bit depth. CThe fourth cropping unit 1004 is connected between the inverse ACT unit 802 and the inverse CRS unit 98 to maintain a fourth dynamic range DR of the residual information corresponding to the fourth bit depth. D .

[0134] In some embodiments associated with the example residual decoding subsystem 1050, in addition to the first clipping unit 904, the third clipping unit 1002, and the fourth clipping unit 1004, the second clipping unit 906 is connected at the output of the inverse JCCR unit 902, for example, connected between the inverse JCCR unit 902 and the inverse ACT unit 802, to maintain the chroma dynamic range DR corresponding to the chroma bit depth CR In the example, the chrominance dynamic range DR CR corresponds to a bit depth of 16 bits, and a second clipping unit 906 is applied so that the reconstructed JCCR chroma residual output by the JCCR unit 902 does not exceed the 16-bit dynamic range DR CR .

[0135] The cropping units 1002, 904, 906 and 1004 are arranged in an ordered sequence on the video decoding path. For the luma residual or the chroma residual, the dynamic range corresponding to the upstream cropping unit on the video decoding path is equal to or includes the dynamic range of the downstream cropping unit on the video decoding path. Specifically, for the luma residual or the chroma residual, respectively, the third bit depth is selectively equal to or greater than the first bit depth or the second bit depth, and the third dynamic range DR C Equal to or including the first dynamic range DR A Or the second dynamic range DR B The second bit depth is selectively equal to or greater than the fourth bit depth, and the second dynamic range DR B Equal to or including the fourth dynamic range DR D In the residual decoding subsystem 1050, the second bit depth of the chroma residual is selectively equal to or greater than the chroma bit depth, and the second dynamic range DR B Equal to or including the chrominance dynamic range DR CR The chroma bit depth is selectively equal to or greater than the fourth bit depth, and the chroma dynamic range DR CR Equal to or including the fourth dynamic range DR D .

[0136] Reference Fig. 10B In the example, the first clipping unit 904, the second clipping unit 906 and the third clipping unit 1002 have a clipping function Clip(-2 15 ,2 15-1,M), i.e., have the same bit depth of 16 bits, while the fourth clipping unit 1004 has a different clipping function Clip(-2 B ,2 B -1,M) defined different dynamic ranges DR D Note that the clipping function clip(min,max,M) clips the input value M to the dynamic range of [min,max], and B is the encoding bit depth of the input video data. In addition, in some embodiments, there is a bitstream consistency constraint to ensure that the parsed residual (i.e., the residual fed to the inverse quantization unit 86) is within the 16-bit dynamic range (i.e., [-2 15 ,2 15 -1]). In some embodiments, Fig. 10A In the example, there is no clipping operation applied to the output of the inverse JCCR unit 902. When JCCR mode 2 with a signaled negative sign is selected (i.e., Cr = -Cb) and the decoded Cb residual samples are equal to the minimum value of a 16-bit signed integer (i.e., -2 15 ), this design may cause the decoded residual after JCCR unit 902 to exceed the 16-bit dynamic range. In this case, the calculated Cr residual sample is equal to 2 15 , 2 15 Within the 16-bit dynamic range [-2 15 ,2 15 -1].

[0137] In some embodiments, the clipping operation of the first clipping unit 904 applied before the inverse JCCR unit 902 is symmetric, for example, with a clipping function Clip(-(2 15 -1),2 15 -1,M), the clipping function Clip(-(2 15 -1),2 15 -1,M) is configured to clip the residual output by the inverse transform processing unit 88 to a symmetrical dynamic range [-(2 15 -1),2 15 -1]. In an example, this symmetrical clipping function clips both the luma residual and the chroma residual after the inverse transform processing unit 88. The first dynamic range DR A and the second dynamic range DR B The two are symmetrical, for example, [-(2 15 -1),2 15 In another example, such a symmetrical clipping function is applied to clip the chroma residual to the second dynamic range DR B , and apply an asymmetric clipping function (e.g., Clip(-2 15 ,215 -1,M)) to clip the associated luminance residual to the first dynamic range DR A . The first dynamic range DR A and the second dynamic range DR B They are [-2 15 ,2 15 -1] and [-(2 15 -1),2 15 -1].

[0138] As explained above, in some embodiments, the first clipping unit 904, the second clipping unit 906, and the third clipping unit 1002 are configured to implement the same clipping operation of clipping the corresponding input residuals to the same dynamic range associated with the same bit depth, while the fourth clipping unit 1004 is configured to implement a different clipping operation of clipping the corresponding input residuals to different dynamic ranges associated with different bit depths. In addition, in some embodiments, the first clipping unit 904, the second clipping unit 906, the third clipping unit 1002, and the fourth clipping unit 1004 are unified to implement the same clipping operation of clipping the corresponding input residuals to the same dynamic range associated with the same bit depth. For example, the first clipping unit 904, the second clipping unit 906, the third clipping unit 1002, and the fourth clipping unit 1004 are unified to apply the clipping function Clip(-2 15 ,2 15 -1,M).

[0139] Fig.11A and Fig. 11B 1 is a flowchart of a video decoding method 1100 implemented by an electronic device (e.g., including a video decoder 30) according to some embodiments. The electronic device reconstructs a plurality of residual blocks of a video frame from a video bitstream (1102), and clips the luminance residual and the chrominance residual of the plurality of residual blocks of the video frame to a first dynamic range DR, respectively. A and the second dynamic range DR B (1104), and for each residual block, determining a luma residual for a luma component and a chroma residual for two chroma components of the video frame according to the clipped luma residuals and chroma residuals of the plurality of residual blocks of the video frame (1106). In some embodiments, the first dynamic range DR A and the second dynamic range DR B The clipped luminance residual and chrominance residual of the plurality of residual blocks are stored in a buffer. The clipping operation helps to save the storage space of the buffer and speed up the corresponding video decoding rate. In the example, the first dynamic range DR A and the second dynamic range DR B At least one of them is asymmetric and is defined as [-2 15 ,215 -1], that is, the dynamic range DR A and DR B One or both of the 15 ,2 15 -1]. In another example, the first dynamic range DR A and the second dynamic range DR B At least one of them is symmetric and is defined as [-(2 15 -1),2 15 -1], that is, the dynamic range DR A and DR B One or both of the 15 -1),2 15 -1]. In yet another example, the first dynamic range DR A and the second dynamic range DR B At least one of them is asymmetric and is defined as [-2 B ,2 B -1], that is, the dynamic range DR A and DR B One or both of the B ,2 B -1], and B is the encoding bit depth of the video frame and is obtained through the video bit stream. In another example, the first dynamic range DR A and the second dynamic range DR B At least one of them is symmetric and is defined as [-(2 B -1),2 B -1], that is, the dynamic range DR A and DR B One or both of the B -1),2 B -1], and B is the encoding bit depth obtained by the video bitstream.

[0140] In some embodiments, a plurality of residual blocks (1102) are reconstructed by inversely quantizing a plurality of quantized transform coefficients in a video bitstream into a plurality of transform coefficients for a video frame (1108) and applying an inverse transform to the plurality of transform coefficients to reconstruct a plurality of residual blocks (1110). The plurality of residual blocks are then clipped to a first dynamic range. Furthermore, in some embodiments, each of the plurality of transform coefficients is clipped to a third dynamic range DR before applying the inverse transform. C In (1112).

[0141] In some embodiments, the electronic device determines whether a joint coding and decoding of chroma residual (JCCR) mode is enabled (1114). Based on determining that the JCCR mode is enabled, the chroma residuals for the two chroma components of the video frame are jointly determined (1116) according to a joint coding and decoding scheme for the chroma residual, for example, by the inverse JCCR unit 902. In some embodiments, the chroma residuals for the two chroma components of the video frame are then clipped to a chroma dynamic range DR associated with the chroma bit depth. CR In some embodiments, the video bitstream has a hierarchical structure, and the electronic device obtains a syntax element associated with a first layer of the hierarchical structure, thereby jointly determining the chroma residuals for two chroma components of the video frame under the first layer (1120). Conversely, in some embodiments, based on determining that the syntax element indicates that the JCCR mode is disabled, the electronic device reconstructs the chroma residuals for two chroma components of the video frame based on the clipped chroma residuals of the plurality of residual blocks (1122).

[0142] In some embodiments, an inverse adaptive color transform is also applied to the two chroma components of the video frame to obtain a substitute chroma residual for the two chroma components (1124). The substitute chroma residual for the two chroma components of the video frame is clipped to the fourth dynamic range DR C The video frame is then processed (1126) and scaled for the two chrominance components (1128).

[0143] In some embodiments, each residual block includes one or more corresponding luma residuals and one or more corresponding chroma residuals. A Equal to the second dynamic range DR B And is symmetric with respect to zero. For each residual block, one or more corresponding luminance residuals are clipped to the first dynamic range DR A and clipping one or more corresponding chroma residuals to a second dynamic range DR B In contrast, in some embodiments, each residual block includes one or more corresponding luminance residuals and one or more corresponding chrominance residuals. The first dynamic range DR A Symmetrical with respect to zero, and the second dynamic range DR B Asymmetric with respect to zero. One or more corresponding luminance residuals of each residual block are clipped to the first dynamic range DR A and clipping one or more corresponding chrominance residuals of each residual block to a second dynamic range DR B middle.

[0144] In some embodiments associated with uniform cropping, the first dynamic range is the same as the second dynamic range. Multiple residual blocks are reconstructed by dequantizing multiple quantized transform coefficients in a video bitstream into multiple transform coefficients for a video frame and applying an inverse transform to the multiple transform coefficients to reconstruct multiple residual blocks. Before applying the inverse transform, the electronic device crops each of the multiple transform coefficients to the first dynamic range. After determining the chroma residual, the electronic device crops the chroma residual for two chroma components of the video frame to the first dynamic range, applies an inverse adaptive color transform to the cropped chroma residuals for the two chroma components of the video frame to obtain alternative chroma residuals for the two chroma components, and crops the alternative chroma residuals for the two chroma components of the video frame to a fourth dynamic range. In some embodiments, the fourth dynamic range is the same as the first dynamic range. In some embodiments, the first dynamic range is [-2 15 ,2 15 -1], and the fourth dynamic range is [-2 B ,2 B -1], where B (i.e., the encoding bit depth) is an integer less than 15.

[0145] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes, and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium or a communication medium, wherein the computer-readable storage medium corresponds to a tangible medium (such as a data storage medium), and the communication medium includes any medium that facilitates the transfer of a computer program (e.g., according to a communication protocol) from one place to another. In this way, a computer-readable medium may generally correspond to (1) a non-volatile tangible computer-readable storage medium or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the embodiments described in the present application. A computer program product may include a computer-readable medium.

[0146] The terms used in the description of the embodiments herein are only used for the purpose of describing specific embodiments, and are not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any one and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "including" and / or "including ... " when used in this specification, specify the presence of stated features, elements and / or components, but do not exclude the presence or addition of one or more other features, elements, components and / or their groups.

[0147] It will also be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first electrode may be referred to as the second electrode, and similarly, the second electrode may be referred to as the first electrode without departing from the scope of the embodiment. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0148] The description of the present application has been presented for the purpose of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. With the benefit of the teachings presented in the foregoing description and the associated drawings, many modifications, variations and alternative embodiments will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to best explain the principles of the invention, the practical application, and to enable other persons skilled in the art to understand the invention for various embodiments, and to best utilize the basic principles and various embodiments with various modifications suitable for the intended specific use. Therefore, it will be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding video data, comprising: reconstructing a plurality of residual blocks of a video frame from a video bitstream; Clipping the luminance residual and the chrominance residual of the plurality of residual blocks of the video frame into a first dynamic range and a second dynamic range respectively; and For each residual block, determining a luma residual for a luma component and a chroma residual for two chroma components according to the clipped luma residuals and chroma residuals of the plurality of residual blocks, Wherein at least one of the first dynamic range and the second dynamic range is asymmetric with respect to zero and is defined as [-2 15 , 2 15 -1], or at least one of the first dynamic range and the second dynamic range is symmetric with respect to zero and is defined as [-(2 15 -1), 2 15 -1], or At least one of the first dynamic range and the second dynamic range is asymmetric with respect to zero and is defined as [-2 B , 2 B -1], or at least one of the first dynamic range and the second dynamic range is symmetric with respect to zero and is defined as [-(2 B -1), 2 B -1], where B is the encoding bit depth of the video frame.

2. The method of claim 1, wherein: The method further comprises: The encoding bit depth is obtained through the video bit stream.

3. The method according to any one of claims 1 to 2, wherein: Each residual block includes one or more corresponding luma residuals and one or more corresponding chroma residuals; The first dynamic range is equal to the second dynamic range and is symmetrical with respect to zero; and The step of trimming the luminance residual and the chrominance residual of the plurality of residual blocks of the video frame further comprises: For each residual block, the one or more corresponding luma residuals are clipped into the first dynamic range and the one or more corresponding chroma residuals are clipped into the second dynamic range.

4. The method according to any one of claims 1 to 2, wherein: Each residual block includes one or more corresponding luma residuals and one or more corresponding chroma residuals; The first dynamic range is asymmetric with respect to zero, and the second dynamic range is symmetric with respect to zero; and The step of trimming the luminance residual and the chrominance residual of the plurality of residual blocks of the video frame further comprises: The one or more corresponding luma residuals of each residual block are clipped into the first dynamic range, and the one or more corresponding chroma residuals of each residual block are clipped into the second dynamic range.

5. The method of any one of claims 1-2, wherein the second dynamic range is smaller than the first dynamic range or is included in the first dynamic range.

6. The method of any one of claims 1-2, wherein reconstructing the plurality of residual blocks from a video bitstream comprises: Dequantizing a plurality of quantized transform coefficients in the video bitstream into a plurality of transform coefficients for the video frame; and An inverse transform is applied to the plurality of transform coefficients to reconstruct the plurality of residual blocks.

7. The method of claim 6, further comprising: Prior to applying the inverse transform, each transform coefficient of the plurality of transform coefficients is clipped into a third dynamic range.

8. The method according to any one of claims 1 to 2, further comprising: The clipped luma residuals and chroma residuals of the plurality of residual blocks are stored in a buffer according to the first dynamic range and the second dynamic range, respectively.

9. The method according to any one of claims 1 to 2, further comprising: applying an inverse adaptive color transform to the two chroma components of the video frame to obtain substitute chroma residuals for the two chroma components; clipping a substitute chroma residual of the two chroma components of the video frame into a fourth dynamic range; and A clipped substitute chroma residual of the two chroma components of the video frame is scaled.

10. The method according to any one of claims 1 to 2, further comprising: It is determined whether a joint coding and decoding (JCCR) mode of chroma residual is enabled, wherein according to determining that the JCCR mode is enabled, the chroma residuals for the two chroma components of the video frame are jointly determined according to a joint coding and decoding scheme of the chroma residual.

11. The method of claim 10, further comprising: For each residual block, the chroma residuals of the two chroma components of the video frame are clipped into a chroma dynamic range.

12. The method of claim 10, wherein the video bitstream has a layered structure, the method further comprising: obtaining syntax elements associated with a first level of the hierarchical structure; The chroma residuals of the two chroma components of the video frame are jointly determined under the first layer.

13. The method of claim 12, determining a luma residual for the luma component and a chroma residual for the two chroma components further comprising: In response to determining that the syntax element indicates that the JCCR mode is disabled, chroma residuals for the two chroma components of the video frame are reconstructed respectively based on the clipped chroma residuals of the plurality of residual blocks.

14. The method of claim 1, wherein: The first dynamic range and the second dynamic range are the same; Reconstructing the plurality of residual blocks from the video bitstream comprises: Dequantizing a plurality of quantized transform coefficients in the video bitstream into a plurality of transform coefficients for the video frame; and applying an inverse transform to the plurality of transform coefficients to reconstruct the plurality of residual blocks; The method further comprises: prior to applying an inverse transform, clipping each transform coefficient of the plurality of transform coefficients to the first dynamic range; clipping chroma residuals for the two chroma components of the video frame into the first dynamic range; applying an inverse adaptive color transform to the clipped chroma residuals of the two chroma components of the video frame to obtain substitute chroma residuals for the two chroma components; and Substitute chroma residuals for the two chroma components of the video frame are clipped into a fourth dynamic range. The method of claim 14 , wherein the fourth dynamic range is the same as the first dynamic range.

16. The method of claim 14, wherein the first dynamic range is [-2 15 , 2 15 -1], the fourth dynamic range is [-2 K , 2 K -1], where K is an integer less than 15.

17. An electronic device comprising: one or more processors; as well as A memory having instructions stored thereon, which, when executed by the one or more processors, cause the processors to perform the method according to any one of claims 1 to 16.

18. A non-transitory computer-readable medium having stored thereon instructions, which when executed by one or more processors cause the processors to perform the method of any one of claims 1-16.

Images