Method and device for video encoding and decoding in 4:4:4 chroma format

Abstract

An electronic device performs a method for decoding video data, the method comprising: obtaining a luma-to-chroma quantization parameter syntax element set from a bitstream; generating a luma-to-chroma quantization parameter QP mapping table based on the luma-to-chroma quantization parameter syntax element set, wherein the luma-to-chroma QP mapping table converts a quantization parameter (QP) of a chroma component into a quantization parameter (QP) of a chroma component. C ) is set equal to the quantization parameter (QP) of the corresponding luminance component L );Get the QP of the brightness component of the coding unit from the data bit stream L ; The QP of the chroma component of the coding unit is mapped according to the luminance to chroma QP mapping table C Determined as the QP of the luma component of the coding unit L Same; using QP L and QP C The luma component and chroma components of a coding unit are decoded from the data bitstream.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 954,326, filed on December 27, 2019, entitled “METHODS AND APPARATUS ON 4:4:4 VIDEO CODING,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates generally to video data encoding and compression, and more particularly to methods and apparatus for improving and simplifying luma-to-chroma quantization parameter (QP) mapping tables. Background Art

[0004] Various electronic devices (such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression standards defined by MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC) and General Video Codec (VVC) standards. Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data. For block-based video coding, a video frame is divided into one or more slices, each with multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or may be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called leaf-CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be encoded and decoded in intra, inter, or intra block copy (IBC) mode. Video blocks in intra-coded (I) slices of a video frame are encoded using spatial predictions about reference samples in neighboring blocks within the same video frame. Video blocks in inter-coded (P (forward predicted picture) or B (bidirectional predicted picture)) slices of a video frame can use spatial predictions about reference samples in neighboring blocks within the same video frame or temporal predictions about reference samples in other previous reference video frames and / or future reference video frames.

[0005] A prediction block for the current video block to be coded is derived based on spatial prediction or temporal prediction of previously coded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished by a block matching algorithm. The residual data representing the pixel differences between the current block to be coded and the prediction block is called a residual block or prediction error. Inter-coded blocks are encoded based on the residual block and a motion vector pointing to a reference block in a reference frame that forms the prediction block. The process of determining the motion vector is generally referred to as motion estimation. Intra-coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain (e.g., the frequency domain) to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, which are then entropy encoded into a video bitstream to achieve even greater compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., a flash memory) for access by another electronic device with digital video capabilities or sent directly to the electronic device by wire or wirelessly. The electronic device then performs video decompression (which is the reverse process of the video compression described above), for example, by parsing the encoded video bitstream to obtain syntax elements from the bitstream, and reconstructing digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and the electronic device presents the reconstructed digital video data on a display of the electronic device.

[0007] As digital video quality changes from HD to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. How to encode / decode video data more efficiently while maintaining the image quality of the decoded video data is a long-standing challenge.

[0008] Certain video content, such as screen content video, is encoded in a 4:4:4 chroma format in which all three components (luminance and two chroma components) have the same resolution. Although the 4:4:4 chroma format includes more redundancy than the 4:2:0 chroma format and the 4:2:2 chroma format (which is detrimental to achieving good compression efficiency), the 4:4:4 chroma format is still the preferred encoding format for many applications that require high fidelity to preserve color information (e.g., sharp edges) in the decoded video. Given the presence of redundancy in 4:4:4 chroma format video, there is evidence that significant codec improvements can be achieved by weakening the correlation between the three color components (e.g., Y, Cb, and Cr in the YCbCr domain; or G, B, and R in the RGB domain) of the 4:4:4 video. Due to these correlations, during the development of the HEVC Screen Content Codec (SCC) extension, the Adaptive Color Space Transform (ACT) tool was used to weaken the correlation between the three color components. Summary of the invention

[0009] The present application describes embodiments related to video data encoding and decoding, and more particularly, describes embodiments related to systems and methods for constructing luma-to-chroma quantization parameter (QP) mapping tables.

[0010] For a video signal originally captured in a 4:4:4 color format, if high fidelity is expected for the decoded video signal and there is a large amount of information redundancy in the original color space (e.g., RGB video), it is preferred to encode the video in the original space. Although some inter-component coding tools in the current VVC standard (e.g., cross-component linear model prediction (CCLM)) can improve the efficiency of 4:4:4 video coding, the redundancy between the three components is not completely removed. This is because only the Y / G components are used to predict the Cb / B and Cr / R components without considering the correlation between the Cb / B and Cr / R components. Accordingly, further decorrelation of the three color components can improve the coding and decoding performance of the 4:4:4 video coding.

[0011] In the current VVC standard, the design of existing inter-frame and intra-frame tools is mainly focused on videos captured in 4:2:0 chroma format. Therefore, in order to achieve a better complexity / performance trade-off, most of these codec tools are only applicable to luma components, while disabling chroma components (e.g., position-dependent intra prediction combination (PDPC), multiple reference lines (MRL), and sub-segmentation prediction (ISP)), or using different operations for luma components and chroma components (e.g., interpolation filters applied to motion compensated prediction). However, compared with 4:2:0 video, the video signal in 4:4:4 chroma format represents very different characteristics. For example, the Cb / B components and Cr / R components of 4:4:4 YCbCr video and RGB video represent richer color information than the chroma components in 4:2:0 video and have more high-frequency information (e.g., edges and textures). In this consideration, the same design of some existing codec tools in VVC may always be optimal for both 4:2:0 video and 4:4:4 video.

[0012] According to a first aspect of the present application, a method for decoding video data includes: obtaining a luminance-to-chrominance quantization parameter syntax element set from a data bit stream; generating a luminance-to-chrominance quantization parameter QP mapping table based on the luminance-to-chrominance quantization parameter syntax element set, wherein the luminance-to-chrominance QP mapping table converts the quantization parameter QP of the chrominance component into a quantization parameter QP of the chrominance component. C Set equal to the quantization parameter QP of the corresponding luminance component L ; Get the QP of the luminance component of the coding unit from the data bitstream L ; The QP of the chroma component of the coding unit is mapped according to the luminance to chroma QP mapping table C Determined as the QP of the luma component of the coding unit L Same; using QP L and QP C The luma component and chroma components of a coding unit are decoded from the data bitstream.

[0013] According to a second aspect of the present application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When the programs are executed by the one or more processing units, the electronic device executes the method for decoding video data as described above.

[0014] According to a third aspect of the present application, a non-volatile computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. When the programs are executed by the one or more processing units, the electronic device executes the method for decoding video data as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of the specification, illustrate the described embodiments and together with the description serve to explain the basic principles. Like reference numerals refer to corresponding parts.

[0016] Figure 1 is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0017] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0018] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0019] FIG. 4A to FIG. 4E is a block diagram illustrating how a frame may be recursively partitioned into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0020] Figure 5A and Figure 5B is a block diagram illustrating an example of applying an adaptive color space conversion (ACT) technique to convert a residual between an RGB color space and a YCgCo color space according to some embodiments of the present disclosure.

[0021] Figure 6 is a block diagram of a technique for applying luma mapping and chroma scaling (LMCS) in an exemplary video data decoding process according to some embodiments of the present disclosure.

[0022] Figure 7 is a block diagram illustrating an exemplary video decoding process in which a video decoder implements an inverse adaptive color space transform (ACT) technique according to some embodiments of the present disclosure.

[0023] Fig. 8A and Figure 8B is a block diagram illustrating an exemplary video decoding process in which a video decoder implements techniques for inverse adaptive color space transform (ACT) and luma mapping and chroma scaling (LMCS) according to some embodiments of the present disclosure.

[0024] Fig. 9 is a block diagram illustrating exemplary decoding logic between performing adaptive color space transform (ACT) and performing block differential pulse code modulation (BDPCM) according to some embodiments of the present disclosure.

[0025] Fig.10is a decoding flow chart for applying different quantization parameter (QP) offsets to different components when luma intra bit depth and chroma intra bit depth are different according to some embodiments of the present disclosure.

[0026] Fig.11 is a flowchart 1100 illustrating an exemplary process for a video codec to encode / decode video data by using a luma-to-chroma quantization parameter (QP) mapping table based on a luma-to-chroma quantization parameter syntax element set according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0028] In some embodiments, these methods are provided to improve the encoding and decoding efficiency of the VVC standard for 4:4:4 video. In general, the main features of the technology in the present disclosure are summarized as follows.

[0029] In some embodiments, these methods are implemented to improve existing ACT designs that enable adaptive color space conversion in the residual domain. In particular, special considerations are made to handle the interaction of ACT with some existing codec tools in VVC.

[0030] In some embodiments, these methods are implemented to improve the efficiency of some existing inter-frame and intra-frame codec tools for 4:4:4 video in the VVC standard, including: 1) enabling 8-tap interpolation filters for chroma components; 2) enabling PDPC for intra-frame prediction of chroma components; 3) enabling MRL for intra-frame prediction of chroma components; 4) enabling ISP splitting for chroma components.

[0031] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0032] In some embodiments, the target device 14 may receive the encoded video data to be decoded via the link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14. In one example, the link 16 may include a communication medium that enables the source device 12 to send the encoded video data directly to the target device 14 in real time. The encoded video data may be modulated according to a communication standard (such as a wireless communication protocol) and sent to the target device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that may be conducive to facilitating communication from the source device 12 to the target device 14.

[0033] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. Subsequently, the encoded video data in the storage device 32 may be accessed by the target device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device 32 may correspond to a file server or another intermediate storage device that can hold the encoded video data generated by the source device 12. The target device 14 may access the stored video data from the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include a web server (e.g., for a website), an FTP server, a network attached storage (NAS) device, or a local disk drive. The target device 14 may access the encoded video data through any standard data connection suitable for accessing encoded video data stored on a file server, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both wireless channels and wired connections. The transmission of the encoded video data from the storage device 32 may be a streaming transmission, a download transmission, or a combination of both streaming transmission and download transmission.

[0034] like Figure 1 As shown in , source device 12 includes video source 18, video encoder 20 and output interface 22. Video source 18 may include sources such as or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or a video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding, and may be applied to wireless and / or wired applications.

[0035] The captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video data may be sent directly to the target device 14 via the output interface 22 of the source device 12. The encoded video data may also (or alternatively) be stored on a storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or a transmitter.

[0036] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data communicated via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included in the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0037] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device or an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0038] The video encoder 20 and the video decoder 30 may operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, MPEG-4, Part 10, Advanced Video Codec (AVC)), or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 may be configured to encode video data according to any of these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 may be configured to decode video data according to any of these current standards or future standards.

[0039] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-volatile computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device.

[0040] Figure 2is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described herein. The video encoder 20 may perform intra-frame prediction coding and inter-frame prediction coding for video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0041] like Figure 2 As shown in FIG. 1 , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A deblocking filter (not shown) may be located between the adder 62 and the DPB 64 to filter the block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, a loop filter (not shown) may also be used to filter the output of the adder 62. The video encoder 20 may take the form of a fixed or programmable hardware unit, or may be dispersed in one or more of the illustrated fixed or programmable hardware units.

[0042] Video data memory 40 may store video data to be encoded by components of video encoder 20. The video data in video data memory 40 may be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data for use by video encoder 20 (e.g., in intra-frame or inter-frame prediction codec mode) when encoding video data. Video data memory 40 and DPB 64 may be formed by any of a variety of memory devices. In various examples, video data memory 40 may be on-chip with other components of video encoder 20, or off-chip relative to those components.

[0043] like Figure 2As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into slices, partitions (tiles), or other larger coding units (CUs) according to a predefined split partitioning structure associated with the video data (such as a quadtree partitioning structure). The video frame may be divided into a plurality of video blocks (or a set of video blocks referred to as partitions). The prediction processing unit 41 may select one of a plurality of feasible prediction codec modes for the current video block based on error results (e.g., codec rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction codec modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (such as motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to the entropy coding unit 56.

[0044] To select an appropriate intra-prediction codec mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction codec of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction codec of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate codec mode for each block of video data.

[0045] In some embodiments, motion estimation unit 42 determines the inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion for a video block. For example, a motion vector may indicate the displacement of a PU of a video block within a current video frame or picture relative to a prediction block (or other coding unit) within a reference frame associated with a current block (or other coding unit) being encoded and decoded within the current frame. The predetermined pattern may designate video frames in a sequence as P frames or B frames. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC encoding and decoding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine the block vector.

[0046] A prediction block is a block of a reference frame that is considered to closely match a PU of a video block to be encoded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some embodiments, video encoder 20 may calculate values ​​for sub-integer pixel positions of reference frames stored in DPB 64. For example, video encoder 20 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of a reference frame. Thus, motion estimation unit 42 may perform motion searches relative to full pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0047] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-prediction codec frame by comparing the position of the PU with the position of a prediction block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy encoding unit 56.

[0048] The motion compensation performed by the motion compensation unit 44 may involve extracting or generating a prediction block based on the motion vector determined by the motion estimation unit 42. After receiving the motion vector for the PU of the current video block, the motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from the DPB 64, and forward the prediction block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit 44 from the pixel values ​​of the current video block being encoded and decoded. The pixel difference values ​​forming the residual video block may include a luma difference component or a chroma difference component or both. The motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining a motion vector for describing the prediction block, any flag indicating a prediction mode, or any other syntax information described herein. It should be noted that the motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0049] In some embodiments, the intra BC unit 48 may generate vectors and extract prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra BC unit 48 may determine the intra prediction mode to be used to encode the current block. In some examples, the intra BC unit 48 may encode the current block using various intra prediction modes, for example, during separate encoding passes, and test their performance through rate-distortion analysis. Next, the intra BC unit 48 may select a suitable intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​for the various tested intra prediction modes using rate-distortion analysis, and select the intra prediction mode with the best rate-distortion characteristics among the tested modes as the suitable intra prediction mode to use. The rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was encoded to generate the coded block, as well as the bit rate (i.e., the number of bits) used to generate the coded block. Intra BC unit 48 may calculate ratios from the distortions and rates for the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0050] In other examples, intra BC unit 48 may use, in whole or in part, motion estimation unit 42 and motion compensation unit 44 to perform such functions for intra BC prediction in accordance with embodiments described herein. In either case, for intra block copying, the prediction block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics, and identification of the prediction block may include calculating values ​​for sub-integer pixel positions.

[0051] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, video encoder 20 can form pixel difference values ​​by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being decoded, thereby forming a residual video block. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0052] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 or the intra-frame block copy prediction performed by the intra BC unit 48 as described above, the intra-frame prediction processing unit 46 may perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 may determine an intra-frame prediction mode for encoding the current block. To this end, the intra-frame prediction processing unit 46 may use various intra-frame prediction modes to encode the current block, for example, during separate encoding passes, and the intra-frame prediction processing unit 46 (or in some examples, the mode selection unit) may select a suitable intra-frame prediction mode from the tested intra-frame prediction modes to use. The intra-frame prediction processing unit 46 may provide information indicating the intra-frame prediction mode selected for the block to the entropy encoding unit 56. The entropy encoding unit 56 may encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0053] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0054] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of the matrix including the quantized transform coefficients. Optionally, entropy encoding unit 56 may perform the scan.

[0055] After quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to video decoder 30, or archived in storage device 32 for later sending to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0056] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0057] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0058] Figure 3 30 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above combined Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0059] In some examples, units of the video decoder 30 may be tasked to perform embodiments of the present application. In addition, in some examples, embodiments of the present disclosure may be dispersed in one or more of the multiple units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30 (such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81 (such as the motion compensation unit 82).

[0060] The video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 may be obtained, for example, from the storage device 32, from a local video source (such as a camera), via a wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. A decoded picture buffer (DPB) 92 of the video decoder 30 stores reference video data for use by the video decoder 30 when decoding the video data (e.g., in an intra-frame or inter-frame prediction codec mode). The video data memory 79 and the DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data memory 79 and the DPB 92 are shown in FIG. Figure 3 9 as two different components of video decoder 30. However, it will be apparent to those skilled in the art that video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, video data memory 79 may be on-chip with other components of video decoder 30, or off-chip relative to those components.

[0061] During the decoding process, the video decoder 30 receives an encoded video bitstream representing video blocks of an encoded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0062] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, intra-prediction unit 84 of prediction processing unit 81 may generate prediction data for a video block of a current video frame based on a signaled intra-prediction mode and reference data from a previously decoded block of the current frame.

[0063] When the video frame is encoded as an inter-frame prediction coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vectors and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks may be generated from a reference frame within one of the reference frame lists. The video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference frames stored in the DPB 92.

[0064] In some examples, when a video block is encoded or decoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0065] The motion compensation unit 82 and / or the intra BC unit 85 determine the prediction information for the video block of the current video frame by parsing the motion vector and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) used to encode and decode the video block of the video frame, the inter prediction frame type (e.g., B or P), the construction information for one or more of the reference frame lists for the frame, the motion vector for each inter prediction coded video block of the frame, the inter prediction state for each inter prediction coded video block of the frame, and other information for decoding the video block in the current video frame.

[0066] Similarly, the intra BC unit 85 may use some of the received syntax elements, such as flags, to determine whether the current video block is predicted using intra BC mode, construction information of which video blocks of the frame are within the reconstruction region and should be stored in the DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding the video block in the current video frame.

[0067] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 from the received syntax elements and use these interpolation filters to generate the prediction blocks.

[0068] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0069] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 to further process the decoded video block. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0070] In a typical video encoding and decoding process, a video sequence usually includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, a frame may be monochrome and therefore include only a two-dimensional array of luma samples.

[0071] like Figure 4A As shown in , the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in a raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set so that all CTUs in a video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. It should be noted, however, that the present application is not necessarily limited to a particular size. As Figure 4BAs shown in , each CTU may include one coding tree block (CTB) of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements for encoding and decoding samples of the coding tree blocks. The syntax elements describe the properties of different types of units of coding pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements for encoding and decoding samples of the coding tree block. The coding tree block may be an N×N sample block.

[0072] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination of both, on the coding tree blocks of the CTU and divide the CTU into smaller coding units (CUs). Figure 4C As depicted in , the 64×64 CTU 400 is first divided into four smaller CUs, each CU having a block size of 32×32. Among the four smaller CUs, CU 410 and CU 420 are respectively divided into four CUs with a block size of 16×16. The two 16×16 CUs 430 and CU 440 are further divided into four CUs with a block size of 8×8. Figure 4D Depicted is a diagram showing Figure 4C The quadtree data partitioning structure is the final result of the partitioning process of the CTU 400 depicted in FIG. 4 , and each leaf node of the quadtree corresponds to a CU of each size ranging from 32×32 to 8×8. Figure 4B Each CU may include a coding block (CB) of luma samples and two corresponding coding blocks of chroma samples of the same size frame, and syntax elements for encoding and decoding the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and a syntax partitioning structure for encoding and decoding the samples of the coding block. It should be noted that Figure 4C and Figure 4D The quadtree partitioning depicted in FIG is for illustrative purposes only, and a CTU may be split into CUs based on quadtree / ternary tree / binary tree partitioning to adapt to varying local characteristics. In a multi-type tree partitioning structure, a CTU is partitioned by a quadtree partitioning structure, and each quadtree leaf CU may be further partitioned by a binary and ternary tree partitioning structure. Figure 4E As shown, there are five types of segmentation, namely, quadruple segmentation, horizontal binary segmentation, vertical binary segmentation, horizontal ternary segmentation, and vertical ternary segmentation.

[0073] In some embodiments, the video encoder 20 may further partition the coding block of the CU into one or more M×N prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples to which the same prediction (inter or intra) is applied. The prediction unit (PU) of the CU may include a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax elements for predicting the prediction blocks. In a monochrome picture or a picture with three separate color planes, the PU may include a single prediction block and a syntax partitioning structure for predicting the prediction block. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma prediction block, the Cb prediction block, and the Cr prediction block of each PU of the CU.

[0074] Video encoder 20 may use intra prediction or inter prediction to generate a prediction block for a PU. If video encoder 20 uses intra prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0075] After the video encoder 20 generates the predicted luma block, the predicted Cb block, and the predicted Cr block for one or more PUs of a CU, the video encoder 20 may generate a luma residual block for the CU by subtracting the predicted luma block of the CU from the original luma coding block of the CU, so that each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, so that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0076] In addition, if Figure 4CAs shown in , the video encoder 20 may decompose the luma residual block, Cb residual block, and Cr residual block of a CU into one or more luma transform blocks, Cb transform blocks, and Cr transform blocks using quadtree partitioning. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A transform unit (TU) of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming the transform block samples. Therefore, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax partitioning structure for transforming the samples of the transform block.

[0077] The video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficient may be a scalar. The video encoder 20 may apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0078] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic coding and decoding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream including a sequence of bits that form a representation of an encoded frame and associated data, and the bitstream is stored in storage device 32 or sent to target device 14.

[0079] After receiving the bitstream generated by the video encoder 20, the video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 may reconstruct a frame of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 may perform an inverse transform on a coefficient block associated with a TU of the current CU to reconstruct a residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coding block of the current CU by adding samples of the prediction block for the PU of the current CU to corresponding samples of the transform block of the TU of the current CU. After reconstructing the coding block for each CU of the frame, the video decoder 30 may reconstruct the frame.

[0080] As described above, video codecs mainly use two modes, namely, intra-frame prediction (or intra-frame prediction) and inter-frame prediction (or inter-frame prediction) to achieve video compression. Palette-based codec is another codec scheme that has been adopted by many video codec standards. In palette-based codecs (which may be particularly suitable for screen-generated content codecs), a video codec (e.g., video encoder 20 or video decoder 30) forms a palette table representing the colors of video data for a given block. The palette table includes the most important (e.g., frequently used) pixel values ​​in a given block. Pixel values ​​that are not frequently represented in the video data of a given block are not included in the palette table or are included in the palette table as escape colors.

[0081] Each entry in the palette table includes an index to the corresponding pixel value in the palette table. The palette index for a sample in a block can be encoded to indicate which entry from the palette table will be used to predict or reconstruct which sample. This palette mode begins with a process of generating a palette predictor for the first block of a picture, slice, tile, or other such grouping of video blocks. As will be explained below, the palette predictor for subsequent video blocks is typically generated by updating the previously used palette predictor. For purposes of illustration, it is assumed that the palette predictor is defined at the picture level. In other words, a picture may include multiple coding blocks, each with its own palette table, but there is one palette predictor for the entire picture.

[0082] In order to reduce the bits required to signal palette entries in a video bitstream, a video decoder may utilize a palette predictor to determine new palette entries in a palette table for reconstructing a video block. For example, the palette predictor may include palette entries from a previously used palette table, or even be initialized with a most recently used palette table by including all entries of a most recently used palette table. In some embodiments, the palette predictor may include fewer entries than all entries from a most recently used palette table, and then incorporate some entries from other previously used palette tables. The palette predictor may have the same size as a palette table used to encode and decode a different block, or may be larger or smaller than a palette table used to encode and decode a different block. In one example, the palette predictor is implemented as a first-in, first-out (FIFO) table that includes 64 palette entries.

[0083] To generate a palette table for a block of video data from the palette predictor, a video decoder may receive a one-bit flag for each entry of the palette predictor from an encoded video bitstream. The one-bit flag may have a first value (e.g., binary one) indicating that the associated entry of the palette predictor is to be included in the palette table or a second value (e.g., binary zero) indicating that the associated entry of the palette predictor is not to be included in the palette table. If the size of the palette predictor is larger than the palette table for the block of video data, the video decoder may stop receiving more flags once a maximum size for the palette table is reached.

[0084] In some embodiments, some entries in the palette table may be directly signaled in the encoded video bitstream without being determined using a palette predictor. For such entries, the video decoder may receive three separate m-bit values ​​from the encoded video bitstream indicating pixel values ​​for the luma component and two chroma components associated with the entry, where m represents the bit depth of the video data. Compared to the multiple m-bit values ​​required for palette entries that are directly signaled, those palette entries derived from the palette predictor require only a one-bit flag. Therefore, signaling some or all palette entries using a palette predictor can significantly reduce the number of bits required to signal new palette table entries, thereby improving the overall codec efficiency of palette mode codecs.

[0085] In many cases, the palette predictor for a block is determined based on the palette table used to encode one or more previously encoded blocks. However, when encoding and decoding the first coding tree unit in a picture, slice, or tile, the palette table of the previously encoded block may not be available. Therefore, the entries of the previously used palette table cannot be used to generate the palette predictor. In this case, a sequence of palette predictor initialization values ​​can be signaled in a sequence parameter set (SPS) and / or a picture parameter set (PPS), and the palette predictor initialization value is a value used to generate the palette predictor when the previously used palette table is not available. SPS generally refers to a syntax partitioning structure of syntax elements applied to a series of consecutive coded video pictures called a coded video sequence (CVS), where the CVS is determined by the content of the syntax elements found in the PPS referred to by the syntax elements found in each slice segment header. PPS generally refers to a syntax partitioning structure of syntax elements applied to one or more individual pictures within a CVS, where the CVS is determined by the syntax elements found in each slice segment header. Therefore, the SPS is generally considered to be a higher-level syntax partitioning structure than the PPS, which means that the syntax elements included in the SPS generally change less frequently and apply to a larger portion of the video data than the syntax elements included in the PPS.

[0086] FIG. 5A to FIG. 5B is a block diagram illustrating an example of applying an adaptive color space conversion (ACT) technique to convert a residual between an RGB color space and a YCgCo color space according to some embodiments of the present disclosure.

[0087] In the HEVC screen content coding extension, ACT is applied to adaptively transform the residual from one color space (e.g., RGB) to another color space (e.g., YCgCo) so that the correlation (e.g., redundancy) between the three color components (e.g., R, G, and B) is significantly reduced in the YCgCo color space. In addition, in the existing ACT design, the adaptation of different color spaces is performed at the transform unit (TU) level by signaling a flag tu_act_enabled_flag for each TU. When the flag tu_act_enabled_flag is equal to 1, it indicates that the residual of the current TU is encoded in the YCgCo space; otherwise (i.e., the flag is equal to 0), it indicates that the residual of the current TU is encoded in the original color space (i.e., there is no color space conversion). In addition, different color space transformation formulas are applied depending on whether the current TU is encoded in lossless mode or in lossy mode. Specifically, in Figure 5A The forward color space transformation formula and the reverse color space transformation formula between RGB color space and YCgCo color space for lossy mode are defined in .

[0088] For lossless mode, a reversible version of the RGB-YCgCo transform (also called YCgCo-LS) is used. The reversible version of the RGB-YCgCo transform is based on Figure 5B The lifting operation depicted in and the related description are implemented.

[0089] like Figure 5A As shown in , the forward color transform matrix and the inverse color transform matrix used in the lossy mode are not normalized. Therefore, after applying the color transform, the amplitude of the YCgCo signal is smaller than the amplitude of the original signal. In order to compensate for the amplitude reduction caused by the forward color transform, an adjusted quantization parameter is applied to the residual in the YCgCo domain. Specifically, when applying the color space transform, the QP value QP used to quantize the YCgCo domain residual is Y , QP Cg and QP Co They are set to QP-5, QP-5, and QP-3 respectively, where QP is the quantization parameter used in the original color space.

[0090] Figure 6 is a block diagram of a technique for applying luma mapping and chroma scaling (LMCS) in an exemplary video data decoding process according to some embodiments of the present disclosure.

[0091] In VVC, LMCS is used as a new codec tool applied before loop filters (e.g., deblocking filter, SAO, and ALF). Generally, LMCS has two main modules: 1) loop mapping of luma components based on an adaptive piecewise linear model; 2) luma-dependent chroma residual scaling. Figure 6 The improved decoding process using LMCS is shown. Figure 6 In the LMCS, the decoding modules performed in the mapped domain include an entropy decoding module, an inverse quantization module, an inverse transform module, a luma intra prediction module, and a luma sample reconstruction module (i.e., luma prediction samples are added to luma residual samples). The decoding modules performed in the original (i.e., non-mapped) domain include a motion compensation prediction module, a chroma intra prediction module, a chroma sample reconstruction module (i.e., chroma prediction samples are added to chroma residual samples), and all loop filter modules (such as a deblocking module, an SAO module, and an ALF module). The new operation modules introduced by LMCS include a luma sample forward mapping module 610, a luma sample reverse mapping module 620, and a chroma residual scaling module 630.

[0092] The loop mapping of LMCS can adjust the dynamic range of the input signal to improve the efficiency of encoding and decoding. The loop mapping of luma samples in existing LMCS designs is based on two mapping functions: a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The forward mapping function is signaled from the encoder to the decoder using a piecewise linear model with sixteen equal-sized segments. The inverse mapping function can be directly derived from the forward mapping function and therefore does not need to be signaled.

[0093] The parameters of the luma mapping model are signaled at the slice level. First, a presence flag is signaled to indicate whether a luma mapping model will be signaled for the current slice. If a luma mapping model exists under the current slice, the corresponding piecewise linear model parameters are further signaled. In addition, at the slice level, another LMCS control flag is signaled to enable / disable LMCS for the slice.

[0094] The chroma residual scaling module 630 is designed to compensate for the mutual influence of quantization precision between the luma signal and its corresponding chroma signal when loop mapping is applied to the luma signal. Whether chroma residual scaling is enabled or disabled for the current slice is also signaled in the slice header. If luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is applied. When luma mapping is not used, luma-dependent chroma residual scaling is always disabled and no additional flag is required. In addition, chroma residual scaling is always disabled for CUs containing less than or equal to four chroma samples.

[0095] Figure 7 is a block diagram illustrating an exemplary video decoding process in which a video decoder implements an inverse adaptive color space transform (ACT) technique according to some embodiments of the present disclosure.

[0096] Similar to the ACT design in HEVC SCC, ACT in VVC converts the intra / inter prediction residual of a CU in a 4:4:4 chroma format from the original color space (e.g., RGB color space) to the YCgCo color space. As a result, the redundancy between the three color components can be reduced to obtain better coding efficiency. Figure 7 A decoding flow chart depicting how to apply inverse ACT in the VVC framework by adding an inverse ACT module 710. When processing a CU encoded with ACT enabled, entropy decoding, inverse quantization, and an inverse DCT / DST-based transform are first applied to the CU. Figure 7As depicted in , the inverse ACT is called to convert the decoded residual from the YCgCo color space to the original color space (e.g., RGB and YCbCr). In addition, because the ACT in the lossy mode is not normalized, the QP adjustment (-5, -5, -3) is applied to the Y component, the Cg component, and the Co component to compensate for the changed magnitude of the transformed residual.

[0097] In some embodiments, the ACT method reuses the same ACT kernel transform of HEVC to perform color conversion between different color spaces. Specifically, two different versions of the color transform are applied depending on whether the current CU is coded in a lossy or lossless manner. The forward color transform and the reverse color transform for the lossy case use the following example: Figure 5A The irreversible YCgCo transformation matrix is ​​shown in Figure 2. For the lossless case, the following is applied: Figure 5B The reversible color transform YCgCo-LS is shown. In addition, unlike the existing ACT design, the following changes are introduced into the ACT scheme to handle its interaction with other codec tools in the VVC standard.

[0098] For example, because the residual of one CU in HEVC can be split into multiple TUs, an ACT control flag is signaled separately for each TU to indicate whether color space conversion needs to be applied. Figure 4E As described, a quadtree nested with binary and ternary partitioning structures is applied in VVC to replace the concept of multiple partitioning types, thereby removing the distinction between individual CUs, PUs, and TUs in HEVC. This means that in most cases, a CU leaf node is also used as a unit of prediction and transform processing without further partitioning, unless the maximum supported transform size is smaller than the width or height of a component of the CU. Based on such a partitioning structure, CU-level ACT is adaptively enabled and disabled. Specifically, a flag cu_act_enabled_flag is signaled for each CU to select between the original color space and the YCgCo color space for encoding and decoding the residual of the CU. If the flag is equal to 1, it indicates that the residuals of all TUs within the CU are encoded and decoded in the YCgCo color space. Otherwise, if the flag cu_act_enabled_flag is equal to 0, all residuals of the CU are encoded and decoded in the original color space.

[0099] In some embodiments, there are different scenarios for disabling ACT. When ACT is enabled for a CU, the residuals of all three components need to be accessed for color space conversion. However, the VVC design cannot guarantee that each CU always contains information of the three components. According to an embodiment of the present disclosure, in those cases where the CU does not contain information of all three components, ACT should be forcibly disabled.

[0100] First, in some embodiments, when a separate tree partitioning structure is applied, the luma samples and chroma samples within a CTU are partitioned into CUs based on different partitioning structures. Therefore, the CU in the luma partitioning tree contains only the coding information of the luma component, and the CU in the chroma partitioning tree contains only the coding information of the two chroma components. According to the current VVC, the switching between the single tree partitioning structure and the separate tree partitioning structure is implemented at the slice level. Therefore, according to an embodiment of the present disclosure, when it is found that a separate tree is applied to a slice, ACT will always be disabled for all CUs (both luma CUs and chroma CUs) within the slice, without signaling the ACT flag, but the ACT flag is inferred to be zero.

[0101] Secondly, in some embodiments, when the ISP mode is enabled (described further below), TU partitioning is applied only to luma samples, while chroma samples are encoded and decoded without being further split into multiple TUs. Assuming that N is the number of ISP sub-partitions (i.e., TUs) of an intra-frame CU, according to the current ISP design, only the last TU contains both luma and chroma components, while the first N-1 ISP TUs consist only of luma components. According to an embodiment of the present disclosure, ACT is disabled in ISP mode. There are two ways to disable ACT for ISP mode. In the first method, an ACT enable / disable flag (i.e., cu_act_enabled_flag) is signaled before the syntax of the ISP mode is signaled. In this case, when the flag cu_act_enabled_flag is equal to 1, the ISP mode will not be signaled in the bitstream, but will always be inferred to be zero (i.e., turned off). In the second method, ISP mode signaling is used to bypass the signaling of the ACT flag. Specifically, in this method, the ISP mode is signaled before the flag cu_act_enabled_flag. When ISP mode is selected, the flag cu_act_enabled_flag is not signaled but inferred to be zero. Otherwise (ISP mode is not selected), the flag cu_act_enabled_flag will still be signaled to adaptively select the color space for residual coding of the CU.

[0102] In some embodiments, in addition to forcibly disabling ACT for CUs with misaligned luma and chroma segmentation structures, LMCS is also disabled for CUs to which ACT is applied. In one embodiment, when a CU selects the YCgCo color space to encode and decode its residual (i.e., ACT is 1), both luma mapping and chroma residual scaling are disabled. In another embodiment, when ACT is enabled for a CU, only chroma residual scaling is disabled, while luma mapping can still be applied to adjust the dynamic range of the output luma samples. In the last embodiment, both luma mapping and chroma residual scaling are enabled for CUs to which ACT is applied to encode and decode residuals. There may be multiple ways to implement chroma residual scaling for CUs to which ACT is applied. In one method, chroma residual scaling is applied before inverse ACT during decoding. By this method, this means that when ACT is applied, chroma residual scaling is applied to chroma residuals (i.e., Cg and Co residuals) in the YCgCo domain. In another method, chroma residual scaling is applied after inverse ACT. Specifically, by the second method, chroma scaling is applied to residuals in the original color space. Assuming the input video is captured in RGB format, this means that chroma residual scaling is applied to the residuals of the B and R components.

[0103] In some embodiments, a syntax element (e.g., sps_act_enabled_flag) is added to a sequence parameter set (SPS) to indicate whether ACT is enabled at the sequence level. In addition, when color space conversion is applied to video content where the luma component and the chroma component have the same resolution (e.g., 4:4:4 chroma format 4:4:4), a bitstream conformance requirement needs to be added so that ACT can be enabled only for the 4:4:4 chroma format. Table 1 shows an improved SPS syntax table with the above syntax added.

[0104]

[0105] Table 1 Improved SPS syntax table

[0106] Specifically, sps_act_enabled_flag equal to 1 indicates that ACT is enabled, and sps_act_enabled_flag equal to 0 indicates that ACT is disabled, so that CUs referencing the SPS do not signal the flag cu_act_enabled_flag but infer cu_act_enabled_flag to be 0. When ChromaArrayType is not equal to 3, a requirement for bitstream conformance is that the value of sps_act_enabled_flag should be equal to 0.

[0107] In another embodiment, instead of always signaling sps_act_enabed_flag, the signaling of the flag is conditional on the chroma type of the input signal. Specifically, assuming that ACT is applicable only when the luma component and the chroma components have the same resolution, the flag sps_act_enabled_flag is signaled only when the input video is captured in 4:4:4 chroma format. In this changed case, the improved SPS syntax table is:

[0108]

[0109] Table 2 Improved SPS syntax table with signaling conditions

[0110] In some embodiments, the syntax design specifications for decoding video data using ACT are shown in the following table.

[0111]

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119] Table 3 Signaling ACT mode specifications

[0120] The flag cu_act_enabled_flag equal to 1 indicates that the residual of the coding unit is encoded in the YCgCo color space, and the flag cu_act_enabled_flag equal to 0 indicates that the residual of the coding unit is encoded in the original color space (e.g., RGB or YCbCr). When the flag cu_act_enabled_flag is not present, it is inferred to be equal to 0.

[0121] In the current VVC working draft, when the input video is captured in a 4:4:4 chroma format, the transform skip mode can be applied to both the luma component and the chroma component. Based on this design, in some embodiments, three methods are used below to handle the interaction between ACT and transform skip.

[0122] In one method, when transform skip mode is enabled for an ACT CU, transform skip mode is only applied to luma components and not to chroma components. In some embodiments, the syntax design specification for this method is shown in the following table.

[0123]

[0124] Table 4 Syntax specification when transform skip mode is applied only to luma component

[0125] In another approach, transform skip mode is applied to both luma and chroma components. In some embodiments, the syntax design specification for this approach is shown in the following table.

[0126]

[0127] Table 5 Syntax specification when transform skip mode is applied to both luma and chroma components

[0128] In yet another approach, when ACT is enabled for a CU, transform skip mode is always disabled.

[0129] In some embodiments, the grammar design specifications for this approach are shown in the following table.

[0130]

[0131] Table 6 Syntax specification when transform skip mode is always disabled

[0132] Fig. 8A and Figure 8B is a block diagram illustrating an exemplary video decoding process in which a video decoder implements techniques for inverse adaptive color space transform (ACT) and luma mapping and chroma scaling according to some embodiments of the present disclosure. In some embodiments, using ACT (e.g., Figure 7 Inverse ACT 710 in ) and chroma residual scaling (e.g., Figure 6 In some other embodiments, the video bitstream is encoded and decoded using chroma residual scaling rather than using both ACT and chroma residual scaling, so inverse ACT 710 is not required.

[0133] More specifically, Fig. 8AAn embodiment is depicted in which the video codec performs chroma residual scaling 630 prior to the inverse ACT 710. Thus, the video codec performs luma mapping and chroma residual scaling 630 in the color space transformed domain. For example, assuming that the input video is captured in RGB format and transformed into the YCgCo color space, the video codec performs chroma residual scaling 630 on the chroma residuals Cg and Co according to the luma residual Y in the YCgCo color space.

[0134] Figure 8B An alternative embodiment is depicted where the video codec performs chroma residual scaling 630 after the inverse ACT 710. Thus, the video codec performs luma mapping and chroma residual scaling 630 in the original color space domain. For example, assuming the input video is captured in RGB format, the video codec applies chroma residual scaling to the B and R components.

[0135] Fig. 9 is a block diagram illustrating exemplary decoding logic between performing adaptive color space transform (ACT) and performing block differential pulse code modulation (BDPCM) according to some embodiments of the present disclosure.

[0136] BDPCM is a codec tool for screen content coding and decoding. In some embodiments, the BDPCM enable flag is signaled in the SPS at the sequence level. The BDPCM enable flag is signaled only when the transform skip mode is enabled in the SPS.

[0137] When BDPCM is enabled, if the CU size is less than or equal to MaxTsSize×MaxTsSize in terms of luma samples, and if the CU is intra-coded, a flag is sent at the CU level, where MaxTsSize is the maximum block size for which transform skip mode is allowed. This flag indicates whether conventional intra-coding or BDPCM is used. If BDPCM is used, another BDPCM prediction direction flag is further sent to indicate whether the prediction is horizontal or vertical. The block is then predicted using the unfiltered reference samples using a conventional horizontal or vertical intra prediction process. The residual is quantized, and the difference between each quantized residual and its predictor, i.e., the previously coded residual of the adjacent position horizontally or vertically (depending on the BDPCM prediction direction), is coded.

[0138] For a block of size M (height) × N (width), let r i，j is the prediction residual, where 0≤i≤M-1, 0≤j≤N-1. Let Q(r i，j ) represents the residual r i，j , where 0≤i≤M-1, 0≤j≤N-1. BDPCM is applied to the quantized residual values, generating a quantized version of The modified M×N array in It is predicted based on its neighboring quantized residual values. For vertical BDPCM prediction mode, for 0≤j≤(N-1), the following is used to derive

[0139]

[0140] For horizontal BDPCM prediction mode, for 0≤i≤(M-1), the following is derived:

[0141]

[0142] On the decoder side, the above process is reversed to calculate Q(r i，j ), 0≤i≤M-1, 0≤j≤N-1, as follows:

[0143] If vertical BDPCM is used, then

[0144] If horizontal BDPCM is used, then

[0145] The inverse quantized residual Q -1 (Q(r i，j )) is added to the intra-frame block prediction value to generate a reconstructed sample value.

[0146] The predicted quantized residual value is converted to 1 using the same residual coding process as in transform skip mode residual coding. Sent to the decoder. For the MPM mode for future intra-mode encoding and decoding, if the BDPCM prediction direction is horizontal or vertical, the horizontal prediction mode or the vertical prediction mode is stored respectively for the BDPCM-encoded CU. For deblocking, if the two blocks on both sides of the block boundary are encoded and decoded using BDPCM, deblocking is not performed on that particular block boundary. According to the latest VVC working draft, when the input video is in 4:4:4 chroma format, BDPCM can be applied to the luma component and the chroma component by signaling two separate flags for the luma and chroma channels at the CU level, namely, intra_bdpcm_luma_flag and intra_bdpcm_chroma_flag.

[0147] In some embodiments, the video codec implements different logic to better handle the interaction between ACT and BDPCM. For example, when ACT is applied to an intra CU, BDPCM is enabled for the luma component, while BDPCM is disabled for the chroma component (910). The corresponding improved syntax table of the coding unit is as follows:

[0148]

[0149]

[0150]

[0151] Table 7 Enable BDPCM only for the luma component

[0152] In some embodiments, when ACT is applied to an intra CU, BDPCM is enabled for both luma and chroma components (920). The corresponding improved syntax table for the coding unit is as follows:

[0153]

[0154]

[0155]

[0156] Table 8 Enables BDPCM for both luma and chroma components

[0157] In some embodiments, when ACT is applied to an intra CU, BDPCM is disabled for both luma and chroma components (930). In this case, no BDPCM-related syntax elements need to be signaled. The corresponding improved syntax table for the coding unit is as follows:

[0158]

[0159]

[0160]

[0161] Table 9 disables BDPCM for both luma and chroma components

[0162] In some embodiments, ACT is processed using lossless codec. In the HEVC standard, the lossless mode of a CU is indicated by signaling a CU-level flag cu_transquant_bypass_flag as 1. However, in the current VVC standardization process, a different lossless enabling method is applied. Specifically, when a CU is encoded and decoded in lossless mode, it only needs to skip the transform and use a quantization step of 1. This can be achieved by signaling the CU-level QP value as 1 and signaling the TU-level transform_skip_flag as 1. Therefore, in one embodiment of the present disclosure, the lossy ACT transform and the lossless ACT transform are switched for a CU / TU according to the value of transform_skip_flag and the QP value. When the flag transform_skip_flag is equal to 1 and the QP value is equal to 4, the lossless ACT transform is applied; otherwise, the lossy version of the ACT transform is applied, as shown below.

[0163] If transform_skip_flag is equal to 1 and QP is equal to 4, the residual sample r Y 、r Cb and r Cr The (nTbW)×(nTbH) array (where x=0..nTbW-1, y=0..nTbH-1) is modified as follows:

[0164] tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0165] r Y [x][y]=tmp+r Cb [x][y]

[0166] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0167] r Cr [x][y]=r Cb [x][y]+r Cr [x][y]

[0168] Otherwise, the residual sample r Y 、r Cb and r Cr The (nTbW)×(nTbH) array (where x=0..nTbW-1, y=0..nTbH-1) is modified as follows:

[0169] tmp=r Y [x][y]–rCb [x][y]

[0170] r Y [x][y]=r Y [x][y]+r Cb [x][y]

[0171] r Cb [x][y]=tmp-r Cr [x][y]

[0172] r Cr [x][y]=tmp+r Cr [x][y]

[0173] In the above description, different ACT transform matrices are used for lossy and lossless codecs. In order to achieve a more unified design, the lossless ACT transform matrix is ​​used for both lossy and lossless codecs. In addition, assuming that the lossless ACT transform will increase the dynamic range of the Cg component and the Co component by 1 bit, an additional 1-bit right shift is applied to the Cg component and the Co component after the positive ACT transform, and a 1-bit left shift is applied to the Cg component and the Co component before the inverse ACT transform. As described below,

[0174] If transform_skip_flag is equal to 0 or QP is not equal to 4, the residual sample r Cb and r Cr The (nTbW)×(nTbH) array (where x=0..nTbW-1, y=0..nTbH-1) is modified as follows:

[0175] r Cb [x][y]=r Cb [x][y]<<1

[0176] r Cr [x][y]=r Cr [x][y]<<1

[0177] Residual sample r Y 、r Cb and r Cr The (nTbW)×(nTbH) array (where x=0..nTbW-1, y=0..nTbH-1) is modified as follows:

[0178] tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0179] r Y [x][y]=tmp+r Cb [x][y]

[0180] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0181] r Cr [x][y]=r Cb [x][y]+r Cr [x][y]

[0182] In addition, as can be seen from the above, when ACT is applied, QP offsets (-5, -5, -3) are applied to the Y, Cg, and Co components. Therefore, for small input QP values ​​(e.g., <5), negative QPs will be used for quantization / dequantization of undefined ACT transform coefficients. To address this issue, a clipping operation is added after the QP adjustment of ACT so that the applied QP value is always equal to or greater than zero, i.e., QP'=max(QP ORG -QP OFFSET , 0), where QP is the original QP, QP offset is the ACT QP offset and QP' is the adjusted QP value.

[0183] In the method described above, although the same ACT transform matrix (ie, lossless ACT transform matrix) is used for both lossy encoding and lossless encoding, the following two problems can still be identified:

[0184] Depending on whether the current CU is a lossy CU or a lossless CU, different inverse ACT operations are still applied. Specifically, for lossless CU, the inverse ACT transform is applied; for lossy CU, an additional right shift needs to be applied before the inverse ACT transform. In addition, the decoder needs to know whether the current CU is encoded and decoded in lossy mode or lossless mode. This is inconsistent with the current VVC lossless design. In detail, unlike the HEVC lossless design that indicates the lossless mode of a CU by signaling a cu_transquant_bypass_flag, the lossless encoding and decoding in VVC is done in a purely non-normative way, i.e., skipping the transform of the prediction residual (enabling the transform skip mode for the luminance component and the chrominance component), selecting an appropriate QP value (i.e., 4), and explicitly disabling codec tools that prevent lossless encoding and decoding, such as loop filters.

[0185] Currently, the QP offset used for normalized ACT transform is fixed. However, in terms of codec efficiency, the choice of the best QP offset may depend on the content itself. Therefore, when enabling the ACT tool in order to maximize its codec gain, it may be more beneficial to allow flexible QP offset signaling.

[0186] Based on the above considerations, a unified ACT design is implemented as follows. First, the lossless ACT forward transform and inverse transform are applied to CUs encoded and decoded in both lossy and lossless modes. Secondly, instead of using fixed QP offsets, the QP offsets applied to the ACT CU are explicitly signaled in the bitstream (i.e., three QP offsets applied to the Y component, the Cg component, and the Co component). Third, in order to prevent possible overflow problems of the QP applied to the ACT CU, the clipping operation is applied to the resulting QP of each ACTCU to obtain a valid QP range. It can be seen that based on the above method, the selection between lossy encoding and lossless encoding can be achieved only by encoder modification (i.e., using different encoder settings). The decoding operation is the same for both lossy and lossless encoding and decoding of the ACT CU. Specifically, in order to achieve lossless encoding and decoding, in addition to the existing encoder-side lossless configuration, the encoder only needs to signal the values ​​of the three QP offsets as zero. On the other hand, in order to achieve lossy encoding and decoding, the encoder can signal non-zero QP offsets. For example, in one embodiment, to compensate for the dynamic range change caused by the lossless ACT transform in the lossy codec, QP offsets (-5, 1, 3) may be signaled for the Y component, the Cg component, and the Co component when the ACT is applied. On the other hand, the ACT QP offset may be signaled at different coding levels (e.g., sequence parameter set (SPS), picture parameter set (PPS), picture header, coding block group level, etc.), which may provide different QP adaptations at different granularities. The following table gives an example of performing QP offset signaling in the SPS.

[0187]

[0188] Table 10 Syntax specification for QP offset signaling in SPS

[0189] In another embodiment, an advanced control flag (e.g., picture_header_act_qp_offset_present_flag) is added at the SPS or PPS. When the flag is equal to zero, it indicates that the QP offset signaled in the SPS or PPS will be applied to all CUs encoded and decoded in ACT mode. Otherwise, when the flag is equal to 1, additional QP offset syntax (e.g., picture_header_y_qp_offset_plus5, picture_header_cg_qp_offset_minus1, and picture_header_co_qp_offset_minus3) can be further signaled in the picture header to individually control the QP value applied to the ACT CU in a specific picture.

[0190] On the other hand, the QP offset sent by the signal should also be applied to clip the final ACT QP value to the effective dynamic range. In addition, different clipping ranges can be applied to CUs that perform codecs with transforms and CUs that do not perform codecs with transforms. For example, when the transform is not applied, the final QP should not be less than 4. Assuming that the ACT QP offset is signaled in the SPS level, the corresponding QP value derivation process for the ACT CU can be described as follows:

[0191] QpY=((qPY_PRED+CuQpDeltaVal+64+2*QpBdOffset+sps_act_y_qp_offset)%(64+QpBdOffset))-QpBdOffset

[0192] Qp′ Cb =Clip3(-QpBdOffset,63,qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb +sps_act_cg_offset)+QpBdOffset

[0193] Qp′ Cr =Clip3(-QpBdOffset,63,qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr +sps_act_co_offset)+QpBdOffset

[0194] Qp′ CbCr =Clip3(-QpBdOffset,63,qP CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffset CbCr +sps_act_cg_offset)+QpBdOffset

[0195] In another embodiment, the ACT enable / disable flag is signaled at the SPS level while the ACT QP offset is signaled at the PPS level to allow the encoder to more flexibly adjust the QP offset applied to the ACT CU in order to improve encoding and decoding efficiency. Specifically, the SPS and PPS syntax tables with the described changes are presented in the following table.

[0196]

[0197]

[0198] Table 11 Syntax specification for signaling ACT enable / disable flag at SPS level and ACT QP offset at PPS level

[0199] pps_act_qp_offset_present_flag equal to 1 specifies that pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are present in the bitstream. When pps_act_qp_offset_present_flag is equal to 0, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are not present in the bitstream. Bitstream conformance is that the value of pps_act_qp_offset_present_flag shall be 0 when sps_act_enabled_flag is equal to 0.

[0200] pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are used to determine the offset applied to the value of the quantization parameter used for the luma and chroma components of coding blocks with cu_act_enabled_flag equal to 1. When not present, the values ​​of pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_cr_qp_offset_minus3 are inferred to be equal to 0.

[0201] In the above PPS signaling, the same QP offset value is applied to the ACT CU when the joint coding and decoding of chroma residual (JCCR) mode is applied or not. Considering that only the residual of one signal chroma component is coded and decoded in the JCCR mode, this design may not be optimal. Therefore, in order to achieve a better codec gain, when the JCCR mode is applied to an ACT CU, a different QP offset can be applied to codec the residual of the chroma component. Based on this consideration, a separate QP offset signaling is added to the PPS for the JCCR mode, as described below.

[0202]

[0203] Table 12 Syntax specification for adding separate QP offset signaling in PPS for JCCR mode

[0204] pps_joint_cbcr_qp_offset is used to determine the offset applied to the value of the quantization parameter used for the chroma residual of a coded block to which joint chroma residual codec is applied. When not present, the value of pps_joint_cbcr_qp_offset is inferred to be equal to 0.

[0205] In some embodiments, Fig.10 A method for handling ACT when the internal luma and chroma bit depths are different is shown. Specifically, Fig.10 is a decoding flow chart for applying different QP offsets to different components when luma and chroma internal bit depths are different according to some embodiments of the present disclosure.

[0206] According to the existing VVC specification, the luminance component and the chrominance component are allowed to use different internal bit depths (expressed as BitDepth Y and BitDepth C ) for encoding and decoding. However, existing ACT designs always assume that the internal luma bit depth and chroma bit depth are the same. In this section, a method is implemented to Y Not equal to BitDepth C Improve ACT design while

[0207] In the first method, the ACT tool is always disabled when the internal luma bit depth is not equal to the bit depth of the chroma components.

[0208] In the second approach, the second solution is implemented as aligning the bit depth of the luma component and the bit depth of the chroma components by left-shifting the component with the smaller bit depth to match the bit depth of the other component; the scaled components will then be readjusted to the original bit depth by right-shifting the bits after the color transform.

[0209] Similar to HEVC, the quantization step size increases by about 2 with each increment of QP 1 / 6 times, and doubles exactly every 6 increments. Based on this design, in the second method, to compensate for the internal bit depth between luma and chroma, the QP value for the component with a smaller internal bit depth is increased by 6Δ, where Δ is the difference between the luma internal bit depth and the chroma internal bit depth. Then, the residual of this component is moved back to the original dynamic range by applying a right shift of Δ bits. Fig.101 shows the corresponding decoding process when the above method is applied. For example, assuming that the input QP value is qp, the default QP values ​​applied to the Y component, Cg component, and Co component are equal to qp-5, qp-5, and qp-3. In addition, assuming that the luma internal bit depth is higher than the chroma bit depth, that is, Δ = BitDepth Y -BitDepth C The final QP values ​​applied to the luma and chroma components are then equal to qp-5, qp-5+6Δ, and qp-3+6Δ.

[0210] In some embodiments, encoder acceleration logic is implemented. In order to select the color space for residual codec of a CU, the most direct approach is to let the encoder check each codec mode (e.g., intra-frame codec mode, inter-frame codec mode, and IBC mode) twice, once when ACT is enabled and once when ACT is disabled. This can approximately double the encoding complexity. In order to further reduce the encoding complexity of ACT, the following encoder acceleration logic is implemented in the present disclosure:

[0211] First, since the YCgCo space is more compact than the RGB space, when the input video is in RGB format, it is implemented to first check the rate-distortion (RD) cost of the ACT tool enabled, and then check the RD cost of the ACT tool disabled. In addition, when ACT is enabled, the RD cost of disabling color space conversion is calculated only when there is at least one non-zero coefficient. Optionally, when the input video is in YCbCr format, it is implemented to check the RD cost of disabling ACT after the RD check of enabling ACT. When ACT is disabled, the second RD check (i.e., enabling ACT) is performed only when there is at least one non-zero coefficient.

[0212] Secondly, in order to reduce the number of tested codec modes, the same codec mode is used for the two color spaces. More specifically, for intra mode, the selected intra prediction mode for full RD cost comparison is shared between the two color spaces; for inter mode, the selected motion vector, reference picture, motion vector predictor and merge index (for inter merge mode) are shared between the two color spaces; for IBC mode, the selected block vector, block vector predictor and merge index (for IBC merge mode) are shared between the two color spaces.

[0213] Third, due to the quaternary / binary / ternary tree partitioning structure used in VVC, a same block partition can be obtained through different partition combinations. In order to speed up the color space selection, it is implemented to use ACT to enable / disable decisions when the same block is achieved through different partitioning paths. Specifically, when a specific CU is first encoded and decoded, the selected color space used to encode and decode the residual of the CU will be stored. Then, when the same CU is obtained through another partitioning path, instead of choosing between the two spaces, the stored color space decision will be directly reused.

[0214] Fourth, considering the strong correlation between a CU and its spatial neighbors, it is implemented to use the color space selection information of the spatial neighboring blocks of the CU to decide how many color spaces need to be checked for the residual encoding and decoding of the current CU. For example, if a sufficient number of spatial neighboring blocks choose the YCgCo space to encode and decode their residuals, it is reasonable to infer that the current CU may choose the same color space. Accordingly, the RD check for encoding and decoding the residual of the current CU in the original color space can be skipped. If enough spatial neighbors choose the original color space, the RD check for residual encoding and decoding in the YCgCo domain can be bypassed. Otherwise, two color spaces need to be tested.

[0215] Fifth, considering the strong correlation between CUs in the same area, a CU may choose the same color space as its parent CU to encode and decode its residual. Or the child CU may derive the color space from the information of its parent CU, such as the selected color space and the RD cost of each color space. Therefore, in order to simplify the coding complexity, if the residual of a CU's parent CU is encoded in the YCgCo domain, the RD cost of residual coding and decoding in the RGB domain will be skipped for the CU; in addition, if the residual of its parent CU is encoded in the RGB domain, the RD cost of residual coding and decoding in the YCgCo domain will be skipped. Another conservative way is to use the RD cost of its parent CU in the two color spaces if two color spaces are tested in the encoding of its parent CU. If its parent CU selects the YCgCo color space and the RD cost of YCgCo is much smaller than the RD cost of RGB, the RGB color space will be skipped, and vice versa.

[0216] In some embodiments, 4:4:4 video codec efficiency is improved by enabling only luma codec tools for chroma components. Because the main focus of VVC design is for videos captured in 4:2:0 chroma format, most existing inter / intra codec tools are enabled only for luma components and disabled for chroma components. However, as mentioned earlier, video signals in 4:4:4 chroma format show completely different characteristics compared to 4:2:0 video signals. For example, similar to the luma component, the Cb / B and Cr / R components of 4:4:4YCbCr / RGB video typically contain useful high-frequency texture and edge information. This is different from the chroma components in 4:2:0 video, which are typically very smooth and contain much less information than the luma components. Based on such analysis, the following method is implemented to extend some of the current luma-only codec tools in VVC to chroma components when the input video is in 4:4:4 chroma format.

[0217] First, a luma interpolation filter is enabled for the chroma component. Similar to HEVC, the VVC standard uses motion compensated prediction technology to exploit redundancy between temporally adjacent pictures, which supports motion vectors accurate to one sixteenth of a pixel for the Y component and motion vectors accurate to one thirty-second of a pixel for the Cb and Cr components. A set of separable 8-tap filters is used to interpolate fractional samples. The fractional interpolation of the Cb and Cr components is essentially the same as the fractional interpolation of the Y component, except that a separable 4-tap filter is used for the 4:2:0 video format. This is because for 4:2:0 video, the Cb and Cr components contain much less information than the Y component, and when compared to using an 8-tap interpolation filter, the 4-tap interpolation filter can reduce the complexity of the fractional interpolation filtering without sacrificing the efficiency of motion compensated prediction of the Cb and Cr components.

[0218] As previously mentioned, existing 4-tap chroma interpolation filters may not be efficient for interpolating fractional samples for motion compensated prediction of chroma components in 4:4:4 video. Therefore, in one embodiment of the present disclosure, it is implemented to use the same set of 8-tap interpolation filters (which are used for luma components in 4:2:0 video) for fractional sample interpolation of both luma components and chroma components in 4:4:4 video. In another embodiment, for a better trade-off between codec efficiency and complexity, it is implemented to enable adaptive interpolation filter selection for chroma samples in 4:4:4 video. For example, an interpolation filter selection flag may be signaled in the SPS, PPS, and / or slice levels to indicate whether an 8-tap interpolation filter (or other interpolation filter) or a default 4-tap interpolation filter is used for chroma components at various encoding levels.

[0219] Second, PDPC and MRL are enabled for chroma components.

[0220] The Position-Dependent Intra Prediction Combining (PDPC) tool in VVC extends the above idea by employing a weighted combination of intra prediction samples with unfiltered reference samples. In the current VVC working draft, PDPC is enabled without signaling for the following intra modes: planar, DC, horizontal (i.e., mode 18), vertical (i.e., mode 50), angular directions close to the lower left diagonal direction (i.e., modes 2, 3, 4, ..., 10), and angular directions close to the upper right diagonal direction (i.e., modes 58, 59, 60, ..., 66). Assuming that the prediction sample located at coordinates (x, y) is pred(x, y), its corresponding value after PDPC is calculated as

[0221] pred(x,y)=(wL×R-1,y+wT×Rx,-1–wTL×R-1,-1+(64–wL–wT+wTL)×pred(x,y)+32)>>6

[0222] Where Rx,-1,R-1,y represent the reference samples located at the top and left of the current sample (x, y), respectively, and R-1,-1 represents the reference sample located at the upper left corner of the current block. The weights wL, wT and wTL in the above equation are adaptively selected according to the prediction mode and the sample position, as described below, assuming that the size of the current coding block is W×H:

[0223] For DC mode,

[0224] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift), wTL=(wL>>4)+(wT>>4)

[0225] For planar mode,

[0226] wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=0

[0227] For horizontal mode:

[0228] wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=wT

[0229] For vertical mode:

[0230] wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),wTL=wL For the lower left diagonal direction:

[0231] wT=16>>((y<<1)>>shift),wL=16>>((x<<1)>>shift),wTL=0

[0232] For the upper right diagonal direction:

[0233] wT=16>>((y<<1)>>shift),wL=16>>((x<<1)>>shift),wTL=0

[0234] Where shift=(log2(W)–2+log2(H)–2+2)>>2.

[0235] Unlike HEVC, which only reconstructs the nearest row / column of samples as reference, multiple reference lines (MRL) are introduced in VVC, where two additional rows / columns are used for intra prediction. The index of the selected reference row / column is signaled from the encoder to the decoder. When non-nearest row / column is selected, planar mode and DC mode are excluded from the set of intra modes that can be used to predict the current block.

[0236] In the current VVC design, the PDPC tool is only adopted by the luma component to reduce / remove discontinuities between intra-prediction samples and reference samples derived from reconstructed neighboring samples. However, as mentioned above, there may be rich texture information in the chroma blocks in the video signal in the 4:4:4 chroma format. Therefore, tools (such as PDPC) that use a weighted average of unfiltered reference samples and intra-prediction samples to improve prediction quality should also be beneficial to improve the chroma encoding and decoding efficiency of 4:4:4 video. Based on this consideration, in one embodiment of the present disclosure, it is implemented to enable the PDPC process for intra-prediction of chroma components in 4:4:4 video.

[0237] The same considerations can also be extended to the MRL tool. In the current VVC, MRL cannot be applied to chroma components. Based on an embodiment of the present disclosure, it is implemented to enable MRL for chroma components of 4:4:4 video by signaling an MRL index for the chroma components of an intra CU. Based on this embodiment, different methods can be used. In one method, an additional MRL index may be signaled and shared by both Cb / B and Cr / R components. In another method, it is implemented to signal two MRL indexes, one for each chroma component. In a third method, it is implemented to reuse the luma MRL index for intra prediction of the chroma component, so that no additional MRL signaling is required to enable MRL for the chroma component.

[0238] Third, enable ISP for chroma components.

[0239] In some embodiments, a codec tool called sub-segmentation prediction (ISP) is introduced into VVC to further improve the efficiency of intra-frame coding and decoding. Conventional intra-frame modes only use reconstructed samples adjacent to a CU to generate intra-frame prediction samples of a block. Based on this design, the spatial correlation between the prediction samples and the reference samples is roughly proportional to the distance between them. Therefore, samples at the internal part (especially samples located at the lower right corner of the block) usually have worse prediction quality than samples near the block boundary. Depending on the block size, ISP divides the current CU into 2 or 4 sub-blocks in the horizontal or vertical direction, and each sub-block contains at least 16 samples. The reconstructed samples in a sub-block can be used as a reference for predicting the samples in the next sub-block. Repeat the above process until all sub-blocks in the current CU are coded and decoded. In addition, in order to reduce signaling overhead, all sub-blocks in an ISP CU share the same intra-frame mode. In addition, according to the existing ISP design, sub-block segmentation is only applicable to luminance components. Specifically, only the luma samples of one ISP CU can be further split into multiple sub-blocks (or TUs), and each luma sub-block is encoded and decoded separately. However, the chroma samples of the ISP CU are not divided. In other words, for the chroma component, the CU is used as a processing unit for intra prediction, transform, quantization, and entropy encoding and decoding without further segmentation.

[0240] In the current VVC, when the ISP mode is enabled, TU splitting is only applied to luma samples, while chroma samples are encoded and decoded without further splitting into multiple TUs. According to an embodiment of the present disclosure, due to the rich texture information in the chroma plane, it is implemented to enable the ISP mode for chroma encoding and decoding in 4:4:4 video as well. Based on this embodiment, different methods can be used. In one method, an additional ISP index is signaled and shared by two chroma components. In another method, it is implemented to separately signal two additional ISP indexes, one additional ISP index for Cb / B and the other additional ISP index for Cr / R. In a third method, it is implemented to reuse the ISP index used for the luma component for ISP prediction of the two chroma components.

[0241] Fourth, matrix-based intra prediction (MIP) is enabled as a new intra prediction technique for chroma components.

[0242] To predict samples of a rectangular block of width W and height H, MIP takes as input a column of H reconstructed neighboring boundary samples on the left side of the block and a row of W reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, they are generated as in conventional intra prediction.

[0243] In some embodiments, the MIP mode is enabled only for the luma component. For the same reason as the ISP mode is enabled for the chroma components, in one embodiment, the implementation enables MIP for the chroma components of 4:4:4 video. Two signaling methods can be applied. In the first method, the implementation separately signals two MIP modes, one MIP mode for the luma component and another MIP mode for the two chroma components. In the second method, the implementation only signals one single MIP mode shared by the luma component and the chroma components.

[0244] Fifth, multiple transform selection (MTS) is enabled for chroma components.

[0245] In addition to DCT-II already adopted in HEVC, the MTS scheme is also used for residual coding and decoding of both inter-frame coded blocks and intra-frame coded blocks. It uses multiple selected transforms from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII.

[0246] In the current VVC, the MTS tool is enabled only for the luma component. In one embodiment of the present disclosure, MTS is enabled for the chroma components of a 4:4:4 video. Two signaling methods can be applied. In the first method, when MTS is enabled for a CU, two transform indexes are respectively signaled, one for the luma component and the other for the two chroma components. In the second method, one transform index shared by the luma component and the chroma component is signaled when MTS is enabled.

[0247] In some embodiments, unlike the HEVC standard, which uses a fixed lookup table to derive the QP used by the chroma component based on the luma quantization parameter (QP), the VVC standard sends a luma-to-chroma mapping table from the encoder to the decoder, which is defined by several pivot points of a piecewise linear function. Specifically, the syntax elements and reconstruction process of the luma-to-chroma mapping table are described as follows:

[0248]

[0249] Table 13 Syntax elements and reconstruction process of luma to chroma mapping table

[0250] same_qp_table_for_chroma equal to 1 specifies that only one chroma QP mapping table is signaled, which applies to both Cb residual and Cr residual, and when sps_joint_cbcr_enabled_flag is equal to 1, the table additionally applies to the joint Cb-Cr residual.

[0251] same_qp_table_for_chroma equal to 0 specifies that the chroma QP map is signaled in the SPS, two chroma QP maps for Cb and Cr, and one chroma QP map additionally for joint Cb-Cr when sps_joint_cbcr_enabled_flag is equal to 1. When same_qp_table_for_chroma is not present in the bitstream, the value of same_qp_table_for_chroma is inferred to be equal to 1.

[0252] qp_table_start_minus26[i] plus 26 specifies the starting luma and chroma QPs used to describe the i-th chroma QP map table. The value of qp_table_start_minus26[i] shall be in the range of -26-QpBdOffset to 36, inclusive. When qp_table_start_minus26[i] is not present in the bitstream, the value of qp_table_start_minus26[i] is inferred to be equal to 0.

[0253] num_points_in_qp_table_minus1[i] plus 1 specifies the number of points used to describe the i-th chroma QP map table. The value of num_points_in_qp_table_minus1[i] shall be in the range of 0 to 63+QpBdOffset, inclusive. When num_points_in_qp_table_minus1[0] is not present in the bitstream, the value of num_points_in_qp_table_minus1[0] is inferred to be equal to 0.

[0254] delta_qp_in_val_minus1[i][j] specifies the delta value used to derive the input coordinates of the j-th pivot point of the i-th chroma QP map. When delta_qp_in_val_minus1[0][j] is not present in the bitstream, the value of delta_qp_in_val_minus1[0][j] is inferred to be equal to 0.

[0255] delta_qp_diff_val[i][j] specifies the delta value used to derive the output coordinates of the j-th pivot point of the i-th chroma QP map.

[0256] The i-th chroma QP mapping table ChromaQpTable[i] (i=0..numQpTables-1) is derived as follows:

[0257] qpInVal[i][0]=-qp_table_start_minus26[i]+26

[0258] qpOutVal[i][0]=qpInVal[i][0]

[0259] for(j=0;j<=num_points_in_qp_table_minus1[i];j++){

[0260] qpInVal[i][j+1]=qpInVal[i][j]+

[0261] delta_qp_in_val_minus1[i][j]+1

[0262] qpOutVal[i][j+1]=qpOutVal[i][j]+

[0263] (delta_qp_in_val_minus1[i][j]^delta_qp_diff_val[i][j])

[0264] }

[0265] ChromaQpTable[i][qpInVal[i][0]]=qpOutVal[i][0]

[0266] for(k=qpInVal[i][0]-1;k>=-QpBdOffset;k--)

[0267] ChromaQpTable[i][k]=Clip3(-QpBdOffset,63,ChromaQpTable[i][k+1]-1)

[0268] for(j=0;j<+num_points_in_qp_table_minus1[i];j++){

[0269] sh=(delta_qp_in_val_minus1[i][j]+1)>>1

[0270] for(k=qpInVal[i][j]+1,m=1;k<=qpInval[i][j+1];k++,m++)

[0271] ChromaQpTable[i][k]=ChromaQpTable[i][qpInVal[i][j]]+((qpOutVal[i][j+1]-qpOutVal[i][j])*m+sh) /

[0272] (delta_qp_in_val_minus1[i][j]+1)

[0273] }

[0274] for(k=qpInVal[i][num_points_in_qp_table_minus1[i]+1]+1; k<=63; k++)

[0275] ChromaQpTable[i][k]=Clip3(-QpBdOffset,63,ChromaQpTable[i][k-1]+1)

[0276] In some embodiments, an improved luma-to-chroma mapping function for RGB video is disclosed herein.

[0277] In the current VTM-7.1 software, when the input video is in RGB format, a two-stage linear function is sent from the encoder to the decoder to map the luma QP to the chroma QP. This is done by setting the syntax elements same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0, and delta_qp_diff_val[0][0]=0. Specifically, the corresponding luma-to-chroma QP mapping function used in VTM-7.1 is defined as

[0278]

[0279] Assuming the intra coding bit depth is 10 bits, Table 14 shows the luma to chroma QP mapping function applied to RGB codec in VTM-7.1.

[0280]

[0281] Table 14 Luma to Chroma QP Mapping Table for RGB Codec in VTM-7.1

[0282] As shown in (5), when the luma QP is greater than 26, unequal QP values ​​are used to encode and decode the luma component and the chroma component. In some embodiments, considering the use of weighted chroma distortion when calculating the RD cost for mode decision, the unequal QP values ​​have an impact not only on the quantization / dequantization process, but also on the decisions made during rate-distortion (RD) optimization, as shown in the following equation.

[0283] J mode =(SSE luma +w chroma ·SSE chroma )+λ mode ·R mode (6)

[0284] Among them SSE luma and SSE chroma are the distortion of the luminance component and the chrominance component respectively; R mode is the number of bits; mode is the Lagrange multiplier; w chroma is a weighting parameter for chroma distortion, which is calculated as

[0285]

[0286] However, there is a stronger correlation between the three channels of RGB video than YCbCr / YUV video. Therefore, when video content is captured in RGB format, strong texture and high-frequency information are usually present in all three components, that is, the information in R, G and B is equally important. Therefore, in one embodiment of the present disclosure, equal QP values ​​are applied to all three channels for RGB encoding and decoding. This can be accomplished by setting the corresponding luma to chroma QP mapping syntax elements to same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=1. Accordingly, using the method disclosed herein, the RGB luma to chroma mapping function is shown by equation (8) and Table 15.

[0287] QP c =QP L (8)

[0288]

[0289] Table 15 Luma to Chroma QP Mapping Table for RGB Codec

[0290] Fig.11 1100 is a flowchart illustrating an exemplary process for a video decoder (e.g., video decoder 30) to decode video data based on a luma-to-chroma quantization parameter (QP) syntax element set by using a luma-to-chroma quantization parameter mapping table according to some embodiments of the present disclosure.

[0291] In some embodiments, video decoder 30 obtains a set of luma-to-chroma quantization parameter syntax elements from a data bitstream ( 1110 ).

[0292] In some embodiments, video decoder 30 generates a luma-to-chroma quantization parameter mapping table based on a luma-to-chroma quantization parameter (QP) syntax element set, wherein the luma-to-chroma QP mapping table maps the quantization parameters (QP) of the chroma components to the quantization parameters (QP) of the chroma components. C ) is set equal to the quantization parameter (QP) of the corresponding luminance component L )(1120).

[0293] In some embodiments, video decoder 30 obtains the QP of the luma component of the coding unit from the data bitstream. L (1130).

[0294] In some embodiments, video decoder 30 sets the QP of the chroma component of the coding unit according to the luma-to-chroma QP mapping table. C Determined as the QP of the luma component of the coding unit L Same (1140).

[0295] In some embodiments, video decoder 30 uses QP L and QP C The luma component and the chroma component of the coding unit are encoded and decoded from the data bitstream (1150).

[0296] In some embodiments, the video data is captured in RGB format prior to encoding and decoding.

[0297] In some embodiments, the intra coding bit depth is 10 bits, and the luma to chroma QP mapping table is shown as follows:

[0298]

[0299] In some embodiments, luma-to-chroma quantization parameter syntax element sets are implemented at the sequence level in a sequence parameter set.

[0300] In some embodiments, the luma-to-chroma quantization parameter syntax element set includes same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0, and delta_qp_diff_val[0][0]=1.

[0301] In some embodiments, the luma-to-chroma quantization parameters syntax element set includes same_qp_table_for_chroma equal to 1, which specifies that only one luma-to-chroma QP mapping table applied to Cb and Cr residuals is signaled.

[0302] In some embodiments, the luma-to-chroma quantization parameters syntax element set includes sps_joint_CbCr_enabled_flag equal to 1, which specifies that only one luma-to-chroma QP mapping table that is additionally applied to the residual of the joint Cb-Cr mode is signaled.

[0303] In some embodiments, the luma-to-chroma quantization parameter syntax element set includes qp_table_start_minus26[i] plus 26, which specifies the starting luma QP for describing the i-th luma-to-chroma QP mapping table.

[0304] In some embodiments, the luma-to-chroma quantization parameter syntax element set includes num_points_in_qp_table_minus1[i] plus 1, which specifies the number of points used to describe the i-th luma-to-chroma QP mapping table.

[0305] In some embodiments, the luma-to-chroma quantization parameter syntax element set includes delta_qp_in_val_minus1[i][j], which specifies the delta value used to derive the input coordinates of the j-th pivot point of the i-th luma-to-chroma QP mapping table.

[0306] In some embodiments, the luma-to-chroma quantization parameter syntax element set includes delta_qp_diff_val[i][j], which specifies the delta value used to derive the output coordinates of the j-th pivot point of the i-th luma-to-chroma QP mapping table.

[0307] In some embodiments, the above constraint values ​​in the luma-to-chroma quantization parameter syntax element set are implemented as a bitstream level constraint. That is, when decoding video data of an RGB sequence, syntax elements are received from the bitstream, but the values ​​of the received syntax elements must be equal to predetermined values, i.e., same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=1.

[0308] In some embodiments, it is proposed to abandon sending the corresponding set of luma to chroma quantization parameter syntax elements in the bitstream. Instead, the values ​​of those syntax elements are set to predetermined values ​​by both video encoder 20 and video decoder 30, i.e., same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=1.

[0309] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes, and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or to a communication medium including any medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to (1) a non-volatile tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data segmentation structures for implementing the implementation described in the present application. A computer program product may include a computer-readable medium.

[0310] The terms used in the description of the embodiments herein are only used for the purpose of describing specific embodiments and are not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms "one", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "including" and / or "including ... " when used in this specification, specify the presence of stated features, elements and / or components, but do not exclude the presence or addition of one or more other features, elements, components and / or groups thereof.

[0311] It will also be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the embodiment, the first electrode may be referred to as the second electrode, and similarly, the second electrode may be referred to as the first electrode. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0312] The description of the present application has been presented for the purpose of illustration and description, and is not intended to be exhaustive or limited to the invention in the disclosed form. With the benefit of the teachings presented in the foregoing description and the associated drawings, many modifications, variations and alternative embodiments will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to best explain the principles of the invention, the practical application, and to enable other persons skilled in the art to understand the various embodiments of the invention, and to best utilize the basic principles and various embodiments with various modifications suitable for the intended specific use. Therefore, it will be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding video data, comprising: Obtaining a set of luminance-to-chrominance quantization parameter syntax elements from a data bitstream; Based on the luma-to-chroma quantization parameter syntax element set, a luma-to-chroma quantization parameter QP mapping table is generated, wherein the luma-to-chroma QP mapping table converts the quantization parameter QP of the chroma component into C Directly set to equal the quantization parameter QP of the corresponding luminance component L ; Get the QP of the luma component of the coding unit from the data bitstream L ; The QP of the chroma component of the coding unit is mapped according to the luma-to-chroma QP mapping table. C Determined as the QP of the luma component of the coding unit L same; Using the QP L and the QP C decoding the luma component and the chroma component of the coding unit from the data bitstream, The luma-to-chroma quantization parameter syntax element set includes: same_qp_table_for_chroma=1, qp_table_start_minus26[i] plus 26, num_points_in_qp_table_minus1[i] plus 1, delta_qp_in_val_minus1[i][j], delta_qp_diff_val[i][j], where same_qp_table_for_chroma equal to 1 specifies that only one luma-to-chroma QP mapping table applied to Cb and Cr residuals is signaled, Wherein the qp_table_start_minus26[i] plus 26 specifies the starting luma QP for describing the i-th luma to chroma QP mapping table, where num_points_in_qp_table_minus1[i] plus 1 specifies the number of points used to describe the i-th luma-to-chroma QP mapping table, wherein delta_qp_in_val_minus1[i][j] specifies the delta value of the input coordinates used to derive the jth pivot point of the i-th luma to chroma QP mapping table, wherein delta_qp_diff_val[i][j] specifies the delta value used to derive the output coordinates of the jth pivot point of the i-th luma-to-chroma QP mapping table, Among them, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=1.

2. The method of decoding video data according to claim 1, wherein the video data is captured in RGB format.

3. The method for decoding video data according to claim 1, wherein the intra-coding bit depth is 10 bits, and the generated luminance to chrominance QP mapping table is as follows. 4 . The method of decoding video data of claim 1 , wherein the set of luma-to-chroma quantization parameter syntax elements is implemented at a sequence level in a sequence parameter set.

5. The method of decoding video data according to claim 1, wherein the luma-to-chroma quantization parameter syntax element set includes sps_joint_cbcr_enabled_flag equal to 1, wherein sps_joint_cbcr_enabled_flag equal to 1 specifies that only one luma-to-chroma QP mapping table is signaled, and the luma-to-chroma QP mapping table is additionally applied to the residual of the joint Cb-Cr mode.

6. An electronic device comprising: one or more processing units; a memory coupled to the one or more processing units; as well as A plurality of programs are stored in the memory, and when the plurality of programs are executed by the one or more processing units, the electronic device executes the method as claimed in any one of claims 1 to 5.

7. A non-volatile computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, causes the electronic device to execute the method as claimed in any one of claims 1 to 5.

Images