Method and device for encoding and decoding 4:4:4 chroma format video

Abstract

An electronic device performs a method for encoding and decoding video data. The method includes: receiving a first syntax element indicating whether adaptive color space transform (ACT) is enabled for a coding unit from video data in a bitstream; if the ACT is enabled for the coding unit, receiving a first set of syntax elements associated with block differential pulse code modulation (BDPCM) for a luminance component of the coding unit, the first set of syntax elements including a second syntax element indicating whether the BDPCM is enabled for the luminance component of the coding unit; if the BDPCM is enabled for the luminance component of the coding unit, assigning corresponding values ​​of the first set of syntax elements associated with BDPCM for the luminance component of the coding unit to a second set of syntax elements associated with BDPCM for chrominance components of the coding unit.

Description

[0001] Related Applications

[0002] This application claims priority to provisional application No. 62 / 956,095, filed on December 31, 2019, entitled “Method and Apparatus for 4:4:4 Video Coding and Decoding,” the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present application relates to video data encoding, decoding and compression, and more specifically, to a method and system for adaptive color-space transform (ACT) using block differential pulse coded modulation (BDPCM). Background Art

[0004] Various electronic devices (e.g., digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression standards specified in standards such as MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC) and General Video Codec (VVC). Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data. For block-based video coding, a video frame is divided into one or more strips, each strip having multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf-CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be encoded and decoded in intra mode, inter mode, or intra block copy (IBC) mode. Video blocks in an intra-coded (I) slice of a video frame are encoded using spatial prediction relative to reference samples in neighboring blocks within the same video frame. Video blocks in an inter-coded (P (forward predicted picture) or B (bidirectionally predicted picture)) slice of a video frame can use spatial prediction relative to reference samples in neighboring blocks within the same video frame or temporal prediction relative to reference samples in other previous reference video frames and / or future reference video frames.

[0005] A prediction block for a current video block to be encoded is generated based on spatial prediction or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding the reference block can be accomplished by a block matching algorithm. The residual data representing the pixel differences between the current block to be encoded and the prediction block is called a residual block or prediction error. Inter-coded blocks are encoded based on motion vectors pointing to reference blocks in a reference frame that form the prediction block, and the residual block. The process of determining motion vectors is generally referred to as motion estimation. Intra-coded blocks are encoded based on the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain (e.g., frequency domain) to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which are then entropy encoded into the video bitstream to achieve even more compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., a flash memory) for access by another electronic device with digital video capabilities or directly sent to the electronic device in a wired or wireless manner. The electronic device then performs video decompression (which is the opposite process of the video compression described above), for example, by parsing the encoded video bitstream to obtain semantic elements from the bitstream, and reconstructing digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and presenting the reconstructed digital video data on a display of the electronic device.

[0007] As digital video quality changes from HD to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. How to encode / decode video data more efficiently while maintaining the image quality of the decoded video data is an ongoing challenge.

[0008] Some video content (e.g., screen content video) is encoded in a 4:4:4 chroma format, in which all three components (luminance component and two chroma components) have the same resolution. Although the 4:4:4 chroma format includes more redundancy than the redundancy of the 4:2:0 chroma format and the 4:2:2 chroma format (which is not conducive to achieving good compression efficiency), the 4:4:4 chroma format is still the preferred encoding format for many applications that require high fidelity to preserve color information (e.g., sharp edges) in the decoded video. Given the redundancy present in 4:4:4 chroma format video, there is evidence that significant codec improvements can be achieved by exploiting the correlation between the three color components of 4:4:4 video (e.g., Y, Cb, and Cr in the YCbCr domain; or G, B, and R in the RGB domain). Due to these correlations, during the development of the HEVC screen content coding (SCC) extension, the Adaptive Color Space Transform (ACT) tool was used to exploit the correlation between the three color components. Summary of the invention

[0009] The present application describes implementations related to video data encoding and decoding, and more particularly to methods and systems for adaptive color space conversion (ACT) using block differential pulse code modulation (BDPCM).

[0010] For video signals originally captured in a 4:4:4 color format, if high fidelity is required for the decoded video signal and there is rich information redundancy in the original color space (such as RGB video), it is preferred to encode the video in the original space. Although some inter-component coding tools in the current VVC standard (such as cross-component linear model prediction (CCLM)) can improve the efficiency of 4:4:4 video encoding and decoding, the redundancy between the three components is not completely eliminated. This is because only the Y / G components are used to predict the Cb / B and Cr / R components without considering the correlation between the Cb / B and Cr / R components. Accordingly, further decorrelation of the three color components can improve the encoding and decoding performance for 4:4:4 video encoding and decoding.

[0011] In the current VVC standard, the design of existing inter-frame and intra-frame tools is mainly focused on videos captured in 4:2:0 chroma format. Therefore, in order to achieve a better complexity / performance trade-off, most codec tools only apply to luma components, but disable chroma components (such as position-dependent intraprediction combination (PDPC), multi-reference line (MRL) and sub-partition prediction (ISP)), or use different operations for luma and chroma components (such as interpolation filters applied to motion compensated prediction). However, compared with 4:2:0 video, video signals in 4:4:4 chroma format show very different characteristics. For example, the Cb / B and Cr / R components of 4:4:4YCbCr and RGB videos show richer color information and have more high-frequency information (such as edges and textures) than the chroma components in 4:2:0 video. Considering this, it may always be best to use the same design of some existing codec tools for 4:2:0 and 4:4:4 videos in VVC.

[0012] According to a first aspect of the present application, a method for decoding video data includes: receiving video data corresponding to a coding unit from a bitstream, wherein the coding unit is encoded in an intra-frame prediction mode; receiving a first syntax element from the video data, wherein the first syntax element indicates whether adaptive color space transform (ACT) is enabled for the coding unit; based on a determination that the ACT is enabled for the coding unit: receiving a first set of syntax elements associated with block differential pulse code modulation (BDPCM) for a luminance component of the coding unit from the video data, wherein the first set of syntax elements includes a second syntax element indicating whether the BDPCM is enabled for the luminance component of the coding unit; based on a determination that the BDPCM is enabled for the luminance component of the coding unit: assigning corresponding values ​​of the first set of syntax elements associated with the BDPCM for the luminance component of the coding unit to a second set of syntax elements associated with the BDPCM for the chrominance components of the coding unit; and decoding the coding unit from the video data based on the first syntax element associated with the ACT and the first and second sets of syntax elements associated with the BDPCM.

[0013] According to a second aspect of the present application, a method for decoding video data comprises: receiving video data corresponding to a coding unit from a bitstream, wherein the coding unit is encoded in an intra-frame prediction mode; receiving a first syntax element from the video data, wherein the first syntax element indicates whether adaptive color space transform (ACT) is enabled for the coding unit; based on a determination that the ACT is enabled for the coding unit: receiving a second syntax element from the video data, wherein the second syntax element indicates a direction of a luminance component in the intra-frame prediction mode for the coding unit; based on a determination that the direction of the luminance component in the intra-frame prediction mode for the coding unit is pure horizontal or pure vertical: receiving a third syntax element indicating whether block differential pulse code modulation (BDPCM) is enabled for a chrominance component of the coding unit; a method element; according to a determination that the third syntax element is non-zero, assigning the value of the direction of the luma component in the intra-prediction mode from the second syntax element to a fourth syntax element indicating the BDPCM direction of the chroma component; decoding the coding unit from the video data according to the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element; according to a determination that the direction of the luma component in the intra-prediction mode for the coding unit is neither purely horizontal nor purely vertical: setting the third syntax element indicating whether the BDPCM is enabled for the chroma component of the coding unit to zero indicating that the BDPCM is not enabled for the chroma component of the coding unit; and decoding the coding unit from the video data according to the first syntax element, the second syntax element, and the third syntax element.

[0014] According to a third aspect of the present application, an electronic device includes: one or more processing units, a memory, and a plurality of programs stored in the memory. When the programs are executed by the one or more processing units, the electronic device implements the method described above.

[0015] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. When the programs are executed by the one or more processing units, the electronic device implements the method as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of the specification, illustrate the described embodiments and together with the description serve to explain the basic principles. Like reference numerals refer to corresponding parts.

[0017] Figure 1is a block diagram illustrating an exemplary system for encoding and decoding video blocks according to some embodiments of the present disclosure.

[0018] Figure 2 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0019] Figure 3 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0020] FIG. 4A to FIG. 4E is a block diagram illustrating how a frame may be recursively partitioned into multiple video blocks of different sizes and shapes according to some embodiments of the present disclosure.

[0021] Figure 5A and 5B is a block diagram illustrating an example of applying an adaptive color space conversion (ACT) technique to convert a residual between an RGB color space and a YCgCo color space according to some embodiments of the present disclosure.

[0022] Figure 6 is a block diagram of applying a luma mapping with chroma scaling (LMCS) technique in an exemplary video data decoding process according to some embodiments of the present disclosure.

[0023] Figure 7 is a block diagram illustrating an exemplary video decoding process in which a video decoder implements an inverse adaptive color space transform (ACT) technique according to some embodiments of the present invention.

[0024] Fig. 8A and 8B is a block diagram illustrating an exemplary video decoding process in which a video decoder implements Inverse Adaptive Color Space Transform (ACT) and Luma Mapping with Chroma Scaling (LMCS) techniques according to some embodiments of the present invention.

[0025] Fig. 9 is a block diagram illustrating exemplary decoding logic between performing Adaptive Color Space Transform (ACT) and Block Differential Pulse Code Modulation (BDPCM) according to some embodiments of the present disclosure.

[0026] Fig.10 is a decoding flow chart for applying different quantization parameter (QP) offsets for different components when luma and chroma internal bit depths are different according to some embodiments of the present disclosure.

[0027] Fig.11is a flowchart illustrating an exemplary process by which a video decoder according to some embodiments of the present disclosure decodes video data by conditionally implementing block differential pulse code modulation (BDPCM) for luma and chroma components based on the interaction between inverse adaptive color space transform (ACT) and BDPCM. DETAILED DESCRIPTION

[0028] Reference will now be made in detail to specific embodiments, examples of which are shown in the accompanying drawings. In the following detailed description, a number of non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.

[0029] In some embodiments, these methods are provided to improve the encoding and decoding efficiency of the VVC standard for 4:4:4 video. In general, the main features of the technology in this disclosure are summarized as follows.

[0030] In some embodiments, these methods are implemented to improve existing ACT designs that enable adaptive color space transformation in the residual domain. In particular, special considerations are made to handle the interaction of ACT with some existing coding tools in VVC.

[0031] In some embodiments, these methods are implemented to improve the efficiency of some existing inter-frame and intra-frame coding tools in the VVC standard for 4:4:4 video, including: 1) enabling 8-tap interpolation filters for chroma components; 2) enabling PDPC for intra-frame prediction of chroma components; 3) enabling MRL for intra-frame prediction of chroma components; 4) enabling ISP partitioning for chroma components.

[0032] Figure 1 is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown in , system 10 includes a source device 12 that generates and encodes video data to be later decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smart phones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0033] In some embodiments, the target device 14 may receive the encoded video data to be decoded via the link 16. The link 16 may include any type of communication medium or device capable of moving the encoded video data from the source device 12 to the target device 14. In one example, the link 16 may include a communication medium that enables the source device 12 to directly send the encoded video data to the target device 14 in real time. The encoded video data may be modulated according to a communication standard (e.g., a wireless communication protocol) and sent to the target device 14. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that may be beneficial to facilitate communication from the source device 12 to the target device 14.

[0034] In some other embodiments, the encoded video data may be sent from the output interface 22 to the storage device 32. Subsequently, the encoded video data in the storage device 32 may be accessed by the target device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a digital versatile disc (DVD), a compact disc read-only memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device 32 may correspond to a file server or another intermediate storage device that may hold the encoded video data generated by the source device 12. The target device 14 may access the stored video data from the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include a network server (e.g., for a website), a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. Target device 14 may access the encoded video data through any standard data connection suitable for accessing encoded video data stored on a file server, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both wireless channels and wired connections. The transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both streaming and download transmissions.

[0035] like Figure 1As shown in , source device 12 includes video source 18, video encoder 20 and output interface 22. Video source 18 may include sources such as or a combination of such sources: a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or a video phone. However, the embodiments described in this application may be generally applicable to video encoding and decoding, and may be applied to wireless and / or wired applications.

[0036] The captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video data may be sent directly to the target device 14 via the output interface 22 of the source device 12. The encoded video data may also (or alternatively) be stored on a storage device 32 for later access by the target device 14 or other devices for decoding and / or playback. The output interface 22 may further include a modem and / or a transmitter.

[0037] Target device 14 includes input interface 28, video decoder 30, and display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data sent over a communication medium, stored on a storage medium, or stored on a file server.

[0038] In some implementations, the target device 14 may include a display device 34, which may be an integrated display device and an external display device configured to communicate with the target device 14. The display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0039] The video encoder 20 and the video decoder 30 can operate according to a proprietary standard or an industry standard (e.g., VVC, HEVC, Part 10 of MPEG-4, AVC) or an extension of such a standard. It should be understood that the present application is not limited to a specific video encoding / decoding standard and can be applied to other video encoding / decoding standards. It is generally believed that the video encoder 20 of the source device 12 can be configured to encode the video data according to any standard in these current standards or future standards. Similarly, it is also generally believed that the video decoder 30 of the target device 14 can be configured to decode the video data according to any standard in these current standards or future standards.

[0040] The video encoder 20 and the video decoder 30 may be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic devices, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in the hardware to perform the video encoding / decoding operations disclosed in the present disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either of the encoders or decoders may be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device.

[0041] Figure 2 is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described herein. The video encoder 20 may perform intra-frame prediction coding and inter-frame prediction coding on video blocks within a video frame. Intra-frame prediction coding relies on spatial prediction to reduce or remove spatial redundancy in video data within a given video frame or picture. Inter-frame prediction coding relies on temporal prediction to reduce or remove temporal redundancy in video data within adjacent video frames or pictures of a video sequence.

[0042] like Figure 2As shown in , the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A loop filter 63 such as a deblocking filter can be located between the adder 62 and the DPB 64 to filter the block boundary to remove the block effect from the reconstructed video. In addition to the deblocking filter, another loop filter (e.g., a sample adaptive offset (SAO) filter and / or an adaptive loop filter (ALF)) can also be used to filter the output of the adder 62. In some examples, the loop filter can be omitted, and the decoded video block can be directly provided to the DPB 64 by the adder 62. Video encoder 20 may take the form of a fixed or programmable hardware unit, or may be distributed among one or more of the illustrated fixed or programmable hardware units.

[0043] Video data memory 40 may store video data to be encoded by components of video encoder 20. Figure 1 The video source 18 shown obtains video data in the video data memory 40. The DPB 64 is a buffer that stores reference video data (e.g., reference frames or pictures) for use by the video encoder 20 (e.g., in intra-frame or inter-frame prediction coding mode) when encoding the video data. The video data memory 40 and the DPB 64 can be formed by any of a variety of memory devices. In various examples, the video data memory 40 can be on-chip with other components of the video encoder 20, or off-chip relative to those components.

[0044] like Figure 2As shown in , after receiving the video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into slices, tiles, or other larger coding units (CUs) according to a predefined splitting structure (e.g., a quadtree structure) associated with the video data. The video frame may be divided into a plurality of video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of a plurality of feasible prediction coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of one or more inter-frame prediction coding modes in a plurality of intra-frame prediction coding modes. The prediction processing unit 41 may provide the resulting intra-frame prediction coding block or inter-frame prediction coding block to the adder 50 to generate a residual block, and to the adder 62 to reconstruct the coding block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements (e.g., motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information) to the entropy coding unit 56.

[0045] To select an appropriate intra-prediction coding mode for the current video block, intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction coding of the current video block in relation to one or more prediction blocks in one or more reference frames to provide temporal prediction. Video encoder 20 may perform multiple encoding passes, for example, to select an appropriate coding mode for each block of video data.

[0046] In some embodiments, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating a motion vector according to a predetermined pattern within a sequence of video frames, the motion vector indicating the displacement of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating a motion vector that estimates the motion of a video block. For example, a motion vector may indicate the displacement of a video block within a current video frame or picture relative to a prediction block within a reference frame associated with a current block being encoded within the current frame. The predetermined pattern may designate a video frame in a sequence as a P frame or a B frame. Intra BC unit 48 may determine a vector (e.g., a block vector) for intra BC coding in a manner similar to the motion vector determined by motion estimation unit 42 for inter prediction, or may utilize motion estimation unit 42 to determine a block vector.

[0047] In terms of pixel differences, the prediction block for the video block may be or may correspond to a block or reference block of a reference frame that is considered to closely match the video block to be encoded, and the pixel differences may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference metrics. In some embodiments, the video encoder 20 may calculate values ​​for sub-integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 may interpolate values ​​for quarter-pixel positions, one-eighth pixel positions, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 may perform motion searches relative to full pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy.

[0048] Motion estimation unit 42 calculates a motion vector for a video block in an inter-prediction coded frame by comparing the position of the video block to the position of a prediction block of a reference frame selected from a first reference frame list (list 0) or a second reference frame list (list 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vector to motion compensation unit 44 and then to entropy encoding unit 56.

[0049] The motion compensation performed by the motion compensation unit 44 may involve obtaining or generating a prediction block based on the motion vector determined by the motion estimation unit 42. After receiving the motion vector for the current video block, the motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from the DPB 64, and forward the prediction block to the adder 50. The adder 50 then forms a residual video block of pixel difference values ​​by subtracting the pixel values ​​of the prediction block provided by the motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include a luminance difference component or a chrominance difference component or both. The motion compensation unit 44 may also generate syntax elements associated with the video block of the video frame for use by the video decoder 30 when decoding the video block of the video frame. The syntax elements may include, for example, syntax elements defining a motion vector for identifying a prediction block, any flag indicating a prediction mode, or any other syntax information described herein. It should be noted that the motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0050] In some embodiments, the intra BC unit 48 may generate vectors and obtain prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but these prediction blocks are in the same frame as the current block being encoded, and these vectors are referred to as block vectors rather than motion vectors. Specifically, the intra BC unit 48 may determine the intra prediction mode to be used to encode the current block. In some examples, the intra BC unit 48 may encode the current block using various intra prediction modes, for example, during separate encoding passes, and test their performance through rate-distortion analysis. Next, the intra BC unit 48 may select a suitable intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, the intra BC unit 48 may calculate rate-distortion values ​​for the various tested intra prediction modes using rate-distortion analysis, and select an intra prediction mode with the best rate-distortion characteristic among the tested modes as a suitable intra prediction mode to use. The rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra BC unit 48 may calculate ratios based on the distortion and rate for the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0051] In other examples, intra BC unit 48 may use, in whole or in part, motion estimation unit 42 and motion compensation unit 44 to perform such functions for intra BC prediction in accordance with embodiments described herein. In either case, for intra block copying, the prediction block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by SAD, SSD, or other difference metrics, and identifying the prediction block may include calculating values ​​for sub-integer pixel positions.

[0052] Regardless of whether the prediction block is from the same frame according to intra-frame prediction or from a different frame according to inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference values ​​forming the residual video block may include both luma component differences and chroma component differences.

[0053] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 or the intra-frame block copy prediction performed by the intra BC unit 48 as described above, the intra-frame prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-frame prediction processing unit 46 can determine the intra-frame prediction mode for encoding the current block. To this end, the intra-frame prediction processing unit 46 can use various intra-frame prediction modes to encode the current block, for example, during separate encoding passes, and the intra-frame prediction processing unit 46 (or in some examples, the mode selection unit) can select a suitable intra-frame prediction mode from the tested intra-frame prediction modes to use. The intra-frame prediction processing unit 46 can provide information indicating the intra-frame prediction mode selected for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode into the bitstream.

[0054] After prediction processing unit 41 determines a prediction block for the current video block via inter-prediction or intra-prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform (e.g., a discrete cosine transform (DCT) or a conceptually similar transform).

[0055] The transform processing unit 52 may send the resulting transform coefficients to a quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may also reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the quantization unit 54 may then perform a scan on the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0056] After quantization, entropy coding unit 56 entropy encodes the quantized transform coefficients into a video bitstream using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding method or technique. The encoded bitstream may then be sent to a video bitstream such as a Figure 1 The video decoder 30 shown, or archived as Figure 1 The video frame may be stored in storage device 32 as shown for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0057] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for use in generating reference blocks for predicting other video blocks. As noted above, motion compensation unit 44 may generate a motion compensated prediction block from one or more reference blocks of a frame stored in DPB 64. Motion compensation unit 44 may also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values ​​for use in motion estimation.

[0058] Adder 62 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 44 to generate a reference block for storage in DPB 64. The reference block may then be used as a prediction block by intra BC unit 48, motion estimation unit 42, and motion compensation unit 44 to inter-predict another video block in a subsequent video frame.

[0059] Figure 3 1 is a block diagram showing an exemplary video decoder 30 according to some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction unit 84, and an intra-frame BC unit 85. The video decoder 30 may perform the above-mentioned Figure 2 The encoding process is substantially the inverse of the decoding process described with respect to video encoder 20. For example, motion compensation unit 82 may generate prediction data based on motion vectors received from entropy decoding unit 80, and intra-prediction unit 84 may generate prediction data based on intra-prediction mode indicators received from entropy decoding unit 80.

[0060] In some examples, units of the video decoder 30 may be tasked to perform embodiments of the present application. In addition, in some examples, embodiments of the present disclosure may be dispersed in one or more of the units of the video decoder 30. For example, the intra BC unit 85 may perform embodiments of the present application alone or in combination with other units of the video decoder 30 (e.g., the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80). In some examples, the video decoder 30 may not include the intra BC unit 85, and the functions of the intra BC unit 85 may be performed by other components of the prediction processing unit 81 (e.g., the motion compensation unit 82).

[0061] The video data memory 79 may store video data, such as an encoded video bitstream, to be decoded by other components of the video decoder 30. The video data stored in the video data memory 79 may be obtained, for example, from the storage device 32, from a local video source (e.g., a camera), via a wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). The video data memory 79 may include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. The DPB 92 of the video decoder 30 stores reference video data for use by the video decoder 30 (e.g., in intra-frame or inter-frame prediction coding mode) when decoding the video data. The video data memory 79 and the DPB 92 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, the video data memory 79 and the DPB 92 are shown in FIG. Figure 3 9 are depicted as two different components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and the DPB 92 can be provided by the same memory device or a separate memory device. In some examples, the video data memory 79 can be on-chip with other components of the video decoder 30, or off-chip relative to those components.

[0062] During the decoding process, the video decoder 30 receives a coded video bitstream representing video blocks of a coded video frame and associated syntax elements. The video decoder 30 may receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 entropy decodes the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors or intra-frame prediction mode indicators, and other syntax elements to the prediction processing unit 81.

[0063] When a video frame is encoded as an intra-prediction coded (I) frame or for intra-coded prediction blocks in other types of frames, the intra-prediction unit 84 of the prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the intra-prediction mode transmitted by the signal and reference data from a previously decoded block of the current frame.

[0064] When the video frame is encoded as an inter-frame prediction coding (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the video block of the current video frame based on the motion vector and other syntax elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated from a reference frame in one of the reference frame lists. The video decoder 30 can construct the reference frame lists, i.e., List 0 and List 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0065] In some examples, when the video block is encoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block as defined by video encoder 20.

[0066] The motion compensation unit 82 and / or the intra BC unit 85 determine prediction information for the video block of the current video frame by parsing the motion vector and other syntax elements, and then use the prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) used to encode the video block of the video frame, the inter prediction frame type (e.g., B or P), the construction information for one or more of the reference frame lists for the frame, the motion vector for each inter prediction encoded video block of the frame, the inter prediction state for each inter prediction encoded video block of the frame, and other information for decoding the video block in the current video frame.

[0067] Similarly, the intra BC unit 85 may use some of the received syntax elements, such as flags, to determine whether the current video block is predicted using intra BC mode, construction information of which video blocks of the frame are within the reconstruction region and should be stored in the DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding the video block in the current video frame.

[0068] Motion compensation unit 82 may also perform interpolation using interpolation filters as used by video encoder 20 during encoding of the video blocks to calculate interpolated values ​​for sub-integer pixels of reference blocks. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 based on the received syntax elements and use these interpolation filters to produce the prediction blocks.

[0069] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters calculated by video encoder 20 for each video block in the video frame to determine the degree of quantization. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0070] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. A loop filter 91 (e.g., a deblocking filter, an SAO filter, and / or an ALF) may be located between the adder 90 and the DPB 92 to further process the decoded video block. In some examples, the loop filter 91 may be omitted, and the decoded video block may be provided directly to the DPB 92 by the adder 90. The decoded video blocks in a given frame are then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a memory device separate from the DPB 92 may also store the decoded video for later presentation on a display device (e.g., Figure 1 on a display device 34).

[0071] In a typical video encoding process, a video sequence typically includes an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore include only a two-dimensional array of luma samples.

[0072] like Figure 4A As shown in , the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of CTUs. A video frame may include an integer number of CTUs sequentially ordered from left to right and from top to bottom in a raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set so that all CTUs in a video sequence have the same size of one of 128 columns, 648 columns, 328 columns, and 168 columns. However, it should be noted that the present application is not necessarily limited to a specific size. As Figure 4BAs shown in , each CTU may include one CTB for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding samples of the coding tree blocks. The syntax elements describe the properties of different types of units of the coded pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome picture or a picture with three separate color planes, a CTU may include a single coding tree block and syntax elements for encoding samples of the coding tree block. The coding tree block may be an N-coded sample block.

[0073] To achieve better performance, the video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination thereof, on the coding tree block of the CTU and divide the CTU into smaller CUs. Figure 4C As depicted in FIG, a 64×64 CTU 400 is first divided into four smaller CUs, each CU having a block size of 32×2. Among the four smaller CUs, CU 410 and CU 420 are respectively divided into four CUs with a block size of 16. Two CUs 430 and CU 440 with a block size of 16 are further divided into four CUs with a block size of 8. Figure 4D Depicted is a diagram showing Figure 4C The quadtree data structure of the final result of the partitioning process of CTU 400 depicted in FIG. 4 corresponds to a CU of a corresponding size ranging from 32 to 82. Figure 4B In the CTU depicted in FIG, each CU may include a CB of luma samples and two corresponding coding blocks of chroma samples of the same size frame, and syntax elements for encoding the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures for encoding the samples of the coding block. It should be noted that Figure 4C and Figure 4D The quadtree partitioning depicted in FIG is for illustrative purposes only, and one CTU can be split into multiple CUs based on quadtree partitioning / ternary tree partitioning / binary tree partitioning to adapt to varying local characteristics. In a multi-type tree structure, one CTU is partitioned according to a quadtree structure, and each quadtree leaf CU can be further partitioned according to a binary and ternary tree structure. Figure 4E As shown, there are five possible partitioning types for a coding block with width W and height H, namely, quadruple partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning.

[0074] In some embodiments, the video encoder 20 may further partition the coding block of the CU into one or more (M one by one) PBs. A PB is a rectangular (square or non-square) sample block to which the same prediction (inter-frame or intra-frame) is applied. The PU of the CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements for predicting the PBs. In a monochrome picture or a picture with three separate color planes, the PU may include a single PB and a syntax structure for predicting the PB. The video encoder 20 may generate a predicted luma block, a predicted Cb block, and a predicted Cr block for the luma prediction block, the Cb prediction block, and the Cr prediction block of each PU of the CU.

[0075] Video encoder 20 may use intra prediction or inter prediction to generate a prediction block for a PU. If video encoder 20 uses intra prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate a prediction block for a PU, video encoder 20 may generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0076] After the video encoder 20 generates the predicted luma block, the predicted Cb block, and the predicted Cr block for one or more PUs of a CU, the video encoder 20 may generate a luma residual block for the CU by subtracting the predicted luma block of the CU from the original luma coding block of the CU, so that each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the predicted luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, the video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, respectively, so that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0077] In addition, if Figure 4CAs shown in , the video encoder 20 can use quadtree partitioning to decompose the luma residual block, Cb residual block and Cr residual block of the CU into one or more luma transform blocks, Cb transform blocks and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. The TU of the CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU may be associated with a luma transform block, a Cb transform block and a Cr transform block. In some examples, the luma transform block associated with the TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax structure for transforming the samples of the transform block.

[0078] The video encoder 20 may apply one or more transforms to the luma transform block of the TU to generate a luma coefficient block for the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficient may be a scalar. The video encoder 20 may apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 may apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0079] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), the video encoder 20 may quantize the coefficient block. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After the video encoder 20 quantizes the coefficient block, the video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, the video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, the video encoder 20 may output a bitstream including a sequence of bits that form a representation of an encoded frame and associated data, and the bitstream is stored in the storage device 32 or sent to the target device 14.

[0080] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. The video decoder 30 can reconstruct the frame of the video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally inverse to the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coding block of the current CU by adding the samples of the prediction block for the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the coding block for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0081] As described above, video codecs mainly use two modes to achieve video compression, i.e., intra-frame prediction (or intra-frame prediction) and inter-frame prediction (or inter-frame prediction). Palette-based coding is another coding scheme that many video coding standards have adopted. In palette-based coding (which may be particularly suitable for screen-generated content coding), a video codec (e.g., video encoder 20 or video decoder 30) forms a palette table representing the color of the video data of a given block. The palette table includes the most important (e.g., commonly used) pixel values ​​in a given block. Pixel values ​​that are not frequently represented in the video data of a given block are not included in the palette table, or are included in the palette table as escape colors.

[0082] Each entry in the palette table includes an index for the corresponding pixel value in the palette table. The palette index for the sample in the block can be encoded to indicate which entry in the palette table will be used to predict or reconstruct which sample. The palette mode starts with a process of generating a palette predictor for the first block, strip, tile, or other such grouping of video blocks of a picture. As will be explained below, the palette predictor for subsequent video blocks is typically generated by updating the previously used palette predictor. For illustrative purposes, it is assumed that the palette predictor is defined at the picture level. In other words, a picture can include multiple coding blocks, each with its own palette table, but there is one palette predictor for the entire picture.

[0083] In order to reduce the bits required to signal palette entries in a video bitstream, a video decoder can utilize a palette predictor to determine new palette entries in a palette table for reconstructing a video block. For example, the palette predictor can include palette entries from a previously used palette table, or even be initialized with a recently used palette table by including all entries of a recently used palette table. In some implementations, the palette predictor can include fewer entries than all entries in the most recently used palette table, and then merge some entries from other previously used palette tables. The palette predictor can have the same size as a palette table used to encode different blocks, or can be larger or smaller than a palette table used to encode different blocks. In one example, the palette predictor is implemented as a first-in-first-out (FIFO) table including 64 palette entries.

[0084] To generate a palette table for a block of video data from a palette predictor, a video decoder may receive a one-bit flag for each entry of the palette predictor from an encoded video bitstream. The one-bit flag may have a first value (e.g., binary 1) indicating that the associated entry of the palette predictor is to be included in the palette table, or a second value (e.g., binary 0) indicating that the associated entry of the palette predictor is not to be included in the palette table. If the size of the palette predictor is larger than the palette table for the block of video data, the video decoder may stop receiving more flags once a maximum size for the palette table is reached.

[0085] In some embodiments, some entries in the palette table can be directly signaled in the encoded video bitstream instead of being determined using a palette predictor. For such entries, the video decoder can receive three separate m-bit values ​​from the encoded video bitstream, indicating the pixel values ​​for the luminance and two chrominance components associated with the entry, where m represents the bit depth of the video data. Compared to the multiple m-bit values ​​required for palette entries that are directly signaled, those palette entries derived from the palette predictor require only a one-bit flag. Therefore, using a palette predictor to signal some or all palette entries can significantly reduce the number of bits required to signal new palette table entries, thereby improving the overall encoding and decoding efficiency of palette mode encoding and decoding.

[0086] In many cases, the palette predictor for a block is determined based on the palette table used to decode one or more previously encoded blocks. However, when encoding the first coding tree unit in a picture, slice, or tile, the palette table of the previously encoded block may be unavailable. Therefore, the palette predictor cannot be generated using the entries of the previously used palette table. In such a case, a sequence of palette predictor initializers can be signaled in a sequence parameter set (SPS) and / or a picture parameter set (PPS), which is the value used to generate the palette predictor when the previously used palette table is unavailable. SPS generally refers to the syntax structure of syntax elements applicable to a series of consecutive coded video pictures (referred to as a coded video sequence (CVS)) as determined by the contents of the syntax elements found in the PPS, and the PPS is referenced by the syntax elements in each slice segment header. PPS generally refers to the syntax structure of syntax elements applicable to one or more individual pictures within the CVS as determined by the syntax elements in each slice segment header. Therefore, the SPS is generally considered to be a higher-level syntax structure than the PPS, which means that the syntax elements included in the SPS generally change less and apply to a larger portion of the video data than the syntax elements included in the PPS.

[0087] Figures 5A to 5B is a block diagram illustrating an example of applying an adaptive color space conversion (ACT) technique to convert a residual between an RGB color space and a YCgCo color space according to some implementations of the present disclosure.

[0088] In the HEVC screen content codec extension, ACT is applied to adaptively transform the residual from one color space (e.g., RGB) to another color space (e.g., YCgCo) so that the correlation (e.g., redundancy) between the three color components (e.g., R, G, and B) is significantly reduced in the YCgCo color space. In addition, in the existing ACT design, adaptation of different color spaces is performed at the transform unit (TU) level by signaling a flag tu_act_enabled_flag for each TU. When the flag tu_act_enabled_flag is equal to 1, it indicates that the residual of the current TU is encoded in the YCgCo space; otherwise (i.e., the flag is equal to 0), it indicates that the residual of the current TU is encoded in the original color space (i.e., there is no color space conversion). In addition, different color space transformation formulas are applied depending on whether the current TU is encoded in lossless mode or lossy mode. Specifically, in Figure 5AThe forward and inverse color space transformation formulas between the RGB color space and the YCgCo color space for lossy mode are defined in .

[0089] For lossless mode, a reversible version of the RGB-YCgCo transform (also called YCgCo-LS) is used. The reversible version of the RGB-YCgCo transform is based on Figure 5B This is achieved by the lifting operations depicted in and related descriptions.

[0090] like Figure 5A As shown in , the forward and inverse color transform matrices used in lossy mode are not normalized. Therefore, after the color transform is applied, the amplitude of the YCgCo signal is smaller than that of the original signal. In order to compensate for the reduction in amplitude caused by the forward color transform, an adjusted quantization parameter is applied to the residual in the YCgCo domain. Specifically, when the color space transform is applied, the QP values ​​QPY, QPCg and QPCo used to quantize the YCgCo domain residual are set to QP-5, QP-5 and QP-3, respectively, where QP is the quantization parameter used in the original color space.

[0091] Figure 6 is a block diagram of applying a luma mapping with chroma scaling (LMCS) technique in an exemplary video data decoding process according to some embodiments of the present disclosure.

[0092] In VVC, LMCS is used as a new codec tool applied before loop filters (e.g., deblocking filter, SAO, and ALF). Generally, LMCS has two main modules: 1) luma component loop mapping based on an adaptive piecewise linear model; 2) luma-dependent chroma residual scaling. Figure 6 A modified decoding process in which LMCS is applied is shown. Figure 6 In the LMCS, the decoding modules performed in the mapped domain include an entropy decoding module, an inverse quantization module, an inverse transform module, a luma intra prediction module, and a luma sample reconstruction module (i.e., the addition of luma prediction samples and luma residual samples). The decoding modules performed in the original (i.e., non-mapped) domain include a motion compensation prediction module, a chroma intra prediction module, a chroma sample reconstruction module (i.e., the addition of chroma prediction samples and chroma residual samples), and all loop filter modules (such as a deblocking module, an SAO module, and an ALF module). The new operation modules introduced by LMCS include a luma sample forward mapping module 610, a luma sample inverse mapping module 620, and a chroma residual scaling module 630.

[0093] The loop mapping of LMCS can adjust the dynamic range of the input signal to improve the encoding and decoding efficiency. In the existing LMCS design, the loop mapping of the luminance sample is established based on two mapping functions: a forward mapping function FwdMap and a corresponding inverse mapping function InvMap. The forward mapping function is sent from the encoder to the decoder through a signal using a piecewise linear model with 16 equal-sized segments. The inverse mapping function can be derived directly from the forward mapping function and therefore does not need to be sent through a signal.

[0094] The parameters of the luma mapping model are signaled at the slice level. First, a presence flag is signaled to indicate whether a luma mapping model will be signaled for the current slice. If a luma mapping model exists in the current slice, the corresponding piecewise linear model parameters are further signaled. In addition, at the slice level, another LMCS control flag is signaled to enable / disable LMCS for the slice.

[0095] The chroma residual scaling module 630 is designed to compensate for the interaction of quantization precision between the luma signal and its corresponding chroma signal when loop mapping is applied to the luma signal. Whether chroma residual scaling is enabled or disabled for the current slice is also signaled in the slice header. If luma mapping is enabled, an additional flag is signaled to indicate whether luma-dependent chroma residual scaling is applied. When luma mapping is not used, luma-dependent chroma residual scaling is always disabled and no additional flag is required. In addition, chroma residual scaling is always disabled for CUs containing less than or equal to four chroma samples.

[0096] Figure 7 is a block diagram illustrating an exemplary video decoding process according to some embodiments of the present disclosure, by which a video decoder implements an inverse adaptive color space transform (ACT) technique.

[0097] Similar to the ACT design in HEVC SCC, ACT in VVC converts the intra / inter prediction residual of one CU with a 4:4:4 chroma format from the original color space (e.g., RGB color space) to the YCgCo color space. As a result, the redundancy between the three color components can be reduced for better coding efficiency. Figure 7 A decoding flow chart is depicted in which how inverse ACT is applied in the VVC framework by adding an inverse ACT module 710. When processing a CU encoded with ACT enabled, entropy decoding, inverse quantization, and inverse DCT / DST-based transform are first applied to the CU. After this, Figure 7As depicted in , the inverse ACT is called to convert the decoded residual from the YCgCo color space to the original color space (e.g., RGB and YCbCr). In addition, since the ACT in lossy mode is not normalized, a QP adjustment of (-5, -5, -3) is applied to the Y, Cg, and Co components to compensate for the varying amplitude of the transformed residual.

[0098] In some embodiments, the ACT method reuses the same ACT core transform of HEVC to convert colors between different color spaces. Specifically, two different versions of the color transform are applied depending on whether the current CU is coded in a lossy or lossless manner. The forward and inverse color transforms for the lossy case use the irreversible YCgCo transform matrix, such as Figure 5A For the lossless case, the application Figure 5B The reversible color transform YCgCo-LS shown in . In addition, unlike the existing ACT design, the following changes are introduced to the proposed ACT scheme to handle its interaction with other codec tools in the VVC standard.

[0099] For example, since the residual of a CU in HEVC can be divided into multiple TUs, an ACT control flag is signaled separately for each TU to indicate whether color space conversion needs to be applied. Figure 4E As described, a quadtree nested with binary and ternary partition structures is applied in VVC to replace the concept of multiple partition types, thereby removing the separate CU, PU and TU partitions in HEVC. This means that in most cases, a CU leaf node is also used as a unit of prediction and transform processing without further partitioning, unless the maximum supported transform size is less than the width or height of a component of the CU. Based on such a partition structure, it is proposed in the present disclosure to adaptively enable and disable ACT at the CU level. Specifically, a flag cu_act_enabled_flag is sent by signal for each CU to select between the original color space and the YCgCo color space to encode and decode the residual of the CU. If the flag is equal to 1, it indicates that the residuals of all TUs within the CU are encoded and decoded in the YCgCo color space. Otherwise, if the flag cu_act_enabled_flag is equal to 0, all residuals of the CU are encoded and decoded in the original color space.

[0100] In some embodiments, there are different scenarios for disabling ACT. When ACT is enabled for a CU, it needs to access the residuals of all three components for color space conversion. However, the VVC design cannot guarantee that each CU always contains information of the three components. According to an embodiment of the present disclosure, in the case where the CU does not contain information of all three components, ACT should be forcibly disabled.

[0101] First, in some embodiments, when a separate tree partition structure is applied, the luma and chroma samples within a CTU are divided into CUs based on a separate partition structure. As a result, the CU in the luma partition tree contains only the coding information of the luma component, and the CU in the chroma partition tree contains only the coding information of the two chroma components. According to the current VVC, the switching between the single tree and the separate tree partition structure is performed at the slice level. Therefore, according to an embodiment of the present disclosure, when it is found that a separate tree is applied to a slice, ACT will always be disabled for all CUs (including luma CUs and chroma CUs) within the slice without sending an ACT flag inferred to be zero through a signal.

[0102] Second, in some embodiments, when the ISP mode is enabled (further described below), the TU partition is applied only to luma samples, while the chroma samples are encoded without further splitting into multiple TUs. Assuming that N is the number of ISP sub-partitions (i.e., TUs) of an intra-frame CU, according to the current ISP design, only the last TU contains luma and chroma components, while the first N-1 ISPTUs consist only of luma components. According to one embodiment of the present disclosure, ACT is disabled in ISP mode. There are two ways to disable ACT in ISP mode. In the first method, an ACT enable / disable flag (i.e., cu_act_enabled_flag) is signaled before the ISP mode syntax is signaled. In this case, when the flag cu_act_enabled_flag is equal to 1, the ISP mode will not be signaled in the bitstream, but it will always be inferred to be zero (i.e., turned off). In the second method, the signaling of the ACT flag is bypassed using ISP mode signaling. Specifically, in this method, the ISP mode is signaled before the flag cu_act_enabled_flag. When ISP mode is selected, the flag cu_act_enabled_flag is not signaled and is inferred to be 0. Otherwise (ISP mode is not selected), the flag cu_act_enabled_flag is still signaled to adaptively select the color space for residual coding of the CU.

[0103] In some embodiments, in addition to forcibly disabling ACT for CUs with misaligned luma and chroma partition structures, LMCS for CUs to which ACT is applied is also disabled. In one embodiment, when a CU selects the YCgCo color space to encode its residual, both luma mapping and chroma residual scaling are disabled (ie, ACT is 1). In another embodiment, when ACT is enabled for a CU, only chroma residual scaling is disabled, while luma mapping can still be applied to adjust the dynamic range of the output luma sample. In the last embodiment, for a CU to which ACT is applied to encode its residual, both luma mapping and chroma residual scaling are enabled. There may be multiple ways to enable chroma residual scaling for CUs to which ACT is applied. In one method, chroma residual scaling is applied before inverse ACT during decoding. By this method, this means that when ACT is applied, chroma residual scaling is applied to chroma residuals (ie, Cg and Co residuals) in the YCgCo domain. In another method, chroma residual scaling is applied after inverse ACT. Specifically, by the second method, chroma scaling is applied to residuals in the original color space. Assuming the input video is captured in RGB format, this means that the chroma residual scaling is applied to the residuals of the B and R components.

[0104] In some embodiments, a syntax element (e.g., sps_act_enabled_flag) is added to a sequence parameter set (SPS) to indicate whether ACT is enabled at the sequence level. In addition, since color space conversion is applied to video content where luma and chroma components have the same resolution (e.g., 4:4:4 chroma format 4:4:4), a bitstream conformance requirement needs to be added so that ACT is only enabled for 4:4:4 chroma formats. Table 1 illustrates the modified SPS syntax table with the above syntax added.

[0105]

[0106] Table 1 Modified SPS syntax table

[0107] Specifically, sps_act_enabled_flag equal to 1 indicates that ACT is enabled, and sps_act_enabled_flag equal to 0 indicates that ACT is disabled, so that the flag cu_act_enabled_flag is not signaled for CUs that reference SPS but are inferred to be 0. When ChromaArrayType is not equal to 3, a requirement for bitstream conformance is that the value of sps_act_enabled_flag shall be equal to 0.

[0108] In another embodiment, instead of always signaling sps_act_enabed_flag, signaling the flag is conditional on the chroma type of the input signal. Specifically, assuming that ACT can only be applied when the luma and chroma components are at the same resolution, the flag sps_act_enabled_flag is signaled only when the input video is captured in 4:4:4 chroma format. With this change, the modified SPS syntax table is:

[0109]

[0110] Table 2 Modified SPS syntax table with signaling conditions

[0111] In some embodiments, the syntax design specifications for decoding video data using ACT are shown in the following table.

[0112]

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123] Table 3 Signaling ACT mode specifications

[0124] The flag cu_act_enabled_flag equal to 1 indicates that the residual of this coding unit is encoded in the YCgCo color space, and the flag cu_act_enabled_flag equal to 0 indicates that the residual of this coding unit is encoded in the original color space (such as RGB or YCbCr). When the flag cu_act_enabled_flag is not present, it is inferred to be equal to 0.

[0125] In the current VVC working draft, when the input video is captured in a 4:4:4 chroma format, the transform skip mode can be applied to the luminance and chroma components. Based on such a design, in some embodiments, the following three methods are used to handle the interaction between ACT and transform skip.

[0126] In one method, when transform skip mode is enabled for an ACT CU, transform skip mode is applied only to luma components and not to chroma components. In some embodiments, the syntax design specification for this method is described in the following table.

[0127]

[0128]

[0129] Table 4 Syntax specification when transform skip mode is used only for luma component

[0130] In another approach, the transform skip mode is applied to both luma and chroma components. In some embodiments, the syntax design specification for this approach is described in the following table.

[0131]

[0132]

[0133] Table 5 Syntax specification when transform skip mode is used for luma component and chroma component

[0134] In yet another approach, when ACT is enabled for a CU, transform skip mode is always disabled. In some embodiments, the syntax design specification for this approach is described in the following table.

[0135]

[0136]

[0137] Table 6 Syntax specification when transform skip mode is always disabled

[0138] Fig. 8A and 8B 1 is a block diagram illustrating an exemplary video decoding process for implementing an inverse adaptive color space transform (ACT) and a luminance mapping technique with chroma scaling in a video decoder according to some embodiments of the present invention. In some embodiments, using ACT (e.g. Figure 7 Inverse ACT 710 in ) and chroma residual scaling (e.g. Figure 6 In some other embodiments, the video bitstream is encoded using chroma residual scaling instead of ACT and thus inverse ACT 710 is not required.

[0139] More specifically, Fig. 8A An embodiment is shown in which the video decoder performs chroma residual scaling 630 prior to the inverse ACT 710. Thus, the video decoder performs luma mapping in the color space transform domain with chroma residual scaling 630. For example, assuming that the input video is captured in RGB format and converted to the YCgCo color space, the video encoder performs chroma residual scaling 630 on the chroma residuals Cg and Co according to the luma residual Y in the YCgCo color space.

[0140] Figure 8B An alternative embodiment is shown where the video decoder performs chroma residual scaling 630 after the inverse ACT 710. Thus, the video decoder performs luma mapping in the original color space domain with chroma residual scaling 630. For example, assuming the input video is captured in RGB format, the video encoder applies chroma residual scaling to the B and R components.

[0141] Fig. 9 is a block diagram illustrating exemplary decoding logic between performing Adaptive Color Space Transform (ACT) and Block Differential Pulse Code Modulation (BDPCM) according to some embodiments of the present disclosure.

[0142] BDPCM is a coding tool for screen content coding. In some embodiments, the BDPCM enable flag is sent by signal at the sequence level in SPS. The BDPCM enable flag is sent by signal only when the conversion skip mode is enabled in SPS.

[0143] When BDPCM is enabled, it is only executed if the size of the CU is less than or equal to MaxTsSize for luma samples. If the CU is intra-coded, a flag is sent at the CU level, where MaxTsSize is the maximum block size allowed for transform skip mode. The flag indicates whether conventional intra coding or BDPCM is used. If BDPCM is used, another BDPCM prediction direction flag is further sent to indicate whether the prediction is horizontal or vertical. The block is then predicted using a conventional horizontal or vertical intra prediction process with unfiltered reference samples. The residuals are quantized and the difference between each quantized residual and its prediction value, i.e. the previously encoded residual of the adjacent position horizontally or vertically (depending on the BDPCM prediction direction), is encoded.

[0144] For a block of size M(height)×N(width), let r i，j ,0≤i≤M-1,0≤j≤N-1 is the prediction residual. Let Q(r i，j), 0≤i≤M-1,0≤j≤N-1 represents the residual r i，j BDPCM is applied to the quantized residual values, resulting in a quantized version of The modified M×N array in, is predicted from its adjacent quantized residual values. For the vertical BDPCM prediction mode, for 0≤j≤(N-1), the following formula is used to derive

[0145]

[0146] Regarding the horizontal BDPCM prediction mode, for 0≤i≤(M-1), the following formula is used to derive

[0147] At the decoding end, the above process is reversed to convert Q(r i，j ), 0≤i≤M-1, 0≤j≤N-1 is calculated as follows:

[0148] If vertical BDPCM is used, then

[0149]

[0150] If horizontal BDPCM is used, then

[0151]

[0152] The inverse quantized residual Q -1 (Q(r i，j )) is added to the intra-block prediction value to produce the reconstructed sample value.

[0153] These predicted quantized residual values ​​are encoded using the same residual coding process as transform skip mode residual coding Sent to the decoder. For the MPM mode for future intra-mode encoding, the horizontal or vertical prediction mode is stored for the BDPCM-encoded CU if the BDPCM prediction direction is horizontal or vertical, respectively. For deblocking, if both blocks on both sides of the block boundary are encoded using BDPCM, then that particular block boundary will not be deblocked. According to the latest VVC working draft, when the input video is in 4:4:4 chroma format, BDPCM can be applied to luma and chroma components by signaling two separate flags, i.e., intra_bdpcm_luma_flag and intra_bdpcm_chroma_flag for luma and chroma channels at the CU level.

[0154] In some embodiments, the video encoder executes different logic to handle the interaction between ACT and BDPCM. For example, when ACT is applied to an intra-frame CU, BDPCM is enabled only for the luma component but disabled for the chroma component (910). In some embodiments, when ACT is applied to an intra-frame CU, BDPCM is enabled for the luma component, but BDPCM signaling for the chroma component is disabled. When the signaling of chroma BDPCM is bypassed, in one embodiment, the values ​​of intra_bdpcm_chroma_flag and intra_bdpcm_chroma_dir are equal to the values ​​of the luma component, i.e., intra_bdpcm_flag and intra_bdpcm_dir_flag (i.e., the same luma BDPCM direction as for chroma BDPCM is used). In another embodiment, the values ​​of intra_bdpcm_chroma_flag and intrra_bdpcm_chroma_dir_flag are set to zero when they are sent by signal for ACT mode (i.e., the chroma BDPCM mode for chroma components is disabled). The corresponding modified syntax table of the coding unit is as follows:

[0155]

[0156]

[0157]

[0158]

[0159] Table 7 BDPCM enabled only for the luminance component

[0160] In some embodiments, when ACT is applied to an intra CU, BDPCM is enabled for luma and chroma components (920). The corresponding modified syntax table for the coding unit is as follows:

[0161]

[0162]

[0163]

[0164]

[0165] Table 8 BDPCM enabled for both luminance and chrominance components

[0166] In some embodiments, when ACT is applied to an intra CU, BDPCM is disabled for luma and chroma components (930). In this case, there is no need to signal BDPCM-related syntax elements. The corresponding modified syntax table for the coding unit is as follows:

[0167]

[0168]

[0169]

[0170]

[0171] Table 9: BDPCM is disabled for both luminance and chrominance components

[0172] In some embodiments, a constrained chroma BDPCM signaling method is used for ACT mode. Specifically, when ACT is applied, the signaling of the chroma BDPCM enable / disable flag (i.e., intra_bdpcm_chroma_flag) is adjusted when the luma BDPCM (i.e., intra_bdpcm_flag) is present. The flag intra_bdpcm_chroma_flag is signaled only when the flag intra_bdpcm_flag is equal to 1 (i.e., luma BDPCM mode is enabled). Otherwise, the flag intra_bdpcm_chroma_flag is inferred to be zero (i.e., chroma BDPCM is disabled). When the flag intra_bdpcm_chroma_flag is equal to 1 (i.e., chroma BDPCM is enabled), the BDPCM direction for the chroma component is always set to be equal to the luma BDPCM direction, i.e., the value of the flag intra_bdpcm_chroma_dir_flag is always set to be equal to the value of intra_bdpcm_dir_flag. The corresponding modified syntax table for this coding unit is as follows:

[0173]

[0174]

[0175]

[0176]

[0177] Table 10 Adjustments to the signaling of the chroma BDPCM enable / disable flag when luma BDPCM is present

[0178] In some embodiments, instead of adjusting the presence of the flag intra_bdpcm_chroma_flag based on the value of intra_bdpcm_flag, the signaling of the chroma BDPCM mode is conditionally enabled when the intra prediction mode of the luma component is horizontal or vertical when ACT is applied. Specifically, with this approach, the flag intra_bdpcm_chroma_flag is signaled only when the luma intra prediction direction is purely horizontal or vertical. Otherwise, the flag intra_bdpcm_chroma_flag is inferred to be 0 (which means that chroma BDPCM is disabled). When the flag intra_bdpcm_chroma_flag is equal to 1 (i.e., chroma BDPCM is enabled), the BDPCM direction applied to these chroma components is always set equal to the luma intra prediction direction. In the table below, the numbers 18 and 50 represent the current intra prediction indices for horizontal and vertical intra predictions in the current VVC draft. The corresponding modified syntax table for this coding unit is as follows:

[0179]

[0180]

[0181]

[0182]

[0183] Table 11 Adjustments to the signaling of chroma BDPCM enable / disable when the intra prediction mode of the luma component is horizontal or vertical

[0184] In some embodiments, context modeling for luma / chroma BDPCM mode is implemented. In the current BDPCM design of VVC, BDPCM signaling for luma and chroma components reuses the same context modeling. In detail, a single context is shared by the luma BDPCM enable / disable flag (i.e., intra_bdpcm_flag) and the chroma BDPCM enable / disable flag (i.e., intra_bdpcm_chroma_flag), and another single context is shared by the luma BDPCM direction flag (i.e., intra_bdpcm_dir_flag) and the chroma BDPCM direction flag (i.e., intra_bdpcm_chroma_dir_flag).

[0185] In some embodiments, to improve codec efficiency, in one method, separate contexts are used to signal BDPCM enable / disable for luma and chroma components. In another embodiment, separate contexts for signaling BDPCM direction flags are used for these luma and chroma components. In yet another embodiment, two additional contexts are used to encode and decode the chroma BDPCM enable / disable flag, wherein the first context is used to signal intra_bdpcm_chroma_flag when luma BDPCM mode is enabled, and the second context is used to signal intra_bdpcm_chroma_flag mode when luma BDPCM mode is disabled.

[0186] In some embodiments, lossless codec is used to process ACT. In the HEVC standard, the lossless mode of a CU is indicated by sending a CU-level flag cu_transquant_bypass_flag as 1 through a signal. However, in the ongoing VVC standardization process, a different lossless enabling method is applied. Specifically, when a CU is encoded in a lossless manner, it is only necessary to skip the transform and use a quantization step of 1. This can be achieved by sending the QP value at the CU level as 1 through a signal and sending the transform_skip_flag at the TU level as 1. Therefore, in one embodiment of the present invention, a CU / TU is switched between lossy ACT transform and lossless ACT transform according to the value of transform_skip_flag and the value of QP. When the flag transform_skip_flag is equal to 1 and the QP value is equal to 4, the lossless ACT transform is applied; otherwise, the lossy version of the ACT transform is applied, as shown below.

[0187] If transform_skip_flag is equal to 1 and QP is equal to 4, the residual sample r with x=0..nTbW-1, y=0..nTbH-1 is Y 、r Cb and r Cr The (nTbW)x(nTbH) array is modified as follows:

[0188] tmp=r Y [x][y]–(r Cb [x][y]>>1)

[0189] r Y [x][y]=tmp+r Cb [x][y]

[0190] r Cb [x][y]=tmp–(r Cr[x][y]>>1)

[0191] r Cr [x][y]=r Cb [x][y]+r Cr [x][y].

[0192] Otherwise, the residual sample r with x = 0..nTbW-1, y = 0..nTbH–1 is Y 、r Cb and r Cr The (nTbW)x(nTbH) array is modified as follows:

[0193] tmp=r Y [x][y]-r Cb [x][y]

[0194] r Y [x][y]=r Y [x][y]+r Cb [x][y]

[0195] r Cb [x][y]=tmp–r Cr [x][y]

[0196] r Cr [x][y]=tmp+r Cr [x][y].

[0197] In the above description, different ACT transform matrices are used for lossy and lossless coding. In order to achieve a more unified design, the lossless ACT transform matrix is ​​used for lossy and lossless coding. In addition, considering that the lossless ACT transform increases the dynamic range of the Cg and Co components by 1 bit, an additional 1-bit right shift is applied to the Cg and Co components after the forward ACT transform, while a 1-bit left shift is applied to the Cg and Co components before the inverse ACT transform, as described below.

[0198] If transform_skip_flag is equal to 0 or QP is not equal to 4, the residual sample r with x=0..nTbW-1, y=0..nTbH-1 is Cb and r Cr The (nTbW)x(nTbH) array is modified as follows:

[0199] r Cb [x][y]=r Cb [x][y]<<1

[0200] r Cr [x][y]=r Cr [x][y]<<1.

[0201] The residual sample r of x = 0..nTbW-1, y = 0..nTbH–1 Y 、r Cb and r Cr The (nTbW)x(nTbH) array is modified as follows:

[0202] tmp=r Y [x][y]–(r Cb [x][y]>>1)

[0203] r Y [x][y]=tmp+r Cb [x][y]

[0204] r Cb [x][y]=tmp–(r Cr [x][y]>>1)

[0205] r Cr [x][y]=r Cb [x][y]+r Cr [x][y].

[0206] Furthermore, as can be seen above, when ACT is applied, QP offsets (-5, -5, -3) are applied to the Y, Cg, and Co components. Therefore, for small input QP values ​​(e.g. <5), negative QPs will be used for quantization / dequantization of the ACT transform coefficients, which are undefined. To address this, a clipping operation is added after the QP adjustment of ACT so that these QP values ​​applied are always equal to or greater than zero, i.e. QP' = max(QP org -QP offset ,0), where QP is the original QP, QP offset is the ACT QP offset and QP' is the adjusted QP value.

[0207] In the method described above, although the same ACT transform matrix (ie, lossless ACT transform matrix) is used for lossy and lossless encoding and decoding, the following two problems can still be identified:

[0208] Different inverse ACT operations are still applied depending on whether the current CU is a lossy CU or a lossless CU. Specifically, for a lossless CU, an inverse ACT transform is applied; for a lossy CU, an additional right shift needs to be applied before the inverse ACT transform. In addition, the decoder needs to know whether the current CU is encoded in lossy mode or lossless mode. This is inconsistent with the current VVC lossless design. In detail, unlike the lossless design in HEVC where the lossless mode of a CU is signaled by a cu_transquant_bypass_flag, the lossless encoding and decoding in VVC is done in a purely non-normative way, that is, skipping the transform of the prediction residual (enabling the transform skip mode for luminance and chrominance components), selecting an appropriate QP value (i.e. 4) and explicitly disabling coding tools that prevent lossless coding, such as loop filters.

[0209] The QP offset for the standardized ACT transform is currently fixed. However, in terms of codec efficiency, the choice of the best QP offset may depend on the content itself. Therefore, it may be more beneficial to allow flexible QP offset signaling when enabling the ACT tool to maximize its codec gain.

[0210] Based on the above considerations, a unified ACT design is implemented as follows. First, lossless ACT forward transform and inverse transform are applied to CUs encoded in lossy mode and lossless mode. Second, instead of using a fixed QP offset, the QP offset applied to the ACT CU in the bitstream is explicitly signaled (i.e., three QP offsets applied to the Y, Cg, and Co components). Third, in order to prevent possible overflow problems of the QP applied to the ACT CU, a trimming operation is applied to the generated QP of each ACT CU to make the QP range valid. It can be seen that based on the above method, the choice between lossy codec and lossless codec can be achieved by pure encoder modification (i.e., using different encoder settings). The decoding operations for lossy and lossless encoding of the ACT CU are the same. Specifically, in order to enable lossless coding, in addition to the existing encoder-side lossless configuration, the encoder only needs to signal the values ​​of the three QP offsets as zero. On the other hand, in order to enable lossy coding, the encoder can signal a non-zero QP offset. For example, in one embodiment, to compensate for the dynamic range change caused by the lossless ACT transform in the lossy codec, QP offsets (-5, 1, 3) for the Y, Cg, and Co components can be signaled when applying ACT. On the other hand, these ACT QP offsets can be signaled at different codec levels (e.g., sequence parameter set (SPS), picture parameter set (PPS), picture header, coding block group level, etc.), which can provide different QP adaptations at different granularities. The following table gives an example of QP offset signaling implemented in SPS.

[0211]

[0212] Table 12 Syntax specification for QP offset signaling implemented in SPS

[0213] In another embodiment, an advanced control flag (such as picture_header_act_qp_offset_present_flag) is added at the SPS or PPS. When the flag is equal to 0, it means that the QP offset signaled in the SPS or PPS will be applied to all CUs encoded in ACT mode. Otherwise, when the flag is equal to 1, additional QP offset syntax (such as picture_header_y_qp_offset_plus5, picture_header_cg_qp_offset_minus1, and picture_header_co_qp_offset_minus3) can be further signaled in the picture header to control the QP value applied to the ACT CU in a specific picture respectively.

[0214] On the other hand, these QP offsets sent by signal should also be applied to clip the final ACT QP value to a valid dynamic range. In addition, different clipping ranges can be applied to CUs coded with and without transform. For example, when no transform is applied, the final QP should not be less than 4. Assuming that these ACT QP offsets are signaled at the SPS level, the corresponding QP value derivation process for the ACT CU can be described as follows:

[0215] QpY=((qPY_PRED+CuQpDeltaVal+64+2*QpBdOffset+sps_act_y_qp_offset)%(64+QpBdOffset))-QpBdOffset

[0216] Qp′ Cb =Clip3(-QpBdOffset,63,qP Cb +pps_cb_qp_offset+slice_cb_qp_offset+CuQpOffset Cb +sps_act_cg_offset)+QpBdOffset

[0217] Qp′ Cr =Clip3(-QpBdOffset,63,qP Cr +pps_cr_qp_offset+slice_cr_qp_offset+CuQpOffset Cr+sps_act_co_offset)+QpBdOffset

[0218] Qp′ CbCr =Clip3(-QpBdOffset,63,qP CbCr +pps_joint_cbcr_qp_offset+slice_joint_cbcr_qp_offset+CuQpOffset CbCr +sps_act_cg_offset)+QpBdOffset.

[0219] In another embodiment, the ACT enable / disable flag is signaled at the SPS level, while the ACT QP offset is signaled at the PPS level to allow the encoder to more flexibly adjust the QP offset applied to the ACT CU to improve encoding and decoding efficiency. Specifically, the SPS and PPS syntax tables with the changes are shown in the following table.

[0220]

[0221]

[0222]

[0223] Table 13 Syntax specification for signaling the ACT enable / disable flag at the SPS level and the ACT QP offset at the PPS level

[0224] pps_act_qp_offset_present_flag equal to 1 specifies that pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are present in the bitstream. When pps_act_qp_offset_present_flag is equal to 0, the syntax elements pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are not present in the bitstream. When sps_act_enabled_flag is equal to 0, the value of pps_act_qp_offset_present_flag shall be 0 for bitstream conformance.

[0225] pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are used to determine the offset applied to the value of the quantization parameter for the luma and chroma components of coding blocks with cu_act_enabled_flag equal to 1. When pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_cr_qp_offset_minus3 are not present, their values ​​are inferred to be equal to 0.

[0226] In the above PPS signaling, the same QP offset value is applied to the ACT CU when the joint coding of chroma residuals (JCCR) mode is applied or not. In view of the fact that only the residual of one signal chroma component is encoded in the JCCR mode, such a design may not be optimal. Therefore, in order to obtain a better coding gain, when the JCCR mode is applied to an ACT CU, a different QP offset can be applied to encode the residual of the chroma component. Based on this consideration, a separate QP offset signaling is added to the JCCR mode in the PPS, as follows.

[0227]

[0228]

[0229] Table 14 Adding separate QP offset signaling to the syntax specification of the PPS for JCCR mode

[0230] pps_joint_cbcr_qp_offset is used to determine the offset applied to the quantization parameter value for the chroma residual of a coding block to which joint chroma residual coding is applied. When pps_joint_cbcr_qp_offset is not present, its value is inferred to be zero.

[0231] In some embodiments, Fig.10 A method of processing ACT when the internal luma and chroma bit depths are different is shown. Specifically, Fig.10 is a decoding flow chart for applying different QP offsets for different components when luma and chroma internal bit depths are different according to some embodiments of the present disclosure.

[0232] According to the existing VVC specification, the brightness and chrominance components are allowed to use different internal bit depths (expressed as BitDepth Yand BitDepth C ) for encoding. However, existing ACT designs always assume that the internal luma and chroma bit depths are the same. In this section, some methods are implemented to Y Not equal to BitDepth C Improve ACT design while

[0233] In the first method, the ACT tool is always disabled when the internal luma bit depth is not equal to the bit depth of the chroma components.

[0234] In the second approach, the bit depth alignment of luma and chroma components is achieved by left-shifting the component with smaller bit depth to match the bit depth of the other component. The scaled components are then rescaled to the original bit depth by right-shifting after color transformation.

[0235] Similar to HEVC, the quantization step size increases by approximately 2 with each increment of QP 1 / 6 times, and doubles exactly every 6 increments. Based on this design, in the second method, in order to compensate for the internal bit depth between luma and chroma, the QP value for the component with a smaller internal bit depth is increased by 6Δ, where Δ is the difference between the luma and chroma internal bit depths. Then, the residual of this component is moved back to the original dynamic range by applying a right shift of Δ bits. Fig.10 The corresponding decoding process when the above method is applied is shown. For example, assuming that the input QP value is qp, the default QP values ​​applied to the Y, Cg, and Co components are equal to qp-5, qp-5, and qp-3. In addition, assuming that the luma internal bit depth is higher than the chroma bit depth, that is, Δ = BitDepth Y –BitDepth C The final QP values ​​applied to the luma and chroma components are then equal to qp-5, qp-5+6Δ, and qp-3+6Δ.

[0236] In some embodiments, encoder acceleration logic is implemented. In order to select the color space for residual codec of a CU, the most direct approach is to have the encoder check each coding mode (e.g., intra-frame coding mode, inter-frame coding mode, and IBC mode) twice, one time with ACT enabled and the other with ACT disabled. This may increase the encoding complexity by about one-fold. In order to further reduce the encoding complexity of ACT, the following encoder acceleration logic is implemented in the present disclosure:

[0237] First, since the YCgCo space is more compact than the RGB space, when the input video is in RGB format, the implementation first checks the rate-distortion (RD) cost with the ACT tool enabled, and then checks the RD cost with the ACT tool disabled. In addition, the calculation of the RD cost with the color space transform disabled is performed only if there is at least one non-zero coefficient when ACT is enabled. Alternatively, when the input video is in YCbCr format, the implementation checks the RD cost with ACT disabled after the RD check with ACT enabled. The second RD check (i.e., with ACT enabled) is performed only if there is at least one non-zero coefficient when ACT is disabled.

[0238] Second, in order to reduce the number of tested codec modes, the same codec mode is used for both color spaces. More specifically, for intra mode, the selected intra prediction mode for full RD cost comparison is shared between the two color spaces; for inter mode, the selected motion vector, reference picture, motion vector predictor and merge index (for inter merge mode) are shared between the two color spaces; for IBC mode, the selected block vector, block vector predictor and merge index (for IBC merge mode) are shared between the two color spaces.

[0239] Third, due to the quadtree / binarytree / ternarytree partition structure used in VVC, the same block partition can be obtained through different partition combinations. In order to speed up the selection of color space, the ACT enable / disable decision is used when the same block is realized through different partition paths. Specifically, when the CU is encoded and decoded for the first time, the selected color space used to encode and decode the residual of a specific CU is stored. Then, when the same CU is obtained through another partition path, it does not choose between the two spaces, but directly reuses the stored color space decision.

[0240] Fourth, considering the strong correlation between a CU and its spatial neighbors, the color space selection information of its spatial neighboring blocks is used to decide how many color spaces need to be checked for residual encoding and decoding of the current CU. For example, if a sufficient number of spatial neighboring blocks choose the YCgCo space to encode and decode their residuals, it can be reasonably inferred that the current CU is likely to choose the same color space. Accordingly, the RD check for encoding and decoding the residual of the current CU in the original color space can be skipped. If enough spatial neighbors choose the original color space, the RD check for residual encoding and decoding in the YCgCo domain can be bypassed. Otherwise, both color spaces need to be tested.

[0241] Fifth, assuming that there is a strong correlation between CUs in the same area, a CU can choose the same color space as its parent CU to encode and decode its residual. Or the child CU can derive the color space from the information of its parent CU, such as the selected color space and the RD cost of each color space. Therefore, in order to simplify the coding complexity, if the residual of its parent CU is encoded in the YCgCo domain, the RD cost of residual encoding and decoding in the RGB domain will be skipped for a CU; in addition, if the residual of its parent CU is encoded in the RGB domain, the RD cost of residual encoding and decoding in the YCgCo domain will be skipped. Another conservative approach is to use the RD cost of its parent CU in both color spaces if two color spaces are tested in the encoding of its parent CU. If its parent CU selects the YCgCo color space and the RD cost of YCgCo is much smaller than that of RGB, the RGB color space will be skipped, and vice versa.

[0242] In some embodiments, 4:4:4 video codec efficiency is improved by enabling luma-only coding tools for chroma components. Since the main focus of VVC design is for video captured in 4:2:0 chroma format, most existing inter / intra codec tools are enabled only for luma components and disabled for chroma components. However, as mentioned earlier, video signals in 4:4:4 chroma format exhibit distinct characteristics compared to 4:2:0 video signals. For example, similar to the luma component, the Cb / B and Cr / R components of 4:4:4YCbCr / RGB video typically contain useful high-frequency texture and edge information. This is different from the chroma components in 4:2:0 video, which are typically very smooth and contain much less information than the luma components. Based on such analysis, the following method is implemented to extend some of the current luma-only codec tools in VVC to chroma components when the input video is in 4:4:4 chroma format.

[0243] First, enable luma interpolation filters for chroma components. Like HEVC, the VVC standard uses motion compensated prediction techniques to exploit redundancy between temporally adjacent pictures, which supports motion vectors accurate to 16 pixels for the Y component and 32 pixels for the Cb and Cr components. Fractional samples are interpolated using a set of separable 8-tap filters. The fractional interpolation of the Cb and Cr components is basically the same as that of the Y component, except that a separable 4-tap filter is used in the case of the 4:2:0 video format. This is because for 4:2:0 video, the Cb and Cr components contain much less information than the Y component, and the 4-tap interpolation filter can reduce the complexity of the fractional interpolation filtering compared to using an 8-tap interpolation filter without affecting the efficiency of motion compensated prediction of the Cb and Cr components.

[0244] As previously noted, existing 4-tap chroma interpolation filters may not be effective for motion compensated prediction interpolation of fractional samples of chroma components in 4:4:4 video. Therefore, in one embodiment of the present disclosure, the same set of 8-tap interpolation filters (used for luma components in 4:2:0 video) is used for fractional sample interpolation of luma and chroma components in 4:4:4 video images. In another embodiment, in order to achieve a better trade-off between coding efficiency and complexity, adaptive interpolation filter selection for chroma samples in 4:4:4 video is enabled. For example, an interpolation filter selection flag may be signaled at the SPS, PPS, and / or slice level to indicate whether an 8-tap interpolation filter (or other interpolation filter) or a default 4-tap interpolation filter is used for chroma components at various coding levels.

[0245] Second, PDPC and MRL are enabled for chroma components.

[0246] The position-dependent intra prediction combination (PDPC) tool in VVC extends the above concept by using a weighted combination of intra prediction samples and unfiltered reference samples. In the current VVC working draft, PDPC is enabled for the following intra modes without signaling: plane, DC, horizontal (i.e., mode 18), vertical (i.e., mode 50), angular direction close to the lower left diagonal direction (i.e., modes 2, 3, 4, ..., 10), and angular direction close to the upper right diagonal direction (i.e., modes 58, 59, 60, ..., 66). Assuming that the predicted sample positioned at coordinates (x, y) is pred(x, y), its corresponding value after PDPC is calculated as:

[0247] pred(x,y)=(wL×R-1,y+wT×Rx,-1–wTL×R-1,-1+(64–wL–wT+wTL)×pred(x,y)+32)>>6;

[0248] Where Rx,-1, R-1,y represent the reference samples located at the top and left of the current sample (x, y), respectively, and R-1,-1 represents the reference sample located at the top left corner of the current block. The weights wL, wT, and wTL in Equation 5 are adaptively selected according to the prediction mode and sample position, as described below, where it is assumed that the size of the current coding block is W×H

[0249] For DC mode,

[0250] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift), wTL=(wL>>4)+(wT>>4).

[0251] For planar mode,

[0252] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift), wTL=0.

[0253] For horizontal mode:

[0254] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift), wTL=wT.

[0255] For vertical mode:

[0256] wT=32>>((y<<1)>>shift), wL=32>>((x<<1)>>shift), wTL=wL.

[0257] For the lower left diagonal direction:

[0258] wT=16>>((y<<1)>>shift), wL=16>>((x<<1)>>shift), wTL=0.

[0259] For the upper right diagonal direction:

[0260] wT=16>>((y<<1)>>shift), wL=16>>((x<<1)>>shift), wTL=0;

[0261] Where shift = (log2(W)–2+log2(H)–2+2)>>2.

[0262] Unlike HEVC, which only uses the nearest row / column of the reconstructed sample as a reference, multiple reference lines (MRL) are introduced in VVC, where two additional rows / columns are used for intra prediction. The index of the selected reference row / column is signaled from the encoder to the decoder. When selecting non-nearest rows / columns, planar and DC modes are excluded from the set of intra modes that can be used to predict the current block.

[0263] In the current VVC design, the PDPC tool is only used by the luma component to reduce / remove discontinuities between intra-prediction samples and their reference samples derived from reconstructed neighboring samples. However, as mentioned above, there may be rich texture information in the chroma blocks in a video signal in a 4:4:4 chroma format. Therefore, tools like PDPC that use a weighted average of unfiltered reference samples and intra-prediction samples to improve prediction quality should also be beneficial for improving the chroma coding efficiency of 4:4:4 video. Based on such considerations, in one embodiment of the present disclosure, a PDPC process is implemented to enable intra-prediction of chroma components in 4:4:4 video.

[0264] The same considerations can also be extended to the MRL tool. In the current VVC, MRL cannot be applied to chroma components. Based on an embodiment of the present invention, MRL for chroma components of 4:4:4 video is enabled by signaling an MRL index for chroma components of an intra CU. Different methods can be used based on this embodiment. In one method, an additional MRL index can be signaled and shared by the Cb / B and Cr / R components. In another method, two MRL indexes are signaled, one for each chroma component. In a third method, the luma MRL index is reused for intra prediction of the chroma components, so that no additional MRL signaling is required to enable MRL for the chroma components.

[0265] Third, ISP is enabled for the chroma components.

[0266] In some embodiments, a coding tool called sub-partition prediction (ISP) is introduced into VVC to further improve the efficiency of intra-frame coding. The traditional intra-frame mode only uses the reconstructed samples adjacent to a CU to generate the intra-frame prediction samples of the block. Based on such a design, the spatial correlation between the prediction samples and the reference samples is roughly proportional to the distance between them. Therefore, the prediction quality of the internal samples (especially the samples located in the lower right corner of the block) is usually worse than the prediction quality of the samples close to the block boundary. Depending on the block size, ISP divides the current CU into 2 or 4 sub-blocks in the horizontal or vertical direction, each sub-block containing at least 16 samples. The reconstructed samples in a sub-block can be used as a reference for predicting the samples in the next sub-block. Repeat the above process until all sub-blocks in the current CU are encoded. In addition, in order to reduce signaling overhead, all sub-blocks in an ISPCU share the same intra-frame mode. In addition, according to the existing ISP design, sub-block division is only applicable to the luminance component. Specifically, only the luminance samples of an ISPCU can be further split into multiple sub-blocks (or TUs), and each luminance sub-block is encoded separately. However, the chroma samples of the ISP CU are not segmented. In other words, for the chroma component, the CU is used as a processing unit for intra prediction, transform, quantization, and entropy coding without further partitioning.

[0267] In the current VVC, when the ISP mode is enabled, TU partitioning is only applied to luma samples, while chroma samples are encoded without further splitting into multiple TUs. According to an embodiment of the present disclosure, due to the rich texture information in the chroma plane, the ISP mode is also enabled for chroma encoding in 4:4:4 video. Different methods can be used based on this embodiment. In one method, an additional ISP index is sent through a signal and shared by two chroma components. In another method, two additional ISP indexes are sent separately through a signal, one for Cb / B and the other for Cr / R. In yet another method, the ISP index used for the luma component is reused for ISP prediction of the two chroma components.

[0268] Fourth, matrix based intraprediction (MIP) is enabled as a new intra prediction technique for these chrominance components.

[0269] To predict samples of a rectangular block of width W and height H, MIP takes as input a row of H reconstructed neighboring boundary samples on the left side of the block and a row of W reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, they are generated as in conventional intra prediction.

[0270] In some embodiments, the MIP mode is enabled only for the luma component. For the same reason as the ISP mode is enabled for the chroma components, in one embodiment, the implementation enables MIP for the chroma components of the 444 videos. Two signaling methods can be applied. In the first method, the implementation separately signals two MIP modes, one for the luma component and the other for the two chroma components. In the second method, the implementation only signals a single MIP mode shared by the luma and chroma components.

[0271] Fifth, multiple transform selection (MTS) is enabled for chroma components.

[0272] In addition to DCT-II already adopted in HEVC, the MTS scheme is used for residual coding of inter-frame and intra-frame coded blocks. It uses multiple selected transforms from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII.

[0273] In the current VVC, the MTS tool is enabled only for the luma component. In one embodiment of the present disclosure, MIP is enabled for the chroma components of 444 videos. Two signaling methods can be applied. In the first method, when MTS is enabled for a CU, two transform indexes are sent by signal, one for the luma component and the other for the two chroma components. In the second method, one transform index is sent by signal when MTS is enabled, and the transform index is shared by the luma and chroma components.

[0274] In some embodiments, unlike the HEVC standard that uses a fixed lookup table to derive the quantization parameter (QP) used from the chroma component based on the luma QP, in the VVC standard, a luma-to-chroma mapping table is sent from the encoder to the decoder, which is defined by several pivot points of a piecewise linear function. Specifically, the syntax elements and reconstruction process of the luma-to-chroma mapping table are described as follows:

[0275]

[0276] Table 15 Syntax elements and reconstruction process of luma to chroma mapping table

[0277] same_qp_table_for_chroma equal to 1 specifies that only one chroma QP map table is signaled, which applies to both Cb and Cr residuals, and also to the joint Cb-Cr residual when sps_joint_cbcr_enabled_flag is equal to 1. same_qp_table_for_chroma equal to 0 specifies that multiple chroma QP maps are signaled in the SPS, two for Cb and Cr, and an additional one for the joint Cb-Cr when sps_joint_cbcr_enabled_flag is equal to 1. When same_qp_table_for_chroma is not present in the bitstream, the value of same_qp_table_for_chroma is inferred to be equal to 1.

[0278] qp_table_start_minus26[i] plus 26 specifies the starting luma and chroma QPs used to describe the i-th chroma QP map. The value of qp_table_start_minus26[i] shall be in the range of -26 - QpBdOffset to 36 (inclusive). When qp_table_start_minus26[i] is not present in the bitstream, the value of qp_table_start_minus26[i] is inferred to be equal to 0.

[0279] num_points_in_qp_table_minus1[i]plus 1 specifies the number of points used to describe the i-th chroma QP map table. The value of num_points_in_qp_table_minus1[i] shall be in the range of 0 to 63+QpBdOffset, inclusive. When num_points_in_qp_table_minus1[0] is not present in the bitstream, the value of num_points_in_qp_table_minus1[0] is inferred to be equal to 0.

[0280] delta_qp_in_val_minus1[i][j] specifies the delta value used to derive the input coordinates of the jth pivot point of the i-th chroma QP map. When delta_qp_in_val_minus1[0][j] is not present in the bitstream, the value of delta_qp_in_val_minus1[0][j] is inferred to be equal to 0.

[0281] delta_qp_diff_val[i][j] specifies the delta value used to derive the output coordinates of the j-th pivot point of the i-th chroma QP map.

[0282] The i-th chroma QP mapping table ChromaQpTable[i] with i=0..numQpTables-1 is derived as follows:

[0283]

[0284]

[0285] In some embodiments, disclosed herein is an improved luma-to-chroma mapping function for RGB video.

[0286] In some embodiments, when the input video is in RGB format, a two-part linear function is transmitted from the encoder to the decoder for mapping luma QP to chroma QP. This is accomplished by setting the syntax elements same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=0. Specifically, the corresponding luma-to-chroma QP mapping function is defined as:

[0287]

[0288] Assuming the intra coding bit depth is 10 bits, Table 16 shows the luma to chroma QP mapping function applied to RGB encoding.

[0289]

[0290] Table 16 Luma to Chroma QP Mapping Function Applied to RGB Coding

[0291] As shown in equation (5), when the luma QP is greater than 26, unequal QP values ​​are used to encode the luma and chroma components. In some embodiments, these unequal QP values ​​have an impact not only on the quantization / dequantization process, but also on the decisions made during rate-distortion (RD) optimization, assuming that a weighted chroma distortion is used when calculating the RD cost for mode decision, which is specified in the following equation.

[0292] J mode =(SSE luma +w chroma ·SSE chroma )+λ mode ·R mode (6);

[0293] Among them, SSE luma and SSE chroma are the distortion of brightness and chrominance components respectively; R mode is the number of bits; mode is the Lagrange multiplier; w chroma is the weighting parameter for chroma distortion, which is calculated as:

[0294]

[0295] However, there is a stronger correlation between the three channels of RGB video than YCbCr / YUV video. Therefore, when video content is captured in RGB format, strong texture and high-frequency information are usually present in all three components, that is, the information in R, G and B is equally important. Therefore, in one embodiment of the present disclosure, equal QP values ​​are applied to all three channels of RGB encoding. This can be accomplished by setting the corresponding luminance to chrominance QP mapping syntax elements to same_qp_table_for_chroma=1, qp_table_start_minus26[0]=0, num_points_in_qp_table_minus1[0]=0, delta_qp_in_val_minus1[0][0]=0 and delta_qp_diff_val[0][0]=1. Accordingly, using the method disclosed herein, the RGB luminance to chrominance mapping function is shown in Equation (8) and Table 17.

[0296] QP c =QP L (8).

[0297] Brightness QP -12 -11 -10 ....... 25 26 27 28 29 ....... 60 61 62 63 Chroma QP -12 -11 -10 ....... 25 26 27 28 29 .... 60 61 62 63

[0298] Table 17 Luma to Chroma QP Mapping Table for RGB Coding

[0299] Fig.11 1100 is a flow chart illustrating an exemplary process by which a video decoder (e.g., video decoder 30) according to some embodiments of the present disclosure decodes video data by conditionally implementing block differential pulse code modulation (BDPCM) for luma and chroma components based on the interaction between an inverse adaptive color space transform (ACT) and BDPCM.

[0300] The video decoder 30 receives video data corresponding to a coding unit from a bitstream, wherein the coding unit is encoded in an intra-frame prediction mode (1110). Then, the video decoder 30 receives a first syntax element from the video data, wherein the first syntax element indicates whether adaptive color space transform (ACT) is enabled for the coding unit (1120).

[0301] Based on a determination that the ACT is enabled for the coding unit, the video decoder 30 receives from the video data a first set of syntax elements associated with block differential pulse code modulation (BDPCM) for a luminance component of the coding unit, wherein the first set of syntax elements includes a second syntax element (1130) indicating whether the BDPCM is enabled for the luminance component of the coding unit.

[0302] Based on a determination that the BDPCM is enabled for the luma component of the coding unit, the video decoder 30 assigns corresponding values ​​of the first set of syntax elements associated with the BDPCM for the luma component of the coding unit to a second set of syntax elements associated with the BDPCM for the chroma components of the coding unit (1140).

[0303] Video decoder 30 decodes the coding unit from the video data based on the first syntax element associated with the ACT and the first set and the second set of syntax elements associated with the BDPCM (1150).

[0304] In some embodiments, the first set of syntax elements includes a third syntax element indicating a BDPCM direction for the luma component of the coding unit.

[0305] In some embodiments, the second set of syntax elements includes a fourth syntax element, the fourth syntax element indicating whether the BDPCM is enabled for the chroma component of the coding unit, and a value of the fourth syntax element is equal to the value of the allocated second syntax element.

[0306] In some embodiments, the second set of syntax elements includes a fifth syntax element, the fifth syntax element indicates a BDPCM direction for the chrominance component of the coding unit, and a value of the fifth syntax element is equal to a value of the allocated third syntax element.

[0307] In some embodiments, the second syntax element indicating whether the BDPCM is enabled for the luma component of the coding unit is intra_bdpcm_flag, and the third syntax element indicating the BDPCM direction for the luma component of the coding unit is intra_bdpcm_dir_flag.

[0308] In some embodiments, a fourth syntax element indicating whether the BDPCM is enabled for the chroma component of the coding unit is intra_bdpcm_chroma_flag.

[0309] In some embodiments, the fifth syntax element indicating the BDPCM direction for the chroma component of the coding unit is intra_bdpcm_chroma_dir.

[0310] In some embodiments, the second set of syntax elements associated with BDPCM for the chroma component of the coding unit is not received from the bitstream prior to the decoding.

[0311] In some other embodiments, the video decoder 30 receives video data corresponding to a coding unit from a bitstream. The coding unit is encoded in an intra-frame prediction mode. The video decoder 30 receives a first syntax element from the video data, the first syntax element indicating whether adaptive color space transform (ACT) is enabled for the coding unit. Based on a determination that the ACT is enabled for the coding unit, the video decoder 30 receives a second syntax element from the video data, the second syntax element indicating a direction of a luma component in the intra-frame prediction mode for the coding unit. Based on a determination that the direction of the luma component in the intra-frame prediction mode for the coding unit is purely horizontal or purely vertical, the video decoder 30 receives a third syntax element indicating whether block differential pulse code modulation (BDPCM) is enabled for a chroma component of the coding unit. Based on a determination that the third syntax element is non-zero, the video decoder 30 assigns a value of the direction of the luma component in the intra-frame prediction mode from the second syntax element to a fourth syntax element indicating a BDPCM direction of the chroma component, and then decodes the coding unit from the video data based on the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element. Based on the determination that the direction of the luma component in the intra-prediction mode for the coding unit is neither purely horizontal nor purely vertical, the video decoder 30 sets the third syntax element indicating whether the BDPCM is enabled for the chroma components of the coding unit to zero, indicating that the BDPCM is not enabled for the chroma components of the coding unit, and then decodes the coding unit from the video data based on the first syntax element, the second syntax element, and the third syntax element.

[0312] In some embodiments, when the direction of the luma component in the intra prediction mode for the coding unit is purely horizontal, the value of the second syntax element is 18.

[0313] In some embodiments, when the direction of the luma component in the intra prediction mode for the coding unit is purely vertical, the value of the second syntax element is 50.

[0314] In some embodiments, the fourth syntax element indicating the BDPCM direction of the chroma component is a predefined value not received from the bitstream.

[0315] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium, or transmitted through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium including any medium that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this way, a computer-readable medium may generally correspond to (1) a non-transient tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the embodiments described in the present application. A computer program product may include a computer-readable medium.

[0316] The terms used in the description of the embodiments herein are only used for the purpose of describing specific embodiments and are not intended to limit the scope of the claims. The singular forms "one" and "the / said" used in the description of the embodiments and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more associated listed items. It will be further understood that when used in this specification, the term "comprising" specifies the presence of stated features, elements and / or components, but does not exclude the presence or addition of other one or more features, elements, components and / or their groups.

[0317] It should also be understood that although the terms first, second, etc. may be used to describe various elements in this article, these elements should not be limited to these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the embodiment, the first electrode may be referred to as the second electrode, and similarly, the second electrode may be referred to as the first electrode. The first electrode and the second electrode are both electrodes, but not the same electrode.

[0318] The description of the present application is presented for the purpose of illustration and description and is not intended to be exhaustive or to limit the invention in the disclosed form. Many modifications, variations, and alternative embodiments will be apparent to those of ordinary skill in the art with the benefit of the teachings presented in the foregoing description and the associated drawings. The embodiments are selected and described in order to best explain the principles of the invention, the practical application, and to enable other persons skilled in the art to understand the various implementations of the invention and to best utilize the basic principles and various implementations with various modifications, as applicable to the intended specific use. Therefore, it should be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding video data, comprising: Receiving video data corresponding to a coding unit from a bitstream, wherein the coding unit is encoded in an intra-frame prediction mode; receiving a first syntax element from the video data, wherein the first syntax element indicates whether adaptive color space transform (ACT) is enabled for the coding unit; Based on the determination that the ACT is enabled for the encoding unit: receiving, from the video data, a first set of syntax elements associated with block differential pulse code modulation (BDPCM) for a luma component of the coding unit, wherein the first set of syntax elements includes a second syntax element indicating whether the BDPCM is enabled for the luma component of the coding unit; Based on the determination of enabling the BDPCM for the luma component of the coding unit: assigning corresponding values ​​of the first set of syntax elements associated with the BDPCM for luma components of the coding unit to a second set of syntax elements associated with the BDPCM for chroma components of the coding unit; and The coding unit is decoded from the video data based on the first syntax element associated with the ACT and the first set and the second set of syntax elements associated with the BDPCM.

2. The method according to claim 1, wherein: The first set of syntax elements includes a third syntax element indicating a BDPCM direction for the luma component of the coding unit.

3. The method according to claim 1, wherein: The second set of syntax elements includes a fourth syntax element, the fourth syntax element indicating whether the BDPCM is enabled for the chroma component of the coding unit, and a value of the fourth syntax element is equal to a value of the allocated second syntax element.

4. The method according to claim 2, wherein: The second set of syntax elements includes a fifth syntax element, the fifth syntax element indicating a BDPCM direction for the chroma component of the coding unit, and a value of the fifth syntax element is equal to a value of the allocated third syntax element.

5. The method according to claim 2, wherein: The second syntax element indicating whether the BDPCM is enabled for the luma component of the coding unit is intra_bdpcm_flag, and the third syntax element indicating a BDPCM direction for the luma component of the coding unit is intra_bdpcm_dir_flag.

6. The method according to claim 3, wherein: The fourth syntax element indicating whether the BDPCM is enabled for the chroma component of the coding unit is intra_bdpcm_chroma_flag.

7. The method according to claim 4, wherein: The fifth syntax element indicating the BDPCM direction for the chroma component of the coding unit is intra_bdpcm_chroma_dir.

8. The method according to claim 1, wherein: The second set of syntax elements associated with the BDPCM for the chroma components is not received from the bitstream prior to the decoding.

9. A method for decoding video data, comprising: Receiving video data corresponding to a coding unit from a bitstream, wherein the coding unit is encoded in an intra-frame prediction mode; receiving a first syntax element from the video data, wherein the first syntax element indicates whether adaptive color space transform (ACT) is enabled for the coding unit; Based on the determination that the ACT is enabled for the encoding unit: receiving a second syntax element from the video data, wherein the second syntax element indicates a direction of a luma component in the intra prediction mode for the coding unit; According to a determination of whether a direction of the luma component in the intra prediction mode for the coding unit is purely horizontal or purely vertical: receiving a third syntax element indicating whether block differential pulse code modulation (BDPCM) is enabled for a chroma component of the coding unit; In accordance with a determination that the third syntax element is non-zero, assigning a value of the direction of the luma component in the intra prediction mode from the second syntax element to a fourth syntax element indicating a BDPCM direction of the chroma components; and decoding the coding unit from the video data according to the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element; Based on the determination that the direction of the luma component in the intra prediction mode for the coding unit is neither purely horizontal nor purely vertical: setting the third syntax element indicating whether the BDPCM is enabled for the chroma components of the coding unit to zero indicating that the BDPCM is not enabled for the chroma components of the coding unit; and The coding unit is decoded from the video data according to the first syntax element, the second syntax element, and the third syntax element.

10. The method according to claim 9, wherein: When the direction of the luma component in the intra prediction mode for the coding unit is purely horizontal, the value of the second syntax element is 18.

11. The method according to claim 9, wherein: When the direction of the luma component in the intra prediction mode for the coding unit is purely vertical, the value of the second syntax element is 50.

12. The method according to claim 9, wherein: The fourth syntax element indicating the BDPCM direction of the chroma component is a predefined value not received from a bitstream.

13. An electronic device comprising: one or more processing units; a memory coupled to the one or more processing units; as well as A plurality of programs stored in the memory, which, when executed by the one or more processing units, enable the electronic device to implement the method according to any one of claims 1-12.

14. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein: When the plurality of programs are executed by the one or more processing units, the electronic device implements the method according to any one of claims 1 to 12.

Images