Video coding and decoding method and device

Abstract

An electronic device performs a method of decoding video data, the method comprising: if adaptive color transform (ACT) is enabled for a residual of a coding unit, receiving, from information in a bitstream, an adjustment value applied to a quantization parameter for inverse quantization, and a coefficient for the residual of the coding unit; reconstructing the residual of the coding unit by applying inverse quantization and inverse transform to the reception coefficient for the residual of the coding unit; and, if the residuals of the coding units have been encoded using ACT, modifying the reconstructed residuals of the coding units by applying inverse ACT to convert the reconstructed residuals of the coding units from the YCgCo color space to the original color space based on the same inverse ACT matrix as one applied to one coding unit encoded in the lossy or lossless encoding mode.

Description

[0001] This application is a divisional application of patent application No. 202080094115.5, filed on December 10, 2020, entitled "Method and Apparatus for Video Encoding and Decoding in 4:4:4 Chroma Format". Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 946,328, filed December 10, 2019, entitled “Methods and Apparatus on 4:4:4 video CODING”, and U.S. Provisional Patent Application No. 62 / 948,054, filed December 13, 2019, both of which are incorporated herein by reference in their entirety. Technical Field

[0003] This application generally relates to video data encoding, decoding and compression, and more specifically, to methods and apparatus for improving and simplifying adaptive color space transformation (ACT) for lossy and lossless coding. Background Technology

[0004] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards defined by standards such as MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC), and Universal Video Codec (VVC). Video compression typically involves performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in the video data. For block-based video codecs, video frames are divided into one or more stripes, each strip containing multiple video blocks, which may also be referred to as codec tree units (CTUs). Each CTU may contain a coding unit (CU) or be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU can be encoded and decoded in intra-frame mode, inter-frame mode, or intra-block copy (IBC) mode. Video blocks in an intra-coded (I) strip of a video frame are encoded using spatial prediction, which is performed with respect to reference samples in adjacent blocks within the same video frame. Video blocks in an inter-coded (P (forward prediction picture) or B (bidirectional prediction picture) strip of a video frame can use either spatial or temporal prediction. Spatial prediction is performed with respect to reference samples in adjacent blocks within the same video frame, while temporal prediction is performed with respect to reference samples in other previous and / or future reference video frames.

[0005] Based on spatial or temporal predictions of previously encoded reference blocks (e.g., neighboring blocks), a predicted block is derived for the current video block to be encoded / decoded. The process of finding the reference block can be accomplished using a block-matching algorithm. Residual data, representing the pixel differences between the current block to be encoded / decoded and the predicted block, is called the residual block or prediction error. Inter-frame coded blocks are encoded based on motion vectors and the residual block, with the motion vectors pointing to reference blocks in the reference frames that form the predicted blocks. The process of determining the motion vectors is often referred to as motion estimation. Intra-frame coded blocks are encoded based on intra-frame prediction modes and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, such as the frequency domain, to obtain residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, and then entropy-encoded into the video bitstream for even more compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities, or transmitted directly to the electronic device via wired or wireless means. The electronic device then performs video decompression (which is the reverse of the video compression process described above) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream, and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and the electronic device displays the reconstructed digital video data on the display of the electronic device.

[0007] As digital video quality evolves from high definition to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining the image quality of decoded video data while simultaneously encoding / decoding it more efficiently remains an ongoing challenge.

[0008] Some video content, such as screen content video, is encoded in a 4:4:4 chroma format, where all three components (luminance component and two chroma components) have the same resolution. While the 4:4:4 chroma format includes more redundancy compared to 4:2:0 and 4:2:2 chroma formats (which is detrimental to achieving good compression efficiency), it remains the preferred encoding format for many applications that require high fidelity to preserve color information (e.g., sharp edges) in decoded video. Given the redundancy in 4:4:4 chroma format video, there is evidence that significant encoding / decoding improvements can be achieved by leveraging the correlations between the three color components of 4:4:4 video (e.g., Y, Cb, and Cr in the YCbCr domain; or G, B, and R in the RGB domain). Due to these correlations, the Adaptive Color Space Transformation (ACT) tool was employed during the development of the HEVC Screen Content Codec (SCC) extension to utilize the correlations between the three color components. Summary of the Invention

[0009] This application describes implementations related to video data encoding and decoding, and more specifically, implementations relate to systems and methods for performing adaptive color space transformation (ACT) under lossy and lossless encoding.

[0010] For video signals initially captured in a 4:4:4 color format, if the decoded video signal requires high fidelity and has abundant information redundancy in the original color space (e.g., RGB video), it is preferable to encode the video in the original color space. Although some inter-component coding / decoding tools in the current VVC standard, such as Cross-Component Linear Model Prediction (CCLM), can improve the efficiency of 4:4:4 video coding / decoding, redundancy between the three components is not completely eliminated. This is because only the Y / G components are used to predict the Cb / B and Cr / R components, without considering the correlation between the Cb / B and Cr / R components. Accordingly, further decorrelation of the three color components can improve the coding / decoding performance of 4:4:4 video.

[0011] In the current VVC standard, existing inter-frame and intra-frame tools are primarily designed for video captured in 4:2:0 chroma format. Therefore, to achieve a better complexity / performance tradeoff, most of these codecs only work with the luma component, disabling the chroma component (e.g., Position-dependent Intra-Prediction Combination (PDPC), Multiple Reference Lines (MRL), and Sub-Segment Prediction (ISP)), or use different operations for the luma and chroma components (e.g., interpolation filters applied to motion-compensated prediction). However, video signals in 4:4:4 chroma format represent significantly different characteristics compared to 4:2:0 video. For example, the Cb / B and Cr / R components in 4:4:4 YCbCr and RGB video represent richer color information and possess more high-frequency information (e.g., edges and textures) compared to the chroma components in 4:2:0 video. Given this consideration, using the same design of some existing codecs for both 4:2:0 and 4:4:4 video in VVC may always be optimal.

[0012] According to a first aspect of this application, a method for decoding video data includes: receiving a first control flag from a bitstream, wherein the first control flag indicates whether to enable Adaptive Color Transform (ACT) to decode residuals of coding units in the bitstream; based on a determination of the first control flag regarding enabling the ACT to decode the residuals of the coding units in the bitstream: receiving a plurality of syntax elements from the bitstream, wherein the plurality of syntax elements indicate adjustment values ​​applied to quantization parameters, the quantization parameters being used to inverse quantize the residuals of the coding units using the ACT; and receiving video data corresponding to the coding units from the bitstream, wherein the video... The data includes coefficients corresponding to the residual of the encoding unit and a second control flag indicating whether the residual of the encoding unit has been encoded in the YCgCo color space using the ACT; the residual of the encoding unit is reconstructed by applying inverse quantization and inverse transform to the received coefficients corresponding to the residual of the encoding unit; and the reconstructed residual of the encoding unit is modified by applying inverse ACT to convert the reconstructed residual of the encoding unit from the YCgCo color space to the original color space based on an inverse ACT matrix, based on the determination of the second control flag that the residual of the encoding unit has been encoded using the ACT.

[0013] In some embodiments, based on determining that the coding unit is encoded using a lossy coding mode, the inverse quantization and inverse transform are applied to the coefficients corresponding to the residuals of the received coding unit.

[0014] In some embodiments, the same inverse ACT matrix is ​​used to transform the reconstructed residuals, regardless of whether the residuals of the coding units are encoded in lossy or lossless mode.

[0015] According to a second aspect of this application, a method for decoding video data includes: receiving a first syntax element from the video data corresponding to an encoding unit, wherein the first syntax element indicates whether the residual of the encoding unit has been encoded using the ACT; receiving a second syntax element indicating whether to apply an inverse transform to decode the residual of the encoding unit based on a determination based on the first syntax element that the residual of the encoding unit has not yet been encoded using the ACT; and assigning a default value to the second syntax element based on the determination based on the first syntax element that the residual of the encoding unit has been encoded using the ACT, the default value indicating that an inverse transform is applied to decode the residual of the encoding unit.

[0016] According to a third aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by the one or more processing units, these programs cause the electronic device to perform the method of decoding video data as described above.

[0017] According to a fourth aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. When executed by the one or more processing units, these programs cause the electronic device to perform a method for decoding video data as described above. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the embodiments and are incorporated herein and form part of the specification, illustrate the described embodiments and, together with the description, serve to explain the underlying principles. Similar reference numerals refer to corresponding parts.

[0019] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0020] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0021] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0022] Figures 4A to 4E This is a block diagram illustrating how a frame, according to some embodiments of the present disclosure, is recursively divided into multiple video blocks of different sizes and shapes.

[0023] Figure 5A and Figure 5B This is a block diagram illustrating an example of a technique for transforming residuals between the RGB color space and the YCgCo color space using adaptive color space transformation (ACT) according to some embodiments of this disclosure.

[0024] Figure 6 This is a block diagram illustrating the application of a luminance mapping technique with chroma scaling (LMCS) during exemplary video data decoding according to some embodiments of this disclosure.

[0025] Figure 7 This is a block diagram illustrating an exemplary video decoding process according to some embodiments of the present disclosure, through which the video decoder implements the technique of inverse adaptive color space transformation (ACT).

[0026] Figure 8A and Figure 8BThis is a block diagram illustrating an exemplary video decoding process according to some embodiments of the present disclosure, through which the video decoder implements techniques of inverse adaptive color space transformation (ACT) and luminance mapping with chroma scaling (LMCS).

[0027] Figure 9 This is a block diagram illustrating exemplary decoding logic between performing Adaptive Color Space Transformation (ACT) and Block Differential Pulse Code Modulation (BDPCM) according to some embodiments of the present disclosure.

[0028] Figure 10 This is a flowchart illustrating an exemplary process according to some embodiments of the present disclosure, in which a video codec decodes video data by using an inverse adaptive color space transformation (ACT) for encoding units encoded in both lossy and lossless modes.

[0029] Figure 11 This is a decoding flowchart illustrating the application of different quantization parameters (QP) offsets to different components when the internal bit depths of luminance and chrominance are different, according to some embodiments of this disclosure. Detailed Implementation

[0030] Reference will now be made in detail to the specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on many types of electronic devices with digital video capabilities.

[0031] In some embodiments, these methods are provided to improve the encoding and decoding efficiency of 4:4:4 video using the VVC standard. Generally, the main features of the techniques in this disclosure are summarized below.

[0032] In some embodiments, these methods are implemented to improve existing ACT designs, enabling adaptive color space conversion in the residual domain. In particular, special consideration has been given to handling the interaction between ACT and some existing codec tools in VVC.

[0033] In some embodiments, these methods are implemented to improve the efficiency of some existing inter-frame and intra-frame codecs for 4:4:4 video in the VVC standard, including: 1) enabling an 8-tap interpolation filter for the chroma component; 2) enabling PDPC for intra-frame prediction of the chroma component; 3) enabling MRL for intra-frame prediction of the chroma component; and 4) enabling ISP segmentation for the chroma component.

[0034] Figure 1 This is a block diagram illustrating an exemplary system 10 for encoding and decoding video blocks in parallel according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes source device 12, which generates and encodes video data that will later be decoded by destination device 14. Source device 12 and destination device 14 can include any electronic device from a variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some embodiments, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0035] In some implementations, destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, link 16 may include a communication medium enabling source device 12 to directly transmit encoded video data to destination device 14 in real time. The encoded video data may be modulated according to communication standards, such as wireless communication protocols, and transmitted to destination device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that may facilitate communication from source device 12 to destination device 14.

[0036] In some other implementations, encoded video data can be sent from output interface 22 to storage device 32. The encoded video data in storage device 32 can then be accessed by destination device 14 via input interface 28. Storage device 32 may include any data storage medium of various distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 may correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. Destination device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and sending encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 can access the encoded video data via any standard data connection suitable for accessing the encoded video data stored on the file server, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or combinations of both. Transmission of the encoded video data from storage device 32 can be streaming, downloading, or a combination of both.

[0037] like Figure 1 As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include a source such as a video capture device (e.g., a camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. As an example, if video source 18 is a camera in a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and can be applied to wireless and / or wired applications.

[0038] Captured video, pre-captured video, or computer-generated video can be encoded by video encoder 20. The encoded video data can be sent directly to destination device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may also include a modem and / or transmitter.

[0039] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem, and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 when decoding the video data. Such syntax elements may be included within encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0040] In some embodiments, destination device 14 may include display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0041] Video encoder 20 and video decoder 30 can operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4 Part 10, Advanced Video Codec (AVC), or extensions of such standards. It should be understood that this application is not limited to specific video encoding / decoding standards and can be applied to other video encoding / decoding standards. It is generally expected that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally expected that the video decoder 30 of destination device 14 can be configured to decode video data according to any of these current or future standards.

[0042] Both the video encoder 20 and the video decoder 30 can be implemented as any circuit system in a variety of suitable encoder circuit systems, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0043] Figure 2This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can perform intra-frame predictive coding and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-frame predictive coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within neighboring video frames or pictures of a video sequence.

[0044] like Figure 2 As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a segmentation unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copy (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A deblocking filter (not shown) may be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, a loop filter (not shown) may be used to filter the output of the adder 62. The video encoder 20 may be in the form of fixed hardware units or programmable hardware units, or may be divided among one or more of the described fixed hardware units or programmable hardware units.

[0045] Video data memory 40 can store video data to be encoded by components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data used by video encoder 20 when encoding video data (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, video data memory 40 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0046] like Figure 2As shown, after receiving video data, the segmentation unit 45 within the prediction processing unit 41 segments the video data into video blocks. This segmentation may also include segmenting the video frame into stripes, tiles, or other larger coding units (CUs) based on a predefined splitting structure (such as a quadtree structure) associated with the video data. The video frame can be divided into multiple video blocks (or a set of video blocks referred to as tiles). The prediction processing unit 41 can select one of several possible predictive coding modes for the current video block based on error results (e.g., encoding / decoding rate and distortion level), such as one of several intra-frame predictive coding modes or one of several inter-frame predictive coding modes. The prediction processing unit 41 can provide the resulting intra-frame or inter-frame predictive coded block to adder 50 to generate a residual block, and to adder 62 to reconstruct the encoded block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-frame mode indicators, segmentation information, and other such syntax information, to entropy coding unit 56.

[0047] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal prediction. The video encoder 20 can perform multiple coding passes, for example, to select a suitable coding mode for each video data block.

[0048] In some implementations, motion estimation unit 42 determines the inter-frame prediction mode for the current video frame by generating motion vectors according to a predetermined pattern within the video frame sequence. These motion vectors indicate the displacement of the prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate motion relative to a video block. The motion vectors may, for example, indicate the displacement of the PU of a video block within the current video frame or picture relative to a prediction block (or other encoded unit) within a reference frame, which is relative to a current block (or other encoded unit) encoded within the current frame. The predetermined pattern may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine the vectors (e.g., block vectors) for intra-frame BC coding in a manner similar to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0049] A prediction block is a block of reference frames that is considered to closely match the PU of the video block to be encoded in terms of pixel differences, which can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute the values ​​of sub-integer pixel positions of the reference frames stored in the DPB 64. For example, the video encoder 20 can interpolate the values ​​of quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frames. Therefore, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0050] The motion estimation unit 42 calculates the motion vector of the PU (Programmable Component) of a video block in an inter-frame predictive coding frame by comparing the position of the PU with the position of the predicted block of a reference frame selected from either a first reference frame list (list 0) or a second reference frame list (list 1). Each of the first and second reference frame lists identifies one or more reference frames stored in the DPB64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0051] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving motion vectors for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel differences forming the residual video block may include a luminance difference component or a chrominance difference component, or both. Motion compensation unit 44 may also generate syntax elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Syntax elements may include, for example, syntax elements defining motion vectors for describing prediction blocks, any flags indicating prediction modes, or any other syntax information described herein. Note that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0052] In some implementations, the intra-BC unit 48 can generate vectors and acquire prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but the prediction blocks are located in the same frame as the current block being encoded, and the vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode to be used for encoding the current block. In some examples, the intra-BC unit 48 can, for example, use various intra-prediction modes to encode the current block during multiple individual encoding passes and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select an appropriate intra-prediction mode to use from the various tested intra-prediction modes and generate an intra-prediction mode indicator accordingly. For example, the intra-BC unit 48 can use rate-distortion analysis to calculate rate-distortion values ​​for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics as the appropriate intra-prediction mode to be used. Rate-distortion analysis typically determines the amount of distortion (or error) between a coded block and the original uncoded block that was encoded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate the ratio from the distortion and rate for various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0053] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to perform such functions for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, the predicted block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference measures, and the identification of the predicted block may include value calculation for sub-integer pixel locations.

[0054] Regardless of whether the prediction block originates from the same frame based on intra-frame prediction or from different frames based on inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luma component difference and chroma component difference.

[0055] As an alternative to the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44 as described above, or the intra-block copy prediction performed by the intra-BC unit 48, the intra-prediction processing unit 46 can perform intra-frame prediction on the current video block. Specifically, the intra-prediction processing unit 46 can determine the intra-prediction mode to be used for encoding the current block. To this end, the intra-prediction processing unit 46 can, for example, use various intra-prediction modes to encode the current block during multiple individual encoding passes, and the intra-prediction processing unit 46 (or, in some examples, a mode selection unit) can select the appropriate intra-prediction mode to be used from the tested intra-prediction modes. The intra-prediction processing unit 46 can provide information to the entropy coding unit 56 indicating the intra-prediction mode selected for the block. The entropy coding unit 56 can encode the information indicating the selected intra-prediction mode in the bitstream.

[0056] After prediction processing unit 41 determines a prediction block for the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 uses a transform to transform the residual video data into residual transform coefficients, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0057] The transform processing unit 52 can send the resulting transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform the scan.

[0058] After quantization, entropy coding unit 56 uses, for example, context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probabilistic interval partitioned entropy (PIPE) coding, or other entropy coding methods or techniques to entropy encode the quantized transform coefficients into a video bitstream. The encoded bitstream can then be sent to video decoder 30, or archived in storage device 32 for later transmission to video decoder 30 or retrieval by video decoder 30. Entropy coding unit 56 can also entropy encode motion vectors and other syntax elements used for the current video frame being encoded.

[0059] The dequantization unit 58 and the inverse transform processing unit 60 apply dequantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate a reference block for predicting other video blocks. As noted above, the motion compensation unit 44 can generate a motion-compensated prediction block from one or more reference blocks of a frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to compute sub-integer pixel values ​​used in motion estimation.

[0060] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to produce a reference block for storage in DPB 64. The reference block can then be used as a prediction block by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 for inter-frame prediction of another video block in subsequent video frames.

[0061] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction processing unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can perform a decoding process, which is generally combined with the above description. Figure 2 The encoding processes described for video encoder 20 are inverses of each other. For example, motion compensation unit 82 can generate prediction data based on motion vectors received from entropy decoding unit 80, while intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from entropy decoding unit 80.

[0062] In some examples, units of the video decoder 30 may be assigned tasks to perform embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be divided among one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may perform embodiments of this application alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra-frame prediction processing unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0063] Video data memory 79 may store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 may be obtained, for example, from storage device 32, from a local video source (such as a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 when decoding video data (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 79 and DPB 92 may be formed of any memory device from a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM, SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are... Figure 3 The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be located on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0064] During the decoding process, the video decoder 30 receives the encoded video bitstream and associated syntax elements, where the encoded video bitstream represents a video block of an encoded video frame. The video decoder 30 can receive syntax elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantized coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 80 then forwards the motion vectors and other syntax elements to the prediction processing unit 81.

[0065] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode transmitted by the signal and reference data from the previous decoded block of the current frame.

[0066] When a video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks for the current video frame based on motion vectors and other syntax elements received from the entropy decoding unit 80. Each prediction block can be generated from a reference frame within one of the reference frame lists. The video decoder 30 can construct the reference frame lists using the default construction technique based on the reference frames stored in the DPB 92: list 0 and list 1.

[0067] In some examples, when a video block is encoded according to the intra-BC mode described herein, the intra-BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from the entropy decoding unit 80. The prediction block may lie within the reconstructed region of the same image as the current video block defined by the video encoder 20.

[0068] Motion compensation unit 82 and / or intra-frame BC unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other syntax elements, and then use the prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frame lists for the frame, motion vectors for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0069] Similarly, the intra-BC unit 85 can use some of the syntax elements (e.g., flags) from the received syntax elements to determine whether the current video block was predicted using the intra-BC mode, which video blocks of the frame are located within the reconstruction region and should be stored in the DPB 92, the block vector of the video block predicted for each intra-BC for the frame, the intra-BC prediction state of the video block predicted for each intra-BC for the frame, and other information for decoding the video blocks in the current video frame.

[0070] The motion compensation unit 82 can also perform interpolation using interpolation filters, such as those used by the video encoder 20 during the encoding of video blocks, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, the motion compensation unit 82 can determine the interpolation filters used by the video encoder 20 from the received syntax elements and use these interpolation filters to generate the prediction block.

[0071] The dequantization unit 86 dequantizes the quantized transform coefficients provided in the bitstream and entropy-decoded by the entropy decoding unit 80 using the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to determine the quantization level. The inverse transform processing unit 88 applies an inverse transform, such as inverse DCT, inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients in order to reconstruct the residual blocks in the pixel domain.

[0072] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by summing the residual block from the inverse transform processing unit 88 and the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92 or a separate memory device may also store the decoded video for later display on a display device (such as...). Figure 1 It is displayed on the display device 34).

[0073] In a typical video encoding and decoding process, a video sequence usually consists of an ordered set of frames or images. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other cases, a frame may be monochromatic and therefore consist of only a two-dimensional array of luminance samples.

[0074] like Figure 4A As shown, the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set such that all CTUs in the video sequence have the same size, namely one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not limited to a specific size. Figure 4BAs shown, each CTU may include a coding tree block (CTB) for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements for encoding the samples of the coding tree blocks. The syntax elements describe the properties of different types of units in the encoded pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter-frame prediction or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In monochrome images or images with three separate color planes, the CTU may include a single coding tree block and syntax elements for encoding the samples of the coding tree block. The coding tree block may be an NxN sample block.

[0075] To achieve better performance, the video encoder 20 can recursively perform tree splitting on the coding tree blocks of the CTU, such as binary tree splitting, ternary tree splitting, quadtree splitting, or combinations thereof, and divide the CTU into smaller coding units (CUs). Figure 4C As depicted, the 64x64 CTU 400 is first divided into four smaller CUs, each with a block size of 32x32. Of these four smaller CUs, CU 410 and CU 420 are each further divided into four 16x16 CUs. The two 16x16 CUs, 430 and CU 440, are each further divided into four 8x8 CUs. Figure 4D The quadtree data structure is described, and the quadtree data structure is shown as follows: Figure 4C The final result of the CTU 400 partitioning process depicted in the diagram is that each leaf node of the quadtree corresponds to a CU of a corresponding size ranging from 32x32 to 8x8. Similar to... Figure 4B The CTU depicted herein may include, for each CU, a coded block (CB) of luminance samples and two corresponding coded blocks of chrominance samples of the same size frame, as well as syntax elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, a CU may include a single coded block and a syntax structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4D The quadtree partitioning depicted is for illustrative purposes only, and a CTU can be split into CUs based on quadtree / ternary / binary tree partitioning to accommodate different local characteristics. In multi-type tree structures, a CTU is partitioned by a quadtree structure, and each quadtree leaf CU can be further partitioned by binary and ternary tree structures. Figure 4E As shown, there are five types of partitioning: quadrilateral partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal trilateral partitioning, and vertical trilateral partitioning.

[0076] In some implementations, the video encoder 20 may further segment the coded blocks of the CU into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) sample block to which the same prediction (inter-frame or intra-frame) is applied. A prediction unit (PU) of the CU may include a prediction block for luma samples, two corresponding prediction blocks for chroma samples, and syntax elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and a syntax structure for predicting the prediction block. The video encoder 20 can generate predicted luma blocks, Cb blocks, and Cr blocks for each PU of the CU, representing the luma, Cb, and Cr prediction blocks.

[0077] Video encoder 20 can generate prediction blocks for a PU using intra-frame prediction or inter-frame prediction. If video encoder 20 uses intra-frame prediction to generate prediction blocks for a PU, it can generate the prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for a PU, it can generate the prediction blocks for the PU based on decoded samples of one or more frames different from the frame associated with the PU.

[0078] After the video encoder 20 generates predicted luminance blocks, Cb blocks, and Cr blocks for one or more PUs of the CU, the video encoder 20 can generate luminance residual blocks for the CU by subtracting the predicted luminance blocks of the CU from their original luminance coding blocks, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0079] In addition, such as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks. A transform block is a rectangular (square or non-square) sample block to which the same transform is applied. A transform unit (TU) of the CU can include a transform block of luminance samples, two corresponding transform blocks of chrominance samples, and syntax elements for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU can be a sub-block of the CU's luminance residual block. A Cb transform block can be a sub-block of the CU's Cb residual block. A Cr transform block can be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and syntax structures for transforming the samples of the transform block.

[0080] Video encoder 20 can apply one or more transforms to the luminance transform block of TU to generate a luminance coefficient block for TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. Video encoder 20 can apply one or more transforms to the Cb transform block of TU to generate a Cb coefficient block for TU. Video encoder 20 can apply one or more transforms to the Cr transform block of TU to generate a Cr coefficient block for TU.

[0081] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy-encode the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 can perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data; the bitstream is stored in storage device 32 or transmitted to destination device 14.

[0082] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain syntax elements. The video decoder 30 can reconstruct frames of video data, at least in part, based on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally the inverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the codec blocks of the current CU by adding samples of the prediction block of the PU for the current CU to the corresponding samples of the transform block of the TU of the current CU. After reconstructing the codec blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0083] As noted above, video encoding and decoding primarily employ two modes for video compression: intra-frame prediction and inter-frame prediction. Palette-based encoding and decoding is another approach adopted by many video encoding and decoding standards. In palette-based encoding and decoding (which may be particularly suitable for screen-generated content encoding and decoding), the video codec (e.g., video encoder 20 or video decoder 30) forms a color palette table representing a given block of video data. The palette table includes the most dominant (e.g., frequently used) pixel values ​​in the given block. Pixel values ​​that are not frequently represented in the video data of a given block are not included in the palette table, or are included as escape colors.

[0084] Each entry in the palette table includes an index to the corresponding pixel value in the palette table. The palette index for a sample within a block can be encoded to indicate which entry from the palette table will be used to predict or reconstruct which sample. This palette pattern begins with the process of generating palette predictions for the first block of a group of images, stripes, tiles, or other such video blocks. As will be explained below, palette predictions for subsequent video blocks are typically generated by updating previously used palette predictions. For illustrative purposes, it is assumed that the palette predictions are defined at the image level. In other words, an image may include multiple codec blocks, each with its own palette table, but only one palette prediction for the entire image.

[0085] To reduce the number of bits required to signal palette entries in the video bitstream, the video decoder can use palette prediction values ​​to determine new palette entries in the palette table used to reconstruct video blocks. For example, the palette prediction values ​​may include palette entries from previously used palette tables, or may even be initialized using the most recently used palette table by including all entries from the most recently used palette table. In some implementations, the palette prediction values ​​may include fewer than all entries from the most recently used palette table, and then include some entries from other previously used palette tables. The palette prediction values ​​may have the same size as, or be larger than, or smaller than, the palette tables used to encode and decode different blocks. In one example, the palette prediction values ​​are implemented as a first-in-first-out (FIFO) table comprising 64 palette entries.

[0086] To generate a palette table for video data blocks from palette predictions, the video decoder can receive a 1-bit flag for each entry of the palette predictions from the encoded video bitstream. This 1-bit flag can have a first value (e.g., binary 1) or a second value (e.g., binary 0), where the first value indicates that the associated entry of the palette prediction will be included in the palette table, and the second value indicates that the associated entry of the palette prediction will not be included in the palette table. If the size of the palette predictions is larger than the palette table for video data blocks, the video decoder can stop receiving more flags once the maximum size for the palette table is reached.

[0087] In some implementations, some entries in the palette table can be signaled directly in the encoded video bitstream, rather than determined using palette prediction values. For such entries, the video decoder can receive three separate m-bit values ​​from the encoded video bitstream, indicating the pixel values ​​of the luma component and two chroma components associated with the entry, where m represents the bit depth of the video data. Those palette entries derived from palette prediction values ​​require only 1 bit of flags compared to the multiple m-bit values ​​required for direct signaling of palette entries. Therefore, signaling some or all palette entries using palette prediction values ​​can significantly reduce the number of bits required to signal new palette table entries, thereby improving the overall encoding / decoding efficiency of palette pattern encoding / decoding.

[0088] In many cases, the palette prediction value for a block is determined based on the palette table used to encode one or more previous codec blocks. However, when encoding and decoding the first codec tree unit in a picture, strip, or tile, the palette table of the previous codec blocks may be unavailable. Therefore, the palette prediction value cannot be generated using entries from the previously used palette table. In this case, a sequence of initial values ​​for the palette prediction value can be sent in the Sequence Parameter Set (SPS) and / or Picture Parameter Set (PPS) as signals. These initial values ​​are used to generate the palette prediction value when the previously used palette table is unavailable. The SPS typically refers to the syntax structure applied to syntax elements of a series of consecutive encoded video pictures, called a Codec Video Sequence (CVS), which is determined by the contents of syntax elements found in the PPS referenced by syntax elements found in the header of each strip segment. The PPS typically refers to the syntax structure applied to syntax elements of one or more individual pictures within a CVS, which is determined by syntax elements found in the header of each strip segment. Therefore, SPS is generally considered to be a higher-level grammatical structure than PPS, meaning that the grammatical elements included in SPS are generally less frequently changed and applied to a larger portion of the video data compared to those included in PPS.

[0089] Figures 5A to 5B This is a block diagram illustrating an example of using an adaptive color space transformation (ACT) technique to transform the residual between the RGB color space and the YCgCo color space according to some embodiments of this disclosure.

[0090] In the HEVC screen content codec extension, ACT is applied to adaptively transform residuals from one color space (e.g., RGB) to another (e.g., YCgCo), thereby significantly reducing the correlation (e.g., redundancy) between the three color components (e.g., R, G, and B) in the YCgCo color space. Furthermore, in existing ACT designs, a flag is sent for each TU using a signal. tu_act_enabled_flag Adaptation of different color spaces is performed at the Transform Unit (TU) level. When the identifier... tu_act_enabled_flag When the value is 1, it indicates that the residual of the current TU is encoded in the YCgCo space; otherwise (i.e., the flag is 0), it indicates that the residual of the current TU is encoded in the original color space (i.e., no color space conversion). Furthermore, different color space transformation formulas are applied depending on whether the current TU is encoded in lossless or lossy mode. Specifically, in Figure 5A The formulas for forward and inverse color space transformation between the RGB color space and the YCgCo color space for lossy modes are defined.

[0091] For lossless mode, a reversible version of the RGB-YCgCo transform (also known as YCgCo-LS) is used. The reversible version of the RGB-YCgCo transform is based on... Figure 5B This is achieved through the lifting operations described in the relevant instructions.

[0092] like Figure 5A As shown, the forward and inverse color transformation matrices used in the lossy mode are not normalized. Therefore, the amplitude of the YCgCo signal is smaller than the amplitude of the original signal after the color transformation is applied. To compensate for the amplitude drop caused by the forward color transformation, an adjusted quantization parameter is applied to the residual in the YCgCo domain. Specifically, when the color space transformation is applied, the QP value QP used to quantize the YCgCo domain residual is... Y QP Cg and QP Co Set them to QP-5, QP-5 and QP-3 respectively, where QP is the quantization parameter used in the original color space.

[0093] Figure 6 This is a block diagram illustrating the application of luminance mapping with chroma scaling (LMCS) in an exemplary video data decoding process according to some embodiments of this disclosure.

[0094] In VVC, LMCS is used as a new codec tool applied before loop filters (e.g., deblocking filters, SAO, and ALF). Generally, LMCS has two main modules: 1) loop mapping of the luminance component based on an adaptive piecewise linear model; and 2) luminance-dependent chrominance residual scaling. Figure 6 A modified decoding process with LMCS applied is shown. Figure 6 In the mapped domain, the decoding modules include: an entropy decoding module, an inverse quantization module, an inverse transform module, a luminance intra-frame prediction module, and a luminance sample reconstruction module (i.e., the sum of luminance prediction samples and luminance residual samples). In the original (i.e., non-mapped) domain, the decoding modules include a motion compensation prediction module, a chrominance intra-frame prediction module, a chrominance sample reconstruction module (i.e., the sum of chrominance prediction samples and chrominance residual samples), and all loop filter modules such as a deblocking module, a SAO module, and an ALF module. New operational modules introduced by LMCS include: a forward mapping module 610 for luminance samples, a reverse mapping module 620 for luminance samples, and a chrominance residual scaling module 630.

[0095] The loop mapping of LMCS can adjust the dynamic range of the input signal to improve encoding and decoding efficiency. In existing LMCS designs, the loop mapping of luminance samples is based on two mapping functions: a forward mapping function... FwdMap and a corresponding reverse mapping function InvMapThe forward mapping function uses a piecewise linear model with 16 equally sized segments to be sent from the encoder to the decoder via a signal. The inverse mapping function can be derived directly from the forward mapping function, so it does not need to be sent via a signal.

[0096] At the strip level, parameters of the luminance mapping model are signaled. First, an presence flag is signaled to indicate whether a luminance mapping model for the current strip should be signaled. If a luminance mapping model exists in the current strip, the corresponding piecewise linear model parameters are further signaled. Additionally, at the strip level, another LMCS control flag is signaled to enable / disable LMCS for the strip.

[0097] The chroma residual scaling module 630 is designed to compensate for the mutual interference of quantization accuracy between the luminance signal and its corresponding chroma signal when loop mapping is applied to the luminance signal. The strip header also signals whether chroma residual scaling is enabled or disabled for the current strip. If luminance mapping is enabled, an additional flag is signaled to indicate whether luminance-dependent chroma residual scaling is applied. When luminance mapping is not used, luminance-dependent chroma residual scaling is always disabled and no additional flag is required. Furthermore, chroma residual scaling is always disabled for CUs containing four or fewer chroma samples.

[0098] Figure 7 This is a block diagram illustrating an exemplary video decoding process according to some embodiments of the present disclosure, through which the video decoder implements the technique of inverse adaptive color space transformation (ACT).

[0099] Similar to the ACT design in HEVC SCC, ACT in VVC converts the intra / inter-frame prediction residuals of a CU in a 4:4:4 chroma format from the original color space (e.g., RGB color space) to the YCgCo color space. As a result, redundancy between the three color components can be reduced, leading to better encoding and decoding efficiency. Figure 7 The flowchart illustrates how the inverse ACT decoding process is applied in the VVC framework by adding the inverse ACT module 710. When processing a CU with ACT encoding enabled, entropy decoding, inverse quantization, and inverse DCT / DST-based transform are first applied to the CU. After that, as... Figure 7 As shown, the inverse ACT is invoked to convert the decoded residual from the YCgCo color space to the original color space (e.g., RGB and YCbCr). Furthermore, since the ACT in lossy mode is not normalized, a QP adjustment of (-5, -5, -3) is applied to the Y, Cg, and Co components to compensate for the changed magnitude of the transformed residual.

[0100] In some embodiments, the ACT method reuses the same ACT core transform of HEVC for color conversion between different color spaces. Specifically, depending on whether the current CU is encoded in a lossy or lossless manner, two different versions of the color transform are applied. For lossy cases, the forward and inverse color transforms use an irreversible YCgCo transform matrix, such as... Figure 5A As shown in the image. For non-destructive cases, the application is as follows: Figure 5B The reversible color transformation YCgCo-LS is shown. Furthermore, unlike existing ACT designs, the following changes were introduced to the ACT scheme to handle its interaction with other codec tools in the VVC standard.

[0101] For example, because the residual of a CU in HEVC can be divided into multiple TUs, an ACT control flag is sent individually for each TU to indicate whether color space conversion needs to be applied. However, as combined with the above... Figure 4E The VVC implementation uses a nested binary and ternary partitioning structure to replace the multi-partition type concept with a quadtree, thus eliminating the separate CU, PU, ​​and TU partitions in HEVC. This means that in most cases, a CU leaf node is also used as a unit for prediction and transform processing without further partitioning, unless the maximum supported transform size is smaller than the width or height of a CU component. Based on this partitioning structure, ACT at the CU level is adaptively enabled and disabled. Specifically, a flag is sent to each CU. cu_act_enabled_flag This allows selection between the native color space and the YCgCo color space for encoding and decoding the residuals of the CU. If the flag is equal to 1, it indicates that the residuals of all TUs within the CU are encoded and decoded in the YCgCo color space. Otherwise, if the flag... cu_act_enabled_flag If the value is 0, then all residuals of the CU are encoded and decoded in the original color space.

[0102] In some embodiments, there are different scenarios where ACT can be disabled. When ACT is enabled for a CU, it needs to access the residuals of all three components for color space conversion. However, VVC design cannot guarantee that each CU always contains information for all three components. According to embodiments of this disclosure, in those cases where the CU does not contain information for all three components, ACT should be forcibly disabled.

[0103] First, in some embodiments, when a single-tree segmentation structure is applied, the luma and chroma samples within a CTU are segmented into CUs based on the single-tree segmentation structure. As a result, the CUs in the luma segmentation tree contain only the encoding / decoding information for the luma component, and the CUs in the chroma segmentation tree contain only the encoding / decoding information for the two chroma components. According to current VVC, the switching between single-tree and single-tree segmentation structures is performed at the stripe level. Therefore, according to embodiments of this disclosure, when a single tree is found to be applied to a stripe, ACT will always be disabled for all CUs (including luma and chroma CUs) within the stripe, without signaling an ACT flag that is inferred to be zero.

[0104] Second, in some embodiments, when ISP mode is enabled (described further below), TU segmentation is applied only to luma samples, while chroma samples are encoded / decoded without further segmentation into multiple TUs. Assuming N is the number of ISP sub-segments (i.e., TUs) for an internal CU, according to current ISP designs, only the last TU contains both luma and chroma components, while the first N-1 ISP TUs consist only of luma components. According to one embodiment of this disclosure, ACT is disabled in ISP mode. There are two ways to disable ACT for ISP mode. In the first method, an ACT enable / disable flag (i.e., ...) is signaled before the ISP mode syntax is signaled. cu_act_enabled_flag In this case, when the flag cu_act_enabled_ flag When equal to 1, ISP mode will not be signaled in the bitstream, but will always be inferred as zero (i.e., off). In the second method, ISP mode signaling is used to bypass the signaling of the ACT flag. Specifically, in this method, the flag... cu_ act_enabled_flag Previously, signal transmission was used in ISP mode. When ISP mode is selected, the signal transmission flag is not used. cu_act_ enabled_flag And the result is inferred to be zero. Otherwise (ISP mode not selected), the flag is... cu_act_enabled_flag The color space for residual encoding and decoding of the CU will still be adaptively selected by signal transmission.

[0105] In some embodiments, LMCS is disabled for CUs where luminance and chrominance segmentation structures are misaligned, in addition to forcibly disabling ACT. In one embodiment, when a CU selects the YCgCo color space to encode and decode its residuals (i.e., ACT is 1), both luminance mapping and chrominance residual scaling are disabled. In another embodiment, when ACT is enabled for a CU, only chrominance residual scaling is disabled, and luminance mapping can still be applied to adjust the dynamic range of the output luminance samples. In a final embodiment, both luminance mapping and chrominance residual scaling are enabled for CUs that apply ACT to encode and decode their residuals. There are several ways to enable chrominance residual scaling for CUs that apply ACT. In one method, chrominance residual scaling is applied before the inverse ACT during decoding. This means that when ACT is applied, chrominance residual scaling is applied to the chrominance residuals in the YCgCo domain (i.e., Cg and Co residuals). In another method, chrominance residual scaling is applied after the inverse ACT. Specifically, the second method applies chrominance scaling to the residuals in the original color space. Assuming the input video is captured in RGB format, this means that chroma residual scaling is applied to the residuals of the B and R components.

[0106] In some embodiments, syntax elements (e.g., sps_act_enabled_flag A bitstream consistency requirement is added to the Sequence Parameter Set (SPS) to indicate whether ACT is enabled at the sequence level. Furthermore, since color space conversion is applied to video content whose luminance and chrominance components have the same resolution (e.g., 4:4:4 chroma format), a bitstream consistency requirement needs to be added to enable ACT only for 4:4:4 chroma format. Table 1 shows the modified SPS syntax table with the above syntax added. Table 1. Modified SPS Syntax Table

[0107] Specifically, sps_act_enabled_flag An equal value of 1 indicates that ACT is enabled, and sps_act_enabled_flag An equal-0 value indicates that ACT is disabled, so that no signaling flag is used for CUs that involve SPS but are inferred to be 0. cu_act_enabled_ flag .when ChromaArrayType When the value is not equal to 3, the requirement for bitstream consistency is: sps_act_enabled_flag The value should be equal to 0.

[0108] In another embodiment, signals are not always sent. sps_act_enabled_flagInstead, it sends a signal flag based on the chroma type of the input signal. Specifically, assuming ACT can only be applied when the luminance and chroma components are at the same resolution, the signal flag is sent only when the input video is captured in 4:4:4 chroma format. sps_act_ enabled_flag Under these changes, the revised SPS syntax table is: Table 2 Modified SPS syntax table with signaling conditions

[0109] In some embodiments, the following table shows the syntax design specifications for using ACT to decode video data. Table 3 Specifications for Signaling ACT Mode

[0110] logo cu_act_enabled_flag A value of 1 indicates encoding and decoding of the residual of the coding unit in the YCgCo color space, while the flag... cu_act_enabled_flag A value of 0 indicates encoding / decoding of the residual of the coding unit in the original color space (e.g., RGB or YCbCr). When the flag... cu_act_enabled_flag If it does not exist, it is inferred to be equal to 0.

[0111] In the current VVC working draft, when capturing input video in 4:4:4 chroma format, a transform skip mode can be applied to both the luma and chroma components. Based on this design, in some embodiments, three methods are used to handle the interaction between ACT and transform skip.

[0112] In one approach, when transform skip mode is enabled for an ACT CU, the transform skip mode applies only to the luma component and not to the chroma component. In some embodiments, the following table illustrates the syntax design specifications for this approach. Table 4 Syntax Specifications When Transform Skip Mode is Applied Only to Luminance Component

[0113] In another approach, the transform skip mode is applied to both the luminance and chrominance components. In some embodiments, the following table illustrates the syntax design specifications for this approach. Table 5 Syntax specifications when transform skip mode is applied to both the luminance and chrominance components.

[0114] In another approach, transform skip mode is always disabled when ACT is enabled for a CU. In some embodiments, the following table shows the syntax design specifications for this approach. Table 6 Syntax Specifications When Transition Skip Mode is Always Disabled

[0115] Figure 8A and Figure 8B This is a block diagram illustrating an exemplary video decoding process according to some embodiments of the present disclosure, through which the video decoder implements techniques for inverse adaptive color space transformation (ACT) and luminance mapping with chroma scaling. In some embodiments, ACT (e.g., Figure 7 Inverse ACT 710) and chromaticity residual scaling (e.g., Figure 6 The video bitstream is encoded and decoded using chroma residual scaling (630). In some other embodiments, the video bitstream is encoded and decoded using chroma residual scaling instead of both ACT, thus eliminating the need for inverse ACT 710.

[0116] More specifically, Figure 8A An embodiment in which the video codec performs chroma residual scaling 630 prior to inverse ACT 710 is described. As a result, the video codec performs a luminance mapping with chroma residual scaling 630 in the color space transform domain. For example, assuming the input video is captured in RGB format and converted to the YCgCo color space, the video codec performs chroma residual scaling 630 on the chroma residuals Cg and Co based on the luminance residual Y in the YCgCo color space.

[0117] Figure 8B An alternative embodiment in which the video codec performs chroma residual scaling 630 after inverse ACT 710 is described. As a result, the video codec performs a luminance mapping with chroma residual scaling 630 in the original color space domain. For example, assuming the input video is captured in RGB format, the video codec applies chroma residual scaling to the B and R components.

[0118] Figure 9 This is a block diagram illustrating exemplary decoding logic between performing Adaptive Color Space Transformation (ACT) and Block Differential Pulse Code Modulation (BDPCM) according to some embodiments of the present disclosure.

[0119] BDPCM is an encoding tool used for screen content encoding. In some embodiments, the BDPCM enable flag is signaled at the sequence level in SPS. The BDPCM enable flag is signaled only if transition skip mode is enabled in SPS.

[0120] When BDPCM is enabled, if the CU size is less than or equal to the MaxTsSize × MaxTsSize luma samples, and if the CU is intra-coded, a flag is sent at the CU level, where MaxTsSize is the maximum block size allowed for transform skipping modes. This flag indicates whether regular intra-coding or BDPCM is used. If BDPCM is used, another BDPCM prediction direction flag is further sent to indicate whether the prediction is horizontal or vertical. The block is then predicted using a regular horizontal or vertical intra-prediction process with unfiltered reference samples. The residuals are quantized, and each quantized residual is encoded as the difference between its predicted value and its adjacent horizontal or vertical (depending on the BDPCM prediction direction) position.

[0121] For a block of size M (height) × N (width), let To predict the residuals, let... Residual The quantized version. BDPCM is applied to the quantized residuals to produce an element-wise... The modified M × N array R̃, where, It is predicted from its adjacent quantization residual values. For the vertical BDPCM prediction mode, for 0≤j≤(N-1), the following is used for derivation. : (1)

[0122] For the horizontal BDPCM prediction mode, for 0 ≤ i ≤ (M-1), the following is used for derivation. : (2)

[0123] On the decoder side, the above process is calculated in reverse. ,as follows: If using vertical BDPCM (3) If using horizontal BDPCM (4)

[0124] Inverse quantization residual Add the predicted values ​​within the block to produce the reconstructed sample values.

[0125] The predicted quantized residual values ​​are encoded using the same residual coding process as transform skip mode residual coding. Send to the decoder. Regarding the MPM mode used for future intra-frame mode coding, if the BDPCM prediction direction is horizontal or vertical, the CU for BDPCM encoding stores the horizontal or vertical prediction mode. For deblocking, if both blocks on either side of a block boundary are encoded using BDPCM, that particular block boundary is not deblocked. According to the latest VVC working draft, when the input video is in 4:4:4 chroma format, this can be achieved by sending two separate flags (i.e., ...) with a signal. intra_bdpcm_luma_flag and intra_ bdpcm_chroma_flag BDPCM is applied to the luminance and chrominance components (targeting the CU-level luminance and chrominance channels).

[0126] In some embodiments, the video codec executes different logic to better handle the interaction between ACT and BDPCM. For example, when ACT is applied to an intra-frame CU, BDPCM enables the luma component but disables the chroma component (910). The corresponding modified syntax table for the coding unit is shown below: Table 7 BDPCM enabled only for the luma component

[0127] In some embodiments, when ACT is applied to an intra-frame CU, BDPCM (920) is enabled for both the luma and chroma components. The corresponding modified syntax table for the coding unit is shown below: Table 8. BDPCM Enabled for Luminance and Chroma Components

[0128] In some embodiments, when ACT is applied to an intra-frame CU, BDPCM is disabled for both the luma and chroma components (930). In this case, it is not necessary to signal the BDPCM-related syntax elements. The corresponding modified syntax table for the coding unit is shown below: Table 9. Disabling BDPCM for both luminance and chrominance components.

[0129] In some embodiments, lossless encoding is used to process ACT. In the HEVC standard, this is achieved by sending a CU-level flag via a signal. cu_transquant_bypass_flagA value of 1 indicates a lossless mode for a CU. However, in the ongoing VVC standardization process, a different lossless enabling method is applied. Specifically, when encoding a CU in a lossless manner, it is only necessary to skip the transform and use a quantization step size of 1. This can be achieved by signaling a CU-level QP value of 1 and signaling a TU-level QP value of 1. transform_skip_flag To achieve this, it is 1. Therefore, in one embodiment of this disclosure, according to transform_skip_flag The values ​​of QP and QP are used to switch between lossy and lossless ACT transformations for a CU / TU. When the flag... transform_skip_flag When the QP value is equal to 1 and the QP value is equal to 4, the lossless ACT transform is applied; otherwise, the lossy version of the ACT transform is applied, as shown below.

[0130] if transform_skip_flag If the value is equal to 1 and QP is equal to 4, then the residual sample r Y r Cb and r Cr The (nTbW)x(nTbH) array (where x = 0..nTbW-1, y = 0..nTbH – 1) is modified as follows: tmp = r Y [x][y] – (r) Cb [x][y] >> 1) r Y [x][y] = tmp + r Cb [ x ][ y ] r Cb [x][y] = tmp – (r) Cr [x][y] >> 1) r Cr [x][y] = r Cb [x][y] + r Cr [ x ][ y ]

[0131] Otherwise, the residual sample r Y r Cb and r Cr The (nTbW)x(nTbH) array (where x = 0..nTbW-1, y =0..nTbH-1) is modified as follows: tmp = r Y [x][y] - r Cb [ x ][ y ] r Y [x][y] = r Y[x][y] + r Cb [ x ][ y ] r Cb [x][y] = tmp – r Cr [ x ][ y ] r Cr [x][y] = tmp + r Cr [ x ][ y ]

[0132] In the above description, different ACT transform matrices are used for lossy and lossless coding. For a more unified design, a lossless ACT transform matrix is ​​used for both lossy and lossless coding. Furthermore, it is assumed that the lossless ACT transform will increase the dynamic range of the Cg and Co components by 1 bit; after the forward ACT transform, an additional 1-bit right shift is applied to the Cg and Co components, while before the inverse ACT transform, a 1-bit left shift is applied to the Cg and Co components. This is described below.

[0133] If transform_skip_flag equals 0 or QP is not equal to 4, then the residual sample r Cb and r Cr The (nTbW)x(nTbH) array (where x = 0..nTbW-1, y = 0..nTbH-1) is modified as follows: r Cb [x][y] = r Cb [x][y]<<1 r Cr [x][y] = r Cr [x][y]<<1

[0134] residual sample r Y r Cb and r Cr The (nTbW)x(nTbH) array (where x = 0..nTbW-1, y = 0..nTbH-1) is modified as follows: tmp = r Y [x][y] – (r) Cb [x][y] >> 1) r Y [x][y] = tmp + r Cb [ x ][ y ] r Cb [x][y] = tmp – (r) Cr [x][y] >> 1) r Cr [x][y] = r Cb [x][y] + r Cr [ x ][ y ]

[0135] Furthermore, as can be seen above, when applying ACT, the QP offset (-5, -5, -3) is applied to the Y, Cg, and Co components. Therefore, for smaller input QP values ​​(e.g., <5), negative QP will be used for the quantization / dequantization of undefined ACT transform coefficients. To address this, a clipping operation is added after the QP adjustment in ACT, ensuring that the applied QP value is always equal to or greater than zero, i.e., QP' = max(QP... org - QP offset , 0), where QP is the original QP, QP offset It is the ACT QP offset, where QP' is the adjusted QP value.

[0136] In the method described above, although the same ACT transform matrix (i.e., the lossless ACT transform matrix) is used for both lossy and lossless encoding, the following two problems can still be identified:

[0137] Depending on whether the current CU is a lossy or lossless CU, different inverse ACT operations are still applied. Specifically, for a lossless CU, an inverse ACT transform is applied; for a lossy CU, an additional right shift needs to be applied before the inverse ACT transform. Furthermore, the decoder needs to know whether the current CU is encoded in lossy or lossless mode. This is inconsistent with the current lossless design of VVC. In detail, this is related to the method of sending a signal... cu_transquant_bypass_flag Unlike HEVC lossless design, which indicates the lossless mode of a CU, lossless coding in VVC is done in a purely non-canonical way, that is, skipping the transform of the prediction residual (for the luminance and chrominance components, the transform skip mode is enabled), choosing an appropriate QP value (i.e., 4), and explicitly disabling coding tools that avoid lossless coding, such as loop filters.

[0138] The QP offset used for the standardized ACT transform is currently fixed. However, the optimal QP offset selection can depend on the content itself in terms of codec efficiency. Therefore, allowing flexible QP offset signaling when ACT tools are enabled may be more beneficial in order to maximize their codec gain.

[0139] Based on the above considerations, a unified ACT design is implemented as follows. First, lossless ACT forward and inverse transforms are applied to CUs encoded in both lossy and lossless modes. Second, instead of using fixed QP offsets, the QP offsets applied to the ACT CUs are explicitly signaled in the bitstream (i.e., three QP offsets applied to the Y, Cg, and Co components). Third, to avoid potential overflow issues with the QPs applied to the ACT CUs, a pruning operation is applied to the resulting QPs of each ACT CU to ensure the QP range is valid. It can be seen that, based on the above method, the choice between lossy and lossless encoding can be achieved simply by modifying the encoder (i.e., using different encoder settings). The decoding operations for lossy and lossless encoding of the ACT CUs are the same. Specifically, to achieve lossless encoding, in addition to the existing lossless encoder-side configuration, the encoder only needs to signal three QP offsets with zero values. On the other hand, to enable lossy encoding, the encoder can signal non-zero QP offsets. For example, in one embodiment, to compensate for dynamic range variations caused by the lossless ACT transform in lossy coding, QP offsets (-5, 1, 3) can be signaled for the Y, Cg, and Co components when applying ACT. Alternatively, ACT QP offsets can be signaled at different coding levels, such as Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header, Code Block Group level, etc., which can provide different QP adaptations at different granularities. The table below provides an example where QP offset signaling is performed in SPS. Table 10 shows the syntax specifications for executing QP offset signaling in SPS.

[0140] In another embodiment, an advanced control flag (e.g., at SPS or PPS) is added. picture_header_ act_qp_offset_present_flag When the flag is equal to 0, this means that the QP offset signaled in SPS or PPS will be applied to all CUs encoded and decoded in ACT mode. Otherwise, when the flag is equal to 1, additional QP offset syntax (e.g., picture_header_y_qp_offset_plus5, picture_header_cg_qp_offset_minus1, and picture_header_co_qp_offset_minus3) can be further signaled in the picture header to individually control the QP value applied to the ACT CU in a specific picture.

[0141] On the other hand, the QP offset transmitted via signaling should also be applied to trim the final ACT QP value to the effective dynamic range. Furthermore, different trimming ranges can be applied to CUs that use transforms and those that do not. For example, when no transform is applied, the final QP should be no less than 4. Assuming the ACT QP offset is transmitted via signaling at the SPS level, the derivation of the corresponding QP value for the ACT CU can be described as follows: QpY = ( ( qPY_PRED + CuQpDeltaVal QpBdOffset + sps_act_y_qp_offset ) % ( 64 + QpBdOffset ) ) - QpBdOffset Qp′ Cb = Clip3( -QpBdOffset, 63, qP Cb + pps_cb_qp_offset + slice_cb_qp_offset + CuQpOffset Cb + sps_act_cg_offset) + QpBdOffset Qp′ Cr = Clip3( -QpBdOffset, 63, qP Cr + pps_cr_qp_offset + slice_cr_qp_offset + CuQpOffset Cr + sps_act_co_offset) + QpBdOffset Qp′ CbCr = Clip3( -QpBdOffset, 63, qP CbCr + pps_joint_cbcr_qp_offset + slice_joint_cbcr_qp_offset +CuQpOffset CbCr + sps_act_cg_offset) +QpBdOffset

[0142] In another embodiment, the ACT enable / disable flag is signaled at the SPS level, while the ACT QP offset is signaled at the PPS level. This allows the encoder to more flexibly adjust the QP offset applied to the ACT CU, thereby improving coding efficiency. Specifically, the SPS and PPS syntax tables with the described changes are presented in the table below. Table 11 shows the syntax for signaling the ACT enable / disable flag at the SPS level and signaling the ACT QP offset at the PPS level.

[0143] A value of 1 for `pps_act_qp_offset_present_flag` indicates the presence of `pps_act_y_qp_offset_plus5`, `pps_act_cg_qp_offset_minus1`, and `pps_act_co_qp_offset_minus3` in the bitstream. When `pps_act_qp_offset_present_flag` is 0, these syntax elements do not exist in the bitstream. Bitstream consistency is maintained by ensuring that when `sps_act_enabled_flag` is 0, the value of `pps_act_qp_offset_present_flag` should be 0.

[0144] pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_co_qp_offset_minus3 are used to determine the offset of the values ​​applied to the quantization parameters for the luma and chroma components of the coded block whose cu_act_enabled_flag is equal to 1. When these parameters are not present, the values ​​of pps_act_y_qp_offset_plus5, pps_act_cg_qp_offset_minus1, and pps_act_cr_qp_offset_minus3 are inferred to be equal to 0.

[0145] In the aforementioned PPS signaling, the same QP offset value is applied to the ACT CU whether the Joint Coding of Chromaticity Residues (JCCR) mode is applied or not. Assuming that only the residual of one signal chroma component is encoded in JCCR mode, this design may not be optimal. Therefore, to obtain a better coding gain, a different QP offset can be applied to encode the residual of the chroma component when JCCR mode is applied to an ACT CU. Based on this consideration, a separate QP offset signaling is added to the PPS for JCCR mode, as follows. Table 12 shows the syntax specification for adding a separate QP offset signaling for JCCR mode in PPS.

[0146] `pps_joint_cbcr_qp_offset` is used to determine the offset applied to the value of the quantization parameter, which is used for the chroma residual of the coded block in which joint chroma residual coding is applied. If it does not exist, the value of `pps_joint_cbcr_qp_offset` is inferred to be 0.

[0147] Figure 10 This is a flowchart 1000 illustrating an exemplary process according to some embodiments of the present disclosure, through which a video encoder decodes video data by using the same inverse adaptive color space transformation (ACT) for encoding units encoded in lossy and lossless modes.

[0148] In some embodiments, the video decoder 30 receives a first control flag in the syntax from the bitstream, wherein the first control flag indicates whether adaptive color transformation (ACT) is enabled to decode the residuals (1010) of the coding units in the bitstream.

[0149] In some embodiments, the video decoder 30 then determines that ACT is enabled to decode the residuals (1020) of the encoded units in the bitstream based on a first control flag (e.g., if the first control flag value is true or non-zero).

[0150] According to the determination in step 1020, the video decoder 30 receives multiple syntax elements (1020-1) from the bitstream. These syntax elements indicate adjustment values ​​applied to quantization parameters used to inverse-quantize the residuals of the coding units using ACT. Next, the video decoder 30 receives video data (1020-2) corresponding to the coding units from the bitstream. The video data includes coefficients corresponding to the residuals of the coding units and a second control flag in the syntax indicating whether the residuals of the coding units have been encoded in the YCgCo color space using ACT.

[0151] The video decoder 30 constructs the residual (1020-3) of the coding unit by applying inverse quantization and inverse transform to the coefficients corresponding to the residual of the coding unit.

[0152] In some embodiments, the video decoder 30 determines, based on a second control flag (e.g., if the value of the second control flag is true or non-zero), that the residual of the coding unit has been encoded in the YCgCo color space using ACT (1030).

[0153] Based on the determination in step 1030 that the residual of the coding unit has been encoded in the YCgCo color space using ACT, the video decoder 30 modifies the reconstructed residual of the coding unit by applying inverse ACT (1030-1). Inverse ACT uses an inverse ACT matrix to transform the reconstructed residual of the coding unit from the YCgCo color space back to the original color space. In some embodiments, the inverse ACT matrix is ​​used to transform the reconstructed residual of the coding unit encoded in both lossy and lossless modes. In other words, the same inverse ACT matrix is ​​used to transform the reconstructed residual regardless of whether the residual of the coding unit is encoded in lossy or lossless mode.

[0154] In some embodiments, based on the determination of encoding the coding unit using a lossy coding mode, inverse quantization and inverse transform are applied to the coefficients corresponding to the residuals of the received coding unit.

[0155] In some embodiments, after applying the adjustment value to the quantization parameter, the video decoder 30 applies a cropping operation to further restrict the adjusted quantization parameter to a valid range. In some embodiments, the valid range is equal to or greater than zero.

[0156] In some embodiments, the adjustment value applied to the quantization parameter is predetermined based on the change in the magnitude of the residual of the encoding unit before and after the application of inverse ACT.

[0157] In some embodiments, the adjustment values ​​applied to the quantization parameters include a first quantization parameter adjustment for the Y component, a second quantization parameter adjustment for the Cg component, and a third quantization parameter adjustment for the Co component. In some embodiments, the adjustment values ​​applied to the quantization parameters further include a fourth quantization parameter adjustment for the color components of the coding unit encoded via the Joint Chromaticity Residual Coding (JCCR) mode.

[0158] In some embodiments, video data is captured in a 4:4:4 chroma format prior to encoding.

[0159] In some embodiments, a first control flag and multiple syntax elements are executed at one or more of the Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header, and Code Block Group level.

[0160] In some embodiments, the decoder receives a first syntax element from video data corresponding to the coding unit, wherein the first syntax element indicates whether the residual of the coding unit has been encoded using ACT; based on the determination from the first syntax element that the residual of the coding unit has not yet been encoded using ACT, the decoder receives a second syntax element indicating whether to apply an inverse transform to decode the residual of the coding unit; and based on the determination from the first syntax element that the residual of the coding unit has been encoded using ACT, the decoder assigns a default value to the second syntax element, the default value indicating that an inverse transform should be applied to decode the residual of the coding unit.

[0161] In some embodiments, Figure 11 This demonstrates a method for handling ACT when the internal luminance and chroma bit depths differ. Specifically, Figure 11 This is a decoding flowchart illustrating the application of different QP offsets to different components when the internal bit depths of luminance and chrominance differ, according to some embodiments of this disclosure.

[0162] According to the existing VVC specification, the luminance and chrominance components are allowed to use different internal bit depths (denoted as BitDepth). Y and BitDepth C Encoding and decoding are performed. However, existing ACT designs always assume that the internal luma and chroma bit depths are the same. In this section, the method is implemented such that when BitDepth... Y Not equal to BitDepth C Improve ACT design in time.

[0163] In the first approach, the ACT tool is always disabled when the internal luminance bit depth is not equal to the chrominance component.

[0164] The second method aligns the bit depths of the luminance and chrominance components by left-shifting the component with the smaller bit depth to match the bit depth of the other component; then, the scaled component is readjusted to its original bit depth by right-shifting after color transformation.

[0165] Similar to HEVC, the quantization step size increases by approximately 2 with each increment of QP. 1⁄6 The value is doubled every 6 increments. Based on this design, in the second method, to compensate for the internal bit depth difference between luminance and chrominance, it increases the QP value for the component with the smaller internal bit depth by 6Δ, where Δ is the difference between the luminance and chrominance internal bit depths. Then, by applying a right shift of Δ bits, the residual of the component is shifted back to the original dynamic range. Figure 11The corresponding decoding process when the above method is applied is illustrated. For example, assuming the input QP value is qp, the default QP values ​​applied to the Y, Cg, and Co components are qp-5, qp-5, and qp-3, respectively. Furthermore, it is assumed that the internal bit depth of the luminance is higher than the bit depth of the chrominance, i.e., Δ = BitDepth. Y –BitDepth C Then, the final QP values ​​applied to the luminance and chrominance components are equal to qp-5, qp-5+6Δ, and qp-3+6Δ.

[0166] In some embodiments, encoder acceleration logic is implemented. To select the color space for residual coding of a CU, the most straightforward approach is to have the encoder check each coding mode (e.g., intra-frame coding mode, inter-frame coding mode, and IBC mode) twice, once with ACT enabled and the other time with ACT disabled. This can approximately double the coding complexity. To further reduce the coding complexity of ACT, the following encoder acceleration logic is implemented in this disclosure:

[0167] First, because the YCgCo color space is more compact than the RGB color space, when the input video is in RGB format, a rate-distortion (RD) cost check is performed first with the ACT tool enabled, and then with the RD cost checked with the ACT tool disabled. Furthermore, the RD cost calculation for disabling color space transformation is only performed if at least one non-zero coefficient is present when ACT is enabled. Alternatively, when the input video is in YCbCr format, the RD cost check for disabling ACT is performed after the ACT-enabled RD check. A second RD check is performed only if at least one non-zero coefficient is present when ACT is disabled (i.e., ACT is enabled).

[0168] Second, to reduce the number of test coding modes, the same coding mode was implemented for both color spaces. More specifically, for intra-frame modes, the selected intra-frame prediction mode used for full RD cost comparison is shared between the two color spaces; for inter-frame modes, the selected motion vectors, reference picture, motion vector predictor, and merge index (for inter-frame merge mode) are shared between the two color spaces; and for IBC modes, the selected block vectors, block vector predictor, and merge index (for IBC merge mode) are shared between the two color spaces.

[0169] Third, because VVC uses a quadtree / binary tree / ternary tree partitioning structure, the same block partition can be obtained through different partitioning combinations. To speed up color space selection, an ACT enable / disable decision is implemented when the same block is obtained through different partitioning paths. Specifically, when a CU is encoded for the first time, the selected color space for encoding the residual of a particular CU is stored. Then, when the same CU is obtained through another partitioning path, instead of choosing between the two spaces, the stored color space decision is directly reused.

[0170] Fourth, assuming a strong correlation between a CU and its spatial neighbors, it is implemented by using the color space selection information of its spatial neighbors to determine how many color spaces need to be checked for the residual encoding of the current CU. For example, if a sufficient number of spatial neighbors choose the YCgCo space to encode their residuals, it is reasonable to infer that the current CU is likely to choose the same color space. Accordingly, the RD check for encoding the residuals of the current CU in the original color space can be skipped. If enough spatial neighbors choose the original color space, the RD check for residual encoding in the YCgCo domain can be bypassed. Otherwise, both color spaces need to be tested.

[0171] Fifth, assuming strong correlation between color spaces (CUs) within the same region, a CU can choose the same color space as its parent CU to encode its residuals. Alternatively, a child CU can deduce its color space from information about its parent CU, such as the chosen color space and the RD cost of each color space. Therefore, to simplify encoding complexity, if the residuals of its parent CU are encoded in the YCgCo domain, the RD cost of the residual encoding in the RGB domain will be skipped for a CU; similarly, if the residuals of its parent CU are encoded in the RGB domain, the RD cost of the residual encoding in the YCgCo domain will be skipped. Another conservative approach is to use the RD cost of the parent CU in both color spaces if both color spaces are tested in the encoding of its parent CU. If the parent CU chooses the YCgCo color space, and the RD cost of YCgCo is much smaller than the RD cost of RGB, the RGB color space will be skipped, and vice versa.

[0172] In some embodiments, 4:4:4 video encoding / decoding efficiency is improved by enabling luma-only codecs for the chroma component. Since VVC is primarily designed for video captured in 4:2:0 chroma format, most existing inter-frame / intra-frame codecs are enabled only for the luma component and disabled for the chroma component. However, as discussed earlier, 4:4:4 chroma format video signals exhibit distinctly different characteristics compared to 4:2:0 video signals. For example, similar to the luma component, the Cb / B and Cr / R components of 4:4:4 YCbCr / RGB video typically contain useful high-frequency texture and edge information. This differs from the chroma component in 4:2:0 video, which is typically very smooth and contains far less information than the luma component. Based on this analysis, when the input video is in 4:4:4 chroma format, the following approach is implemented to extend some of the luma-only codecs in current VVC to the chroma component.

[0173] First, a luminance interpolation filter is enabled for the chroma component. Similar to HEVC, the VVC standard utilizes motion compensation prediction to leverage redundancy between temporally adjacent images. This motion compensation prediction technique supports motion vectors accurate to 1 / 16 of a pixel for the Y component and 1 / 32 of a pixel for the Cb and Cr components. A set of separable 8-tap filters is used to interpolate the fractional samples. The fractional interpolation of the Cb and Cr components is essentially the same as that of the Y component, except that a separable 4-tap filter is used for the 4:2:0 video format. This is because for 4:2:0 video, the Cb and Cr components contain significantly less information than the Y component, and the 4-tap interpolation filter reduces the complexity of fractional interpolation filtering compared to using an 8-tap filter without affecting the efficiency of motion compensation prediction for the Cb and Cr components.

[0174] As previously noted, existing 4-tap chroma interpolation filters may not be effective for interpolating fractional samples used for motion compensation prediction of the chroma components in 4:4:4 video. Therefore, in one embodiment of this disclosure, the same set of 8-tap interpolation filters (for the luma component in 4:2:0 video) is implemented for fractional sample interpolation of both the luma and chroma components in 4:4:4 video. In another embodiment, to better balance encoding / decoding efficiency and complexity, adaptive interpolation filter selection is enabled for chroma samples in 4:4:4 video. For example, an interpolation filter selection flag can be signaled at the SPS, PPS, and / or slice levels to indicate whether an 8-tap interpolation filter (or other interpolation filter) or the default 4-tap interpolation filter is used for the chroma components at various encoding / decoding levels.

[0175] Second, enable PDPC and MRL for the chromaticity components.

[0176] The Position-Related Intra-Prediction Combination (PDPC) tool in VVC extends this idea by employing a weighted combination of intra-predicted samples and unfiltered reference samples. In the current VVC working draft, PDPC is enabled for the following signalless intra-modes: planar, DC, horizontal (i.e., mode 18), vertical (i.e., mode 50), angular direction near the lower left diagonal (i.e., modes 2, 3, 4, ..., 10), and angular direction near the upper right diagonal (i.e., modes 58, 59, 60, ..., 66). Assuming the predicted sample at coordinates (x, y) is pred(x, y), its corresponding value after PDPC is calculated as follows: pred(x,y) = ( wL × R-1,y + wT × Rx,-1 – wTL × R-1,-1 + (64 – wL –wT + wTL) × pred(x,y) + 32 ) >> 6 Where Rx,-1, R-1,y represent the reference samples located above and to the left of the current sample (x, y), respectively, and R-1,-1 represents the reference sample located at the top left corner of the current block. The weights wL, wT, and wTL in the above formula are adaptively selected based on the prediction mode and sample position, as described below, where it is assumed that the current codec block has a size of W × H:

[0177] For DC mode, wT = 32>>((y<<1)>>shift), wL = 32>>((x<<1)>>shift), wTL =(wL>>4)+(wT>>4)

[0178] For planar mode, wT = 32>>((y<<1)>>shift), wL = 32>>((x<<1)>>shift), wTL = 0

[0179] For horizontal mode: wT = 32>>((y<<1)>>shift), wL = 32>>((x<<1)>>shift), wTL = wT

[0180] For vertical mode: wT = 32>>((y<<1)>>shift), wL = 32>>((x<<1)>>shift), wTL = wL

[0181] For the bottom left diagonal direction: wT = 16>>((y<<1)>>shift), wL = 16>>((x<<1)>>shift), wTL = 0

[0182] For the top right diagonal direction: wT = 16>>((y<<1)>>shift), wL = 16>>((x<<1)>>shift), wTL = 0 Where shift = (log2(W) – 2 + log2(H) – 2 + 2) >> 2.

[0183] Unlike HEVC, which uses only the nearest row / column of the reconstructed sample as a reference, VVC introduces multiple reference lines (MRLs), where two additional rows / columns are used for intra-frame prediction. The index of the selected reference row / column is sent from the encoder to the decoder via a signal. When a non-nearest row / column is selected, planar modes and DC modes are excluded from the set of intra-frame modes available for predicting the current block.

[0184] In current VVC designs, the PDPC tool is used solely by the luma component to reduce / remove discontinuities between intra-predicted samples and reference samples derived from reconstructed neighboring samples. However, as mentioned above, chroma blocks in 4:4:4 chroma format video signals may contain rich texture information. Therefore, tools like PDPC, which use a weighted average of unfiltered reference samples and intra-predicted samples to improve prediction quality, should also benefit the efficiency of chroma encoding and decoding in 4:4:4 video. Based on this consideration, in one embodiment of this disclosure, a PDPC-enabled process is implemented to perform intra-prediction of chroma components in 4:4:4 video.

[0185] The same considerations can be extended to MRL tools. In current VVCs, MRL cannot be applied to chroma components. Based on embodiments of this disclosure, MRL for chroma components of 4:4:4 video is enabled by signaling an MRL index for the chroma components of an intra-frame CU. Different methods can be used based on this embodiment. In one method, an additional MRL index can be signaled and shared by the Cb / B and Cr / R components. In another method, two MRL indices are signaled, one for each chroma component. In a third method, the luma MRL index is reused for intra-frame prediction of the chroma components, thus eliminating the need for additional MRL signaling to enable MRL for the chroma components.

[0186] Third, enable ISP for the chroma component.

[0187] In some embodiments, a coding / decoding tool called Sub-Segmentation Prediction (ISP) is introduced into VVC to further improve intra-frame coding / decoding efficiency. Traditional intra-frame modes utilize only reconstructed samples adjacent to a CU to generate intra-frame prediction samples for that block. Based on this design, the spatial correlation between prediction samples and reference samples is roughly proportional to the distance between them. Therefore, samples inside the block (especially those located in the lower right corner) typically have worse prediction quality than samples near the block boundary. Depending on the block size, the ISP divides the current CU into 2 or 4 sub-blocks horizontally or vertically, with each sub-block containing at least 16 samples. Reconstructed samples in one sub-block can be used as a reference for predicting samples in the next sub-block. This process is repeated until all sub-blocks within the current CU have been encoded / decoded. Furthermore, to reduce signaling overhead, all sub-blocks within an ISP CU share the same intra-frame mode. Additionally, according to existing ISP designs, sub-block segmentation applies only to the luma component. Specifically, the luma samples of an ISP CU can be further divided into multiple sub-blocks (or TUs), and each luma sub-block can be encoded / decoded separately. However, the chroma samples of the ISP CU were not segmented. In other words, for the chroma components, the CU was used as a processing unit for intra-frame prediction, transform, quantization, and entropy encoding / decoding without further segmentation.

[0188] In current VVC, when ISP mode is enabled, TU segmentation is applied only to luma samples, while chroma samples are encoded / decoded without further segmentation into multiple TUs. According to one embodiment of this disclosure, due to the rich texture information in the chroma plane, ISP mode is also enabled for chroma encoding / decoding in 4:4:4 video. Based on this embodiment, different methods can be employed. In one method, an additional ISP index is signaled and shared by both chroma components. In another method, two additional ISP indices are signaled separately, one for Cb / B and the other for Cr / R. In a third method, the ISP index used for the luma component is reused for ISP prediction of the two chroma components.

[0189] Fourth, matrix-based intra-frame prediction (MIP) is enabled for the chrominance component as a new intra-frame prediction technique.

[0190] To predict samples for a rectangular block of width W and height H, MIP takes H reconstructed adjacent boundary samples from the left row of the block and W reconstructed adjacent boundary samples from the top row of the block as input. If reconstructed samples are unavailable, they are generated in the manner used in regular intra-frame prediction.

[0191] In some embodiments, MIP mode is enabled only for the luma component. For the same reason as enabling ISP mode for the chroma component, in one embodiment, MIP is enabled for the chroma component of 4:4:4 video. Two signaling methods can be applied. In a first method, two MIP modes are signaled separately, one for the luma component and the other for both chroma components. In a second method, a single MIP mode shared by both the luma and chroma components is signaled only.

[0192] Fifth, enable Multi-Transform Selection (MTS) for the chromaticity component.

[0193] In addition to DCT-II used in HEVC, the MTS scheme is used for residual coding and decoding of both inter-frame and intra-frame codec blocks. It uses several selected transforms from DCT8 / DST7. The newly introduced transform matrices are DST-VII and DCT-VIII.

[0194] In the current VVC, the MTS tool is enabled only for the luma component. In one embodiment of this disclosure, MIP is enabled for the chroma component of 4:4:4 video. Two signaling methods can be applied. In the first method, when MTS is enabled for a CU, two transform indices are signaled separately, one for the luma component and the other for both chroma components. In the second method, when MTS is enabled, only a single transform index is signaled, which is shared by the luma and chroma components.

[0195] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium or a communication medium, where a computer-readable storage medium corresponds to a tangible medium such as a data storage medium, and a communication medium includes any medium that facilitates (e.g., according to a communication protocol) the transfer of a computer program from one place to another. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the embodiments described herein. A computer program product may include a computer-readable medium.

[0196] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0197] It will also be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0198] The description in this application is presented for illustrative and descriptive purposes and is not intended to be exhaustive or limited to the invention in the forms disclosed. Many modifications, variations, and alternative embodiments will become apparent to those skilled in the art from the teachings presented in the foregoing description and the accompanying drawings. The embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention and to best utilize the basic principles and various embodiments with modifications suitable for the intended particular use. Therefore, it will be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method of decoding video data, comprising: receiving a first control flag from a bitstream, wherein the first control flag indicates whether adaptive color transform (ACT) is enabled for a coding unit in the bitstream; receiving video data corresponding to the coding unit in the bitstream from the bitstream; obtaining coefficients corresponding to a residual of the coding unit from the video data; and in accordance with a determination, based on the first control flag, that the ACT is enabled for the coding unit in the bitstream: receiving a second control flag from the video data that indicates whether the residual of the coding unit has been encoded in a YCgCo color space using the ACT; determining a plurality of adjustment values from the bitstream and applying the plurality of adjustment values to a quantization parameter to obtain an adjusted quantization parameter for dequantizing the obtained coefficients of the coding unit; reconstructing the residual of the coding unit by applying dequantization to the obtained coefficients corresponding to the residual of the coding unit using the adjusted quantization parameter, and applying an inverse transform to the dequantized coefficients; and in accordance with a determination, based on the second control flag, that the residual of the coding unit has been encoded using the ACT, modifying the reconstructed residual of the coding unit by applying an inverse ACT to convert the reconstructed residual of the coding unit from the YCgCo color space to an original color space based on an inverse ACT matrix.

2. The method of claim 1, wherein: in accordance with a determination that the coding unit is encoded using a lossy encoding mode, applying the dequantization and inverse transform.

3. The method of claim 1, further comprising: applying a clipping operation to further limit the adjusted quantization parameter to a valid range after applying the adjustment values to the quantization parameter.

4. The method of claim 3, wherein, a minimum value in the valid range is equal to or greater than zero.

5. The method of claim 1, wherein, the adjustment values applied to the quantization parameter are predetermined based on a change in magnitude of the residual of the coding unit before and after applying the inverse ACT.

6. The method of claim 2, wherein, the adjustment values applied to the quantization parameter include a first quantization parameter adjustment for a Y component, a second quantization parameter adjustment for a Cg component, and a third quantization parameter adjustment for a Co component.

7. The method of claim 6, wherein, the adjustment values applied to the quantization parameter further include a fourth quantization parameter adjustment for color components of a coding unit encoded by a joint chrominance residual coding (JCCR) mode.

8. The method of claim 1, wherein, the video data is captured in a 4:4:4 chroma format before encoding.

9. The method of claim 1, wherein, the inverse ACT matrix is used to convert the reconstructed residual of the coding unit encoded in a lossy mode or a lossless mode.

10. The method of claim 1, wherein, the first control flag is signaled in one or more of a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, and a coding block group level.

11. A method of decoding video data, comprising: receiving a first syntax element from video data corresponding to a coding unit, wherein the first syntax element indicates whether a residual of the coding unit has been encoded using adaptive color transform (ACT); in accordance with a determination, based on the first syntax element, that the residual of the coding unit has not been encoded using the ACT, receiving a second syntax element that indicates whether inverse transform is applied to decode the residual of the coding unit; and in accordance with a determination, based on the first syntax element, that the residual of the coding unit has been encoded using the ACT, assigning a default value to the second syntax element, the default value indicating that inverse transform is applied to decode the residual of the coding unit.

12. An electronic device, comprising: one or more processing units; memory coupled to the one or more processing units; and a plurality of programs stored in the memory that, when executed by the one or more processing units, cause the electronic device to perform the method of any of claims 1-11.

13. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic apparatus having one or more processing units, wherein, The plurality of programs, when executed by the one or more processing units, cause the electronic device to perform the method of any of claims 1-11.

14. A computer program product, comprising at least one instruction that, when executed by at least one processor, causes the at least one processor to perform the method of any of claims 1-11.

Images