Lossless coding mode for video coding

Abstract

An electronic device performs a method of encoding and decoding video data. The method includes receiving, from a video bitstream, a first syntax element for a current picture, receiving, from the video bitstream, a second syntax element for the current picture according to a determination that the first syntax element indicates that residual coding for transform skip is not disabled for the current picture, setting the second syntax element to a default value that disables dependent quantization for the current picture according to a determination that the first syntax element indicates that residual coding for transform skip is disabled for the current picture, and performing residual decoding and inverse quantization for the current picture according to the first syntax element and the second syntax element.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 971,841, filed on February 7, 2020, entitled “LOSSLESS CODING MODES FOR VIDEOCODING,” which is incorporated herein by reference in its entirety. Technical Field

[0003] The present application relates generally to video data coding and compression, and more particularly to methods and systems for improved and simplified lossless coding for video coding. Background Art

[0004] Various electronic devices (such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc.) support digital video. Electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression standards defined by MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC) and Versatile Video Codec (VVC) standards. Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove redundancy inherent in video data. For block-based video codecs, a video frame is partitioned into one or more slices, each slice having multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or be recursively split into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf-CU) contains one or more transform units (TUs) and each CU also contains one or more prediction units (PUs). Each CU can be coded in intra, inter, or IBC mode. Video blocks in an intra-coded (I) slice of a video frame are coded using spatial prediction with respect to reference samples in neighboring blocks within the same video frame. Video blocks in an inter-coded (P or B) slice of a video frame can use spatial prediction with respect to reference samples in neighboring blocks within the same video frame or temporal prediction with respect to reference samples in other previous reference video frames and / or future reference video frames.

[0005] A prediction block for a current video block to be coded is derived based on either spatial or temporal prediction of previously coded reference blocks (e.g., neighboring blocks). The process of finding the reference blocks can be done through block matching algorithms. Residual data representing pixel differences between the current block to be coded and the prediction block is referred to as a residual block or prediction error. An inter-coded block is coded according to the residual block and a motion vector pointing to a reference block in the reference frame that forms the prediction block. The process of determining the motion vector is commonly referred to as motion estimation. An intra-coded block is coded according to an intra prediction mode and the residual block. To further compress, the residual block is transformed from the pixel domain to a transform domain (e.g., frequency domain), resulting in residual transform coefficients, which can then be quantized. The quantized transform coefficients, which are initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector, and then entropy coded into a video bitstream for even more compression.

[0006] The encoded video bitstream is then saved in a computer-readable storage medium (e.g., flash memory) to be accessed by another electronic device with digital video capabilities or directly transmitted to the electronic device wired or wirelessly. The electronic device then performs video decompression (which is the reverse of video compression described above), e.g., by parsing the encoded video bitstream to obtain syntax elements from the bitstream, and reconstructing the digital video data from the encoded video bitstream into its original format based at least in part on the syntax elements obtained from the bitstream, and the electronic device presents the reconstructed digital video data on a display of the electronic device.

[0007] As the quality of digital videos increases from high definition to 4K x 2K or even 8K x 4K, the amount of video data to be encoded / decoded grows exponentially. It is a long-standing challenge in how to more efficiently code / decode video data while maintaining the image quality of the decoded video data. SUMMARY

[0008] This application describes implementations related to video data encoding and decoding, and more specifically, implementations related to improved and simplified methods and systems for lossless coding for video coding.

[0009] According to a first aspect of the present application, a method of decoding video data includes receiving, from a video bitstream, a first syntax element for a current picture, receiving, from the video bitstream, a second syntax element for the current picture according to a determination that the first syntax element indicates that residual coding for transform skip is not disabled for the current picture, setting the second syntax element to a default value that disables dependent quantization for the current picture according to a determination that the first syntax element indicates that residual coding for transform skip is disabled for the current picture, and performing residual decoding and inverse quantization for the current picture according to the first syntax element and the second syntax element.

[0010] According to a second aspect of the present application, an electronic device includes one or more processing units, memory, and a plurality of programs stored in the memory. The programs, when executed by the one or more processing units, cause the electronic device to perform the method of decoding video data as described above.

[0011] According to a third aspect of the present application, a non-transitory computer-readable storage medium stores a plurality of programs for execution by an electronic device having one or more processing units. The programs, when executed by the one or more processing units, cause the electronic device to perform the method of decoding video data as described above. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, included to provide a further understanding of the implementations and are incorporated in and constitute a part of this specification, illustrate the described implementations and together with the description serve to explain the basic principles. Like reference numerals refer to corresponding parts throughout.

[0013] Figure 1 is a block diagram illustrating an exemplary video encoding and decoding system, in accordance with some implementations of the present disclosure.

[0014] Figure 2 is a block diagram illustrating an exemplary video encoder, in accordance with some implementations of the present disclosure.

[0015] Figure 3 is a block diagram illustrating an exemplary video decoder, in accordance with some implementations of the present disclosure.

[0016] Figures 4A to 4E is a block diagram illustrating how frames are recursively partitioned into video blocks of different sizes and shapes, in accordance with some implementations of the present disclosure.

[0017] Figures 5A to 5B is a block diagram illustrating an example of transform coefficient coding using context coding and bypass coding, in accordance with some implementations of the present disclosure.

[0018] Figure 6 is a block diagram illustrating an exemplary process that relies on scalar quantization, in accordance with some implementations of the present disclosure.

[0019] Figure 7 is a block diagram illustrating an exemplary state machine for switching between two different scalar quantizers, in accordance with some implementations of the present disclosure.

[0020] Figure 8 is a flowchart illustrating an exemplary process of a video decoder decoding a current picture of a video bitstream, in accordance with some implementations of the present disclosure.

[0021] Figure 9 is a block diagram illustrating an example context adaptive binary arithmetic coding (CABAC) engine in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0022] Reference will now be made in detail to specific implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth in order to assist in understanding the subject matter presented herein. But it will be apparent to one of ordinary skill in the art that the subject matter can be practiced without these specific details, and that numerous alternatives can be implemented. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein can be implemented on many types of electronic devices having digital video capabilities.

[0023] Figure 1 is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel, in accordance with some embodiments of the present disclosure. As shown in Figure 1 System 10 includes a source device 12 that generates and encodes video data to be decoded at a later time by a destination device 14, as shown in

[0024] In some embodiments, destination device 14 can receive encoded video data to be decoded via a link 16. Link 16 can comprise any type of communication medium or device capable of moving the encoded video data from source device 12 to destination device 14. In one example, link 16 can comprise a communication medium to enable source device 12 to transmit encoded video data directly to destination device 14 in real-time. The encoded video data can be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium can comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet. The communication medium can include routers, switches, base stations, or any other equipment that can be useful to facilitate communication from source device 12 to destination device 14.

[0025] In some other embodiments, the encoded video data may be sent from the output interface 22 to a storage device 32. The encoded video data in the storage device 32 may then be accessed by the target device 14 via the input interface 28. The storage device 32 may include any of a variety of distributed or locally accessible data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device 32 may correspond to a file server or another intermediate storage device that can hold the encoded video data generated by the source device 12. The target device 14 may access the stored video data from the storage device 32 via streaming or downloading. The file server may be any type of computer capable of storing encoded video data and sending the encoded video data to the target device 14. Exemplary file servers include a web server (e.g., for a website), an FTP server, a network attached storage (NAS) device, or a local disk drive. Target device 14 may access the encoded video data via any standard data connection suitable for accessing encoded video data stored on a file server, including a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both. The transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0026] like Figure 1 As shown in , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include a source such as a video capture device (e.g., a video camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. As an example, if video source 18 is a camera of a security monitoring system, source device 12 and target device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding, and may be applied to wireless and / or wired applications.

[0027] Captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be sent directly to target device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored on storage device 32 for later access by target device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or a transmitter.

[0028] Target device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 can include a receiver and / or a modem and receive encoded video data over link 16. The encoded video data communicated over link 16, or provided on a storage device 32, can include a variety of syntax elements generated by video encoder 20 for use in decoding the video data by video decoder 30. Such syntax elements can be included in the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0029] In some implementations, target device 14 can include a display device 34, which can be an integrated display device and an external display device configured to communicate with target device 14. Display device 34 displays the decoded video data to a user, and can include any of a variety of display devices such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0030] Video encoder 20 and video decoder 30 can operate according to a proprietary standard or industry standard, such as VVC, HEVC, MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions thereof. It is to be understood that the present application is not limited to a specific video coding / decoding standard and can be applicable to other video coding / decoding standards. In general, it is contemplated that video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is also contemplated that video decoder 30 of target device 14 can be configured to decode video data according to any of these current or future standards.

[0031] Video encoder 20 and video decoder 30 can be implemented as any of a variety of suitable encoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware or any combinations thereof. When implemented partially in software, an electronic device can store instructions for the software in a suitable, non- volatile computer-readable medium and execute the instructions using one or more processors of the electronic device to perform the video coding / decoding operations disclosed in the present disclosure. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, either of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0032] Figure 2is a block diagram illustrating an example video encoder 20, in accordance with some implementations described in this application. Video encoder 20 can perform intra-prediction coding and inter-prediction coding of video blocks within a video frame. Intra-prediction coding relies on spatial prediction to reduce or remove spatial redundancy in the video data within a given video frame or picture. Inter-prediction coding relies on temporal prediction to reduce or remove temporal redundancy in the video data within adjacent video frames or pictures of a video sequence.

[0033] As Figure 2 As shown in

[0034] Video data memory 40 can store video data to be encoded by the components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer that stores reference video data used by video encoder 20 in encoding video data (e.g., in intra- or inter-prediction coding modes). Video data memory 40 and DPB 64 can be formed by any of a variety of memory devices, such as those described with respect to memory 34 of video source 18. In various examples, video data memory 40 can be on-chip with other components of video encoder 20, or off-chip relative to those components.

[0035] As Figure 2As shown in FIG. 1, after receiving video data, partitioning unit 45 within prediction processing unit 41 partitions the video data into video blocks. This partitioning can also include partitioning of video frames into slices, tiles, or other larger coding units (CUs) in accordance with a predefined splitting structure, such as a quadtree structure, associated with the video data. Video frames can be divided into multiple video blocks (or sets of video blocks referred to as tiles). Prediction processing unit 41 can select one of a plurality of possible predictive coding modes, such as one of a plurality of intra-prediction coding modes or one of a plurality of inter-prediction coding modes, for the current video block based on error results (e.g., rate and level of distortion). Prediction processing unit 41 can provide the resulting intra- or inter-predicted block to summer 50 to generate a residual block, and to summer 62 to reconstruct the coded block for use in later processing as part of a reference frame. Prediction processing unit 41 also provides syntax elements, such as motion vectors, intra-mode indicators, partitioning information, and other such syntax information, to entropy encoding unit 56.

[0036] To select an appropriate intra-prediction coding mode for a current video block, intra-prediction processing unit 46 within prediction processing unit 41 can perform intra-prediction coding of the current video block in relation to one or more neighboring blocks in the same frame as the current block being coded to provide spatial prediction. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-prediction coding of the current video block in relation to one or more predictive blocks in one or more reference frames to provide temporal prediction. Video encoder 20 can perform multiple coding passes, e.g., to select an appropriate coding mode for each block of video data.

[0037] In some implementations, motion estimation unit 42 determines an inter-prediction mode for a current video frame by generating motion vectors that indicate displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block within a reference video frame in accordance with a predetermined pattern within a sequence of video frames. Motion estimation performed by motion estimation unit 42 is a process that generates motion vectors that estimate motion of video blocks. For example, a motion vector can indicate displacement of a PU of a video block within a current video frame or picture relative to a prediction block (or other coding unit) within a reference frame that is relevant to a current block (or other coding unit) being coded within the current frame. The predetermined pattern can designate video frames in the sequence as P-frames or B-frames. Intra-BC unit 48 can determine vectors (e.g., block vectors) for intra-BC coding in a similar manner as motion vectors determined by motion estimation unit 42 for inter-prediction, or can utilize block vectors determined by motion estimation unit 42.

[0038] In terms of pixel differences, a prediction block is a block of a reference frame that is deemed to closely match a PU of the video block being coded, and pixel differences can be determined by a sum of absolute differences (SAD), a sum of squared differences (SSD), or other difference metrics. In some implementations, video encoder 20 can calculate values for sub-integer pixel positions of reference frames stored in DPB 64. For example, video encoder 20 can interpolate values for quarter-pel positions, eighth-pel positions, or other fractional-pel positions of a reference frame. Thus, motion estimation unit 42 can perform a motion search with respect to both full-pel positions and fractional-pel positions and output motion vectors with fractional-pel precision.

[0039] Motion estimation unit 42 calculates motion vectors for PUs of a video block in an inter-predicted coded frame by comparing a location of the PU to a location of a prediction block of a reference frame selected from a first reference frame list (List 0) or a second reference frame list (List 1), each of which identifies one or more reference frames stored in DPB 64. Motion estimation unit 42 sends the calculated motion vectors to motion compensation unit 44, which then sends to entropy encoding unit 56.

[0040] Motion compensation performed by motion compensation unit 44 can involve extracting or generating a prediction block based on the motion vectors determined by motion estimation unit 42. Upon receiving a motion vector for a PU of a current video block, motion compensation unit 44 can locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to summer 50. Summer 50 then forms a residual video block of pixel difference values by subtracting pixel values of the prediction block provided by motion compensation unit 44 from pixel values of the current video block being coded. The pixel difference values forming the residual video block can include luma difference components or chroma difference components, or both. Motion compensation unit 44 can also generate syntax elements associated with the video block of the video frame for use by video decoder 30 when decoding the video block of the video frame. The syntax elements can include, for example, syntax elements defining motion vectors used to identify the prediction block, any flags indicating the mode of prediction, or any other syntax information described herein. It is noted that motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are illustrated separately for conceptual purposes.

[0041] In some implementations, intra BC unit 48 can generate vectors and extract prediction blocks in a manner similar to that described above in connection with motion estimation unit 42 and motion compensation unit 44, but the prediction blocks are in the same frame as the current block being coded, and the vectors are referred to as block vectors rather than motion vectors. In particular, intra BC unit 48 can determine an intra prediction mode to be used for coding the current block. In some examples, intra BC unit 48 can code the current block using various intra prediction modes, e.g., during a separate encoding pass, and test their performance through rate-distortion analysis. Next, intra BC unit 48 can select an appropriate intra prediction mode to use among the various tested intra prediction modes and generate an intra mode indicator accordingly. For example, intra BC unit 48 can compute rate-distortion values for the various tested intra prediction modes using rate-distortion analysis, and select the intra prediction mode with the best rate-distortion characteristics among the tested modes as the appropriate intra prediction mode to use. Rate-distortion analysis generally determines the amount of distortion (or error) between a coded block and the original uncoded block that was coded to generate the coded block, and the bit rate (i.e., number of bits) used to generate the coded block. Intra BC unit 48 can compute the rate from the distortion and the rate for various coded blocks to determine which intra prediction mode exhibits the best rate-distortion values for the block.

[0042] In other examples, intra BC unit 48 can perform such functions for intra BC prediction according to the implementations described herein using motion estimation unit 42 and motion compensation unit 44 in whole or in part. In either case, for intra block copy, in terms of pixel difference, the prediction block can be a block that is deemed to closely match the block to be coded, the pixel difference can be determined by sum of absolute difference (SAD), sum of squared difference (SSD), or other difference metric, and the identification of the prediction block can include computing values for sub-integer pixel positions.

[0043] Whether the prediction block is from the same frame according to intra prediction or a different frame according to inter prediction, video encoder 20 can form pixel difference values by subtracting pixel values of the prediction block from pixel values of the current video block being coded, thereby forming a residual video block. The pixel difference values forming the residual video block can include both luma component differences and chroma component differences.

[0044] As an alternative to inter prediction performed by motion estimation unit 42 and motion compensation unit 44 or intra block copy prediction performed by intra BC unit 48, as described above, intra prediction processing unit 46 can intra predict the current video block. In particular, intra prediction processing unit 46 can determine an intra prediction mode to use for encoding the current block. To this end, intra prediction processing unit 46 may, for example, encode the current block using various intra prediction modes during a separate encoding pass, and intra prediction processing unit 46 (or, in some examples, a mode selection unit) can select an appropriate intra prediction mode to use from among the tested intra prediction modes. Intra prediction processing unit 46 can provide information indicating the selected intra prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 can encode information indicating the selected intra prediction mode into the bitstream.

[0045] After prediction processing unit 41 determines a prediction block for the current video block via inter prediction or intra prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block can be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform.

[0046] Transform processing unit 52 can send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting a quantization parameter. In some examples, quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 can perform the scan.

[0047] Following quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients using, for example, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), Probability Interval Partitioning Entropy (PIPE) coding or another entropy encoding methodology or technique, into a video bitstream. The encoded bitstream can then be transmitted to video decoder 30, or archived (e.g., stored) on a storage device 32 for later transmission to or retrieval by video decoder 30. Entropy encoding unit 56 can also entropy encode motion vectors and other syntax elements for the current video frame being encoded.

[0048] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain for generating a reference block used to predict other video blocks. As noted above, the motion compensation unit 44 can generate a motion compensated prediction block from one or more reference blocks of frames stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to calculate sub-integer pixel values for use in motion estimation.

[0049] The summer 62 adds the reconstructed residual block to the motion compensated prediction block generated by the motion compensation unit 44 to generate a reference block for storage in the DPB 64. The reference block can then be used by the intra BC unit 48, the motion estimation unit 42, and the motion compensation unit 44 as a prediction block to inter predict another video block in a subsequent video frame.

[0050] Figure 3 FIG. 1 is a block diagram illustrating an example video decoder 30, in accordance with some embodiments of the present application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, a summer 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra prediction unit 84, and an intra BC unit 85. The video decoder 30 can perform a decoding process substantially reciprocal to that described above in connection with the video encoder 20. Figure 2 The decoding process described in relation to the video encoder 20 is substantially reciprocal. For example, the motion compensation unit 82 can generate prediction data based on motion vectors received from the entropy decoding unit 80, while the intra prediction unit 84 can generate prediction data based on intra prediction mode indicators received from the entropy decoding unit 80.

[0051] In some examples, the units of the video decoder 30 can be tasked to perform embodiments of the present application. Moreover, in some examples, embodiments of the present disclosure can be dispersed among one or more of the plurality of units of the video decoder 30. For example, the intra BC unit 85 can perform embodiments of the present application, alone or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra prediction unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 can not include the intra BC unit 85, and the functionality of the intra BC unit 85 can be performed by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0052] Video data memory 79 can store video data to be decoded by the other components of video decoder 30, such as an encoded video bitstream. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source, such as a camera, via wired or wireless network communication of video data, or by accessing physical data storage media (e.g., a flash drive or hard disk). Video data memory 79 can include a coded picture buffer (CPB) that stores encoded video data from an encoded video bitstream. Decoded picture buffer (DPB) 92 of video decoder 30 stores reference video data for use in decoding video data by video decoder 30 (e.g., in intra- or inter-coding prediction modes). Video data memory 79 and DPB 92 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magneto resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For Figure 3 illustrative purposes, video data memory 79 and DPB 92 are depicted as two distinct components of video decoder 30. But it will be readily apparent to one of ordinary skill in the art that video data memory 79 and DPB 92 can be provided by the same memory device or separate memory devices. In some examples, video data memory 79 can be on-chip with other components of video decoder 30, or off-chip relative to those components.

[0053] During the decoding process, video decoder 30 receives an encoded video bitstream that represents encoded video frames and associated syntax elements. Video decoder 30 can receive the syntax elements at the video frame level and / or video block level. Entropy decoding unit 80 of video decoder 30 entropy decodes the bitstream to generate quantized coefficients, motion vectors, or intra-prediction mode indicators, among other syntax elements. Entropy decoding unit 80 then forwards the motion vectors and other syntax elements to prediction processing unit 81.

[0054] When a video frame is encoded as an intra-predicted coded (I) frame or intra coded prediction blocks in other types of frames, intra-prediction unit 84 of prediction processing unit 81 can generate prediction data for a video block of the current video frame based on the signaled intra-prediction mode and reference data from previously decoded blocks of the current frame.

[0055] When the video frame is encoded as an inter-frame prediction coded (i.e., B or P) frame, motion compensation unit 82 of prediction processing unit 81 generates one or more prediction blocks for a video block of the current video frame based on the motion vectors and other syntax elements received from entropy decoding unit 80. Each of the prediction blocks may be generated from a reference frame in one of the reference frame lists. Video decoder 30 may construct the reference frame lists, List 0 and List 1, using a default construction technique based on the reference frames stored in DPB 92.

[0056] In some examples, when a video block is encoded or decoded according to the intra BC mode described herein, intra BC unit 85 of prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other syntax elements received from entropy decoding unit 80. The prediction block may be within a reconstructed region of the same picture as the current video block, as defined by video encoder 20.

[0057] The motion compensation unit 82 and / or the intra BC unit 85 determine prediction information for a video block of the current video frame by parsing the motion vectors and other syntax elements, and then uses the prediction information to generate a prediction block for the current video block being decoded. For example, the motion compensation unit 82 uses some of the received syntax elements to determine the prediction mode (e.g., intra prediction or inter prediction) used to encode or decode the video block of the video frame, the inter prediction frame type (e.g., B or P), construction information for one or more of the reference frame lists for the frame, the motion vector for each inter-prediction-encoded video block of the frame, the inter prediction state for each inter-prediction-encoded video block of the frame, and other information used to decode the video block in the current video frame.

[0058] Similarly, intra BC unit 85 may use some of the received syntax elements, such as flags, to determine whether the current video block is predicted using intra BC mode, construction information of which video blocks of the frame are within the reconstruction region and should be stored in DPB 92, block vectors for each intra BC predicted video block of the frame, intra BC prediction status for each intra BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0059] Motion compensation unit 82 may also perform interpolation to calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters, as used during encoding of the video block by video encoder 20. In this case, motion compensation unit 82 may determine the interpolation filters used by video encoder 20 from received syntax elements and use these interpolation filters to generate the prediction block.

[0060] Inverse quantization unit 86 inverse quantizes the quantized transform coefficients provided in the bitstream and entropy decoded by entropy decoding unit 80, using the same quantization parameters that determine the degree of quantization calculated by video encoder 20 for each video block in the video frame. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0061] After the motion compensation unit 82 or the intra BC unit 85 generates a prediction block for the current video block based on the vector and other syntax elements, the adder 90 reconstructs the decoded video block for the current video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82 and the intra BC unit 85. An in-loop filter (not shown) may be located between the adder 90 and the DPB 92 to further process the decoded video block. The decoded video block in a given frame is then stored in the DPB 92, which stores reference frames for subsequent motion compensation of the next video block. The DPB 92, or a memory device separate from the DPB 92, may also store the decoded video for later presentation on a display device (e.g., Figure 1 on the display device 34).

[0062] In a typical video encoding and decoding process, a video sequence typically consists of an ordered set of frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other cases, a frame may be monochrome and therefore consist of only a two-dimensional array of luma samples.

[0063] like Figure 4A As shown in , the video encoder 20 (or more specifically, the segmentation unit 45) generates an encoded representation of a frame by first segmenting the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs ordered consecutively from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in a sequence parameter set so that all CTUs in a video sequence have the same size, one of 128×128, 64×64, 32×32, and 16×16. However, it should be noted that the present application is not necessarily limited to a specific size. As Figure 4BAs shown in FIG. 1, each CTU can include one coding tree block (CTB) of luma samples, two corresponding coding tree blocks of chroma samples, and syntax elements used to code the samples of the coding tree blocks. The syntax elements describe properties of different types of units that code the pixel blocks and how the video sequence can be reconstructed at the video decoder 30, including inter or intra prediction, intra prediction modes, motion vectors, and other parameters. In monochrome pictures or pictures with three separate color planes, a CTU can include a single coding tree block and syntax elements used to code the samples of the coding tree block. A coding tree block can be an NxN block of samples.

[0064] To achieve better performance, the video encoder 20 can recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination of the two, on the coding tree blocks of a CTU and divide the CTU into smaller coding units (CUs). As Figure 4C As depicted in FIG. 4, a 64x64 CTU 400 is first divided into four smaller CUs, each having a block size of 32x32. Among the four smaller CUs, CUs 410 and 420 are each divided into four CUs having a block size of 16x16. Two 16x16 CUs 430 and 440 are each further divided into four CUs having a block size of 8x8. Figure 4D A quad tree data structure showing the final result of the partitioning process of the CTU 400 as Figure 4C depicted in FIG. 4, each leaf node of the quad tree corresponds to one CU of various sizes ranging from 32x32 to 8x8. Similar to the binary tree data structure depicted in FIG. 3, each CU can include one coding block (CB) of luma samples and two corresponding coding blocks of chroma samples of the same size of a frame, and syntax elements used to code the samples of the coding blocks. In monochrome pictures or pictures with three separate color planes, a CU can include a single coding block and syntax elements used to code the samples of the coding block. It is noted that, Figure 4B Figure 4C and Figure 4D The quad tree partitioning depicted in FIG. 4 is for illustrative purposes only, and one CTU can be split into CUs based on quad / ternary / binary tree partitioning to adapt to varying local characteristics. In multi-type tree structure, one CTU is partitioned by a quad tree structure, and each quad tree leaf CU can be further partitioned by binary and ternary tree structures. As shown in FIG. 5, there are five types of partitioning, namely, quad partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning. Figure 4E

[0065] ​​In some implementations, video encoder 20 can further partition the coding block of a CU into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) block of samples on which the same prediction (inter or intra) is applied. A prediction unit (PU) of a CU can comprise a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax elements used to perform the prediction on the prediction block. In a monochrome picture or a picture having three separate color planes, a PU can comprise a single prediction block and syntax structures used to perform the prediction on the prediction block. Video encoder 20 can generate prediction luma, Cb, and Cr blocks for each PU of a CU.

[0066] Video encoder 20 can use intra prediction or inter prediction to generate the prediction block for a PU. If video encoder 20 uses intra prediction to generate the prediction block for a PU, video encoder 20 can generate the prediction block for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter prediction to generate the prediction block for a PU, video encoder 20 can generate the prediction block for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0067] After video encoder 20 generates the prediction luma, Cb, and Cr blocks for one or more PUs of a CU, video encoder 20 can generate a luma residual block for the CU by subtracting the prediction luma blocks for the CU from the original luma coding block for the CU, such that each sample in the luma residual block for the CU indicates a difference between a luma sample in one of the prediction luma blocks for the CU and a corresponding sample in the original luma coding block for the CU. Similarly, video encoder 20 can generate Cb and Cr residual blocks for the CU, respectively, such that each sample in the Cb residual block for the CU indicates a difference between a Cb sample in one of the prediction Cb blocks for the CU and a corresponding sample in the original Cb coding block for the CU, and each sample in the Cr residual block for the CU can indicate a difference between a Cr sample in one of the prediction Cr blocks for the CU and a corresponding sample in the original Cr coding block for the CU.

[0068] Furthermore, as Figure 4CAs shown in , video encoder 20 may decompose the luma residual block, Cb residual block, and Cr residual block of a CU into one or more luma transform blocks, Cb transform blocks, and Cr transform blocks using quadtree partitioning. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax elements for transforming the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a subblock of the luma residual block of the CU. The Cb transform block may be a subblock of the Cb residual block of the CU. The Cr transform block may be a subblock of the Cr residual block of the CU. In a monochrome picture or a picture with three separate color planes, a TU may include a single transform block and a syntax structure for transforming the samples of the transform block.

[0069] Video encoder 20 may apply one or more transforms to the luma transform block of a TU to generate a luma coefficient block for the TU. A coefficient block may be a two-dimensional array of transform coefficients. A transform coefficient may be a scalar. Video encoder 20 may apply one or more transforms to the Cb transform block of a TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to the Cr transform block of a TU to generate a Cr coefficient block for the TU.

[0070] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to the process by which transform coefficients are quantized to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream comprising a sequence of bits forming a representation of an encoded frame and associated data, which is stored in storage device 32 or sent to target device 14.

[0071] After receiving the bitstream generated by video encoder 20, video decoder 30 can parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 can reconstruct the frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing the video data is generally reciprocal to the encoding process performed by video encoder 20. For example, video decoder 30 can perform inverse transforms on the coefficient blocks associated with the TUs of the current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs the coding blocks of the current CU by adding the samples of the prediction blocks for the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. After reconstructing the coding blocks for each CU of a frame, video decoder 30 can reconstruct the frame.

[0072] As described above, video coding primarily uses two modes (i.e., intra prediction (or intra-frame prediction) and inter prediction (or inter-frame prediction)) to achieve video compression. Palette-based coding is another coding scheme that has been adopted by many video coding standards. In palette-based coding, which can be particularly suitable for screen-generated content coding, a video coder (e.g., video encoder 20 or video decoder 30) forms a palette table that represents the colors of the video data of a given block. The palette table includes the most dominant (e.g., frequently used) pixel values in the given block. Pixel values that are not frequently represented in the video data of the given block are either not included in the palette table or are included in the palette table as escape colors.

[0073] Each entry in the palette table includes an index for a corresponding pixel value in the palette table. The palette index for a sample in a block can be coded to indicate which entry from the palette table is to be used to predict or reconstruct which sample. This palette mode begins with a process of generating a palette predictor for a first block of a picture, slice, tile, or other such grouping of video blocks. As will be explained below, the palette predictor for subsequent video blocks is typically generated by updating the previously used palette predictor. For purposes of illustration, assume that the palette predictor is defined at the picture level. In other words, a picture can include multiple coding blocks, each having its own palette table, but there is one palette predictor for the entire picture.

[0074] To reduce the bits needed to signal palette entries in a video bitstream, a video decoder can utilize a palette predictor to determine new palette entries in a palette table used to reconstruct a video block. For example, the palette predictor can include palette entries from a previously used palette table, or even be initialized by including all entries of a recently used palette table. In some implementations, the palette predictor can include fewer entries than all entries from a recently used palette table, and then incorporate some entries from other previously used palette tables. The palette predictor can have the same size as a palette table used to code a different block, or can be larger or smaller than a palette table used to code a different block. In one example, the palette predictor is implemented as a first-in-first-out (FIFO) table including 64 palette entries.

[0075] To generate a palette table for a block of video data from a palette predictor, a video decoder can receive from an encoded video bitstream a one-bit flag for each entry of the palette predictor. The one-bit flag can have a first value (e.g., binary one) indicating that the associated entry of the palette predictor is to be included in the palette table, or a second value (e.g., binary zero) indicating that the associated entry of the palette predictor is not to be included in the palette table. If the size of the palette predictor is larger than the palette table for the block of video data, the video decoder can stop receiving more flags once a maximum size for the palette table is reached.

[0076] In some implementations, some entries in the palette table can be directly signaled in the encoded video bitstream without being determined using a palette predictor. For such entries, the video decoder can receive from the encoded video bitstream three separate m-bit values indicating pixel values for a luma component and two chroma components associated with the entry, where m represents the bit depth of the video data. Those palette entries derived from the palette predictor only require a one-bit flag, as compared to the multiple m-bit values needed for directly signaled palette entries. Thus, signaling some or all of the palette entries using a palette predictor can significantly reduce the number of bits needed to signal the entries of a new palette table, thereby improving the overall coding efficiency of palette mode coding.

[0077] In many cases, the palette predictor for a block is determined based on the palette table used to encode one or more previously encoded blocks. But when coding the first coding tree unit in a picture, slice or tile, the palette table of previously coded blocks can not be available. Therefore, entries of the previously used palette table cannot be used to generate the palette predictor. In this case, a sequence of palette predictor initialization values can be signaled in the sequence parameter set (SPS) and / or the picture parameter set (PPS), which are values used to generate the palette predictor when the previously used palette table is not available. The SPS generally refers to a syntax structure of syntax elements that applies to a series of consecutive coded video pictures referred to as a coded video sequence (CVS), where the CVS is determined by the content of the syntax elements found in the PPS referred to by the syntax elements found in each slice segment header. The PPS generally refers to a syntax structure of syntax elements that applies to one or more individual pictures within a CVS, where the CVS is determined by the syntax elements found in each slice segment header. Therefore, the SPS is generally considered to be a more high-level syntax structure than the PPS, which means that the syntax elements included in the SPS generally change less frequently and apply to a larger portion of the video data than the syntax elements included in the PPS.

[0078] Figures 5A to 5B is a block diagram illustrating an example of transform coefficient coding using context coding and bypass coding according to some embodiments of the present disclosure.

[0079] Transform coefficient coding in VVC is similar to that in HEVC as they both use non-overlapping groups of coefficients (also referred to as CGs or sub-blocks). However, there are also some differences between the two schemes. In HEVC, each CG of coefficients has a fixed size of 4x4. In VVC Draft 6, the CG size becomes dependent on the TB size. Therefore, various CG sizes (1x16, 2x8, 8x2, 2x4, 4x2 and 16x1) are available in VVC. The CGs within a coding block and the transform coefficients within a CG are coded according to a predefined scan order.

[0080] To limit the maximum number of context coded bins (CCB) per pixel, the area of the TB and the type of video component (i.e. luma component vs. chroma component) are used to derive the maximum number of context coded bins (CCB) for the TB. In some embodiments, the maximum number of context coded bins is equal to TB_zosize x 1.75. Here, TB_zosize represents the number of samples within the TB after coefficient zero-out. Note that coded_sub_block_flag, which is a flag indicating whether a CG contains non-zero coefficients, is not considered for CCB counting.

[0081] Coefficient zero-out is an operation performed on a transform block such that coefficients located in a certain region of the transform block are set to zero. For example, in the current VVC, a 64x64 TB has an associated zero-out operation. Thus, transform coefficients located outside the top-left 32x32 region of the 64x64 TB are all forced to zero. In fact, in the current VVC, for any transform block whose size along a certain dimension exceeds 32, a coefficient zero-out operation is performed along that dimension such that coefficients located outside the top-left 32x32 region are zeroed.

[0082] In the transform coefficient coding in VVC, a variable remBinsPassl is first set to the maximum number of context-coded bins allowed (MCCB). During the coding process, this variable is decremented by one each time a context-coded bin is signaled. When remBinsPassl is greater than or equal to 4, coefficients are signaled with syntax elements including sig_coeff_flag, abs_level_gtl_flag, par_level_flag, and abs_level_gt3_flag, all of which use context-coded bins in the first pass. The rest of the coefficient’s level information is coded in the second pass using Golomb-Rice coding and bypass-coded bins with syntax element abs_remainer. When remBinsPassl becomes less than 4 while coding in the first pass, the current coefficient is not coded in the first pass but is directly coded in the second pass using Golomb-Rice coding and bypass-coded bins with syntax element dec_abs_level. After all the above level coding, the sign (sign_flag) for all scan positions with sig_coeff_flag equal to 1 is finally coded as bypass bins. The process is depicted in FIG. 1. Figure 5A The process is depicted in FIG. 1. remBinsPassl is reset for each TB. The transition from context-coded bins for sig_coeff_flag, abs_level_gtl_flag, par_level_flag, and abs_level_gt3_flag to bypass-coded bins for the rest of the coefficients happens at most once per TB. For a coefficient subblock, if remBinsPassl is less than 4 before coding its first coefficient, the entire coefficient subblock is coded using bypass-coded bins.

[0083] Unlike HEVC where a single residual codec scheme is designed for coding both transform coefficients and transform skip coefficients, in VVC, two separate residual codec schemes are used for transform coefficients and transform skip coefficients (ie, residuals), respectively.

[0084] For example, it is observed that the statistical characteristics of the residual in transform skip mode are different from those of the transform coefficients and there is no energy compression around low frequency components. The residual codec is modified to take into account the different signal characteristics of the (spatial) transform skip residual, which include:

[0085] (1) No signal is used to send the final x / y position;

[0086] (2) When all previous flags are equal to 0, coded_sub_block_flag is encoded and decoded for each sub-block except the DC sub-block;

[0087] (3) sig_coeff_flag context modeling with two adjacent coefficients;

[0088] (4) par_level_flag using only one context model;

[0089] (5) Additional signs greater than 5, 7, or 9;

[0090] (6) Derivation of modified Rice parameters for binarization of the remaining part;

[0091] (7) Determine context modeling for the sign flag based on the left and above neighboring coefficient values, and parse the sign flag after sig_coeff_flag to keep all context-coded bins together;

[0092] like Figure 5B As shown in , the syntax elements sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, par_level_flag are coded and decoded in an interleaved manner on a residual sample basis in the first pass, followed by coding and decoding the abs_level_gtX_flag bit plane in the second pass, and coding and decoding abs_remainder in the third pass.

[0093] Channel 1: sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, par_level_flag

[0094] Channel 2: abs_level_gt3_flag, abs_level_gt5_flag, abs_level_gt7_flag, abs_level_gt9_flag

[0095] Channel 3: abs_remainder

[0096] Figure 6 is a block diagram illustrating an exemplary process for dependent scalar quantization according to some embodiments of the present disclosure.

[0097] In current VVC, the maximum QP value is extended from 51 to 63, and the signaling of initial QP is changed accordingly. When a non-zero value of slice_qp_delta is coded, the initial value of SliceQpY can be modified at slice segment level. For transform skip blocks, the minimum allowed QP is defined as 4 because the quantization step size becomes 1 when QP equals 1.

[0098] In addition, the scalar quantization used in HEVC is adapted to a new concept called “dependent scalar quantization”. Dependent scalar quantization refers to a method in which the set of allowable reconstructed values for a transform coefficient depends on the values of transform coefficient levels preceding the current transform coefficient level in the reconstruction order. When compared to the regular independent scalar quantization used in HEVC, the allowable reconstructed vectors are more densely packed in the N-dimensional vector space (N denotes the number of transform coefficients in a transform block). That is, for a given average number of allowable reconstructed vectors per N-dimensional unit volume, the average distortion between the input vector and the closest reconstructed vector is reduced. The method of dependent scalar quantization is realized by (a) defining two scalar quantizers with different reconstruction levels, and (b) defining a process for switching between the two scalar quantizers.

[0099] The two scalar quantizers used, denoted by Q0 and Q1, are illustrated in Figure 6 The position of the available reconstruction levels is uniquely specified by the quantization step size Δ. The scalar quantizer used (Q0 or Q1) is not explicitly signaled in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity of the transform coefficient levels preceding the current transform coefficient in the coding or reconstruction order.

[0100] Figure 7 is a block diagram illustrating an exemplary state machine for switching between two different scalar quantizers according to some embodiments of the present disclosure.

[0101] As Figure 7As shown, the switching between the two scalar quantizers (Q0 and Q1) is implemented via a state machine with four quantizer states (QState). QState can take four different values: 0, 1, 2, 3. It is uniquely determined by the parity of the transform coefficient level preceding the current transform coefficient in the coding / reconstruction order. At the beginning of the dequantization for a transform block, the state is set equal to 0. The transform coefficients are reconstructed in the scan order, i.e., in the same order as they were entropy decoded. After the current transform coefficient is reconstructed, the state is updated as shown, where k denotes the value of the transform coefficient level. Figure 7

[0102] Signaling of default and user-defined scaling matrices is also supported. The default mode scaling matrices are all flat, with elements equal to 16 for all TB sizes. IBC and intra coding modes currently share the same scaling matrices. Therefore, for the case of user-defined (USER_DEFINED) matrices, the number of MatrixType and MatrixType DC is updated as follows:

[0103] MatrixType: 30 = 2 (2 for intra and IBC / inter) x 3 (Y / Cb / Cr components) x 5 (square TB sizes: from 4x4 to 64x64 for luma and from 2x2 to 32x32 for chroma)

[0104] MatrixType DC: 14 = 2 (2 for intra and IBC / inter x 1 for Y component) x 3 (TB sizes: 16x16, 32x32, 64x64) + 4 (2 for intra and IBC / inter x 2 for Cb / Cr components) x 2 (TB sizes: 16x16, 32x32)

[0105] The DC values are coded for the following scaling matrices, respectively: 16x16, 32x32, and 64x64. For TBs with size smaller than 8x8, all elements in one scaling matrix are signaled. If the size of the TB is greater than or equal to 8x8, only 64 elements in one 8x8 scaling matrix are signaled as the base scaling matrix. To obtain square matrices with size larger than 8x8, the 8x8 base scaling matrix is upsampled (by copying of elements) to the corresponding square size (i.e., 16x16, 32x32, 64x64). When zeroing of high frequency coefficients for 64-point transform is applied, the corresponding high frequencies of the scaling matrix are also zeroed. That is, if the width or height of the TB is greater than or equal to 32, only the left or top half of the coefficients are kept, and the remaining coefficients are assigned as zero. Furthermore, the number of elements signaled for the 64x64 scaling matrix is also reduced from 8x8 to three 4x4 sub-matrices, since the bottom right 4x4 elements are never used. ​

[0106] The selection of the probability model for the syntax element related to the absolute value of the transform coefficient level depends on the values of the absolute levels in the local neighborhood or the partially reconstructed absolute levels.

[0107] The selected probability model depends on the sum of the absolute levels in the local neighborhood (or the partially reconstructed absolute levels) and the number of absolute levels greater than 0 in the local neighborhood (given by the number of sig_coeff_flag equal to 1). The context modeling and binarization depends on the following metrics of the local neighborhood:

[0108] . numSig: the number of non-zero levels in the local neighborhood;

[0109] . sumAbsl: the sum of the partially reconstructed absolute levels (absLevell) after the first pass in the local neighborhood;

[0110] . sumAbs: the sum of the reconstructed absolute levels in the local neighborhood

[0111] . diagonal position (d): the sum of the horizontal and vertical coordinates of the current scan position within the transform block

[0112] Based on the values of numSig, sumAbsl and d, the probability models used to code sig_coeff_flag, abs_level_gtl_flag, par_level_flag and abs_level_gt3_flag are selected. Based on the values of sumAbs and numSig, the Rice parameters used to binarize abs_remainder and dec_abs_level are selected.

[0113] In the current VVC, the reduced 32-point MTS (also referred to as RMTS32) is based on skipping high frequency coefficients and is used to reduce the computational complexity of 32-point DST-7 / DCT-8. Also, it comes with a coefficient coding change including all types of zero-out (i.e., the existing zero-out for high frequency components in RMTS32 and DCT2). Specifically, the binarization of the last non-zero coefficient position coding is coded based on the reduced TU size and the context model selection for the last non-zero coefficient position coding is determined by the original TU size. In addition, sig_coeff_flag of the transform coefficients is coded using 60 context models. The selection of the context model index is based on the sum of the maximum five previously partially reconstructed absolute levels (referred to as locSumAbsPassl) and the state of the dependent quantization QState, as follows:

[0114] If cldx is equal to 0, ctxInc is derived as follows:

[0115] ctxInc = 12 x Max(0, QState - 1) + Min((locSumAbsPass1 + 1) » 1, 3) + (d < 2? 8 : (d < 5? 4 : 0))

[0116] Otherwise (cldx is greater than 0), ctxInc is derived as follows:

[0117] ctxInc = 36 + 8 x Max(0, QState - 1) + Min((locSumAbsPass1 + 1) » 1, 3) + (d < 2? 4 : 0)

[0118] In current VVC, a slice-level syntax element “slice_ts_residual_coding_disabled_flag” is employed to indicate whether to disable residual coding for transform skip at slice level, i.e., even if the current TU is in transform skip mode. For example, if slice_ts_residual_coding_disabled_flag is equal to 1, then for the current TU, residual coding for transform skip is disabled, and residual coding for non-transform skip mode is invoked. In addition, the transform coefficients in residual coding for non-transform skip mode depend on the quantizer state (QState). But since the dependent scalar quantization is applied specifically for transform skip mode, if residual coding for non-transform skip mode is applied to the current TU with transform skip mode, the dependent scalar quantization should be disabled.

[0119] In some embodiments, a picture header syntax element “ph_ts_residual_coding_disabled_flag” is employed to indicate whether to disable residual coding for transform skip mode at picture level. If ph_ts_residual_coding_disabled_flag is equal to 1, then residual coding for transform skip is disabled for the current picture and all slices of the current picture associated with the picture header. Therefore, since ph_ts_residual_coding_disabled_flag is equal to 1, the corresponding slice_ts_residual_coding_disabled_flag does not need to be signaled in the respective slice header associated with the picture header (which is inferred to be 1); if ph_ts_residual_coding_disabled_flag is equal to 0, the syntax element slice_ts_residual_coding_disabled_flag can still be signaled to indicate whether to enable residual coding for transform skip in the current slice.

[0120] In some embodiments, if residual coding for transform skip mode is disabled, then dependent scalar quantization is disabled (including the use of dependent quantization state to derive context modeling and rice parameters for parsing residual samples, as described in the Introduction section). In other words, if ph ts residual coding disabled flag is equal to 1, then ph dep quant enabled flag is set to 0. If ph dep quant enabled flag is equal to 1, then ph ts residual coding disabled flag is set to 0. In another example, if ph dep quant enabled flag is equal to 1, then slice ts residual coding disabled flag is set to 0. In yet another example, ph dep quant enabled flag and slice ts residual coding disabled flag can both be equal to 1; but for transform skip, the application of dependent quantization, dependent state context model and rice parameter derivation is ignored, e.g., check if the current TU is in transform skip mode, i.e.,!transform skip flag [x0] [y0] [cldx].

[0121] In some embodiments, the syntax element ph ts residual coding disabled flag equal to 1 indicates that residual coding for transform skip is disabled for the current picture and only residual coding for non-transform skip is invoked, and the syntax element ph ts residual coding disabled flag equal to 0 indicates that residual coding for transform skip is applied for the current picture. If the syntax element ph ts residual coding disabled flag is not present, it is inferred to be 0. The syntax element ph dep quant enabled flag equal to 0 indicates that dependent quantization is disabled for the current picture, and the syntax element ph dep quant enabled flag equal to 1 indicates that dependent quantization is enabled for the current picture. When the syntax element ph dep quant enabled flag is not present, it is inferred to be 0. The following table shows an example syntax design where the syntax element ph ts residual coding disabled flag is used to determine the presence of the syntax element ph dep quant enabled flag.

[0122]

[0123] In some embodiments, the inverse quantization method (including both dependent quantization and dependent state based context modeling and state dependent LPS parameter derivation) is enabled for transform skip mode. With this design, the following combinations are supported by changing the values of ph_dep_quant_enabled_flag and slice_ts_residual_coding_disabled_flag:

[0124] . When both flags are equal to 1, dependent quantization is applied to transform skip mode. In addition, residual coding for non-transform skip mode is applied to transform skip TUs together with dependent state based context modeling and dependent state based LPS parameter derivation

[0125] . When slice_ts_residual_coding_disabled_flag is equal to 1 and ph_dep_quant_enabled_flag is equal to 0. In this case, regular quantization method is applied to transform skip TUs. Meanwhile, residual coding for non-transform skip mode is applied to transform skip TUs without dependent state based context modeling and dependent state based LPS parameter derivation

[0126] . When slice_ts_residual_coding_disabled_flag is equal to 0 and ph_dep_quant_enabled_flag is equal to 1. In this case, dependent quantization is applied to transform skip mode and also residual coding for transform skip mode is applied to transform skip TUs, i.e., without dependent state based context modeling and LPS parameter derivation, etc. applied to transform skip TUs.

[0127] Figure 8 is a flowchart illustrating an exemplary process 800 of decoding, by a video decoder according to some embodiments of the present disclosure, a current picture of a video bitstream. As mentioned above, because dependent scalar quantization is applied to transform skip mode exclusively, if residual coding for non-transform skip mode is applied to a current picture with transform skip mode, dependent scalar quantization should be disabled.

[0128] In particular, a video decoder (e.g., video decoder 30) receives a first syntax element (e.g., ph ts residual coding disabled flag) of a current picture from a video bitstream (810). The video decoder then checks whether residual coding for transform skip is disabled for the current picture (820). According to a determination that the first syntax element indicates that residual coding for transform skip is not disabled for the current picture (820 - No), the video decoder then receives a second syntax element (e.g., ph dep quant enabled flag) of the current picture from the video bitstream (830); according to a determination that the first syntax element indicates that residual coding for transform skip is disabled for the current picture (820 - Yes), the video decoder sets the second syntax element to a default value that disables dependent quantization for the current picture (840). The video decoder then performs residual decoding and inverse quantization for the current picture according to the first syntax element and the second syntax element (850).

[0129] In some embodiments, the video decoder performs residual coding for non-transform skip for the current picture when the first syntax element indicates that residual coding for transform skip is disabled for the current picture (860); the video decoder performs residual coding for transform skip for the current picture when the first syntax element indicates that residual coding for transform skip is not disabled for the current picture or the first syntax element is not present in the video bitstream for the current picture (870).

[0130] In some embodiments, the video decoder disables dependent quantization for the current picture when the second syntax element indicates that dependent quantization is disabled for the current picture or the second syntax element is not present in the video bitstream for the current picture (880); the video decoder enables dependent quantization for the current picture when the second syntax element indicates that dependent quantization is enabled for the current picture (890).

[0131] In some embodiments, when residual coding for transform skip is not disabled for the current picture, the video decoder receives a third syntax element (e.g., slice ts residual coding disabled flag) from the video bitstream, where the third syntax element indicates whether residual coding for transform skip is enabled for a current slice associated with the current picture. When residual coding for transform skip is disabled for the current picture, the video decoder sets the third syntax element to a default value that disables residual coding for transform skip for all slices associated with the current picture. In some embodiments, the video decoder derives context modeling and rice parameters for parsing residual samples of the current picture independently of a dependent quantization state of the residual samples of the current picture. In other words, there is no dependency between the derivation of the context modeling and rice parameters and the use of the dependent quantization state.

[0132] Figure 9 is a block diagram illustrating an example context adaptive binary arithmetic coding (CABAC) engine according to some embodiments of the present disclosure.

[0133] Context adaptive binary arithmetic coding (CABAC) is a form of entropy coding used in many video coding standards, such as H.264 / MPEG-4 AVC, High Efficiency Video Coding (HEVC), and VVC. CABAC is based on arithmetic coding with some innovations and changes to make it suitable for the needs of video coding standards. For example, CABAC codes binary symbols, which keeps the complexity low and allows for more frequent usage of probability modeling for any symbol. Because the coding pattern is usually locally well-correlated, the probability model is adaptively selected based on a local context, allowing for better modeling of the probabilities. Finally, CABAC uses range division without multiplication by using a quantized probability range and probability state.

[0134] CABAC has multiple probability modes for different contexts. It first converts all non-binary symbols to binary. Then, for each binary bit (or called bit), the encoder selects which probability model to use, and then uses information from nearby elements to optimize the probability estimation. Finally, arithmetic coding is applied to compress the data.

[0135] Context modeling provides an estimate of the conditional probability of the coded symbol. With a suitable context model, one can exploit the given inter-symbol redundancy by switching between different probability models depending on the nearby already coded symbols. Coding a data symbol involves the following stages.

[0136] Binarization: CABAC uses binary arithmetic coding, which means that only binary decisions (1 or 0) are coded. Non-binary-valued symbols (such as transform coefficients or motion vectors) are "binarized" or converted to binary codes before arithmetic coding. This process is similar to the process of converting data symbols to variable-length codes, but the binary codes are further encoded (by the arithmetic encoder) before transmission. These stages are repeated for each binary bit (or "bit") of a binarized symbol.

[0137] Context model selection: A "context model" is a probability model for one or more binary bits of a binarized symbol. The model can be selected from a selection of available models according to the statistics of recently coded data symbols. The context model stores the probability of each binary bit being "1" or "0".

[0138] Arithmetic coding: The arithmetic encoder encodes each binary bit according to the selected probability model. Note that there are only two sub-ranges for each binary bit (corresponding to "0" and "1").

[0139] Probability update: The selected context model is updated based on the actual coded value (e.g., if the binary bit value is "1", then the frequency count "1" is increased).

[0140] By decomposing each non-binary syntax element value into a sequence of binary bits, the further processing of each binary bit value in CABAC depends on the associated coding mode decision, which can be selected as either regular mode or bypass mode. The latter is selected for binary bits that are assumed to be uniformly distributed, so for these binary bits the entire regular binary arithmetic coding (and decoding) process is simply bypassed. In regular coding mode, each binary bit value is coded by using a regular binary arithmetic coding engine, where the associated probability model is determined by a fixed selection based on the type of the syntax element and the bin position or bin index (binldx) in the binarized representation of the syntax element, or the associated probability model is adaptively selected from two or more probability models depending on the related side information (e.g., spatial neighbors, components, depth or size of the CU / PU / TU, or position within the TU). The selection of the probability model is referred to as context modeling. As an important design decision, the latter case is typically only applied to the most frequently observed binary bits, while the others, which are typically less frequently observed, will be processed using a joint, typically zeroth order probability model. In this way, CABAC is able to perform selective adaptive probability modeling on a sub-symbol level, thus providing an efficient tool to exploit inter-symbol redundancy, significantly reducing the overall modeling or learning cost. Note that for both the fixed and adaptive cases, in principle, the switching from one probability model to another can occur between any two consecutive regular coded binary bits. In general, the design of the context models in CABAC reflects a good trade-off between the conflicting goals of avoiding unnecessary modeling cost overhead and exploiting statistical correlation to a large extent.

[0141] The parameters of the probability models in CABAC are adaptive, meaning that the adaptation of the model probabilities to the statistical variations of the source of the binary bits is performed in the encoder and the decoder on a per-binary bit basis in a backward adaptive and synchronized manner; this process is referred to as probability estimation. To this end, each probability model in CABAC can utilize an associated model probability value p taking one out of 126 different states, with the associated model probability value p ranging in the interval [0:01875; 0:98125]. The two parameters of each probability model are stored in the context memory in the form of 7-bit entries: 6 bits for each of the 63 probability states of the model probability pLPS representing the least probable symbol (LPS), and 1 bit for nMPS, the value of the most probable symbol (MPS).

[0142] In one or more examples, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media can include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of the computer program from one place to another, e.g., according to a communication protocol. In this manner, computer- readable media generally can correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the implementations described in the present disclosure. A computer program product can include a computer-readable medium.

[0143] The terminology used in the description of the implementations herein is for the purpose of describing particular implementations only and is not intended to limit the scope of the claims. As used in the description of the implementations and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0144] It will also be understood that, although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first electrode could be termed a second electrode, and, similarly, a second electrode could be termed a first electrode, without departing from the scope of the present implementations. The first electrode and the second electrode are both electrodes, but they are not the same electrode.

[0145] The description of the application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the application in the form disclosed. Many modifications, variations, and alternatives of the present application will be apparent to those of ordinary skill in the art, once the benefits of the present application have been explained. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, and to enable others skilled in the art to understand the application of the various embodiments with various modifications as are suited to the particular use contemplated, and to best enable others skilled in the art to best utilize the application in various embodiments and with various modifications as are suited to the particular use contemplated. Therefore, it is to be understood that the scope of the claims is not to be limited to the specific examples disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding video data, comprising: receiving a first syntax element of a current picture from a video bitstream; receiving, based on determining that the first syntax element indicates that residual codec skipping for transform is not disabled for the current picture, a second syntax element for the current picture from the video bitstream, wherein the second syntax element indicates whether dependent quantization is disabled for the current picture; According to determining that the first syntax element indicates disabling of transform-skipped residual codec for the current picture, setting the second syntax element to a default value of disabling dependent quantization for the current picture; and Residual decoding and inverse quantization are performed on the current picture according to the first syntax element and the second syntax element.

2. The method according to claim 1, wherein performing residual decoding on the current picture according to the first syntax element and the second syntax element further comprises: When the first syntax element indicates that residual coding and decoding for transform skipping is disabled for the current picture, performing residual coding and decoding for non-transform skipping on the current picture; and When the first syntax element indicates that residual coding and decoding for transform skipping is not disabled for the current picture or the first syntax element does not exist in the video bitstream for the current picture, residual coding and decoding for transform skipping is performed on the current picture.

3. The method of claim 1 , wherein performing inverse quantization on the current picture according to the first syntax element and the second syntax element further comprises: When the second syntax element indicates that dependent quantization is disabled for the current picture or the second syntax element does not exist in the video bitstream for the current picture, disabling dependent quantization for the current picture; and When the second syntax element indicates that dependent quantization is enabled for the current picture, dependent quantization is enabled for the current picture.

4. The method according to claim 1, further comprising: When transform-skipped residual codec is not disabled for the current picture, a third syntax element is received from the video bitstream, wherein the third syntax element indicates whether transform-skipped residual codec is enabled for a current slice associated with the current picture.

5. The method according to claim 4, further comprising: When residual coding and decoding for transform skipping is disabled for the current picture, the third syntax element is set to a default value that disables residual coding and decoding for transform skipping for all slices associated with the current picture.

6. The method according to claim 1, further comprising: When residual coding and decoding for transform skipping is disabled for the current picture, context modeling and Rice parameters for parsing the residual samples of the current picture are derived independently of the dependent quantization state of the residual samples of the current picture. The method according to claim 1 , wherein when the first syntax element is 1, the second syntax element is 0. The method according to claim 1 , wherein when the second syntax element is 1, the first syntax element is 0.

9. An electronic device comprising: one or more processing units; a memory coupled to the one or more processing units; as well as A plurality of programs are stored in the memory, and when the plurality of programs are executed by the one or more processing units, the plurality of programs cause the electronic device to perform the method according to any one of claims 1 to 8.

10. A non-volatile computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, causes the electronic device to perform the method according to any one of claims 1 to 8.

Images