Method and apparatus for residual and coefficient coding

Abstract

An electronic device implements a method of decoding video data. The method includes receiving, from a bitstream, video data corresponding to a transform skip mode coded block, decoding, from the video data, a first codeword, a second codeword, and a first group of codewords for a pixel point within the transform skip mode coded block, deriving an initial horizontal value from the first group of codewords, converting the first codeword to a remainder of the pixel point according to a predefined mapping relationship generated using a constant Rice parameter, converting the second codeword to a sign value of the remainder, and deriving a quantized residual of the pixel point from the remainder, the sign value, and the initial horizontal value.

Description

[0001] This application is a divisional application of the application filed on September 24, 2020, with application number 202080053580.4 and invention title "Method and apparatus for encoding and decoding residuals and coefficients".

[0002] Related applications This application claims priority to U.S. Provisional Patent Application No. 62 / 905,342, filed September 24, 2019, entitled “Residual and Coefficients Coding for Video Coding,” which is incorporated herein by reference in its entirety. Technical Field

[0003] This application generally relates to video data encoding and decoding and compression, and more specifically, to improved methods and systems for residual and coefficient encoding and decoding of video. Background Technology

[0004] Digital video is supported by a variety of electronic devices, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. These electronic devices transmit, receive, encode, decode, and / or store digital video data by implementing video compression / decompression standards defined by MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Codec (AVC), High Efficiency Video Codec (HEVC), and Universal Video Codec (VVC). Video compression typically involves implementing spatial domain (intra-frame) prediction and / or temporal domain (inter-frame) prediction to reduce or remove redundancy inherent in the video data. For block-based video codecs, video frames are divided into one or more stripes, each strip containing multiple video blocks, also known as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or be recursively subdivided into smaller CUs until a predefined minimum CU size is reached. Each CU (also called a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU can be coded in intra-frame, inter-frame, or IBC mode. Video blocks in the intra-frame coded (I) strip of a video frame are encoded using spatial domain prediction relative to reference samples in adjacent blocks within the same video frame. Video blocks in the inter-frame coded (P or B) strip of a video frame can be encoded using spatial domain prediction relative to reference samples in adjacent blocks within the same video frame or temporal prediction relative to reference samples in other previous and / or future reference video frames.

[0005] A prediction block for the current video block to be encoded is generated based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding reference blocks can be accomplished using a block-matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called the residual block or prediction error. Inter-frame coded blocks are encoded based on motion vectors and the residual block, the motion vectors pointing to reference blocks in the reference frame that form the prediction block. The process of determining motion vectors is often called motion estimation. Intra-frame coded blocks are encoded based on intra-frame prediction modes and the residual block. For further compression, the residual block is transformed from the pixel domain to a transform domain, such as the frequency domain, producing residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, which are then entropy-encoded into the video bitstream for even further compression.

[0006] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., flash memory) for access by another electronic device with digital video capabilities or transmitted directly to the electronic device via wired or wireless means. The electronic device then performs video decompression (the reverse of the video compression process described above) by, for example, parsing the encoded video bitstream to obtain semantic elements from the bitstream and reconstructing the digital video data from the encoded video bitstream to its original format based at least in part on the semantic elements obtained from the bitstream, and presents the reconstructed digital video data on the display of the electronic device.

[0007] As digital video quality progresses from high definition to 4Kx2K or even 8Kx4K, the amount of video data to be encoded / decoded grows exponentially. Maintaining image quality while encoding / decoding video data more efficiently remains a continuous challenge. Summary of the Invention

[0008] This application describes implementations related to video data encoding and decoding, and more specifically, describes implementations related to systems and methods for video encoding and decoding using a palette pattern.

[0009] According to a first aspect of this application, a method for decoding video data includes: an electronic device implementing the method of decoding video data. The method includes: receiving video data corresponding to a transform skip mode coding block from a bitstream; decoding a first codeword, a second codeword, and a first set of codewords for pixels within the transform skip mode coding block from the video data; deriving an initial level value from the first set of codewords; converting the first codeword into a remainder of the pixel according to a predefined mapping relationship generated using a constant Rice parameter; converting the second codeword into a sign value of the remainder; and deriving a quantization residual of the pixel from the remainder, the sign value, and the initial level value.

[0010] According to a second aspect of this application, an electronic device includes one or more processing units, a memory, and a plurality of programs stored in the memory. When executed by the one or more processing units, the programs cause the electronic device to perform the method of decoding video data as described above.

[0011] According to a third aspect of this application, a non-transitory computer-readable storage medium stores a plurality of programs executed by an electronic device having one or more processing units. When executed by the one or more processing units, the programs cause the electronic device to implement the method of decoding video data as described above. Attached Figure Description

[0012] The accompanying drawings, which are included herein and form part of the specification, illustrate the described embodiments and, together with the description, serve to explain the underlying principles. Similar reference numerals denote corresponding parts.

[0013] Figure 1 This is a block diagram illustrating an exemplary video encoding and decoding system according to some embodiments of the present disclosure.

[0014] Figure 2 This is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.

[0015] Figure 3 This is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.

[0016] Figures 4A to 4E This is a block diagram illustrating how a frame, according to some embodiments of the present disclosure, is recursively divided into multiple video blocks of different sizes and shapes.

[0017] Figures 5A to 5B This is a block diagram illustrating examples of transform coefficient encoding and decoding for use scenarios and bypass encoding and decoding according to some embodiments of this disclosure.

[0018] Figure 6 This is a block diagram illustrating an exemplary process of dependent scalar quantization according to some embodiments of the present disclosure.

[0019] Figure 7 This is a block diagram illustrating an exemplary state machine for switching between two different scalar quantizers according to some embodiments of the present disclosure.

[0020] Figure 8 This is a flowchart illustrating an exemplary process by which a video decoder performs residual encoding and decoding for a transform skip mode coding block according to some embodiments of the present disclosure.

[0021] Figure 9 This is a block diagram illustrating an example scenario adaptive binary arithmetic codec (CABAC) engine according to some embodiments of this disclosure. Detailed Implementation

[0022] Reference will now be made in detail to the specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented on a wide variety of electronic devices with digital video capabilities.

[0023] Figure 1 This is a block diagram illustrating an exemplary system 10 for parallel encoding and decoding of video blocks according to some embodiments of the present disclosure. Figure 1 As shown, system 10 includes a source device 12 that generates and encodes video data for later decoding by a destination device 14. The source device 12 and destination device 14 can include any of a variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some embodiments, the source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0024] In some implementations, destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, link 16 may include a communication medium enabling source device 12 to transmit encoded video data directly to destination device 14 in real time. Encoded video data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to destination device 14. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, switch, base station, or any other equipment that may be used to facilitate communication from source device 12 to destination device 14.

[0025] In some other embodiments, encoded video data can be transferred from output interface 22 to storage device 32. The destination device 14 can then access the encoded video data in storage device 32 via input interface 28. Storage device 32 can include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 can correspond to a file server or another intermediate storage device that can hold the encoded video data generated by source device 12. The destination device 14 can access the stored video data from storage device 32 via streaming or downloading. The file server can be any type of computer capable of storing and transferring encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 can access encoded video data via any standard data connection, including wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or combinations thereof, suitable for accessing encoded video data stored on a file server. Transmission of encoded video data from storage device 32 can be streaming, downloading, or a combination thereof.

[0026] like Figure 1As shown, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Video source 18 may include sources such as: a video capture device (e.g., a video camera), a video archive containing previously captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of these sources. As an example, if video source 18 is a video camera in a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, the embodiments described in this application are generally applicable to video encoding and decoding and can be applied to wireless and / or wired applications.

[0027] Captured, pre-captured, or computer-generated video can be encoded by video encoder 20. The encoded video data can be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data can also (or alternatively) be stored on storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may also include a modem and / or transmitter.

[0028] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or a modem and receives encoded video data via link 16. The encoded video data transmitted via link 16 or provided on storage device 32 may include various semantic elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such semantic elements may be included within the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0029] In some embodiments, destination device 14 may include display device 34, which may be an integrated display device or an external display device configured to communicate with destination device 14. Display device 34 displays decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0030] Video encoder 20 and video decoder 30 may operate according to proprietary or industry standards, such as VVC, HEVC, MPEG-4 Part 10, Advanced Video Codec (AVC), or extensions of such standards. It should be understood that this application is not limited to any particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally contemplated that the video encoder 20 of source device 12 can be configured to encode video data according to any of these current or future standards. Similarly, it is generally contemplated that the video decoder 30 of destination device 14 can be configured to decode video data according to any of these current or future standards.

[0031] Both the video encoder 20 and the video decoder 30 can be implemented as any of a variety of suitable encoder circuit systems, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When partially implemented in software, the electronic device may store software instructions in a suitable non-transitory computer-readable medium and execute said instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of the video encoder 20 and the video decoder 30 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0032] Figure 2 This is a block diagram illustrating an exemplary video encoder 20 according to some embodiments described in this application. The video encoder 20 can implement intra-frame and inter-frame predictive coding of video blocks within a video frame. Intra-frame predictive coding relies on spatial domain prediction to reduce or remove spatial domain redundancy in video data within a given video frame or image. Inter-frame predictive coding relies on temporal domain prediction to reduce or remove temporal domain redundancy in video data within neighboring video frames or images of a video sequence.

[0033] like Figure 2As shown, the video encoder 20 includes a video data memory 40, a prediction processing unit 41, a decoded picture buffer (DPB) 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The prediction processing unit 41 further includes a motion estimation unit 42, a motion compensation unit 44, a partitioning unit 45, an intra-frame prediction processing unit 46, and an intra-frame block copying (BC) unit 48. In some embodiments, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62 for video block reconstruction. A deblocking filter (not shown) may be located between the adder 62 and the DPB 64 to filter block boundaries to remove block artifacts from the reconstructed video. In addition to the deblocking filter, a loop filter (not shown) may be used to filter the output of the adder 62. The video encoder 20 may be in the form of a fixed or programmable hardware unit, or may be partitioned into one or more of the fixed or programmable hardware units shown in the figure.

[0034] Video data memory 40 can store video data to be encoded by components of video encoder 20. The video data in video data memory 40 can be obtained, for example, from video source 18. DPB 64 is a buffer storing reference video data for use by video encoder 20 during video data encoding (e.g., in intra-frame or inter-frame predictive coding modes). Video data memory 40 and DPB 64 can be formed from any of a variety of memory devices. In various examples, video data memory 40 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

[0035] like Figure 2 As shown, after receiving video data, the partitioning unit 45 within the prediction processing unit 41 divides the video data into video blocks. This partitioning may also include dividing the video frame into strips, tiles, or other larger coding units (CUs) based on a predefined partitioning structure associated with the video data, such as a quadtree structure. The video frame may be divided into multiple video blocks (or groups of video blocks referred to as tiles). The prediction processing unit 41 may select one of several possible predictive coding modes for the current video block based on error results (e.g., coding rate and distortion level), such as one of several intra-frame predictive coding modes or one of several inter-frame predictive coding modes. The prediction processing unit 41 may provide the resulting intra-frame or inter-frame predictive coded block to adder 50 to generate a residual block and to adder 62 to reconstruct the coded block for subsequent use as part of a reference frame. The prediction processing unit 41 also provides semantic elements, such as motion vectors, intra-frame mode indicators, partitioning information, and other such semantic information, to entropy coding unit 56.

[0036] To select a suitable intra-predictive coding mode for the current video block, the intra-predictive processing unit 46 within the prediction processing unit 41 can perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame as the current block to be encoded to provide spatial domain prediction. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more prediction blocks in one or more reference frames to provide temporal domain prediction. The video encoder 20 can perform multi-round coding, for example, to select a suitable coding mode for each block of video data.

[0037] In some implementations, motion estimation unit 42 determines the inter-frame prediction mode of the current video frame by generating motion vectors that indicate the displacement of a prediction unit (PU) of a video block within the current video frame relative to a prediction block in a reference video frame, according to a predetermined mode within the video frame sequence. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of the video block. For example, the motion vectors may indicate the displacement of the PU of a video block within the current video frame or picture relative to a prediction block (or other coding unit) in a reference frame with respect to the current block (or other coding unit) encoded within the current video frame. The predetermined mode may designate video frames in the sequence as P-frames or B-frames. Intra-frame BC unit 48 may determine vectors for intra-frame BC coding, such as block vectors, in a manner similar to how motion estimation unit 42 determines motion vectors for inter-frame prediction, or the block vectors may be determined using motion estimation unit 42.

[0038] A prediction block is a block of reference frames that is considered to closely match the PU of the video block to be encoded in terms of pixel differences, which can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some implementations, the video encoder 20 can compute the values ​​of the segmented integer pixel positions of the reference frame stored in the DPB 64. For example, the video encoder 20 can interpolate the values ​​of the quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference frame. Thus, the motion estimation unit 42 can perform motion search relative to the full pixel positions and fractional pixel positions and output a motion vector with fractional pixel precision.

[0039] The motion estimation unit 42 calculates the motion vector of the PU in the inter-frame predictive coding frame by comparing the position of the PU with the position of the prediction block of a reference frame selected from the first reference frame list (list 0) or the second reference frame list (list 1). Each identifier in the first reference frame list (list 0) or the second reference frame list (list 1) is stored in one or more reference frames in the DPB64. The motion estimation unit 42 sends the calculated motion vector to the motion compensation unit 44, and then to the entropy coding unit 56.

[0040] Motion compensation implemented by motion compensation unit 44 may involve retrieving or generating prediction blocks based on motion vectors determined by motion estimation unit 42. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the prediction block pointed to by the motion vector in one of the reference frame lists, retrieve the prediction block from DPB 64, and forward the prediction block to adder 50. Adder 50 then forms a residual video block of pixel differences by subtracting the pixel values ​​of the prediction block provided by motion compensation unit 44 from the pixel values ​​of the current video block being encoded. The pixel differences forming the residual video block may include luma or chroma difference components or both. Motion compensation unit 44 may also generate semantic elements associated with video blocks of a video frame for use by video decoder 30 when decoding video blocks of a video frame. Semantic elements may include, for example, semantic elements defining motion vectors for identifying the prediction block, any identifier indicating a prediction mode, or any other semantic information described herein. Note that motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are described separately for conceptual purposes.

[0041] In some implementations, the intra-BC unit 48 can generate vectors and retrieve prediction blocks in a manner similar to that described above in conjunction with the motion estimation unit 42 and the motion compensation unit 44, but the prediction blocks are in the same frame as the current block being encoded, and the vectors are referred to as block vectors rather than motion vectors. Specifically, the intra-BC unit 48 can determine the intra-prediction mode for encoding the current block. In some examples, the intra-BC unit 48 can encode the current block using various intra-prediction modes, for example, during individual encoding rounds, and test their performance through rate-distortion analysis. Next, the intra-BC unit 48 can select a suitable intra-prediction mode from the various tested intra-prediction modes to use accordingly and generate an intra-mode indicator. For example, the intra-BC unit 48 can use rate-distortion analysis for various tested intra-prediction modes to calculate rate-distortion values ​​and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes as the suitable intra-prediction mode to use. Rate-distortion analysis typically determines the amount of distortion (or error) between a coded block and a raw uncoded block, which is encoded to produce the coded block, and determines the bit rate (i.e., the number of bits) used to produce the coded block. Intra-frame BC unit 48 can calculate a ratio based on the distortion and rate of various coded blocks to determine which intra-frame prediction mode exhibits the optimal rate-distortion value for the block.

[0042] In other examples, the intra-frame BC unit 48 may use, in whole or in part, the motion estimation unit 42 and the motion compensation unit 44 to implement such functionality for intra-frame BC prediction according to the embodiments described herein. In any case, for intra-frame block copying, the predicted block may be a block that is considered to closely match the block to be encoded in terms of pixel differences, which may be determined by the sum of absolute differences (SAD), the sum of squared differences (SSD), or other difference measures, and the identification of the predicted block may include the calculation of values ​​for segmented integer pixel positions.

[0043] Regardless of whether the prediction block originates from the same frame based on intra-frame prediction or from different frames based on inter-frame prediction, the video encoder 20 can form a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded. The pixel difference forming the residual video block can include both luminance and chrominance component differences.

[0044] Intra-prediction processing unit 46 can perform intra-prediction on the current video block as an alternative to inter-frame prediction implemented by motion estimation unit 42 and motion compensation unit 44, or intra-block copy prediction implemented by intra-BC unit 48, as described above. Specifically, intra-prediction processing unit 46 can determine the intra-prediction mode for encoding the current block. To this end, intra-prediction processing unit 46 can encode the current block using various intra-prediction modes, for example, during individual encoding rounds, and intra-prediction processing unit 46 (or, in some instances, mode selection unit) can select an appropriate intra-prediction mode from the tested intra-prediction modes for use. Intra-prediction processing unit 46 can provide information indicating the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 can encode the information indicating the selected intra-prediction mode in the bitstream.

[0045] After prediction processing unit 41 determines the prediction block of the current video block via inter-frame prediction or intra-frame prediction, adder 50 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more transform units (TUs) and provided to transform processing unit 52. Transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform such as discrete cosine transform (DCT) or a conceptually similar transform.

[0046] The transform processing unit 52 can send the obtained transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can also reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan on a matrix including the quantized transform coefficients. Alternatively, the entropy coding unit 56 can perform this scan.

[0047] After quantization, entropy coding unit 56 entropy-encodes the quantized transform coefficients into a video bitstream using, for example, context-adaptive variable-length codec (CAVLC), context-adaptive binary arithmetic codec (CABAC), semantic-based context-adaptive binary arithmetic codec (SBAC), probability interval partitioning entropy (PIPE) codec, or other entropy coding methods or techniques. The encoded bitstream can then be transmitted to video decoder 30, or archived in storage device 32 for later transmission to video decoder 30 or retrieval by video decoder 30. Entropy coding unit 56 can also entropy-encode the motion vectors and other semantic elements of the currently encoded video frame.

[0048] The inverse quantization unit 58 and the inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual video block in the pixel domain to generate a reference block for predicting other video blocks. As described above, the motion compensation unit 44 can generate a motion-compensated prediction block based on one or more reference blocks of the frame stored in the DPB 64. The motion compensation unit 44 can also apply one or more interpolation filters to the prediction block to calculate piecewise integer pixel values ​​for motion estimation.

[0049] Adder 62 adds the reconstructed residual block to the motion-compensated prediction block generated by motion compensation unit 44 to generate a reference block, which is stored in DPB 64. The reference block can then be used by intra-frame BC unit 48, motion estimation unit 42, and motion compensation unit 44 as a prediction block for inter-frame prediction of another video block in subsequent video frames.

[0050] Figure 3 This is a block diagram illustrating an exemplary video decoder 30 according to some embodiments of this application. The video decoder 30 includes a video data memory 79, an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, and a DPB 92. The prediction processing unit 81 further includes a motion compensation unit 82, an intra-frame prediction processing unit 84, and an intra-frame prediction (BC) unit 85. The video decoder 30 can be implemented in conjunction with the above. Figure 2 The encoding process described for video encoder 20 is largely the opposite of the decoding process. For example, motion compensation unit 82 can generate prediction data based on motion vectors received from entropy decoding unit 80, while intra-frame prediction unit 84 can generate prediction data based on intra-frame prediction mode indicators received from entropy decoding unit 80.

[0051] In some examples, units of the video decoder 30 may be assigned tasks to implement embodiments of this application. Furthermore, in some examples, embodiments of this disclosure may be divided among one or more units of the video decoder 30. For example, the intra-frame BC unit 85 may implement embodiments of this application individually or in combination with other units of the video decoder 30, such as the motion compensation unit 82, the intra-frame prediction processing unit 84, and the entropy decoding unit 80. In some examples, the video decoder 30 may not include the intra-frame BC unit 85, and the functionality of the intra-frame BC unit 85 may be implemented by other components of the prediction processing unit 81, such as the motion compensation unit 82.

[0052] Video data memory 79 can store video data, such as encoded video bitstreams, to be decoded by other components of video decoder 30. The video data stored in video data memory 79 can be obtained, for example, from storage device 32, from a local video source (such as a camera), via wired or wireless network communication of video data, or by accessing a physical data storage medium (e.g., a flash drive or hard disk). Video data memory 79 may include an encoded picture buffer (CPB) that stores encoded video data from the encoded video bitstream. Decoding picture buffer (DPB) 92 of video decoder 30 stores reference video data for use by video decoder 30 when decoding video data (e.g., according to intra-frame or inter-frame predictive coding modes). Video data memory 79 and DPB 92 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. For illustrative purposes, video data memory 79 and DPB 92 are... Figure 3 The video data memory 79 and DPB 92 are depicted as two distinct components of the video decoder 30. However, it will be apparent to those skilled in the art that the video data memory 79 and DPB 92 may be provided by the same memory device or separate memory devices. In some examples, the video data memory 79 may be on-chip along with other components of the video decoder 30, or off-chip relative to those components.

[0053] During the decoding process, the video decoder 30 receives an encoded video bitstream, which represents video blocks of encoded video frames and associated semantic elements. The video decoder 30 may receive semantic elements at the video frame level and / or the video block level. The entropy decoding unit 80 of the video decoder 30 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors or intra-frame prediction mode indicators, and other semantic elements. The entropy decoding unit 80 then forwards the motion vectors and other semantic elements to the prediction processing unit 81.

[0054] When a video frame is encoded as an intra-predictive coded (I) frame or used as an intra-coded prediction block in other types of frames, the intra-predictive processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video frame based on the intra-predictive mode sent by signaling and reference data from the previous decoded block of the current frame.

[0055] When the video frame is encoded as an inter-frame predictive coded (i.e., B or P) frame, the motion compensation unit 82 of the prediction processing unit 81 generates one or more prediction blocks of the video blocks of the current video frame based on the motion vectors and other semantic elements received from the entropy decoding unit 80. Each of the prediction blocks can be generated based on reference frames within one of the reference frame lists. The video decoder 30 can construct the reference frame list, i.e., list 0 and list 1, based on the reference frames stored in the DPB 92 using a default construction technique.

[0056] In some examples, when a video block is encoded according to the intra-frame BC mode described herein, the intra-frame BC unit 85 of the prediction processing unit 81 generates a prediction block for the current video block based on the block vector and other semantic elements received from the entropy decoding unit 80. The prediction block can be located within the reconstructed region of the same image as the current video block defined by the video encoder 20.

[0057] Motion compensation unit 82 and / or intra-frame BC unit 85 determine prediction information for video blocks in the current video frame by parsing motion vectors and other semantic elements, and then use this prediction information to generate a prediction block for the current video block being decoded. For example, motion compensation unit 82 uses some of the received semantic elements to determine the prediction mode (e.g., intra-frame or inter-frame prediction) for encoding video blocks in the video frame, the inter-frame prediction frame type (e.g., B or P), construction information for one or more reference frame lists for the frame, the motion vector for each inter-frame prediction encoded video block in the frame, the inter-frame prediction state for each inter-frame prediction encoded video block in the frame, and other information for decoding video blocks in the current video frame.

[0058] Similarly, the intra-BC unit 85 may use some of the received semantic elements (e.g., identifiers) to determine which video block was predicted using the intra-BC mode, which video blocks of the frame have construction information in the reconstruction region and should be stored in the DPB 92, the block vector of each intra-BC predicted video block of the frame, the intra-BC prediction state of each intra-BC predicted video block of the frame, and other information for decoding video blocks in the current video frame.

[0059] The motion compensation unit 82 may also use an interpolation filter, such as that used by the video encoder 20 during the encoding of a video block, to perform interpolation to calculate the interpolated values ​​for segmented integer pixels of the reference block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the video encoder 20 based on the received semantic elements and use the interpolation filter to generate a prediction block.

[0060] The inverse quantization unit 86 uses the same quantization parameters calculated by the video encoder 20 for each video block in the video frame to inverse quantize the quantization transform coefficients provided in the bitstream and entropy decoded by the entropy decoding unit 80 to determine the degree of quantization. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to reconstruct the residual block in the pixel domain.

[0061] After the motion compensation unit 82 or the intra-frame BC unit 85 generates a prediction block for the current video block based on vectors and other semantic elements, the adder 90 reconstructs the decoded video block of the current video block by summing the residual block from the inverse transform processing unit 88 and the corresponding prediction block generated by the motion compensation unit 82 and the intra-frame BC unit 85. A loop filter (not shown) may be located between the adder 90 and the DPB 92 for further processing of the decoded video block. The decoded video block in a given frame is then stored in the DPB 92, which stores a reference frame for subsequent motion compensation for the next video block. The DPB 92 or a separate memory device may also store the decoded video for later display on a display device (such as...). Figure 1 It is displayed on the display device 34).

[0062] In a typical video encoding and decoding process, a video sequence usually consists of a set of ordered frames or pictures. Each frame may include three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples. SCb is a two-dimensional array of Cb chrominance samples. SCr is a two-dimensional array of Cr chrominance samples. In other instances, a frame may be monochrome and therefore consist of only a two-dimensional luminance sample array.

[0063] like Figure 4A As shown, the video encoder 20 (or more specifically, partitioning unit 45) generates a coded representation of a frame by first dividing the frame into a set of coding tree units (CTUs). A video frame may include an integer number of CTUs, which are sequentially ordered from left to right and from top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by the video encoder 20 in the sequence parameter set such that all CTUs in the video sequence have the same size, i.e., 128×128, 64×64, 32×32, and 16×16. However, it should be noted that this application is not necessarily limited to specific sizes. Figure 4BAs shown, each CTU may include a coding tree block (CTB) for luminance samples, two corresponding coding tree blocks for chrominance samples, and semantic elements for encoding samples of the coding tree blocks. The semantic elements describe the attributes of different types of units in the encoded pixel block and how the video sequence can be reconstructed at the video decoder 30, including inter-frame or intra-frame prediction, intra-frame prediction mode, motion vectors, and other parameters. In a monochrome image or an image with three separate color planes, the CTU may include a single coding tree block and semantic elements for encoding samples of the coding tree block. The coding tree block may be an NxN block of samples.

[0064] To achieve better performance, the video encoder 20 can recursively perform tree partitioning on the coding tree blocks of the CTU, such as binary tree partitioning, ternary tree partitioning, quadtree partitioning, or a combination of both, and divide the CTU into smaller coding units (CUs). For example... Figure 4C As depicted, the 64x64 CTU 400 is first divided into four smaller CUs, each with a block size of 32x32. Of these four smaller CUs, CU 410 and CU 420 are each further divided into four 16x16 CUs. The two 16x16 CUs, 430 and 440, are then further divided into four 8x8 CUs. Figure 4D A quadtree data structure is depicted, and the quadtree data structure is illustrated as follows: Figure 4C The final result of the partitioning process of the CTU 400, as depicted in the diagram, is that each leaf node of the quadtree corresponds to a CU with a corresponding size range from 32x32 to 8x8. Just like... Figure 4B Similar to the CTU depicted, each CU can include a coded block (CB) of luma samples and two corresponding coded blocks of chroma samples of the same size frame, as well as semantic elements for encoding the samples of the coded blocks. In monochrome images or images with three separate color planes, the CU can include a single coded block and a semantic structure for encoding the samples of the coded block. It should be noted that... Figure 4C and Figure 4D The quadtree partitioning depicted is for illustrative purposes only, and a CTU can be partitioned into CUs based on quadtree / ternary / binary tree partitioning to accommodate different local characteristics. In multi-type tree structures, a CTU is partitioned according to a quadtree structure, and the leaf CUs of each quadtree can be further partitioned according to binary and ternary tree structures. Figure 4E As shown, there are five partition types: quadrilateral partition, horizontal binary partition, vertical binary partition, horizontal ternary partition, and vertical ternary partition.

[0065] In some implementations, the video encoder 20 may further divide the coded blocks of the CU into one or more MxN prediction blocks (PBs). A prediction block is a rectangular (square or non-square) sample block on which the same inter-frame or intra-frame prediction is applied. A prediction unit (PU) of the CU may include a prediction block for luma samples, two corresponding prediction blocks for chroma samples, and semantic elements for predicting the prediction blocks. In a monochrome image or an image with three separate color planes, a PU may include a single prediction block and a semantic structure for predicting the prediction block. The video encoder 20 may generate predicted luma, Cb, and Cr blocks for each PU of the CU.

[0066] Video encoder 20 can use intra-frame prediction or inter-frame prediction to generate prediction blocks for the PU. If video encoder 20 uses intra-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of the frame associated with the PU. If video encoder 20 uses inter-frame prediction to generate prediction blocks for the PU, then video encoder 20 can generate prediction blocks for the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0067] After the video encoder 20 generates predicted luminance, Cb, and Cr blocks for one or more PUs of the CU, the video encoder 20 can generate a luminance residual block of the CU by subtracting the predicted luminance block of the CU from its original luminance coding block, such that each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predicted luminance blocks of the CU and a corresponding sample in the original luminance coding block of the CU. Similarly, the video encoder 20 can generate Cb residual blocks and Cr residual blocks for the CU, such that each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predicted Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predicted Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0068] In addition, such as Figure 4CAs shown, the video encoder 20 can use quadtree partitioning to decompose the luminance, Cb, and Cr residual blocks of the CU into one or more luminance, Cb, and Cr transform blocks. Transform blocks are rectangular (square or non-square) sample blocks to which the same transform is applied. A transform unit (TU) of the CU may include a transform block for the luminance sample, two corresponding transform blocks for the chrominance sample, and semantic elements for transforming the samples of the transform blocks. Therefore, each TU of the CU may be associated with a luminance transform block, a Cb transform block, and a Cr transform block. In some examples, the luminance transform block associated with a TU may be a sub-block of the CU's luminance residual block. The Cb transform block may be a sub-block of the CU's Cb residual block. The Cr transform block may be a sub-block of the CU's Cr residual block. In a monochrome image or an image with three separate color planes, a TU may include a single transform block and a semantic structure of samples for transforming the transform block.

[0069] The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block for the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block for the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block for the TU.

[0070] After generating coefficient blocks (e.g., luminance coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), video encoder 20 can quantize the coefficient blocks. Quantization generally refers to the process of quantizing transform coefficients to reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After quantizing the coefficient blocks, video encoder 20 can entropy-encode the semantic elements indicating the quantized transform coefficients. For example, video encoder 20 can perform context-adaptive binary arithmetic encoding / decoding (CABAC) on the semantic elements indicating the quantized transform coefficients. Finally, video encoder 20 can output a bitstream comprising a bit sequence that forms a representation of coded frames and associated data, which is stored in storage device 32 or transmitted to destination device 14.

[0071] After receiving the bitstream generated by the video encoder 20, the video decoder 30 can parse the bitstream to obtain semantic elements. The video decoder 30 can reconstruct frames of video data based at least in part on the semantic elements obtained from the bitstream. The process of reconstructing the video data is generally the reverse of the encoding process performed by the video encoder 20. For example, the video decoder 30 can perform an inverse transform on the coefficient block associated with the TU of the current CU to reconstruct the residual block associated with the TU of the current CU. The video decoder 30 also reconstructs the coded blocks of the current CU by adding samples of the predicted blocks of the PU of the current CU to corresponding samples of the transformed blocks of the TU of the current CU. After reconstructing the coded blocks for each CU of the frame, the video decoder 30 can reconstruct the frame.

[0072] As mentioned above, video encoding and decoding primarily use two modes (intra-frame prediction and inter-frame prediction) to achieve video compression. Palette-based encoding and decoding is another encoding and decoding scheme adopted by many video encoding and decoding standards. In palette-based encoding and decoding, which may be particularly suitable for encoding and decoding screen-generated content, the video codec (e.g., video encoder 20 or video decoder 30) forms a color palette table that represents a given block of video data. The palette table includes the most prominent (e.g., frequently used) pixel values ​​in the given block. Pixel values ​​that are not frequently represented in the given block of video data are either not included in the palette table or are included as escape colors.

[0073] Each entry in the palette table includes an index to the corresponding pixel value in the palette table. The palette index for a sample within a block can be encoded to indicate which entry in the palette table should be used to predict or reconstruct which sample. The palette pattern begins with generating a palette predictor for the first block of a group of images, strips, mosaics, or other such video blocks. As will be explained below, palette predictors for subsequent video blocks are typically generated by updating previously used palette predictors. For illustrative purposes, it is assumed that the palette predictor is defined at the image level. In other words, an image may consist of multiple coded blocks, each with its own palette table, but only one palette predictor for the entire image.

[0074] To reduce the number of bits required to signal palette entries in the video bitstream, the video decoder can utilize a palette predictor to determine new palette entries in the palette table for reconstructing video blocks. For example, the palette predictor can include palette entries from previously used palette tables, or it can even be initialized using the most recently used palette table by including all entries from the most recently used palette table. In some implementations, the palette predictor can include a subset of entries from the most recently used palette table and then incorporate some entries from other previously used palette tables. The palette predictor can have the same size as the palette table used to encode different blocks, or it can be larger or smaller than the palette table used to encode different blocks. In one example, the palette predictor is implemented as a first-in-first-out (FIFO) table comprising 64 palette entries.

[0075] To generate a palette table for blocks of video data based on the palette predictor, the video decoder can receive a one-bit identifier for each entry of the palette predictor from the encoded video bitstream. This one-bit identifier may have a first value (e.g., binary 1) indicating that the associated entry of the palette predictor will be included in the palette table, or a second value (e.g., binary 0) indicating that the associated entry of the palette predictor will not be included in the palette table. If the size of the palette predictor is larger than the palette table for blocks of video data, the video decoder can stop receiving more identifiers once the maximum size of the palette table is reached.

[0076] In some implementations, some entries in the palette table can be directly signaled in the encoded video bitstream instead of being determined using a palette predictor. For such entries, the video decoder can receive three separate m-bit values ​​from the encoded video bitstream, indicating the pixel values ​​associated with the entry for the luma and two chroma components, where m represents the bit depth of the video data. Those palette entries derived from the palette predictor require only one bit of identification compared to the multiple m-bit values ​​needed to directly signal the palette entries. Therefore, using a palette predictor to signal some or all palette entries can significantly reduce the number of bits required to signal new palette table entries, thereby improving the overall encoding / decoding efficiency of palette pattern encoding / decoding.

[0077] In many cases, the palette predictor for a block is determined based on the palette table used to encode one or more previously encoded blocks. However, when encoding the first coding tree unit in a picture, strip, or mosaic, the palette table for previously encoded blocks may be unavailable. Therefore, entries from previously used palette tables cannot be used to generate the palette predictor. In this case, a sequence of palette predictor initialization values ​​can be sent via signaling in the Sequence Parameter Set (SPS) and / or Picture Parameter Set (PPS). These initialization values ​​are used to generate the palette predictor when the previously used palette table is unavailable. The SPS typically refers to the semantic structure of semantic elements applied to a series of consecutively encoded video pictures (called a Coded Video Sequence (CVS)), determined by the content of the semantic elements found in the PPS, which are referred to by the semantic elements found in the header of each strip segment. The PPS typically refers to the semantic structure of semantic elements applied to one or more individual pictures within a CVS, determined by the semantic elements found in the header of each strip segment. Therefore, SPS is generally considered to be a higher-level semantic structure than PPS, which means that the semantic elements included in SPS are usually less frequently changed and applied to a larger portion of the video data compared to the semantic elements included in PPS.

[0078] Figures 5A to 5B This is a block diagram illustrating examples of transform coefficient encoding and decoding for use scenarios and bypass encoding and decoding according to some embodiments of this disclosure.

[0079] Transform coefficient encoding and decoding in VVC is similar to that in HEVC because they both use non-overlapping coefficient groups (also known as CGs or sub-blocks). However, there are some differences between the two schemes. In HEVC, the size of each CG is fixed at 4x4. In VVC draft 6, the CG size becomes dependent on the TB size. Therefore, various CG sizes (1x16, 2x8, 8x2, 2x4, 4x2, and 16x1) can be used in VVC. CGs within a block and transform coefficients within a CG are encoded according to a predefined scan order.

[0080] To limit the maximum number of context-coded bits (CCB) per pixel, the area of ​​the TB and the types of video components (i.e., luma and chroma components) are used to derive the maximum number of context-coded bits (CCB) per TB. In some embodiments, the maximum number of context-coded bits is equal to TB_zosize * 1.75. Here, TB_zosize represents the number of samples within the TB after the coefficients are zeroed out. Note that... coded_sub_block_flag This is an indicator that CG contains non-zero coefficients, and this indicator is not considered when counting CCB.

[0081] Coefficient zeroing is an operation performed on a transform block to force coefficients located in a specific region of the transform block to be set to zero. For example, in the current VVC, a 64x64 TB has an associated zeroing operation. As a result, all transform coefficients located outside the top-left 32x32 region of the 64x64 TB are forced to zero. In fact, in the current VVC, for any transform block with a size exceeding 32 along a certain dimension, a coefficient zeroing operation is performed along that dimension to force coefficients located outside the top-left 32x32 region to be zero.

[0082] In transform coefficient encoding and decoding in VVC, the variable `remBinsPass1` is initially set to the maximum allowed number of context-coded bits (MCCB). During the encoding and decoding process, the variable is decremented by 1 each time context-coded bits are sent via signaling. When `remBinsPass1` is greater than or equal to four, coefficients are sent via signaling using semantic elements including `sig_coeff_flag`, `abs_level_gt1_flag`, `par_level_flag`, and `abs_level_gt3_flag`, all of which use context-coded bits in the first round. The remaining coefficient level information is encoded in the second round using Golomb-Rice codes and bypass-coded bits using the semantic elements of `abs_remainder`. When `remBinsPass1` becomes less than four during encoding in the first round, the current coefficient is not encoded in the first round but is directly encoded in the second round using Golomb-Rice codes and bypass-coded bits using the semantic elements of `dec_abs_level`. After all the above levels of encoding, the sign (sign_flag) at all scan positions where sig_coeff_flag is equal to 1 is finally encoded as bypass bits. This process... Figure 5A The description is as follows. `remBinsPass1` is reset for each TB. The conversion from using context-coded bits for `sig_coeff_flag`, `abs_level_gt1_flag`, `par_level_flag`, and `abs_level_gt3_flag` to using bypass-coded bits for the remaining coefficients occurs at most once per TB. For a coefficient subblock, if `remBinsPass1` is less than 4 before encoding its first coefficient, the entire coefficient subblock is encoded using bypass-coded bits.

[0083] Unlike HEVC, which uses a single residual codec scheme to encode both transform coefficients and transform skip coefficients, VVC uses two separate residual codec schemes for the transform coefficients and transform skip coefficients (i.e., residuals).

[0084] For example, it was observed that the statistical properties of the residuals in the transform skip mode differed from those of the transform coefficients, and there was no energy compression around the low-frequency components. The residual encoding / decoding was modified to take into account the different signal characteristics of the (spatial domain) transform skip residuals, including: (1) No signaling for the last x / y position; (2) Encode the coded_sub_block_flag for each sub-block except the DC sub-block when all previous flags are equal to 0; (3) Scenario modeling using the sig_coeff_flag of two adjacent coefficients; (4) par_level_flag uses only one context model; (5) More than 5, 7, or 9 additional identifiers; (6) Exporting the rice parameter for modification of remainder binarization; (7) The context modeling of the symbol flag is determined based on the left and top adjacent coefficient values, and the symbol flag is parsed after sig_coeff_flag to keep all context-encoded bits together; like Figure 5B As shown, the semantic elements sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, and par_level_flag are encoded in an interleaved manner from one residual sample to another in the first round, followed by the abs_level_gtX_flag bit plane in the second round, and the abs_remainder encoding in the third round.

[0085] Round 1: sig_coeff_flag, coeff_sign_flag, abs_level_gt1_flag, par_level_flag Round 2: abs_level_gt3_flag, abs_level_gt5_flag, abs_level_gt7_flag, abs_level_gt9_flag Round 3: abs_remainder.

[0086] Figure 6 This is a block diagram illustrating an exemplary process of dependent scalar quantization according to some embodiments of the present disclosure.

[0087] In the current VVC, the maximum QP value has been extended from 51 to 63, and the signaling for the initial QP has also changed accordingly. The initial value of SliceQpY can be modified at the slice segmentation layer when a non-zero value of slice_qp_delta is encoded. For transform skip blocks, the minimum allowed QP is limited to 4 because the quantization step size becomes 1 when QP equals 1.

[0088] Furthermore, a new concept called "dependent scalar quantization" is used to adapt scalar quantization used in HEVC. Dependent scalar quantization is a method in which a set of permissible reconstructed values ​​for transform coefficients depends on the values ​​of transform coefficient levels preceding the current transform coefficient level in reconstruction order. Compared to the regular independent scalar quantization used in HEVC, the permissible reconstructed vectors are compressed more densely in the N-dimensional vector space (where N represents the number of transform coefficients in the transform block). That is, for a given average number of permissible reconstructed vectors per N-dimensional unit volume, the average distortion between the input vector and the nearest reconstructed vector is reduced. The dependent scalar quantization method is implemented by (a) defining two scalar quantizers with different reconstruction levels and (b) defining a process for switching between the two scalar quantizers.

[0089] The two scalar quantizers used (represented by Q0 and Q1) Figure 6 The diagram shows the available reconstruction level. The position of the available reconstruction level is uniquely specified by the quantization step size Δ. The scalar quantizer used (Q0 or Q1) is not explicitly sent via signaling in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity of the transform coefficient level preceding the current transform coefficient in the encoding / decoding or reconstruction order.

[0090] Figure 7 This is a block diagram illustrating an exemplary state machine for switching between two different scalar quantizers according to some embodiments of the present disclosure.

[0091] like Figure 7 As shown, the switching between the two scalar quantizers (Q0 and Q1) is implemented via a state machine with four quantizer states (QState). QState can take four distinct values: 0, 1, 2, and 3. It is uniquely determined by the parity of the level of the transform coefficients preceding the current transform coefficient in the encoding / decoding / reconstruction order. At the start of inverse quantization of the transform block, the state is set to 0. The transform coefficients are reconstructed in scan order (i.e., in the same order in which they are entropy decoded). After reconstructing the current transform coefficient, the state is updated, as shown... Figure 7 As shown in the figure, k represents the value of the transformation coefficient level.

[0092] It also supports sending default and user-defined scaling matrices via signaling. The default (DEFAULT) mode scaling matrices are all flat, with elements equal to 16 for all TB sizes. IBC and intra-frame coding modes currently share the same scaling matrix. Therefore, for the case of user-defined (USER_DEFINED) matrices, the number of MatrixType and MatrixType_DC are updated as follows: MatrixType: 30 = 2 (2 for intra-frame & IBC / inter-frame) × 3 (Y / Cb / Cr components) × 5 (square TB size: for luma from 4×4 to 64×64, for chroma from 2×2 to 32×32) MatrixType_DC: 14 = 2 (2 for intra-frame & IBC / inter-frame × 1 for Y component) × 3 (TB size: 16×16, 32×32, 64×64) + 4 (2 for intra-frame & IBC / inter-frame × 2 for Cb / Cr component) × 2 (TB size: 16×16, 32×32).

[0093] DC values ​​are individually encoded for the following scaling matrices: 16×16, 32×32, and 64×64. For TBs smaller than 8×8, all elements of a scaling matrix are sent via signaling. If the TB size is greater than or equal to 8×8, only 64 elements of an 8×8 scaling matrix are sent via signaling as the basic scaling matrix. To obtain square matrices larger than 8×8, the 8×8 basic scaling matrix is ​​upsampled (by copying the elements) to the corresponding square size (i.e., 16×16, 32×32, 64×64). When the high-frequency coefficients of the 64-point transform are zeroed out, the corresponding high frequencies of the scaling matrix are also zeroed out. That is, if the width or height of the TB is greater than or equal to 32, only the left or upper half of the coefficients are retained, and the remaining coefficients are assigned zeros. Furthermore, the number of elements sent via signaling for a 64×64 scaling matrix is ​​reduced from 8×8 to three 4×4 submatrices because the bottom right 4×4 elements are never used.

[0094] The choice of probabilistic model for semantic elements related to the absolute value of the transform coefficient level depends on the absolute level value in the local neighborhood or the absolute level value of the partial reconstruction.

[0095] The chosen probabilistic model depends on the sum of the absolute levels (or partially reconstructed absolute levels) in the local neighborhood and the number of absolute levels greater than 0 in the local neighborhood (given by the number of sig_coeff_flags equal to 1). Contextual modeling and binarization depend on the following metrics of the local neighborhood: •numSig: The number of non-zero levels in the local neighborhood; •sumAbs1: The sum of the absolute levels (absLevel1) of the partial reconstruction in the local neighborhood after the first round; •sumAbs: The sum of the absolute levels of reconstruction in the local neighborhood. • Diagonal position (d): The sum of the x and y coordinates of the current scan position within the transform block.

[0096] Based on the values ​​of numSig, sumAbs1, and d, a probabilistic model is selected for encoding sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag. The Rice parameter is selected for binarizing abs_remainder and dec_abs_level based on the values ​​of sumAbs and numSig.

[0097] In the current VVC, the reduced 32-point MTS (also known as RMTS32) is based on skipping high-frequency coefficients and is used to reduce the computational complexity of 32-point DST-7 / DCT-8. Furthermore, it is accompanied by changes in coefficient encoding and decoding, including all types of zeroing (i.e., RMTS32 and the existing zeroing for high-frequency components in DCT2). Specifically, the binarization of the encoding and decoding of the last non-zero coefficient position is based on a reduced TU size, and the choice of context model for encoding and decoding this last non-zero coefficient position is determined by the original TU size. Additionally, 60 context models are used to encode the sig_coeff_flag of the transform coefficients. The selection of the context model index is based on the maximum value among the five previously reconstructed absolute levels called locSumAbsPass1 and the sum of the dependent quantized state QState, as follows: If cIdx equals 0, then ctxInc is exported as follows: ctxInc = 12 * Max(0, QState – 1) + Min((locSumAbsPass1 + 1)>>1, 3) +(d<2 8: (d<5 4 : 0)) Otherwise, (if cIdx is greater than 0), ctxInc is exported as follows: ctxInc = 36 + 8 * Max(0, QState − 1) + Min((locSumAbsPass1 + 1)>>1,3) + (d<2 4:0).

[0098] Figure 8This is a flowchart illustrating an exemplary process by which a video decoder performs residual encoding and decoding for a transform skip mode coding block according to some embodiments of the present disclosure.

[0099] In some embodiments, when encoding the residuals, a uniform (identical) Rice parameter (ricePar) is derived and used to send the semantic elements abs_remainder and dec_abs_level via signaling. The only difference is that baseLevel is set to 4 and 0 for encoding abs_remainder and dec_abs_level, respectively. The Rice parameter is determined not only based on the sum of the absolute levels of the five adjacent transform coefficients in the local template, but also based on the corresponding base level, as follows: RicePara = RiceParTable[max(min(31, sumAbs – 5 * baseLevel), 0)].

[0100] In other words, the binary codewords for the semantic elements `abs_remainder` and `dec_abs_level` are adaptively determined based on the level information of adjacent coefficients. Since this codeword determination is performed on a per-sample basis, it requires additional logic to handle this codeword adaptation for coefficient encoding. Similarly, when encoding residual blocks, the binary codeword for the semantic element `abs_remainder` is adaptively determined based on the level information of adjacent coefficients.

[0101] Furthermore, when encoding semantic elements related to residual encoding / decoding or transform coefficient encoding / decoding, the chosen probabilistic model depends on the level information of adjacent levels, and therefore requires additional logic and additional context models to handle the context selection.

[0102] In some embodiments, the video codec uses a fixed binary codeword for residual encoding of the semantic element `abs_remainder`. The binary codeword can be formed using (but is not limited to) the following methods (details of which can be found in the VVC draft specification): (1) The process of determining the codeword of abs_remainder is the same as the process used in VVC with fixed rice parameters (e.g., 1, 2, or 3). (2) Fixed-length binarization (3) Truncated Rice binarization (4) Truncated binary (TB) binarization process (5) k-th order Exp-Golomb binarization process (EGk) (6) Finite k-order Exp-Golomb binarization (7) Fixed-length binarization.

[0103] When decoding a transform skip mode coded block using a constant Rice parameter, the video decoder first receives the video data corresponding to the transform skip mode coded block from the bitstream (810). It is assumed here that the block is encoded using the transform skip mode and that the video decoder does not perform an inverse transform during the decoding process.

[0104] Next, the video decoder decodes from the received video data the first codeword (e.g., abs_reaminder), the second codeword (e.g., sign_coeff_flag), and the first set of codewords (820) for the pixels within the transform skip mode coding block.

[0105] The decoder derives the initial level value (e.g., dec_abs_level) from the first set of codewords (830).

[0106] The video decoder then converts the first codeword to a remainder (840) of a pixel based on a predefined mapping generated using a constant Rice parameter. For example, the Rice parameter can have a constant value of 1, 2, or 3. In some embodiments, the Rice parameter is set to 1.

[0107] In some embodiments, the generation of a predefined mapping relationship between a specific remainder value abs_remainder and the corresponding codeword involves the following steps: 1) The constant binarization parameter cMax is determined based on the constant Rice parameter as follows: cMax = 6< <cRiceParam; 2) By calling the truncated Rice binarization process, the prefix value, prefixVal, of the corresponding codeword is exported as follows: prefixVal = Min(cMax, abs_remainder); and 3) When the prefix binary string is equal to a string of length 6 with all bits equal to 1, derive the suffix value suffixVal of the corresponding codeword, as follows: suffixVal = abs_remainder–cMax.

[0108] As mentioned above, the suffix value of the codeword corresponding to a specific remainder value is optional because it only exists if the prefix value of the codeword satisfies certain conditions. In some embodiments, this predefined mapping is constructed into the memory of the video encoder / decoder in the form of a lookup table, allowing the video encoder to quickly find the codeword, including the prefix value and the optional suffix value, in the lookup table for a given remainder value. Similarly, the video decoder can quickly find the remainder for the corresponding codeword in the lookup table.

[0109] Next, the video decoder converts the second codeword into the sign value (e.g., positive or negative) of the remainder (obtained from the first codeword) (850).

[0110] The video decoder then derives the quantization residual (860) of the pixel from the remainder, the sign value, and the initial level value.

[0111] In some embodiments, the video codec uses fixed codewords for coefficient encoding and decoding for the semantic elements abs_remainder and dec_abs_level. The binary codewords can be formed using (but are not limited to) the following methods.

[0112] (1) The process of determining the codewords for abs_remainder and dec_abs_level is the same as that used in VVC with fixed Rice parameters (e.g., 1, 2, or 3). The baseLevel used in the current VVC for abs_remainder and dec_abs_level can still be different (e.g., baseLevel is set to 4 and 0 for encoding abs_remainder and dec_abs_level, respectively). (2) The process for determining the codewords for abs_remainder and dec_abs_level is the same as that used in VVC with fixed Rice parameters (e.g., 1, 2, or 3). The baseLevels for abs_remainder and dec_abs_level are also uniform (e.g., 0 for both or 4 for both). (3) Fixed-length binarization (4) Truncated Rice Binarization (5) Truncated binary (TB) binarization process (6) k-th order Exp-Golomb binarization process (EGk) (7) Finite k-order Exp-Golomb binarization (8) Fixed-length binarization.

[0113] In some embodiments, the video codec uses a single context to encode semantic elements (e.g., abs_level_gtx_flag) associated with residual or coefficient coding and may remove context selection based on adjacent decoded level information.

[0114] Figure 9 This is a block diagram illustrating an example scenario adaptive binary arithmetic codec (CABAC) engine according to some embodiments of this disclosure.

[0115] Context-Adaptive Binary Arithmetic Codec (CABAC) is a form of entropy coding used in many video codec standards, such as H.264 / MPEG-4 AVC, High Efficiency Video Codec (HEVC), and VVC. CABAC is based on arithmetic coding with several modifications to adapt it to the needs of video codec standards. For example, CABAC encodes binary symbols, which maintains low complexity and allows for probabilistic modeling of the more frequently used bits of any symbol. The probability model is adaptively chosen based on local context to allow for better modeling of probabilities, as coding and decoding patterns are generally well-correlated locally. Finally, CABAC uses a non-multiplicative range division by employing quantized probability ranges and probability states.

[0116] CABAC employs multiple probability models for different scenarios. It first converts all non-binary symbols to binary. Then, for each binary bit (also called a "bit"), the codec selects which probability model to use and optimizes the probability estimate using information from nearby elements. Finally, arithmetic encoding and decoding are applied to compress the data.

[0117] Context modeling provides estimates of the conditional probabilities of encoding and decoding symbols. Using a suitable context model, redundancy between given symbols can be utilized by switching between different probability models, based on the already encoded symbols in the neighborhood of the current symbol to be encoded. Encoding data symbols involves the following stages.

[0118] Binarization: CABAC uses binary arithmetic encoding and decoding, meaning that only binary decisions (1 or 0) are encoded. Before arithmetic encoding and decoding, non-binary value symbols (such as transform coefficients or motion vectors) are "binarized" or converted into binary code. This process is similar to converting data symbols into variable-length code, but the binary code is further encoded (performed by the arithmetic codec) before transmission. The individual stages are repeated for each binary bit (or "bit") of the binarized symbol.

[0119] Context Model Selection: A "context model" is a probabilistic model for one or more bits of a binary symbol. This model can be selected based on statistics of recently encoded data symbols, using a selection of available models. The context model stores the probability that each bit is "1" or "0".

[0120] Arithmetic Encoding: The arithmetic encoder encodes each bit according to the selected probability model. Note that each bit has only two subranges (corresponding to "0" and "1").

[0121] Probability update: Update the selected scenario model based on the actual encoded value (e.g., if the binary bit value is "1", increment the frequency count of "1").

[0122] In CABAC, further processing of each bit value depends on an associated encoding mode decision, which can be either a regular mode or a bypass mode, by decomposing each non-binary semantic element value into a series of bits. The latter is chosen for the bits, assuming a uniform distribution, and thus simply bypasses the entire regular binary arithmetic encoding (and decoding) process for the bits. In the regular encoding mode, each bit value is encoded using a regular binary arithmetic encoding engine, where the associated probabilistic model is determined either by a fixed choice based on the semantic element's type and the bit position or bit index (binIdx) in the semantic element's binary representation, or adaptively from two or more probabilistic models based on relevant lateral information such as spatial domain neighbors, components, depth or size of the CU / PU / TU, or position within the TU. The selection of the probabilistic model is called contextual modeling. As an important design decision, the latter case is typically applied only to the most frequently observed bits, while another, less frequently observed bit will be processed using a joint, typically zero-order probabilistic model. In this way, CABAC achieves selective adaptive probabilistic modeling at the sub-symbol level, and thus provides an efficient tool for leveraging inter-symbol redundancy with a significantly reduced overall modeling or learning cost. Note that, in principle, for both fixed and adaptive cases, the switch from one probabilistic model to another can occur between any two consecutive conventionally encoded bits. Generally, the design of the context model in CABAC reflects a conflicting goal of finding a good trade-off between avoiding unnecessary modeling costs and making substantial use of statistical dependencies.

[0123] In CABAC, the parameters of the probabilistic models are adaptive, meaning that the adaptation of the model probabilities to the statistical changes of the binary bit source is performed bit-by-bit in a backward adaptive and synchronous manner in both the encoder and decoder; this process is called probability estimation. For this purpose, each probabilistic model in CABAC can be chosen from 126 different states, with the associated model probability value p ranging from [0:01875;0:98125]. Two parameters for each probabilistic model are stored as 7-bit entries in the context memory: 6 bits for each of the 63 probability states representing the least likely symbol (LPS) pLPS, and 1 bit for nMPS, the value of the most likely symbol (MPS).

[0124] In one or more examples, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted through a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include: a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium; or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the embodiments described in this application. A computer program product can include a computer-readable medium.

[0125] The terminology used in the description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof.

[0126] It will also be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the embodiments, a first electrode may be referred to as a second electrode, and similarly, a second electrode may be referred to as a first electrode. Both the first electrode and the second electrode are electrodes, but they are not the same electrode.

[0127] The description in this application is presented for illustrative and descriptive purposes and is not intended to be exhaustive or limiting of the invention in the disclosed forms. Many modifications, variations, and alternative embodiments will be apparent to those skilled in the art from the teachings presented in the foregoing description and the accompanying drawings. The embodiments were chosen and described in order to best explain the principles of the invention, its practical application, and to enable others skilled in the art to understand the various embodiments of the invention and to best utilize the basic principles and various embodiments with various modifications, as suited to the particular intended use. Therefore, it should be understood that the scope of the claims is not limited to the specific examples of the disclosed embodiments, and that modifications and other embodiments are intended to be included within the scope of the appended claims.

Claims

1. A method for decoding video data, comprising: Receive video data corresponding to the transform skip mode coded blocks from the bitstream; Determine the first codeword, the second codeword, and the first group of codewords for the pixels within the transform skip mode coding block from the video data; Derive the initial level value from the first group of codewords; The first codeword is converted into the remainder of the pixel based on a predefined mapping relationship generated using a constant Rice parameter for the transform skip mode coding block; The sign for the remainder is determined based on the second codeword; The quantization residual of the pixel is derived from the remainder, the sign, and the initial level value; as well as Reconstructed samples of pixels are obtained based on the quantization residuals.

2. The method according to claim 1, wherein the predefined mapping relationship is generated in the following manner: The constant binarization parameter is calculated using the constant Rice parameter; and The constant binarization parameter is used to determine the prefix value of a codeword and the optional suffix value of the corresponding codeword for a candidate remainder value.

3. The method according to claim 2, wherein, According to the transformation skip mode, the constant Rice parameter is set to 1.

4. The method of claim 2, wherein the prefix value of the codeword is determined using a truncated Rice binarization process.

5. The method according to claim 1, wherein the initial level value is used to derive the quantization residual of the pixel based on the position of the pixel within the transform skip mode coding block.

6. The method according to claim 1, wherein the first codeword is a fixed binary codeword of the remainder of the pixel having a predefined value.

7. The method according to claim 1, wherein the first codeword corresponds to the semantic element abs_remainder, and the second codeword corresponds to the semantic element sign_coefficient_flag.

8. The method of claim 1, wherein a single context model is used to decode the first codeword, the second codeword, and the first set of codewords.

9. A method for encoding video data, comprising: Divide video images into video blocks; Determine the quantization residual of the pixels within the video block; The remainder of the pixel, the sign of the remainder, and the initial level value are determined based on the quantization residual; The remainder of the pixel is converted into the first codeword for the pixel within the transform skip mode coding block according to a predefined mapping relationship generated using a constant Rice parameter, wherein the constant Rice parameter is set to 1 according to the transform skip mode; The second codeword for the pixel within the transform skip mode coding block is determined based on the sign of the remainder; Based on the initial level value, determine the first set of codewords for the pixels within the transform skip mode coding block; The first codeword, the second codeword, and the first group of codewords are encoded into video data corresponding to the transform skip mode coding block; as well as Video data corresponding to the transformed skip mode coded block is inserted into the bitstream.

10. The method of claim 9, wherein the predefined mapping relationship is generated in the following manner: The constant binarization parameter is calculated using the constant Rice parameter; and The constant binarization parameter is used to determine the prefix value of a codeword and the optional suffix value of the corresponding codeword for a candidate remainder value.

11. The method of claim 10, wherein the prefix value of the codeword is determined using a truncated Rice binarization process.

12. The method of claim 9, wherein the initial level value of the quantization residual of the pixel is derived based on the position of the pixel within the transform skip mode coding block.

13. The method of claim 9, wherein the first codeword is a fixed binary codeword of the remainder of the pixel having a predefined value.

14. The method according to claim 9, wherein the first codeword corresponds to the semantic element abs_remainder, and the second codeword corresponds to the semantic element sign_coefficient_flag.

15. The method of claim 9, wherein a single context model is used to encode the first codeword, the second codeword, and the first set of codewords.

16. An electronic device comprising: One or more processing units; Memory, coupled to the one or more processing units; and A plurality of programs stored in the memory, which, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 1-8.

17. An electronic device comprising: One or more processing units; Memory, coupled to the one or more processing units; and A plurality of programs stored in the memory, which, when executed by the one or more processing units, cause the electronic device to perform the method of any one of claims 9-15.

18. A non-transitory computer-readable storage medium storing a plurality of programs for execution by an electronic device having one or more processing units, wherein the plurality of programs, when executed by the one or more processing units, cause the electronic device to perform the method according to any one of claims 9-15, wherein the non-transitory computer-readable storage medium stores a bit stream thereon, the bit stream being generated by the method according to any one of claims 9-15.

19. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-8.

20. A method for storing a bit stream, comprising: A bit stream is generated by performing the method according to any one of claims 9-15; as well as Store the bit stream.

21. A method for transmitting a bit stream, comprising: A bit stream is generated by performing the method according to any one of claims 9-15; as well as Send the bit stream.

Images