Scalar quantizer decision scheme for dependent scalar quantization

Abstract

A scalar quantizer decision scheme for dependent scalar quantization is presented. In one implementation, in a first scan pass for entropy decoding transform coefficients of a block in a picture, a first set of syntax flags and a second set of syntax flags are entropy decoded, the first set of syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than 1, and the second set of syntax flags indicating whether the absolute value of the corresponding transform coefficient is greater than 1. The second group of syntax flags indicate whether the absolute value of the corresponding transform coefficient is greater than 3; entropy decoding a first type of syntax elements indicating a remaining level of transform coefficients in a second scan pass for entropy decoding the transform coefficients of the block; entropy decoding a second type of syntax elements indicating symbols of the transform coefficients in a third scan pass for entropy decoding the transform coefficients of the block; obtaining a transform coefficient of the block based on the first group of syntax flags, the second group of syntax flags, the first type of syntax elements and the second type of syntax elements; and reconstructing the block in response to the acquired transform coefficients of the block.

Description

[0001] This application is a divisional application of the Chinese invention patent application with the application date of September 13, 2019, application number 201980061637.2, invention name “Scalar Quantizer Decision Scheme for Dependent Scalar Quantization” and the applicant being Interactive Digital VC Holdings Co., Ltd. Technical Field

[0002] Embodiments of the present invention generally relate to methods and apparatus for video encoding or decoding. Background Art

[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame picture correlation, and then the difference between the original block and the predicted block (often expressed as prediction error or prediction residual) is transformed, quantized and entropy encoded. To reconstruct the video, the compressed data is decoded by the inverse process corresponding to the entropy encoding, quantization, transformation and prediction. Summary of the invention

[0004] According to one embodiment, a video decoding method is provided, comprising: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; entropy decoding the first and second sets of parameters in a first pass for the block, wherein the first pass is performed before one or more other passes for entropy decoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1 and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2; and reconstructing the block in response to the decoded transform coefficients.

[0005] According to one embodiment, a method for video encoding is provided, comprising: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a conventional mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy encoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before one or more other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2.

[0006] According to another embodiment, an apparatus for video decoding is provided, comprising one or more processors, wherein the one or more processors are configured to: access a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; access a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a conventional mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; entropy decode the first and second sets of parameters in a first pass for the block, wherein the first pass is performed before one or more other passes for entropy decoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1 and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2; and reconstruct the block in response to the decoded transform coefficients. The apparatus may also include one or more memories coupled to the one or more processors.

[0007] According to another embodiment, a device for video encoding is provided, comprising one or more processors, wherein the one or more processors are configured to: access a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; access a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a regular mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy encode the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before one or more other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2.

[0008] According to another embodiment, an apparatus for video decoding is provided, comprising: means for accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; means for accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a regular mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; means for entropy decoding the first and second sets of parameters in a first pass for the block, wherein the first pass is performed before one or more other passes for entropy decoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1 and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2; and means for reconstructing the block in response to the decoded transform coefficients.

[0009] According to another embodiment, a video encoding apparatus is provided, comprising: a component for accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; a component for accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a conventional mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and a component for entropy encoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before one or more other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2.

[0010] According to another embodiment, a signal comprising an encoded video is generated by performing the following operations: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in coding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a conventional mode, wherein context modeling of at least one parameter of the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy encoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before one or more other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a first flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and a second flag indicating whether the absolute value of the corresponding transform coefficient is greater than 2.

[0011] According to one embodiment, a method is provided, comprising: entropy decoding a first group of syntax flags and a second group of syntax flags in a first scan pass for entropy decoding transform coefficients of a block in a picture, the first group of syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than 1, and the second group of syntax flags indicating whether an absolute value of the corresponding transform coefficient is greater than 3; entropy decoding a first type of syntax element indicating a residual level of a transform coefficient in a second scan pass for entropy decoding the transform coefficients of the block; entropy decoding a second type of syntax element indicating a sign of a transform coefficient in a third scan pass for entropy decoding the transform coefficients of the block; obtaining the transform coefficients of the block based on the first group of syntax flags, the second group of syntax flags, the first type of syntax element, and the second type of syntax element; and reconstructing the block in response to the obtained transform coefficients of the block.

[0012] According to one embodiment, a device is provided, comprising one or more processors, wherein the one or more processors are configured to: in a first scan pass for entropy decoding transform coefficients of a block in a picture, entropy decode a first group of syntax flags and a second group of syntax flags, the first group of syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than 1, and the second group of syntax flags indicating whether an absolute value of the corresponding transform coefficient is greater than 3; in a second scan pass for entropy decoding the transform coefficients of the block, entropy decode a first type of syntax element indicating a residual level of the transform coefficient; in a third scan pass for entropy decoding the transform coefficients of the block, entropy decode a second type of syntax element indicating a sign of the transform coefficient; obtain the transform coefficients of the block based on the first group of syntax flags, the second group of syntax flags, the first type of syntax element, and the second type of syntax element; and reconstruct the block in response to the obtained transform coefficients of the block.

[0013] According to one embodiment, a method is provided, comprising: obtaining transform coefficients of a block in a picture; entropy encoding a first group of syntax flags and a second group of syntax flags in a first scan pass for entropy encoding the transform coefficients of the block in the picture, wherein the first group of syntax flags indicates whether an absolute value of the corresponding transform coefficient is greater than 1, and the second group of syntax flags indicates whether an absolute value of the corresponding transform coefficient is greater than 3; entropy encoding a first type of syntax element indicating a residual level of the transform coefficient in a second scan pass for entropy encoding the transform coefficients of the block; and entropy encoding a second type of syntax element indicating a sign of the transform coefficient in a third scan pass for entropy encoding the transform coefficients of the block.

[0014] According to one embodiment, a device is provided, comprising at least one memory and one or more processors, wherein the one or more processors are configured to: obtain transform coefficients of a block in a picture; in a first scan pass for entropy encoding the transform coefficients of the block in the picture, entropy encode a first group of syntax flags and a second group of syntax flags, the first group of syntax flags indicating whether the absolute value of the corresponding transform coefficient is greater than 1, and the second group of syntax flags indicating whether the absolute value of the corresponding transform coefficient is greater than 3; in a second scan pass for entropy encoding the transform coefficients of the block, entropy encode a first type of syntax element indicating a residual level of the transform coefficient; and in a third scan pass for entropy encoding the transform coefficients of the block, entropy encode a second type of syntax element indicating a sign of the transform coefficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A block diagram of a system is shown in which aspects of the present embodiments may be implemented.

[0016] Figure 2 A block diagram of an embodiment of a video encoder is shown.

[0017] Figure 3 A block diagram of an embodiment of a video decoder is shown.

[0018] Figure 4 is a diagrammatic example showing two scalar quantizers used in dependency quantization proposed in JVET-J0014.

[0019] Figure 5 is a graphical example showing state transition and quantizer selection for dependency quantization proposed in JVET-J0014.

[0020] Figure 6 is a pictorial example showing the order of coefficient bits in CG as proposed in JVET-J0014.

[0021] Figure 7 is a diagrammatic example showing a SIG-based state transition as proposed in JVET-K0319.

[0022] Figure 8 is a pictorial example showing the order of coefficient bits in CG as proposed in JVET-K0319.

[0023] Fig. 9 is a pictorial example showing the order of coefficient bits in a CG according to one embodiment.

[0024] Fig.10is a diagrammatic example showing a state transition based on SUM(SIG, gt1, gt2) according to one embodiment.

[0025] Fig.11 is a diagrammatic example showing a state transition based on XOR(SIG, gt1, gt2) according to one embodiment.

[0026] Fig.12 is a diagrammatic example showing a gt1-based state transition according to one embodiment.

[0027] Fig.13 A process of encoding a current block according to one embodiment is shown.

[0028] Fig.14 A process of decoding a current block according to one embodiment is shown. DETAILED DESCRIPTION

[0029] Figure 1 A block diagram of an example of a system in which various aspects and embodiments can be implemented is shown. System 100 can be embodied as a device including various components described below, and is configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances and servers. The elements of system 100 can be embodied in a single integrated circuit, multiple ICs and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed over multiple ICs and / or discrete components. In various embodiments, system 100 is coupled to other similar systems or other electronic devices via, for example, a communication bus or by a dedicated input and / or output port communication. In various embodiments, system 100 is configured to implement one or more aspects described in this application.

[0030] The system 100 includes at least one processor 110, which is configured to execute instructions loaded therein to implement various aspects described in the present application, for example. The processor 110 may include embedded memory, input and output interfaces, and various other circuits known in the art. The system 100 includes at least one memory 120 (e.g., a volatile storage device and / or a non-volatile storage device). The system 100 includes a storage device 140, which may include a non-volatile memory and / or a volatile memory, including but not limited to an EEPROM, a ROM, a PROM, a RAM, a DRAM, an SRAM, a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 140 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0031] The system 100 includes an encoder / decoder module 130, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that can be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 130 may be implemented as a separate element of the system 100, or may be incorporated into the processor 110 as a combination of hardware and software, which is well known to those skilled in the art.

[0032] Program code to be loaded onto the processor 110 or the encoder / decoder 130 to perform various aspects described in the present application may be stored in the storage device 140 and subsequently loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, the memory 120, the storage device 140, and the encoder / decoder module 130 may store one or more of various items during the execution of the processes described in the present application. Such stored items may include, but are not limited to, input video, decoded video or a portion of decoded video, bitstreams or signals, matrices, variables, and intermediate or final results from equations, formulas, operations, and operational logic.

[0033] In several embodiments, memory internal to the processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory may be a memory 120 and / or a storage device 140, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations such as MPEG-2, HEVC, or VVC.

[0034] Input to the elements of system 100 may be provided through various input devices as indicated in block 105. Such input devices include, but are not limited to: (i) an RF section that receives an RF signal wirelessly transmitted, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0035] In various embodiments, the input device of block 105 has associated corresponding input processing elements known in the art. For example, the RF portion may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal that sends on wired (for example cable) medium, and filter to the frequency band of expectation again by filtering, down-conversion and perform frequency selection.Each embodiment rearranges the order of above-mentioned (and other) element, removes some in these elements, and / or adds other elements that perform similar or different functions.Adding element can include and insert element between existing element, for example, inserts amplifier and analog-to-digital converter.In various embodiments, the RF part comprises antenna.

[0036] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or in the processor 110 as desired. Similarly, various aspects of USB or HDMI interface processing may be implemented, for example, in a separate interface IC or in the processor 110 as desired. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 110 and an encoder / decoder 130 that operates in combination with memory and storage elements to process the data stream for presentation on an output device as desired.

[0037] The various elements of the system 100 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data may be transmitted between them using suitable connection means 115 (eg, an internal bus known in the art, including an I2C bus, wiring, and printed circuit boards).

[0038] The system 100 includes a communication interface 150 that enables communication with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data over the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or a network card, and the communication channel 190 may be implemented, for example, in a wired and / or wireless medium.

[0039] In various embodiments, a Wi-Fi network such as IEEE 802.11 is used to stream data to the system 100. The Wi-Fi signals of these embodiments are received on a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 of these embodiments is typically connected to an access point or router that provides access to an external network including the Internet to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to the system 100, which delivers data through the HDMI connection of the input block 105. Other embodiments also use the RF connection of the input block 105 to provide streaming data to the system 100.

[0040] The system 100 can provide output signals to various output devices including a display 165, a speaker 175, and other peripherals 185. In various examples of embodiments, the other peripherals 185 include one or more of a stand-alone DVR, a hard disk player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 100. In various embodiments, control signals are transmitted between the system 100 and the display 165, the speaker 175, or other peripherals 185 using signaling such as AV.Link, CEC, or other communication protocols that can achieve device-to-device control with or without user intervention. The output devices can be communicatively coupled to the system 100 via dedicated connections through the respective interfaces 160, 170, and 180. Alternatively, the output devices can be connected to the system 100 using a communication channel 190 via the communication interface 150. The display 165 and the speaker 176 can be integrated into a single unit with other components of the system 100 in an electronic device (e.g., a television). In various embodiments, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0041] For example, if the RF portion of input 105 is part of a separate set-top box, the display 165 and speaker 175 may alternatively be separate from one or more other components. In various embodiments where the display 165 and speaker 175 are external components, the output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0042] Figure 2 An example video encoder 200 is shown, such as a High Efficiency Video Coding (HEVC) encoder. Figure 2 Also shown are encoders in which improvements are made to the HEVC standard or encoders that employ techniques similar to HEVC, such as VVC (Versatile Video Coding) being developed by JVET (Joint Video Exploration Team).

[0043] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "encoded" or "encoded" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0044] Prior to encoding, the video sequence may undergo pre-encoding processing (201), such as color conversion of an input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or remapping of input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0045] To encode a video sequence having one or more pictures, a picture is partitioned (202) into, for example, one or more slices, where each slice may include one or more fragments. In HEVC, a fragment is organized into coding units, prediction units, and transform units. The HEVC specification distinguishes between "blocks" and "units", where a "block" addresses a specific area in the sample array (e.g., luma, Y), and a "unit" includes a concatenated block of all coded color components (Y, Cb, Cr or monochrome), syntax elements, and prediction data (e.g., motion vectors) associated with these blocks.

[0046] For encoding according to HEVC, a picture is partitioned into square coding tree blocks (CTBs) of configurable size (typically 64x64, 128x128 or 256x256 pixels), and a set of consecutive coding tree blocks are grouped into a slice. A coding tree unit (CTU) contains the CTBs of the encoded color components. The CTB is the root of a quadtree partitioned into coding blocks (CBs), and a coding block can be partitioned into one or more prediction blocks (PBs) and forms the root of a quadtree partitioned into transform blocks (TBs). Transform blocks (TBs) larger than 4x4 are partitioned into 4x4 sub-blocks of quantized coefficients, called coefficient groups (CGs). Corresponding to coding blocks, prediction blocks and transform blocks, a coding unit (CU) includes a prediction unit (PU) and a tree-structured set of transform units (TUs), the PU including prediction information for all color components and the TU including a residual coding syntax structure for each color component. The sizes of the CB, PB and TB of the luminance component apply to the corresponding CU, PU and TU. In this application, the term "block" may be used to refer to any of, for example, CTU, CU, PU, ​​TU, CG, CB, PB, and TB. In addition, the term "block" may also be used to refer to macroblocks and partitions specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0047] In encoder 200, a picture is encoded by encoder elements as described below. The picture to be encoded is processed in units of, for example, CUs. Each coding unit is encoded using intra or inter mode. When a coding unit is encoded in intra mode, it performs intra prediction (260). In inter mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) which of intra mode or inter mode to use to encode the coding unit, and indicates the intra / inter decision by a prediction mode flag. The prediction residual is calculated by subtracting (210) the predicted block from the original image block.

[0048] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy encoded (245) to output a bitstream. As a non-limiting example, context-based adaptive binary arithmetic coding (CABAC) may be used to encode syntax elements into the bitstream.

[0049] In order to encode with CABAC, the non-binary syntax element values ​​are mapped to a binary sequence called a bit string by a binarization process. For a bit, a context model is selected. A "context model" is a probability model for one or more bits, and is selected from a selection of available models based on the statistics of the most recently encoded symbols. The context model of each bit is identified by a context model index (also called a "context index"), and different context indexes correspond to different context models. The context model stores the probability of each bit as '1' or '0', and can be adaptive or static. The static model triggers the coding engine with equal probabilities of 1' and '0' for the bit. In the adaptive coding engine, the context model is updated based on the actual coded value of the bit. The operating modes corresponding to the adaptive model and the static model are respectively referred to as normal mode and bypass mode. Based on the context, the binary arithmetic coding engine encodes or decodes the bit according to the corresponding probability model.

[0050] The scan mode converts a 2D block into a 1D array and defines the order in which samples or coefficients are processed.A scan pass is an iteration over the transform coefficients in a block (according to the selected scan mode) in order to encode a particular syntax element.

[0051] In HEVC, a scan pass on a TB consists in processing each CG in sequence according to the scan mode (diagonal, horizontal, vertical), and also scanning the 16 coefficients within each CG according to the considered scan order. The scan starts with the last significant coefficient in the TB and processes all coefficients until the DC coefficient. The CGs are scanned sequentially. A maximum of five scans can be applied to the CG. Each scan pass encodes the syntax elements of the coefficients within the CG as follows:

[0052] ·Significant_coeff_flag (SIG): The significance of the coefficient (zero / non-zero). ·Coefficient absolute level greater than 1 flag (gt1, coeff_abs_level_greater1_flag): Indicates whether the absolute value of the coefficient level is greater than 1.

[0053] Coefficient absolute level greater than 2 flag (gt2, coeff_abs_level_greater2_flag): indicates whether the absolute value of the coefficient level is greater than 2.

[0054] · Coefficient sign flag (coeff_sign_flag): The sign of the significant coefficient (0: positive, 1: negative).

[0055] · Coefficient remaining absolute level (coeff_abs_level_remaining): The remaining value of the absolute value of the coefficient level (if the value is larger than after encoding in the previous pass). Up to 8 coeff_abs_level_greater1_flags can be encoded in a given CG, and coeff_abs_level_greater2_flag is encoded only for the first coefficient in the CG with a magnitude greater than 1.

[0056] In each pass, syntax is encoded only if it is determined necessary by the previous pass. For example, if a coefficient is unimportant, then the remaining passes are unnecessary for that coefficient. The bits in the first three passes are encoded in normal mode, where the context model index depends on the position of the particular coefficient in the TB and the value of the previously encoded coefficient in the adjacent domain covered by the local template. The bits in passes 4 and 5 are encoded in bypass mode, so that all bypass bits in the CG are grouped together.

[0057] The encoder may also skip the transform and apply quantization directly to the untransformed residual signal, for example on a 4x4 TU basis. The encoder may also bypass both transform and quantization, i.e., encode the residual directly without applying a transform or quantization process. In direct PCM encoding, no prediction is applied and the coding unit samples are encoded directly into the bitstream.

[0058] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) and the image block is reconstructed. A loop filter (265) is applied to the reconstructed picture, for example, to perform deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (280).

[0059] Figure 3 A block diagram of an example video decoder 300, such as a HEVC decoder, is shown. In the decoder 300, a bitstream is decoded by decoder elements as described below. Figure 2 As described in , video decoder 300 typically performs a decoding pass corresponding to an encoding pass, which performs video decoding as part of encoding video data. Figure 3 A decoder in which improvements are made to the HEVC standard or a decoder that adopts a technique similar to HEVC (such as a VVC decoder) may also be shown.

[0060] In particular, the input to the decoder includes a video bitstream, which may be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, picture segmentation information, and other encoded information. If CABAC is used for entropy coding, the context model is initialized in the same way as the encoder context model, and syntax elements are decoded from the bitstream based on the context model.

[0061] The picture partitioning information indicates how the picture is partitioned, for example, the size of the CTU, and the manner in which the CTU is split into CUs and possibly PUs when applicable. Thus, the decoder can divide (335) the picture into, for example, CTUs and each CTU into CUs according to the decoded picture partitioning information. The transform coefficients are dequantized (340) and inversely transformed (350) to decode the prediction residual.

[0062] The decoded prediction residual and the prediction block are combined (355) and the image block is reconstructed. The prediction block can be obtained (370) from intra-frame prediction (360) or motion compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0063] The decoded picture may further undergo post-decoding processing (385), such as an inverse color transform (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0064] In the document JVET-J0014, 10th meeting: San Diego, USA, April 10-20, 2018 (hereinafter referred to as "JVET-J0014"), entitled "Description of SDR, HDR and 360° video coding technology proposal by Fraunhofer HHI", dependent scalar quantization is proposed, in which two scalar quantizers with different reconstruction levels are switched for quantization. Compared to traditional independent scalar quantization (as used in HEVC and VTM-1), the set of allowable reconstruction values ​​for transform coefficients depends on the values ​​of transform coefficient levels that are earlier than the current transform coefficient level in the reconstruction order.

[0065] Dependent scalar quantization is implemented by (a) defining two scalar quantizers with different reconstruction levels, and (b) defining a process of switching between the two scalar quantizers.

[0066] Figure 4 The two scalar quantizers used are shown in , denoted by Q0 and Q1. The location of the available reconstruction levels is uniquely specified by the quantization step size Δ. If we ignore the fact that the actual reconstruction of the transform coefficients uses integer arithmetic, the two scalar quantizers Q0 and Q1 are characterized as follows:

[0067] Q0: The reconstruction level of the first quantizer Q0 is given by an even integer multiple of the quantization step size Δ. When this quantizer is used, the reconstructed transform coefficient t' is calculated according to the following formula

[0068] t'=2·k·Δ

[0069] where k represents the associated transform coefficient level. It should be noted that the term "transform coefficient level" (k) refers to the quantized transform coefficient value, for example, which corresponds to the TransCoeffLevel described in the residual_coding syntax structure below. The term "reconstructed transform coefficient" (t') refers to the dequantized transform coefficient value.

[0070] Q1: The reconstruction level of the second quantizer Q1 is given by an odd integer multiple of the quantization step size Δ, otherwise the reconstruction level is equal to zero. The mapping of the transform coefficient level k to the reconstructed transform coefficient t' is specified by:

[0071] t'=(2·k–sgn(k))·Δ

[0072] Where sgn(·) represents the sign function, sgn(x)=(k==0?0:(k<0?–1:1)).

[0073] The used scalar quantizer (Q0 or Q1) is not explicitly signaled in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity check of the transform coefficient levels that precede the current transform coefficient in the coding / reconstruction order.

[0074] like Figure 5As shown in , the switching between the two scalar quantizers (Q0 and Q1) is performed via a state machine with four states. The state can take four different values: 0, 1, 2, 3. The state is uniquely determined by the parity of the transform coefficient level that precedes the current transform coefficient in the encoding / reconstruction order. At the beginning of inverse quantization of the transform block, the state is set equal to 0. The transform coefficients are reconstructed in scanning order (i.e., in the same order in which they are entropy encoded / decoded). After reconstructing the current transform coefficient, the state is updated as shown in Table 1, where k represents the value of the transform coefficient level. Note that the next state depends only on the current state and the parity of the current transform coefficient level k (k&1). With k representing the value of the current transform coefficient level, the state update can be written as

[0075] state=stateTransTable[state][k&1]

[0076] Where stateTransTable represents Figure 5 As shown in Table 1, the operator & specifies the bitwise "and" operator in two's complement arithmetic.

[0077] The state uniquely specifies the scalar quantizer to be used. If the state of the current transform coefficient is equal to 0 or 1, the scalar quantizer Q0 is used. Otherwise (the state is equal to 2 or 3), the scalar quantizer Q1 is used.

[0078] Table 1

[0079]

[0080] More generally, the quantizer for the transform coefficients may be selected among more than two scalar quantizers, and the state machine may have more than four states. Alternatively, quantizer switching may be handled via other possible mechanisms.

[0081] In JVET-J0014, a coefficient coding scheme coupled with dependent scalar quantization is also proposed, by which the context modeling for the quantized coefficients depends on the quantizer used. Specifically, each of the significance flag (SIG) and the greater than 1 flag (gt1) has two sets of context models, and the set of models selected for a particular SIG or gt1 depends on the quantizer used for the associated coefficient. Therefore, the coefficient coding proposed in JVET-J0014 requires the absolute level (absLevel) of the quantized coefficient to be fully reconstructed before moving to the next scanning position in order to know the parity check used to determine the quantizer and therefore the context set for the next coefficient. That is, in order to obtain the context model for entropy decoding of coefficient (n+1), entropy decoding of coefficient n (SIG, gt1, ..., gt4, sign flag, absolute residual level) needs to be completed. Therefore, some conventionally coded bits of coefficient (n+1) need to wait for the decoding of some bypass coded bits of coefficient n, and therefore, the bypass coded bits of different coefficients are interleaved with the conventionally coded bits, such as Figure 6 as shown in .

[0082] like Figure 6 As shown in , the coefficient coding design in JVET-J0014 reduces the throughput compared to the design in HEVC or VTM-1, as follows:

[0083] 1. Increased regular coded bits. Due to context selection and interval subdivision calculations, regular coded bits are processed slower than bypassed coded bits. When using coefficient coding in JVET-J0014, the number of regular coded bits in a coefficient group (CG) is up to 80, while it is 25 in VTM-1 (16 in SIG, 8 in gt1, and 1 in gt2).

[0084] 2. Bypass coded bits are not grouped. Grouping bypass bits into longer chains increases the number of bits processed per cycle, thereby reducing the number of cycles required to process a single bypass bit. However, when using coefficient encoding in JVET-J0014, the bypass coded bits in CG are not grouped; instead, they are interleaved with the regular coded bits of each coefficient.

[0085] This application is directed to a decision scheme for a scalar quantizer to achieve nearly the same level of throughput as the coefficient coding designs in HEVC and VTM-1, while maintaining most of the gains provided by dependent scalar quantization.

[0086] In the JVET-K0319 document (see “CE7-Related: TCQ with High Throughput Coefficient Coding”, document JVET-K0319, 11th JVET Meeting: Ljubljana, SI, July 10-18, 2018, hereinafter referred to as “JVET-K0319”), the parity-based state transition proposed in JVET-J0014 is replaced by the SIG-based state transition, e.g. Figure 7 As shown in Table 2, other related scalar quantization designs in JVET-J0014 remain unchanged. By doing so, the scalar quantizer used to quantize the current transform coefficient is determined by the SIG of the quantized coefficient that precedes the current transform coefficient in the scanning order.

[0087] Table 2

[0088]

[0089] The coefficient coding proposed in JVET-K0319 is based on the HEVC and VTM-1 coefficient coding. The difference is that there are two sets of context models for each of SIG and gt1, and the entropy encoder selects the context set for a particular SIG or gt1 based on the quantizer used for the associated coefficient. Therefore, changing the scalar quantizer of the dependent scalar quantization from parity-based to SIG-based can achieve a high throughput design similar to the HEVC and VTM-1 coefficient coding. The proposed order of coefficient bits in CG is Figure 8 That is, using the coefficient coding proposed in JVET-K0319, the conventional coded bits for each CG are up to 25, which remains the same as the HEVC and VTM-1 coefficient coding; all bypass coded bits in the CG are grouped together.

[0090] The dependent quantization method in JVET-J0014 was tested in the test 7.2.1 software of CE7, and the simulation results showed a 4.99% reduction in AI (all Intra), 3.40% RA (random access), and 2.70% LDB (low latency B) BD rate compared to the VTM-1.0 anchor point. However, the simulation results in JVET-K0319 showed a 3.98% reduction in AI, 2.58% RA, and 1.80% LDB BD rate compared to the VTM-1.0 anchor point. That is, compared to the proposal in JVET-J0014 (based on the complete absLevel switching of the quantized coefficients), the scalar quantizer used to quantize the transform coefficients as in JVET-K0319 (based only on SIG switching) may reduce the coding efficiency.

[0091] The present application proposes some alternative decision schemes for scalar quantizers for dependency scalar quantization to achieve a good compromise between high throughput and coding efficiency. Instead of using absolute level parity or SIG values, state transitions and context model selection based on conventional coded bits are proposed. In the following, several embodiments of deciding a scalar quantizer for dependency scalar quantization are described.

[0092] For the dependent scalar quantization proposed in JVET-J0014, it is necessary to complete the reconstruction of the absolute level (absLevel) of the quantized coefficient before moving to the next scanning position in order to know the parity check used to determine the quantizer for the next coefficient. Therefore, the bypass coded bits in CG are not grouped, and each of their coefficients is interleaved with conventional coded bits. In addition, in contrast to HEVC and VTM-1, the maximum number of conventional coded bits for each transform coefficient level has increased (in the method proposed in JVET-J0014, there can be up to 5 conventional coded bits per transform coefficient level), as shown in the syntax table below. Changes relative to VTM-1 are in italics. The entropy encoder selects the context set for a specific SIG or gt1 according to the "state", which depends on the information of the change coefficient level and is used to determine the quantizer. The encoding order of the bits is illustrated in the following syntax, where the function getSigCtxId(XC, YC, state) is used to derive the context of the syntax sig_coeff_flag based on the current coefficient scanning position (xC, yC) and the state, decodeSigCoeffFlag(sigCtxId) is used to decode the syntax sig_coeff_flag with the associated context sigCtxId, getGreater1CtxId(xC, yC, state) is used to derive the context of the syntax abs_level_gt1_flag based on the current coefficient scanning position (xC, yC) and the state, and decodeAbsLevelGt1Flag(greater1CtxId) is used to decode the syntax abs_level_gt1_flag with the associated context greater1CtxId.

[0093]

[0094]

[0095]

[0096] As mentioned before, there are potential issues regarding high throughput hardware implementation due to these syntax modifications. In our proposal, alternative approaches are proposed to achieve almost the same throughput level as HEVC while supporting dependent scalar quantization. In the following, HEVC is taken as an example to illustrate the proposed modifications.

[0097] In one embodiment, the maximum number of normally coded bits per transform coefficient level remains at 3 instead of 5 (SIG, gt1, and gt2 are normally coded). For each CG, the normally coded bits and the bypass coded bits are separated in coding order; first, all the normally coded bits for the CG are sent, then the bypass coded bits are sent. The recommended order of coefficient bits in the CG is Fig. 9 The bits for the CG are encoded in multiple passes over the scan positions in the CG:

[0098] Pass 1: Coding of the significance (SIG, sig_coeff_flag), greater than 1 flag (gt1, abs_level_gt1_flag), and greater than 2 flag (gt2, abs_level_gt2_flag) in coding order. The greater than 1 flag is only present if sig_coeff_flag is equal to 1. Coding of the greater than 2 flag (abs_level_gt2_flag) is only performed for scan positions with abs_level_gt1_flag equal to 1. If gt1 and gt2 are not present in the bitstream, their values ​​are inferred to be 0. The SIG, gt1, and gt2 flags are coded in normal mode, and the context modeling selection for the SIG depends on which state is selected for the associated coefficient.

[0099] Pass 2: Coding of the syntax element abs_level_remaining for all scan positions with abs_level_gt2_flag equal to 1. Non-binary syntax elements are binarized and the resulting bits are encoded in the bypass mode of the arithmetic coding engine;

[0100] • Pass 3: Coding of symbols (coeff_sign_flag) for all scan positions with sig_coeff_flag equal to 1. Symbols are encoded in bypass mode.

[0101] The above embodiments show the proposed modifications compared to EVC. These modifications can also be based on other solutions. For example, if JVET-J0014 is used as a basis, the context modeling of SIG and gt1 will depend on the choice of quantizer, and other greater than x flags (gtx, x=3 and 4) can be encoded in the 1st or 2nd pass. If there are other conventionally encoded bits, such as gt5, gt6, gt7 flags, they can be encoded in the 1st or 2nd pass. In addition, the flag in the 3rd pass (coeff_sign_flag) can also be encoded in the regular mode.

[0102] At the March 2019 meeting, JVET adopted a new residual coding process for transform skipped residual blocks. When transform skip (TS) is enabled, transform is skipped for prediction residuals. The residual level of the coefficient group (CG) is coded three times over the scan position as follows:

[0103] Pass 1: Signal the following signs

[0104] o sig_coeff_flag

[0105] o coeff_sign_flag

[0106] o Greater than 1 flag (abs_level_gtx_flag[0])

[0107] oParity check (par_level_flag) flag

[0108] Second pass: Signal the sign

[0109] o Greater than 3 flag (abs_level_gtx_flag[1])

[0110] o greater than 5 flag (abs_level_gtx_flag[2])

[0111] o greater than 7 flag (abs_level_gtx_flag[3])

[0112] o greater than 9 flag (abs_level_gtx_flag[4])

[0113] Pass 3: Bypass encoding of the remaining absolute level (abs_remainder) using Golomb-Rice encoding.

[0114] The above-mentioned embodiments can also be applied to the newly adopted TS residual coding, where the position of the greater than 3 flag is moved to the first pass, for example as shown below:

[0115] Pass 1: Signal the following signs

[0116] o sig_coeff_flag

[0117] o coeff_sign_flag

[0118] o Greater than 1 flag (abs_level_gtx_flag[0])

[0119] oParity check (par_level_flag) flag

[0120] o Greater than 3 flag (abs_level_gtx_flag[1])

[0121] Second pass: Signal the sign

[0122] o greater than 5 flag (abs_level_gtx_flag[2])

[0123] o greater than 7 flag (abs_level_gtx_flag[3])

[0124] o greater than 9 flag (abs_level_gtx_flag[4])

[0125] Pass 3: Bypass encoding of the remaining absolute level (abs_remainder) using Golomb-Rice encoding.

[0126] Example 1 - Scalar quantizer decision scheme based on function SUM(SIG, gt1, gt2)

[0127] To address the issue of high throughput hardware implementation, a full reconstruction of the absolute level (absLevel) of the quantized coefficients will not be performed to decide the state, and the switching between the two scalar quantizers will not depend on the parity check of the absolute level of the complete transform coefficients. As mentioned earlier, a scalar quantizer determined only by SIG as in JVET-K0319 may reduce coding efficiency. In one embodiment, we propose to decide the scalar quantizer based on the function SUM(SIG, gt1, gt2), which will jointly consider the SIG, gt1 and gt2 values ​​of the current transform coefficient.

[0128] Table 3

[0129] Tag value m gt2 gt1 SIG Possible absolute level of change 0 0 0 0 0 1 0 0 1 1 2 0 1 1 2 3 1 1 1 ≥3

[0130] The possible combinations of SIG, gt1 and gt2 values ​​for the transform coefficients are shown in Table 3. There is a one-to-one mapping from these four different combinations to the four possible marker level values, where m represents the marker value for these four possible cases. The function that derives the marker value m from the SIG, gt1 and gt2 values ​​can be written as:

[0131] m=SUM(SIG,gt1,gt2)=SIG+gt1+gt2

[0132] like Fig.10 As shown in , the switch between these two scalar quantizers is uniquely determined by the parity check of the marker value m of the transform coefficient level that precedes the current transform coefficient in the coding / reconstruction order. At the beginning of inverse quantization for a transform block, the state is set equal to 0. After reconstructing the normal coded bits of the current transform coefficient, as shown in Fig.10 and update the state as shown in Table 4. Note that the next state depends only on the current state and the parity of the signature value m of the current transform coefficient level (m&1). The state update can be written as:

[0133] state=stateTransTable[state][m&1]

[0134] Where stateTransTable represents Fig.10 The table shown in Table 4 and the operator & specifies the bitwise "and" operator in two's complement arithmetic.

[0135] Table 4

[0136]

[0137]

[0138]

[0139] After transformation and quantization, the magnitude of most transform coefficients is usually at very low values. When the absolute level of the transform coefficient is less than 3, as shown in Tables 3 and 4, our proposed method can obtain almost the same results as JVET-J0014. At the same time, our proposed method does not require full reconstruction of the absolute level (absLevel) of the quantized coefficients, thus solving the problem of high-throughput hardware implementation.

[0140] The syntax table above gives details about the coding order and the presence of bits and the reconstruction of the transform coefficient levels from the transmitted data. For ease of illustration, the different passes in the scan position are annotated in the syntax table. Changes relative to HEVC and VTM-1 are shown in italics. The entropy encoder selects the context set for a particular SIG based on the "state", which depends on the information of the transform coefficient levels and is used to determine the quantizer.

[0141] Example 2 - Scalar quantizer decision scheme based on function XOR(SIG, gt1, gt2)

[0142] In another embodiment, the SIG, gt1 and gt2 values ​​of the current transform coefficient are considered, and a scalar quantizer is selected based on the function XOR(SIG, gt1, gt2). The function that derives the XOR value x from the SIG, gt1 and gt2 values ​​can be written as:

[0143] x=XOR(SIG,gt1,gt2)=SIG^gt1^gt2

[0144] The corresponding XOR values ​​x for the possible combinations of SIG, gt1 and gt2 values ​​of the transform coefficients are shown in Table 5.

[0145] Table 5

[0146] gt2 gt1 SIG XOR value x 0 0 0 0 0 0 1 1 0 1 1 0 1 1 1 1

[0147] Compared with the first embodiment, the switch between the two scalar quantizers is uniquely determined by the XOR value x of the SIG, gt1 and gt2 flags. The state update can be written as:

[0148] state=stateTransTable[state][x]

[0149] Among them, stateTransTable represents Fig.11 and the table shown in Table 6. The rest of the state machine remains similar to the previous method.

[0150] Table 6

[0151]

[0152] When the absolute level of the transform coefficient is less than 3, our proposed method can obtain almost the same results as JVET-J0014 as shown in Tables 5 and 6. At the same time, our proposed method does not require full reconstruction of the absolute level (absLevel) of the quantized coefficients, thus solving the problem of high-throughput hardware implementation.

[0153] Example 3 - Scalar quantizer decision scheme based on one of the conventional coded bits

[0154] According to the above embodiment, all the conventional coded bits of the current transform coefficient (SIG, gt1 and gt2) will be considered to decide the scalar quantizer. In another embodiment, the switch between the two scalar quantizers can be based on one of the conventional coded bits, for example, the gt1 flag. In this embodiment, the previous state transition can be replaced by a state transition based on gt1, such as Fig.12 As shown in , while other designs remain unchanged. By doing so, the scalar quantizer used to quantize the current transform coefficient is determined by the gt1 flag of the quantized coefficient that precedes the current transform coefficient in the scan order. The state update can be written as:

[0155] state=stateTransTable[state][gt1]

[0156] Among them, stateTransTable represents Fig.12 and the table shown in Table 7.

[0157] In this embodiment, the bits for the CG are encoded in 3 scan passes at scan positions in the CG: a first pass for sig, gt1, and gt2, a second pass for the remaining absolute levels, and a third pass for the signature information. In a variation, the bits for the CG are encoded in 4 scan passes at scan positions in the CG: a first pass for sig and gt1, a second pass for gt2, a third pass for the remaining absolute levels, and a fourth pass for the signature information. This variation can further reduce bit-to-bit dependencies compared to the 3 scan passes proposed in the previous embodiment.

[0158] Table 7

[0159]

[0160] Alternatively, the scalar quantizer used to quantize the current transform coefficient is determined by the gt2 flag of the quantized coefficient that precedes the current transform coefficient in the scan order. More generally, the scalar quantizer used to quantize the current transform coefficient is determined by one conventionally encoded bit (e.g., the gtx flag) of the quantized coefficient that precedes the current transform coefficient in the scan order.

[0161] In the above examples, we have shown several embodiments based on HEVC, which use 3 conventional coding bits (SIG, gt1, gt2) for coefficients. When the conventional coding bits of the coefficients are different from the conventional coding bits in HEVC, the proposed embodiments can be performed by using different numbers (greater or less than 3) of conventional coding bits for each transform coefficient.

[0162] In the above, the sum and XOR functions are considered, the proposed embodiment can also be performed with a state update derivation (1 / 0) function where each transform coefficient level is different from the conventional coded bit.

[0163] In the above, the description is mainly about dequantization. It should be noted that the quantization is adjusted accordingly. The scalar quantizer (Q0 or Q1) used is not explicitly signaled in the bitstream. For example, the quantization module on the encoder side will select which quantizer to use for the current transform coefficient based on the state. If the state of the current transform coefficient is equal to 0 or 1, the scalar quantizer Q0 is used. Otherwise (state is equal to 2 or 3), the scalar quantizer Q1 is used. The state is used as Figure 10-12The method described in is uniquely determined by the information of the transform coefficient levels, and the decoder will use the same quantizer in order to decode the bitstream accordingly.

[0164] Fig.13 A method (1300) for encoding a current coding unit according to an embodiment is shown. At step 1305, the initial state is set to zero. In order to encode the coding unit, the coefficients in the coding unit are scanned. The scanning pass for the coding unit sequentially processes each CG of the coding unit according to the scanning mode (diagonal, horizontal, vertical), and scans the coefficients within each CG according to the scanning order considered. The scanning starts at the CG with the last significant coefficient in the coding unit (1315), and all coefficients are processed until the first CG with a DC coefficient.

[0165] If the CG does not contain the last significant coefficient or DC coefficient (1320), a flag (coded_sub_block_flag) indicating whether the CG contains any non-zero coefficients is encoded (1325). For a CG containing the last non-zero horizontal or DC coefficient, coded_sub_block_flag is inferred to be equal to 1 and does not appear in the bitstream.

[0166] If coded_sub_block_flag is true (1330), 3 scan passes are applied to the CG. In the first pass (1335-1360), the SIG flag (sig_coeff_flag) is encoded for the coefficient (1335). To encode the SIG flag, the state is used to determine the context mode index, for example, sigCtxId=getSigCtxId(state). If the SIG flag is true (1340), the gt1 flag (abs_level_gt1_flag) is encoded (1345). If the gt1 flag is true (1350), the gt2 flag (abs_level_gt2_flag) is encoded (1355). Based on one or more of the SIG, gt1, and gt2 flags, for example, using Figure 10-12 The method described updates the state (1360).

[0167] In the second pass (1365, 1370), the encoder checks if the gt2 flag is true (1365). If so, the remaining absolute level (abs_level_remaining) is encoded (1370). In the third pass (1375, 1380), the encoder checks if the SIG flag is true (1375). If so, the sign flag (coeff_sign_flag) is encoded (1380). In step 1385, the encoder checks if there are more CGs to process. If so, it proceeds to the next CG to be processed (1390).

[0168] Fig.14 A method (1400) for decoding a current coding unit according to an embodiment is shown. At step 1405, the initial state is set to zero. Similar to the encoder side, in order to decode the coding unit, the coefficient positions in the coding unit are scanned. The scanning starts with the CG with the last significant coefficient in the coding unit (1415), and then all coefficients are processed until the first CG with a DC coefficient.

[0169] If the CG does not contain the last significant coefficient or DC coefficient (1420), a flag (coded_sub_block_flag) indicating whether the CG contains any non-zero coefficients is decoded (1425). For a CG containing the last non-zero horizontal or DC coefficient, coded_sub_block_flag is inferred to be equal to 1.

[0170] If coded_sub_block_flag is true (1430), 3 scan passes are applied to the CG. In the first pass (1435-1460), the SIG flag (sig_coeff_flag) is decoded for the coefficient (1435). To decode the SIG flag, the state is used to determine the context mode index, for example, sigCtxId = getSigCtxId(state). If the SIG flag is true (1440), the gt1 flag (abs_level_gt1_flag) is decoded (1445). If the gt1 flag is true (1450), the gt2 flag (abs_level_gt2_flag) is decoded (1455). Based on one or more of the SIG, gt1, and gt2 flags, for example, using Figure 10-12 The method described in updates the state (1460).

[0171] In the second scan pass (1470, 1475), the decoder checks whether the gt2 flag is true (1470). If true, the remaining absolute level (abs_level_remaining) is decoded (1475). In the third scan pass (1480, 1485), the decoder checks whether the SIG flag is true (1480). If true, the sign flag (coeff_sign_flag) is decoded (1485). In step 1487, the decoder calculates the transform coefficients based on the available SIG, gt1, gt2, sign flag and remaining absolute value.

[0172] The decoder checks if there are more CGs to process (1490). If so, it proceeds to the next CG to be processed (1495). If all coefficients are entropy decoded, the transform coefficients are dequantized using dependent scalar quantization (1497). The scalar quantizer (Q0 or Q1) used for the transform coefficients is determined by the state, which is the state of the transform coefficients using Figure 10-12 , derived using information on the decoded transform coefficient levels.

[0173] Various methods are described herein, and each method includes one or more steps or actions for realizing the described method.Unless the correct operation of the method requires specific steps or action sequence, otherwise the order and / or use of specific steps and / or actions can be modified or combined.In addition, terms such as "first", "second" etc. can be used to modify elements, components, steps, operations, etc., such as, "first decoding" and "second decoding" in various embodiments.Unless specifically required, the use of these terms does not mean that the modified operation is sorted.Therefore, in this example, the first decoding does not need to be performed before the second decoding, and can, for example, occur before, during, or in a time period overlapping with the second decoding.

[0174] Various methods and other aspects described in this application can be used to modify modules, such as Figure 2 and Figure 3 The entropy encoding and decoding modules (245, 330) of the video encoder 200 and decoder 300 shown in FIG. 24 are shown in FIG. 24. In addition, the current aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations, as well as extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0175] Various numerical values ​​are used in this application. The specific values ​​are for example purposes, and the described aspects are not limited to these specific values.

[0176] According to one embodiment, a video decoding method is provided, comprising: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; entropy decoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy decoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a gt1 flag and a gt2 flag, the gt1 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and the gt2 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 2; and reconstructing the block in response to the decoded transform coefficients.

[0177] According to one embodiment, a method for video encoding is provided, comprising: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy encoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a gt1 flag and a gt2 flag, the gt1 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and the gt2 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 2.

[0178] According to another embodiment, an apparatus for video decoding is provided, the apparatus comprising one or more processors, wherein the one or more processors are configured to: access a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; access a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; entropy decode the first and second sets of parameters in a first pass for the block, wherein the first pass is performed before other passes for entropy decoding the transform coefficients of the block, and wherein each of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a gt1 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1 and a gt2 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 2; and reconstruct the block in response to the decoded transform coefficients. The apparatus also comprises one or more memories coupled to the one or more processors.

[0179] According to another embodiment, a device for video encoding is provided, the device comprising one or more processors, wherein the one or more processors are configured to: access a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; access a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a regular mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy encode the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a gt1 flag and a gt2 flag, wherein the gt1 flag indicates whether an absolute value of the corresponding transform coefficient is greater than 1, and the gt2 flag indicates whether an absolute value of the corresponding transform coefficient is greater than 2.

[0180] According to another embodiment, a video decoding device is provided, comprising: a component for accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in decoding order; a component for accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on decoding of the first transform coefficient; a component for entropy decoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy decoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a gt1 flag and a gt2 flag, the gt1 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1, and the gt2 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 2; and a component for reconstructing the block in response to the decoded transform coefficients.

[0181] According to another embodiment, a video encoding device is provided, comprising: a component for accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; a component for accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy encoded in a normal mode, wherein context modeling of at least one parameter in the second set of parameters for the second transform coefficient depends on the first transform coefficient; and a component for entropy encoding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy encoding the transform coefficients of the block, and wherein each set of the first and second sets of parameters for the first and second transform coefficients includes at least one of a gt1 flag and a gt2 flag, wherein the gt1 flag indicates whether the absolute value of the corresponding transform coefficient is greater than 1, and the gt2 flag indicates whether the absolute value of the corresponding transform coefficient is greater than 2.

[0182] According to another embodiment, a signal comprising an encoded video is formed by performing the following steps: accessing a first set of parameters associated with a first transform coefficient in a block of a picture, the first transform coefficient preceding a second transform coefficient in the block of the picture in a coding order; accessing a second set of parameters associated with the second transform coefficient, the first and second sets of parameters being entropy coded in a conventional mode, wherein context modeling of at least one parameter of the second set of parameters for the second transform coefficient depends on the first transform coefficient; and entropy coding the first and second sets of parameters in a first scan pass for the block, wherein the first scan pass is performed before other scan passes for entropy coding the transform coefficients of the block, and wherein each of the first and second sets of parameters for the first and second transform coefficients comprises at least one of a gt1 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 1 and a gt2 flag indicating whether an absolute value of the corresponding transform coefficient is greater than 2.

[0183] According to an embodiment, the context modeling of at least one parameter of the second set of parameters for the second transform coefficient depends on the decoding of the first set of parameters for the first transform coefficient and is independent of (1) parameters used to represent the first transform coefficient, and (2) parameters entropy encoded in bypass mode.

[0184] According to an embodiment, a SIG flag is also encoded or decoded in the first scanning pass, the SIG flag indicating whether the corresponding transform coefficient is zero.

[0185] According to an embodiment, the gt1 flag is encoded or decoded in a first scanning pass, and the gt2 flag is encoded or decoded in a second scanning pass.

[0186] According to an embodiment, a dequantizer for dequantizing the second transform coefficient is selected between two or more quantizers based on the first transform coefficient.

[0187] According to an embodiment, the dequantizer is selected based on the first set of parameters for the first transform coefficients.

[0188] According to an embodiment, context modeling of at least one parameter of said second set of parameters for said second transform coefficient depends on decoding of said first and second transform coefficients.

[0189] According to an embodiment, the dequantizer is selected based on the sum of the SIG, gt1 and gt2 flags.

[0190] According to an embodiment, the dequantizer is selected based on an XOR function of the SIG, gt1 and gt2 flags.

[0191] According to an embodiment, the dequantizer is selected based on the gt1 flag, the gt2 flag or the gtx flag, the gtx flag indicating whether the absolute value of the corresponding transform coefficient is greater than x.

[0192] According to an embodiment, (1) parameters for representing the transform coefficients in the block, and (2) parameters encoded in the bypass mode are entropy encoded or decoded in one or more scanning passes after (1) parameters for representing the transform coefficients in the block and (2) parameters encoded in the normal mode.

[0193] According to an embodiment, the context modeling of the SIG, gt1, gt2 flag or gtx flag is based on the state used in the quantizer or quantizer selection.

[0194] An embodiment provides a computer program including instructions, and when the instructions are executed by one or more processors, the instructions cause the one or more processors to perform the encoding method or decoding method according to any of the above embodiments. One or more of the present embodiments also provide a computer-readable storage medium on which instructions for encoding or decoding video data according to the above method are stored. One or more embodiments also provide a computer-readable storage medium on which a bit stream generated according to the above method is stored. One or more embodiments also provide a method and apparatus for sending or receiving a bit stream generated according to the above method.

[0195] Various implementations involve decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to refer broadly to a broader decoding process will be apparent and is believed to be fully understood by those skilled in the art.

[0196] Various implementations involve encoding.In a similar manner to the above discussion on "decoding", "encoding" as used in this application may encompass all or part of the processes performed on an input video sequence, for example, to produce an encoded bitstream.

[0197] Note that the syntax elements used herein (eg, sig_coeff_flag, abs_level_gt1_flag) are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0198] The implementations and aspects described herein can be implemented, for example, in methods or processes, devices, software programs, data streams, or signals. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features discussed can also be implemented in other forms (e.g., devices or programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (e.g., a processor), which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0199] References to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof mean that the particular features, structures, characteristics, etc. described in conjunction with the embodiment are included in at least one embodiment. Therefore, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same embodiment.

[0200] Additionally, the present application may refer to “determining” various information. Determining information may include, for example, one or more of determining information, estimating information, calculating information, predicting information, or retrieving information from a memory.

[0201] Furthermore, the present application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0202] Additionally, the present application may involve "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is typically involved in one way or another.

[0203] It should be understood that the use of any of the following “ / ”, “and / or”, and “at least one of…”, for example, in the context of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the context of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). As will be apparent to one of ordinary skill in this and related arts, this can be extended to as many items as listed.

[0204] As will be apparent to one of ordinary skill in the art, implementations may generate various signals formatted to carry information, which may be stored or transmitted, for example. The information may include, for example, instructions for executing the method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, the signal may be sent via a variety of wired or wireless links. The signal may be stored on a processor readable medium.

Claims

1. A method comprising: In a first scan pass for entropy decoding transform coefficients of blocks in a picture, entropy decoding a first set of syntax flags indicating whether absolute values ​​of corresponding transform coefficients are greater than 1 and a second set of syntax flags indicating whether absolute values ​​of the corresponding transform coefficients are greater than 3; entropy decoding a first type of syntax element indicating a remaining level of a transform coefficient in a second scanning pass for entropy decoding the transform coefficients of the block; entropy decoding a syntax element of a second type indicating a sign of a transform coefficient in a third scanning pass for entropy decoding the transform coefficients of the block; Obtaining the transform coefficients of the block based on the first group of syntax flags, the second group of syntax flags, the first type of syntax elements, and the second type of syntax elements; as well as The block is reconstructed in response to the obtained transform coefficients of the block.

2. The method according to claim 1, wherein: All syntax elements in the first scan pass are entropy decoded in a normal mode.

3. The method according to claim 1, wherein: All syntax elements in the second scan pass are entropy decoded in bypass mode.

4. The method according to claim 1, wherein: All syntax elements in the third scan pass are entropy decoded in bypass mode.

5. The method according to claim 1, wherein: One or more syntax flags are entropy decoded in the first scanning pass or the second scanning pass, the one or more syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than x, where x is an integer greater than 3.

6. An apparatus comprising one or more processors, wherein the one or more processors are configured to: In a first scan pass for entropy decoding transform coefficients of blocks in a picture, entropy decoding a first set of syntax flags indicating whether absolute values ​​of corresponding transform coefficients are greater than 1 and a second set of syntax flags indicating whether absolute values ​​of the corresponding transform coefficients are greater than 3; entropy decoding a first type of syntax element indicating a remaining level of a transform coefficient in a second scanning pass for entropy decoding the transform coefficients of the block; entropy decoding a syntax element of a second type indicating a sign of a transform coefficient in a third scanning pass for entropy decoding the transform coefficients of the block; Obtaining the transform coefficients of the block based on the first group of syntax flags, the second group of syntax flags, the first type of syntax elements, and the second type of syntax elements; as well as The block is reconstructed in response to the obtained transform coefficients of the block.

7. The device according to claim 6, wherein: All syntax elements in the first scan pass are entropy decoded in a normal mode.

8. The device according to claim 6, wherein: All syntax elements in the second scan pass are entropy decoded in bypass mode.

9. The device according to claim 6, wherein: All syntax elements in the third scan pass are entropy decoded in bypass mode.

10. The device according to claim 6, wherein: One or more syntax flags are entropy decoded in the first scanning pass or the second scanning pass, the one or more syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than x, where x is an integer greater than 3.

11. A method comprising: Get the transform coefficients of the blocks in the picture; entropy encode a first set of syntax flags indicating whether absolute values ​​of corresponding transform coefficients are greater than 1 and a second set of syntax flags indicating whether absolute values ​​of corresponding transform coefficients are greater than 3 in a first scan pass for entropy encoding the transform coefficients of the block in the picture; entropy encoding a first type of syntax element indicating a remaining level of a transform coefficient in a second scanning pass for entropy encoding the transform coefficients of the block; as well as In a third scanning pass for entropy encoding the transform coefficients of the block, a second type of syntax element indicating a sign of the transform coefficient is entropy encoded.

12. The method according to claim 11, wherein: All syntax elements in the first scan pass are entropy encoded in a normal mode.

13. The method according to claim 11, wherein: All syntax elements in the second scan pass are entropy encoded in bypass mode.

14. The method according to claim 11, wherein: All syntax elements in the third scan pass are entropy encoded in bypass mode.

15. The method according to claim 11, wherein: One or more syntax flags are entropy encoded in the first scanning pass or the second scanning pass, the one or more syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than x, where x is an integer greater than 3.

16. An apparatus comprising at least one memory and one or more processors, wherein: The one or more processors are configured to: Get the transform coefficients of the blocks in the picture; In a first scan pass for entropy encoding transform coefficients of blocks in a picture, entropy encoding a first set of syntax flags indicating whether absolute values ​​of corresponding transform coefficients are greater than 1 and a second set of syntax flags indicating whether absolute values ​​of the corresponding transform coefficients are greater than 3; entropy encoding a first type of syntax element indicating a remaining level of a transform coefficient in a second scanning pass for entropy encoding the transform coefficients of the block; as well as In a third scanning pass for entropy encoding the transform coefficients of the block, a second type of syntax element indicating a sign of the transform coefficient is entropy encoded.

17. The device according to claim 16, wherein: All syntax elements in the first scan pass are entropy encoded in a normal mode.

18. The device according to claim 16, wherein: All syntax elements in the second scan pass are entropy encoded in bypass mode.

19. The device according to claim 16, wherein: All syntax elements in the third scan pass are entropy encoded in bypass mode.

20. The device according to claim 16, wherein: One or more syntax flags are entropy encoded in the first scanning pass or the second scanning pass, the one or more syntax flags indicating whether an absolute value of a corresponding transform coefficient is greater than x, where x is an integer greater than 3.

Images