Integration of Context Coded BIN (CCB) counting method

Abstract

In one embodiment, the CCB counting method is unified between the transform residual coding process and the transform skip (TS) residual coding process. In one example, in TS residual coding, the CCB counting excludes coeff_sign_flag so that the syntax used for CCB counting is unified for the two residual coding processes. Also, a separate maximum number of context coding bins may be specified and used only for coeff_sign_flag. In another example, in TS residual coding, the maximum number of CCB counts is reduced from TB_size*2 to TB_size*1.75 for the TB, or more generally, both transform residual coding and TS residual coding are set to the same value, and the maximum CCB count is unified for the two residual coding processes.

Description

[Technical field]

[0001] The present embodiment generally relates to integrating a context-coded bins (CCB) scheme for transform residual mode and transform skip residual mode. [Background technology]

[0002] To achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. In general, intra- or inter-prediction is used to exploit intra- or inter-frame image correlation, and then the difference between the original block and the predicted block, often denoted as prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to the entropy coding, quantization, transformation, and prediction. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6),” Document JVET-O2002, 15th Meeting: Gothenburg, SE, 3-12 Jul. 2019 Summary of the Invention

[0004] According to one embodiment, a method of video encoding is provided, comprising: obtaining a prediction residual for a block to be encoded in an image; counting a number of context coding bins used to code the prediction residual of the block, the prediction residual of the block to be encoded using one of a transform residual coding process and a transform skip residual coding process, the transform residual coding process and the transform skip residual coding process using the same limit to limit the number of context coding bins; and adjusting an entropy coding method of one or more syntax elements in one of the transform residual coding process and the transform skip residual coding process depending on the limit and the number of context coding bins.

[0005] According to another embodiment, a method of video decoding is provided, comprising: counting a number of context coding bins used to decode a prediction residual of a block to be decoded using one of a transform residual decoding process and a transform skip residual decoding process, the transform residual decoding process and the transform skip residual decoding process using a same limit to limit the number of context coding bins; adjusting an entropy decoding method of one or more syntax elements in one of the transform residual decoding process and the transform skip residual decoding process according to the limit and the number of context coding bins; and obtaining a prediction residual of a block to be decoded in an image.

[0006] According to another embodiment, an apparatus for video encoding is provided, comprising one or more processors configured to obtain a prediction residual for a block to be encoded in an image, count a number of context coding bins used to code the prediction residual of the block, the prediction residual of the block to be coded using one of a transform residual coding process and a transform skip residual coding process, the transform residual coding process and the transform skip residual coding process using a same limit to limit the number of context coding bins, and adjust an entropy coding method of one or more syntax elements in one of the transform residual coding process and the transform skip residual coding process depending on the limit and the number of context coding bins.

[0007] According to another embodiment, an apparatus for video decoding is provided, comprising one or more processors configured to: count a number of context coding bins used to decode a prediction residual of a block to be decoded using one of a transform residual decoding process and a transform skip residual decoding process, the transform residual decoding process and the transform skip residual decoding process using the same limit to limit the number of context coding bins; adjust an entropy decoding method of one or more syntax elements in one of the transform residual decoding process and the transform skip residual decoding process according to the limit and the number of context coding bins; and obtain a prediction residual of a block to be decoded in an image.

[0008] One or more embodiments provide a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above-described embodiments. One or more embodiments also provide a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to the above-described methods. One or more embodiments also provide a computer-readable storage medium having stored thereon a bitstream generated according to the above-described methods. One or more embodiments also provide methods and apparatus for transmitting or receiving a bitstream generated according to the above-described methods. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates a block diagram of a system in which aspects of the present embodiments may be implemented. [Diagram 2] FIG. 2 shows a block diagram of an embodiment of a video encoder. [Diagram 3] FIG. 2 shows a block diagram of an embodiment of a video decoder. [Figure 4] FIG. 2 illustrates the concept of a coding tree unit and a coding tree for representing a compressed image. [Diagram 5] FIG. 2 illustrates a residual coding structure for a transform block. [Figure 6] FIG. 1 illustrates a residual coding structure for a transform skip block. [Figure 7] A diagram showing the residual coding process of a transform block in VTM6.0. [Figure 8] A diagram showing the residual coding process of transform skip blocks in VTM6.0. [Figure 9] FIG. 2 illustrates a residual coding process for a transform skip block according to one embodiment. [Figure 10] FIG. 13 illustrates a residual coding process for a transform skip block according to another embodiment. [Figure 11]FIG. 13 illustrates a residual coding process for a transform skip block according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] FIG. 1 illustrates a block diagram of an example of a system in which various aspects and embodiments may be implemented. System 100 may be embodied as a device including the various components described below and configured to implement one or more aspects described herein. Examples of such devices include various electronic devices, such as, but not limited to, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 100, alone or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more aspects described herein.

[0011] The system 100 includes at least one processor 110 configured to execute loaded instructions to implement various aspects described herein, for example. The processor 110 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 100 includes at least one memory 120 (e.g., volatile and / or non-volatile memory devices). The system 100 includes a storage device 140 that may include non-volatile and / or volatile memory including, but not limited to, EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drives, and / or optical disk drives. The storage device 140 may include, as non-limiting examples, internal storage, attached storage, and / or network-accessible storage.

[0012] The system 100 includes an encoder / decoder module 130 configured to process data to provide, for example, encoded or decoded video, and the encoder / decoder module 130 may include its own processor and memory. The encoder / decoder module 130 represents a module that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 130 may be implemented as a separate element of the system 100 or may be incorporated within the processor 110 as a combination of hardware and software, as is known to those skilled in the art.

[0013] Program code to be loaded into the processor 110 or the encoder / decoder 130 to perform various aspects described herein may be stored in the storage device 140 and then loaded onto the memory 120 for execution by the processor 110. According to various embodiments, one or more of the processor 110, the memory 120, the storage device 140, and the encoder / decoder module 130 may store one or more of various items during performance of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0014] In some embodiments, memory within the processor 110 and / or the encoder / decoder module 130 is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be the processor 110 or the encoder / decoder module 130) is used for one or more of these functions. The external memory can be the memory 120 and / or the storage device 140, such as dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, a high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0015] Input to the elements of system 100 may be provided through various input devices, shown in block 105. Such input devices include, but are not limited to, (i) an RF section for receiving RF signals transmitted over the air, for example by a broadcast station, (ii) a composite input, (iii) a USB input, and / or (iv) an HDMI® input.

[0016] In various embodiments, the input devices of block 105 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrow band of frequencies to select a signal frequency band, which in some embodiments may be referred to as a channel (for example), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner for performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and A / D converters. In various embodiments, the RF section includes an antenna.

[0017] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting the system 100 to other electronic devices via USB and / or HDMI connections. It should be appreciated that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in separate input processing ICs or within the processor 110, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, in separate interface ICs or within the processor 110, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, the processor 110 and the encoder / decoder 130 operating in combination with memory and storage elements, to process the data stream as desired for presentation on an output device.

[0018] The various elements of system 100 may be provided within an integrated housing in which the various elements are interconnected and data may be transmitted therebetween using a suitable connection arrangement 115, e.g., internal buses known in the art, including an Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0019] System 100 includes a communication interface 150 that enables communication with other devices over a communication channel 190. Communication interface 150 may include, but is not limited to, a transceiver configured to transmit and receive data over communication channel 190. Communication interface 150 may include, but is not limited to, a modem or a network card, and communication channel 190 may be implemented in a wired and / or wireless medium, for example.

[0020] In various embodiments, data is streamed to the system 100 using a Wi-Fi network, such as IEEE 802.11. The Wi-Fi signal in these embodiments is received via a communication channel 190 and communication interface 150 adapted for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, enabling streaming applications and other over-the-top communications. Other embodiments provide streaming data to the system 100 using a set-top box that routes data through an HDMI connection in the input block 105. Still other embodiments provide streaming data to the system 100 using an RF connection in the input block 105.

[0021] System 100 can provide output signals to various output devices, including display 165, speakers 175, and other peripheral devices 185. Other peripheral devices 185, in various example embodiments, include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, control signals are communicated between system 100 and display 165, speakers 175, or other peripheral devices 185 using signaling such as AV.Link, CEC, or other communication protocols that allow inter-device control with or without user intervention. Output devices may be communicatively coupled to system 100 via dedicated connections through respective interfaces 160, 170, and 180. Alternatively, output devices may be connected to system 100 using communication channel 190 via communication interface 150. Display 165 and speakers 175 may be integrated in a single unit with other components of system 100 in an electronic device such as a television. In various embodiments, the display interface 160 includes a display driver, such as a timing controller (T Con) chip.

[0022] Alternatively, the display 165 and speakers 175 may be separate from one or more of the other components, for example if the RF portion of input 105 is part of a separate set-top box. In various embodiments where the display 165 and speakers 175 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0023] 2 illustrates an example video encoder 200, such as a High Efficiency Video Coding (HEVC) encoder. FIG. 2 may also illustrate an encoder that incorporates improvements to the HEVC standard or that uses technology similar to HEVC, such as the Versatile Video Coding (VVC) encoder being developed by the Joint Video Exploration Team (JVET).

[0024] In this application, the terms "reconstructed" and "decoded" are used interchangeably, and the terms "image", "picture" and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, and the term "decoded" is used on the decoder side.

[0025] Before being encoded, a video sequence may undergo pre-encoding processing (201), for example to apply a color transformation to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or to perform a remapping of the input picture components to make the signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0026] To code a video sequence containing one or more images, the images are divided (202), for example, into one or more slices, each of which may contain one or more slice segments. In HEVC, slice segments are organized into coding units, prediction units, and transform units. The HEVC standard distinguishes between "blocks" and "units," where a "block" addresses a particular region (e.g., luma, Y) in a sample array, and a "unit" contains a collocated block of all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data (e.g., motion vectors) associated with the block.

[0027] When coding according to HEVC, an image is divided into square coding tree blocks (CTBs) of configurable size (typically 64x64, 128x128, or 256x256 pixels), and a set of coding tree blocks is grouped into a slice. A coding tree unit (CTU) contains the CTB of a coded color component. The CTB is the root of a quad-tree divided into coding blocks (CBs), which in turn form the root of a quad-tree divided into one or more predictive blocks (PBs) and transform blocks (TBs). Transform blocks (TBs) larger than 4x4 are divided into 4x4 sub-blocks of quantized coefficients called coefficient groups (CGs). Corresponding to coding blocks, predictive blocks, and transform blocks, a coding unit (CU) contains a prediction unit (PU) and a set of tree-structured transform units (TUs), where a PU contains prediction information for all color components, and a TU contains a residual coding syntax structure for each color component. The sizes of CB, PB, and TB for the luma components apply to the corresponding CU, PU, ​​and TU. In this application, the term "block" can be used to refer to any of the following: CTU, CU, PU, ​​TU, CG, CB, PB, and TB. Additionally, the term "block" can also refer to macroblocks and partitions as specified in H.264 / AVC or other video coding standards, or more generally to arrays of data of various sizes.

[0028] In the encoder 200, pictures are coded by the encoder elements as described below. A picture to be coded is processed, for example, in units of CUs. Each coding unit is coded using either intra mode or inter mode. If the coding unit is coded in intra mode, intra prediction is performed (260). In inter mode, motion estimation (275) and motion compensation (270) are performed. The encoder decides (205) whether to use intra mode or inter mode to code the coding unit, and indicates the intra mode / inter mode decision by a prediction mode flag. Prediction residuals are calculated by subtracting (210) the prediction block from the original image block.

[0029] The prediction residual is transformed (225) and quantized (230). The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded (245) to output a bitstream. As a non-limiting example, context-based adaptive binary arithmetic coding (CABAC) can be used to code the syntax elements into the bitstream.

[0030] To code with CABAC, the values ​​of non-binary syntax elements are mapped to binary sequences, called bin strings, through a binarization process. For a bin, a context model is selected. A "context model" is a probability model for one or more bins, selected from a selection of available models depending on the statistics of recently coded symbols. The context model for each bin is identified by a context model index (also called a "context index"), with different context indices corresponding to different context models. A context model stores the probability that each bin is a "1" or a "0" and can be adaptive or static. A static model triggers the coding engine with equal probability for bins "0" and "1". In an adaptive coding engine, the context model is updated based on the actual coded value of the bin. The operating modes corresponding to the adaptive and static models are called normal and bypass modes, respectively. Based on the context, the binary arithmetic coding engine encodes or decodes the bin according to the corresponding probability model.

[0031] The encoder can also skip the transform and apply quantization directly to the untransformed residual signal, for example on a 4x4TU basis. The encoder can bypass both the transform and the quantization, i.e. the residual is directly coded without application of a transform or quantization process. In direct PCM coding, no prediction is applied and the samples of the coding unit are coded directly into the bitstream.

[0032] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual is combined (255) with the prediction block to reconstruct an image block. An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (280).

[0033] Figure 3 illustrates a block diagram of an exemplary video decoder 300, such as an HEVC decoder. In the decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs a decoding pass that is an inverse of the encoding pass as described in Figure 2, which performs video decoding as part of the encoding of the video data. Figure 3 may also illustrate a decoder that improves upon the HEVC standard or employs techniques similar to HEVC, such as a VVC decoder.

[0034] In particular, the input of the decoder includes a video bitstream, which may be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, image partition information, and other coding information. When using CABAC for entropy coding, a context model is initialized in the same manner as the encoder context model, and syntax elements are decoded from the bitstream based on the context model.

[0035] The image partitioning information indicates how the image is partitioned, e.g., the size of the CTUs, and how the CTUs are partitioned into CUs and, if applicable, into PUs. Thus, the decoder can partition the image, e.g., into CTUs and partition each CTU into CUs (335) according to the decoded image partitioning information. The transform coefficients are inverse quantized (340) and inverse transformed (350) to decode the prediction residual.

[0036] The decoded prediction residual is combined with the prediction block (355) to reconstruct an image block. The prediction block can be obtained from intra prediction (360) or motion compensated prediction (i.e., inter prediction) (375) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0037] The decoded image may further undergo post-decoding processing (385), such as an inverse color conversion (e.g., converting from YCbCr 4:2:0 to RGB 4:4:4), or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0038] In HEVC, the transform coefficients of a coding block are coded using non-overlapping coefficient groups (CGs or sub-blocks), where each CG contains the coefficients of a 4x4 block of the coding block. In VVC Draft 6, the choice of coefficient group size now depends only on the TB size, i.e., it is no longer dependent on the channel type. As a result, a variety of CGs (1x16, 2x8, 8x2, 2x4, 4x2, and 16x1) are available. The CGs in a coding block, and the transform coefficients within a CG, are coded according to a predefined scan order.

[0039] In order to limit the maximum number of context-coded bins (bins coded in normal mode) per pixel, the area of ​​the TB is used to derive the maximum number of context-coded bins allowed for a TB in VVC draft 6 (Non-Patent Document 1). Figure 7 shows the residual coding process of a transform block in VTM 6.0, and Figure 8 shows the residual coding process of a transform skip block in VTM 6.0. For a TB, the maximum number of context-coded bins (CCBs) is set to TB_zosize*1.75, as shown in step 710 of Figure 7, where TB_zosize indicates the number of samples in the TB excluding the zeroed coefficients. It is known that a large block partition usually results in less residual and the energy is concentrated in the low frequency coefficients of the transform domain. In VTM (VVC Test Model) 6.0, high frequency transform coefficients outside the 32x32 area are zeroed for transform blocks whose size is greater than 32. It should be noted that coded_sub_block_flag is not considered for the CCB count, as shown in step 720 of Figure 7.

[0040] In VTM6.0, transform skip mode can be used for luma blocks of size up to 32x32 (inclusive). When a CU is coded in transform skip mode, its prediction residual is quantized (no transform, i.e. transform skip) and coded using a transform skip residual coding process. Unlike HEVC, where residual coding is designed for transform coefficient level statistics and signal characteristics, two separate residual coding structures are used for transform residual (transformed residual, transform not skipped) and transform skip residual (untransformed residual, transform skipped), respectively. For example, the maximum number of CCBs is set to TB_size*2 in transform skip mode, as shown in step 810 of FIG. 8, where TB_size indicates the number of samples in a TB. Also, the coeff_sign_flag in transform skip residual mode is context coded and considered as CCB count, as shown in step 840 of FIG. 8. Residual coding of transform residuals is also called "transform coefficients coding," "transform residual coding," or "residual coding for transform coefficients." Residual coding of transform skip residuals is also called "Transform Skip (TS) residual coding."

[0041] Residual coding of transform coefficients

[0042] In VVC Draft 6, to code transform coefficients, if the significance of the CG is equal to 1 (720), the bins of the CG (sub-block) are coded in multiple passes at the scan positions of the CG: Pass 1: The significance (sig_coeff_flag), flags greater than 1 (gt1, abs_level_gtx_flag[0]), parity flags (PAR, par_level_flag), and flags greater than 3 (gt3, abs_level_gtx_flag[1]) are coded in coding order. Flags greater than 1 are present only if sig_coeff_flag is equal to 1. Coding of parity flags and flags greater than 3 (abs_level_gtx_flag[1]) is performed only for scan positions where abs_level_gtx_flag[0] is equal to 1. The values ​​of gt1, PAR, and gt3 are inferred to be 0 if not present in the bitstream. The SIG, PAR, gt1, and gt3 flags are coded in normal mode (740). MaxCcbs is decremented by 4 after these flags are coded for a coefficient, and a counter n is incremented by 1 to track the number of coefficients in a subblock for which the SIG, PAR, gt1, and gt3 flags are coded in normal mode (750), where n is reset as 0 for each subblock. Pass 1 stops when all coefficients in a subblock have been coded (755) or when the CCB count limit has been reached (730), so that coding of some scan positions may be skipped in pass 1. Pass 2: For scan positions coded in pass 1, the syntax element abs_remainder is coded (760) for scan positions where gt3 is equal to 1. Non-binary syntax elements are binarized using Golomb-Rice coding and the resulting bins are coded in bypass mode of the arithmetic coding engine. For scan positions not coded in pass 1, the syntax element dec_abs_level is binarized using Golomb-Rice coding and the resulting bins are coded in bypass mode (760). Pass 3: The signs (coeff_sign_flag) of all scan positions where sig_coeff_flag is equal to 1 are encoded. The signs (sign) are encoded in bypass mode (780).

[0043] As mentioned above, for transform coefficient coding, the variable remBinsPass1 (MaxCcbs) is initially set to the maximum number of context-coded bins (MCCB) and is decremented by one when a context-coded bin is signaled. While remBinsPass1 is greater than or equal to 4, the flags for the first coding pass include sig_coeff_flag, abs_level_gtx_flag[0] (gt1, flags greater than 1), par_level_flag, and abs_level_gtx_flag[1] (gt3, flags greater than 3), which are coded using the context-coded bins. If the number of context-coded bins is not greater than the MCCB for the first pass coding, the remainder of the level information indicated in the first pass to be further coded is coded in the abs_remainder syntax element using Golomb-Rice codes and bypass coding bins. If remBinsPass1 is less than 4, the remaining coefficients that were not coded in the first pass are directly coded in the second pass using the syntax element dec_abs_level with Golomb-Rice coding and bypass coding bins. remBinsPass1 is reset every TB. The transition from using context coded bins for sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] to using bypass coding bins for the remaining syntax elements occurs at most once per TB. For coefficient sub-blocks, if remBinsPass1 is less than 4, the entire coefficient sub-block is coded using bypass coding bins. After all the above level coding, the signs (coeff_sign_flag) of all scan positions with sig_coeff_flag equal to 1 are finally bypass coded. This process is performed for all sub-blocks in the transform block (790) before the process ends (799).

[0044] For example, in FIG. 5, after coding sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] in normal mode for scan positions C15,...,C3, the number of CCBs reaches the limit and coding in pass 1 stops. In pass 2, if pass 1 indicates that more level information is to be coded for the current scan position, abs_remainder is bypass coded for scan positions C15,...,C3. Scan positions C2, C1, and C0 are not coded in pass 1, and absolute coefficient levels for these positions are directly coded with the syntax element dec_abs_level using bypass mode. In pass 3, if the coefficient in the current scan position is not zero, the syntax element coeff_sign_flag is bypass coded.

[0045] Transform-skip residual coding

[0046] In transform skip mode, the statistical properties of the residual signal differ from those of the transform coefficients, and no energy compaction around low frequency components is observed. The residual coding is modified to take into account different signal properties of the (spatial) transform skip residual, including: -The coefficient scan is forward scan; -No signaling of final x / y position; - coded_sub_block_flag is coded for all sub-blocks except DC sub-block if all previous coded_sub_block_flag are equal to 0; -sig_coeff_flag context modeling with reduced templates; - abs_level_gtx_flag[0] and par_level_flag each use only one context model; - Additional flags greater than 5, 7, 9; - Derivation of modified Rice parameters for residual binarization; The sign flag coeff_sign_flag is context coded, and the context modeling of the sign flag is determined based on the left and upper adjacent coefficient values, and the sign flag is parsed after sig_coeff_flag to summarise all the context coded bins.

[0047] 6 and 8, if the CG importance is 1 (820), the syntax elements sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], and par_level_flag are interleaved coded for each residual sample in a first pass (830-860), followed by coding (880) of the abs_level_gtx_flag bitplane, abs_remainder in a second pass (870). This process is performed (890) for all sub-blocks in the transform block before the process ends (899). - Path 1 (830 to 860): sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], and par_level_flag. - Path 2 (870): abs_level_gtx_flag[1], abs_level_gtx_flag[2] (flags greater than 5), abs_level_gtx_flag[3] (flags greater than 7), abs_level_gtx_flag[4] (flags greater than 9). - Path 3 (880): abs_remainder.

[0048] In transform skip mode, when the CCB count exceeds the maximum number, these remaining coefficient flags are bypass coded with the same syntax structure, which is quite different from the case of the transform residual.

[0049] As mentioned above, the syntax used for CCB count, the maximum number of CCB count, and the syntax design after the CCB count exceeds the maximum number are different for residual coding of transform blocks and transform skip blocks. Therefore, some unification can be proposed here.

[0050] As mentioned above, to limit the maximum number of context coding bins (CCBs) per pixel, the area of ​​the TB is used to derive the maximum number of context coding bins for the TB in VVC Draft 6. Furthermore, in the March 2019 meeting, JVET adopted a new residual coding process for transform-skip residual blocks. This transform-skip (TS) residual coding process is significantly different from the transform residual coding process for transform blocks. The main differences between TS residual coding and transform residual coding with respect to CCBs are as follows: 1. In transform residual coding, the sign (coeff_sign_flag) is bypass coded and is not automatically included as a syntax used for CCB counting, whereas coeff_sign_flag is context coded in TS residual coding and is included as a syntax used for CCB counting; 2. In transform residual coding, the maximum number of CCBs is equal to TB_zosize*1.75 of the TB, while in TS residual coding, the maximum number of CCBs is equal to TB_size*2 of the TB.

[0051] In order to unify the CCB restrictions between the transform and TS residual coding processes, some unifications have been proposed, while preserving most of the gains provided by the two separate residual coding structures used for transform and TS blocks.

[0052] In one embodiment, in TS residual coding, coeff_sign_flag is excluded from the CCB count so that the syntax used for the CCB count is unified for the two residual coding processes. In addition, a separate maximum number of context coding bins is specified to be used only for coeff_sign_flag. In another embodiment, in TS residual coding, the maximum number of CCB counts is reduced from TB_size*2 to TB_size*1.75 of the TB. More generally, the maximum CCB counts for both transform residual coding and TS residual coding are set to the same value so that the maximum CCB counts are unified for the two residual coding processes. In the following, several methods of unifying the CCB limits between the transform residual coding process and the TS residual coding process are described in detail.

[0053] Consolidation of syntax used for CCB counting

[0054] In VVC draft 6, as shown in step 740 and step 750 of FIG. 7, coefficient flags such as sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] are included in the CCB count of the transform residual mode. On the other hand, as shown in step 830 to step 870 of FIG. 8, coefficient flags such as sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[j] (1≦j≦4) are included in the CCB count of the transform skip residual mode. The difference between these two modes is whether coeff_sign_flag is included in the CCB count. Table 1 shows the syntax structures of the two residual coding modes of VTM6.0. As shown in Table 1, the residual of a coding block is coded using non-overlapping sub-blocks (SubBlocks), each containing 16 coefficients. For a given sub-block, the coefficients are coded in a loop (SubBlock loop) until all coefficients in that sub-block can be reconstructed before processing the next sub-block. For a given coefficient, the coefficient-level syntax is coded in several passes. In each pass, all predefined syntax in the coefficient pass is coded before starting to process the next coefficient. The entropy coding / decoding of the residual comprises up to three passes and is used to code the following syntax elements: coded_sub_block_flag: Specifies the significance (zero / non-zero) of a sub-block of 16 coefficients sig_coeff_flag: Specifies the significance of the coefficient (zero / non-zero) abs_level_gtx_flag[j]: Specifies whether the absolute value of the coefficient level is greater than (j<<1)+1 par_level_flag: Specifies the parity of the coefficients abs_remainder: specifies the remaining value of the absolute value of the coefficient level (if the value is larger than the value coded in the previous pass) dec_abs_level: Specifies the absolute value of the coefficient level (if the coefficient was not coded in a previous pass) coeff_sign_flag: Specifies the sign of the significant coefficient (0: positive, 1: negative)

[0055] [Table 1]

[0056] 1 shows a detailed syntax table of transform skip residual coding in VVC Draft 6.

[0057] [Table 2-1]

[0058] [Table 2-2]

[0059] As mentioned before, coeff_sign_flag is context coded and included in the CCB count for TS residual mode, as shown in step 840 of Fig. 8. In one embodiment, it is proposed to exclude coeff_sign_flag from the CCB count for TS residual mode, as shown in Fig. 9 and Table 3, where the changes compared to VTM6.0 are underlined. The proposed syntax table for transform skip residual coding is shown in Table 4, where the changes related to VVC draft 6 are strikethrough (i.e., "MaxCcbs--" is removed after coeff_sign_flag is signaled (940), and the other steps in Fig. 9 are the same as those in Fig. 8). According to a variant of this embodiment, it is proposed to exclude abs_lever_gtx_flag[j] (j>1) from the CCB count in TS residual coding.

[0060] [Table 3]

[0061] [Table 4]

[0062] Consolidation of maximum number of CCB counts

[0063] In VVC draft 6, as shown (also in Figures 7 and 8), the number of context coding bins per coefficient for luma and chroma coefficients is limited to 1.75 bins / coefficient at the TB level, as shown in step 710 of Figure 7. TS residual coding, on the other hand, sets this limit to 2 bins / coefficient at the TB level, as shown in step 810 of Figure 8.

[0064] Another integration proposed here is to align the maximum allowed number of context coding bins per coefficient for transform and TS residual coding, for example by allocating 1.75 bins / coefficient at the TB level, as shown in step 1010 of FIG. 10 (other steps in FIG. 10 are the same as those in FIG. 8). For TS residual coding, it is desirable to reduce the worst case number of context coding bins per residual from 2 to 1.75. This change improves the overall CABAC throughput and makes the CABAC limits more consistent. Table 5 shows the maximum number of CCB allocations for the two residual coding modes proposed in this embodiment, with the changes compared to VTM6.0 underlined. The proposed syntax table for the maximum number CCB count allocation for transform skip residual coding is shown in Table 6, where the changes related to VVC draft 6 are underlined.

[0065] [Table 5]

[0066] [Table 6]

[0067] According to a variant of the second embodiment, the maximum allowed number of context coding bins per coefficient for transform residual coding and TS residual coding can be aligned by assigning another value x instead of 1.75 bins / coefficient at the TB level.

[0068] According to another variant of the second embodiment, the second embodiment can be combined with the first embodiment, which reduces the maximum number of CCBs for the residual syntaxes other than coeff_sign_flag to 1.75 bins.

[0069] Assignment of another number of context coding bins CCB_SIGN to coeff_sign_flag

[0070] According to a first embodiment, it is proposed that coeff_sign_flag is excluded from the CCB count for TS residual mode, which reduces the overall CABAC throughput since the worst-case number of context coding bins per residual increases beyond 2 for TS residual coding.

[0071] To remedy this shortcoming, one solution is to combine the previous embodiments described in Tables 3 and 5, which reduces the maximum number of CCBs for other residual syntaxes except for coeff_sign_flag. Another solution is to assign a different number of context coding bins CCB_SIGN to coeff_sign_flag. In this case, the CABAC throughput for coeff_sign_flag can be controlled separately.

[0072] Using the area of ​​the TB, the maximum number of context coding bins CCB_SIGN for coeff_sign_flag in the TB can be derived. For example, the maximum number of CCB_SIGN can be equal to TB_size*0.5, as shown in step 1110 of FIG. 11, where TB_size indicates the number of samples in the TB as shown in Table 7, and the changes to VTM6.0 are underlined. As shown in step 1140 of FIG. 11, a new CCB count MaxCcbs_sign is used for coeff_sign_flag instead of MaxCcbs (steps other than 1110 and 1140 of FIG. 11 are the same as those of FIG. 8). The transform skip residual code syntax table with additional CCB_SIGN is shown in Table 8, where the changes to VVC draft 6 are shown with strikethrough or underline.

[0073] According to a variant of the third embodiment, it can be applied together with the second embodiment. According to a variant of the third embodiment, CCB_SIGN can also be set and applied at the CG level.

[0074] [Table 7]

[0075] [Table 8]

[0076] It should be noted that the methods described in Figures 7-11 can be used at the encoder side or the decoder side. When used at the encoder side, the term "code" in the figures can be understood as "encoding", while when used at the decoder side, the term "code" in the figures can be understood as "decoding". In the case of Figures 9-11, where the residual coding process is shown, when used at the encoder side, the quantized prediction residual of the transform skip block is used as input, and the coded prediction residual is output. When Figures 9-11 are used to explain the residual coding process at the decoder side, the bins corresponding to the quantized prediction residual in the bitstream are decoded, and the quantized prediction residual is output to the transform skip block.

[0077] Various methods are described herein, each method including one or more steps or actions for achieving the method described herein. The order and / or use of certain steps and / or actions may be varied or combined, except where a particular order of steps or actions is necessary for proper operation of the method. Additionally, terms such as "first", "second", etc. may be used in various embodiments to vary elements, components, steps, operations, etc., e.g., "first decode" and "second decode". Use of such terms does not imply a varied order of operations, unless specifically required. Thus, in this example, the first decode need not be performed before the second decode, but may occur, for example, before, during, or during an overlapping period with the second decode.

[0078] Various methods and other aspects described herein can be used to modify modules, such as the entropy encoding module (245) of the video encoder 200 and the entropy decoding module (330) of the video decoder 300 as shown in Figures 2 and 3. Additionally, aspects of the present application are not limited to VVC or HEVC, but can be applied, for example, to other standards and recommendations, and extensions of such standards and recommendations. Unless otherwise specified or technically precluded, aspects described herein can be used individually or in combination.

[0079] Various numerical values ​​are used in this application, the particular values ​​are for illustrative purposes and the aspects being described are not limited to these particular values.

[0080] One embodiment provides a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above embodiments. One or more embodiments also provide a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to the above methods. One or more embodiments also provide a computer-readable storage medium having stored thereon a bitstream generated according to the above methods. One or more embodiments also provide methods and apparatus for transmitting or receiving a bitstream generated according to the above methods.

[0081] Various implementations include decoding. As used herein, "decoding" may include all or some of the processes performed on a received encoded sequence to generate a final output suitable for display, for example. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Whether the term "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will be clear based on the context of a particular description and will be well understood by one of ordinary skill in the art.

[0082] Various implementations include encoding. In a manner similar to the above description of "decoding," "encoding" as used herein may encompass all or part of the processes performed on an input video sequence to generate, for example, a coded bitstream.

[0083] It should be noted that syntax elements as used herein are descriptive terms and therefore do not preclude the use of other syntax element names.

[0084] The implementations and aspects described herein can be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. An implementation of a discussed feature, even if discussed only in the context of a single implementation form (e.g., discussed only as a method), can be implemented in other forms (e.g., an apparatus or a program). An apparatus can be implemented, for example, in appropriate hardware, software, and firmware. A method can be implemented in an apparatus, for example, a processor, where a processor generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device, for example, a computer, a cell phone, a portable / personal digital assistant (PDA), and other devices that facilitate communication of information between end users.

[0085] References to "one embodiment" or "embodiment" or "one implementation" or "implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearance of the phrases "in one embodiment" or "in an embodiment" or "in one implementation" or "in an implementation," as well as any other variations, in various places throughout this application are not necessarily all referring to the same embodiment.

[0086] This application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0087] This application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0088] This application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically relates in some way to an operation, such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0089] For example, it should be recognized that the use of any of " / ", "and / or", and "at least one of" in the cases of "A / B", "A and / or B", and "at least one of A and B" is intended to include the selection of only the first enumerated option (A), or the selection of only the second enumerated option (B), or the selection of both options (A and B). As yet another example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such language is intended to include the selection of only the first enumerated option (A), or the selection of only the second enumerated option (B), or the selection of only the third enumerated option (C), or the selection of only the first and second enumerated options (A and B), or the selection of only the first and third enumerated options (A and C), or the selection of only the second and third enumerated options (B and C), or the selection of all three options (A and B and C). This may be extended to the number of items enumerated as would be clear to one of ordinary skill in the art and related arts.

[0090] As will be apparent to one skilled in the art, implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bit stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links as known. The signal can be stored in a processor-readable medium.

Claims

1. A method of video encoding, comprising: counting a number of context coding bins used to code one or more syntax elements associated with a prediction residual of a block, the limit being used to limit the number of context coding bins; counting a second number of context coding bins used to code a syntax element indicating a sign of the prediction residual of the block, wherein another limit is used to limit the second number of context coding bins; A method for providing the above.

2. The method of claim 1 , wherein the prediction residual for the block is coded using one of a transform residual coding process and a transform skip residual coding process.

3. A method for video decoding, comprising: counting a number of context coding bins used to decode one or more syntax elements associated with a prediction residual of a block, wherein a limit is used to limit the number of context coding bins; counting a second number of context coding bins used to decode a syntax element indicating a sign of the prediction residual of the block, wherein another limit is used to limit the second number of context coding bins; A method for providing the above.

4. The method of claim 3, wherein the prediction residual of the block is decoded using one of a transform residual coding process and a transform skip residual coding process.

5. 5. The method of claim 1, wherein the limit is set to 1.75*TB_size, where TB_size indicates the number of samples in the block.

6. The method of claim 1 , wherein a syntax element indicating a sign of a prediction residual is excluded from counting the number of context coding bins.

7. The method of claim 1 , wherein a syntax element indicating whether an absolute level of a prediction residual is greater than a value is excluded from counting the number of context coding bins.

8. 5. The method of claim 1, wherein the further limit depends on TB_size, which indicates the number of samples in the block.

9. The method of claim 8 , wherein the another limit is set to 0.5*TB_size.

10. The method of claim 2 or claim 4, wherein the transform residual coding process and the transform skip residual coding process use the same set of syntax elements when counting the number of context coding bins.

11. 1. An apparatus for video encoding, comprising one or more processors, The one or more processors: counting a number of context coding bins used to code one or more syntax elements associated with a prediction residual of a block, the limit being used to limit the number of context coding bins; counting a second number of context coding bins used to code a syntax element indicating a sign of the prediction residual of the block, wherein another limit is used to limit the second number of context coding bins; An apparatus configured to:

12. The apparatus of claim 11 , wherein the prediction residual for the block is coded using one of a transform residual coding process and a transform skip residual coding process.

13. An apparatus for video decoding, comprising one or more processors, The one or more processors: counting a number of context coding bins used to decode one or more syntax elements associated with a prediction residual of a block, wherein a limit is used to limit the number of context coding bins; counting a second number of context coding bins used to decode a syntax element indicating a sign of the prediction residual of the block, wherein another limit is used to limit the second number of context coding bins; An apparatus configured to:

14. The apparatus of claim 13, wherein the prediction residual of the block is decoded using one of a transform residual coding process and a transform skip residual coding process.

15. 15. The apparatus of claim 11, wherein the limit is set to 1.75*TB_size, where TB_size indicates the number of samples in the block.

16. The apparatus of claim 11 , wherein a syntax element indicating a sign of a prediction residual is excluded from counting the number of context coding bins in a transform skip residual coding process.

17. 15. The apparatus of claim 11, wherein a syntax element indicating whether an absolute level of a prediction residual is greater than a value is excluded from counting the number of context coding bins in a transform skip residual coding process.

18. 15. The apparatus of claim 11, wherein the further limit depends on TB_size, which indicates the number of samples in the block.

19. The apparatus of claim 18 , wherein the another limit is set to 0.5*TB_size.

20. The apparatus of claim 12 or 14, wherein the transform residual coding process and the transform skip residual coding process use a same set of syntax elements when counting the number of context coding bins.

Images