Unified context coded bin (ccb) count method

Abstract

In one implementation, the context coding bin (CCB) count method is unified between the transform residual coding process and the transform skip (TS) residual coding process. In one example, in TS residual coding, the CCB count excludes coeff_sign_flag, such that the syntax used for CCB count is unified for both residual coding processes. Further, the maximum number of separate context coded bins can be specified and used only for coeff_sign_flag. In another example, in TS residual coding, the maximum number of CCB counts is reduced from TB_size*2 to TB_size*1.75 for TB, or more generally, the maximum CCB count is set to the same value for both the transform residual coding and the TS residual coding, such that the maximum CCB count is unified for both residual coding processes.

Description

Technical Field

[0001] This embodiment generally relates to a unified context codec binary (CCB) scheme for transform residuals and transform skip residual modes. Background Technology

[0002] To achieve high compression efficiency, image and video codec schemes typically employ prediction and transform to utilize spatial and temporal redundancy in video content. Generally, intra-frame or inter-frame prediction is used to leverage intra-frame or inter-frame image correlations, and then the differences between the original and predicted blocks (typically represented as prediction error or prediction residuals) are transformed, quantized, and entropy encoded / decoded. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy encoding / decoding, quantization, transform, and prediction. Summary of the Invention

[0003] According to an embodiment, a video encoding method is provided, comprising: acquiring a prediction residual of a block to be encoded in an image; counting the number of context codec bits used to encode the prediction residual in the block, wherein the prediction residual of the block will be encoded using one of a transform residual encoding / decoding process and a transform skip residual encoding / decoding process, wherein the transform residual encoding / decoding process and the transform skip residual encoding / decoding process use the same bound to limit the number of context codec bits; and adjusting an entropy encoding / decoding method for one or more syntax elements in one of the transform residual encoding / decoding process and the transform skip residual encoding / decoding process in response to the bound and the number of context codec bits.

[0004] According to another embodiment, a video decoding method is provided, comprising: counting the number of context codec bits used to decode prediction residuals in a block, wherein the prediction residuals of the block will be decoded using one of a transform residual decoding process and a transform skip residual decoding process, wherein the transform residual decoding process and the transform skip residual decoding process use the same bound to limit the number of context codec bits; adjusting an entropy decoding method for one or more syntax elements in one of the transform residual decoding process and the transform skip residual decoding process in response to the bound and the number of context codec bits; and obtaining the prediction residuals of the block to be decoded in an image.

[0005] According to another embodiment, an apparatus for video encoding includes one or more processors, wherein the one or more processors are configured to: acquire prediction residuals of blocks to be encoded in an image; count the number of context codec bits used to encode the prediction residuals in the blocks, wherein the prediction residuals of the blocks will be encoded using one of a transform residual encoding / decoding process and a transform skip residual encoding / decoding process, wherein the transform residual encoding / decoding process and the transform skip residual encoding / decoding process use the same bound to limit the number of context codec bits; and adjust an entropy encoding / decoding method for one or more syntax elements in one of the transform residual encoding / decoding process and the transform skip residual encoding / decoding process in response to the bound and the number of context codec bits.

[0006] According to another embodiment, an apparatus for video decoding includes one or more processors, wherein the one or more processors are configured to: count the number of context codec bits for decoding prediction residuals in a block, wherein the prediction residuals of the block will be decoded using one of a transform residual decoding process and a transform skip residual decoding process, wherein the transform residual decoding process and the transform skip residual decoding process use the same bound to limit the number of context codec bits; adjust an entropy decoding method for one or more syntax elements in one of the transform residual decoding process and the transform skip residual decoding process in response to the bound and the number of context codec bits; and acquire the prediction residuals of the block to be decoded in an image.

[0007] One or more embodiments also provide a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above embodiments. One or more embodiments in this embodiment also provide a computer-readable storage medium storing instructions for encoding or decoding video data according to the above methods. One or more embodiments also provide a computer-readable storage medium storing a bitstream generated according to the above methods. One or more embodiments also provide a method and apparatus for transmitting or receiving a bitstream generated according to the above methods. Attached Figure Description

[0008] Figure 1 A block diagram of a system in which various aspects of this embodiment can be implemented is shown.

[0009] Figure 2 A block diagram of an embodiment of a video encoder is shown.

[0010] Figure 3 A block diagram of an embodiment of a video decoder is shown.

[0011] Figure 4 The codec tree unit representing a compressed image and the codec tree concept are shown.

[0012] Figure 5 The residual encoding / decoding structure of the transform block is shown.

[0013] Figure 6 The residual encoding / decoding structure of the transform skip block is shown.

[0014] Figure 7 The residual encoding and decoding process of the transform block in VTM 6.0 is shown.

[0015] Figure 8 The residual encoding and decoding process for transform skip blocks in VTM 6.0 is shown.

[0016] Figure 9 The residual encoding / decoding process for a transform skip block according to an embodiment is illustrated.

[0017] Figure 10 The residual encoding / decoding process for a transform skip block according to another embodiment is illustrated.

[0018] Figure 11 The residual encoding / decoding process for a transform skip block according to another embodiment is illustrated. Detailed Implementation

[0019] Figure 1 A block diagram illustrating an example of a system in which various aspects and embodiments may be implemented is shown. System 100 may be embodied as a device including various components described below and configured to perform one or more of the aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 100 may be embodied individually or in combination in a single integrated circuit, multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 100 are distributed across multiple ICs and / or discrete components. In various embodiments, system 100 is communicatively coupled to other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 100 is configured to implement one or more of the aspects described in this application.

[0020] System 100 includes at least one processor 110 configured to execute instructions loaded thereon for implementing various aspects described herein, such as those described. Processor 110 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 100 includes at least one memory 120 (e.g., a volatile memory device and / or a non-volatile memory device). System 100 includes a storage device 140, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 140 may include internal storage devices, additional storage devices, and / or network-accessible storage devices.

[0021] System 100 includes an encoder / decoder module 130 configured to, for example, process data to provide encoded and / or decoded video, and the encoder / decoder module 130 may include its own processor and memory. Encoder / decoder module 130 represents multiple modules that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Furthermore, encoder / decoder module 130 may be implemented as a separate element of system 100, or may be incorporated into processor 110 as a combination of hardware and software known to those skilled in the art.

[0022] Program code to be loaded onto processor 110 or encoder / decoder 130 to execute the various aspects described in this application may be stored in storage device 140 and subsequently loaded onto memory 120 for execution by processor 110. According to various embodiments, one or more of processor 110, memory 120, storage device 140, and encoder / decoder module 130 may store one or more of various items during the execution of the processes described in this application. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing equations, formulas, operations, and operational logic.

[0023] In several embodiments, the memory within processor 110 and / or encoder / decoder module 130 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be processor 110 or encoder / decoder module 130) is used for one or more of these functions. External memory may be memory 120 and / or storage device 140, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, fast external volatile memory, such as RAM, is used as working memory for video encoding / decoding operations, such as MPEG-2, HEVC, or VVC.

[0024] Inputs to the components of system 100 can be provided by various input devices, as shown in block 105. Such input devices include, but are not limited to, the following: (i) an RF section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a composite input terminal; (iii) a USB input terminal; and / or (iv) an HDMI input terminal.

[0025] In various embodiments, the input device of block 105 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform (including, for example, down-converting received signals to lower frequencies (e.g., intermediate frequencies or near-baseband frequencies) or baseband frequencies) various of these functions. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0026] Furthermore, USB and / or HDMI terminals may include corresponding interface processors for connecting system 100 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or processor 110. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed within a separate interface IC or processor 110. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements (including, for example, processor 110 and encoder / decoder 130), which operates in combination with memory and storage elements to process the data stream as needed for presentation on the output device.

[0027] The various components of system 100 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected and transmit data therebetween using a suitable connection arrangement 115 (e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards).

[0028] System 100 includes a communication interface 150 capable of communicating with other devices via a communication channel 190. The communication interface 150 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 190. The communication interface 150 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 190 may be implemented, for example, within a wired and / or wireless medium.

[0029] In various embodiments, a Wi-Fi network, such as IEEE 802.11, is used to stream data to system 100. The Wi-Fi signal in these embodiments is received via a communication channel 190 and a communication interface 150 suitable for Wi-Fi communication. The communication channel 190 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other super-top-level communications. Other embodiments use a set-top box that delivers data via an HDMI connection to input block 105 to provide streaming data to system 100. Still other embodiments use an RF connection to input block 105 to provide streaming data to system 100.

[0030] System 100 can provide output signals to various output devices, including display 165, speaker 175, and other peripheral devices 185. In various examples of embodiments, other peripheral devices 185 include one or more of the following: a standalone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 100. In various embodiments, control signals are communicated between system 100 and display 165, speaker 175, or other peripheral devices 185 using signaling such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. Output devices can be communicatively coupled to system 100 via dedicated connections through corresponding interfaces 160, 170, and 180. Alternatively, output devices can be connected to system 100 via communication interface 150 using communication channel 190. Display 165 and speaker 175 can be integrated into a single unit with other components of system 100 in electronic devices such as televisions. In various embodiments, display interface 160 includes a display driver, such as a timing controller (TCon) chip.

[0031] For example, if the RF portion of input 105 is part of a separate set-top box, then display 165 and speaker 175 may alternatively be separate from one or more of the other components. In various embodiments where display 165 and speaker 175 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0032] Figure 2 An example video encoder 200, such as the High Efficiency Video Codec (HEVC) encoder, is shown. Figure 2 Encoders that improve upon the HEVC standard or employ similar HEVC technology can also be shown, such as the VVC (Various Video Codec) encoder developed by JVET (Joint Video Exploration Team).

[0033] In this application, the terms "reconstructed" and "decoded" are used interchangeably, as are the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0034] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0035] To encode a video sequence having one or more images, the images are segmented (202) into, for example, one or more stripes, where each strip may include one or more strip segments. In HEVC, strip segments are organized into encoding / decoding units, prediction units, and transform units. The HEVC specification distinguishes between “blocks” and “units,” where a “block” addresses a specific region (e.g., luma, Y) in a sample array, and a “unit” includes a juxtaposed block of all encoded color components (Y, Cb, Cr, or monochromatic), syntax elements, and prediction data (e.g., motion vectors) associated with the block.

[0036] To encode and decode according to HEVC, the image is segmented into square codec tree blocks (CTBs) of configurable size (typically 64×64, 128×128, or 256×256 pixels), and contiguous sets of CTBs are grouped into stripes. A codec tree unit (CTU) contains the CTBs encoding the color components. A CTB is the root of a quadtree segmented into codec blocks (CBs), and a codec block can be segmented into one or more prediction blocks (PBs), forming the root of a quadtree segmented into transform blocks (TBs). Transform blocks (TBs) larger than 4x4 are divided into 4x4 sub-blocks of quantization coefficients called coefficient groups (CGs). Corresponding to the codec blocks, prediction blocks, and transform blocks, a codec unit (CU) comprises a set of prediction units (PUs) and tree-structured transform unit (TUs). PUs contain prediction information for all color components, and TUs contain the residual codec syntax structure for each color component. The sizes of the CBs, PBs, and TBs for the luma component are appropriate for the corresponding CUs, PUs, and TUs. In this application, the term "block" can be used to refer to, for example, any one of CTU, CU, PU, ​​TU, CG, CB, PB, and TB. Furthermore, the term "block" can also refer to macroblocks and segments specified in H.264 / AVC or other video codec standards, and more generally refers to data arrays of various sizes.

[0037] In encoder 200, images are encoded by encoder elements as described below. Images to be encoded are processed, for example, in units of CUs. Each encoding / decoding unit is encoded using either an intra-frame or inter-frame mode. When an encoding / decoding unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes is used to encode the encoding / decoding unit, indicating the intra / inter-frame decision via a prediction mode flag. The prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0038] The predicted residuals are then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vectors and other syntax elements, are entropy encoded (245) to output a bitstream. As a non-limiting example, context-based adaptive binary arithmetic encoding and decoding (CABAC) can be used to encode syntax elements into a bitstream.

[0039] To use CABAC encoding, a binarization process maps non-binary syntax element values ​​to binary sequences called binary bit (bin) strings. For each binary bit, a context model is selected. A "context model" is a probabilistic model for one or more binary bits, chosen from available models based on statistics of recently encoded / decoded symbols. The context model for each binary bit is identified by a context model index (also called a "context index"), and different context indices correspond to different context models. The context model stores the probability that each binary bit is "1" or "0" and can be adaptive or static. A static model triggers the encoding / decoding engine for binary bits "0" and "1" with equal probability. In an adaptive encoding / decoding engine, the context model is updated based on the actual encoded / decoded value of the binary bit. The operating modes corresponding to adaptive and static models are called normal mode and bypass mode, respectively. Based on this context, the binary arithmetic encoding / decoding engine encodes or decodes the binary bit according to the corresponding probabilistic model.

[0040] The encoder can also skip the transform and directly quantize the untransformed residual signal, for example, based on a 4×4 TU. The encoder can also bypass both the transform and quantization, i.e., directly encode and decode the residual without applying either the transform or quantization process. In direct PCM encoding and decoding, no prediction is applied, and the encoder-decoder unit samples are directly encoded and decoded into a bitstream.

[0041] The encoder decodes the coded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (255) to reconstruct the image blocks. For example, a loop filter (265) is applied to the reconstructed image to perform deblocking / sampling adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference image buffer (280).

[0042] Figure 3 A block diagram of an example video decoder 300, such as an HEVC decoder, is shown. In decoder 300, the bitstream is decoded by decoder elements as described below. Video decoder 300 typically performs operations similar to... Figure 2 The decoding rounds, which are the opposite of the encoding rounds described in the text, perform video decoding as part of the encoded video data. Figure 3Decoders that improve upon the HEVC standard or employ technologies similar to HEVC, such as VVC decoders, may also be shown.

[0043] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy-decoded (330) to obtain transform coefficients, motion vectors, image segmentation information, and other encoding / decoding information. If CABAC is used for entropy encoding / decoding, the context model is initialized in the same manner as the encoder's context model, and syntax elements are decoded from the bitstream based on the context model.

[0044] Image segmentation information indicates how the image is segmented, for example, the size of the CTU and how the CTU is divided into CUs (and possibly into PUs where applicable). Therefore, the decoder can segment (335) the image into, for example, CTUs, and then divide each CTU into CUs based on the decoded image segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals.

[0045] The predicted residual and predicted block are combined (355) to reconstruct an image block. The predicted block can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (380).

[0046] The decoded image can undergo further post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding process can use metadata derived in the pre-encoding process and signaled in the bitstream.

[0047] In HEVC, transform coefficients of the codec block are encoded and decoded using non-overlapping coefficient groups (CGs or sub-blocks), and each CG contains the coefficients of a 4×4 block of the codec block. In VVC draft 6, the choice of coefficient group size becomes solely dependent on the TB size, thus removing the dependency on the channel type. Therefore, various CGs (1x16, 2x8, 8x2, 2x4, 4x2, and 16x1) become available. CGs within a codec block and transform coefficients within a CG are encoded and decoded according to a predefined scan order.

[0048] To constrain the maximum number of context codec bits per pixel (bits encoded in regular mode), the area of ​​the TB is used to derive the maximum number of context codec bits allowed by the TB in VVC draft 6 (see “Algorithm description of Universal Video Codec and Test Model 6 (VTM 6)”, document JVET-O2002, 15th meeting: Gothenburg, Sweden, July 3-12, 2019). Figure 7 The residual encoding and decoding process of the transform block in VTM 6.0 is shown, and Figure 8 This illustrates the residual encoding / decoding process for transform skip blocks in VTM 6.0. For TB, the maximum number of context codec bits (CCB) is set to TB_zosize*1.75, as shown below. Figure 7 Step 710 in the diagram shows that TB_zosize indicates the number of samples within TB, excluding coefficients that are zeroed out. It is well known that large block partitions typically exhibit fewer residuals, and the energy is more concentrated in the low-frequency coefficients within the transform domain. In VTM (VVC Test Model) 6.0, for transform blocks larger than 32, high-frequency transform coefficients outside the 32×32 region are zeroed out. Note that, as... Figure 7 As shown in step 720, the coded_sub_block_flag is not considered for CCB counting.

[0049] VTM 6.0 allows transform-skip mode for luma blocks up to 32x32 (including 32x32). When encoding / decoding a CU in transform-skip mode, its prediction residuals are quantized (without transform, i.e., transform-skipped) and encoded / decoded using a transform-skipped residual encoding / decoding process. Unlike HEVC, which designs residual encoding / decoding based on statistical and signal characteristics at the transform coefficient level, two separate residual encoding / decoding structures are used for transform residuals (residuals of the transform, where the transform is not skipped) and transform-skipped residuals (residuals of the untransformed, where the transform is skipped). For example, as... Figure 8 As shown in step 810, in transform skip mode, the maximum number of CCBs is set to TB_size*2, where TB_size indicates the number of samples within TB. Furthermore, in transform skip residual mode, coeff_sign_flag is context-coded and considered for CCB counting, as... Figure 8 The step 840 is shown in the diagram. Residual encoding and decoding of transform residuals is also known as "transform coefficient encoding and decoding," "transform residual encoding and decoding," or "residual encoding and decoding of transform coefficients." Residual encoding and decoding of transform skip residuals is also known as "transform skip (TS) residual encoding and decoding."

[0050] Residual encoding and decoding of transform coefficients

[0051] In VVC draft 6, in order to encode transform coefficients, if the importance of a CG (sub-block) is equal to 1 (720), then the binary bits of the CG are encoded and decoded multiple times at the scan positions in that CG:

[0052] Round 1: The importance (sig_coeff_flag), the greater than 1 flag (gt1, abs_level_gtx_flag[0]), the parity flag (PAR, par_level_flag), and the greater than 3 flag (gt3, abs_level_gtx_flag[1]) are encoded and decoded in the order of encoding and decoding. The greater than 1 flag exists only if sig_coeff_flag is equal to 1. The parity flag and the greater than 3 flag (abs_level_gtx_flag[1]) are encoded and decoded only at scan positions where abs_level_gtx_flag[0] is equal to 1. If the values ​​of gt1, PAR, and gt3 do not exist in the bitstream, these values ​​are inferred to be 0. The SIG, PAR, gt1, and gt3 flags are encoded and decoded in normal mode (740). After these flags are encoded into coefficients, MaxCcbs is decremented by 4, and counter n is incremented by 1 to track the number of coefficients in a sub-block whose SIG, PAR, gt1, and gt3 flags are encoded in normal mode (750), where n is reset to 0 for each sub-block. Round 1 stops if all coefficients in the sub-block have been encoded (755) or the CCB count reaches the limit (730), thus allowing some scan positions to be skipped in round 1.

[0053] Round 2: For scan positions encoded and decoded in Round 1, the syntax element `abs_remainder` is encoded and decoded for scan positions where `gt3` equals 1 (760). Non-binary syntax elements are binaryized using Golomb-rice code, and the resulting binary bits are encoded and decoded in bypass mode of the arithmetic codec engine. For scan positions not encoded and decoded in Round 1, the syntax element `dec_abs_level` is binaryized using Golomb-rice code, and the resulting binary bits are encoded and decoded in bypass mode (760).

[0054] Round 3: Encode and decode the symbols (coeff_sign_flag) at all scan positions where sig_coeff_flag equals 1. Encode and decode the flags in bypass mode (780).

[0055] As described above, in transform coefficient encoding and decoding, the variable remBinsPass1(MaxCcbs) is first set to the maximum number of context codec bits (MCCB), and decremented by 1 when signaling informs the context codec bits. When remBinsPass1 is greater than or equal to 4, the flags in the first encoding round (including sig_coeff_flag, abs_level_gtx_flag[0](gt1, greater than 1 flag), par_level_flag, and abs_level_gtx_flag[1](gt3, greater than 3 flag)) are encoded and decoded using the context codec bits. If the number of context codec bits in the first encoding round is not greater than MCCB, the remaining portion of the level information indicating further encoding and decoding in the first round is encoded and decoded using the syntax elements of abs_remainder with the Golomb-rice code and bypass encoding and decoding bits. When remBinsPass1 becomes less than 4, in the second round, the remaining coefficients that were not encoded in the first round are directly encoded using the syntax element dec_abs_level by using Golomb-Rice code and bypass encoding / decoding bits. For each TB, remBinsPass1 is reset. The conversion from using context encoding / decoding bits for sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] to using bypass encoding / decoding bits for the remaining syntax elements occurs at most once per TB. For coefficient subblocks, if remBinsPass1 is less than 4, the entire coefficient subblock is encoded / decoded by using bypass encoding / decoding bits. After encoding / decoding at all the above levels, the sign (coeff_sign_flag) of all scan positions where sig_coeff_flag is equal to 1 is finally bypass encoded / decoded. This process (790) is performed on all subblocks of the transform block before the end (799).

[0056] For example, in Figure 5 In the middle, using the conventional mode to scan position C 15 After encoding sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] of C3, the number of CCBs reaches the limit and round 1 encoding / decoding stops. In round 2, if round 1 indicates that more level information should be encoded / decoded for the current scan position, abs_remainder is bypassed for scanning position C. 15C2, C1, and C0 are not encoded or decoded in round 1, and the absolute coefficient levels of these positions are directly encoded and decoded using the syntax element dec_abs_level in bypass mode. In round 3, if the coefficient of the current scan position is not zero, the syntax element coeff_sign_flag will be bypassed and encoded / decoded.

[0057] Transform skipped residual encoding and decoding

[0058] In transform skip mode, the statistical properties of the residual signal differ from those of the transform coefficients, and no energy compression is observed around the low-frequency components. The residual encoding / decoding is modified to account for the different signal characteristics of the (spatial) transform skip residuals, including:

[0059] Coefficient scanning is a forward scan;

[0060] No signaling notification for the final x / y position;

[0061] Encode and decode coded_sub_block_flag for each sub-block except the DC sub-block when all previous coded_sub_block_flags are equal to 0;

[0062] Modeling the sig_coeff_flag context using a simplified template;

[0063] abs_level_gtx_flag[0] and par_level_flag each use only one context model;

[0064] Additional markers greater than 5, 7, or 9;

[0065] Derivation of the corrected rice parameter for remainder binarization;

[0066] The symbol flag `coeff_sign_flag` is context-coded and its context modeling is determined based on the adjacent coefficient values ​​to the left and above; furthermore, the symbol flag is parsed after `sig_coeff_flag` to keep all context-coded bits together.

[0067] like Figure 6 and Figure 8As shown, if the importance of CG is 1 (820), the syntax elements sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], and par_level_flag are locally interleaved and encoded in the first round (830-860) by residual sampling, followed by the abs_level_gtx_flag bit plane (870) and abs_remainder encoding and decoding (880) as the second round. This process (890) is performed on all sub-blocks of the transform block before the end (899).

[0068] · Round 1 (830-860): sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], par_level_flag.

[0069] Round 2 (870): abs_level_gtx_flag[1], abs_level_gtx_flag[2] (flag greater than 5), abs_level_gtx_flag[3] (flag greater than 7), abs_level_gtx_flag[4] (flag greater than 9).

[0070] Round 3 (880): abs_remainder.

[0071] In transform skip mode, when the CCB count exceeds the maximum value, the remaining coefficient flags are bypassed and encoded / decoded using the same syntax structure, which is completely different from the transform residual case.

[0072] As mentioned above, the syntax for CCB counting, the maximum number of CCB counts, and the syntax design after the CCB count exceeds the maximum number are different between residual encoding and decoding of transform blocks and transform skip blocks. Therefore, some unification can be proposed here.

[0073] As mentioned above, to limit the maximum number of context codec bits (CCB) per pixel, VVC draft 6 uses the area of ​​the TB to derive the maximum number of context codec bits per TB. Furthermore, at the March 2019 meeting, JVET adopted a new residual coding process for transform skip (TS) residual blocks. This transform skip (TS) residual coding process differs significantly from the transform residual coding process for transform blocks. The main differences between transform residual coding and TS residual coding for CCBs are listed below:

[0074] 1. In transform residual encoding and decoding, the sign (coeff_sign_flag) is bypassed and not automatically included as syntax for CCB counting. On the other hand, in TS residual encoding and decoding, coeff_sign_flag is context-coded and included as syntax for CCB counting;

[0075] 2. In transform residual encoding and decoding, the maximum number of CCBs in a TB is equal to TB_zosize * 1.75. On the other hand, in TS residual encoding and decoding, the maximum number of CCBs in a TB is equal to TB_size * 2.

[0076] To unify the CCB constraints between the transform residual coding and decoding process and the TS residual coding and decoding process, some unifications are proposed while maintaining most of the gains provided by the two separate residual coding and decoding structures used for the transform block and the TS block.

[0077] In one embodiment, in TS residual encoding / decoding, we exclude `coeff_sign_flag` from the CCB count, making the syntax for CCB counting consistent across the two residual encoding / decoding processes. Furthermore, the maximum number of individual context encoding / decoding bits can be specified and used only for `coeff_sign_flag`. In another embodiment, in TS residual encoding / decoding, the maximum number of CCB counts for the TB is reduced from `TB_size*2` to `TB_size*1.75`. More generally, the maximum CCB count for both transform residual encoding / decoding and TS residual encoding / decoding is set to the same value, making the maximum CCB count consistent across the two residual encoding / decoding processes. Several methods for unifying CCB constraints between transform residual encoding / decoding and TS residual encoding / decoding processes are described in detail below.

[0078] Unified syntax for CCB counting

[0079] In VVC draft 6, coefficient flags such as sig_coeff_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[1] are included in the CCB count in the transform residual mode, such as Figure 7 Steps 740 and 750 are shown in the diagram. On the other hand, coefficient flags such as sig_coeff_flag, coeff_sign_flag, abs_level_gtx_flag[0], par_level_flag, and abs_level_gtx_flag[j] (1≤j≤4) are included in the CCB count in the transform skip residual mode, as shown in the diagram. Figure 8Steps 830-870 are shown in the table. In these two modes, there is a difference between whether `coeff_sign_flag` is included in the CCB count. Table 1 shows the syntax structure of the two residual encoding / decoding modes in VTM 6.0. As shown in Table 1, the residual of the encoding / decoding block is encoded / decoded using non-overlapping sub-blocks (sub-blocks), and each sub-block contains 16 coefficients. For a given sub-block, the coefficients are encoded / decoded inside a loop (sub-block loop) before processing the next sub-block, until all coefficients in that sub-block can be reconstructed. For a given coefficient, the coefficient-level syntax is encoded / decoded inside several rounds. In each round, after all the predefined syntax within that round of the coefficient has been encoded / decoded, it begins processing the next coefficient. The residual entropy encoding / decoding involves up to three rounds, which are used to encode / decode the following syntax elements:

[0080] coded_sub_block_flag: Specifies the importance of the 16-coefficient sub-block (zero / non-zero).

[0081] sig_coeff_flag: Specifies the importance of the coefficient (zero / non-zero).

[0082] abs_level_gtx_flag[j]: Specifies whether the absolute value of the coefficient level is greater than (j<<1)+1.

[0083] par_level_flag: Specifies the parity of the coefficients.

[0084] abs_remainder: Specifies the residual value of the absolute value at the coefficient level (if the value is greater than the value encoded or decoded in the previous round).

[0085] dec_abs_level: Specifies the absolute value of the coefficient level (if the coefficient has not been encoded or decoded in previous rounds).

[0086] coeff_sign_flag: Specifies the sign of the importance coefficient (0: positive, 1: negative).

[0087]

[0088] Table 1: Residual syntax structure of two residual encoding / decoding modes in VTM 6.0. The detailed syntax table of transform skip residual encoding / decoding in VVC draft 6 is shown in Table 2.

[0089]

[0090]

[0091]

[0092] Table 2: Syntax table for transform skip residual encoding / decoding in VVC Draft 6

[0093] As mentioned above, such as Figure 8 As shown in step 840, coeff_sign_flag is context-encoded and included in the CCB count in the TS residual mode. In one embodiment, it is proposed to exclude coeff_sign_flag from the CCB count in the TS residual mode, as... Figure 9 As shown in Table 3, changes relative to VTM 6.0 are underlined. Table 4 presents the syntax table for the proposed transform skip residual encoding / decoding, where changes relative to VVC draft 6 are strikethroughs (i.e., "MaxCcbs-" is removed after coeff_sign_flag is signaled (940)). Figure 9 Other steps in Figure 8 The other steps are the same. According to a variation of this embodiment, it is proposed to exclude abs_level_gtx_flag[j] (j>1) from the CCB count in TS residual encoding and decoding.

[0094]

[0095] Table 3: Residual syntax structure of the two proposed residual encoding / decoding modes

[0096]

[0097]

[0098] Table 4: Syntax table of the proposed transform skip residual encoding / decoding

[0099] Maximum number of unified CCB counts

[0100] In VVC Draft 6, as shown in Table 1 (also in...) Figure 7 and Figure 8 As shown in the diagram, it limits the number of context codec bits for each coefficient of luminance and chrominance coefficients to 1.75 bits / coefficient at the TB level, such as... Figure 7 As shown in step 710. On the other hand, TS residual codec sets this limitation to 2 bits / coefficient at the TB level, as... Figure 8 Step 810 is shown in the diagram.

[0101] Another unification proposed here is, for example, aligning the maximum allowed number of context codec bits for each coefficient used in transform residual codec and TS residual codec by allocating 1.75 bits / coefficient at the TB level, such as... Figure 10 Step 1010 is shown in the middle ( Figure 10 Other steps in Figure 8 The other steps are the same. In TS residual coding and decoding, it is expected that the worst-case number of context coding bits per residual will be reduced from 2 to 1.75. This change will improve the overall CABAC throughput and make CABAC constraints more consistent. Table 5 shows the maximum number of CCB allocations for the two residual coding and decoding modes proposed in this embodiment, with changes relative to VTM 6.0 underlined. Table 6 gives the syntax table of the proposed maximum number of CCB count allocations for transform skip residual coding and decoding, with changes relative to VVC draft 6 underlined.

[0102]

[0103] Table 5: Proposed Maximum Number of CCB Counts Allocation

[0104] residual_ts_coding(x0,y0,log2TbWidth,log2TbHeight,cIdx){ descriptor log2SbSize=(Min(log2TbWidth,log2TbHeight)<2?1:2) numSbCoeff=1<<(log2SbSize<<1) lastSubBlock=(1<<(log2TbWidth+log2TbHeight-2*log2SbSize))-1 inferSbCbf=1 <![CDATA[ MaxCcbs=(28*(1< <log2TbWidth)*(1<<log2TbHeight))> >4 ]]> …

[0105] Table 6: Proposed syntax table for the maximum number of CCB counts allocated for transform skip residual encoding / decoding

[0106] According to a variation of the second embodiment, the maximum allowed number of context codec bits for each coefficient of the transformation residual codec and the TS residual codec can be aligned by assigning another value x at the TB level instead of 1.75 bits / coefficient.

[0107] According to another variation of the second embodiment, the second embodiment can be combined with the first embodiment, which reduces the maximum number of CCBs for residual syntax other than coeff_sign_flag to 1.75 bits.

[0108] Allocate another number of context encoding / decoding bits (CCB_SIGN) for coeff_sign_flag.

[0109] According to the first embodiment, it is proposed to exclude coeff_sign_flag from the CCB count in TS residual mode. Since this increases the worst-case number of context codec bits for each residual to more than 2 in TS residual encoding / decoding, it will reduce the overall CABAC throughput.

[0110] To compensate for this deficiency, one solution is to combine the embodiments described in Tables 3 and 5 above, which reduces the maximum number of CCBs for residual syntaxes other than coeff_sign_flag. Another solution is to allocate an additional number of context codec bits (CCB_SIGN) for coeff_sign_flag. In this case, the CABAC throughput of coeff_sign_flag can be controlled independently.

[0111] The TB region can be used to export the maximum number of context codec bits (CCB_SIGN) of coeff_sign_flag in the TB. For example, as Figure 11 As shown in step 1110, the maximum number of CCB_SIGNs can be equal to TB_size * 0.5. Here, TB_size indicates the number of samples within TB, as shown in Table 7, where changes relative to VTM 6.0 are underlined. Figure 11 As shown in step 1140, the new CCB count MaxCcbs_sign instead of MaxCcbs is used for coeff_sign_flag (except for...). Figure 11 Steps other than 1110 and 1140 in the text Figure 8 The steps are the same as in the previous section). Table 8 presents the syntax table for the proposed transform skip residual encoding / decoding with the additional CCB_SIGN, where changes relative to VVC draft 6 are marked with strikethroughs or underlines.

[0112] According to a variation of the third embodiment, it can be applied in conjunction with the second embodiment. According to a variation of the third embodiment, CCB_SIGN can also be set and applied at the CG level.

[0113]

[0114] Table 7: Proposed residual syntax structure with additional CCB_SIGN

[0115]

[0116]

[0117] Table 8: Proposed syntax table for transform skipping residual encoding / decoding with additional CCB_SIGN

[0118] Notice, Figure 7-11 The methods described can be used on either the encoder or decoder side. When used on the encoder side, the term "encode / decode" in the figure can be understood as "encoding"; when used on the decoder side, the term "encode / decode" in the figure can be understood as "decoding". For the residual encoding / decoding process shown... Figure 9-11 When used on the encoder side, the quantization prediction residual of the transform skip block is used as input, and the encoder-decoder prediction residual is output. Figure 9-11 When describing the residual encoding and decoding process on the decoder side, the binary bits corresponding to the quantization prediction residual in the bitstream are decoded, and the quantization prediction residual is output for transforming skip blocks.

[0119] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first" and "second" can be used in various embodiments to modify elements, components, steps, operations, etc., e.g., "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within a time period overlapping with the second decoding.

[0120] The various methods and other aspects described in this application can be used to modify the module, for example, such as Figure 2 and Figure 3 The entropy encoding and decoding modules (245, 330) of the video encoder 200 and decoder 300 shown are illustrated. Furthermore, this aspect is not limited to VVC or HEVC and can be applied to, for example, other standards and recommendations, as well as any extensions of such standards and recommendations. Unless otherwise stated or technically excluded, the aspects described in this application may be used alone or in combination.

[0121] Various numerical values ​​are used in this application. Specific values ​​are used for illustrative purposes and the aspects described are not limited to these specific values.

[0122] One embodiment provides a computer program including instructions that, when executed by one or more processors, cause the one or more processors to perform an encoding or decoding method according to any of the above embodiments. One or more embodiments of this invention also provide a computer-readable storage medium storing instructions for encoding or decoding video data according to the above methods. One or more embodiments also provide a computer-readable storage medium storing a bitstream generated according to the above methods. One or more embodiments also provide a method and apparatus for transmitting or receiving a bitstream generated according to the above methods.

[0123] Various implementations involve decoding. As used herein, “decoding” can encompass, for example, all or part of the processing performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, these processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. It will be clear, and is considered well understood by those skilled in the art, whether the phrase “decoding process” is intended to specifically refer to a subset of operations or generally to a broader decoding process, based on the context of the specific description.

[0124] Various implementations involve encoding. In a manner similar to the discussion above regarding "decoding," the term "encoding" as used in this application can encompass, for example, all or part of the processing performed on the input video sequence to produce an encoded bitstream.

[0125] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0126] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the features under discussion can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in suitable hardware, software, and firmware. These methods can be implemented, for example, in apparatuses, such as processors, which generally refer to processing devices (including, for example, computers, microprocessors, integrated circuits, or programmable logic devices). Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0127] References to "an embodiment" or "an embodiment" or "an implementation" or "an implementation," and other variations thereof, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the appearance of the phrases "in an embodiment" or "in an embodiment" or "in an implementation" or "in an implementation," and any other variations appearing throughout this application, do not necessarily refer to the same embodiment.

[0128] Furthermore, this application may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, or information retrieved from memory.

[0129] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0130] Furthermore, this application may relate to "receiving" various types of information. Receiving, like "accessing," is a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory). Moreover, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0131] It should be understood that the use of any of the following “ / ”, “and / or”, and “…at least one of…”, for example, in the cases of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be clear to those skilled in the art and related fields, this can be extended to as many items as are listed.

[0132] It will be apparent to those skilled in the art that implementations can generate various signals that are formatted to carry, for example, information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of a described embodiment. Such a signal may be formatted as an electromagnetic wave or as a baseband signal, for example (e.g., using the radio frequency portion of a spectrum). Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

Claims

1. A method for video encoding or decoding, comprising: During the transform skip residual encoding / decoding process, a first number of context codec bits for encoding or decoding a set of syntax elements associated with level information of the prediction residual in the block is counted, wherein the set of syntax elements indicates at least one of the importance of the prediction residual, whether the absolute value of the prediction residual is greater than a certain value, and the parity of the prediction residual, wherein a limit is used to limit the first number of context codec bits, and wherein the set of syntax elements is context-coded before the first number of context codec bits reaches the limit of context codec bits, and then the set of syntax elements is bypassed and decoded before the first number of context codec bits exceeds the limit of context codec bits; as well as During the transform skip residual encoding / decoding process, a second number of context codec bits for the set of syntax flags used to encode or decode symbols indicating the predicted residuals in the block is counted, wherein another limit is used to limit the second number of context codec bits, and wherein the set of syntax flags is context-coded before the second number of context codec bits reaches the other limit of context codec bits, and then the set of syntax flags is bypassed after the second number of context codec bits exceeds the other limit of context codec bits.

2. The method according to claim 1, wherein, The limit is set to 1.75 * TB_size, where TB_size indicates the number of samples in the block.

3. The method according to claim 1, wherein, Syntax elements indicating whether the absolute level of the predicted residual is greater than the value are excluded from the first count of the binary bits encoded in the context.

4. The method according to claim 1, wherein, The other limit depends on TB_size, where TB_size indicates the number of samples in the block.

5. The method according to claim 4, wherein, The other limit is set to 0.5 * TB_size.

6. An apparatus for video encoding or decoding, comprising one or more processors, wherein the one or more processors are configured to: During the transform skip residual encoding / decoding process, a first number of context-coded bits for the set of syntax elements associated with level information of the prediction residuals in the block is counted, wherein a limit is used to constrain the first number of context-coded bits, and wherein the set of syntax elements is context-coded before the first number of context-coded bits reaches the limit of context-coded bits, and then the set of syntax elements is bypassed and decoded after the first number of context-coded bits exceeds the limit of context-coded bits; and During the transform skip residual encoding / decoding process, a second number of context codec bits for the set of syntax flags used to encode or decode symbols indicating the predicted residuals in the block is counted, wherein another limit is used to limit the second number of context codec bits, and wherein the set of syntax flags is context-coded before the second number of context codec bits reaches the other limit of context codec bits, and then the set of syntax flags is bypassed after the second number of context codec bits exceeds the other limit of context codec bits.

7. The apparatus according to claim 6, wherein, The limit is set to 1.75 * TB_size, where TB_size indicates the number of samples in the block.

8. The apparatus according to claim 6, wherein, During the transform skip residual encoding / decoding process, syntax elements indicating whether the absolute level of the predicted residual is greater than the value are excluded from the first count of the context encoding / decoding bits.

9. The apparatus according to claim 6, wherein, The other limit depends on TB_size, where TB_size indicates the number of samples in the block.

10. The apparatus according to claim 9, wherein, The other limit is set to 0.5 * TB_size.

11. A non-transitory computer-readable storage medium storing instructions, said instructions implementing an encoding or decoding method when executed by a processor, said encoding or decoding method comprising: During the transform skip residual encoding / decoding process, a first number of context codec bits for the set of syntax elements associated with the level information of the prediction residual in the block is counted, wherein a limit is used to limit the first number of context codec bits, and wherein the set of syntax elements is context-encoded before the first number of context codec bits reaches the limit of context codec bits, and then the set of syntax elements is bypassed decoded after the first number of context codec bits exceeds the limit of context codec bits; as well as During the transform skip residual encoding / decoding process, a second number of context codec bits for the set of syntax flags used to encode or decode symbols indicating the predicted residuals in the block is counted, wherein another limit is used to limit the second number of context codec bits, and wherein the set of syntax flags is context-coded before the second number of context codec bits reaches the other limit of context codec bits, and then the set of syntax flags is bypassed after the second number of context codec bits exceeds the other limit of context codec bits.

12. The medium of claim 11, wherein the other limit depends on TB_size, wherein TB_size indicates the number of samples in the block.

Images