Method and apparatus for video coding and decoding

By optimizing the maximum usage constraints of conventional CABAC codec and flexible allocation of conventional binary bits in video encoding and decoding, the problem of low allocation efficiency of conventional binary bits in the prior art is solved, and the compression efficiency and overall performance of video encoding and decoding are improved.

CN118945361APending Publication Date: 2024-11-12INTERDIGITAL VC HOLDINGS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411204807.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-05-23
Filing Date
2020-03-06
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the residual codec process of existing video encoding and decoding, the allocation efficiency of conventional binary bits is low, resulting in insufficient use of compression efficiency. Especially in the inter-frame prediction and transformation process, the constraints of conventional binary bits are too strict. It affects the overall performance of video encoding and decoding.

Method used

By optimizing the maximum usage constraints of conventional CABAC codecs in the residual codec of blocks and their coefficients, high-level constraints are followed, and flexibly allocating conventional binary bits between multiple codec groups, improving compression efficiency . Specific methods include setting higher levels of constraints at the transform block, transform unit or codec unit level, allowing the number of conventional binary bits per unit area, and thus allocating the budget more efficiently on smaller units.

Benefits of technology

Improve the compression efficiency of video encoding and decoding, ensure compliance with high-level constraints, and better allocate conventional binary bits at different encoding and decoding levels, improving the overall performance of the encoding and decoding process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945361A_ABST
    Figure CN118945361A_ABST
Patent Text Reader

Abstract

In at least one embodiment, a method and apparatus for encoding / decoding a video, where the video encoding method includes determining a number of regular bins, and performing CABAC encoding based on the determined number of regular bins.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the application with application number 202080020023.2, application date March 6, 2020, national entry date September 9, 2021, and invention name “Flexible allocation of conventional binary bits in residual coding for video coding”. Technical Field

[0002] At least one of the present embodiments generally relates to a conventional allocation of bins in a residual codec for video encoding or decoding. Background Art

[0003] In order to achieve high compression efficiency, image and video coding schemes usually use prediction and transformation to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit frame correlation within or between frames, and then the difference between the original block and the predicted block (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded and decoded. In order to reconstruct the video, the compressed data is decoded by the inverse process corresponding to entropy encoding and decoding, quantization, transformation, and prediction. Summary of the invention

[0004] One or more of the embodiments herein process the high-level constraints on the maximum usage of conventional CABAC codecs of binary bits during residual coding of blocks and their coefficient groups, such that the high-level constraints are observed and compression efficiency is improved compared to current methods.

[0005] According to a first aspect of at least one embodiment, a video encoding method comprises: a residual encoding process, which encodes and decodes a syntax element representing a picture area including a codec group using a limited number of regular binary bits, wherein the codec is CABAC encoding, and wherein the number of regular binary bits is determined based on a budget allocated among multiple codec groups.

[0006] According to a second aspect of at least one embodiment, a video decoding method comprises: a residual decoding process, which uses regular binary bits to parse a bit stream representing a picture area including a codec group, wherein the decoding is completed using CABAC decoding, and wherein the number of regular binary bits is determined based on a budget allocated among multiple codec groups.

[0007] According to a third aspect of at least one embodiment, a device comprises: an encoder for encoding picture data of at least one block in a picture or video, wherein the encoder is configured to perform a residual coding process using a limited number of regular binary bits to encode and decode syntax elements representing a picture area including a codec group, wherein the codec is CABAC coding, and wherein the number of regular binary bits is determined based on a budget allocated between multiple codec groups.

[0008] According to a fourth aspect of at least one embodiment, a device comprises: a decoder for decoding picture data of at least one block in a picture or video, wherein the decoder is configured to perform a residual decoding process using regular binary bits to parse a bitstream representing a picture area including a codec group, wherein the decoding is performed using CABAC decoding, and wherein the number of regular binary bits is determined based on a budget allocated between multiple codec groups.

[0009] According to a fifth aspect of at least one embodiment, there is presented a computer program comprising program code instructions executable by a processor, the computer program implementing the steps of the method according to at least the first or second aspect.

[0010] According to a sixth aspect of at least one embodiment, there is presented a computer program product stored on a non-transitory computer-readable medium and comprising program code instructions executable by a processor, which implements the steps of the method according to at least the first or second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A block diagram of an example of a video encoder 100 , such as a High Efficiency Video Coding (HEVC) encoder, is shown.

[0012] Figure 2 A block diagram of an example of a video decoder 200 , such as a HEVC decoder, is shown.

[0013] Figure 3 A block diagram is shown of an example of a system in which various aspects and embodiments are implemented.

[0014] Figure 4A Examples of codec tree units and codec trees in the compressed domain are shown.

[0015] Figure 4B An example of dividing a CTU into a coding unit, a prediction unit, and a transform unit is shown.

[0016] Figure 5 The use of two scalar quantizers in dependent scalar quantization is shown.

[0017] Fig. 6A and Figure 6B An example mechanism for switching between scalar quantizers is shown.

[0018] Figure 6C An example of the scanning order between CG and coefficients as used in VVC in an 8×8 transform block is shown.

[0019] Fig. 7A An example of syntax elements for encoding / parsing of transform blocks is shown.

[0020] Figure 7B An example of CG level residual codec syntax is shown.

[0021] Fig. 8A An example of CU level syntax is shown.

[0022] Figure 8B An example of the transform_tree syntax structure is shown.

[0023] Figure 8C The transform unit level syntax arrangement is shown.

[0024] Fig. 9A and Fig. 9B The CABAC decoding and encoding processes are shown separately.

[0025] Fig.10 A CABAC encoding process including a CABAC optimizer is shown, in accordance with an embodiment of the present principles.

[0026] Fig.11 An example flow chart of a CABAC optimizer used in an encoding process according to an embodiment of the present principles is shown.

[0027] Fig.12 An example of a modified process for encoding / decoding a codec group using a CABAC optimizer is shown.

[0028] Fig.13A An example embodiment is shown in which the budget of the conventional bins is determined at the transform block level.

[0029] Fig. 13B An example embodiment is shown in which the budget of the normal bins is determined at the transform block level according to the position of the last significant coefficient.

[0030] Fig.14A An example embodiment is shown in which the budget of regular bins is determined at transform unit level.

[0031] Fig. 14BAn example embodiment is shown, where regular bins are determined at transform unit level and allocated between transform blocks according to the relative surfaces between different transform blocks and according to the regular bins used in transform blocks that have been encoded / parsed in the transform unit under consideration.

[0032] Fig. 14C An example embodiment of a process for TB-level residual encoding / parsing is shown.

[0033] Fig.15A An example embodiment is shown in which a budget of regular bins is allocated at the codec unit level.

[0034] Fig. 15B An example embodiment is shown for encoding / parsing a transform tree associated with a current CU based on a regular bin budget allocated at the CU level.

[0035] Fig. 15C A variant embodiment of the codec tree encoding / parsing process when the budget is fixed at the CU level is shown.

[0036] Fig.15D and Fig.15E The transform unit encoding / decoding / parsing process is shown, which is applicable to the present embodiment when the conventional binary bit budget is fixed at the CU level.

[0037] Fig.16A and Fig. 16B An example embodiment is shown where a budget for regular bins is allocated at a higher level than the CG level, and two types of CG encoding / decoding / parsing processes are to be used. DETAILED DESCRIPTION

[0038] Various embodiments relate to entropy coding of quantized transform coefficients. This stage of the video codec is also referred to as the residual coding step. At least one embodiment aims to optimize the coding efficiency of the video codec under the constraint of the maximum number of binary bits per unit area of ​​conventional coding.

[0039] The various methods and other aspects described in this application can be used to modify Figure 1 and Figure 2At least the entropy encoding and decoding and / or entropy decoding modules (145, 230) of the video encoder 100 and decoder 200 shown. In addition, although the principles related to a specific draft of the VVC (Versatile Video Codec) or HEVC (High Efficiency Video Codec) specification are described, the various aspects of the present application are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations (whether existing or developed in the future) and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated, or technically excluded, the various aspects described in the present application can be used alone or in combination.

[0040] Figure 1 A block diagram of an example of a video encoder 100 , such as a HEVC encoder, is shown. Figure 1 It is also possible to show encoders in which improvements are made to the HEVC standard or encoders that adopt techniques similar to HEVC, such as the JEM (Joint Exploration Model) or VTM (VVC Test Model) encoders developed by JVET (Joint Video Exploration Team) for VVC.

[0041] Before being encoded, a video sequence may undergo a pre-encoding process (101). This is performed, for example, by applying a color transform to the input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0042] In HEVC, to encode a video sequence having one or more pictures, the pictures are partitioned (102) into one or more slices, where each slice may include one or more slice segments. Slice segments are organized into codec units, prediction units, and transform units. The HEVC specification distinguishes between "blocks" and "units", where a "block" addresses a specific area in the sample array (e.g., luma, Y), and a "unit" includes a collocated block of all coded color components (Y, Cb, Cr, or monochrome), syntax elements, and prediction data associated with the block (e.g., motion vectors).

[0043] For codecs in HEVC, pictures are partitioned into square codec tree blocks (CTBs) of configurable size, and a continuous set of codec tree blocks are grouped into a slice. A codec tree unit (CTU) contains the CTBs of the coded color components. The CTB is the root of a quadtree partitioned into codec blocks (CBs), which can be partitioned into one or more prediction blocks (PBs) and form the root of a quadtree partitioned into transform blocks (TBs). Corresponding to the codec blocks, prediction blocks, and transform blocks, the codec unit (CU) includes a prediction unit (PU) and a tree-structured transform unit (TU) group, the PU includes prediction information for all color components, and the TU includes a residual codec syntax structure for each color component. The sizes of the CB, PB, and TB of the luminance component apply to the corresponding CU, PU, ​​and TU. In the present application, the term "block" may be used to refer to, for example, any of CTU, CU, PU, ​​TU, CB, PB, and TB. In addition, "block" can also be used to refer to macroblocks and partitions specified in H.264 / AVC or other video coding standards, and more generally to data arrays of various sizes.

[0044] As described below, in the example of encoder 100, a picture is encoded by an encoder element. The picture to be encoded is processed in units of CUs. Each CU is encoded using intra or inter mode. When a CU is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and motion compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the CU, and indicates the intra / inter decision through a prediction mode flag. The prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0045] A CU in intra mode is predicted from neighboring samples reconstructed within the same strip. A set of 35 intra prediction modes are available in HEVC, including DC, planar, and 33 angular prediction modes. The intra prediction reference is reconstructed from rows and columns adjacent to the current block. The reference extends twice the block size in the horizontal and vertical directions using the available samples from the previously reconstructed block. When an angular prediction mode is used for intra prediction, the reference samples can be copied along the direction indicated by the angular prediction mode.

[0046] Two different options can be used to encode and decode the applicable luma intra prediction mode for the current block. If the applicable mode is included in a constructed list of three most probable modes (MPMs), the mode is signaled by its index in the MPM list. Otherwise, the mode is signaled by a fixed-length binarization of the mode index. The three most probable modes are derived from the intra prediction modes of the top and left neighboring blocks.

[0047] For an inter CU, the corresponding codec block is further partitioned into one or more prediction blocks. Inter prediction is performed at the PB level, and the corresponding PU contains information about how to perform inter prediction. Motion information (e.g., motion vector and reference picture index) can be signaled in two ways, namely "merge mode" and "advanced motion vector prediction (AMVP)".

[0048] In merge mode, the video encoder or decoder assembles a candidate list based on the blocks that have been coded and decoded, and the video encoder signals the index of one of the candidates in the candidate list. At the decoder side, the motion vector (MV) and reference picture index are reconstructed based on the signaled candidates.

[0049] In AMVP, a video encoder or decoder assembles a candidate list based on motion vectors determined from blocks that have already been coded. The video encoder then signals an index into the candidate list to identify the motion vector predictor (MVP) and signals the motion vector difference (MVD). On the decoder side, the motion vector (MV) is reconstructed as MVP+MVD. The applicable reference picture index is also explicitly coded in the PU syntax of AMVP.

[0050] The prediction residual is then transformed (125) and quantized (130), which includes at least one embodiment for adjusting the chrominance quantization parameters described below. The transform is typically based on a separable transform. For example, a DCT transform is first applied in the horizontal direction and then in the vertical direction. In recent codecs (such as JEM), the transforms used in the two directions can be different (e.g., DCT in one direction and DST in the other), which leads to a wide variety of 2D transforms, while in previous codecs the variety of 2D transforms for a given block size was typically limited.

[0051] The quantized transform coefficients are entropy encoded (145) along with motion vectors and other syntax elements to output a bitstream. The encoder can also skip the transform and apply quantization directly to the untransformed residual signal on a 4×4TU basis. The encoder can also bypass both transform and quantization, that is, encode and decode the residual directly without applying a transform or quantization process. In direct PCM encoding and decoding, prediction is not applied and the codec unit samples are encoded and decoded directly into the bitstream.

[0052] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. An in-loop filter (165) is applied to the reconstructed picture, for example, to perform deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered picture is stored in a reference picture buffer (180).

[0053] Figure 2 1 shows a block diagram of an example of a video decoder 200, such as a HEVC decoder. In the example of the decoder 200, a bitstream is decoded by a decoder element as described below. The video decoder 200 generally performs the same Figure 1 The decoding process is the reverse of the encoding process described in , where the encoding process performs video decoding as part of encoding the video data. Figure 2 It is also possible to show decoders that improve the HEVC standard or decoders that adopt techniques similar to HEVC, such as JEM or VVC decoders.

[0054] Specifically, the input of the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, picture segmentation information and other encoding and decoding information. The picture segmentation information indicates the size of the CTU, and the way the CTU is divided into CUs, and if applicable, may also indicate the way the CTU is divided into PUs. Therefore, the decoder can divide (235) the picture into CTUs and each CTU into CUs based on the decoded picture segmentation information. The transform coefficients are dequantized (240), which includes at least one embodiment for adjusting the chroma quantization parameters described below, and the transform coefficients are inversely transformed (250) to decode the prediction residual.

[0055] The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block can be obtained (270) from intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). As described above, AMVP and merge mode techniques can be used to derive motion vectors for motion compensation, which can use interpolation filters to calculate interpolation of sub-integer samples of the reference block. An in-loop filter (265) is applied to the reconstructed image. The filtered picture is stored in a reference picture buffer (280).

[0056] The decoded picture may further undergo post-decoding processing (285), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing an inverse remapping that is the opposite of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0057] Figure 3 A block diagram of an example of a system in which various aspects and embodiments are implemented is shown. System 300 may be embodied as a device including various components described below, and is configured to perform one or more aspects described in this application. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, encoders, code converters, and servers. The elements of system 300 may be embodied in a single integrated circuit, multiple ICs, and / or discrete components, either alone or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 300 are distributed over multiple ICs and / or discrete components. In various embodiments, the elements of system 300 are communicatively coupled via an internal bus 310. In various embodiments, system 300 is communicatively coupled to other similar systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports. In various embodiments, system 300 is configured to implement one or more aspects described in this document, such as the modified video encoder 100 and video decoder 200 described above and described below.

[0058] System 300 includes at least one processor 301, which is configured to execute instructions loaded therein, for implementing various aspects described in this document, for example. Processor 301 may include embedded memory, input and output interfaces, and various other circuits known in the art. System 300 includes at least one memory 302 (e.g., a volatile memory device and / or a non-volatile memory device). System 300 includes storage device 304, which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash memory, disk drive, and / or optical drive. As a non-limiting example, storage device 304 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0059] The system 300 includes a coder / decoder module 303, which is configured to, for example, process data to provide encoded video or decoded video, and the coder / decoder module 303 may include its own processor and memory. The coder / decoder module 303 represents a (multiple) module that can be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, as known to those skilled in the art, the coder / decoder module 303 may be implemented as a separate element of the system 300, or may be incorporated in the processor 301 as a combination of hardware and software.

[0060] Program code to be loaded onto the processor 301 or encoder / decoder 303 to perform various aspects described in this document may be stored in the storage device 304 and subsequently loaded onto the memory 302 for execution by the processor 301. According to various embodiments, one or more of the processor 301, memory 302, storage device 304, and encoder / decoder module 303 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0061] In several embodiments, memory internal to the processor 301 and / or encoder / decoder module 303 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 301 or the encoder / decoder module 303) is used for one or more of these functions. The external memory may be a memory 302 and / or a storage device 304, such as a dynamic volatile memory and / or a non-volatile flash memory. In several embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, for example for MPEG-2, HEVC, or VVC, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations.

[0062] As shown in block 309, input may be provided to the elements of system 300 via various input devices. Such input devices include, but are not limited to, (i) an RF section that receives an RF signal transmitted over the air, such as by a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0063] In various embodiments, the input device of block 309 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with the following elements required: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal sent by wired (for example, cable) medium, and filter to the desired frequency band again to perform frequency selection by filtering, down-conversion and filtering. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some in these elements, and / or add other elements of similar or different functions. Adding element can include inserting element between existing elements, for example inserting amplifier and analog-to-digital converter. In various embodiments, the RF part includes antenna.

[0064] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 300 to other electronic devices via common USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within the processor 301. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed within a separate interface IC or within the processor 301. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 301 and the encoder / decoder 303, which operates in conjunction with memory and storage elements to process the data stream as needed for presentation on an output device.

[0065] The various components of system 300 may be provided in an integrated housing. In the integrated housing, the various components may be connected to each other and transmit data between them using suitable connection means (e.g., internal buses known in the art, including I2C buses, wiring, and printed circuit boards).

[0066] The system 300 includes a communication interface 305 that enables communication with other devices via a communication channel 320. The communication interface 305 may include, but is not limited to, a transceiver configured to send and receive data through the communication channel 320. The communication interface 305 may include, but is not limited to, a modem or a network card, and the communication channel 320 may be implemented, for example, within a wired and / or wireless medium.

[0067] In various embodiments, data is streamed to the system 300 using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signals of these embodiments are received through a communication channel 320 and a communication interface 305 suitable for Wi-Fi communication. The communication channel 320 of these embodiments is typically connected to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications. Other embodiments provide streaming data to the system 300 using a set-top box that delivers data through an HDMI connection of an input box 309. Still other embodiments provide streaming data to the system 300 using an RF connection of an input box 309.

[0068] The system 300 can provide output signals to various output devices, including a display 330, a speaker 340, and other peripherals 350. In various examples of embodiments, the other peripherals 350 include one or more of a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system 300. In various embodiments, control signals are transmitted between the system 300 and the display 330, the speaker 340, or other peripherals 350 using signaling (such as AV link, CEC, or other communication protocols that enable device-to-device control regardless of user interference). The output devices can be communicatively coupled to the system 300 via dedicated connections through respective interfaces 306, 307, and 308. Alternatively, the output devices can be connected to the system 300 using a communication channel 320 via a communication interface 305. The display 330 and the speaker 340 can be integrated into a single unit with other components of the system 300 in an electronic device (e.g., a television). In various embodiments, the display interface 306 includes a display driver, such as a timing controller (T Con) chip.

[0069] For example, if the RF portion of input 309 is a part of a separate set-top box, then the display 330 and the speaker 340 can also be selectively separated from one or more other components. In various embodiments where the display 330 and the speaker 340 are external components, an output signal can be provided via a dedicated output connection (including, for example, an HDMI port, a USB port, or a COMP output). The implementation described herein can be implemented in, for example, a method or process, a device, a software program, a data stream, or a signal. Even if only discussed in the context of a single implementation form (for example, discussed only as a method), the implementation of the discussed feature can also be implemented in other forms (for example, a device or program). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device such as a processor, and the processor generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0070] Figure 4A An example of a codec tree unit and a codec tree in the compressed domain is shown. In the HEVC video compression standard, motion compensated temporal prediction is used to exploit the redundancy existing between consecutive pictures of a video. To this end, pictures are partitioned into so-called codec tree units (CTUs), which are typically 64×64, 128×128 or 256×256 pixels in size. Each CTU is represented by a codec tree in the compressed domain (e.g. in a quadtree partitioning of the CTU). Each leaf is called a codec unit (CU).

[0071] Figure 4B An example of dividing a CTU into codec units, prediction units and transform units is shown. Each unit is then given some intra or inter prediction parameters (prediction information). For this purpose, it is spatially split into one or more prediction units (PUs), each of which is assigned some prediction information, such as motion vectors. Intra or inter codec modes are assigned at the CU level.

[0072] In this application, the terms "reconstructed" and "decoded" can be used interchangeably, the terms "encoded" or "encoded and decoded" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side. The term "block" or "picture block" may be used to refer to any of CTU, CU, PU, ​​TU, CB, PB and TB. In addition, the term "block" or "picture block" may be used to refer to macroblocks, partitions, and sub-blocks specified in H.264 / AVC or other video codec standards, and more generally to sample arrays of various sizes.

[0073] Figure 5 The use of two scalar quantizers in dependent scalar quantization is shown. Dependent scalar quantization as proposed in JVET (contribution JVET-J0014) uses two scalar quantizers with different reconstruction levels for quantization. The main effect of this approach compared to conventional scalar quantization (such as used in HEVC and VTM-1) is that the set of admissible reconstruction values ​​of the transform coefficients depends on the values ​​of the transform coefficient levels that precede the current transform coefficient level in the reconstruction order. The method of dependent scalar quantization is implemented by (a) defining two scalar quantizers with different reconstruction levels, and (b) defining a switching procedure between the two scalar quantizers. As Figure 5 As shown, the two scalar quantizers used, denoted by Q0 and Q1, are shown. The positions of the available reconstruction levels are uniquely specified by the quantization step size Δ. When ignoring the fact that the actual reconstruction of the transform coefficients uses integer arithmetic, the characteristics of the two scalar quantizers Q0 and Q1 are as follows:

[0074] Q0: The reconstruction level of the first quantizer Q0 is given by an even integer multiple of the quantization step size Δ. When this quantizer is used, the dequantized transform coefficient t' is calculated according to the following formula

[0075] t'=2·k·Δ,

[0076] Where k represents the associated quantization coefficient (transmitted quantization index).

[0077] Q1: The reconstruction level of the second quantizer Q1 is given by an odd integer multiple of the quantization step size Δ plus a reconstruction level equal to zero. Based on the quantization coefficient k, the dequantized transform coefficient t' is calculated as follows:

[0078] t'=(2·k-sgn(k))·Δ,

[0079] where sgn(·) is a symbolic function, defined as follows:

[0080] sgn(x)=(k==0?0:(k<0?-1:1)).

[0081] The scalar quantizer used (Q0 or Q1) is not explicitly signaled in the bitstream. Instead, the quantizer used for the current transform coefficient is determined by the parity of the quantized coefficients preceding the current transform coefficient in the encoding / reconstruction order and the state of its finite state machine introduced below.

[0082] Fig. 6A and Figure 6B The mechanism of switching between scalar quantizers in VVC is shown. Fig. 6AAs shown, the switching between the two scalar quantizers (Q0 and Q1) is implemented via a finite state machine with four states, which are labeled 0, 1, 2 or 3 respectively. For a given quantization coefficient, the state of the finite state machine considered is uniquely determined by the parity of the quantization coefficient k before the current quantization coefficient in the encoding / decoding / reconstruction order and the state of the finite state machine considered when processing the previous coefficient. At the beginning of inverse quantization of the transform block, the state is set equal to 0. The transform coefficients are reconstructed in a scan order (i.e., in the same order as entropy decoding). After the current transform coefficient is reconstructed, the state is updated. k is the quantization coefficient. The next state depends on the current state and the parity of the current quantization coefficient k (k&1):

[0083] state=stateTransTable[current_state][k&1],

[0084] Where stateTransTable represents Figure 6B In the state transition table shown, the operator & specifies a bitwise AND operation in two's-complement arithmetic.

[0085] Furthermore, the quantized coefficients contained in so-called transform blocks (TBs) may be entropy coded and decoded as described below.

[0086] First, the transform block is divided into 4×4 sub-blocks of quantized coefficients called codec groups (sometimes also called coefficient groups and abbreviated CG). Entropy encoding / decoding consists of several scanning passes, which are based on Figure 6C The TBs are scanned in the diagonal scan order shown.

[0087] The transform coefficient coding and decoding in VVC includes five main steps: scanning, last significant coefficient coding and decoding, significance map coding and decoding, coefficient level remainder coding and decoding, absolute level and sign data coding and decoding.

[0088] Figure 6C The scanning order between the CGs and coefficients used in VVC in an 8×8 transform block is shown. The scanning pass over the TB then consists of sequentially processing each CG according to the diagonal scanning order, and the 16 coefficients within each CG are also scanned according to the scanning order considered. The scanning pass starts with the last significant coefficient in the TB and processes all coefficients until the DC coefficient.

[0089] Entropy coding of transform coefficients includes up to 7 syntax elements from the following list:

[0090] -coded_sub_block_flag: Validity of coefficient group (CG)

[0091] -sig_flag: Validity of coefficient (zero / non-zero).

[0092] -gt1_flag: Indicates whether the absolute value of the coefficient level is greater than 1

[0093] -par_flag: Indicates the parity of coefficients greater than 1

[0094] -gt3_flag: Indicates whether the absolute value of the coefficient level is greater than 3

[0095] - Remainder: the remainder of the absolute value of the coefficient level (if it is larger than the value encoded in the previous pass)

[0096] - abs_level: value of the absolute value of the coefficient level (if no CABAC bin is signaled for the current coefficient for maximum number of bin budget issues)

[0097] -sign_data: The signs of all significant coefficients contained in the considered CG. It consists of a series of bits, each bit signaling the sign (0: positive, 1: negative) of each non-zero transform coefficient.

[0098] Once the absolute value of a quantized coefficient is known by decoding a subset of the above elements (except the sign), no further syntax elements are encoded or decoded for the absolute value of this coefficient. In the same way, the sign flag is signaled only for non-zero coefficients.

[0099] Before entering the next CG, all necessary scan passes of a given CG are encoded and decoded until all quantized coefficients in that CG can be reconstructed.

[0100] The entire decoding TB parsing process consists of the following main steps:

[0101] 1. Decode the last significant coefficient coordinates. This includes the following syntax elements: last_sig_coeff_x_prefix, last_sig_coeff_y_prefix, last_sig_coeff_x_suffix, and last_sig_coeff_y_suffix. This provides the decoder with the spatial position (x-coordinate and y-coordinate) of the last non-zero coefficient in the entire TB.

[0102] Then, for each consecutive CG from the CG containing the last significant coefficient in the TB to the top-left CG in the TB, apply the following steps:

[0103] 2. Decode the CG validity flag (called coded_sub_block_flag in the VVC specification).

[0104] 3. Decode the significant coefficient flag for each coefficient in the considered CG. This corresponds to the syntax element sig_flag. This indicates which coefficients in the CG are non-zero.

[0105] In the considered CG, for coefficients known to be non-zero, the next parsing stage is related to the coefficient level. This involves the following syntax elements:

[0106] 4.gt1_flag: This flag indicates whether the absolute value of the current coefficient is greater than 1. Otherwise, the absolute value is equal to 1.

[0107] 5.par_flag: This flag indicates whether the current quantized coefficient is even. If the gt1_flag of the current quantized coefficient is true, it is encoded and decoded. If par_flag is zero, the quantized coefficient is even, otherwise it is odd. After par_flag is parsed on the decoder side, part of the decoded quantized coefficient is set equal to (1+gt1_flag+par_flag).

[0108] 6.gt3_flag: This flag indicates whether the absolute value of the current coefficient is greater than 3. If not, the absolute value is equal to 1+gt1_flag+par_flag. If 1+gt1_flag+par_flag is greater than or equal to 2, gt3_flag is encoded and decoded. Once gt3_flag is parsed, the quantized coefficient value becomes 1+gt1_flag+par_flag+(2*GT3_flag) on ​​the decoder side.

[0109] 7. Remainder: This remainder encodes the absolute value of the coefficient. This applies if the partially decoded absolute value is greater than or equal to 4. Note that in the example of VVC Draft 3, the maximum number of regular codec bit budgets is fixed for each codec group. Therefore, for some coefficients, only the sig_flag, gt1_flag and par_flag elements may be signaled, while for other coefficients, gt3_flag may also be signaled. Therefore, the remainder value for coding and parsing is calculated based on the partially decoded quantized coefficients, relative to the already decoded flags of the coefficient under consideration.

[0110] 8. abs_level: This indicates the absolute value of the coefficients for which no flag (among sig_flag, gt1_flag, papr_flag or gt3_flag) is coded in the considered CG, with respect to the maximum number of bins for conventional coding. This syntax element is similar to the remainder syntax element Rice-Golomb binarization and bypass coding.

[0111] 9.sign_flag: This indicates the sign of the non-zero coefficient. This is for bypass coding.

[0112] Fig. 7A The syntax elements for encoding / decoding / parsing of transform blocks according to VVC draft 4 are shown. The encoding / decoding / parsing includes a 4-pass process. As can be seen, for a given transform block, the position of the last significant coefficient is first encoded and decoded. Then, for each CG from the CG containing the last significant coefficient (excluded) to the first CG, the validity of the CG is signaled (coded_sub_block_flag), and as shown Figure 7B As shown, when CG is valid, residual encoding and decoding of the CG occurs.

[0113] Figure 7B The CG level residual codec syntax specified in VVC draft 4 is shown. It signals the previously introduced syntax elements sig_flag, gt1_flag, par_flag, gt3_flag, remainder, abs_level and the symbol data in the considered CG. The figure shows the signaling and parsing of the sig_flag, gt1_flag, par_flag, gt3_flag, remainder and abs_level syntax elements according to VVC draft 4. EP means "equal probability", which means that the relevant binary bits are not arithmetically encoded or decoded, but encoded or decoded in bypass mode. The bypass mode directly writes / parses the bit, which is usually equal to the binary syntax element (binary bit) to be encoded or parsed.

[0114] Furthermore, VVC draft 4 specifies a hierarchical syntax arrangement from the CU level to the residual sub-block (CG) level.

[0115] Fig. 8AThe CU level syntax specified in the example of VVC draft 4 is shown. It includes signaling the codec mode of the considered CU. If the CU is in merge skip mode, cu_skip_flag signals it. If not, cu_pred_mode_flag indicates the value of the variable cuPredMode if the prediction mode of the current CU is coded by intra prediction (cuPredMode=MODE_INTRA) or inter prediction (cuPredMode=MODE_INTER).

[0116] In the case of non-skip and non-intra modes, the next flag indicates that the current CU uses intra block copy (IBC) mode. This syntax is followed by intra or inter prediction data for the current CU. The end of the syntax is dedicated to the signaling of the transform tree associated with the current CU. This starts with the cu_cbf syntax element, which indicates that some non-zero residual data is encoded and decoded for the current CU. Figure 8B The syntax arrangement shown, when this flag is equal to true, signals the transform_tree associated with the current CU.

[0117] Figure 8B The transform_tree syntax structure of an example of VVC draft 4 is shown. It basically includes signaling if a CU contains one or more transform units (TUs). First, if the CU size is higher than the maximum allowed TU size, the CU is divided into 4 sub-transform trees in a quadtree manner. If not, and if no ISP (intra-subpartitioning) or SBT (sub-block transform) is used for the current CU, the current transform tree is not split and contains only one TU, which is achieved by Figure 8C Otherwise, if the CU is intra-frame and the ISP (intra-sub-partitioning) mode is used, the CU is also divided into multiple (actually 2) TUs. Figure 8C The transform unit syntax table is used to signal each of the two TUs continuously. Otherwise, if the CU is inter and SBT (sub-block transform) mode is used, the CU is divided into two TUs according to the CU-level SBT-related decoded syntax. Figure 8C The transform unit syntax table is used to signal each of the two TUs consecutively.

[0118] Figure 8CThe transform unit level syntax arrangement in the example of VVC Draft 4 is shown. It consists of the following parts. First, the tu_cbf_luma, tu_cbf_cb, and tu_cbf_cr flags indicate that non-zero residual data is contained in each transform block in the current TU, corresponding to each component. For each component, if the corresponding cbf flag is true, the transform block level residual_tb syntax table is used to encode and decode the corresponding residual transform block. The transform_unit level syntax also includes some block-related syntax for encoded and decoded transforms and quantization, namely incremental QP information (if present) and the transform type used to encode and decode the TU under consideration.

[0119] Fig. 9A and Fig. 9B The CABAC decoding and encoding processes are shown respectively. CABAC stands for Context-Adaptive Binary Arithmetic Coding (CABAC) and is a form of entropy coding used in HEVC or VVC, for example, because it provides lossless compression with good compression efficiency. Fig. 9A The input to the process consists of a coded bitstream, typically compliant with the HEVC specification or its further developments. At any point in the decoding process, the decoder knows which syntax element to decode next. This is fully described in the standardized bitstream syntax and decoding process. In addition, it knows how the current syntax element to be decoded is binarized (i.e. represented as a sequence of binary symbols called bins, each bin equal to "1" or "0") and how each bin of the bin string is encoded.

[0120] Therefore, the first stage of the CABAC decoding process ( Fig. 9A ) to the left of the bit stream. For each bit, it is known whether it is encoded according to the bypass mode or the normal mode. In the bypass mode, only one bit is read from the bit stream and the obtained value is assigned to the current bit. The advantage of this mode is that it is simple, therefore fast, and does not require a lot of resources. Therefore, it is generally effective for bits with uniform statistical distribution (i.e., equal probability of being '1' or '0').

[0121] On the contrary, if the current bin is not coded in bypass mode, it means that it has been coded in so-called conventional codec (ie by context-based arithmetic coding). This mode is more resource-intensive.

[0122] In this case, the decoding of the considered bin is performed as follows. First, the context for decoding the current bin is obtained. It is given by Fig. 9AThe context modeler is given by . The context objective is to obtain the conditional probability that the current bin has the value “0” given some context prior or information X. Here the prior X is the value of some already decoded syntax element available at both the encoder and decoder side in a synchronized manner when the current bin is decoded.

[0123] Typically, a prior X for decoding a bin is specified in the standard and is chosen because it is statistically correlated with the current bin to be decoded. The benefit of using this contextual information is that it reduces the rate cost of encoding and decoding the bin. This is based on the fact that the conditional entropy of a bin given X is low because the bin and X are correlated. The following relationship is well known in information theory:

[0124] H(bin|X) <H(bin)

[0125] This means that if the bin and X are statistically correlated, then the conditional entropy of the bin knowing X is lower than the entropy of the bin. The context information X is therefore used to obtain the probability that the bin is "0" or "1". Given these conditional probabilities, the conventional decoding engine of FIG. 14 performs arithmetic decoding of the binary-valued bin. Then, knowing the current context information X, the value of the bin is used to update the value of the conditional probability associated with the current bin. This is called Fig. 9A The context model update step on . The context model of each bin is updated as long as the bin is being decoded (or encoded), which allows to gradually refine the context modeling of each bin element. Therefore, the CABAC decoder gradually learns the statistical behavior of each conventionally encoded bin.

[0126] The context modeler and context model update steps are strictly identical operations on the encoder and decoder sides.

[0127] Conventional arithmetic decoding of the current bit or its bypass decoding (depending on how it is encoded) produces a series of decoded bits.

[0128] like Fig. 9A The second stage of CABAC decoding, shown on the right, consists in converting this sequence of binary symbols into higher-level syntax elements. The syntax element may take the form of a flag, in which case it directly takes the value of the currently decoded binary bit. On the other hand, if the binarization of the current syntax element corresponds to a set of multiple binary bits according to the specification of the considered standard, then a Fig. 9A The above is called the conversion step of "binary codeword to syntax element".

[0129] This was carried out with Fig. 9BThe inverse step of the binarization step performed by the encoder shown is therefore performed here by obtaining the values ​​of these syntax elements based on their respective decoded binary versions.

[0130] Figure 1 Encoder 100, Figure 2 The decoder 200 and Figure 3 The system 1000 is suitable for implementing at least one of the following embodiments.

[0131] In current video codec systems (e.g., VVC draft 4), some hard constraints are imposed on the coefficient group coding process, where the sig_flag, gt1_flag, and parity_flag syntax elements on one side and the gt3_flag syntax element on the other side can use a hard-coded maximum number of regular CABAC bits. More precisely, the constraints at the 4×4 codec group (or coefficient group, or CG) level ensure that the CABAC decoder engine per unit area must parse the maximum number of regular bits. However, due to the limited use of regular bit CABAC codec, it may result in non-rate distortion optimal video compression. In fact, when a picture allows a limited number of regular bits, then the overall budget may be largely underutilized when a part of the picture to be coded under consideration contains a valid area coded in skip mode (thus without any residual) and another valid area of ​​the picture will be coded with residual. In this case, the low-level (i.e., 4×4 CG level) constraint on the maximum number of regular bits may result in the total number of regular bits in the picture being well below the acceptable picture level threshold.

[0132] At least one embodiment is directed to handling high-level constraints on the maximum usage of conventional CABAC codec bits during residual coding of blocks and their coefficient groups, such that the high-level constraints are observed and compression efficiency is improved compared to current methods. In other words, the budget for conventionally coded bits is allocated over a larger picture area than the CG (thus covering multiple CGs), and the budget is determined by the average allowed number of conventionally coded bits per unit area. For example, an average of 1.75 conventionally coded bits per sample may be allowed. The budget can then be more efficiently allocated to smaller units. In different embodiments, higher-level constraints are set at the transform block or transform unit or codec unit or codec tree unit or picture level. These embodiments may be implemented as follows: Fig.10 The CABAC optimizer shown is implemented.

[0133] In at least one embodiment, a higher level constraint is set at the transform unit level. In such an embodiment, the number of regular binary bits allowed for encoding / decoding of the entire transform unit is determined based on the average allowed number of regular binary bits per unit area. From the obtained budget of regular binary bits for the TU, the number of regular binary bits allowed for each transform block (TB) in the TU is derived. A transform block is a set of transform coefficients belonging to the same TU and the same color component. Next, given the number of regular binary bits allowed in the transform block, residual encoding or decoding is applied under the constraint of the number of regular binary bits allowed for the entire TB. Therefore, an improved residual encoding and decoding process is proposed here, which takes into account the TB level regular binary bit budget instead of the CG level regular binary bit budget. Several embodiments are proposed for this purpose.

[0134] Fig.10 A CABAC encoding process including a CABAC optimizer according to an embodiment of the present principles is shown. In at least one embodiment, the CABAC optimizer 190 uses a conventional codec for a set of codec groups to process a budget representing the maximum number of bins that are encoded (or to be encoded). The budget is determined, for example, by multiplying the per-sample budget of the bins of the conventional codec by the surface of the data unit under consideration (i.e., the number of samples contained therein). For example, where the budget of the bins of the conventional codec is fixed at the TB level, the allowed number of bins of the conventional codec per sample is multiplied by the number of samples contained in the transform block under consideration. The output of the CABAC optimizer 190 controls the codec in a conventional or bypass codec mode, thereby greatly affecting the efficiency of the codec.

[0135] Fig.11 An example flow chart of a CABAC optimizer used in an encoding process according to an embodiment of the present principles is shown. The flow chart is executed for each new set of codec groups under consideration. Therefore, according to different embodiments, it can occur on each picture, each CTU, each CU, each TU or each TB. In step 191, the processor 301 allocates a budget for the conventional codec determined as described above. In step 192, the processor 301 loops the set of codec groups and checks in step 193 whether the last codec group has been reached. In step 194, for each codec group, the processor processes the codec group with an input budget of the binary bits of the conventional codec. During the processing of the codec group under consideration, the budget of the binary bits of the conventional codec is decreasing, that is, it decreases every time the binary bits are encoded and decoded in the conventional mode. The budget of the binary bits of the conventional codec is returned as an output parameter of the codec group encoding or decoding process.

[0136] Fig.12An example of a modified process for encoding / decoding a codec group using a CABAC optimizer is shown. As described above, the allocation of the number of regular CABAC bins dedicated to residual coding is performed at a higher level than the 4×4 codec group. To this end, the process for encoding / decoding a codec group is modified by adding an input parameter representing the current budget of regular bins when the CG is being encoded or decoded, compared to the conventional function. Thus, the budget is modified by encoding or decoding the current CG, depending on the number of regular bins used to encode / decode the current CG. The input / output parameter representing the budget of regular bins is provided in Fig.12 The process of encoding and decoding the residual data itself at a given CG can be the same as conventionally, except that the number of allowed regular CABAC bins is given by external means. In addition, as Fig.12 In the case of , the budget is decremented by one each time a regular binary bit is encoded / decoded.

[0137] Therefore, at least one embodiment of the present disclosure includes processing the budget of regular bins as an input / output parameter of the coefficient group codec / parse function. Therefore, the budget of regular bins can be determined at a level higher than the CG codec / parse level. Different embodiments propose to determine the budget at different levels: transform block or transform unit or codec unit or codec tree unit or picture level.

[0138] Fig.13A An example embodiment is shown in which the budget for regular bits is determined at the transform block level. In such an embodiment, the budget is determined as a function of two main parameters: the size of the current transform block and a basic budget of regular bits that is fixed for a given unit picture area. Here, the area unit considered is a sample. In the current VVC draft 2, since 32 regular bits are allowed for 4×4 CG, this means that an average of 2 regular bits per component sample is allowed. The proposed transform block level regular bit allocation is based on the average ratio of regular bits per unit area.

[0139] In at least one embodiment, the budget allocated for the transform block under consideration is calculated as the product of the transform block surface and the normal number of allocations per sample point. Next, for each significant coefficient group in the transform block under consideration, the budget is passed to the CG residual encoding / parsing process.

[0140] Fig. 13BAn example embodiment is shown in which the budget of regular bits is determined at the transform block level according to the position of the last significant coefficient. According to this variant embodiment, the number of allowed regular bits for the current transform block is calculated according to the position of the last significant coefficient in the transform block under consideration. The advantage of such an approach is that when allocating regular bits, the energy contained in the transform block under consideration is better fitted. Typically, more bits can be allocated to high energy transform blocks, while fewer regular bits can be allocated to low energy transform blocks.

[0141] Fig.14A An example embodiment is shown in which the budget for regular bins is determined at the transform unit level. As shown, unitary_budget, i.e., the average ratio of regular bins allowed per sample, remains equal to 2. The budget for regular bins at the TU level is determined by multiplying this ratio by the total number of samples contained in the transform unit under consideration, i.e., width×height×3 / 2 in the example of a 4:2:0 color format. This overall budget is then used for the encoding and decoding of all transform blocks contained in the transform unit under consideration. It is passed as an input / output parameter to the residual transform block encoding / parsing process for each color component. In other words, each time a transform block in the transform unit under consideration is encoded / parsed, the budget is reduced by the number of regular bins in the encoding / parsing of the transform blocks that are successively encoded / parsed in the transform unit.

[0142] Fig. 14B An example embodiment is shown in which a budget for regular bits is determined at transform block unit level and allocated between transform blocks according to the relative surfaces between different transform blocks and according to the regular bits used in the transform blocks that have been encoded / parsed in the transform unit under consideration. In this embodiment, the regular bits allocated to the luma TB are two thirds of the overall TU level budget. Next, the Cb TB budget is half of the remaining budget after the luma TB is encoded / parsed. Finally, the Cr TB budget is set to the remaining TU level budget after the first two TBs are encoded / parsed. Furthermore, note that in another embodiment, the budget for the entire transform unit can be shared between transform blocks in a TU based on the knowledge that one or more TBs in the TU are encoded with empty residuals. This can be known, for example, by analyzing the tu_cbf_luma, tu_cbf_cb or tu_cbf_cr flags. Thus, if Fig. 14CAs shown in the modified process shown, the TB level residual encoding / decoding / parsing process is applicable to this variant embodiment. Such an embodiment leads to better encoding / decoding efficiency than when the budget is allocated at a lower level in the hierarchy. In fact, if some CGs or TBs have very low energy, it may adopt a reduced number of bins compared to other CGs or TBs. Therefore, in CGs or TBs that need to encode / parse a larger number of bins, a larger number of regular bins can be used.

[0143] Fig.15A An example embodiment is shown in which a budget for regular bins is allocated at the codec unit level. Its calculation is similar to the previous TU level fixed embodiment, but is based on the sampling rate of the regular bins and the CU size. The calculated regular bin budget is then passed to Fig. 15B FIG. 1 shows a transform tree encoding / parsing process of FIG. 1 , which shows the adaptive encoding / parsing of the transform tree associated with the current CU based on the CU level assigned a normal bit budget. Here, the CU level is basically passed to the encoding / decoding of each transform unit contained in the transform tree under consideration in succession. In addition, the number of normal bits used in the encoding / decoding of each TU is also reduced. Fig. 15C A variant embodiment of the codec tree encoding / parsing process when the budget is fixed at the CU level is shown. Typically, when the current TU is encoded and decoded in the transform tree under consideration, the process updates the overall CU level budget based on the number of regular binary bits used in the TU that has been encoded / parsed. For example, in the case of SBT (sub-block transform) mode in an inter CU, VVC Draft 4 states that the CU is divided into 2 TUs, and one of them has an empty residual. In this case, the embodiment proposes that the entire CU-level regular binary bit budget is dedicated to TUs with non-zero residuals, which makes the coding efficiency for such inter CUs better in SBT mode. In addition, in the case of ISP (intra-frame sub-partitioning), this VVC partitioning mode of intra CUU divides the CU into multiple TUs. In this case, the encoding / parsing of the TU attempts to reuse the unused regular binary bit budget allocated to the previous TU in the same CU. Fig.15D and Fig.15E The transform unit encoding / decoding process is shown, which is applicable to this embodiment when the conventional binary bit budget is fixed at the CU level. They include Fig.14A and Fig. 14B The process is adapted to the respective embodiments of the present invention.

[0144] In at least one embodiment, the budget of regular CABAC bins used in residual codec is fixed at the CTU level. This allows for better utilization of the overall budget of allowed regular bins, thus improving codec efficiency compared to previous embodiments. Typically, bins originally dedicated to a skipped CU can be advantageously used for other non-skipped CUs.

[0145] In at least one embodiment, the budget of regular CABAC bits used in residual coding is fixed at the picture level. This allows even better utilization of the overall budget of allowed regular bits, thus improving codec efficiency compared to previous embodiments. Typically, bits originally dedicated to skipped CUs can be advantageously used for other non-skipped CUs.

[0146] In at least one embodiment, the average ratio of conventional binary bits allowed in a given picture is fixed according to the temporal layer / depth to which the picture under consideration belongs. For example, in a random access codec structure, the temporal structure of a group of pictures conforms to the hierarchical B picture arrangement. In this configuration, pictures are organized in temporal scalable layers. Pictures from higher layers depend on reference pictures from lower temporal layers. In contrast, pictures in lower layers do not depend on any pictures in higher layers. Higher temporal layers are typically encoded and decoded with higher quantization parameters than pictures in lower layers, so the quality is lower. In addition, pictures from lower layers seriously affect the overall codec efficiency of the entire sequence. Therefore, it makes sense to encode these pictures with optimal codec efficiency. According to this embodiment, it is proposed to assign a higher ratio of conventional CABAC binary bits based on samples to pictures from lower layers than to pictures from higher temporal layers.

[0147] Fig.16A and Fig. 16B An example embodiment is shown in which a budget for conventional bins is allocated at a higher level than the CG level, and two types of CG encoding / decoding / parsing processes are used. Fig.16A The first type shown is called "all_bypass" and consists in encoding and decoding all bins associated with the CG in bypass mode. This means that the amplitudes of the transform coefficients in the CG are only encoded and decoded through the abs_level syntax element. Fig. 16BThe second type shown is called "all_regular", which includes encoding and decoding all bins corresponding to the sig_flag, gt1_flag, par_flag and gt3_flag syntax elements in a regular mode. In this embodiment, given a regular bin budget fixed at a level higher than the CG level, the TB encoding and decoding process can switch between the "all_regular" CG encoding and decoding mode and the "all_bypass" CG encoding and decoding mode depending on whether the consideration budget of the regular bin currently considered has been fully used.

[0148] Various implementations include decoding. "Decoding" as used in this application may encompass all or part of the processes performed on a received encoded sequence, for example, to produce a final output suitable for display. In various embodiments, these processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes may also include or alternatively include processes performed by a decoder of the various implementations described in this application, such as Figure 10 to Figure 1 The embodiment presented in 6.

[0149] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or generally to a broader decoding process will be clear based on the context of the particular description and is considered to be well understood by those skilled in the art.

[0150] Various implementations include encoding. In a manner similar to the discussion above regarding "decoding", "encoding" as used in this application can encompass all or part of the processes performed on an input video sequence to produce an encoded bitstream, for example. In various embodiments, these processes include one or more processes typically performed by an encoder, such as segmentation, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also include or alternatively include processes performed by an encoder of the various implementations described in this application, such as, Figure 10 to Figure 1 6. Embodiment of the invention.

[0151] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will be clear based on the context of the particular description and is considered to be well understood by those skilled in the art.

[0152] Note that the syntax elements used herein are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0153] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in detail and, at least to show individual characteristics, are often described in a way that may sound restrictive. However, this is for the purpose of clarity of description and does not limit the application or scope of those aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with aspects described in earlier applications. The aspects described and contemplated in this application can be implemented in many different forms. The above Figure 1 , Figure 2 and Figure 3 Some embodiments are provided, but other embodiments are also contemplated, and the discussion of the figures does not limit the breadth of implementations.

[0154] In this application, the terms "reconstructed" and "decoded" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably. Usually, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0155] Various methods are described herein, and each method includes one or more steps or actions for implementing the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions may be modified or combined.

[0156] Various numerical values ​​are used in this application, such as with respect to block size. The specific values ​​are for example purposes, and the described aspects are not limited to these specific values.

[0157] Reference to "one embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" or "in an implementation" or "in an implementation", as well as any other variations appearing in various places throughout the specification, do not necessarily all refer to the same embodiment.

[0158] Furthermore, this application or its claims may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory.

[0159] Furthermore, this application or its claims may involve "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, predicting information, or estimating information.

[0160] Furthermore, this application or its claims may involve "receiving" various information. Like "accessing," receiving is intended to be a broad term. Receiving information can include, for example, one or more of accessing information or retrieving information (e.g., from memory or optical storage). Furthermore, during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information, "receiving" is generally involved in one way or another.

[0161] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of " / ," any of "and / or," and "at least one of" is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to as many of the listed items as are readily apparent to one of ordinary skill in the art and related arts.

[0162] It will be apparent to those skilled in the art that implementations may generate various signals formatted to carry information that may be stored or transmitted, for example. Information may include, for example, instructions for executing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, a signal may be sent over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

Claims

1. A video encoding method, comprising determining the number of regular binary bits, and performing CABAC encoding based on the determined number of regular binary bits.

2. A video decoding method, comprising determining the number of regular binary bits, and performing CABAC decoding based on the determined number of regular binary bits.

3. A video encoding apparatus, comprising a processor configured to determine the number of regular binary bits and perform CABAC encoding based on the determined number of regular binary bits.

4. A video decoding apparatus, comprising a processor configured to determine the number of regular binary bits and perform CABAC decoding based on the determined number of regular binary bits.