Method and device for probability calculation for context-based adaptive binary arithmetic coding

Abstract

To provide a method, apparatus and non-transitory computer-readable storage medium thereof for video decoding and encoding related to improving the accuracy of a probability estimation module for context-based adaptive binary arithmetic coding (CABAC).SOLUTION: In a method of video decoding, one binary arithmetic decoder obtains multiple hypothesis probabilities of one binary symbol of one given context model of the binary arithmetic decoder according to adaptive weights. The multi-hypothesis probability indicates a probability that a binary symbol is equal to a binary value, wherein the binary symbol is from a plurality of binary symbols associated with the context model. The method further includes decoding, by the decoder, the one binary symbol according to the multiple hypothesis probabilities.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on and claims priority to U.S. Provisional Application No. 63 / 294,692, entitled "Methods and Devices on Probability Calculation for Context-Based Adaptive Binary Arithmetic Coding," filed December 29, 2021, which is incorporated by reference in its entirety for all purposes.

[0002] This disclosure relates to video encoding and compression, and more particularly, but not exclusively, to methods and apparatus for improving the accuracy of probability estimation modules for context-based adaptive binary arithmetic coding (CABAC), an entropy coding method used in modern video codecs. [Background technology]

[0003] Various video coding techniques can be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), High Efficiency Video Coding (H.265 / HEVC), Advanced Video Coding (H.264 / AVC), Moving Picture Expert Group (MPEG) coding, and others. Video coding typically utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit redundancy present in a video image or sequence. An important goal of video coding techniques is to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0004] The first version of the VVC standard was completed in July 2020 and offers approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video coding standard, HEVC. While the VVC standard offers significant coding improvements over previous standards, there is evidence that superior coding efficiency can be achieved using additional coding tools. Recently, the Joint Video Exploration Team (JVET), in collaboration with ITU-T VECG and ISO / IEC MPEG, began exploring advanced technologies that could enable significant improvements in coding efficiency beyond VVC. In April 2021, a single software codebase, called the Enhanced Compression Model (ECM), was established for future video coding exploration work. The ECM reference software is based on the VVC Test Model (VTM) developed by JVET for VVC, with several existing modules (e.g., intra / inter prediction, transform, in-loop filter, etc.) further extended and / or improved. In the future, any new coding tools that go beyond the VVC standard will need to be integrated into the ECM platform and tested using JVET's common test conditions (CTC). Summary of the Invention [Problem to be solved by the invention]

[0005] This disclosure provides examples of techniques for improving the accuracy of the probability estimation module for CABAC. [Means for solving the problem]

[0006] According to a first aspect of the present disclosure, a method for video decoding is provided. The method includes: a binary arithmetic decoder may obtain a multi-hypothesis probability of a binary symbol of a given context model of the binary arithmetic decoder according to an adaptive weight; the multi-hypothesis probability may determine a probability that the binary symbol is equal to a binary value, the binary symbol being from a plurality of binary symbols associated with the context model. Further, the decoder may encode the binary symbol according to the multi-hypothesis probability. Also, the decoder may decode the binary symbol according to the multi-hypothesis probability.

[0007] According to a second aspect of the present disclosure, a method for video encoding is provided, the method including: a binary arithmetic encoder may determine, according to adaptive weights, a multiple hypothesis probability of a binary symbol from a plurality of binary symbols of a given context model of the binary arithmetic encoder, the multiple hypothesis probability indicating a probability that the binary symbol is equal to a binary value; and the encoder may encode the binary symbol according to the multiple hypothesis probability.

[0008] According to a third aspect of the present disclosure, there is provided a method for video decoding, the method including: a decoder may obtain an initial context state of one or more context models of a current slice by selecting one or more slices coded before a current slice, inheriting a context state of the one or more context models of the one slice coded before the current slice, and decoding binary symbols associated with the one or more context models in the current slice according to the initial context state.

[0009] According to a fourth aspect of the present disclosure, there is provided a method for video encoding, the method including: an encoder may determine initial context states of one or more context models of a current slice by selecting one or more slices encoded before a current slice, inheriting context states of the one or more context models from the one slice encoded before the current slice, and encoding binary symbols associated with the one or more context models of the current slice according to the initial context states.

[0010] According to a fifth aspect of the present disclosure, there is provided an apparatus for video decoding, the apparatus including: one or more processors; and a memory, coupled to the one or more processors, configured to store instructions executable by the one or more processors, wherein the one or more processors, upon execution of the instructions, are configured to perform a method according to the first or third aspect.

[0011] According to a sixth aspect of the present disclosure, there is provided an apparatus for video encoding, the apparatus including: one or more processors; and a memory, coupled to the one or more processors, configured to store instructions executable by the one or more processors, wherein the one or more processors, upon execution of the instructions, are configured to perform a method according to the second or fourth aspect.

[0012] According to a seventh aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to the first or third aspect based on the bitstream.

[0013] According to an eighth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a method according to the second or fourth aspect, encoding a current video block into a bitstream, and transmitting the bitstream.

[0014] A more particular description of examples of the present disclosure will be made with reference to specific examples that are illustrated in the accompanying drawings, in which the examples will be described and explained with additional specificity and detail, considering that these drawings illustrate only some examples and therefore are not to be considered limiting in scope. [Brief explanation of the drawings]

[0015] [Figure 1A] FIG. 1 is a block diagram illustrating a system for encoding and decoding video blocks according to some examples of the present disclosure. [Figure 1B] FIG. 2 is a block diagram of an encoder according to some examples of this disclosure. [Figures 1C-1F] FIG. 10 is a block diagram illustrating how a frame is recursively divided into multiple video blocks of different sizes and shapes, in accordance with some examples of this disclosure. [Figure 2] FIG. 2 is a block diagram of a decoder according to some examples of this disclosure. [Figure 3A] FIG. 10 illustrates block division in a multi-type tree structure according to some examples of the present disclosure. [Figure 3B] FIG. 10 illustrates block division in a multi-type tree structure according to some examples of the present disclosure. [Figure 3C] FIG. 10 illustrates block division in a multi-type tree structure according to some examples of the present disclosure. [Figure 3D] FIG. 10 illustrates block division in a multi-type tree structure according to some examples of the present disclosure. [Figure 3E]FIG. 10 illustrates block division in a multi-type tree structure according to some examples of the present disclosure. [Figure 4] FIG. 1 illustrates a computing environment coupled with a user interface according to some examples of the present disclosure. [Figure 5] 1 is a flowchart illustrating a method for video decoding according to some examples of this disclosure. [Figure 6] 6 is a flowchart illustrating a method for video encoding corresponding to the method for video decoding shown in FIG. 5 according to some examples of this disclosure. [Figure 7] 1 is a flowchart illustrating a method for video decoding according to some examples of this disclosure. [Figure 8] 8 is a flowchart illustrating a method for video encoding corresponding to the method for video decoding shown in FIG. 7 according to some examples of this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0016] Reference will now be made in detail to particular implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented in many types of electronic devices having digital video capabilities.

[0017] The terms used in this disclosure are employed only for the purpose of describing particular embodiments and are not intended to limit the disclosure. In this disclosure and the appended claims, the singular forms "a / an," "said," and "the" are intended to include the plural forms unless otherwise clearly indicated throughout this disclosure. The term "and / or," as used in this disclosure, should also be understood to refer to and include one or any or all possible combinations of the associated listed items.

[0018] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that a particular feature, structure, or characteristic described is included in at least one embodiment or example. A feature, structure, element, or characteristic described in connection with one or more embodiments may also be applicable to other embodiments, unless stated otherwise.

[0019] Throughout this disclosure, all terms such as "first," "second," "third," etc., do not imply any spatial or chronological order unless otherwise specified, but are used solely as nomenclature to refer to related elements, e.g., devices, components, compositions, steps, etc. For example, a "first device" and a "second device" may refer to two separately formed devices or two portions, components, or operating states of the same device, and may be arbitrarily named.

[0020] The terms "module," "sub-module," "circuit," "sub-circuit," "circuitry," "sub-circuitry," "unit," or "sub-unit" may include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits that may or may not contain stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached or located adjacent to each other.

[0021] As used herein, the terms "if" or "when" may be understood to mean "upon" or "in response to," depending on the context. When these terms appear in a claim, they may not indicate that the associated limitation or feature is conditional or optional. For example, a method may include: i) a function or action X' is performed when or if condition X exists; and ii) a function or action Y' is performed when or if condition Y exists. The method may be implemented with both the ability to perform function or action X' and the ability to perform function or action Y'. Thus, functions X' and Y' may both be performed at different times during multiple executions of the method.

[0022] A unit or module may be implemented purely by software, purely by hardware, or by a combination of hardware and software. In a purely software implementation, for example, a unit or module may include functionally related code blocks or software components that are directly or indirectly linked together to perform a particular function.

[0023] 1A is a block diagram illustrating an example system 10 for encoding and decoding video blocks in parallel, according to some implementations of the present disclosure. As shown in FIG. 1A, system 10 includes a source device 12 that generates and encodes video data to be subsequently decoded by a destination device 14. Source device 12 and destination device 14 may include any of a wide variety of electronic devices, including desktop or laptop computers, tablet computers, smartphones, set-top boxes, digital televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some implementations, source device 12 and destination device 14 are equipped with wireless communication capabilities.

[0024] In some implementations, destination device 14 may receive encoded video data to be decoded via link 16. Link 16 may include any type of communication medium or device capable of moving encoded video data from source device 12 to destination device 14. In one example, link 16 may include a communication medium to enable source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium may include any wireless or wired communication medium, such as, for example, the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from source device 12 to destination device 14.

[0025] In some other implementations, the encoded video data may be transmitted from output interface 22 to storage device 32. The encoded video data in storage device 32 may then be accessed by destination device 14 via input interface 28. Storage device 32 may include any of a variety of distributed or locally accessible data storage media, such as a hard drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In a further example, storage device 32 may correspond to a file server or other intermediate storage device that may hold the encoded video data generated by source device 12. Destination device 14 may access the stored video data from storage device 32 by streaming or downloading. The file server may be any type of computer capable of storing encoded video data and transmitting the encoded video data to destination device 14. Exemplary file servers include web servers (e.g., for websites), file transfer protocol (FTP) servers, network-attached storage (NAS) devices, or local disk drives. Destination device 14 may access the encoded video data over any standard data connection suitable for accessing the encoded video data stored on the file server, including a wireless channel (e.g., a Wireless Fidelity (Wi-Fi) connection), a wired connection (e.g., a Digital Subscriber Line (DSL), a cable modem, etc.), or a combination of both. Transmission of the encoded video data from storage device 32 may be a streaming transmission, a download transmission, or a combination of both.

[0026] As shown in FIG. 1A , source device 12 includes video source 18, video encoder 20, and output interface 22. Video source 18 may include, for example, a video capture device such as a video camera, a video archive containing captured video, a video feed interface for receiving video from a video content provider, and / or a computer graphics system for generating computer graphics data as source video, or a combination of such sources. As an example, if video source 18 is a video camera in a security surveillance system, source device 12 and destination device 14 may form a camera phone or video phone. However, implementations described herein may be applicable to video encoding generally and may be applied to wireless and / or wired applications.

[0027] Captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video data may be transmitted directly to destination device 14 via output interface 22 of source device 12. The encoded video data may also (or alternatively) be stored in storage device 32 for later access by destination device 14 or other devices for decoding and / or playback. Output interface 22 may further include a modem and / or a transmitter.

[0028] Destination device 14 includes an input interface 28, a video decoder 30, and a display device 34. Input interface 28 may include a receiver and / or modem and receive encoded video data over link 16. The encoded video data transmitted over link 16 or provided on storage device 32 may include various syntax elements generated by video encoder 20 for use by video decoder 30 in decoding the video data. Such syntax elements may be included within the encoded video data transmitted over a communications medium, stored on a storage medium, or stored on a file server.

[0029] In some implementations, destination device 14 may include display device 34, which may be an integrated display device and an external display device configured to communicate with destination device 14. Display device 34 displays the decoded video data to a user and may include any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0030] Video encoder 20 and video decoder 30 may operate in accordance with proprietary or industry standards, e.g., VVC, HEVC, MPEG-4 Part 10, AVC, or extensions to such standards. It should be understood that the present application is not limited to a particular video encoding / decoding standard and may be applicable to other video encoding / decoding standards. It is generally contemplated that video encoder 20 of source device 12 may be configured to encode video data in accordance with any of these current or future standards. Similarly, it is generally contemplated that video decoder 30 of destination device 14 may be configured to decode video data in accordance with any of these current or future standards.

[0031] Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. If implemented partially in software, an electronic device may store software instructions on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the video encoding / decoding operations disclosed in this disclosure. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, any of which may be integrated into the respective device as part of a combined encoder / decoder (CODEC).

[0032] Like HEVC, VVC is built on a block-based hybrid video coding framework. Figure 1B is a block diagram illustrating a block-based video encoder according to some implementations of this disclosure. In encoder 100, an input video signal is processed by blocks called coding units (CUs). Encoder 100 may be the video encoder 20 shown in Figure 1A. In VTM-1.0, a CU can be up to 128x128 pixels. However, unlike HEVC, which divides blocks based only on a quadtree, VVC divides a single coding tree unit (CTU) into CUs based on a quadtree, binary tree, or ternary tree to adapt to various local characteristics. Furthermore, the concept of multiple division unit types in HEVC has been eliminated. That is, the separation of CUs, prediction units (PUs), and transform units (TUs) no longer exists in VVC. Instead, each CU is always used as the basic unit for both prediction and transformation without further division. In the multi-type tree structure, a CTU is first divided into a quadtree structure, and then each quadtree leaf node can be further divided into a binary tree structure and a ternary tree structure.

[0033] 3A-3E are schematic diagrams illustrating multi-type tree partitioning modes according to some implementations of the present disclosure, each showing five partitioning types, including 4-way partitioning (FIG. 3A), 2-way vertical partitioning (FIG. 3B), 2-way horizontal partitioning (FIG. 3C), 3-way vertical partitioning (FIG. 3D), and 3-way horizontal partitioning (FIG. 3E).

[0034] For each given video block, spatial prediction and / or temporal prediction may be performed. Spatial prediction (or "intra prediction") predicts the current video block using pixels from samples of previously coded neighboring blocks (called reference samples) within the same video picture / slice. Spatial prediction reduces spatial redundancy inherent in the video signal. Temporal prediction (also called "inter prediction" or "motion-compensated prediction") predicts the current video block using reconstructed pixels from previously coded video pictures. Temporal prediction reduces temporal redundancy inherent in the video signal. The temporal prediction signal for a given CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal references. If multiple reference pictures are supported, an additional reference picture index is sent, which identifies which reference picture in the reference picture store the temporal prediction signal belongs to.

[0035] After spatial prediction and / or temporal prediction, an intra / inter mode decision circuit 121 in encoder 100 selects the best prediction mode, for example, based on a rate-distortion optimization method. A block predictor 120 is then subtracted from the current video block, and the resulting prediction residual is decorrelated using a transform circuit 102 and a quantization circuit 104. The resulting quantized residual coefficients are inverse quantized by an inverse quantization circuit 116 and inverse transformed by an inverse transform circuit 118 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Further, in-loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO), and / or an adaptive in-loop filter (ALF), may be applied to the reconstructed CU, which is then placed in a reference picture store in a picture buffer 117 and used to encode future video blocks. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit 106 for further compression and packing to form the bitstream.

[0036] For example, a deblocking filter is available in the latest versions of AVC, HEVC, and VVC. HEVC defines an additional in-loop filter called SAO to further improve coding efficiency. Another in-loop filter, called ALF, is being actively researched for the latest version of the VVC standard and may well be included in the final standard.

[0037] These in-loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. They can also be turned off as a decision made by encoder 100 to reduce computational complexity.

[0038] Note that intra prediction is typically based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels if these filter options are turned on by the encoder 100.

[0039] FIG. 2 is a block diagram illustrating a block-based video decoder 200 that can be used in conjunction with many video coding standards. This decoder 200 is similar to the reconstruction-related portion present in the encoder 100 of FIG. 1B. The block-based video decoder 200 may be the video decoder 30 shown in FIG. 1A. In the decoder 200, an incoming video bitstream 201 is first decoded by entropy decoding 202 to derive quantized coefficient levels and prediction-related information. The quantized coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain a reconstructed prediction residual. A block prediction mechanism implemented in an intra / inter mode selector 212 is configured to perform either intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residual from the inverse transform 206 with the prediction output generated by the block prediction mechanism using a summer 214.

[0040] The reconstructed blocks may further pass through an in-loop filter 209 and then be stored in a picture buffer 213, which serves as a reference picture store. The reconstructed video in the picture buffer 213 can be sent to drive a display device and can also be used to predict future video blocks. In situations where the in-loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels to derive the final reconstructed video output 222.

[0041] In the current VVC and AVS3 standards, the motion information of the current coding block is either copied from spatially or temporally neighboring blocks specified by a merge candidate index or obtained by explicit signaling of motion estimation. The focus of this disclosure is to improve the accuracy of motion vectors in affine merge mode by improving the method for deriving affine merge candidates. To facilitate the explanation of this disclosure, the existing affine merge mode design in the VVC standard is used as an example to explain the proposed idea. While the existing affine mode design in the VVC standard is used as an example throughout this disclosure, those skilled in the art of modern video coding technology should note that the proposed technology may also be applied to affine motion prediction modes with different designs or other coding tools with the same or similar design concepts.

[0042] In a typical video coding process, a video sequence typically contains an ordered set of frames or pictures. Each frame may contain three sample arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luma samples. SCb is a two-dimensional array of Cb chroma samples. SCr is a two-dimensional array of Cr chroma samples. In other examples, a frame may be monochrome and therefore contain only one two-dimensional array of luma samples.

[0043] As shown in FIG. 1C, video encoder 20 (or more specifically, a partitioning unit within a prediction processing unit of video encoder 20) generates a coded representation of a frame by first partitioning the frame into a set of CTUs. A video frame may include an integer number of CTUs arranged consecutively from left to right and top to bottom in raster scan order. Each CTU is the largest logical coding unit, and the width and height of the CTU are signaled by video encoder 20 in the sequence parameter set so that all CTUs in a video sequence have the same size, which may be one of 128x128, 64x64, 32x32, and 16x16. However, note that the present application is not necessarily limited to a particular size. As shown in FIG. 1D, each CTU may include one CTB for luma samples, two corresponding coding tree blocks for chroma samples, and syntax elements used to encode the coding tree block samples. The syntax elements describe the properties of various types of units of coded blocks of pixels and how a video sequence may be reconstructed at video decoder 30, including inter- or intra-prediction, intra-prediction mode, motion vectors, and other parameters. For monochrome pictures or pictures with three distinct color planes, a CTU may contain a single coding tree block and the syntax elements used to encode the samples of that coding tree block. A coding tree block may be an NxN block of samples.

[0044] To achieve better performance, video encoder 20 may recursively perform tree partitioning, such as binary tree partitioning, ternary tree partitioning, quad tree partitioning, or a combination thereof, on the coding tree blocks of a CTU to divide the CTU into smaller CUs. As shown in FIG. 1E, 64×64 CTU 400 is first partitioned into four smaller CUs, each with a 32×32 block size. Of the four smaller CUs, CU 410 and CU 420 are each partitioned into four CUs with a 16×16 block size. Two 16×16 CUs, 430 and 440, are each further partitioned into four CUs with an 8×8 block size. FIG. 1F shows a quad tree data structure illustrating the final result of the partitioning process for CTU 400 shown in FIG. 1E, where each leaf node of the quad tree corresponds to one CU of a respective size ranging from 32×32 to 8×8. Similar to the CTU shown in Figure 1D, each CU may contain two corresponding coding blocks of luma samples and chroma samples of the same size frame, along with the syntax elements used to encode the samples of the coding block. In monochrome pictures or pictures with three distinct color planes, a CU may contain a single coding block and the syntax structure used to encode the samples of that coding block. Note that the quadtree partitioning shown in Figures 1E-1F is for illustrative purposes only; a CTU can be divided into CUs based on quadtree, ternary, or binary tree partitioning to accommodate various local characteristics. In a multi-type tree structure, a CTU is divided by a quadtree structure, and each quadtree leaf CU can be further divided by binary and ternary tree structures. As shown in Figures 3A-3E, there are five possible partition types for a coding block with width W and height H: quad, horizontally bipartitioned, vertically bipartitioned, horizontally tripartitioned, and vertically tripartitioned.

[0045] In some implementations, video encoder 20 may further divide a coding block of a CU into one or more M×N PBs. A PB is a rectangular (square or non-square) block of samples to which the same inter or intra prediction is applied. A PU of a CU may include a PB of luma samples, two corresponding PBs of chroma samples, and syntax elements used to predict the PBs. In monochrome pictures or pictures with three separate color planes, a PU may include a single PB and syntax structures used to predict that PB. Video encoder 20 may generate predictive luma, Cb, and Cr blocks for the luma, Cb, and Cr PBs of each PU of a CU.

[0046] Video encoder 20 may use intra prediction or inter prediction to generate the predictive blocks of a PU. If video encoder 20 uses intra prediction to generate the predictive blocks of a PU, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of a frame associated with the PU. If video encoder 20 uses inter prediction to generate the predictive blocks of the PU, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of one or more frames other than the frame associated with the PU.

[0047] After video encoder 20 generates predictive luma, Cb, and Cr blocks for one or more PUs of a CU, video encoder 20 may generate a luma residual block for the CU by subtracting the predictive luma block of the CU from its original luma coding block, where each sample in the luma residual block of the CU indicates a difference between a luma sample in one of the predictive luma blocks of the CU and a corresponding sample in the original luma coding block of the CU. Similarly, video encoder 20 may generate a Cb residual block and a Cr residual block for the CU, where each sample in the Cb residual block of the CU indicates a difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb coding block of the CU, and where each sample in the Cr residual block of the CU indicates a difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr coding block of the CU.

[0048] Further, as shown in FIG. 1E, video encoder 20 may use quadtree partitioning to decompose the luma, Cb, and Cr residual blocks of a CU into one or more luma, Cb, and Cr transform blocks, respectively. A transform block is a rectangular (square or non-square) block of samples to which the same transform is applied. A TU of a CU may include a transform block of luma samples and two corresponding transform blocks of chroma samples, as well as syntax elements used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. In some examples, the luma transform block associated with a TU may be a sub-block of the luma residual block of the CU. The Cb transform block may be a sub-block of the Cb residual block of the CU. The Cr transform block may be a sub-block of the Cr residual block of the CU. In monochrome pictures or pictures with three separate color planes, a TU may include a single transform block and syntax structures used to transform the samples of that transform block.

[0049] Video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block of the TU. The coefficient block may be a two-dimensional array of transform coefficients. The transform coefficients may be scalar quantities. Video encoder 20 may apply one or more transforms to a Cb transform block of the TU to generate a Cb coefficient block of the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of the TU to generate a Cr coefficient block of the TU.

[0050] After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to a process by which transform coefficients are quantized to minimize the amount of data used to represent the transform coefficients, thereby achieving further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform CABAC on the syntax elements indicating the quantized transform coefficients. Finally, video encoder 20 may output a bitstream including a sequence of bits forming a representation of the coded frame and associated data, which may be stored in storage device 32 or transmitted to destination device 14.

[0051] After receiving the bitstream generated by video encoder 20, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct frames of video data based at least in part on the syntax elements obtained from the bitstream. The process of reconstructing video data is generally reciprocal to the encoding process performed by video encoder 20. For example, video decoder 30 may perform an inverse transform on coefficient blocks associated with TUs of a current CU to reconstruct residual blocks associated with the TUs of the current CU. Video decoder 30 also reconstructs coding blocks of the current CU by adding samples of predictive blocks of PUs of the current CU to corresponding samples of transform blocks of TUs of the current CU. After reconstructing the coding blocks for each CU of the frame, video decoder 30 may reconstruct the frame.

[0052] As mentioned above, video coding achieves video compression using two main modes: intra-frame prediction (or intra-prediction) and inter-frame prediction (or inter-prediction). Note that IBC can be considered as either intra-frame prediction or a third mode. Between the two modes, inter-frame prediction contributes more to coding efficiency than intra-frame prediction because it uses motion vectors to predict the current video block from a reference video block.

[0053] However, as video data capture technology advances and video block sizes become finer to preserve video data details, the amount of data required to represent the motion vectors of the current frame also increases significantly. One way to overcome this challenge is to benefit from the fact that a group of neighboring CUs in both the spatial and temporal domains not only have similar video data for prediction, but also similar motion vectors between these neighboring CUs. Therefore, the motion information of spatially neighboring CUs and / or temporally co-located CUs can be used as an approximation of the motion information (e.g., motion vector) of the current CU by examining their spatial and temporal correlations, which is also called the "motion vector predictor (MVP)" of the current CU.

[0054] As described above in connection with FIG. 1B, instead of encoding the actual motion vector of the current CU determined by the motion estimation unit into the video bitstream, the motion vector predictor of the current CU is subtracted from the actual motion vector of the current CU to generate a motion vector difference (MVD) for the current CU. In this way, the motion vector determined by the motion estimation unit for each CU of a frame does not need to be encoded into the video bitstream, and the amount of data used to represent motion information in the video bitstream can be significantly reduced.

[0055] Similar to the process of selecting a predictive block in a reference frame during inter-frame prediction of a codeblock, a motion vector candidate list (also known as a "merge list") for the current CU must be constructed using potential candidate motion vectors associated with CUs spatially neighboring and / or temporally co-located CUs of the current CU, and then a set of rules for selecting one member from the motion vector candidate list as the motion vector predictor for the current CU must be chosen by both the video encoder 20 and the video decoder 30. In this way, the motion vector candidate list itself does not need to be transmitted from the video encoder 20 to the video decoder 30; the index of the selected motion vector predictor in the motion vector candidate list is sufficient for the video encoder 20 and the video decoder 30 to encode and decode the current CU using the same motion vector predictor in the motion vector candidate list.

[0056] The primary objective of this disclosure is to improve the efficiency of CABAC techniques used in hybrid video coding frameworks. Specifically, several improvements are proposed to increase the accuracy of probability estimation for sequences of binary symbols (also known as bins for short) of syntax elements generated when compressing video signals. Below, a detailed analysis of existing CABAC techniques applied in modern video coding standards such as AVC, HEVC, and VVC is first provided. Then, several deficiencies / limitations of existing CABAC designs are discussed. Afterwards, a method for improving CABAC efficiency through increased accuracy of probability estimation is proposed.

[0057] Probability Estimation Techniques for CABAC in AVC and HEVC CABAC was originally introduced in the H.264 / AVC standard as one of two supported entropy coding schemes. In CABAC, arithmetic coding consists of two modules: codeword mapping (also known as binarization) and probability estimation. In the codeword mapping process, syntax elements are mapped to sequences of bins. This mapping is achieved by a binarizer, which converts syntax elements into groups of bins based on various binarization schemes. In practice, various binarization schemes, such as fixed-length codes, unary codes, truncated unary codes, and k-th order exponential-Golomb codes, can be applied to such conversions. The purpose of the probability estimation module is to determine the likelihood that a bin has a value of 1 or 0. In AVC, bin probabilities are calculated based on an exponential aging model, where the probability that a current bin is equal to 1 or 0 depends on the values ​​of previous coded bins. Furthermore, general data statistics show that the influence of bins immediately preceding a current bin is usually greater than that of bins coded much earlier. Considering this, CABAC introduces one parameter α to control the number N of coded bins used to estimate the probability of the current bin, i.e., N=1 / α. This parameter translates into an adaptation speed at which the probability is updated with the increasing number of coded bins. Specifically, using the adaptation parameter α, the probability that a bin is the least probable symbol (LPS) is calculated recursively as follows: p(t+1)=p(t)·(1-α)+x(t)·α (1) where p(t) is the probability of the LPS symbol at instant t, p(t+1) is the updated probability of the LPS symbol at instant t+1, and x(t) is equal to 1 if the current bin is an LPS symbol and equal to 0 if the current bin is a most probable symbol (MPS). In the CABAC engines of AVC and HEVC, this probability is updated independently for each syntax element with a fixed value of α≈1 / 19.69 according to (1), i.e., approximately 19.69 coded bins are considered when estimating the probability of one current bin. Furthermore, to avoid multiplication during probability estimation, the probability p(t) in equation (1), which is real and ranges from 0 to 1, is quantized to a set of fixed probability states. For example, in both AVC and HEVC, the probability has a precision of 7 bits, which corresponds to 128 probability states.

[0058] In AVC and HEVC, a video bitstream typically consists of one or more independently decodable slices. At the beginning of each slice, the probabilities of all contexts are initialized to some predefined value. Theoretically, if the statistical properties of one given context are known, then it is possible to estimate the probability distribution of the contexts from a uniform distribution (i.e., p init = 0.5) to initialize the probabilities of the contexts. However, it has been found to be beneficial to provide each context with some appropriate initial probability values ​​(which may not be equal probabilities) to make the probability of one context quickly catch up to its corresponding statistical distribution. Specifically, in AVC and HEVC, given the initial QP of one slice, SliceQPY, the initial probability state of one context, InitProbState, is calculated as follows: m=SlopeIdx·5-45 n=(OffsetIdx<<3)-16 InitProbState=Clip3(1,127,(m·SliceQP Y )>>4+n) (2) Here, SlopeIdx and OffsetIdx (both in the range of 0 to 15) are two initialization parameters, which are predefined and stored as a look-up table (LUT) to calculate the initial probability of one context. As shown in Equation (2), the initial probability state is modeled by a linear function of slice QP with slope equal to (m>>4) and offset equal to n.

[0059] CABAC Probability Estimation Technique in VVC The probability estimation module applied in VVC is almost the same as that in AVC and HEVC, except for the following important differences: first, VVC maintains two probability estimates per context, each with its own probability adaptation rate α in Equation (1). The final probability actually used for arithmetic coding is the average of the two estimates. Second, in VVC, multiple probability LUTs are predefined and used to initialize the probabilities of different contexts in a slice. Meanwhile, similar to AVC and HEVC, the initial probability estimates are constructed based on a linear model that takes the slice QP as input. However, in VVC, the derived values ​​represent actual probability values, while in AVC / HEVC, they represent probability state indices.

[0060] Multiple Hypothesis Probability Estimation It is clear that using one fixed adaptation parameter for all syntax elements may not be optimal due to their different statistical properties. On the other hand, some scientific studies have shown that using multiple probability estimators can achieve higher estimation accuracy compared to a single estimator. Therefore, one multi-hypothesis probability estimation scheme has been applied in the CABAC design of VVC, in which two different adaptation parameters α0 and α1, corresponding to low and high rates of probability adaptation, are utilized. In this way, using two adaptation parameters, two different probabilities can be calculated for each bin, which are then averaged to generate the final probability for the bin, i.e., p0(t+1)=p0(t)·(1-α0)+x(t)·α0 p1(t+1)=p1(t)·(1-α1)+x(t)·α1 p(t+1)=(p0(t+1)+p1(t+1)) / 2 (3) where α0 and α1 are two adaptive parameters associated with two probability hypotheses. In VVC, the values ​​of α0 and α1 are selected independently for each context using a single training algorithm designed to jointly optimize the adaptive parameters and initial probabilities. Specifically, the current design allows each context to select α0 from one predefined set of values ​​{1 / 4, 1 / 8, 1 / 16, 1 / 32} and α1 from another predefined set of values ​​{1 / 32, 1 / 64, 1 / 128, 1 / 256, 1 / 512}.

[0061] Initial probability calculation Similar to AVC / HEVC, the CABAC process in VVC also invokes one QP-dependent probability initialization process at the beginning of each slice. However, compared to AVC / HEVC, which initializes the states of one probability state machine, the actual values ​​of the initial probabilities are directly derived as follows:

number

number

number

[0062] Problem statement Compared with the CABAC design of AVC / HEVC, the probability estimation scheme of VVC can capture the true statistical distribution of each context bin more accurately, leading to improved CABAC efficiency. However, the design can still be further improved. Specifically, this disclosure identifies the following deficiencies that exist in the current probability estimation of the CABAC process of VVC:

[0063] First, as discussed above, VVC applies a multi-hypothesis probability estimation scheme, in which two probability estimators (one with a fast adaptation rate and the other with a slow adaptation rate) estimate the probability of each bin in each context model. Furthermore, in existing designs, the probability of a bin is simply the average of the two probability estimators. Considering that the fixed weights may not be flexible enough to adapt to various data statistics in different contexts, such a design is suboptimal.

[0064] On the other hand, in existing VVC designs, the probabilities of all contexts within a slice are initialized based on three sets of initial context values ​​predetermined for different slice types (i.e., I, B, and P slices). Among these, the set of initial context values ​​for I slices is allowed to be used only for I slices, while the set of initial context values ​​for B and P slices is allowed to be used for either B or P slices. Due to their inherent characteristics, the bins of each video bitstream typically exhibit very different statistical properties. Therefore, using only three fixed sets of initial context values ​​seems far from optimal for providing an efficient starting point for a probability estimator to quickly capture the true probability distribution of each context. On the other hand, due to strong temporal correlation within a video sequence, the probability statistics of contexts from slices coded before the current slice may provide more accurate estimates for initializing the probabilities of contexts within the current slice.

[0065] This disclosure proposes a method to solve the problems / deficiencies of existing probability estimation schemes in VVC. Specifically, the following method is proposed to further improve the accuracy of probability estimation while considering compatibility with hardware codec implementation.

[0066] First, to improve the accuracy of probability estimation, a binary arithmetic coding method with weighted multiple hypothesis probability updates is proposed. Specifically, instead of using a simple average, the final probability used to encode one bin of each context is calculated as a weighted combination of the two probability estimators p0 and p1 associated with that context. Furthermore, several initialization methods are proposed to indicate the initial weight parameters of the contexts at the start of a slice.

[0067] Second, an improved initialization scheme is proposed for initializing the state parameters of the context of an inter-coded slice. Specifically, in addition to using an existing fixed context initialization table, the proposed scheme enables the state parameters of the context in one inter-coded slice (e.g., two probability estimators, an adaptation rate, and a weight coefficient for the combination of the two probability estimators) to be initialized to be copied from the corresponding state parameters of a coded slice.

[0068] Multihypothesis probability estimation using adaptive weights In VVC, multi-hypothesis probability estimation is applied, and the final probability for encoding each bin of a context is calculated as the average of two probability estimators. Considering the inherent statistical properties of different video bitstreams, it is clear that such a scheme (i.e., using equal weights (i.e., 0.5)) may not always be flexible enough to capture the true symbol statistics when combining the two probability estimators of a context. Therefore, this section proposes multi-hypothesis probability estimation with adaptive weights (MHP-AW) to further improve the probability estimation accuracy of VVC. Specifically, similar to the CABAC design for VVC, two separate probability estimators p0 and p1 are maintained for each context and updated based on their unique adaptation rates α0 and α1. However, instead of using a fixed average, the proposed scheme introduces multiple weight parameters, and the final probability p used for binary arithmetic coding of a context is derived based on a weighted combination of the two probability estimators. In detail, the proposed probability estimation can be formulated as follows: p0(t+1)=p0(t)·(1-α0)+x(t)·α0 p1(t+1)=p1(t)·(1-α1)+x(t)·α1 p(t+1)=(1-ω)·p0(t)+ω·p1(t) (5) where ω is the weight used to combine the two probability estimates, and its value is taken from the range [0,1]. In equation (5), the weight ω represents a real value, which needs to be quantized to an integer for hardware / software codec implementation. In practice, various methods can be applied to convert the value of ω to an integer. For example, to approximate the actual weight value, a quantization step q step A uniform quantizer using ω=ω int ·q step(6) where ω int is an integer weight value. Furthermore, the quantization step, which is also a real value, can be approximated as a single right shift operation of M bits as follows: ω=ω int ·q step =ω int >>M (7)

[0069] As shown in Equation (7), the hardware / software implementation of the proposed MHP-AW scheme requires an integer weight value ω int , additional memory is required to store the set of integer weights. Meanwhile, as shown in Equation (5), the precision of the integer weights (i.e., M) also determines the bit width of the multiplier required for the weighted combination of the two probability estimators. Therefore, in practice, different sets of integer weight values ​​and representation precisions may be applied to achieve various trade-offs between coding efficiency and hardware / software implementation complexity. For example, assuming the representation precision M is equal to 5, different sets of integer values ​​may be applied. In one example, it is proposed to select the optimal weight for each context from one predefined set {0, 3, 6, 10, 13, 16, 19, 22, 26, 29, 32}. In another example, it is proposed to set the weights from one predefined set {0, 6, 11, 16, 21, 26, 32}. In yet another example, it is proposed to use a predefined set of integer weight values ​​{0, 8, 12, 16, 20, 24, 32}.

[0070] Similar to the probability and adaptation rate, in the proposed MHP-AW scheme, the weight ω intOne initial value of MHP-AW weights needs to be provided for each context at the beginning of a slice. Various schemes for initializing MHP-AW weights are proposed below. The first method proposes defining multiple different predefined tables, each containing a set of weight initialization values ​​for all contexts in a slice. Before encoding / decoding a slice, one predefined table may be selected, and the corresponding MHP-AW weights are initialized based on the corresponding weight values ​​in the table. For example, in one embodiment, several slice-type-dependent initial weight tables may be derived, e.g., three sets of weight initialization tables specifically designed for I, P, and B slices. In this way, for a slice, the video encoder may select one of the three predefined tables to initialize the MHP-AW weights to better adapt to the symbol statistics within the slice. When such a scheme is applied, one additional syntax element, sh_cabac_weight_init_idx, may be signaled for each slice, indicating which initial weight table is selected for that slice, as shown in Table 1 below.

[0071] [Table 1] Here, the syntax pps_cabac_weight_init_present_flag is a control flag signaled in the picture parameter set (PPS) that indicates whether selecting a different initial weight table per slice is permitted. When this flag is enabled, another syntax sh_cabac_weight_init_idx is further signaled at the slice level to indicate the selected initial weight table. In another embodiment, it is proposed that the MHP-AW weights of an I slice are only allowed to be initialized by the initial weight table associated with the I slice type, and the MHP-AW weights of P and B slices are only allowed to be initialized from one of the predefined weight initialization tables. Specifically, it is proposed that, in addition to the initial weight table associated with the slice type of the current slice, one P (or B) slice is only allowed to be initialized with the initial weight table of the B (or P) slice type. Correspondingly, in such a case, only one flag needs to be signaled per slice for the initial table selection of the MHP-AW weights of the P / B slices, as shown in Table 2 below.

[0072] [Table 2] If the flag sh_cabac_weight_init_flag is equal to 0, it means that the initial weight table corresponding to the slice type of the current slice is used to initialize the MHP-AW weight value of that slice; if the flag is equal to 1, it means that if the current slice is a B slice, the initial weight table corresponding to the P slice type is used to initialize the MHP-AW weight value of that slice; if the current slice is a P slice, the initial weight table corresponding to the B slice type is used to initialize the MHP-AW weight value of that slice.

[0073] Furthermore, in another embodiment of the present disclosure, it is proposed to extend the existing CABAC initialization table to include MHP-AW weights corresponding to each context. Specifically, after such modification, each element of one CABAC initialization table includes three different categories of information: 1) initial probability values; 2) adaptation rates used to initialize the probabilities and adaptation speeds of two probability hypotheses; and 3) MHP-AW weights used to combine the two hypotheses when updating the probability of each context. Similar to existing VVC designs, when such a scheme is applied, multiple CABAC initialization tables may be pre-determined, and syntax elements may be signaled from the encoder to the decoder to indicate which initialization table is selected for each slice to initialize the two probabilities, two adaptation rates, and corresponding values ​​of the combining weights associated with each slice. In one specific embodiment, it is proposed to reuse existing CABAC initialization syntax elements, namely, pps_cabac_init_present_flag and sh_cabac_init_flag, to indicate the selection of the CABAC initialization table for a slice. When such a method is applied, the CABAC states (i.e., probabilities, adaptation rates, and connection weights) of an I slice are only allowed to be initialized by an I slice-type initialization table, and the CABAC states of a P (or B) slice are only allowed to be initialized by a B (or P) slice-type initialization table. In another embodiment, it is proposed to predetermine several (>3) CABAC tables and allow the CABAC states of one slice to be arbitrarily initialized from one of the predefined tables.

[0074] In all the above methods, a fixed value is used to initialize the MHP-AW weights of a context when encoding one slice, which may not be accurate enough to provide reliable probability estimation for arithmetic coding. To solve such problems, we propose giving the encoder the flexibility to calculate the optimal MHP-AW weights for each slice and signal the corresponding optimal MHP-AW weights to the decoder.

[0075] In one embodiment, it is proposed to directly signal the weight value of each context element. For example, a flag may be first signaled to indicate whether the MHP-AW weights of contexts in a slice are initialized with one fixed initialization table. If the flag is equal to 1, another syntax element may be signaled to inform the decoder which weight initialization table is applied to the current slice; otherwise, if the flag is equal to 0, the MHP-AW weights of contexts in the slice are initialized by values ​​parsed from the bitstream. In practice, various binarization methods may be applied to generate codewords for the MHP-AW weight values, such as fixed-length codes, unary codes, and k-th order exponential-Golomb codes.

[0076] In another embodiment, an adaptive signaling method is proposed. Specifically, at the start of each slice, this scheme first transmits a binary map weightMap[] from the encoder to the decoder, where each element indicates whether the corresponding context uses the initialized weight value from the selected default initialization weight table. If the i-th weightMap[] map entry is equal to 0, this means that the MHP-AW weight of the i-th context of the current slice is initialized by the corresponding value in the selected initialization weight table. If the i-th weightMap[] map entry is equal to 1, this means that the MHP-AW weight of the i-th context of the current slice is initialized by the initial value indicated in the bitstream. Various methods can be applied to encode the map weightMap[]. In one example, it is proposed to use run-length coding to encode the binary value of the map, and a "run" value is transmitted to indicate the number of consecutive 0s (or 1s) before encountering a single 1 (or 0).

[0077] Inheriting initial CABAC states from previously coded slices As discussed above, symbols of the same context in different video bitstreams usually exhibit completely different statistical characteristics. We assume that the probability state is fixed in a predefined CABAC table and cannot adapt to the unique characteristics of various slices. If the predefined initial table deviates from the true symbol statistics of the video bitstream, the coding efficiency may decrease. To improve CABAC efficiency, an improved CABAC initialization scheme is proposed to initialize the context from the context state after encoding / decoding one previous slice. There are various possible ways to identify the previous slice for CABAC initialization.

[0078] In one embodiment, it is proposed to maintain output context states of N coded slices. When a current slice is coded, the video encoder selects the best one from the N coded slices and signals an index to the decoder to initialize the context of the current slice from the corresponding context state of the selected coded slice (indicated by the signaled index value).

[0079] In other embodiments, instead of directly signaling the selected coded slice in the bitstream, some implicit decoder-side selection schemes may be applied to select the corresponding coded slice for context initialization of the current slice, including:

[0080] Rule #1: It is suggested to directly select the slice that is coded immediately before the slice according to the coding order.

[0081] Rule #2: It is suggested to select the coded slice that is closest in order to the current slice and has the same slice type.

[0082] Rule #3: It is proposed to select the coded slice that is closest to the current slice in order and has the smallest QP difference with the current slice.

[0083] Rule #4: It is suggested to select the coded slice that is closest in order to the current slice and has the same temporal layer as the current slice.

[0084] Although the above implicit selection rules are proposed separately, they can be combined and applied together in the proposed initial CABAC state inheritance scheme. In one particular example, we propose combining rules #2, #3, and #4 together. Specifically, based on such a combination, the encoder / decoder may select a coded slice of the same slice type that is closest to the current slice according to coding order and has the smallest QP difference with the current slice. If no such coded slice exists, one of the existing predefined CABAC initialization tables may be applied to initialize the context state of the current slice.

[0085] Furthermore, in the proposed inheritance-based context initialization scheme, the inherited context state may include various categories of state information, e.g., probability values, adaptation rates, and combining weights (when the proposed MHP-AW scheme is applied). In one embodiment of the present disclosure, it is proposed to inherit only one state information from the selected coded slice, and the other state information of the current slice is initialized using an existing pre-defined CABAC initialization table. In another embodiment, it is proposed to inherit only two state information from the selected coded slice. In yet another embodiment, it is proposed to inherit all state information of the current slice from the corresponding context state of the selected coded slice.

[0086] The above inheritance-based CABAC initialization can improve the efficiency of context initialization, but it may introduce parse dependencies between different slices, because the entropy coding of one current slice cannot be invoked until the entropy coding of its reference slice (i.e., the selected coded slice) is completely finished.

[0087] To provide better control over the efficiency and parallelism of entropy coding, an adaptive CABAC initialization scheme is proposed. The context state of a current slice can be initialized in one of two ways: 1) using one of the predefined CABAC initialization tables, or 2) by the resulting context state of a previously coded slice. Specifically, in the proposed scheme, a binary flag is first signaled at the beginning of a slice. If the flag is equal to zero, this means that the context of the current slice is initialized by one of the existing predefined CABAC initialization tables, for example, indicated by the syntax element sh_cabac_init_flag. If the flag is equal to one, this means that the inheritance-based context initialization method is applied, and the initialization context value is set to the context state output from the selected slice coded before the current slice.

[0088] 4 illustrates a computing environment (or computing device) 410 coupled with a user interface 460. The computing environment 410 may be part of a data processing server. In some embodiments, the computing device 410 may perform any of the various methods or processes (e.g., encoding / decoding methods or processes) described above in accordance with various examples of this disclosure. The computing environment 410 may include a processor 420, a memory 440, and an I / O interface 450.

[0089] The processor 420 typically controls the overall operation of the computing environment 410, such as operations associated with display, data acquisition, data communication, and image processing. The processor 420 may include one or more processors for executing instructions for performing all or a portion of the steps in the methods described above. Additionally, the processor 420 may include one or more modules that facilitate interaction between the processor 420 and other components. The processor may be a central processing unit (CPU), a microprocessor, a single-chip machine, a GPU, etc.

[0090] Memory 440 is configured to store various types of data to support the operation of computing environment 410. Memory 440 may include predefined software 442. Examples of such data include instructions for any applications or methods run on computing environment 410, video data sets, image data, etc. Memory 440 may be implemented using any type of volatile or non-volatile memory device, or combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0091] The I / O interface 450 provides an interface between the processor 420 and a peripheral interface module, such as a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, a start scan button, and a stop scan button. The I / O interface 450 may be coupled to an encoder and a decoder.

[0092] In some embodiments, a non-transitory computer-readable storage medium containing a plurality of programs executable by the processor 420 in the computing environment 410 for performing the methods described above, e.g., contained in the memory 440, is also provided. For example, the non-transitory computer-readable storage medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0093] A non-transitory computer-readable storage medium has stored thereon a plurality of programs that are executed by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, cause the computing device to perform the method for motion prediction described above.

[0094] In some embodiments, the computing environment 410 may be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphical processing units (GPUs), controllers, microcontrollers, microprocessors, or other electronic components for performing the methods described above.

[0095] FIG. 5 is a flowchart illustrating a method for video decoding according to an example of this disclosure.

[0096] In step 501, the processor 420 may, at one binary arithmetic decoder side, obtain a multiple hypothesis probability of one binary symbol of one given context model of the binary arithmetic decoder according to an adaptive weight, where the multiple hypothesis probability indicates the probability that one binary symbol is equal to a binary value, and the one binary symbol is from multiple binary symbols associated with the context model.

[0097] For example, the adaptive weight can be the weight ω in equation (5), which is used to combine two probability estimates and whose value is taken from the range [0,1]. In equation (5), the weight ω represents a single real value, which needs to be quantized to an integer for hardware / software codec implementation. In practice, various methods can be applied to convert its value to an integer. Therefore, the multiple hypothesis probability is the combination of the two probability estimates using the adaptive weight ω.

[0098] In some examples, the processor 420 may obtain a first probability of one binary symbol according to the first adaptive parameter and a second probability of one binary symbol according to the second adaptive parameter. Further, the processor 420 may obtain multiple hypothesis probabilities according to the adaptive weights, the first probability, and the second probability. For example, the first probability may be p0(t+1) in Equation (5), and the second probability may be p1(t+1) in Equation (5).

[0099] In some examples, the processor 420 may obtain the adaptive weights according to the context model from a set of predetermined integer weight values ​​in a weight initialization table.

[0100] As discussed in the "Multiple Hypothesis Probability Estimation with Adaptive Weights" section, it has been proposed to select optimal weights for each context from a predefined set. For example, it has been proposed to use the predefined sets of integer weight values ​​{0, 3, 6, 10, 13, 16, 19, 22, 26, 29, 32}, {0, 6, 11, 16, 21, 26, 32}, or {0, 8, 12, 16, 20, 24, 32}.

[0101] In some examples, the decoder may obtain a weight initialization table for each slice type, and in response to determining that the current slice is of a first slice type, select a weight initialization table according to the first slice type, the first slice type including an I type, a P type, or a B type, and obtain adaptive weights according to the selected weight initialization table.

[0102] In some examples, in response to determining that the control syntax element is enabled, processor 420 may obtain, at the slice level, an adaptive weight syntax element for each slice that indicates a weight initialization table selected for each slice according to the slice type of each slice.

[0103] In some examples, processor 420 may obtain a first weight initialization table for each I slice, a second weight initialization table for each P slice, and a third weight initialization table for each B slice. Further, processor 420 may select the first weight initialization table in response to determining that the current slice is an I slice, and may select the second weight initialization table or the third weight initialization table in response to determining that the current slice is a P slice or a B slice, and obtain adaptive weights according to the selected weight initialization table, which is shown in Table 2.

[0104] In step 502, the processor 420 may decode one binary symbol according to multiple hypothesis probabilities.

[0105] FIG. 6 is a flow chart illustrating a method for video encoding that corresponds to the method for video decoding shown in FIG.

[0106] In step 601, the processor 420 may determine, at the binary arithmetic decoder side, a multiple hypothesis probability of a binary symbol of a given context model of the binary arithmetic decoder according to an adaptive weight, where the multiple hypothesis probability indicates a probability that the binary symbol is equal to a binary value from among multiple binary symbols associated with the context model. For example, the encoder may obtain a multiple hypothesis probability of a binary symbol of a given context model of the binary arithmetic decoder according to an adaptive weight.

[0107] For example, the adaptive weight can be the weight ω in equation (5), which is used to combine two probability estimates and whose value is taken from the range [0,1]. In equation (5), the weight ω represents a real value, which needs to be quantized to an integer for hardware / software codec implementation. In practice, various methods can be applied to convert the value of the adaptive weight to an integer. Therefore, the multiple hypothesis probability is the combination of two probability estimates using the adaptive weight ω.

[0108] In some examples, the processor 420 may determine a first probability of one binary symbol according to the first adaptive parameter and a second probability of one binary symbol according to the second adaptive parameter. Further, the processor 420 may determine multiple hypothesis probabilities according to the adaptive weights, the first probability, and the second probability. For example, the first probability may be p0(t+1) in Equation (5), and the second probability may be p1(t+1) in Equation (5).

[0109] In some examples, the processor 420 may determine the adaptive weights according to the context model from a set of predetermined integer weight values ​​in a weight initialization table.

[0110] As discussed in the "Multiple Hypothesis Probability Estimation with Adaptive Weights" section, it has been proposed to select optimal weights for each context from a predefined set. For example, it has been proposed to use the predefined sets of integer weight values ​​{0, 3, 6, 10, 13, 16, 19, 22, 26, 29, 32}, {0, 6, 11, 16, 21, 26, 32}, or {0, 8, 12, 16, 20, 24, 32}.

[0111] In some examples, the decoder may determine a weight initialization table for each slice type, and in response to determining that the current slice is of a first slice type, select a weight initialization table according to the first slice type, the first slice type including an I type, a P type, or a B type, and obtain adaptive weights according to the selected weight initialization table.

[0112] In some examples, in response to determining that the control syntax element is enabled, processor 420 may determine, at the slice level, an adaptive weight syntax element for each slice that indicates a weight initialization table selected for each slice according to the slice type of each slice.

[0113] In some examples, processor 420 may determine a first weight initialization table for each I slice, a second weight initialization table for each P slice, and a third weight initialization table for each B slice. Further, processor 420 may select the first weight initialization table in response to determining that the current slice is an I slice, select the second weight initialization table or the third weight initialization table in response to determining that the current slice is a P slice or a B slice, and determine the adaptive weights according to the selected weight initialization table, as shown in Table 2.

[0114] In step 602, the processor 420 may encode a binary symbol according to multiple hypothesis probabilities. For example, the encoder may encode a binary symbol according to multiple hypothesis probabilities.

[0115] FIG. 7 is a flowchart illustrating a method for video decoding according to an example of this disclosure.

[0116] In step 701, the processor 420 may select one or more slices coded before the current slice at the decoder side.

[0117] For example, as discussed in the "Inheriting Initial CABAC States from Previously Encoded Slices" section, the one or more slices may be N previously coded slices. When a current slice is to be coded, the video encoder selects the best one from the N previously coded slices and signals an index to the decoder to initialize the context of the current slice from the corresponding context state of the selected previously coded slice (indicated by the signaled index value).

[0118] In some examples, the one or more slices may include a slice that was coded immediately before the current slice.

[0119] In some examples, one or more slices may include an uncoded slice immediately preceding the current slice, where the slice satisfies at least one of the following conditions: the slice has the same slice type as the current slice, the slice has a minimum quantization parameter (QP) difference from the current slice, or the slice has the same temporal layer as the current slice.

[0120] For example, some implicit decoder-side selection schemes may be applied to select a corresponding coded slice for context initialization of the current slice according to Rule #1, Rule #2, Rule #3, Rule #4, or any combination of these rules.

[0121] In step 702, processor 420 may obtain an initial context state of one or more context models of the current slice by inheriting a context state of one or more context models of a slice coded before the current slice.

[0122] By inheriting the context state, one or more context models of the current slice use the context state of one or more context models of one previously coded slice (i.e., one slice coded before the current slice) as their initial context state.

[0123] In some examples, the one or more initial context states may include at least one of a probability value, an adaptation rate, or an adaptation weight parameter.

[0124] For example, the inherited context state may include various categories of state information, e.g., probability values, adaptation rates, and combining weights (if the proposed MHP-AW scheme is applied). In one embodiment of the present disclosure, it is proposed to inherit only one state information from the selected coded slice, and the other state information of the current slice is initialized using an existing pre-defined CABAC initialization table. In another embodiment, it is proposed to inherit only two state information from the selected coded slice. In yet another embodiment, it is proposed to inherit all state information of the current slice from the corresponding context state of the selected coded slice.

[0125] In some examples, the processor 420 may obtain at least one second initial context state for a current slice according to one or more existing context-based adaptive binary arithmetic coding (CABAC) initialization tables, the current slice including the initial context state and the at least one second initial context state. Further, the processor 420 may decode binary symbols in the current slice according to the initial context state and the at least one second initial context state.

[0126] In step 703, the processor 420 may decode binary symbols associated with one or more context models in the current slice according to the initial context state.

[0127] FIG. 8 is a flow chart illustrating a method for video encoding that corresponds to the method for video decoding shown in FIG.

[0128] In step 801, the processor 420 may select one or more slices coded before the current slice at the encoder side.

[0129] For example, as discussed in the "Inheriting Initial CABAC States from Previously Encoded Slices" section, the one or more slices may be N previously coded slices. When a current slice is to be coded, the video encoder selects the best one from the N previously coded slices and signals an index to the decoder to initialize the context of the current slice from the corresponding context state of the selected previously coded slice (indicated by the signaled index value).

[0130] In some examples, the one or more slices may include a slice that was coded immediately before the current slice.

[0131] In some examples, one or more slices may include an uncoded slice immediately preceding the current slice, where the slice satisfies at least one of the following conditions: the slice has the same slice type as the current slice, the slice has the smallest quantization parameter (QP) difference from the current slice, or the slice has the same temporal layer as the current slice.

[0132] For example, several implicit selection schemes may be applied to select a corresponding coded slice for context initialization of the current slice according to Rule #1, Rule #2, Rule #3, Rule #4, or any combination of these rules.

[0133] In step 802, processor 420 may determine an initial context state of one or more context models of the current slice by inheriting a context state of one or more context models of a slice coded before the current slice.

[0134] By inheriting the context state, one or more context models of the current slice use the context state of one or more context models of one previously coded slice (i.e., one slice coded before the current slice) as their initial context state.

[0135] In some examples, the one or more initial context states may include at least one of a probability value, an adaptation rate, or an adaptation weight parameter.

[0136] For example, the inherited context state may include various categories of state information, e.g., probability values, adaptation rates, and combining weights (if the proposed MHP-AW scheme is applied). In one embodiment of the present disclosure, it is proposed to inherit only one state information from the selected coded slice, and the other state information of the current slice is initialized using an existing pre-defined CABAC initialization table. In another embodiment, it is proposed to inherit only two state information from the selected coded slice. In yet another embodiment, it is proposed to inherit all state information of the current slice from the corresponding context state of the selected coded slice.

[0137] In step 803, the processor 420 may encode binary symbols associated with one or more context models in the current slice according to the initial context state.

[0138] In some examples, the processor 420 may determine at least one second initial context state for a current slice according to one or more existing context-based adaptive binary arithmetic coding (CABAC) initialization tables, the current slice including the initial context state and the at least one second initial context state. Further, the processor 420 may encode binary symbols in the current slice according to the initial context state and the at least one second initial context state.

[0139] In some examples, an apparatus for video encoding is provided that includes a processor 420 and a memory 440 configured to store instructions executable by the processor, the processor being configured, upon execution of the instructions, to perform any of the methods described above.

[0140] In some other examples, a non-transitory computer-readable storage medium having instructions stored thereon is provided. When executed by processor 420, the instructions cause the processor to perform any method described in this disclosure. In one example, multiple programs may be executed by processor 420 in computing environment 410 to receive (e.g., from video encoder 20 of FIG. 2 ) a bitstream or datastream including coded video information (e.g., video blocks representing coded video frames and / or one or more associated syntax elements, etc.) and to perform the above-described decoding method according to the received bitstream or datastream. In other examples, multiple programs may be executed by processor 420 in computing environment 410 to perform the above-described encoding method to encode video information (e.g., video blocks representing video frames and / or one or more associated syntax elements, etc.) into a bitstream or datastream and to transmit the bitstream or datastream (e.g., to video decoder 30 of FIG. 3 ). Alternatively, the non-transitory computer-readable storage medium may store a bitstream or datastream containing encoded video information (e.g., video blocks representing encoded video frames and / or one or more associated syntax elements, etc.) generated by an encoder (e.g., video encoder 20 of FIG. 2) using, for example, the encoding method described above, for use by a decoder (e.g., video decoder 30 of FIG. 3) in decoding video data. The non-transitory computer-readable storage medium may be, for example, a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0141] Other examples of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure in accordance with the general principles of the present disclosure, including departures from the present disclosure that are within known or customary practice in the art. It is intended that the specification and examples be considered exemplary only.

[0142] It will be understood that the present disclosure is not limited to the exact examples described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope of the present disclosure.

Claims

1. 1. A method for video decoding, comprising: obtaining, by a binary arithmetic decoder, a multi-hypothesis probability of a binary symbol of a given context model of the binary arithmetic decoder according to an adaptive weight, the multi-hypothesis probability indicating a probability that the binary symbol is equal to a binary value, the binary symbol being from a plurality of binary symbols associated with the context model; decoding, by the decoder, the one binary symbol according to the multiple hypothesis probabilities; A method comprising:

2. obtaining, by the decoder, a first probability of the one binary symbol according to a first adaptation parameter; obtaining, by the decoder, a second probability of the one binary symbol according to a second adaptation parameter; obtaining, by the decoder, the multiple hypothesis probabilities according to the adaptive weights, the first probability, and the second probability; The method of claim 1 further comprising:

3. obtaining, by the decoder, the adaptive weights according to the context model from a set of predetermined integer weight values ​​in a weight initialization table; The method of claim 1 further comprising:

4. obtaining, by the decoder, a weight initialization table for each slice type; in response to determining that a current slice is a first slice type, selecting, by the decoder, the weight initialization table according to the first slice type, wherein the first slice type comprises an I-type, a P-type, or a B-type; obtaining, by the decoder, the adaptive weights according to the selected weight initialization table; The method of claim 1 further comprising:

5. Obtaining, by the decoder, a control syntax element in a picture parameter set (PPS) that indicates whether to select a weight initialization table for each slice. The method of claim 4 further comprising:

6. and in response to determining that the control syntax element is enabled, obtaining, by the decoder, at a slice level, an adaptive weight syntax element for each slice that indicates a weight initialization table selected for each slice according to the slice type of each slice. The method of claim 5 further comprising:

7. obtaining, by the decoder, a first weight initialization table for each I slice, a second weight initialization table for each P slice, and a third weight initialization table for each B slice; selecting, by the decoder, the first weight initialization table in response to determining that the current slice is an I slice; selecting, by the decoder, the second weight initialization table or the third weight initialization table in response to determining that the current slice is a P slice or a B slice; obtaining, by the decoder, the adaptive weights according to the selected weight initialization table; The method of claim 1 further comprising:

8. In response to determining that the current slice is not an I-slice, obtaining, by the decoder, a control syntax element in a picture parameter set (PPS) that indicates whether to select a weight initialization table for each slice; In response to determining that the control syntax element is enabled, obtaining, by the decoder, an adaptive weight syntax element for each non-I slice that indicates a weight initialization table selected from the second weight initialization table and the third weight initialization table at a slice level; The method of claim 7 further comprising:

9. 1. A method for video encoding, comprising: determining, by a binary arithmetic coder, a multiple hypothesis probability of a binary symbol from a plurality of binary symbols of a given context model of the binary arithmetic coder according to an adaptive weight, wherein the multiple hypothesis probability indicates a probability that the binary symbol is equal to a binary value; encoding, by the encoder, the one binary symbol according to the multiple hypothesis probabilities; A method comprising:

10. determining, by the encoder, a first probability of the one binary symbol according to a first adaptation parameter; determining, by the encoder, a second probability of the one binary symbol according to a second adaptation parameter; determining, by the encoder, the multiple hypothesis probabilities according to the adaptive weights, the first probability, and the second probability; 10. The method of claim 9, further comprising:

11. predetermining, by the encoder, the adaptive weights as weight values ​​according to a set of predetermined integer weight values ​​in a weight initialization table; 10. The method of claim 9, further comprising:

12. determining, by the encoder, a weight initialization table for each slice type; in response to determining that a current slice is of a first slice type, selecting, by the encoder, the weight initialization table according to the first slice type, wherein the first slice type comprises an I-type, a P-type, or a B-type; determining, by the encoder, the adaptive weights according to the selected weight initialization table; 10. The method of claim 9, further comprising:

13. signaling, by the encoder, a control syntax element in a picture parameter set (PPS) to indicate whether to select a weight initialization table for each slice; The method of claim 12 further comprising:

14. signaling, by the encoder, at a slice level, an adaptive weight syntax element for each slice to indicate a selected weight initialization table for each slice in response to determining that the control syntax element is enabled.

14. The method of claim 13, further comprising:

15. determining, by the encoder, a first weight initialization table for each I slice, a second weight initialization table for each P slice, and a third weight initialization table for each B slice; selecting, by the encoder, the first weight initialization table in response to determining that the current slice is an I slice; selecting, by the encoder, the second weight initialization table or the third weight initialization table in response to determining that the current slice is a P slice or a B slice; determining, by the encoder, the adaptive weights according to the selected weight initialization table; 10. The method of claim 9, further comprising:

16. signaling, by the encoder, a control syntax element in a picture parameter set (PPS) to indicate whether to select a weight initialization table for each slice in response to determining that the current slice is not an I-slice; signaling, by the encoder, at a slice level, an adaptive weight syntax element for each non-I slice to indicate a weight initialization table selected from the second weight initialization table and the third weight initialization table in response to determining that the control syntax element is enabled; 16. The method of claim 15, further comprising:

17. 1. A method for video decoding, comprising: selecting, by a decoder, one or more slices coded before the current slice; obtaining, by the decoder, an initial context state of one or more context models of the current slice by inheriting a context state of one or more context models of a slice coded before the current slice; decoding, by the decoder, binary symbols associated with the one or more context models in the current slice according to the initial context state; A method comprising:

18. The method of claim 17 , wherein the one or more slices include a slice that was coded immediately before the current slice.

19. The one or more slices include an uncoded slice immediately before the current slice, the slice comprising: the condition that the slice has the same slice type as the current slice; the slice has the smallest quantization parameter (QP) difference with the current slice; or the condition that the slice has the same temporal layer as the current slice; The method of claim 7 , wherein at least one of the following is satisfied:

20. 20. The method of claim 17, wherein the one or more initial context states include at least one of a probability value, an adaptation rate, or an adaptation weight parameter.

21. obtaining, by the decoder, at least one second initial context state for the current slice according to one or more existing context-based adaptive binary arithmetic coding (CABAC) initialization tables, wherein the current slice includes the initial context state and the at least one second initial context state; decoding, by the decoder, the binary symbols in the current slice according to the initial context state and the at least one second initial context state; 20. The method of claim 17, further comprising:

22. 1. A method for video encoding, comprising: selecting, by an encoder, one or more slices coded before a current slice; determining, by the encoder, an initial context state of one or more context models for the current slice by inheriting a context state of one or more context models from a slice coded before the current slice; encoding, by the encoder, binary symbols associated with the one or more context models of the current slice according to the initial context state; A method comprising:

23. 23. The method of claim 22, wherein the one or more slices include a slice that was coded immediately before the current slice.

24. The one or more slices include an uncoded slice immediately before the current slice, the slice comprising: the condition that the slice has the same slice type as the current slice; the slice has the smallest quantization parameter (QP) difference with the current slice; or the condition that the slice has the same temporal layer as the current slice; 23. The method of claim 22, wherein at least one of the following is satisfied:

25. 23. The method of claim 22, wherein the one or more initial context states include at least one of a probability value, an adaptation rate, or an adaptation weight parameter.

26. determining, by the encoder, at least one second initial context state for the current slice according to one or more existing context-based adaptive binary arithmetic coding (CABAC) initialization tables, wherein the current slice includes the initial context state and the at least one second initial context state; encoding, by the encoder, the binary symbols in the current slice according to the initial context state and the at least one second initial context state; 23. The method of claim 22, further comprising:

27. 1. An apparatus for video decoding, comprising: one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; Equipped with Apparatus, wherein the one or more processors are configured to, upon execution of the instructions, perform the method of any one of claims 1 to 8 and 17 to 21.

28. 1. An apparatus for video encoding, comprising: one or more processors; a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors; Equipped with Apparatus, wherein the one or more processors are configured to, upon execution of the instructions, perform the method of any one of claims 9 to 16 and 22 to 26.

29. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to receive a bitstream and perform a method according to any one of claims 1 to 8 and 17 to 21 based on the bitstream.

30. 27. A non-transitory computer-readable storage medium for storing computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform a method according to any of claims 9-16 and 22-26, encoding a current video block into a bitstream and transmitting the bitstream.

Images