Systems and methods for regularization-free multi-hypothesis arithmetic coding - Patents.com

Abstract

Various embodiments described herein include methods and systems for coding video. In one aspect, the method includes obtaining video data including syntax elements having a corresponding alphabet of M elements, a first set of occurrence probabilities for each of the M elements of the syntax elements, and a second set of occurrence probabilities for each of the M elements of the syntax elements. The method further includes entropy coding a first portion of the video data according to an aggregation of the first and second sets, and encountering a first syntax element while entropy coding the first portion of the video data. The method also includes updating the first set at a first rate, and updating the second set at a second rate that is faster than the first rate. The method further includes adaptively entropy coding the second portion of the video data.

Description

[Technical field]

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 322,411, entitled "Regularization-Free Multi-Hypothesis Arithmetic Coding," filed March 22, 2022, and is a continuation of, and claims priority to, U.S. Provisional Patent Application No. 18 / 121,432, entitled "Systems and Methods for Regularization-Free Multi-Hypothesis Arithmetic Coding," filed March 14, 2023, both of which are incorporated by reference in their entirety.

[0002] TECHNICAL FIELD Embodiments of this disclosure relate generally to video coding, including, but not limited to, systems and methods for multi-hypothesis arithmetic coding. [Background technology]

[0003] Digital video is supported by a variety of electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, video streaming devices, etc. The electronic devices transmit, receive, or communicate digital video data over communication networks and / or store the digital video data in storage devices. Because communication networks have limited bandwidth capacity and storage devices have limited memory resources, video coding may be used to compress the video data according to one or more video coding standards before the video data is communicated or stored.

[0004] Several video codec standards have been developed. For example, video encoding standards include AOMedia Video 1 (AV1), Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit the redundancy inherent in video data. Video coding aims to compress video data into a format that uses a lower bitrate while avoiding or minimizing degradation of video quality.

[0005] HEVC, also known as H.265, is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC published the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC), also known as H.266, is a video compression standard intended as a successor to HEVC. ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as a replacement for HEVC. The valid version 1.0.0 of this specification was released on January 8, 2019, containing Errata 1. Summary of the Invention [Means for solving the problem]

[0006] As mentioned above, encoding (compression) reduces bandwidth and / or storage space requirements. As will be described in more detail below, both lossless and lossy compression can be employed. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal via a decoding process. Lossy compression refers to a coding / decoding process where the original video information is not fully preserved when coded and cannot be fully restored when decoded. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is made small enough to make the reconstructed signal useful for its intended application. The amount of distortion that can be tolerated depends on the application. For example, a user of a particular consumer video streaming application may tolerate higher distortion than a user of a movie or television broadcast application. The compression ratio achievable by a particular coding algorithm can be selected or adjusted to reflect various distortion tolerances, i.e., generally, higher distortion tolerance allows for coding algorithms that result in higher losses and higher compression ratios.

[0007] A video encoder and / or decoder may utilize techniques from several broad categories and steps, including, for example, motion compensation, Fourier transform, quantization, and entropy coding. During entropy coding, data regarding the operation may be transmitted to an entropy encoder. The entropy encoder may output a bitstream (coded video sequence), which may be transmitted to another device via a transmission channel. During a video decoding process, the bitstream may be transmitted to an entropy decoder. The entropy decoder may output data regarding the operation, which may include intra prediction information, residual information, etc., based on the bitstream. In some embodiments, the entropy coding / decoding utilizes an arithmetic coding algorithm based on the probability of occurrence of a symbol (or character) as the basis for arithmetic coding. In some embodiments, the probability of occurrence of a symbol (or character) is dynamically updated during the coding / decoding process. For example, there are only two possible characters ("a" and "b"), and the probability of "a" occurring is denoted as p_a and the probability of "b" occurring is denoted as p_b, where p_a+p_b=1 (or any other constant value). Thus, when "a" is encountered in the coding / decoding process, p_a may be updated to a larger value and p_b may be updated to a smaller value since their sum may be constant. This probability update process is sometimes called the "probability transition process" or "probability state index update process."

[0008] According to some embodiments, a method of video coding is provided, comprising the steps of: (i) obtaining video data including a plurality of syntax elements, the plurality of syntax elements including a first syntax element having a corresponding alphabet of M elements; (ii) obtaining a first set of occurrence probabilities for each of the M elements of the first syntax element; (iii) obtaining a second set of occurrence probabilities for each of the M elements of the first syntax element; (iv) entropy coding a first portion of the video data according to an aggregation of the first and second sets; (v) while entropy coding the first portion of the video data, encountering the first syntax element; and (vi) entropy coding the first set by determining updated occurrence probabilities for the M elements of the first set according to the first syntax element. (vii) updating the second set by determining updated probabilities of occurrence of the M elements of the second set in accordance with the first syntax element, the second set being updated at a second rate faster than the first rate; (viii) entropy coding a second portion of the video data in accordance with a first aggregation of the updated first and second sets according to at least one of the updated probabilities of the second set being less than a threshold probability value; and (ix) entropy coding the second portion of the video data in accordance with a second aggregation of the updated first and second sets according to at least each of the updated probabilities of the second set being at least the threshold probability value.

[0009] According to some embodiments, a computing system, such as a streaming system, a server system, a personal computer system, or other electronic device, is provided. The computing system includes control circuitry and a memory that stores one or more instruction sets. The one or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and / or a decoder component.

[0010] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more sets of instructions for execution by a computing system. The one or more sets of instructions include instructions for performing any of the methods described herein.

[0011] Accordingly, disclosed are devices and systems having methods for coding video. Such methods, devices, and systems may complement or replace conventional methods, devices, and systems for video coding.

[0012] The features and advantages described herein are not necessarily all-inclusive, and in particular, some additional features and advantages will be apparent to those skilled in the art upon consideration of the drawings, specification, and claims provided in this disclosure.Furthermore, it should be noted that the language used herein has been selected primarily for purposes of readability and explanation, and not necessarily to describe or limit the subject matter described herein.

[0013] So that the present disclosure may be more fully understood, a more detailed description may be made by reference to features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings are merely illustrative of the relevant features of the present disclosure and therefore should not be considered necessarily limiting, since the description may recognize other useful features as understood by those skilled in the art upon reading the present disclosure. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating an exemplary communication system in accordance with some embodiments. [Figure 2A] FIG. 2 is a block diagram illustrating exemplary elements of an encoder component according to some embodiments. [Figure 2B] 2 is a block diagram illustrating exemplary elements of a decoder component according to some embodiments. [Diagram 3] FIG. 1 is a block diagram illustrating an exemplary server system according to some embodiments. [Figure 4] FIG. 11 is a flow diagram illustrating an example flow for decoding a binary decision. [Figure 5A] FIG. 1 illustrates an example approach to enforcing minimum probabilities, according to some embodiments. [Figure 5B] FIG. 13 is another diagram illustrating an example approach to enforcing minimum probabilities, according to some embodiments. [Figure 6] 1 is a flow diagram illustrating an example method for coding video according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] According to common practice, the various features illustrated in the drawings are not necessarily drawn to scale, and like reference numerals may be used to denote like features throughout the specification and drawings.

[0016] This disclosure describes a regularization-free multi-hypothesis arithmetic coding for entropy encoding / decoding. For example, a minimum probability is defined, and as each syntax element probability is updated, the updated probability is compared to the minimum probability. In this example, if the updated probability is less than the minimum probability, a different hypothesis is used or different weighting is applied to the multiple hypotheses. In this way, the final (combined) probability of each syntax element is prevented from approaching 0, reducing coding inefficiencies, preventing Bjontegaard delta (BD) rate losses, and / or reducing modeling errors. Furthermore, the regularization-free approach described herein reduces / eliminates the computational cost associated with performing a splitting operation during the regularization process.

[0017] Exemplary Systems and Devices 1 is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a number of electronic devices 120 (e.g., electronic devices 120-1 through 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system for use in video-enabled applications, such as, for example, video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0018] The source device 102 includes a video source 104 (e.g., a camera component or media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to generate an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from the video source 104 may be of a high data volume compared to the encoded video bitstream 108 generated by the encoder component 106. Because the encoded video bitstream 108 is of a low data volume (less data) compared to the video stream from the video source, the encoded video bitstream 108 requires less bandwidth to transmit and less storage space to store compared to the video stream from the video source 104. In some embodiments, the source device 102 does not include an encoder component 106 (e.g., configured to transmit uncompressed video data to the network(s) 110).

[0019] The one or more networks 110 represent any number of networks that convey information between the source device 102, the server system 112, and / or the electronic device 120, including, for example, wired (wired) and / or wireless communication networks. The one or more networks 110 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0020] The one or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from the source device 102). The server system 112 includes a coder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the coder component 114 includes an encoder component and / or a decoder component. In various embodiments, the coder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the coder component 114 is configured to decode the encoded video bitstream 108 and re-encode the video data using a different encoding standard and / or methodology to generate the encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings from the encoded video bitstream 108.

[0021] In some embodiments, the server system 112 functions as a media-enabled network element (MANE). For example, the server system 112 may be configured to prune the encoded video bitstream 108 to accommodate potentially different bitstreams for one or more of the electronic devices 120. In some embodiments, a MANE is provided separate from the server system 112.

[0022] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, decoder component 122 is configured to decode encoded video data 116 to generate an output video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes media storage). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0023] The source device and / or the electronic devices 120 may also be referred to as “terminal devices” or “user devices.” In some embodiments, the source device 102 and / or one or more of the electronic devices 120 are instances of a server system, a personal computer, a portable device (e.g., a smartphone, tablet, or laptop), a wearable device, a videoconferencing device, and / or other types of electronic devices.

[0024] In an exemplary operation of the communication system 100, the source device 102 transmits an encoded video bitstream 108 to the server system 112. For example, the source device 102 may code a stream of pictures captured by the source device. The server system 112 may receive the encoded video bitstream 108 and decode and / or encode the encoded video bitstream 108 using a coder component 114. For example, the server system 112 may apply optimal encoding to the video data via network transmission and / or storage. The server system 112 may transmit the encoded video data 116 (e.g., one or more coded video bitstreams) to one or more of the electronic devices 120. Each electronic device 120 may decode the encoded video data 116 to recover and optionally display the video pictures.

[0025] In some embodiments, the aforementioned transmission is a unidirectional data transmission. The unidirectional data transmission may be utilized in media serving applications, etc. In some embodiments, the aforementioned transmission is a bidirectional data transmission. The bidirectional data transmission may be utilized in video conferencing applications, etc. In some embodiments, the encoded video bitstream 108 and / or the encoded video data 116 are encoded and / or decoded according to any of the video coding / compression standards described herein, such as HEVC, VVC, and / or AV1.

[0026] FIG. 2A is a block diagram illustrating exemplary elements of the encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide the source video sequence in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 Y CrCb, or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. Video data may be provided as multiple individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples depending on the sampling structure, color space, etc. in use. Those skilled in the art can readily understand the relationship between pixels and samples. The following discussion focuses on samples.

[0027] The encoder component 106 is configured to code and / or compress pictures of a source video sequence into a coded video sequence 216 in real-time or under other time constraints required by an application. Enforcing an appropriate coding rate is one function of the controller 204. In some embodiments, the controller 204 controls and is operatively coupled to other functional units described below. Parameters set by the controller 204 may include rate control related parameters (e.g., picture skip, quantizer, and / or lambda values ​​for rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art can readily identify other functions of the controller 204 that may be associated with the encoder component 106 being optimized for a particular system design.

[0028] In some embodiments, the encoder component 106 is configured to operate in a coding loop. In a simplified example, the coding loop includes a source coder 202 (responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and reference picture(s)) and a (local) decoder 210. The decoder 210 reconstructs the symbols to generate sample data, similar to a (remote) decoder (when the compression between the symbols and the coded video bitstream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory 208. Since the decoding of the symbol stream results in a bit-exact result regardless of the location of the decoder (local or remote), the contents of the reference picture memory 208 are also bit-exact between the local and remote encoders. In this way, the predictor of the encoder interprets the same sample values ​​as reference picture samples as the decoder interprets when using prediction during decoding. This principle of reference picture synchrony (and the resulting drift if synchrony cannot be maintained due to, for example, channel errors) is known to those skilled in the art.

[0029] The operation of the decoder 210 may be the same as that of a remote decoder, such as the decoder component 122, described in detail below in connection with Figure 2B. However, with brief reference to Figure 2B, because symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder 214 and the parser 254 may be lossless, the entropy decoding portion of the decoder component 122, including the buffer memory 252 and the parser 254, may not be fully implemented in the local decoder 210.

[0030] At this point, it can be said that any decoder technology, except for analysis / entropy decoding, present in the decoder must also necessarily exist in the corresponding encoder in substantially the same functional form. For this reason, the subject matter of this disclosure focuses on the operation of the decoder. A description of the encoder technology can be omitted, since it is the inverse of the decoder technology, which is described generically. Only in certain areas is a more detailed description necessary, which is presented below.

[0031] As part of its operation, the source coder 202 may perform motion-compensated predictive coding, which predictively codes an input frame with reference to one or more previously coded frames from the video sequence, designated as reference frames. In this manner, the coding engine 212 codes differences between pixel blocks of the input frame and pixel blocks of the reference frame(s) that may be selected as the predictive reference(s) to the input frame. The controller 204 may manage the coding operations of the source coder 202, including, for example, setting parameters and subgroup parameters used to encode the video data.

[0032] The decoder 210 decodes the coded video data of a frame that may be designated as a reference frame based on the symbols created by the source coder 202. The operation of the coding engine 212 may preferably be a lossy process. When the coded video data is decoded in a video decoder (not shown in FIG. 2A), the reconstructed video sequence may be a replica of the source video sequence, usually with some errors. The decoder 210 may reproduce the decoding process that may be performed by a remote video decoder on the reference frame and store the reconstructed reference frame in the reference picture memory 208. In this way, the encoder component 106 locally stores a copy of the reconstructed reference frame that has common content as the reconstructed reference frame that will be obtained by the remote video decoder (without transmission errors).

[0033] The predictor 206 may perform a predictive search for the coding engine 212. That is, for a new frame to be coded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that may serve as suitable predictive references for the new picture. The predictor 206 may operate on sample blocks pixel block by pixel block to find a suitable predictive reference. In some cases, as determined by the search results obtained by the predictor 206, the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory 208.

[0034] The output of all the aforementioned functional units may undergo entropy coding in entropy coder 214. Entropy coder 214 converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding).

[0035] In some embodiments, the output of the entropy coder 214 is coupled to a transmitter. The transmitter may be configured to buffer the coded video sequence(s) generated by the entropy coder 214 to prepare for transmission over a communication channel 218, which may be a hardware / software link to a storage device that stores the coded video data. The transmitter may be configured to merge the coded video data from the source coder 202 with other data to be transmitted, e.g., coded audio data and / or ancillary data streams (sources not shown). In some embodiments, the transmitter may transmit additional data along with the coded video. The source coder 202 may include such data as part of the coded video sequence. The additional data may include other forms of redundant data, such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, supplemental enhancement information (SEI) messages, visual usability information (VUI) parameter set fragments, etc.

[0036] The controller 204 may manage the operation of the encoder component 106. During coding, the controller 204 may assign a specific coded picture type to each coded picture, which may affect the coding technique applied to the respective picture. For example, a picture may be assigned as an intra picture (I picture), a predicted picture (P picture), or a bidirectionally predicted picture (B picture). An intra picture may be coded and decoded without using any other frame in the sequence as a source of prediction. Some video codecs allow various types of intra pictures, including, for example, independent decoder refresh (IDR) pictures. Those skilled in the art will recognize those variations of I pictures and their respective uses and characteristics, so they will not be repeated here. A predicted picture may be coded and decoded using intra prediction or inter prediction, which uses at most one motion vector and reference index to predict sample values ​​of each block. A bidirectionally predicted picture may be coded and decoded using intra prediction or inter prediction, which uses at most two motion vectors and reference indexes to predict sample values ​​of each block. Similarly, a multi-predicted picture can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0037] A source picture may generally be spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and coded block by block. A block may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the block's respective picture. For example, a block of an I picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). A pixel block of a P picture may be non-predictively coded via spatial prediction with reference to one previously coded reference picture or via temporal prediction. A pixel block of a B picture may be non-predictively coded via spatial prediction with reference to one or two previously coded reference pictures or via temporal prediction.

[0038] A video may be captured in time sequence as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) uses spatial correlation within a given picture, while inter-picture prediction uses correlation (temporal or other) between pictures. In one example, a particular picture being coded / decoded, called the current picture, is divided into blocks. When a block in the current picture is similar to a reference block in a reference picture that has been previously coded and is still buffered in the video, the block in the current picture may be coded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0039] Encoder component 106 may perform coding operations according to a given video coding technique or standard, such as any of those described herein. In its operations, encoder component 106 may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0040] 2B is a block diagram illustrating exemplary elements of the decoder component 122 according to some embodiments. The decoder component 122 of FIG. 2B is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to a loop filter unit 256 and configured to transmit data to the display 124 (e.g., via a wired or wireless connection).

[0041] In some embodiments, the decoder component 122 includes a receiver coupled to the channel 218 and configured to receive data from the channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more coded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each coded video sequence is independent of the other coded video sequences. Each coded video sequence may be received from the channel 218, which may be a hardware / software link to a storage device that stores the coded video data. The receiver receives the coded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to their respective usage entities (not shown). The receiver may separate the coded video sequence from the other data. In some embodiments, the receiver receives additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence(s). The additional data may be used by the decoder component 122 to decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, a temporal, spatial or SNR enhancement layer, redundant slices, redundant pictures, forward error correction codes, etc.

[0042] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra-picture prediction unit 262, a motion compensated prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuitry. In some embodiments, the decoder component 122 is implemented at least partially in software.

[0043] The buffer memory 252 is coupled between the channel 218 and the parser 254 (e.g., to combat network jitter). In some embodiments, the buffer memory 252 is separate from the decoder component 122. In some embodiments, a separate buffer memory is provided between the output of the channel 218 and the decoder component 122. In some embodiments, in addition to the buffer memory 252 in the decoder component 122 (e.g., configured to handle playout timing), a separate buffer memory is provided outside the decoder component 122 (e.g., to combat network jitter). When receiving data from a storage / forwarding device of sufficient bandwidth and controllability, or from an isosynchronous network, the buffer memory 252 may not be necessary or may be small. For use in a best-effort packet network such as the Internet, the buffer memory 252 may be required, and may be relatively large, advantageously adaptively sized, and at least partially implemented in an operating system or similar element (not shown) outside the decoder component 122.

[0044] The parser 254 is configured to reconstruct symbols 270 from the coded video sequence. The symbols may include, for example, information used to manage the operation of the decoder component 122 and / or information for controlling a rendering device such as the display 124. The control information for the rendering device(s) may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser 254 parses (entropy decodes) the coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without contextual dependency, etc. The parser 254 may extract a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. Parser 254 may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0045] The reconstruction of symbols 270 may involve a number of different units, depending on the type of video picture or portion thereof coded (inter-picture and intra-picture, inter-block and intra-block, etc.), and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by parser 254. The flow of such subgroup control information between parser 254 and the following units is not depicted for convenience of explanation.

[0046] In addition to the functional blocks already mentioned, the decoder component 122 can be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the conceptual subdivision into the following functional units is maintained:

[0047] The scalar / inverse transform unit 258 receives the quantized transform coefficients as well as control information (e.g., which transform to use, block size, quantization coefficients, and / or quantization scaling matrix) as symbol(s) 270 from the parser 254. The scalar / inverse transform unit 258 may output blocks containing sample values ​​that may be input to an aggregator 268.

[0048] In some cases, the output samples of the scaler / inverse transform unit 258 relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 may generate a block of the same size and shape as the block being reconstructed using surrounding already reconstructed information fetched from the current (partially reconstructed) picture from the current picture memory 264. The aggregator 268 may add, for each sample, the prediction information generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258.

[0049] In other cases, the output samples of the scaler / inverse transform unit 258 are associated with an inter-coded, potentially motion-compensated block. In such cases, the motion compensated prediction unit 260 may access the reference picture memory 266 to fetch samples used for prediction. After motion compensating the fetched samples according to the symbols 270 associated with the block, these samples may be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 to generate output sample information (in this case referred to as residual samples or residual signals). The addresses in the reference picture memory 266 from which the motion compensated prediction unit 260 fetches the prediction samples may be controlled by a motion vector. The motion vector may be available to the motion compensated prediction unit 260, for example, in the form of the symbols 270, which may have X, Y, and reference picture components. Motion compensation may also include interpolation of fetched sample values ​​from the reference picture memory 266 when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0050] The output samples of aggregator 268 may be subjected to various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filter techniques controlled by parameters contained in the coded video bitstream and made available to loop filter unit 256 as symbols 270 from parser 254, but may also be responsive to meta-information obtained during decoding of previous (in decoding order) parts of the coded picture or coded video sequence, or to previously reconstructed and loop filtered sample values.

[0051] The output of the loop filter unit 256 may be a sample stream that may be output to a rendering device, such as the display 124, as well as stored in a reference picture memory 266 for use in future inter-picture prediction.

[0052] Once fully reconstructed, a particular coded picture can be used as a reference picture for future prediction. Once an encoded picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by parser 254), the current reference picture can become part of reference picture memory 266, and a new current picture memory can be reallocated before beginning reconstruction of the next coded picture.

[0053] The decoder component 122 may perform decoding operations according to a given video compression technique, which may be documented in a standard, such as any of the standards described herein. The coded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense of adhering to the syntax of the video compression technique or standard, as specified in the video compression technique document or standard, specifically in a profile document therein. Also, to conform to some video compression techniques or standards, the complexity of the coded video sequence may also be within a range prescribed by a level of the video compression technique or standard. In some cases, the level limits a maximum picture size, a maximum frame rate, a maximum reconstructed sample rate (e.g., measured in megasamples per second), a maximum reference picture size, etc. The limits set by the level may be further limited, in some cases, by a specification of a hypothetical reference decoder (HRD) and metadata for HRD buffer management signaled in the coded video sequence.

[0054] 3 is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, a memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., a CPU, a GPU, and / or a DPU). In some embodiments, the control circuit includes one or more field programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuits).

[0055] The network interface(s) 304 may be configured to interface with one or more communication networks (e.g., wireless, wired, and / or optical networks). The communication networks may be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of communication networks include local area networks such as Ethernet, cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable, satellite and terrestrial television, vehicular and industrial including CANBus, etc. Such communications may be unidirectional, receive only (e.g., broadcast television), unidirectional transmit only (e.g., CANbus to a particular CANbus device), or bidirectional (e.g., to another computer system using a local or wide area digital network). Such communications may include communications to one or more cloud computing networks.

[0056] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device(s) 310 may include one or more of a keyboard, a mouse, a trackpad, a touch screen, a data glove, a joystick, a microphone, a scanner, a camera, etc. The output device(s) 308 may include one or more of an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0057] Memory 314 may include high-speed random access memory (such as DRAM, SRAM, DDR RAM, and / or other random access solid-state memory devices) and / or non-volatile memory (such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). Memory 314 optionally includes one or more storage devices located remotely from control circuitry 302. Memory 314, or the non-volatile solid-state memory device(s) within memory 314, comprises a non-transitory computer-readable storage medium. In some embodiments, memory 314, or the non-transitory computer-readable storage medium of memory 314, stores the following programs, modules, instructions, and data structures, or a subset or superset thereof: • An operating system 316 that contains procedures for handling various basic system services and performing hardware-dependent tasks; ● A network communications module 318 used to connect the server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections); A coding module 320 for performing various functions related to encoding and / or decoding of data, such as video data. In some embodiments, the coding module 320 is an instance of the coder component 114. The coding module 320 may include, but is not limited to, one or more of the following: o A decoding module 322 for performing various functions related to decoding of encoded data, such as those described above with respect to the decoder component 122; o An encoding module 340 for performing various functions on the encoded data, such as those described above with respect to the encoder component 106; ● A picture memory 352 for storing pictures and picture data, e.g., for use with coding module 320. In some embodiments, the picture memory 352 includes one or more of the reference picture memory 208, the buffer memory 252, the current picture memory 264, and the reference picture memory 266.

[0058] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions described above with respect to the parser 254), a transform module 326 (e.g., configured to perform various functions described above with respect to the scaler / inverse transform unit 258), a prediction module 328 (e.g., configured to perform various functions described above with respect to the motion compensation prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (e.g., configured to perform various functions described above with respect to the loop filter unit 256).

[0059] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions described above with respect to the source coder 202, the coding engine 212, and / or the entropy coder 214) and a prediction module 344 (e.g., configured to perform various functions described above with respect to the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 include a subset of the modules shown in FIG. 3. For example, a shared prediction module is used by both the decoding module 322 and the encoding module 340.

[0060] Each of the above identified modules stored in memory 314 corresponds to a set of instructions for performing functions described herein. The above identified modules (e.g., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be combined or otherwise rearranged in various embodiments. For example, coding module 320 optionally does not include separate decoding and encoding modules, but rather uses the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the above identified modules and data structures. In some embodiments, memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0061] In some embodiments, the server system 112 includes a web or HyperText Transfer Protocol (HTTP) server, a File Transfer Protocol (FTP) server, as well as web pages and applications implemented using Common Gateway Interface (CGI) scripts, PHP Hypertext Preprocessor (PHP), Active Server Pages (ASP), HyperText Markup Language (HTML), Extensible Markup Language (XML), Java, JavaScript, Asynchronous JavaScript and XML (AJAX), XHP, Javelin, Wireless Universal Resource Files (WURFL), and the like.

[0062] 3 illustrates a server system 112 according to some embodiments, however, FIG. 3 is not intended as a structural schematic of the embodiments described herein, but rather as a functional description of various features that may be present in one or more server systems. In practice, and as will be recognized by those skilled in the art, items shown separately may be combined and some items may be separated. For example, some items shown separately in FIG. 3 may be implemented on a single server, and single items may be implemented by one or more servers. The actual number of servers used to implement the server system 112, and how functionality is allocated among them, will vary from implementation to implementation and will optionally depend in part on the amount of data traffic the server system processes during peak and average usage periods.

[0063] Entropy Coding As mentioned above, during entropy coding, data regarding the operation may be transmitted to an entropy encoder (e.g., entropy coder 214). The entropy encoder may output a bitstream, which may be transmitted to another device via a transmission channel. During a video decoding process, the bitstream may be transmitted to an entropy decoder. The entropy decoder may be configured to reconstruct, from a coded picture, certain symbols that represent syntax elements of which the coded picture is composed. Such symbols may include, for example, prediction information (e.g., intra- or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-predictive mode, merged sub-mode, or another sub-mode), certain samples or metadata used for prediction by an intra- or inter-decoder, residual information, e.g., in the form of quantized transform coefficients, etc.

[0064] For example, in HEVC, the entropy coder / decoder may use a context-adaptive binary arithmetic coding (CABAC) algorithm. HEVC's CABAC engine uses a table-based probability transition process between 64 different representative probability states.

[0065] Syntax elements describing video frame content may undergo binary arithmetic coding to obtain a stream encoded as a binary bin stream. During CABAC, the initial interval [0,1) may be stretched by an integer multiplier (e.g., 512), and the least likely symbol probabilities (pLPS) may be presented as integer divisions by rounding their quotients. Then, the interval division operation by general arithmetic coding may be performed as an approximation calculation using integer arithmetic of the specified resolution. The updated interval length corresponding to the LPS (rLPS) may be calculated as rLPS=R*pLPS, where R is the value of the current interval length. To save time and increase efficiency, the above computationally intensive multiplication operation may be replaced with a look-up table (LUT) populated with pre-calculated multiplication results, and thus the updated interval length corresponding to the LPS (ivLpsRange) may be obtained by two indexes pStateIdx and qRangeIdx, for example, ivlLpsRange=rangeTabLps[pStateIdx][qRangeIdx].

[0066] During encoding / decoding, the probability value pLPS may be recursively updated each time a new value of a bin (binVal) to be encoded / decoded is obtained. For example, at the kth step (i.e., during encoding or decoding of the kth bin), a new value of pLPS may be calculated to be a larger value when binVal is the value of the LPS, or a new value of pLPS may be calculated to be a smaller value when binVal is the value of the Most Probable Symbol (MPS).

[0067] In some embodiments, pLPS may be one of 64 possible values ​​indexed by a 6-bit pStateIdx variable. Updating the probability value may be accomplished by updating the index pStateIdx, which may be performed by looking up a value from a pre-computed table to save computational power and / or improve efficiency.

[0068] In some embodiments, the range ivlCurrRange representing the state of the coding engine may be quantized to a set of four values ​​before the calculation of the new interval range. The state transition may be implemented using a table containing all 64×4 8-bit pre-calculated values ​​to approximate the value of ivlCurrRange*pLPS(pStateIdx). Also, the decoding decision may be implemented using a pre-calculated LUT. The LUT is used to obtain the first ivlLpsRange, and then the ivlLpsRange is used to update the ivlCurrRange and calculate the output binVal.

[0069] As an example, in VVC, the probability may be linearly represented by the probability index pStateIdx. Thus, all calculations may be done using equations without LUT operations. To improve the accuracy of the probability estimation, a multi-hypothesis probability update model may be used, as shown in FIG. 4. In this example, pStateIdx used for interval repartition in the binary arithmetic coder is a combination of two probabilities pStateIdx0 and pStateIdx1. The two probabilities are associated with each context model and updated independently with different adaptation rates. The adaptation rates of pStateIdx0 and pStateIdx1 for each context model may be pre-trained based on the statistics of the associated bin. The probability estimate pStateIdx may be the average of the estimates from the two hypotheses.

[0070] 4 is a flow diagram illustrating an example flow for decoding a single decision (DecodeDecision) including a renormalization process (RenomD) in an arithmetic decoding engine. In some embodiments, the inputs to DecodeDecision are a context table (ctxTable) and a context index (ctxIdx). The value of the variable ivlLpsRange is derived as shown at 402. Given the current value of ivlCurrRange, the variable qRangeIdx is derived as qRangeIdx=ivlCurrRange>>5. Given qRangeIdx, pStateIdx0, and pStateIdx1 associated with ctxTable and ctxIdx, valMps and ivlLpsRange are derived as pState=pStateIdx1+16*pStateIdx0, valMps=pState>>14, and ivlLpsRange=(qRangeIdx*((valMps?32767-pState:pState)>>9)>>1)+4. The variable ivlCurrRange is set to ivlCurrRange-ivlLpsRange.

[0071] If ivlOffset is greater than or equal to ivlCurrRange, then the variable binVal is set equal to 1-valMps, ivlOffset is decremented by ivlCurrRange, and ivlCurrRange is set equal to ivlLpsRange; otherwise, the variable binVal is set equal to valMps.

[0072] To update the probabilities, during the state transition process, the inputs to this process are the current pStateIdx0 and pStateIdx1, and the decoded value binVal, and the outputs of this process are the updated pStateIdx0 and pStateIdx1 of the context variables associated with ctxTable and ctxIdx. The variables shift0 and shift1 are derived from the shiftIdx value associated with ctxTable and ctxIdx in 402, where shift0=(shiftIdx>>2)+2 and shift1=(shiftIdx&3)+3+shift0, and according to the decoded value binVal the updates of the two variables pStateIdx0 and pStateIdx1 associated with ctxTable and ctxIdx are derived as pStateIdx0=pStateIdx0-(pStateIdx0>>shift0)+(1023*binVal>>shift0) and pStateIdx1=pStateIdx1-(pStateIdx1>>shift1)+(16383*binVal>>shift1).

[0073] As an example, VVC CABAC may have a quantization parameter (QP) dependent initialization process that is invoked at the beginning of each slice. Given an initial value of luma QP for the slice, the initial probability state of the context model, denoted as preCtxState, may be derived by m=slopeIdx×5-45, n=(offsetIdx<<3)+7, and preCtxState=Clip3(1,127,((m×(QP-32))>>4)+n).

[0074] In some embodiments, slopeIdx and offsetIdx are limited to 3 bits, and the total initialization value is represented with 6-bit precision. The probability state preCtxState may directly represent the probability in the linear domain. Thus, preCtxState only needs a suitable shift operation before being input to the arithmetic coding engine, and the mapping from logarithmic to linear domain and the 256-byte table may be predefined and stored / preserved in memory. pStateIdx0 and pStateIdx1 may be obtained by pStateIdx0=preCtxState<<3, and pStateIdx1=preCtxState<<7.

[0075] In some embodiments, the CABAC algorithm may use a binary system that includes two possible letters / symbols (e.g., "0" and "1"). In a binary-based arithmetic coding algorithm, the two possible letters / symbols may also be represented as a least likely symbol (LPS) and a most likely symbol (MPS).

[0076] In some embodiments, an entropy encoder or decoder may use an arithmetic algorithm in an M-ary basis that includes M possible characters / symbols. For example, M may be any integer value between 2 and 16. For example, if M is equal to 5, then the M-ary basis includes 5 possible characters / symbols that can be represented as "0", "1", "2", "3", and "4".

[0077] An M-ary arithmetic coding engine is used to entropy code syntax elements. Each syntax element is associated with an alphabet of M elements. As an input to an encoder or decoder, a coding context may include a sequence of M-ary symbols with a set of M probabilities. Each of the M probabilities may correspond to each of the M-ary symbols and may be represented by a cumulative distribution function (CDF).

[0078] The cumulative distribution function of the M-ary symbol is C=[c0,c 1, …,c (M-2) ,c (M-1) The cumulative distribution function of an M-ary symbol may be represented by an array of M 15-bit integers, where c (M-1) =2 15 and c n / 32768 is the probability that the symbol is less than or equal to n, where n is an integer between 0 and M-1.

[0079] In some embodiments, the M probabilities (e.g., an array of cumulative distribution functions) are updated after coding / parsing each syntax element. In some embodiments, the M probabilities are updated after coding / decoding each M-ary symbol. For example, for M=4, the array of cumulative distribution functions is [c0,c 1, c2, c3].

[0080] In some embodiments, the updating of the M probabilities is performed according to the following formula:

number

[0081] In some embodiments, the M-ary arithmetic coding process may follow a conventional arithmetic coding engine design, but only the most significant 9 bits of the 15-bit probability value are input to the arithmetic encoder / decoder. The probability update rate α associated with a symbol is calculated based on the number of occurrences of the associated symbol when parsing the bitstream, and the value of α is reset at the beginning of a frame or tile using the following formula:

[0082] As an example,

number

[0083] Updating the rate α can result in a strongly biased distribution where certain symbol probabilities decrease close to zero. Probabilities close to zero can result in BD rate loss. To combat this effect, regularization techniques can be used to reduce the probability (p m ) is the threshold (P thr ), then the regularization term is applied to all probabilities, and p m P thr The regularization term may be taken from a uniform distribution or may depend on the sample space of the syntax element.

[0084] 5A and 5B are diagrams illustrating an example approach to enforcing a minimum probability according to some embodiments. As an example, in AV1, a minimum probability P minis applied (sometimes called the "minimum guard" approach). In the minimum guard approach shown in FIG. 5A, subsequent model updates increase p2 in stage n, which may drive p1 closer to 0, potentially resulting in coding efficiency loss in the presence of model error.

[0085] Regularization can be used as an alternative to the minimum guard approach and is illustrated in Figure 5B. For example, at the end of the update, P min Probability of falling below, say, p m If there is a uniform distribution, then one can apply it as a regularization term to adjust all probabilities simultaneously.

[0086] The CDF of the M-ary uniform distribution is

number

number

number

number

number

number

[0087] A multi-hypothesis probability model for encoding M-ary symbols may be used in which only a single probability model is used by the encoder / decoder. For example, a data adaptive model may be used for probability update, with the update rate being higher for fewer occurrences of the syntax element and lower for more observations. Multi-hypothesis estimation, where each syntax element maintains two or more probability tables with different update rates, can result in additional compression efficiency. A multi-hypothesis probability model may be used with two update rates, such as α1 and α2, as follows:

number

[0088] 6 is a flow diagram illustrating a method 600 for coding video according to some embodiments. Method 600 may be performed in a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and memory storing instructions for execution by the control circuitry. In some embodiments, method 600 is performed by executing instructions stored in a memory (e.g., memory 314) of the computing system.

[0089] The system obtains video data including a plurality of syntax elements, the plurality of syntax elements including a first syntax element having a corresponding alphabet of M elements (602). The system obtains a first set of respective occurrence probabilities for the M elements of the first syntax element (604). The system obtains a second set of respective occurrence probabilities for the M elements of the first syntax element (606). The system entropy codes a first portion of the video data according to an aggregation of the first and second sets (608). While entropy coding the first portion of the video data, the system encounters a first syntax element (610). The system updates the first set by determining updated occurrence probabilities for the M elements of the first set according to the first syntax element, the first set being updated at a first rate (614). The system adaptively entropy codes a subsequent portion of the video data (616).

[0090] The adaptive entropy coding includes entropy coding a subsequent portion of the video data in accordance with the updated first set according to at least one of the updated probabilities of the second set being less than a threshold probability value (618). The adaptive entropy coding includes entropy coding a subsequent portion of the video data in accordance with a first aggregation of the updated first and second sets according to each of the updated probabilities of the second set being at least a threshold probability value (620).

[0091] In some embodiments, the convention for weighting individual hypotheses is predefined for the multi-hypothesis probability model. In some embodiments, when updating the probabilities of individual hypotheses during entropy coding of syntax R, candidate probability updates are tested, and the candidate probability updates are determined to be at least one

number

number

[0092] In some embodiments, for a two-hypothesis model that includes one hypothesis with a faster update rate and one hypothesis with a slower update rate, the hypothesis with the faster update rate has a predefined minimum probability value P thr At least one lower

number

[0093] In some embodiments, for a two-hypothesis model that includes one hypothesis with a faster update rate and one hypothesis with a slower update rate, each of the two hypotheses has a predefined minimum probability value P thr At least one lower

number

[0094] In some embodiments, for a two-hypothesis model that includes one hypothesis with a faster update rate and one hypothesis with a slower update rate, the hypothesis with the faster update rate has a predefined minimum probability value P thr At least one lower

number

[0095] In some embodiments, for a two-hypothesis model that includes one hypothesis with a faster update rate and one hypothesis with a slower update rate, the hypothesis with the faster update rate has a predefined minimum probability value P thr At least one lower

number

[0096] In some embodiments, adaptive entropy coding is performed only on a subset of the plurality of syntax elements. In some embodiments, the probabilities / hypotheses are adjusted when the updated probabilities are lower than a predefined minimum probability for only selected syntax elements. In some embodiments, the probability adjustment method (when the updated probabilities are lower than a predefined minimum probability) is applied only to syntax elements having N or fewer symbols. Example values ​​of N include, but are not limited to, 2, 4, 6, 8, 10, 12.

[0097] In some embodiments, whether the probability adjustment method (if the updated probability is lower than a predefined minimum probability) is applied is specified by a flag for each syntax or group of syntaxes, and the flag can be signaled in the high-level syntax.

[0098] 6 shows some logical stages in a particular order, stages that are not order dependent may be rearranged and other stages may be combined or split. Some rearrangements or other groupings not specifically mentioned will be apparent to those of ordinary skill in the art, and thus the rearrangements and groupings presented herein are not exhaustive. Furthermore, it should be recognized that the various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0099] We now turn to some exemplary embodiments. In the following, R is a symbol space ε[s0,s1,...,s k ,…s M ], probability ε[p0,p1,…,p k ,…p M ], where M can take on values ​​ranging from 2 to 16, including but not limited to, and k≦M.

[0100] (A1) In one aspect, some embodiments include a method of video coding (e.g., method 600). In some embodiments, the method is performed in a computing system (e.g., server system 112) having memory and control circuitry. In some embodiments, the method is performed in a coding module (e.g., coding module 320). In some embodiments, the method is performed in an entropy coder (e.g., entropy coder 214). In some embodiments, the method is performed in a parser (e.g., parser 254).The method includes the steps of: (i) obtaining video data including a plurality of syntax elements, the plurality of syntax elements including a first syntax element having a corresponding alphabet of M elements; (ii) obtaining a first set of occurrence probabilities for each of the M elements of the first syntax element; (iii) obtaining a second set of occurrence probabilities for each of the M elements of the first syntax element; (iv) determining a predefined threshold probability value, wherein a first hypothesis having a probability value smaller than the predefined threshold probability value is used differently during decoding of the first syntax element than a second hypothesis having a probability value larger than the predefined threshold probability value; and (vi) updating the first set by determining updated occurrence probabilities for the M elements of the first set according to the first syntax element, (vii) updating the second set by determining updated probabilities of occurrence of M elements of the second set according to the first syntax element, the second set being updated at a second rate faster than the first rate; and (viii) adaptively entropy coding portions of the video data, the method comprising: (a) entropy coding the portions of the video data according to the updated first set according to at least one of the updated probabilities of the second set being less than a predefined threshold probability value; and (b) entropy coding the portions of the video data according to a first aggregation of the updated first and second sets according to each of the updated probabilities of the second set being at least a predefined threshold probability value. In some embodiments, the number of elements M ranges from 2 to 16. In some embodiments, the number of elements M is greater than 16. In some embodiments, the range is predefined based on a coding protocol, standard, and / or configuration.In some embodiments, the method further includes entropy coding a first portion of the video data according to an aggregation of the first and second sets, and encountering a first syntax element while entropy coding the first portion of the video data, wherein the first and second sets are updated in response to encountering the first syntax element.

[0101] (A2) In some embodiments of A1, entropy coding the portion of the video data in accordance with the updated first set includes assigning a weight of 0 to the updated second set in a second aggregation of the updated first and second sets. For example, the second aggregation is a weighted average of the updated first and second sets, and the weight assigned to the updated second set is 0.

[0102] (A3) In some embodiments of A1 or A2, entropy coding the portion of the video data according to the updated first set includes not using the updated second set to entropy code the portion, e.g., the portion of the video data is entropy coded using only the updated first set.

[0103] (A4) In some embodiments of any of A1-A3, the threshold probability value is selected from a set of threshold probability values ​​based on one or more characteristics of the first syntax element. For example, the threshold probability value may be based on the number M of elements.

[0104] (A5) In some embodiments of any of A1-A4, the method further includes (i) discarding the updated first set in accordance with at least one of the updated probabilities of the second set being less than a threshold probability value and in accordance with at least one of the updated probabilities of the first set being less than the threshold probability value, and (ii) entropy coding the portion of the video data in accordance with the first set. For example, if both updated sets include probabilities below the threshold probability value, the second coding uses the first set with no updates applied.

[0105] (A6) In some embodiments of any of A1-A5, entropy coding the portion of the video data according to the updated first set includes entropy coding the portion of the video data according to a second aggregation of the updated first and second sets, the second aggregation including different weights than the first aggregation. For example, the second set is downweighted according to one of its elements having a probability lower than a threshold probability value.

[0106] (A7) In some embodiments of any of A1-A6, the method further includes decreasing an update rate of the second set in accordance with at least one of the updated probabilities of the second set being less than a threshold probability value. For example, the second set is updated at a lower rate to prevent the probability of any of its elements quickly approaching zero.

[0107] (A8) In some embodiments of any of A1-A7, the plurality of syntax elements includes a second syntax element having a corresponding alphabet of N elements, and the method includes (i) obtaining a first set of occurrence probabilities for each of the N elements of the second syntax element, (ii) obtaining a second set of occurrence probabilities for each of the N elements of the second syntax element, (iii) encountering the second syntax element while entropy coding the first portion of the video data, (iv) updating the first set of occurrence probabilities for each of the N elements by determining updated occurrence probabilities for the N elements of the first set in accordance with the second syntax element, (v) updating the second set of occurrence probabilities for each of the N elements by determining updated occurrence probabilities for the N elements of the second set in accordance with the second syntax element, and (vi) entropy coding the second portion of the video data in accordance with an aggregation of the updated first and second sets of occurrence probabilities for each of the N elements. In some embodiments, N is greater than M. In some embodiments, N is greater than a threshold number (eg, 16, 12, or 10) and M is less than or equal to a threshold number.

[0108] (A9) In some embodiments of any of A1-A8, the portion of the video data is adaptively entropy coded according to M being less than a threshold number. For example, M is less than 16, 12, or 10.

[0109] (A10) In some embodiments of A9, the method further includes discontinuing adaptive entropy coding the portion of the video data in accordance with M being greater than the threshold number. In some embodiments, in accordance with M being greater than the threshold number, the portion of the video data is coded using the first aggregation of the updated first and second sets.

[0110] (A11) In some embodiments of any of A1-A10, the portion of the video data is adaptively entropy coded in accordance with metadata associated with a first syntax element that enables use of adaptive entropy coding. For example, a flag associated with the first syntax element indicates whether to use adaptive entropy coding.

[0111] The methods described herein may be used separately or combined in any order. Each method may be performed by a processing circuit (e.g., one or more processors or one or more integrated circuits). In some embodiments, the processing circuit executes a program stored in a non-transitory computer-readable medium.

[0112] The term "block" may be interpreted as a prediction block, a coding block, or a coding unit (CU). In some embodiments, the term "block" is used to refer to a transform block. The term "block size" refers to the width or height of a block, the maximum width and height, the minimum width and height, the area size (width*height), and / or the aspect ratio of a block (width:height or height:width). A faster update rate refers to a larger value of the update rate and / or a smaller probability update window size. A slower update rate refers to a smaller value of the update rate and / or a larger probability update window size.

[0113] In another aspect, some embodiments include a computing system (e.g., server system 112) including a control circuit (e.g., control circuit 302) and a memory (e.g., memory 314) coupled to the control circuit, the memory storing one or more instruction sets configured to be executed by the control circuit, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1-A11 above).

[0114] In yet another aspect, some embodiments include a non-transitory computer-readable storage medium storing one or more instruction sets for execution by control circuitry of a computing system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1-A11 above).

[0115] It will be understood that although terms such as "first," "second," and the like may be used herein to describe various elements, these elements are not to be limited by these terms. These terms are used only to distinguish one element from another.

[0116] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the scope of the claims. As used in the description of the embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises" and / or "comprising", as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0117] As used herein, the term "if" can be interpreted to mean "when" or "upon" or "in response to determining" or "in accordance with a determination" or "in response to detecting" a stated precondition is true, depending on the context. Similarly, the phrase "if it is determined that [a stated precondition is true]" or "if [a stated precondition is true]" or "when [a stated precondition is true]" can be interpreted to mean "upon determining" or "in response to determining" or "in accordance with a determination" or "upon detecting" or "in response to detecting" a stated precondition is true, depending on the context.

[0118] The foregoing description has been described with reference to specific embodiments for purposes of explanation. However, the above exemplary description is not intended to be exhaustive or to limit the scope of the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been selected and described in order to best explain the principles of operation and practical application and thereby be accessible to others skilled in the art. [Explanation of symbols]

[0119] 100 Communication Systems 102 Source Device 104 Video Sources 106 Encoder Components 108 Encoded Video Bitstream 110 Network 112 Server System 114 Coda Constituents 116 Encoded Video Data 120 Electronic Devices 122 Decoder Components 124 Display 202 Source Coder 204 Controller 206 Predictors 208 Reference Picture Memory 210 Decoder 212 Coding Engine 214 Entropy Coder 216 Coded Video Sequences 218 Channels 252 Buffer Memory 254 Parser 256 Loop Filter Unit 258 Scaler / Descaler Unit 260 Motion Compensation Prediction Unit 262 Intra-picture Prediction Unit 264 Current Picture Memory 266 Reference Picture Memory 268 Aggregator 270 Symbols 302 Control circuit 304 Network Interface 306 User Interface 308 Output Device 310 Input Devices 314 Memory 316 Operating Systems 318 Network Communication Module 320 Coding Module 324 Analysis Module 326 Conversion Module 328 Prediction Module 330 Filter Module 340 Encoding Module 342 Code Module 344 Prediction Module 352 Picture Memory

Claims

1. A video coding method performed by a computing system having memory and one or more processors, wherein the method is A step of obtaining video data containing multiple syntax elements, wherein the multiple syntax elements include a first syntax element having M elements with corresponding alphabets, A step of obtaining a first set of occurrence probabilities for each of the M elements of the first syntax element, A step of obtaining a second set of occurrence probabilities for each of the M elements of the first syntax element, A step of determining a predefined threshold probability value, wherein a first hypothesis having a probability value smaller than the predefined threshold probability value is used differently from a second hypothesis having a probability value larger than the predefined threshold probability value during decoding of the first syntax element. A step of updating the first set by determining the updated occurrence probabilities of the M elements of the first set according to the first syntax element, wherein the first set is updated at a first rate, A step of updating the second set by determining the updated occurrence probabilities of the M elements of the second set according to the first syntax element, wherein the second set is updated at a second rate that is faster than the first rate. A step of adaptively entropy coding the portion of the video data, The steps of entropy coding the portion of the video data according to the updated first set, such that at least one of the updated probabilities of the second set is less than the predefined threshold probability value, and The step of entropy coding the portion of the video data according to a first aggregation of the updated first and second sets, such that each of the updated probabilities of the second set is at least the predefined threshold probability value. Steps and Methods that include...

2. The method according to claim 1, wherein the step of entropy coding the portion of the video data according to the updated first set includes the step of assigning a weight of 0 to the updated second set in a second aggregation of the updated first and second sets.

3. The method according to claim 1, wherein the step of entropy coding the portion of the video data according to the first updated set includes the step of not using the second updated set to entropy code the portion.

4. The method according to claim 1, wherein the threshold probability value is selected from a set of threshold probability values ​​based on one or more properties of the first syntax element.

5. In accordance with the fact that at least one of the updated probabilities of the second set is less than the threshold probability value, and in accordance with the fact that at least one of the updated probabilities of the first set is less than the threshold probability value, The steps include discarding the updated first set, The method according to claim 1, further comprising the step of entropy coding the portion of the video data according to the first set.

6. The method according to claim 1, wherein the step of entropy coding the portion of the video data according to the first updated set is the step of entropy coding the portion of the video data according to a second aggregation of the first and second updated sets, wherein the second aggregation includes different weights than the first aggregation.

7. The method according to claim 1, further comprising the step of reducing the update rate of the second set in accordance with the fact that at least one of the updated probabilities of the second set is less than the threshold probability value.

8. The plurality of syntax elements include a second syntax element having N elements with corresponding alphabets, and the method is A step of obtaining a first set of occurrence probabilities for each of the N elements of the second syntax element, A step of obtaining a second set of occurrence probabilities for each of the N elements of the second syntax element, A step of updating the first set of occurrence probabilities for each of the N elements by determining the updated occurrence probabilities for the N elements of the first set according to the second syntax element, A step of updating the second set of occurrence probabilities for each of the N elements by determining the updated occurrence probabilities for the N elements of the second set according to the second syntax element, The method according to claim 1, further comprising the step of entropy coding the portion of the video data according to the aggregation of the updated first and second sets of the probability of occurrence of each of the N elements.

9. The method according to claim 1, wherein the portion of the video data is adaptively entropy coded according to the condition that M is less than a threshold number.

10. The method according to claim 9, further comprising the step of discontinuing adaptive entropy coding of the portion of the video data in accordance with the fact that M is greater than the threshold number.

11. The method according to claim 1, wherein the portion of the video data is adaptively entropy coded according to metadata associated with a first syntax element that enables the use of adaptive entropy coding.

12. A computing system configured to perform the method described in any one of claims 1 to 11.

13. A computer program for causing a computing device to perform the method described in any one of claims 1 to 11.

Images