Probability model update adaptation

Abstract

Techniques for video data processing are disclosed. Probabilities of a probability model for a current frame are initialized based on a probability model associated with a reference frame. Further, counts of the probability model for the current frame are initialized based on prior counts from the probability model associated with a reference frame. The probability model for the current frame is subsequently updated based on these initialized counts. The counts may be initialized by applying a function to the prior counts, such as retaining three-quarters of the prior counts.

Description

Atty. Doc. No. GOGL-2284-A-WO PATENTPROBABILITY MODEL UPDATE ADAPTATIONCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This disclosure claims the benefit of U.S. Provisional Patent Application No. 63 / 725,471 filed November 26, 2024, the disclosure of which is incorporated by reference herein in its entirety.BACKGROUND

[0002] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high- definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including compression and other coding techniques. These techniques may include both lossy and lossless coding techniques.SUMMARY

[0003] This disclosure relates generally to encoding and decoding video data and more particularly relates to probability model update adaptation.

[0004] In an aspect of the disclosure, a method for initializing a probability model for a current frame is provided. The method includes initializing probabilities of the probability model for the current frame based on a probability model associated with a reference frame, initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame, and updating the probability model for the current frame based on the counts.

[0005] In another aspect of the disclosure, a method for updating a probability model for a current frame is provided. The method includes updating the probability model for the current frame using a technique selected based on whether the probability model for the current frame was initialized based on a previously updated probability model.

[0006] In another aspect of the disclosure, a device is provided. The device includes a processor configured to perform operations for initializing a probability model for a current frame. The operations include initializing probabilities of the probability model for the current frame based on a probability model associated with a reference frame, initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame, and updating the probability model for the current frame based on the counts.

[0007] In another aspect of the disclosure, a non-transitory computer-readable storage medium is provided. The medium includes executable instructions that, when executed by a processor, facilitate performance of operations. The operations include initializing probabilities of a probability model for a current frame based on a probability model associated with a reference frame, initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame, and updating the probability model for the current frame based on the counts.

[0008] In another aspect of the disclosure, a non-transitory computer-readable storage medium is provided. An encoded bitstream is stored on the non-transitory computer-readable storage medium. The encoded bitstream is generated by an encoder or decodable by a decoder that performs initializing probabilities of a probability model for a current frame based on a probability model associated with a reference frame, initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame, and updating the probability model for the current frame based on the counts.

[0009] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims, and the accompanying figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The description herein refers to the accompanying drawings described below wherein like reference numerals refer to like parts throughout the several views.

[0011] FIG. 1 is a schematic of a video encoding and decoding system.

[0012] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0013] FIG. 3 is a diagram of an example of a video stream to be encoded and subsequently decoded.

[0014] FIG. 4 is a block diagram of an encoder.

[0015] FIG. 5 is a block diagram of a decoder.

[0016] FIG. 6 is a flowchart describing techniques for probability initialization including saving a generated probability model including counts.

[0017] FIG. 7 is a block diagram illustrating a first storage configuration for caching and saving probability models including counts and an associated data flow.

[0018] FIG. 8 is a block diagram illustrating a second storage configuration for saving candidate probability models including counts.

[0019] FIG. 9 is a flowchart describing a first technique for probability model update for a tile based on candidate probability model selection

[0020] FIG. 10 is a flowchart describing a second technique for probability model update for a tile based on candidate probability model selection

[0021] FIG. 11 is a flowchart describing a technique for probability model update for a tile based on use of probability model initialized from a reference frame.

[0022] FIG. 12 is a flowchart describing a technique for initializing and updating a probability model for a current frame.DETAILED DESCRIPTION

[0023] As mentioned above, compression schemes related to coding video streams may include breaking images (i.e., original or source images) into blocks and generating a digital video output bitstream using one or more techniques to limit the information included in the output. A received encoded bitstream can be decoded to re-create the blocks and the source images from the limited information. A video stream may be encoded using a variety of tools resulting in a variety of syntax elements which are stored in a compressed bitstream to enable the transfer of the encoded information between an encoder and a decoder. These syntax elements may be encoded in the compressed bitstream using lossless encoding.

[0024] One type of lossless coding is called entropy coding. Entropy is generally considered the degree of disorder or randomness in a system. Entropy coding is designed to compress a sequence (e.g., including bits representing a syntax element) in an informationally efficient way. A lower bound of the length of the compressed sequence is the entropy of the original sequence. An efficient algorithm for entropy coding aims to generate a code (e.g., in bits) whose length approaches this entropy. For a particular sequence of syntax elements, the entropy associated with the code may be measured as a function of the probability distribution(s) of observations (e.g., symbols, values, outcomes, hypotheses, etc.) for thesyntax elements over the sequence. In some implementations, separate probability distributions may be measured for different syntax elements in order to obtain a more compact set of probability distributions. Arithmetic coding, for example, can use a measured probability distribution to construct a code used to encode the sequence.

[0025] A codec may not receive a sequence together with its actual probability distribution(s). Instead, probability estimation may be used in video codecs to implement entropy coding. The probability distribution(s) in an encoded bitstream (e.g., for a frame or a tile in the encoded bitstream) may be estimated using one or more probability models that model the distribution occurring in an encoded bitstream. A probability model can include multiple sets of probabilities each representing, for example, a probability distribution for a particular syntax element or syntax elements. A probability model can include multiple sets of probabilities for a particular syntax element, where the set of probabilities utilized may depend on a context of previously encoded or decoded information. The probability models are ideally designed so that the estimated probability distribution approaches the actual probability distribution. Using these techniques, entropy coding can reduce the number of bits required to represent the input data to close to a theoretical minimum (i.e., the lower bound). The probability models may be expressed or given by various mathematical functions, including a probability mass function (PMF) or Cumulative Distribution Functions (CDFs).

[0026] Generally, a probability model (e.g., including a set of CDFs) is initialized at the start of a frame or tile to be encoded or decoded. The initialization, for example, may be performed using sets of default probabilities (e.g., where no previously computed probability models are available). The initialization may also be performed by utilizing a probability model from a prior frame, such as a largest tile in a prior frame, to initialize the probability model for the current frame (e.g., which may include the probability models used for all tiles within the current frame). As symbols (e.g., corresponding to particular syntax elements) are encountered during encoding or decoding, the initialized probability model is updated according to the actual data encountered during encoding or decoding. For example, after each encoded or decoded symbol, the set of probabilities that applies to the prior symbol may be updated based on the observation of an additional example of a particular symbol (e.g., the probability of the occurrence of that symbol may be increased). This process repeats (using the same procedure at encoded and decoder) until the end of the frame or tile. The resulting probability model (once the frame or tile is encoded / decoded) is likely more accurate than the initialized probability model because of the observations made during encoding / decoding and the updates that were accordingly made to the probability model. Generally speaking, whentiles are utilized, each tile utilizes its own probability model which is updated separately in the course of encoding / decoding each such tile (because tiles, in many cases, may be encoded or decoded independently of each other).

[0027] As probability models are updated for a tile or frame, counts of updates are maintained for sets of probabilities in the probability model. For example, in some implementations, when a set of probabilities is updated for a particular syntax element or syntax element and context, the associated count (which may also be referred to as an accumulated symbol count) for that set of probabilities for the particular syntax element or syntax element and context is incremented by one. In some implementations the count is capped at a maximum number, such as 32 or 128 (e.g., once the cap is reached, the count may be maintained but not incremented after additional probability updates). The counts may be maintained separately for each set of probabilities in the probability model. For example, in some implementations, the probability model may be implemented as an array of sets of probabilities, where each set of probabilities is stored in an array having a length of the number of probabilities in the set of probabilities plus one, where the count for the set of probabilities is stored in the last element of the array.

[0028] Updated probability models may be utilized to initialize probability models in a later decoded frame. For example, a single probability model associated with a reference frame may be used to initialize the probability models for the tiles in a current frame. For example, the single probability model may be selected from the largest tile in the reference frame, or may be an average or weighted average of a subset or all of the probability models of the tiles of the reference frame. For example, initializing a probability model for a current tile in a current frame may be based on a set of candidate probability models selected from probability models determined for respective tiles in a reference frame and saved to a reference frame buffer. For example, up to a pre-determined maximum number of probability models corresponding to tiles in the reference frame may be saved in the reference frame buffer to permit those probability models to be adaptively utilized when initializing probability models for tiles of the current frame. The selection of these candidate probability models for inclusion in the reference frame buffer may be determined based on a common technique utilized by both encoder or decoder or may be performed by the encoder and identification of the tiles selected (from which the candidate probability models are saved) may be included in the compressed bitstream so that the decoder is able to utilize the same candidate probability models as the encoder.

[0029] Conventionally, the counts associated with a probability model are set to zero at initialization, regardless of whether a probability model is initialized using a default probability model or based on a previously updated probability model (e.g., from a reference frame). Such an approach may lead to problems when utilizing the initialized probability model because the approach used to update the probability model is not dependent on whether a previously updated probability model is used for initialization (in which case the initialized probability model is likely to more closely match the actual probability distribution of the current frame or tile as compared to if initialization was performed using a default probability model). For example, the practice of starting with a count of zero with a consistent probability update technique may lead to probability updates that result in a probability model that deviates further from an actual probability distribution than may be otherwise possible, resulting in a compressed bitstream that utilizes more bits than necessary and resulting in the use of additional computer memory, computation, and network bandwidth than necessary.

[0030] For example, a probability model may be updated more aggressively than may be needed at first (e.g., until a certain number of syntax elements are encoded or decoded), which may result in updated probabilities that deviate from an actual probability distribution more than may otherwise be necessary. This is because, generally speaking, probability updates are designed such that a larger count results in a slower update rate and a smaller count results in a more aggressive (faster) update rate.

[0031] For example, such an implementation may update a set of probabilities according to an implementation consistent with or similar to the following pseudocode: static const int nsymbs2speed

[0017] = { 0, 0, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2 , 2 , 2 , 2 , 2 } ; const int time_interval = cdf [nsymbs] > 31 ? 2 : cdf [nsymbs] > 15 ? 1 : 0; rate = 3 + time_interval + nsymbs2speed [nsymbs] + para_adjustment_list [cdf [nsymbs + 1] ] [time_interval] ; int i = 0 ; do { if (i < val) { cdf [i] += ( (1<<15) - cdf [i] ) >> rate; } else { cdf [i] -= cdf [i] >> rate;}} while (++i < nsymbs - 1) ; cdf [nsymbs] += (count < 32) ;

[0032] In the foregoing pseudocode, the first nysmbs elements of the cdf array includes cumulative distribution functions (e.g., probabilities), which for a given cdf [i] are expressed as ( 1« 15)-cdf_i where cdf_i is the probability of a random variable being less than or equal to i. The last element of the cdf array (cdf [nsymbs]) stores the count of updates made to the cdf array. The cdf array corresponds to a set of probabilities of a probability model as described herein. The nsymbs variable is an integer indicating the number of symbols for which the cdf provides probabilities for. The nysmbs2speed array may instead be implemented with Min( FloorLog2( nsymbs ), 2 ) where FloorLog2(n) is defined to be the floor of the base 2 logarithm of the input nsymbs. The para_adjustment_list array may be a pre- determined lookup table. In some implementations, the para_adjustment_list array may be implemented as follows: para_adjustment_list [NUM_PARA_COMBINATIONS] [NUM_PARA_INTERVALS] = {{ 0, 0, 0 }, { 0, 0, -1 }, { 0, 0, -2 }, { 0, 0, 1 },{ 0, 0, 2 }, { 0, -1, 0 }, { 0, -1, -1 }, { 0, -1, -2 },{ 0, -1, 1 }, { 0, -1, 2 }, { 0, -2, 0 }, { 0, -2, -1 },{ 0, -2, -2 }, { 0, -2, 1 }, { 0, -2, 2 }, { 0, 1, 0 },{ 0, 1, -1 }, { 0, 1, -2 }, { 0, 1, 1 }, { 0, 1, 2 },{ 0, 2, 0 }, { 0, 2, -1 }, { 0, 2, -2 }, { 0, 2, 1 },{ 0, 2, 2 }, { -1, 0, 0 }, { -1, 0, -1 }, { -1, 0, -2 },{ -1, 0, 1 }, { -1, 0, 2 }, { -1, -1, 0 }, { -1, -1, -1 },{ -1, -1, -2 }, { -1, -1, 1 }, { -1, -1, 2 }, { -1, -2, 0 },{ -1, -2, -1 }, { -1, -2, -2 }, { -1, -2, 1 }, { -1, -2, 2 },{ -1, 1, 0 }, { -1, 1, -1 }, { -1, 1, -2 }, { -1, 1, 1 },{ -1, 1, 2 }, { -1, 2, 0 }, { -1, 2, -1 }, { -1, 2, -2 },{ -1, 2, 1 }, { -1, 2, 2 }, { -2, 0, 0 }, { -2, 0, -1 },{ -2, 0, -2 }, { -2, 0, 1 }, { -2, 0, 2 }, { -2, -1, 0 },{ -2, -1, -1 }, { -2, -1, -2 }, { -2, -1, 1 }, { -2, -1, 2 },{ -2, -2, 0 }, { -2, -2, -1 }, { -2, -2, -2 }, { -2, -2, 1 },{ -2, -2, 2 }, { -2, 1, 0 }, { -2, 1, -1 }, { -2, 1, -2 },{ -2, !, ! }, { -2, 1, 2 }, { -2, 2, 0 }, { -2, 2, -1 },{ -2, 2, -2 }, { -2, 2, 1 }, { -2, 2, 2 }, { 1, 0, 0 },{ 1, 0, -1 }, { 1, 0, -2 }, { 1, 0, 1 }, { 1, 0, 2 },{ 1, -1, 0 }, { 1, -1, -1 }, { 1, -1, -2 }, { 1, -1, 1 },{ 1, -1, 2 }, { 1, -2, 0 }, { 1, -2, -1 }, { 1, -2, -2 },{ 1, -2, 1 }, { 1, -2, 2 }, { 1, 1, 0 }, { 1, 1, -1 },{ 1, 1, -2 }, { 1, 1, 1 }, { 1, 1, 2 }, { 1, 2, 0 },{ 1, 2, -1 }, { 1, 2, -2 }, { 1, 2, 1 }, { 1, 2, 2 },{ 2, 0, 0 }, { 2, 0, -1 }, { 2, 0, -2 }, { 2, 0, 1 },{ 2, 0, 2 }, { 2, -1, 0 }, { 2, -1, -1 }, { 2, -1, -2 },{ 2 , -1 , 1 } , { 2 , -1 , 2 } , { 2 , -2 , 0 } , { 2 , -2 , -1 } ,{ 2 , -2 , -2 } , { 2 , -2 , 1 } , { 2 , -2 , 2 } , { 2 , 1 , 0 } ,{ 2 , 1 , -1 } , { 2 , 1 , -2 } , { 2 , 1 , 1 } , { 2 , 1 , 2 } ,{ 2 , 2 , 0 } , { 2 , 2 , -1 } , { 2 , 2 , -2 } , { 2 , 2 , 1 } ,{ 2 , 2 , 2 } ,

[0033] Problems such as those described above may be solved by implementations that adapt how probabilities are updated based on whether the probability model for a tile or frame is initialized based on a previously updated probability model, such as from a reference frame or by implementations that initialize counts for an initialized probability model for a current frame based on counts from or associated with a probability model use to initialize the initialized probability model.

[0034] For example, in some implementations, the counts for an initialized probability model for a current frame or tile may be copied from or otherwise obtained from the counts of the probability model (or generated probability model or other metadata) obtained from the reference frame from which the probability model is initialized.

[0035] For example, in some implementations, the counts for an initialized probability model for a current frame or tile may be based on the counts of the probability model (or generated probability model or other metadata) obtained from the reference frame from which the probability model is initialized. For example, the initialized counts may be half or three quarters of the counts obtained from the reference frame, or some other modified number obtained using a formula used by both the encoder and decoder. In some implementations, the same calculation may be utilized regardless of the values of the counts obtained from the reference frame (e.g., if the prior count value is zero or a default count value of zero is utilized, the same calculation may be utilized to, e.g., arrive at a zero value to initialize the count). This may have the effect of simplifying an implementation even though, in such a case, the calculation does not result in a change in the zero-valued count.

[0036] For example, in some implementations, the counts for a current frame or tile are initialized at zero, but the probability update technique is adapted based on whether or not the initialization of the probability model utilizes a previously updated probability model from a prior frame, such as a reference frame. For example, if initialization is performed using a previously updated probability model, probability updates may be made in an attenuated manner as compared to a typical probability model update technique, for example, at half or less of rate that may be typically utilized when updating a probability model initialized using default probabilities.

[0037] In some implementations, a hybrid approach may be utilized such as by combining aspects of the previously described implementations.

[0038] Further details of probability model update adaptation are described herein with initial reference to a system in which it can be implemented. FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG.2. However, other suitable implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0039] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102 and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0040] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0041] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, e.g., a Hypertext Transfer Protocol (HTTP) video streaming protocol.

[0042] When used in a video conferencing system, for example, the transmitting station 102 and / or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a videoconference server (e.g., the transmitting station 102) to decode and view and further encodes and transmits its own video bitstream to the video conference server for decoding and viewing by other participants.

[0043] FIG. 2 is a block diagram of an example of a computing device 200 (e.g., an apparatus) that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0044] A CPU 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the CPU 202 can be any other type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. Although the disclosed implementations can be practiced with one processor as shown, e.g., the CPU 202, advantages in speed and efficiency can be achieved using more than one processor.

[0045] A memory 204 in computing device 200 can be a read only memory (ROM) device or a random-access memory (RAM) device in an implementation. Any other suitable type of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the CPU 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the CPU 202 to perform the techniques described here. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described here. Computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device. Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0046] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the CPU 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, thedisplay can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display or light emitting diode (LED) display, such as an organic LED (OLED) display.

[0047] The computing device 200 can also include or be in communication with an image-sensing device 220, for example a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0048] The computing device 200 can also include or be in communication with a sound-sensing device 222, for example a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0049] Although FIG. 2 depicts the CPU 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the CPU 202 can be distributed across multiple machines (wherein individual machines can have one or more of processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0050] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individualframes, e.g., a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 (e.g., which may also be referred to as tiles) can be subsets of frames that permit parallel processing, for example. The configuration of tiles in a frame may vary depending on the implementation, and may take the form of columns, rows, rectangular areas, or other collections of blocks, depending on the implementation. The use of tiles may be configured such that encoding a tile does not have dependencies on or has limited dependencies on other tiles to permit tiles to be decoded independently of each other. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0051] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macro-block are used interchangeably herein.

[0052] FIG. 4 is a block diagram of an encoder 400. The encoder 400 can be implemented, as described above, in the transmitting station 102 such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the CPU 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0053] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0054] When the video stream 300 is presented for encoding, respective frames 304, such as the frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a prediction block can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0055] Next, still referring to FIG. 4, the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated. The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block, which may include for example the type of prediction used, transform type, MVs and quantizer value, are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0056] The reconstruction path in FIG. 4 (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process that are discussed in more detail below, including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0057] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. For example, a non-transform-based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In another implementation, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0058] FIG. 5 is a block diagram of a decoder 500. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the CPU 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0059] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512 and a post-loop filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0060] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients and other decoded syntax elements needed for the decoding process. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create the same prediction block as was created in the encoder 400, e.g., at the intra / inter prediction stage 402. At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts.

[0061] As can be appreciated from the description of the encoder 400 and the decoder 500 above, bits are generally used for content prediction (e.g., inter mode / motion vectorcoding, intra prediction mode coding, etc.), residual or coefficient coding (e.g., transform coefficients), or other side information needed to decode the compressed bitstream. Encoders may use techniques to decrease the bits spent on representing this data. For example, a coefficient token tree (which may also be referred to as a binary token tree) may specify the scope of the value, with forward-adaptive probabilities for each branch in this token tree. The token base value is subtracted from the value to be coded to form a residual, then the block is coded using the probabilities applicable for that residual. A similar scheme with minor variations including backward-adaptivity is also possible. Adaptive techniques can alter the probability models as the video stream is being encoded to adapt to changing characteristics of the data. In any event, a decoder is informed of (or has available) the probability model used to encode an entropy-coded video bitstream so the decoder can decode the video bitstream.

[0062] That is, and as described initially above, a video codec may use arithmetic coding to effectuate the entropy coding of syntax elements (such as the data referenced above). The coding efficiency is dependent on the accuracy of the probability model used for the tiles and / or frames in the video bitstream. The probability model may be, for example, represented by a PMF or a CDF for each or groups of the various syntax elements in the bitstream. Additional PMFs or CDFs may be provided for a given syntax element(s) which may be selected depending on context. For example, a probability model may include a large number of CDFs each representative of the current probability distribution for a given syntax element / context combination.

[0063] FIG. 6 is a flowchart describing techniques 600 for probability initialization. Techniques 600 may be performed by an encoder or decoder, such as encoder 400 using entropy encoding stage 408 or decoder 500 using entropy decoding stage 502.

[0064] At step 602, a probability model for a current frame is initialized from a probability model stored in a reference frame buffer (RFB). For example, a reference frame buffer may include storage for information about multiple reference frames (e.g., four or seven), and for each such reference frame, the decoded reference frame and a probability model for the reference frame may be stored. A syntax element may be included in a compressed bitstream identifying which reference frame to utilize for a current frame (e.g., in the case of a decoder) or the reference frame to utilize for a current frame may be determined (e.g., in the case of an encoder). The saved probability model corresponding to the identified or determined reference frame can be used to initialize the probability model for the current frame. In certain implementations, the same initialized probability model is used for all tilesin the current frame. In some implementations, initializing the probability model for the current frame includes initializing counts for the probability model for the current frame based on prior counts obtained based on updating the probability model stored in the RFB (e.g., a previously updated probability model). In such an implementation, the prior counts may be stored in the RFB with the stored probability model and may be retrieved along with the stored probability model to initialize the counts.

[0065] At step 604, current tiles in the current frame are encoded or decoded (e.g., depending on whether techniques 600 are performed by an encoder or decoder) using steps 606-612. The group of steps 606-612 may be performed for each current tile concurrently, in parallel, or some combination thereof, depending on the implementation.

[0066] At step 606, a symbol is encoded or decoded using the probability model for the tile. The symbol is a value corresponding to a syntax element in a compressed bitstream. If no prior syntax elements have been encoded or decoded, the probability model is the same as the initialized probability model from step 602. If prior syntax elements have been encoded or decoded the probability model has been updated (perhaps many times) by step 608 and has changed according to the actual occurrences of symbols encoded or decoded for the current tile. For example, a syntax element may represent an x component of a motion vector and the symbol may indicate the value of the x component.

[0067] At step 608, the probability model is updated based on the occurrence of the symbol encoded or decoded at step 606. For example, a count of the symbol for the associated syntax element may be incremented and a set of probabilities for that syntax may be updated based on the updated count.

[0068] At step 610, if there are more syntax elements to be encoded or decoded for the current tile, control passes back to step 606 to encode or decode the next syntax element. If all syntax elements have been encoded or decoded, control passes to step 612.

[0069] At step 612, a determination is made as to whether to cache the probability model for the current tile. In some implementations, the determination is made by a technique common to both the encoder and decoder. For example, if a maximum of four probability models may be cached, the probability models from the four largest tiles may be cached or the four probability models having the largest number of probability updates may be cached. In another example, probability models may be cached based on tile location.

[0070] In some implementations, probability models may be cached along with an indication of the tile from which the probability model was cached or information needed to determine whether a probability model from a later encoded or decoded tile should be cachedinstead of the previously cached probability model. For example, the probability models for the first four tiles encoded or decoded for a current frame may be cached along with an indication of those tiles sizes. When step 612 is reached for the following encoded or decoded tiles, the size of those tiles may be compared to the size of the tiles from which the previously cached probability models were obtained. If the later tile is larger in size than the prior tiles from which the cached probability models were obtained, the probability model from the later tile may be cached instead of the previously cached probability model associated with the smallest tile.

[0071] In some implementations, probability models may be cached including counts associated with the sets of probabilities of the probability model. For example, as the probabilities in the probability model are updated (e.g., at step 608), counts associated with the encountered syntax elements may be incremented. In some implementations, the counts may be incremented up to a maximum value such as 32 or 128. Each set of probabilities may have its own separate count (e.g., the count for that set of probabilities is only incremented when the set of probabilities is applied to encode or decode a syntax element). In some implementations, the sets of probabilities for a probability model are stored in an array having a length equal to the number of probabilities in each set of probabilities plus one. The count associated with a set of probabilities may be stored in the last element of the array for the set of probabilities. In such an implementation, the entire array for each of the sets of probabilities of the probability model may be cached, including the count stored in each array.

[0072] Control passes to step 614 once step 604 is completed (e.g., all tiles have completed steps 606-612). At step 614, a probability model is generated that is stored in a reference frame buffer. For example, a combined probability model is generated from the cached probability models. The combined probability model may be generated, for example, by averaging or weighted averaging the cached probability models.

[0073] The combined probability model may include counts associated with the sets of probabilities in the combined probability model. For example, the counts may be averaged using the same or similar function used to average the probabilities (e.g., an average or weighted average). In some implementations, a different function may be used to obtain a generated count for each of the sets of probabilities in the combined probability model. For example, the generated counts may be calculated as half or three quarters of the average or weighted average of the counts of the cached probability models.

[0074] At step 616, the current frame and combined probability model is saved into a reference frame buffer. The combined probability model, as described with respect to step 614, may include counts associated with sets of probabilities in the combined probability model. For example, if the reference frame buffer has room for multiple reference frames, the current frame buffer and combined probability model may be saved in a location identified by a number x, e.g., RFB[x]. In some implementations, the encoder may determine whether or where to store the current frame and combined probability model in the RFB and this determination may be included in the compressed bitstream so that the decoder may do the same. In the case where the reference frame is not stored in the reference frame buffer, steps 614 and 616 may be skipped for a given current frame.

[0075] At step 618, if there are more frames, control passes back to step 602 to initialize the probability model for the next frame.

[0076] Variations of technique 600 and the description of FIG. 6 are possible. For example, in some implementations, counts are not generated in step 614, generated counts are not saved in the RFB in step 616, and counts are not initialized using prior counts in step 602. In such an implementation, counts may instead be initialized to zero in step 602. In such an implementation, step 608 may be modified such that updating probabilities of the probability model is performed differently based on whether the probability model for the current frame was initialized based on a previously updated probability model. For example, a first technique may be selected to update the probability model if the probability model for the current fame is initialized based on a previously updated probability model and a second technique may be selected to update the probability model otherwise.

[0077] For example, in such an implementation, the first technique may include updating a probability of the probability model for the current frame by a smaller amount than would be utilized if the probability model for the current frame was not initialized based on a previously updated probability model. For example, in such an implementation, the first technique may include updating a probability of the probability model for the current frame by an amount determined based on a measure of correlation between the current frame and a reference frame from which the previously updated probability model is associated. In such an implementation, the measure of correlation may be determined based on a display order difference between the current frame and the reference frame, the display order difference may be based on an encoding order, a slower adaptation rate may be utilized when the reference frame is adjacent to the current frame, or a combination thereof. In some implementations, the amount may be determined based on the measure of correlationaccording to intervals applied to a display order difference between the current frame and the reference frame. In some implementations, the first technique may include updating a probability of the probability model adaptively based on whether the current frame, tile, or block is encoded using inter coding or intra coding

[0078] FIG. 7 is a block diagram illustrating a first storage configuration for caching and saving probability models and an associated data flow. FIG. 7 includes a first storage 702 and a second storage 750. First storage 702 may be implemented using a faster storage mechanism, such as SRAM on a hardware encoder or decoder. Second storage 750 may be implemented using a slower storage mechanism, such as DRAM connected to a hardware encoder or decoder.

[0079] First storage 702 may include probability model cache storage areas 704, 706 (e.g., n storage areas as depicted) and multiple metadata storage areas 710, 712 (e.g., n storage areas as depicted). Probability model cache storage areas 704, 706 may be utilized to store cached probability models that, e.g., are cached at step 612 of FIG. 6. Metadata storage areas 710, 712 may be utilized to store additional information relating to the cached probability models that may be utilized by the encoder or decoder to determine how to cache probability models (e.g., such as the size of the tile, number of probability updates, or location of the tile from which the associated cached probability model is obtained). For example, the information in metadata[0] may correspond to the probability model in probability model cache[0]. In some implementations, metadata storage areas 710, 712 may be omitted or implemented differently, for example if identifications of the tiles to use for caching probability models are determined by the encoder and transmitted in the compressed bitstream. In some implementations, metadata storage areas 710, 712 may be omitted, counts may be stored with the probability model in probability model cache storage areas 704, 706 (e.g., in arrays such as previously described), or combinations thereof.

[0080] Second storage 750 may include a reference frame buffer (RFB) that includes multiple storage areas 760, 770 (e.g., as shown, n storage areas) for storing reference frame data such as frame data 762, 772 and probability model including counts 764, 774. For example, one of the storage areas 760, 770 may be used to store the frame data and combined probability model stored in step 616 of FIG. 6. For example, the probability model including counts 764, 774 may store both the sets of probabilities for the probability model and counts associated with each of the sets of probabilities.

[0081] Data stored in first storage 702 may be accessed and utilized in order to execute technique 790 of generating a combined probability model for the RFB and savingthe probability model into RFB[x] (e.g., one of storage areas 760, 770) which is in second storage 750. For example, technique 790 may include copying counts from one or more of probability model cache storage areas 704, 706, generating counts to be saved into probability model including counts 764, 774, or combinations thereof. Technique 790 may, for example, correspond to steps 614 and 616 of FIG. 6. As previously described, generating the probability model (e.g., a combined probability model) may be performed using an average or weighted average of probability models. It is advantageous for the average, weighted average, or other combination of probability models to be performed using the cached probability models stored in first storage 702 because first storage 702 is implemented using memory with a faster access speed, such as SRAM (as compared to, for example, DRAM for second storage 750). By comparison, performing the combination using information stored in DRAM or other slower storage may disadvantageously increase the time required to perform the combination such that the encoding or decoding process will be delayed.

[0082] FIG. 8 is a block diagram illustrating a second storage configuration for saving candidate probability models. FIG. 8 includes second storage 800. Second storage 800 may be implemented using a slower storage mechanism than SRAM, such as DRAM connected to a hardware encoder or decoder.

[0083] Second storage 800 may include a reference frame buffer (RFB) that includes multiple storage areas 810, 820 (e.g., as shown, n storage areas) for storing reference frame data such as frame data 812, 822 and probability models[] (PM[]) 814, 816, 824, 826. The PM[] as shown for the storage areas 810, 820 are configured to permit the storage of up to n candidate probability models for each storage area in the RFB . In some implementations, the PM[] 814, 816, 824, 826 may store both the sets of probabilities for the respective probability model PM[x] and counts associated with each of the sets of probabilities for the respective probability model PM[x],

[0084] Technique 850 may be utilized to save probability model(s) into one of the storage areas 810, 820 (e.g., identified by x in FIG. 8). For example, in some implementations, with respect to FIG. 6, technique 850 may replace step 612 and steps 614 and 616 may be omitted. In such an implementation, the encoder and / or decoder may determine the tiles from which the candidate models will be obtained from. This, for example, may be an encoder determination based on, e.g., a rate distortion analysis, the result of which (e.g., tile indexes from which the candidate probability models are saved) may beincluded in the compressed bitstream so that during decoding, the decoder may also save the same probability models as candidate models.

[0085] In other implementations, technique 850 may utilize a modification of steps 612, 614, and / or 616 of FIG. 6. For example, step 614 may be configured to generate multiple combined probability models from a reference frame that are separately saved as candidate probability models. For example, certain areas of a reference frame or groups of tiles in the reference frame may separately have probability models cached by step 612, combined by step 614, and saved by step 616 in distinct candidate probability model slots in the PM[] array. In such implementations, second storage 800 may replace second storage 750 as described above with respect to FIG. 7. In some implementations, PM[] 814, 816, 824, 826 may not include counts and technique 850 may omit saving counts into PM[] 814, 816, 824, 826.

[0086] Further variations of the above are possible, including hybrid arrangements between the implementations described above.

[0087] FIG. 9 is a flowchart describing a first technique 900 for probability model update for a tile based on candidate probability model selection. Technique 900 may be performed by an encoder or decoder, such as encoder 400 using entropy encoding stage 408 or decoder 500 using entropy decoding stage 502.

[0088] At step 902, a probability model is initialized for a next tile in a current frame. Step 902 takes as input reference frame selection information 920 (also referred to as RFS) and model selection information 930 (also referred to as MSI) and utilizes RFB 910 which includes storage areas including storage area 912 and for each storage area a candidate probability model array such as depicted by probability model[0.. .n] 914, 916. Probability model[0.. .n] 914, 916 may include counts associated with sets of probabilities includes in the probability models stored in probability model[0...n] 914, 916. The candidate probability model used to initialize the next tile in the current frame is obtained from the RFB using RFS and MSI, such as by using the RFS and MSI as indexes to the RFB data structure:RFB [RFS] [MSI]. The probability model for the next tile is initialized using the obtained candidate probability model. Step 902 may include initializing counts associated with the probability model (e.g., which may include a count for each set of probabilities included in a probability model) based on the counts stored in probability model[0.. .n] 914, 916. For example, counts may be initialized according to or similar to implementations described above with respect to FIG. 6 or later with respect to FIG. 11.

[0089] At step 904, the tile is decoded using the initialized probability model with a probability update using the initialized counts. The probability update using the initialized counts may vary depending on the implementation. Examples of such implementations are described elsewhere, such as with respect to FIGS. 6 and 11. This step and step 902 may be performed in a non-blocking manner. In other words, control may pass to step 906 before steps 902 and / or 904 completes. In some implementations, control may block (e.g., pause) if a maximum number of parallel executions of steps 902 and / or 904 have been reached.

[0090] At step 906, if there are more tiles to be encoded or decoded in the current frame, control passes back to step 902.

[0091] FIG. 10 is a flowchart describing a first technique 1000 for probability model update for a tile based on candidate probability model selection. Technique 1000 may be performed by an encoder or decoder, such as encoder 400 using entropy encoding stage 408 or decoder 500 using entropy decoding stage 502.

[0092] At step 1002, a probability model is initialized for a next tile in a current frame. Step 1002 takes as input reference frame selection information 1020 (also referred to as RFS) and model selection information 1030 (also referred to as MSI) and utilizes RFB 1010 which includes storage areas including storage area 1012 and for each storage area a candidate probability model array such as depicted by probability model[0.. .n] 1014, 1016. Probability model[0.. .n] 1014, 1016 does not include counts associated with sets of probabilities included in the probability models stored in probability model[0.. .n] 1014, 1016. The candidate probability model used to initialize the next tile in the current frame is obtained from the RFB using RFS and MSI, such as by using the RFS and MSI as indexes to the RFB data structure: RFB [RFS] [MSI]. The probability model for the next tile is initialized using the obtained candidate probability model. Step 1002 may include initializing counts associated with the probability model (e.g., which may include a count for each set of probabilities included in a probability model) to zero.

[0093] At step 1004, the tile is decoded using the initialized probability model with a modified probability update. The modified probability update may vary depending on the implementation. For example, the modified probability update may operate differently depending on whether the probability model for the tile was initialized using a previously updated probability model (e.g., as obtained from probability model[0...n] 1014, 1016. Further variations are possible, depending on the implementation. Examples of such implementations are described elsewhere, such as with respect to FIGS. 6 and 11. This step and step 1002 may be performed in a non-blocking manner. In other words, control may passto step 1006 before steps 1002 and / or 1004 completes. In some implementations, control may block (e.g., pause) if a maximum number of parallel executions of steps 1002 and / or 1004 have been reached.

[0094] At step 1006, if there are more tiles to be encoded or decoded in the current frame, control passes back to step 1002.

[0095] Variations of or combinations of techniques 900 and 1000 and the description with respect to FIGS. 9 and 10 are possible depending on the implementation. For example, hybrid techniques that combine aspects of techniques 900 and 1000 are possible, including those that both incorporate the use of counts generated or obtained based on or from counts associated with a probability model stored associated with a reference frame and that utilize a modified probability update dependent on whether a probability model for the current tile is initialized based on a previously updated probability model (e.g., as obtained from probability model[0...n] 1014, 1016).

[0096] FIG. 11 is a flowchart describing a technique 1100 for probability model update for a current frame based on use of probability model initialized from a reference frame. Technique 1100 may be performed by an encoder or decoder, such as encoder 400 using entropy encoding stage 408 or decoder 500 using entropy decoding stage 502.

[0097] Step 1102 includes updating the probability model for the current frame using a technique selected based on whether the probability model for the current frame was initialized based on a previously updated probability model. For example, in some implementations, step 1102 may operate according to or using techniques described with respect to FIGS. 6 (e.g., including step 608), 9 (e.g., including step 904), 10 (e.g., including step 1004), steps thereof, or modifications thereof. In some implementations, technique 1100 may include additional steps in addition to updating the probability model, including steps described or consistent with description provided elsewhere in this disclosure, including with respect to FIGS. 6-10.

[0098] In some implementations, the probability model for the current frame is for a current tile in the current frame. For example, in some implementations a probability model is initialized for and updated with respect to a particular tile in the current frame (e.g., including a current tile). This may be repeated for some or all of the tiles in the current frame.

[0099] In some implementations, the probability model for the current frame was initialized based on a previously updated probability model and the selected technique for updating the probability model for the current frame includes initializing counts for the probability model for the current frame based on prior counts obtained based on updating thepreviously updated probability model. For example, initialization of counts may be performed such as described with respect to step 602, step 902, or step 1002. For example, the previously updated probability model may be obtained from a reference frame buffer (e.g., such as RFB 752, RFB 802, storage area 912, storage area 1012) and may include counts obtained from when the previously updated probability model was updated (e.g., at step 608, step 904) or generated (e.g., at step 614, technique 790).

[0100] In some implementations, initializing the counts for the probability model for the current frame includes initializing the counts using outputs obtained by applying a function to the prior counts. For example, in some implementations, the function obtains an output of the outputs by dividing a count of the prior counts by two or by multiplying a count of the prior counts by three quarters. Multiplying the prior counts by three quarters may be implemented, for example, by multiplying the prior count by three and bit shifting right by two, such as described below with respect to FIG. 12. For example, in some implementations, the function obtains an output of the outputs based on a display order difference between the current frame and the reference frame from which the previously updated probability model is associated. In some implementations, the selected technique includes updating a probability of the probability model by an amount determined based on a current count associated with the probability model for the current frame that has been updated from the initialized counts for the probability model for the current frame based on occurrences of a symbol encountered while encoding or decoding the current frame.

[0101] For example, in some implementations, the function obtains outputs used to initialize the counts based on a display order difference between the current frame and the reference frame. For example, in some implementations utilizing a technique similar to or consistent with the previously disclosed pseudocode, the calculation of a variable symbol_count representing as input a symbol count for a set of probabilities obtained based on a previously updated probability model and as output the initialized symbol count for a set of probabilities of an initialized probability model for a current frame or tile may be varied depending on the display order difference. For example, if a display order difference is greater than 2, the symbol_count may be reduced by half: display_order_dif f erence = abs ( current_f rame_display_order - primary_ref_f rame_display_order ) ; if ( display order difference > 2 ) { symbol_count = symbol_count / 2 ;} else { symbol_count = symbol_count ;}

[0102] A different display order difference limit may be utilized and a different function may be used to obtain symbol_count. For example, in some implementations, the output symbol_count is obtained by multiplying symbol_count by % instead of by dividing by two.

[0103] In some implementations, the probability model for the current frame was initialized based on a previously updated probability model and the selected technique includes updating a probability of the probability model for the current frame by a smaller amount than would be utilized if the probability model for the current frame was not initialized based on a previously updated probability model. For example, in such implementations, the selected technique may include initializing counts for the probability model for the current frame to zero.

[0104] In some implementations, the probability model for the current frame was initialized based on a previously updated probability model and the selected technique includes updating a probability of the probability model for the current frame by an amount determined based on a measure of correlation between the current frame and a reference frame from which the previously updated probability model is associated. For example, in such implementations, the measure of correlation may be determined based on a display order difference between the current frame and the reference frame. For example, in such implementations, the display order difference may be based on an encoding order. For example, in such implementations, a slower adaptation rate is utilized when the reference frame is adjacent to the current frame. A slower adaptation rate means that probabilities are updated at a slower rate (e.g., less than) when the current frame and reference frame are adjacent compared to when they are not. For example, a current frame and reference frame may be adjacent when the current frame is immediately after the reference frame in decoding order, the current frame is immediately prior to or following the reference frame in display order, or some combination thereof.

[0105] For example, in some implementations, the amount is determined based on the measure of correlation according to intervals applied to a display order difference between the current frame and the reference frame. For example, in some implementations utilizing a technique similar to or consistent with the previously disclosed pseudocode, the calculation of time_interval may be varied depending on the display order difference. For example, if a display order difference is greater than 2, 15 and 7 may be used as thresholds, whereas for smaller display order differences, 7 and 3 may be used as thresholds:display_order_dif f erence = abs ( current_f rame_display_order - primary_ref_f rame_display_order ) ; int thrl = ( display order difference > 2 ) ? 15 : 7 ; int thr2= ( display order difference > 2 ) ? 7 : 3 ; const int time_interval = cdf [ nsymbs ] > thrl ? 2 : cdf [ nsymbs ] > thr2 ? 1 : 0 ;

[0106] A different display order difference limit may be utilized and different thresholds may be utilized, depending on the implementation.

[0107] In some implementations, the selected technique updates a probability of the probability model adaptively based on whether the current frame, tile, or block is encoded using inter coding or intra coding. For example, in some implementations utilizing a technique similar to or consistent with the previously disclosed pseudocode, the calculation of time_interval may be varied depending on whether the current frame (or tile) is an intra frame (or tile) or inter frame (or tile). For example, for intra coding, 31 and 15 may be used as thresholds, whereas for inter coding, 7 and 3 may be used as thresholds: / / I f frame or tile utili zes intra coding : const int time_interval = cdf [ nsymbs ] > 31 ? 2 : cdf [ nsymbs ] > 15 ? 1 : 0 ; / / I f frame or tile utili zes inter coding : const int time_interval = cdf [ nsymbs ] > 7 ? 2 : cdf [ nsymbs ] > 3 ? 1 : 0 ;

[0108] Different thresholds may be utilized, depending on the implementation.

[0109] In some implementations, the previously updated probability model is at least one of a probability model generated by averaging or weighted averaging a plurality of probability models of a reference frame, a candidate probability model selected for a current tile of the current frame, or a probability model obtained from a reference tile of the reference frame. The previously updated probability model may be generated, updated, or obtained using other techniques, such as using a probability model from a largest tile in a reference frame or a prior frame.

[0110] In some implementations, implementations described with respect to one or more of the previous figures may be modified according to or consistent with implementations described with respect to FIG. 11.

[0111] FIG. 12 is a flowchart describing a technique 1200 for initializing and updating a probability model for a current frame. Technique 1200 may be performed by an encoder or decoder, such as encoder 400 using entropy encoding stage 408 or decoder 500 using entropy decoding stage 502.

[0112] At step 1202, probabilities of a probability model for the current frame are initialized based on a probability model associated with a reference frame. For example, as described above, the probability model may include sets of probabilities respectively corresponding to a syntax element or a syntax element in a particular context. In such an example, the probabilities refers to one or more of the sets of probabilities. An individual probability in a set of probabilities may indicate the likelihood that a particular value corresponding to the probability will be coded in a corresponding syntax element. The sets of probabilities may take the form of CDFs, such as previously described.

[0113] In some implementations, the probability model for the current frame is for a current tile in the current frame. In such an implementation, the probability model for the current frame may be selected and / or used on a tile-by-tile basis for entropy encoding or entropy decoding syntax elements corresponding to a given tile.

[0114] The reference frame may be selected based on reference frame selection information decoded from an encoded bitstream. In some implementations, the probability model associated with the reference frame is selected from candidate probability models stored in a reference frame buffer storage area selected based on the reference frame selection information. For example, the reference frame selection information may be utilized to access an array of candidate probability models corresponding to the selected reference frame. The combined probability model may be selected based on model selection information. The probability model associated with the reference frame may be selected from the candidate probability models based on the model selection information.

[0115] The probability model associated with the reference frame may be a combined probability model generated by averaging or weighted averaging probability models of the reference frame, a candidate probability model selected for a current tile of the current frame, or a probability model obtained from a reference tile of the reference frame. In some implementations, a first probability model of the plurality of probability models is associated with a first tile of the reference frame, and the first probability model includes a probability and a prior count that were updated during encoding or decoding of the first tile. Other variations of initializing the probabilities of the probability model for the current frame are possible, including by incorporating some or all of the alternative techniques for initializing a probability model that have previously been described.

[0116] In some implementations, the initialization of probabilities described in step 1202 may be performed as part of a broader probability initialization process, such as the process described with respect to FIG. 6. For example, step 1202 may correspond to theinitialization of a probability model for a current frame from a reference frame buffer at step 602 of FIG. 6. Alternatively, when processing individual tiles, step 1202 may be implemented according to the per-tile initialization shown in step 902 of FIG. 9 where a specific candidate probability model is selected from a reference frame buffer.

[0117] At step 1204, counts of the probability model for the current frame are initialized based on prior counts associated with the probability model associated with the reference frame. For example, if the probability model associated with the reference frame is obtained from the decoding of a tile of the reference frame, the prior counts are the counts obtained as a result of the decoding of the tile (e.g., by updating the probability model associated with the reference frame as syntax elements were decoded for the tile). In other examples, the probability model associated with the reference frame is generated or constructed based on multiple probability models from the reference frame and the prior counts associated with the probability model associated with the reference frame may also be generated or constructed (e.g., in a similar or different way from how the probabilities of the probability model were generated or constructed). For example, the counts from the multiple probability models may be averaged (e.g., on a per probability basis).

[0118] Where the probability model associated with the reference frame is a combined probability model, it may include counts generated as a function of respective counts associated with probability models of the plurality of probability models of the reference frame. For example, the function of respective counts may be an average of the respective counts.

[0119] Initializing the counts for the probability model for the current frame based on prior counts may include applying a function to the prior counts. For example, the function may include dividing a count of the prior counts by two or multiplying a count of the prior counts by three quarters. In a specific implementation, multiplying by three quarters may be implemented by multiplying the prior counts by three to produce an intermediate result and bit shifting the intermediate result right by two.

[0120] For example, in some implementations, the counts for the probability model for the current frame may be initialized based on prior counts using the following function: void reset_cdf_symbol_counter ( aom_cdf_prob *cdf_ptr , int num_cdfs , int cdf_stride , int nsymbs ) { for ( int i = 0 ; i < num cdfs ; i++ ) {nsymbs ] = ( cdf_ptr [ i * cdf_stride + nsymbs ] * 3 ) >> 2 ; }} where reset_cdf_symbol_counter is a function called to initialize or reset counts for a probability model or a portion thereof, *cdf_ptr is a pointer to the start of one of the set of probabilities in the probability model being initialized, num_cdfs indicates a number of sets of probabilities to be initialized, cdf_stride indicates a size of memory (e.g., in bytes) that a single cdf utilizes, and nsymbs indicates a number of symbols represented in a set of probabilities. In such an implementation, the probability model has previously been initialized with probabilities based on a probability model associated with a reference frame and includes the prior counts from the probability model associated with a reference frame. As shown, the counts of the sets of probabilities starts with the prior counts and each count is initialized by multiplying the prior count. The reset_cdf_symbol_counter function may be invoked multiple times for different sets of probabilities or groups of sets of probabilities in the probability model.

[0121] The initialization of counts at step 1204 may be performed in implementations where a previously updated probability model, including its prior counts, is utilized. For example, this corresponds to the scenario described with respect to FIG. 6, where a combined probability model including counts is saved to a reference frame buffer at step 616 and subsequently used for initialization at step 602. In a per-tile context, step 1204 may be part of the initialization described in step 902 of FIG. 9, which includes initializing the counts from a selected candidate model.

[0122] At step 1206, the probability model for the current frame may be updated based on the counts initialized in step 1204. For example, when encoding or decoding a tile and a first syntax element is encountered (for coding or decoding) corresponding to a set of probabilities in the probability model where the count for that set of probabilities has been initialized to a non-zero count, the probability corresponding to the value of that first syntax element may be updated by a different amount following the coding or decoding of the first syntax element as compared to if the count was initialized to zero. For example, the probability may be updated by a lesser amount as compared to if the count was initialized to a default of zero. This update adaptation reflects that the probability model has been initialized based on probability updates previously performed when the reference frame was encoded or decoded, and thus the probability model is likely to more accurately reflect the statistical distribution of the current frame or tile. As a result, the probabilities in the probability model do not have to be updated as aggressively as might be done, for example, if the probability model were initialized to a default model with counts of zero.

[0123] In some implementations, updating the probability model for the current frame based on the counts includes updating a probability of the probability model by an amount determined based on a current count associated with the probability of the probability model for the current frame that has been updated from the initialized counts for the probability model for the current frame based on occurrences of a symbol and / or a context encountered while encoding or decoding the current frame. For example, the symbol may be a particular value of a syntax element. The update amount may be smaller than would be utilized if the probability model for the current frame was not initialized based on the probability model associated with the reference frame. In some examples, updating the probability model for the current frame based on the counts includes updating a probability of the probability model for the current frame at a first rate when a count of the counts associated with the probability is greater than a threshold and at a second rate when the count of the counts associated with the probability is not greater than the threshold. For example, such implementations may utilize an adaptation based on a time_interval calculated as previously described.

[0124] The updating of the probability model at step 1206 may be performed during the encoding or decoding of symbols for a current frame or tile. For example, step 1206 may correspond to one of the repeated updates performed at step 608 within step 604 of FIG. 6. Similarly, it may correspond to the probability update using initialized counts described at step 904 of FIG. 9.

[0125] In some implementations, implementations described with respect to one or more of the previous figures may be utilized with or otherwise be modified according to or consistent with implementations described with respect to FIG. 12.

[0126] Some implementations may be implemented according to one or more of the following clauses.

[0127] Clause 1. A method for initializing a probability model for a current frame, the method comprising: initializing probabilities of the probability model for the current frame based on a probability model associated with a reference frame; initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame; and updating the probability model for the current frame based on the counts.

[0128] Clause 2. The method of clause 1, wherein the probability model for the current frame is for a current tile in the current frame.

[0129] Clause 3. The method of clause 2, wherein initializing the counts for the probability model for the current frame based on prior counts includes applying a function to the prior counts.

[0130] Clause 4. The method of clause 3, wherein the function includes dividing a count of the prior counts by two or multiplying a count of the prior counts by three quarters.

[0131] Clause 5. The method of clause 3, wherein the function includes multiplying the prior counts by three to produce an intermediate result and bit shifting the intermediate result right by two.

[0132] Clause 6. The method of any of clauses 1 to 5, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model by an amount determined based on a current count associated with the probability model for the current frame that has been updated from the initialized counts for the probability model for the current frame based on occurrences of a symbol encountered while encoding or decoding the current frame.

[0133] Clause 7. The method of any of clauses 1 to 6, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model for the current frame by a smaller amount than would be utilized if the probability model for the current frame was not initialized based on the prior counts associated with the probability model associated with the reference frame.

[0134] Clause 8. The method of any of clauses 1 to 7, wherein the probability model associated with the reference frame is a combined probability model generated by averaging a plurality of probability models of the reference frame.

[0135] Clause 9. The method of any of clauses 1 to 7, wherein the probability model associated with the reference frame is a combined probability model generated by weighted averaging a plurality of probability models of a reference frame, a candidate probability model selected for a current tile of the current frame, or a probability model obtained from a reference tile of the reference frame.

[0136] Clause 10. The method of any of clauses 8 to 9, wherein a first probability model of the plurality of probability models is associated with a first tile of the reference frame and the first probability model includes a probability and a prior count that were updated during decoding of the first tile.

[0137] Clause 11. The method of any of clauses 1 to 10, wherein the reference frame is selected based on reference frame selection information decoded from an encoded bitstream.

[0138] Clause 12. The method of clause 11, wherein the probability model associated with the reference frame is selected from a plurality of candidate probability models stored in a reference frame buffer storage area selected based on the reference frame selection information.

[0139] Clause 13. The method of any of clauses 8 to 12, wherein the combined probability model includes counts generated as a function of respective counts associated with probability models of the plurality of probability models of the reference frame.

[0140] Clause 14. The method of clause 13, wherein the function of respective counts is an average of the respective counts.

[0141] Clause 15. The method of any of clauses 1 to 14, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model for the current frame at a first rate when a count of the counts associated with the probability is greater than a threshold and at a second rate when the count of the counts associated with the probability is not greater than the threshold.

[0142] Clause 16. A method for updating a probability model for a current frame, the method comprising: updating the probability model for the current frame using a technique selected based on whether the probability model for the current frame was initialized based on a previously updated probability model.

[0143] Clause 17. The method of clause 16, wherein the probability model for the current frame is for a current tile in the current frame.

[0144] Clause 18. The method of clause 16, wherein the probability model for the current frame was initialized based on the previously updated probability model and the selected technique includes: initializing counts for the probability model for the current frame based on prior counts obtained based on updating the previously updated probability model.

[0145] Clause 19. The method of clause 18, wherein initializing the counts for the probability model for the current frame includes initializing the counts using outputs obtained by applying a function to the prior counts.

[0146] Clause 20. The method of clause 19, wherein the function obtains an output of the outputs by dividing a count of the prior counts by two or multiplying a count of the prior counts by three quarters.

[0147] Clause 21. The method of clause 19, wherein the function obtains an output of the outputs based on a display order difference between the current frame and a reference frame from which the previously updated probability model is associated.

[0148] Clause 22. The method of clause 19, wherein the selected technique includes updating a probability of the probability model by an amount determined based on a current count associated with the probability model for the current frame that has been updated from the initialized counts for the probability model for the current frame based on occurrences of a symbol encountered while encoding or decoding the current frame.

[0149] Clause 23. The method of clause 16, wherein the probability model for the current frame was initialized based on the previously updated probability model and the selected technique includes: updating a probability of the probability model for the current frame by a smaller amount than would be utilized if the probability model for the current frame was not initialized based on the previously updated probability model.

[0150] Clause 24. The method of clause 23, wherein the selected technique includes initializing counts for the probability model for the current frame to zero.

[0151] Clause 25. The method of clause 24, wherein the probability model for the current frame was initialized based on the previously updated probability model and the selected technique includes: updating a probability of the probability model for the current frame by an amount determined based on a measure of correlation between the current frame and a reference frame from which the previously updated probability model is associated.

[0152] Clause 26. The method of clause 25, wherein the measure of correlation is determined based on a display order difference between the current frame and the reference frame.

[0153] Clause 27. The method of clause 26, wherein the display order difference is based on an encoding order.

[0154] Clause 28. The method of clause 26, wherein a slower adaptation rate is utilized when the reference frame is adjacent to the current frame.

[0155] Clause 29. The method of clause 25, wherein the amount is determined based on the measure of correlation according to intervals applied to a display order difference between the current frame and the reference frame.

[0156] Clause 30. The method of clause 26, wherein the selected technique updates a probability of the probability model adaptively based on whether the current frame, tile, or block is encoded using inter coding or intra coding.

[0157] Clause 31. The method of clause 26, wherein the previously updated probability model is at least one of a combined probability model generated by averaging or weighted averaging a plurality of probability models of a reference frame, a candidateprobability model selected for a current tile of the current frame, or a probability model obtained from a reference tile of the reference frame.

[0158] Clause 32. A device, comprising: a processor that is configured to perform the method of any of clauses 1 to 31.

[0159] Clause 33. A device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of any of clauses 1 to 31.

[0160] Clause 34. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of any of clauses 1 to 31.

[0161] Clause 35. A non-transitory computer- readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is decoded by a decoder using the method of any of clauses 1 to 31.

[0162] Clause 36. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is generated by an encoder performing the method of any of clauses 1 to 31.

[0163] For simplicity of explanation, the foregoing techniques are depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a method in accordance with the disclosed subject matter.

[0164] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0165] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X includes A or B” is intended to mean any of the natural inclusive permutations. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoinginstances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. Moreover, use of the term “an implementation” or “one implementation” throughout is not intended to mean the same embodiment or implementation unless described as such.

[0166] Implementations of the transmitting station 102 and / or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and / or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0167] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general- purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms and / or instructions described herein. In addition, or alternatively, for example, a special purpose computer / processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

[0168] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server and the receiving station 106 can be implemented on a device separate from the server, such as a hand-held communications device. In this instance, the transmitting station 102 can encode content using an encoder 400 into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer ratherthan a portable communications device and / or a device including an encoder 400 may also include a decoder 500.

[0169] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a non-transitory computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport a program including instructions for use by or in connection with any processor. For example, a processor may be configured to perform executed instructions stored in the memory (e.g., computer readable medium) to perform techniques embodied in the instructions. For example, a non-transitory computer-readable storage medium may include executable instructions that, when executed by a processor, facilitate performance of operations corresponding to techniques described in this disclosure. For example, a non- transitory computer-readable storage medium may store an encoded bitstream that is encodable or decodable using techniques described in this disclosure. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0170] The above-described embodiments, implementations and aspects have been described in order to allow easy understanding of the present invention and do not limit the present invention. On the contrary, the invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structure as is permitted under the law.

Claims

What is claimed is:

1. A method for initializing a probability model for a current frame, the method comprising: initializing probabilities of the probability model for the current frame based on a probability model associated with a reference frame; initializing counts of the probability model for the current frame based on prior counts associated with the probability model associated with the reference frame; and updating the probability model for the current frame based on the counts.

2. The method of claim 1, wherein the probability model for the current frame is for a current tile in the current frame.

3. The method of claim 2, wherein initializing the counts for the probability model for the current frame based on prior counts includes applying a function to the prior counts.

4. The method of claim 3, wherein the function includes dividing a count of the prior counts by two or multiplying a count of the prior counts by three quarters.

5. The method of claim 3, wherein the function includes multiplying the prior counts by three to produce an intermediate result and bit shifting the intermediate result right by two.

6. The method of any of claims 1 to 5, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model by an amount determined based on a current count associated with the probability model for the current frame that has been updated from the initialized counts for the probability model for the current frame based on occurrences of a symbol encountered while encoding or decoding the current frame.

7. The method of any of claims 1 to 6, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model for the current frame by a smaller amount than would be utilized if the probabilitymodel for the current frame was not initialized based on the prior counts associated with the probability model associated with the reference frame.

8. The method of any of claims 1 to 7, wherein the probability model associated with the reference frame is a combined probability model generated by averaging a plurality of probability models of the reference frame.

9. The method of any of claims 1 to 7, wherein the probability model associated with the reference frame is a combined probability model generated by weighted averaging a plurality of probability models of a reference frame, a candidate probability model selected for a current tile of the current frame, or a probability model obtained from a reference tile of the reference frame.

10. The method of any of claims 8 to 9, wherein a first probability model of the plurality of probability models is associated with a first tile of the reference frame and the first probability model includes a probability and a prior count that were updated during decoding of the first tile.

11. The method of any of claims 1 to 10, wherein the reference frame is selected based on reference frame selection information decoded from an encoded bitstream.

12. The method of claim 11, wherein the probability model associated with the reference frame is selected from a plurality of candidate probability models stored in a reference frame buffer storage area selected based on the reference frame selection information.

13. The method of any of claims 8 to 12, wherein the combined probability model includes counts generated as a function of respective counts associated with probability models of the plurality of probability models of the reference frame.

14. The method of claim 13, wherein the function of respective counts is an average of the respective counts.

15. The method of any of claims 1 to 14, wherein updating the probability model for the current frame based on the counts includes updating a probability of the probability model for the current frame at a first rate when a count of the counts associated with the probability is greater than a threshold and at a second rate when the count of the counts associated with the probability is not greater than the threshold.

16. A device, comprising: a processor that is configured to perform the method of any of claims 1 to 15.

17. A device, comprising: a memory; and a processor, the processor configured to execute instructions stored in the memory to perform the method of any of claims 1 to 15.

18. A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising operations that perform the method of any of claims 1 to 15.

19. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is decoded by a decoder using the method of any of claims 1 to 15.

20. A non-transitory computer-readable storage medium having stored thereon an encoded bitstream, wherein the encoded bitstream is generated by an encoder performing the method of any of claims 1 to 15.

Images