CABAC (Adaptive Context-Based Adaptive Binary Arithmetic Coding) Initial State Selection from Coded Pictures

Abstract

Aspects of the present disclosure provide methods and apparatus that include processing circuitry that determines prior probability information associated with each entropy coded region of a plurality of regions in a previous picture. The processing circuitry selects a region from among the plurality of regions based on (i) a location or quantization parameter of an independently decodable coding segment in the current picture, or (ii) syntax information of the independently decodable coding segment. Initial probability information for one or more current syntax elements of a block in the independently decodable coding segment is determined based on the prior probability information associated with the selected region. The one or more current syntax elements are first entropy decoded in the independently decodable coding segment. The processing circuitry entropy decodes coded bits associated with the one or more current syntax elements into a bin string based on the initial probability information.

Description

[Technical field]

[0001] This application claims benefit of priority to U.S. Provisional Application No. 17 / 985,809, "ADAPTIVE CONTEXT-BASED ADAPTIVE BINARY ARITHMETIC CODING (CABAC) INITIAL STATE SELECTION FROM CODED PICTURES," filed November 11, 2022, which claims benefit of priority to U.S. Provisional Application No. 63 / 307,522, "ADAPTIVE CABAC INITIAL STATE SELECTION FROM CODED PICTURES," filed February 7, 2022. The entire disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] This disclosure describes embodiments generally related to video coding. [Background technology]

[0003] The background provided herein is intended to generally present the context of the present disclosure. The work of the present inventors is not admitted, expressly or impliedly, as prior art to the present disclosure to the extent that their work is described in this Background section, and aspects of the description that may not otherwise qualify as prior art at the time of filing.

[0004] Uncompressed digital images and / or videos may include a sequence of pictures, each having spatial dimensions of, for example, 1920x1080 luma samples and associated chroma samples. The sequence of pictures may have a fixed or variable picture rate, for example, 60 pictures / second or 60Hz. Uncompressed images and / or videos have specific bitrate requirements. For example, 1080p60 4:2:0 video (1920x1080 luma sample resolution at a frame rate of 60Hz) at 8 bits per sample requires a bandwidth approaching 1.5Gbit / s. One hour of such video requires more than 600Gbytes of storage space.

[0005] One objective of image and / or video coding and decoding is the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth and / or storage space needs by more than two orders of magnitude in some cases. Although the description herein uses video encoding / decoding as an illustrative example, the same techniques can be applied to image encoding / decoding in a similar manner without departing from the spirit of the present disclosure. Both lossless and lossy compression, as well as combinations thereof, can be used. Lossless compression refers to techniques where an exact copy of the original signal can be reconstructed from the compressed original signal. When using lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original signal and the reconstructed signal is small enough to make the reconstructed signal useful for the intended application. For video, lossy compression is widely used. The amount of distortion that is tolerated depends on the application, for example, a user of a particular consumer streaming application may tolerate higher distortion than a user of a television distribution application. The achievable compression ratio can reflect that a higher tolerable / acceptable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from a number of broad categories, including, for example, motion compensation, transform processing, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding. In intra-coding, sample values ​​are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into blocks of samples. When all blocks of samples are coded in intra mode, the picture can be an intra picture. Intra pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and therefore can be used as the first picture in a coded video bitstream and video session or as a still image. Samples of an intra block can be subjected to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction can be a technique that minimizes sample values ​​in the pre-transform domain. In some cases, the smaller the DC value and the smaller the AC coefficients after the transform, the fewer bits are needed for a given quantization step size to represent the block after entropy coding.

[0008] For example, traditional intra-coding, used in MPEG-2 generation coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to make predictions based on surrounding sample data and / or metadata obtained during encoding and / or decoding of a block of data. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that in at least some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from reference pictures.

[0009] There may be various forms of intra prediction. When one or more of such techniques may be used in a given video coding technique, the particular technique used may be coded as a particular intra prediction mode using the particular technique. In certain cases, an intra prediction mode may have sub-modes and / or parameters that may be coded separately or may be included in a mode codeword that defines the prediction mode being used. Which codeword for a given mode, sub-mode, and / or parameter combination affects the coding efficiency gain through intra prediction and may also affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] A specific mode of intra prediction was introduced in H.264, improved in H.265, and further improved in newer coding techniques such as the Joint Search Model (JEM), Generic Video Coding (VVC), and Benchmark Set (BMS). A predictor block may be formed using neighboring sample values ​​of already available samples. The sample values ​​of the neighboring samples are copied to the predictor block according to the direction. The reference to the direction used may be coded in the bitstream or may itself be predicted.

[0011] Referring to FIG. 1A, depicted at the bottom right is a subset of 9 predictor directions known from the 33 possible predictor directions defined in H.265 (corresponding to the 33 angle modes of the 35 intra modes). The point where the arrows converge (101) represents the sample to be predicted. The arrows represent the direction in which the sample is predicted. For example, arrow (102) indicates that sample (101) is predicted from a sample or samples at an angle of 45 degrees from the horizontal toward the top right. For example, arrow (103) indicates that sample (101) is predicted from a sample or samples at an angle of 22.5 degrees from the horizontal toward the bottom left.

[0012] 1A, at the top left, a square block (104) of 4×4 samples is depicted (indicated by a dashed bold line). The square block (104) contains 16 samples, each labeled with "S", its position in the Y dimension (e.g., row index), and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample (from the top) in the Y dimension and the first sample (from the left) in the X dimension. Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. Since the size of the block is 4×4 samples, S44 is at the bottom right. Additionally, reference samples are shown following a similar numbering scheme. The reference samples are labeled with R, their Y position (e.g., row index), and X position (column index) relative to the block (104). In both H.264 and H.265, the predicted samples are adjacent to the block being reconstructed, so there is no need to use negative values.

[0013] Intra-picture prediction can work by copying reference sample values ​​from adjacent samples indicated by the signaled prediction direction. For example, assume that the coded video bitstream includes signaling for this block indicating a prediction direction that coincides with the arrow (102). That is, the sample is predicted from the top right sample at an angle of 45 degrees from the horizontal. Then samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Then sample S44 is predicted from reference sample R08.

[0014] In certain cases, especially when the orientation is not evenly divisible by 45 degrees, the values ​​of multiple reference samples may be combined, for example through interpolation, to calculate the reference sample.

[0015] With the development of video coding technology, the number of possible directions is increasing. In H.264 (2003), 9 different directions could be represented. This was increased to 33 in H.265 (2013). Currently, JEM / VVC / BMS can support up to 65 directions. Experiments are performed to identify the most likely directions, and certain techniques in entropy coding are used to represent those possible directions with a small number of bits, allowing certain penalties for less likely directions. Furthermore, the direction itself can sometimes be predicted from neighboring directions used in neighboring, already decoded blocks.

[0016] FIG. 1B shows a schematic (110) depicting 65 intra prediction directions with JEM to illustrate the increasing number of prediction directions over time.

[0017] The mapping of intra-prediction direction bits representing directions in the coded video bitstream may vary from one video coding technique to another. Such mappings may range, for example, from simple direct mappings to codewords, to complex adaptation schemes with most likely modes, and similar techniques. However, in most cases, there may be certain directions that are statistically less likely to occur in the video content than certain other directions. Since the goal of video compression is to reduce redundancy, in well-behaved video coding techniques, less likely directions are represented by more bits than more likely directions.

[0018] Image and / or video coding and decoding may be performed using inter-picture prediction with motion compensation. Motion compensation may be a lossy compression technique and may refer to a technique in which blocks of sample data from a previously reconstructed picture or part thereof (reference picture) are used for prediction of a new reconstructed picture or picture partition after being spatially shifted in a direction indicated by a motion vector (hereinafter MV). In some cases, the reference picture may be the same as the picture currently being reconstructed. The MV has two dimensions, X and Y, or three dimensions, with the third dimension being an index of the reference picture in use (the latter may indirectly be a temporal dimension).

[0019] In some video compression techniques, the MV applicable to a given area of ​​sample data may be predicted from other MVs, e.g., MVs associated with another area of ​​sample data that is spatially proximate to the area under reconstruction and precedes that MV in decoding order. By doing so, the amount of data required to code the MV may be significantly reduced, thereby removing redundancy and increasing compression. MV prediction may be effective when coding, for example, an input video signal derived from a camera (known as natural video), since areas larger than the area to which a single MV is applicable have a statistical likelihood of moving in a similar direction and may possibly be predicted using similar motion vectors derived from MVs of adjacent areas. This results in an MV that is found to be similar or the same for a given area as the MV predicted from the surrounding MVs, which, after entropy coding, may be represented with a smaller number of bits than would be used if coding the MV directly. In some cases, MV prediction may be an example of lossless compression (i.e., MV) of a signal derived from the original signal (i.e., sample stream). In other cases, the MV prediction itself may be lossy, e.g., due to rounding errors when computing a predictor from several surrounding MVs.

[0020] Various MV prediction mechanisms are available for H.265 / HEVC (ITU-T Rec. H.265, "High Efficiency Video Coding", December 2016). Among the many MV prediction mechanisms offered by H.265, one that is described with reference to FIG. 2 is a technique hereafter referred to as "spatial merging".

[0021] Referring to Figure 2, the current block (201) contains samples that the encoder found during the motion search process to be predictable from a spatially shifted preceding block of the same size. Instead of coding the MV directly, the MV can be derived from metadata associated with one or more reference pictures, e.g., from the most recent reference picture (in decoding order) using MVs associated with any of the five surrounding samples denoted A0, A1, and B0, B1, B2 (202 to 206, respectively). In H.265, MV prediction can use predictors from the same reference picture that neighboring blocks use. Summary of the Invention

[0022] Aspects of the present disclosure provide methods and apparatus for video and / or image encoding / decoding. In some examples, the apparatus for video / image decoding includes processing circuitry. The processing circuitry receives a bitstream including a current picture and a previous picture. The previous picture includes a plurality of regions, each region of the plurality of regions being entropy coded. The processing circuitry determines prior probability information associated with each entropy coded region of the plurality of regions. The processing circuitry selects a region from among the plurality of regions in the previous picture based on (i) a location or quantization parameter of an independently decodable coding segment in the current picture or (ii) syntax information of the independently decodable coding segment, the decodable coding segment being (i) the current picture or (ii) a region within the current picture. The processing circuitry determines initial probability information for entropy decoding one or more current syntax elements of a block in the independently decodable coding segment based on the determined prior probability information associated with entropy decoding the selected region. In one example, the one or more current syntax elements are first entropy decoded from among the independently decodable syntax elements in the coding segment, and the processing circuitry entropy decodes coded bits associated with the one or more current syntax elements into bin strings associated with the one or more current syntax elements based on the initial probability information and a probability estimation model for the coded bits.

[0023] In one embodiment, the multiple regions in the previous picture include coding tree units (CTUs) located at four corners and a center of the previous picture, respectively. The current picture includes four corner regions and one center region. The processing circuitry selects a CTU located at one of the four corners of the previous picture based on the block being located in one of the four corner regions of the current picture. The CTU located at one of the four corners may be within the selected region. The processing circuitry selects a CTU located at a center of the previous picture based on the block being located in the center region of the current picture. The CTU located at a center may be within the selected region.

[0024] In one example, the independently decodable coding segment is one of an intra slice (I slice), a bi-predictive slice (B slice), and a predictive slice (P slice) in the current picture.

[0025] In one example, the independently decodable coding segment is the current picture. The processing circuitry selects the region based on syntactic information indicating which of the multiple regions is to be selected.

[0026] In one example, the syntactic information includes an index indicating which of the multiple regions is to be selected, and the processing circuitry selects the region based on the index.

[0027] In one example, the processing circuitry obtains one or more parameters in the probability estimation model based on the probability estimation model for entropy decoding of the selected region.

[0028] In one example, the one or more parameters include adaptation rates and adaptation weights associated with a plurality of probability estimates for a bin string, each of the adaptation rates associated with each probability estimate indicating a quantization step size between two adjacent probabilities for the bin string.

[0029] In one example, the processing circuitry selects a preceding picture from the plurality of pictures in the bitstream such that a first level in the hierarchical temporal structure associated with the preceding picture is lower than a second level in the hierarchical temporal structure associated with the current picture, each picture in the first level being decoded prior to decoding each of the one or more pictures in the second level.

[0030] In one example, the processing circuitry stores the determined prior probability information at a first quantization precision that is lower than a second quantization precision of the determined prior probability information used for entropy decoding each region of the prior picture, and the processing circuitry determines initial probability information based on the determined prior probability information stored at the first quantization precision.

[0031] In one example, a flag in the bitstream indicates that the initial probability information is determined based on previous probability information determined from a previous picture.

[0032] In one example, the probability estimation model is a context-adaptive model. The processing circuitry entropy decodes the coded bits with context-adaptive binary arithmetic coding (CABAC).

[0033] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video / image decoding. [Brief description of the drawings]

[0034] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

[0035] [Figure 1A] FIG. 2 is a schematic diagram of an example subset of intra-prediction modes.

[0036] [Figure 1B] FIG. 2 is a diagram of an example intra-prediction direction.

[0037] [Diagram 2] An example of the current block (201) and surrounding samples is shown.

[0038] [Diagram 3] FIG. 3 is a schematic diagram of an example block diagram of a communication system (300).

[0039] [Figure 4] FIG. 4 is a schematic diagram of an example block diagram of a communication system (400).

[0040] [Diagram 5] FIG. 2 is a schematic diagram of an example block diagram of a decoder.

[0041] [Figure 6] FIG. 2 is a schematic diagram of an example block diagram of an encoder;

[0042] [Figure 7] 1 shows a block diagram of an exemplary encoder.

[0043] [Figure 8] 1 shows a block diagram of an exemplary decoder.

[0044] [Figure 9] 13 shows an example of a sub-block scan order.

[0045] [Figure 10] 13 shows an example of a sub-block scan process.

[0046] [Figure 11]13 shows an example of a local template used for context selection of a current bin corresponding to a current position.

[0047] [Figure 12A] 1 illustrates an exemplary context-based adaptive binary arithmetic coding (CABAC) based entropy encoder, according to one embodiment.

[0048] [Figure 12B] 1 illustrates an exemplary CABAC-based entropy decoder according to one embodiment.

[0049] [Figure 13] Here is an example of CABAC used in an encoder:

[0050] [Figure 14] 13 shows an example of CABAC initialization of a current picture using context state from a reference picture.

[0051] [Figure 15] 1 illustrates an example of region-based CABAC context state initialization mapping between a current picture and a reference picture.

[0052] [Figure 16] An example of a hierarchical time structure is shown below.

[0053] [Figure 17] 1 shows a flowchart outlining an encoding process according to some embodiments of the present disclosure.

[0054] [Figure 18] 1 shows a flowchart outlining a decoding process according to some embodiments of the present disclosure.

[0055] [Figure 19] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0056] FIG. 3 illustrates an example block diagram of a communication system (300). The communication system (300) includes a plurality of terminal devices that can communicate with each other, for example, via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) perform a unidirectional transmission of data. For example, the terminal device (310) may code video data (e.g., a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. Unidirectional data transmission is commonplace, such as in media service applications.

[0057] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) for bidirectional transmission of coded video data, e.g., during a video conference. For bidirectional transmission of data, in one example, each of the terminal devices (330) and (340) may code video data (e.g., a stream of video pictures captured by the terminal device) for transmission to the other of the terminal devices (330) and (340) over the network (350). Each of the terminal devices (330) and (340) may also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures on an accessible display device in accordance with the recovered video data.

[0058] In the example of FIG. 3, terminal devices (310), (320), (330), and (340) may be depicted as a server, a personal computer, and a smartphone, respectively, although the principles of the present disclosure are not so limited. Embodiments of the present disclosure find application with laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network (350) represents any number of networks that carry coded video data between terminal devices (310), (320), (330), and (340), including, for example, wired and / or wireless communication networks. Communication network (350) may exchange data in circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of network (350) is not important to the operation of the present disclosure, unless otherwise described below.

[0059] 4 illustrates a video encoder and video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter may be equally applicable to other video-enabled applications including, for example, video conferencing, digital TV, streaming services, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0060] The streaming system may include a video source (401) that creates a stream of uncompressed video pictures (402), a capture subsystem (413) that may include, for example, a digital camera. In one example, the stream of video pictures (402) includes samples taken by a digital camera. The stream of video pictures (402), depicted as thick lines to highlight the high data volume when compared to the encoded video data (404) (or coded video bitstream), may be processed by an electronic device (420) that includes a video encoder (403) coupled to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (or coded bitstream), depicted as thin lines to highlight the lower data volume when compared to the stream of video pictures (402), may be stored in a streaming server (405) for future use. One or more streaming client subsystems, such as the client subsystems (406) and (408) of FIG. 4, can access the streaming server (405) to obtain copies (407, 409) of the encoded video data (404). The client subsystem (406) can include a video decoder (410), for example within an electronic device (430). The video decoder (410) decodes the incoming copy (407) of the encoded video data and creates an outgoing stream (411) of video pictures that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (404), (407), and (409) (e.g., a video bitstream) can be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265.In one example, the developing video coding standard is informally known as Versatile Video Coding (VVC), and the disclosed subject matter may be used in the context of VVC.

[0061] It is noted that the electronic devices (420) and (430) can include other components (not shown). For example, the electronic device (420) can include a video decoder (not shown), and the electronic device (430) can also include a video encoder (not shown).

[0062] 5 shows an example block diagram of a video decoder (510). The video decoder (510) may be included in an electronic device (530). The electronic device (530) may include a receiver (531) (e.g., receiving circuitry). The video decoder (510) may be used in place of the video decoder (410) in the example of FIG. 4.

[0063] The receiver may receive one or more coded video sequences to be decoded by the video decoder (510). In one embodiment, one coded video sequence is received at a time, and the decoding of each coded video sequence is independent of the decoding of the other coded video sequences. The coded video sequences may be received from a channel (501), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (531) may receive the encoded video data together with other data, such as coded audio data and / or auxiliary data streams, which may be transferred using respective entities (not depicted). The receiver (531) may separate the coded video sequences from the other data. To combat network jitter, a buffer memory (515) may be coupled between the receiver (531) and the entropy decoder / analyzer (520) (hereinafter "analyzer (520)"). In certain applications, the buffer memory (515) is part of the video decoder (510). In other cases, it may be external to the video decoder (510) (not depicted). In yet other cases, there may be a buffer memory (not depicted) external to the video decoder (510), e.g., to combat network jitter, and additionally, there may be another buffer memory (515) within the video decoder (510), e.g., to handle playback timing. When the receiver (531) is receiving data from a storage / forwarding device with sufficient bandwidth and controllability, or from an isochronous network, the buffer (515) may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory (515) may be required and may be relatively large, advantageously of adaptive size, and may be at least partially implemented in an operating system or similar element (not depicted) external to the video decoder (510).

[0064] The video decoder (510) may include a parser (520) for reconstructing symbols (521) from the coded video sequence. These categories of symbols include information used to manage the operation of the video decoder (510) and potential information for controlling a rendering device that is not an integral part of the electronic device (530), such as a rendering device (512) (e.g., a display screen), but may be coupled to the electronic device (530) as shown in FIG. 5. The rendering device control information may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser (520) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The analyzer (520) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on the at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The analyzer (520) may also extract information from the coded video sequence, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0065] The parser (520) may perform an entropy decoding / parsing operation on the video sequence received from the buffer memory (515) to produce symbols (521).

[0066] The reconstruction of the symbols (521) can involve several different units, depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors. Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (520). The flow of such subgroup control information between the parser (520) and the following units is not depicted for clarity.

[0067] In addition to the functional blocks already mentioned, the decoder (510) may be conceptually subdivided into several functional units, as described below. In a practical implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0068] The first unit is a scalar / inverse transform unit (551), which receives the quantized transform coefficients as symbols (521) from the analyzer (520), along with control information including the transform to use, block size, quantization factor, quantization scaling matrix, etc. The scalar / inverse transform unit (551) can output blocks containing sample values ​​that can be input to an aggregator (555).

[0069] In some cases, the output samples of the scaler / inverse transform unit (551) may relate to intra-coded blocks, which are blocks that do not use prediction information from a previously reconstructed picture, but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (552). In some cases, the intra-picture prediction unit (552) generates blocks of the same size and shape of the block being reconstructed using surrounding information fetched from the current picture buffer (558) that has already been reconstructed. The current picture buffer (558) buffers, for example, the partially reconstructed and / or the fully reconstructed current picture. The aggregator (555) adds the prediction information generated by the intra-prediction unit (552) to the output sample information provided by the scaler / inverse transform unit (551), possibly on a sample-by-sample basis.

[0070] In other cases, the output samples of the scalar / inverse transform unit (551) may be inter-coded and potentially related to a motion compensated block. In such cases, the motion compensated prediction unit (553) may access the reference picture memory (557) to fetch samples used for prediction. After motion compensation of the fetched samples according to the symbols (521) related to the block, these samples may be added by the aggregator (555) to the output of the scalar / inverse transform unit (551) (in this case referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensation unit (553) fetches the prediction samples may be controlled by motion vectors available to the motion compensation unit in the form of symbols (521), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory (557) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, etc.

[0071] The output samples of the aggregator (555) may be subjected to various loop filtering techniques in a loop filter unit (556). Video compression techniques may include in-loop filter techniques that are controlled by parameters contained in the coded video sequence (also called a coded video bitstream) and made available to the loop filter unit (556) as symbols (521) from the analyzer (520). Video compression may also be responsive to meta-information obtained during decoding of a previous portion (in decoding order) of the coded picture or coded video sequence, and to previously reconstructed loop filtered sample values.

[0072] The output of the loop filter unit (556) may be a sample stream that can be output to a rendering device (512) and also stored in a reference picture memory (557) for use in future inter-picture prediction.

[0073] Once a particular coded picture is fully reconstructed, it may be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the analyzer (520)), the current picture buffer (558) may become part of the reference picture memory (557), and a fresh current picture buffer may be reallocated before beginning reconstruction of a later coded picture.

[0074] The video decoder (510) may perform decoding operations in accordance with a given video compression technology or standard, such as ITU-T Rec. H.265. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used in the sense that the coded video sequence conforms to both the syntax of the video compression technology or standard and to a profile documented in the video compression technology or standard. In particular, a profile may select a particular tool from all tools available in the video compression technology or standard as the only tool available for use under that profile. Also, what is required for compliance may be that the complexity of the coded video sequence is within a range defined by a level of the video compression technology or standard. In some cases, the level limits the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (e.g., measured in megasamples per second), the maximum reference picture size, etc. The limits set by the level may be further limited in some cases via a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0075] In one embodiment, the receiver (531) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (510) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0076] 6 shows an example block diagram of a video encoder (603). The video encoder (603) is included in an electronic device (620). The electronic device (620) includes a transmitter (640) (e.g., transmitting circuitry). The video encoder (603) may be used in place of the video encoder (403) in the example of FIG.

[0077] The video encoder (603) may receive video samples from a video source (601) (which is not part of the electronic device (620) in the example of FIG. 6) that may capture video images to be coded by the video encoder (603). In another example, the video source (601) is part of the electronic device (620).

[0078] The video source (601) may provide a source video sequence to be coded by the video encoder (603) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 YCrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media services system, the video source (601) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (601) may be a camera that captures local image information as a video sequence. The video data may be provided as a number of individual pictures that give motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc. in use. Those skilled in the art can easily understand the relationship between pixels and samples. The following description focuses on samples.

[0079] According to one embodiment, the video encoder (603) may code and compress pictures of a source video sequence into a coded video sequence (643) in real-time, or under any other time constraint as needed. Enforcing an appropriate coding rate is one function of the controller (650). In some embodiments, the controller (650) controls and is operatively coupled to other functional units as described below. Coupling is not depicted for clarity. Parameters set by the controller (650) may include rate control related parameters (picture skip, quantizer, lambda value for rate distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (650) may be configured with other suitable functions related to the video encoder (603) that are optimized for a particular system design.

[0080] In some embodiments, the video encoder (603) is configured to operate in a coding loop. As a rather simplified description, in one example, the coding loop can include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (633) embedded in the video encoder (603). The decoder 633 reconstructs the symbols to create sample data in a similar manner as a (remote) decoder would create them. The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since the decoding of the symbol stream produces bit-wise accurate results that are independent of the location of the decoder (local or remote), the contents in the reference picture memory (634) are also bit-wise accurate between the local and remote encoders. In other words, the predictive part of the encoder "sees" exactly the same sample values ​​as the reference picture samples that the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchrony (eg, if synchrony cannot be maintained due to channel errors, drift will result) is used in several related techniques as well.

[0081] The operation of the "local" decoder (633) can be the same as that of a "remote" decoder, such as the video decoder (510), already described in detail above in conjunction with Figure 5. With brief reference again to Figure 5, because symbols are available and the encoding / decoding of symbols into a coded video sequence can be lossless by the entropy coder (645) and analyzer (520), the entropy decoding portion of the video decoder (510), including the buffer memory (515) and analyzer (520), may not be fully implemented in the local decoder (633).

[0082] In one embodiment, the decoder techniques, except for analysis / entropy decoding, present in the decoder are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on the decoder operation. The description of the encoder techniques can be omitted since they are the inverse of the decoder techniques described generically. In certain areas, more detailed descriptions are provided below.

[0083] In operation, in some examples, the source coder (630) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (632) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as predictive references for the input picture.

[0084] The local video decoder (633) may decode the coded video data of pictures that may be designated as reference pictures based on the symbols created by the source coder (630). The operation of the coding engine (632) may advantageously be a lossy process. When the coded video data may be decoded in a video decoder (not shown in FIG. 6), the reconstructed video sequence may be a replica of the source video sequence, typically with some errors. The local video decoder 633 may replicate the decoding process that may be performed on the reference pictures by the video decoder, such that the reconstructed reference pictures are stored in the reference picture memory (634). In this way, the video encoder (603) may locally store copies of reconstructed reference pictures that have a common content with the reconstructed reference pictures that would be obtained by the far-end video decoder (in the absence of transmission errors).

[0085] The predictor (635) may perform a predictive search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors, block shapes, etc., that serve as suitable prediction references for the new picture. The predictor (635) may operate on a sample block-by-pixel block basis to find suitable prediction references. In some cases, the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (634), as determined by search results obtained by the predictor (635).

[0086] The controller (650) may manage the coding operations of the source coder (630), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0087] The output of all the aforementioned functional units may be subjected to entropy coding in an entropy coder (645), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, or arithmetic coding.

[0088] The transmitter (640) may buffer the coded video sequence created by the entropy coder (645) and prepare it for transmission over a communication channel (660), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (640) may merge the coded video data from the video encoder (603) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0089] A controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to the respective picture. For example, pictures are often assigned as one of the following picture types:

[0090] An intra picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art understand these variations of I-pictures and their respective applications and characteristics.

[0091] A predictive picture (P picture) may be one that can be coded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values ​​of each block.

[0092] Bidirectionally predictive pictures (B-pictures) may be those that can be coded and decoded using intra- or inter-prediction, using at most two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multi-predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of one block.

[0093] A source picture is typically spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded on a block-by-block basis. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to the respective picture of the block. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0094] The video encoder (603) may perform coding operations in accordance with a given video coding technique or standard, such as ITU-T Rec. H.265. In operation, the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence, so that the coded video data may conform to a syntax specified by the video coding technology or standard being used.

[0095] In one embodiment, the transmitter (640) may transmit additional data along with the encoded video. The source coder (630) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0096] A video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture under encoding / decoding, called the current picture, is partitioned into blocks. When a block in the current picture resembles a reference block in a reference picture that was previously coded in the video and is still buffered, the block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in a reference picture and may have a third dimension that identifies the reference picture when multiple reference pictures are in use.

[0097] In some embodiments, bi-prediction techniques may be used for inter-picture prediction. According to bi-prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used that are both previous in decoding order (but may be past and future, respectively, in display order) of a current picture in a video. A block in the current picture may be coded by a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block may be predicted by a combination of the first reference block and the second reference block.

[0098] Furthermore, merge mode techniques may be used for inter-picture prediction to improve coding efficiency.

[0099] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction, intra-picture prediction, etc., is performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are partitioned into coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be split recursively into one or more coding units (CUs) in a quad tree. For example, a CTU of 64×64 pixels may be split into one CU of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. A CU is split into one or more prediction units (PUs) depending on temporal and / or spatial predictability. In general, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels, such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0100] 7 shows an example block diagram of a video encoder (703). The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. In one example, the video encoder (703) is used in place of the video encoder (403) in the example of FIG. 4.

[0101] In an HEVC example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a predictive block of 8x8 samples. The video encoder (703) determines whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode, for example using rate-distortion optimization. When the processing block is to be coded in intra mode, the video encoder (703) may use intra prediction techniques to encode the processing block into a coded picture, and when the processing block is to be coded in inter mode or bi-predictive mode, the video encoder (703) may use inter prediction techniques or bi-predictive techniques, respectively, to encode the processing block into a coded picture. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one embodiment, the video encoder (703) includes other components, such as a mode decision module (not shown) for determining the mode of the processing block.

[0102] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual calculator (723), a switch (726), a residual encoder (724), a general controller (721), and an entropy encoder (725), coupled together as shown in FIG.

[0103] The inter-encoder (730) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in preceding and subsequent pictures), generate inter-prediction information (e.g., an account of redundant information due to inter-encoding techniques, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a prediction block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that is decoded based on the encoded video information.

[0104] The intra encoder (722) receives samples of a current block (e.g., a processing block), possibly compares the block to previously coded blocks in the same picture, generates transformed and quantized coefficients, and possibly also generates intra prediction information (e.g., intra prediction direction information according to one or more intra encoding techniques). In one example, the intra encoder (722) also calculates intra prediction results (e.g., prediction blocks) based on the intra prediction information and reference blocks in the same picture.

[0105] The generic controller (721) is configured to determine generic control data and control other components of the video encoder (703) based on the generic control data. In one example, the generic controller (721) determines the mode of the block and provides a control signal to the switch (726) based on the mode. For example, when the mode is an intra mode, the generic controller 721 controls the switch 726 to select an intra mode result used by the residual calculator 723, and controls the entropy encoder 725 to select intra prediction information and include the intra prediction information in the bitstream, and when the mode is an inter mode, the generic controller 721 controls the switch 726 to select an inter prediction result used by the residual calculator 723, and controls the entropy encoder 725 to select inter prediction information and include the inter prediction information in the bitstream.

[0106] The residual calculator (723) is configured to calculate a difference (residual data) between a received block and a prediction result selected from the intra-encoder (722) or the inter-encoder (730). The residual encoder (724) operates on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (724) is configured to convert the residual data from a spatial domain to a frequency domain to generate transform coefficients. The transform coefficients then undergo a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data may be suitably used by the intra-encoder (722) and the inter-encoder (730). For example, the inter-encoder (730) can generate decoded blocks based on the decoded residual data and the inter-prediction information, and the intra-encoder (722) can generate decoded blocks based on the decoded residual data and the intra-prediction information. The decoded blocks are suitably processed to generate decoded pictures, which may be buffered in a memory circuit (not shown) and used as reference pictures in some examples.

[0107] The entropy encoder (725) is configured to format a bitstream to include the encoded block. The entropy encoder (725) is configured to include various information in the bitstream according to a suitable standard, such as the HEVC standard. In one example, the entropy encoder (725) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other suitable information in the bitstream. It is noted that, according to the disclosed subject matter, no residual information is present when coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode.

[0108] 8 shows an example diagram of a video decoder (810). The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and to decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (810) is used in place of the video decoder (410) in the example of FIG. 4.

[0109] In the example of FIG. 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872), coupled together as shown in FIG.

[0110] The entropy decoder (871) may be configured to reconstruct from the coded picture certain symbols representing syntax elements from which the coded picture is produced. Such symbols may include, for example, prediction information (e.g., intra- or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra- or inter-prediction mode, the latter two in a merged submode or another submode) and the specific samples or metadata used for prediction by the intra- or inter-decoder (872) or (880), respectively. The symbols may also include residual information, for example, in the form of quantized transform coefficients. As an example, when the prediction mode is an inter- or bi-prediction mode, the inter-prediction information is provided to the inter-decoder (880), and when the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra-decoder (872). The residual information may undergo inverse quantization and is provided to the residual decoder (873).

[0111] The inter decoder (880) is configured to receive the inter prediction information and to generate inter prediction results based on the inter prediction information.

[0112] The intra decoder (872) is configured to receive intra prediction information and to generate a prediction result based on the intra prediction information.

[0113] The residual decoder (873) is configured to perform inverse quantization to extract inverse quantized transform coefficients and process the inverse quantized transform coefficients to convert the residual information from the frequency domain to the spatial domain. The residual decoder (873) may also require certain control information, including Quantizer Parameters (QP), which may be provided by the entropy decoder (871) (this may be only low volume control information, so the data path is not depicted).

[0114] The reconstruction module (874) is configured to combine, in the spatial domain, the residual information as output by the residual decoder (873) and the prediction result (possibly as output by an inter- or intra-prediction module) to form a reconstructed block, which may be part of a reconstructed picture, which may be part of a reconstructed video. It should be noted that other suitable operations, such as a deblocking operation, may be performed to improve visual quality.

[0115] It should be noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technique. In one embodiment, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0116] An entropy coder, such as the entropy coder (645) in the example of FIG. 6 or the entropy encoder (725) in the example of FIG. 7, may perform entropy coding (e.g., entropy encoding) for encoding various information. Referring back to FIG. 7, the entropy encoder (725) may encode control information (e.g., prediction information) related to video / picture coding, residual signals (e.g., transform coefficients), etc. The prediction information may include intra prediction information (e.g., intra prediction mode information), inter prediction information (e.g., motion information, inter prediction mode information), prediction information for other prediction modes, etc. For example, the motion information may include a motion vector difference (MVD).

[0117] Entropy coding may be performed after a video / image signal is converted into syntax elements. Entropy coding may be a lossless compression scheme that uses statistical properties to compress data. For example, by performing entropy coding on a set of syntax elements, bits representing the syntax elements (called bins) may be converted into fewer bits (called coded bits) in the bitstream.

[0118] Various methods may be used to convert the above mentioned control information and residual signal into syntax elements. In one embodiment, the residual signal of a transform block is converted from the spatial domain to the frequency domain (e.g., spatial frequency domain) to generate a block of transform coefficients. Quantization may be performed to quantize the block of transform coefficients into a block of transform coefficient levels (e.g., integers). Various techniques may be used to convert the residual signal into transform coefficient levels. The block of transform coefficient levels may be processed to generate syntax elements. The syntax elements may be input to an entropy encoder and encoded into bits in a bitstream.

[0119] In one embodiment, syntax elements are generated from the transform coefficient levels as follows: A block of the transform coefficient levels may contain one or more sub-blocks. The sub-blocks may be processed according to a predefined scan order. FIG. 9 shows an example of a sub-block scan order, such as a reverse diagonal scan order as indicated by the arrows. A block (910) is partitioned into 16 sub-blocks. The sub-block (901) at the bottom right corner is processed first, and the sub-block (902) at the top left corner is processed last. For a sub-block with all zero transform coefficient levels, the sub-block may be skipped without being processed.

[0120] For a sub-block having at least one non-zero transform coefficient level, multiple (e.g., four) scan passes may be performed. An example of transform coefficient levels for a sub-block (905) is shown. The sub-block (905) includes four non-zero transform coefficient levels (e.g., −9, −1, 6, and 3). During each pass, 16 locations (in the spatial frequency domain) within the sub-block (905) may be scanned, for example, in a reverse diagonal scan order.

[0121] The transform coefficient levels may be binarized using any suitable method. Figure 10 shows an example of binarization of a group of transform coefficient levels, e.g. in a sub-block.

[0122] Figure 10 shows an example of a sub-block scan process (1000). Sixteen positions (e.g., coefficient positions) (1010) corresponding to the 16 transform coefficient levels in a sub-block (e.g., the sub-block of Figure 9) are shown in one dimension at the bottom of Figure 10. The positions (1010) are numbered from 0 to 15 to reflect their respective scan orders (0 to 15). During a first pass, the positions (1010) may be scanned from 0 to 15, generating three bin strings (1001) to (1003).

[0123] The bin string (1001) may include a significance flag (e.g., sig_coeff_flag) at each position (1010). Each bin (e.g., significance flag) in the bin string (1001) may indicate whether the absolute transform coefficient level (e.g., absLevel) of the respective transform coefficient is 0 (e.g., sig_coeff_flag is 0) or greater than 0 (e.g., sig_coeff_flag is 1).

[0124] The bin string (1002) may include a parity flag (e.g., par_level_flag) in each position (1010). Each bin (e.g., parity flag) in the bin string (1002) may indicate the parity of the absLevel of the respective transform coefficient.

[0125] The bin string (1003) may include a greater than 1 flag (e.g., rem_abs_gt1_flag) in each position (1010). Each bin (e.g., greater than 1 flag) in the bin string (1003) may indicate whether (absLevel-1)>>1 is greater than 0. The greater than 1 flag is generated only when the absLevel of the respective transform coefficient is non-zero.

[0126] In one example, a bin string (1004) is generated during the second pass. The bin string (1004) may include a greater than 2 flag (e.g., rem_abs_gt2_flag). Each bin (e.g., greater than 2 flag) in the bin string (1004) may indicate whether the absLevel of the respective transform coefficient is greater than 4. The greater than 2 flag is generated only when (absLevel-1)>>1 is greater than 0 for the respective transform coefficient.

[0127] In one example, a bin string (1005) is generated during the third pass. The bin string (1005) may include a remainder value (e.g., abs_remainder). Each bin (e.g., remainder value) in the bin string (1005) may indicate a remainder value of the absLevel of the respective transform coefficient that is greater than 4. The remainder value is generated only when the absLevel of the respective transform coefficient is greater than 4.

[0128] In one example, a bin string (1006) is generated during the fourth pass. The bin string (1006) can indicate the sign of each non-zero coefficient level. Each bin (e.g., sign) in the bin string (1006) can indicate the sign of a respective transform coefficient level.

[0129] Bin string (1001) contains 16 bins corresponding to 16 positions (1010). The other bin strings (1002)-(1006) may contain 16 or fewer bins. If there is no corresponding information in any of the 16 positions (1010), then the respective bin string is not generated. For example, if there are no remainder values ​​in the 16 positions (1010), then bin string (1005) is not generated.

[0130] 10, each transform coefficient level (e.g., the transform coefficient level at position 3) may be determined based on a corresponding set of bins (e.g., bin (1020)) at the same position (e.g., position 3). The set of bins (e.g., (1020)) may include respective bins (e.g., if bins are available) from each of the bin strings (1001)-(1006).

[0131] The bin strings (1001)-(1006) may be encoded by an entropy encoder, for example, as described in Figures 6-7. In one example, the entropy encoder implements Context-Based Adaptive Binary Arithmetic Coding (CABAC), which is a form of entropy encoding. For example, CABAC is used in various video coding standards, such as H.264 / MPEG-4 AVC, HEVC standards, etc. CABAC may be a lossless compression technique. CABAC is based on arithmetic coding. Binary symbols may be encoded, which may allow low complexity and probability modeling for the more frequently used bits of any symbol.

[0132] In CABAC, a context model (also called a probability model) that provides a probability estimate for a bin in a bin string (e.g., one of (1001) to (1006)) may be determined for that bin. In some examples, the probability model is determined by a fixed selection without context modeling.

[0133] In some examples, when bins in a bin string are locally well correlated, a probability model is determined using context modeling, e.g., based on a local context associated with the bin to be coded within the bin. Thus, which context model is used to code the bin to be coded may be based on the local context of the particular bin. Adaptively selecting a context model based on the local context can provide better probability modeling. For example, referring to FIG. 10, the local context for the bin associated with position 3 (1001(3)) may include transform coefficient level information associated with positions 1-2, position information of position 3, etc.

[0134] FIG. 11 shows an example of a local template (or context template) (1130) used for context selection of a current bin corresponding to a current position (or current coefficient) (1120). The local template (1130) may cover a set of neighboring positions (e.g., A-E) or neighboring transform coefficients of the current position (1120) in a coefficient block (1110). In the example of FIG. 11, the coefficient block (1110) has 8×8 positions and includes transform coefficient levels at 64 positions. The local template (1130) may be defined as a 5 position template at the bottom right of the current coefficient (1120). The local template may include fewer positions or more positions. In one example, the local template includes two positions (e.g., A-B) neighboring the current coefficient (1120) and does not include C-E. When reverse diagonal scan order is used for multiple passes over the locations in the coefficient block (1110), the locations in the local template (1130) are processed before the current coefficient (1120).

[0135] During context modeling, a context model for a current bin corresponding to a current coefficient (1120) may be determined based on information of a local template (1130), such as information of transform coefficient levels in the local template (1130). A magnitude of the template may be defined to indicate a magnitude of a transform coefficient or transform coefficient level in the local template (1130). In one example, the context model is selected based on the magnitude of the template.

[0136] In one example, the template magnitude is defined as the sum (e.g., sumAbs1) of the partially reconstructed absolute transform coefficient levels in the local template (1130). The partially reconstructed absolute transform coefficient levels may be determined according to a bin string (e.g., bin strings (1001)-(1003)). In one embodiment, the partially reconstructed absolute transform coefficient level at location (x,y) may be determined according to:

number

[0137] In some embodiments, the current bin is the bin in the bin string (1001) that corresponds to the current coefficient (1120), and the context model is selected based on sumAbs1. In one example, the context model for the current bin is selected based on sumAbs1 and the diagonal position d of the current bin, where d is x0+y0, where x0 and y0 are the coordinates of the current position (1120) relative to the top-left corner of the coefficient block (1110). The diagonal position d in the scan path can indicate the spatial frequency of the current coefficient (1120), and adaptive context modeling based on the diagonal position d is interpreted as spatial-frequency-dependent context modeling.

[0138] 11, when bins such as bins in bin strings (1001)-(1003) are statistically correlated, which context model is selected for a current bin at a current position (1120) may depend on the context. The context may include, for example, information of neighboring positions in the local template (1130). The context may include position information (e.g., d) of the current bin, which indicates spatial frequency dependent context modeling.

[0139] Different types of bin strings may have distinct statistical properties and therefore may be encoded with different sets of context models. For example, a first set of context models may be set for bin strings (1001) and a second set of context models may be set for bin strings (1002). Other sets of context models may be set to code other types of bin strings, such as bin strings associated with MVD.

[0140] For example, different context models may be selected for bins within the same bin string (eg, (1001)) based on diagonal position d and / or sumAbs1.

[0141] In one example, a non-binary syntax element is represented by a single bin string, and the bins within the single bin string are statistically correlated.

[0142] Certain types of bin strings are not coded in the context model, for example, bin string (1006) is encoded using the bypass coding mode described in FIG.

[0143] FIG. 12A illustrates an exemplary CABAC-based entropy encoder (1200A) according to one embodiment. For example, the entropy encoder (1200A) may be implemented with the example entropy coder (645) of FIG. 6 or the example entropy encoder (725) of FIG. 7. The entropy encoder (1200A) may include a context modeler (1210) and a binary arithmetic encoder (1220). In one example, various bin strings are input to the entropy encoder (1200A). The bin strings may include bin strings converted from binary syntax elements and / or non-binary syntax elements.

[0144] In one example, the context modeler (1210) performs a context modeling process to select a context model for a bin in the received bin string. The context model may be determined for a received bin based on, for example, the type of bin (e.g., (1001) or (1003)), the color component type of the transform component, the position of the transform coefficient associated with the received bin (e.g., diagonal position d), and previously processed neighboring transform coefficients associated with the bin (e.g., local template (1130)). The context model may provide a probability estimate for the next bin in the bin string based on, for example, the value of the received bin (e.g., “0” or “1”) and the probability estimate for the bin.

[0145] Various probability models can be used to determine (e.g., update) probability estimates for bins in the received bin string. With reference to FIG. 10, in one embodiment, a probability estimate p[k+1] for bin b[k+1] (e.g., 1001(2)) can depend on a probability estimate p[k] for bin b[k] (e.g., 1001(1)) and the value of bin b[k]. The probability model f a can be modified by varying the parameter a, as shown in equation (2).

number

[0146] p[k+1] may indicate the probability that bin b[k+1] is "1" or "0". In one example, p[k+1] indicates the probability that bin b[k+1] is "1". The probability that bin b[k+1] is "0" may be determined based on p[k+1]. Alternatively, if p[k+1] indicates the probability that bin b[k+1] is "0", the probability that bin b[k+1] is "1" may be determined based on p[k+1].

[0147] p[k] is the 7-bit integer k n (e.g., 0 to 127) np[k+1] may be represented by p[k], where the probability is discretized by a probability state (e.g., 128 states). The update of the probability may be implemented using, for example, a lookup table (e.g., a state transition table) that maps from one state (e.g., a first number indicating p[k]) to another state (e.g., a second number indicating p[k+1]) depending on, for example, the value of b[k]. The parameter "a" may indicate an adaptation rate, for example, a difference Δp between two adjacent probability states. For example, p[k+1] indicates the probability that bin b[k+1] is "1". If b[k] is 1, then p[k+1] is greater than p[k] (e.g., Δp>0), and it is estimated that bin b[k+1] is more likely to be "1" than b[k]. Otherwise, if b[k] is 0, then p[k+1] is smaller than p[k] (e.g., Δp<0), and bin b[k+1] is estimated to be less likely to be a "1" than b[k].

[0148] In one example, the arithmetic coding engine can use the probability states directly. In CABAC, a multiplication-free range split can be used by using quantized probability ranges and probability states.

[0149] In another embodiment, a multi-hypothesis probability estimation is applied. The probability can be estimated by averaging multiple estimates of different adaptation rates (e.g., two estimates q1[k] and q2[k]) as shown in equations (3) to (5).

number

[0150] The parameters a1 and a2 indicate the adaptation rates of the two estimates q1[k] and q2[k], respectively. a1 may be different from a2. The explanation of formula (2) can be suitably applied to formulas (3) to (4). When W1 is different from W2, formula (5) indicates the weighted average p[k+1] of the multiple estimates q1[k+1] and q2[k+1].

[0151] The context model list (1202) may include context models. The context model list (1202) may be stored in the memory (1201). Each entry in the context model list (1202) may represent a context model. Each context model may be assigned an index, referred to as a context model index or context index. In one example, the memory (1201) stores a probability estimate p[k] or a probability state indicative of the probability estimate. During context modeling, the context modeler (1210) may select a context model from the context model list (1202) and assign the selected context model to a bin.

[0152] The probability estimates of the context models may be initialized, for example, for the first bin in the bin string to be encoded. After a context model on the context model list (1202) is assigned to encode a bin, the context model may be subsequently updated according to the value of the bin with the updated probability estimate, for example, using Equation 2 or Equations 3-5.

[0153] In one embodiment, a binary arithmetic encoder (1220) receives the bins and associated context models (e.g., probability estimates) and performs a binary arithmetic coding process based on the probability estimates, thus converting a sequence of bins into coded bits. The bins may be encoded based on the actual state of the associated adaptive probability model passed along with the bin values. Coded bits may be generated and transmitted in the bitstream. The context model may be adaptive, e.g., the context selection for the next bin in the received string may depend on the value of the previously encoded / decoded bin, as described in FIG. 11.

[0154] FIG. 12B illustrates an exemplary CABAC-based entropy decoder according to one embodiment. For example, the entropy decoder (1200B) may be implemented in the example analyzer (520) of FIG. 5 or the example entropy decoder (871) of FIG. 8. The entropy decoder (1200B) may include a binary arithmetic decoder (1230) and a context modeler (1240). The binary arithmetic decoder (1230) receives coded bits from a bitstream and performs a binary arithmetic decoding process to recover bins from the coded bits. The context modeler (1240) may operate similarly to the context modeler (1210). For example, the context modeler (1240) may select a context model from a context model list (1204) stored in the memory (1203) and provide the selected context model to the binary arithmetic decoder (1230). The memory (1203) and the context model list (1204) may be similar to or identical to the memory (1201) and the context model list (1202) shown in FIG. 12A.

[0155] In one example, the context modeler (1240) determines a context model based on the bins recovered from the binary arithmetic decoder (1230).

[0156] Different entropy encoding schemes may be used to encode different bin strings. For example, the significance flag, parity flag, greater than 1 flag, and greater than 2 flag may be encoded with a CABAC-based entropy encoder such as the one described in the example of FIG. 12A. The syntax elements generated during the third and fourth passes may be encoded using a CABAC-bypassed entropy encoder (e.g., a binary arithmetic encoder with fixed probability estimates for the input bins).

[0157] FIG. 13 shows an example of CABAC as used in an encoder (e.g., (645), (725)). CABAC can have multiple probability modes for different contexts. In CABAC, coding the data symbols can include one or more stages such as binarization, context modeling, binary arithmetic coding, etc. One or more stages may be omitted or modified. Additional stages may be added.

[0158] In some embodiments, non-binary symbols such as non-binary syntax elements (e.g., all non-binary symbols) may be converted to binary values ​​such as bin strings by binarization by a binarizer (1301). In one example, a bin string is generated based on the non-binary syntax elements. For bits (or bins) in the bin string, the coder may choose which probability model to use (context modeling) and may optimize the probability estimate using information from nearby elements. Arithmetic coding may be applied to compress the data, for example, using a binary arithmetic coder (1305). If for some binary symbols the probability of being a "1" or a "0" is equal without a good context to better estimate the probability, the binary symbol may be coded without using a context model, and thus bypass coding is used. The embodiment or variant shown in FIG. 13 may be used in H.264 / AVC or other video coding standards. Context modeling may provide estimates of the conditional probabilities of the coding symbols. With the aid of a suitable context model, a given inter-symbol redundancy can be exploited by switching between different probability models according to the symbols already coded in the neighbourhood of the current symbol to be encoded.

[0159] In one embodiment, CABAC uses binary arithmetic coding, so that only binary decisions (1 or 0) are encoded. In binarization, non-binary symbols (e.g., transform coefficients or motion vectors) may be binarized or converted to binary by a binarizer (1301) prior to arithmetic coding. The binarization process may be similar to the process of converting data symbols to variable length codes, except that the binary codes are further encoded (e.g., by an arithmetic coder) prior to transmission. If the symbol or syntax element is a binary syntax element, binarization may be skipped. The bins from the binarizer (1301) may be input to a binary arithmetic coding engine.

[0160] The bin string may be a binarized symbol or a binary syntax element from the binarizer (1301). In one example, the bins in the bin string are processed consecutively. One or more stages in CABAC may be repeated for each bit (or "bin") of the bin string.

[0161] The coding mode decision between normal coding mode and bypass coding mode may be made for each bin, for example, based on a statistical correlation between each bin and other bins in the bin string. For example, the bypass coding mode may be selected for the bins associated with the code information (e.g., bins (1006)) or for the lower significant bins (e.g., bins (1005)), which may be evenly distributed. When the bypass coding mode is selected, the normal binary arithmetic encoding process may be bypassed for the bins. In the bypass coding mode, the bins are coded in the bypass coding engine (1304). For example, arithmetic coding based on a fixed probability model, such as using Golomb-Rice, may be used.

[0162] In normal coding mode, the context modeler (1302) can select a context model, such as a probability model, for a bin. A context model is a probability model for one or more bins of a binary symbol. As described in FIG. 12A, the context model may be chosen from a selection of available context models (e.g., in (1202)), for example, depending on statistics of recently coded data symbols. The context model can store the probability that each bin is a "1" or a "0".

[0163] In one example, the context modeler (1302) may select a context model based on a local template, a diagonal position, etc., as described in, for example, FIG 11. When a context model is adaptively selected, different bins in a bin string (e.g., the (1001) bin) may have different context models associated with different context model indexes.

[0164] The description of the context modeler 1210 may apply to the context modeler 1302. In one example, the context modeler 1302 is similar to or identical to the context modeler 1210.

[0165] Context models may be designed for various syntax elements. As mentioned above, context models may be designed for various types of syntax elements. Exemplary classes of syntax elements include residual information (e.g., transform coefficients), control information (e.g., MVD, skip mode flags, prediction mode information, block information, etc.). One type of syntax element (e.g., MVD) may be associated with multiple context models.

[0166] The previous coding results of the syntax element (which is either "1" or "0") can be used to update the context state or probability state of each binarized symbol (e.g., the probability that each bin is "1" or "0"), making the context model more adaptive to the local characteristics of the syntax element. By having an accurate prediction from the updated context state, the coding efficiency can be improved. In general, if after some adaptation, the symbol is more likely to be "1" (e.g., from historical results), the context state can be updated to lean towards "1" instead of "0".

[0167] The context modeler (1302) can output a bin and a context model (e.g., a probability state). The selected context model can be updated based on the actual coded value (e.g., if the bin value is "1", the frequency count of "1" is increased).

[0168] The arithmetic coder (1305) can encode each bin according to a selected probability model. In one example, two sub-ranges (corresponding to "0" and "1") are used for each bin. When the bypass coding mode is selected, the bypass coding engine (1304) is used to code the bin. When the normal coding mode is selected, the normal coding engine (1303) is selected to code the bin. In one example, the normal coding mode is applied to code bins whose probability of the bin's value is predictable given a given previously coded bin's value. The bypass coding engine (1304) can be a fast branch of a coding engine with significantly reduced complexity, while the normal coding engine (1303) encodes a given bin value but depends on the actual state of the associated adaptive probability model passed along with the bin value.

[0169] When the normal coding mode is selected, the selected context model may be updated based on the actual coded value (e.g., if the bin value was "1", the frequency count of "1" is increased).

[0170] In some embodiments, for random access, parallel processing purposes, etc., image and video signals are split into smaller coding segments that can be coded (e.g., coded independently) without relying on previously decoded data in the decoding order. For each binarized symbol, the context model can have an initial state at the beginning of the coding segment instead of inheriting the existing context state of the previous coding result. A coding segment can include a picture, a slice, a tile, a CTU row, a CTU, etc. Without a good initialization, the CABAC engine may require a series of adaptations to adjust the context state of the model after coding some input symbols. Thus, the compression performance may be affected in the early stages of a coding segment if the initial state of the context model and the actual probability of the coded symbol do not match.

[0171] FIG. 14 shows an example of CABAC initialization of a current picture using context states from a reference picture. In some examples, the CABAC context state (or CABAC probability state) at a fixed location of a picture or slice may be stored as an estimated initial state for the CABAC context state at the start of decoding / encoding the next picture or next slice. Pictures A-C in coding order are shown in FIG. 14. In one example, picture A is coded before coding pictures B-C. The initial probability state for the CABAC context state used for coding the start of picture B (e.g., the first bin string or first syntax element to be coded) may be determined (e.g., estimated) based on the CABAC context state in a CTU (1401) in picture A, which is a previously coded picture of picture B.

[0172] In one example, the initial probability state for the CABAC context state used to code the start of picture C (e.g., the first bin string or first syntax element to be coded) may be determined (e.g., estimated) based on the CABAC context state in a CTU (1402) in picture B, which is a previously coded picture of picture C.

[0173] Local characteristics of signals (e.g., residual information, control information) may vary from one region to another within the same picture or slice. Using the embodiment described in Fig. 14 to perform context state initialization may not be optimal in terms of prediction accuracy. According to one embodiment of the present disclosure, adaptive CABAC initial state selection from the coded picture can be performed to estimate the probability state of the context model at the start of a new coding segment.

[0174] To accommodate different statistics, the initial state table may be predefined or predicted separately for intra (I), bi-predictive (B), and predictive (P) slice types. An I slice (intra slice) may be coded (e.g., decoded) using only intra prediction. A B slice may be coded (e.g., decoded) using intra prediction or inter prediction, e.g., using at most two motion vectors and reference indexes to predict sample values ​​of each block, and a P slice may be coded (e.g., decoded) using intra prediction or inter prediction, e.g., using at most one motion vector and reference index to predict sample values ​​of each block. In one example, an initial state table of an I slice type is used only for intra slices. An initial state table of a B slice type and a P slice type may be selected for an inter slice. The above description of an I slice type, a B slice type, and a P slice type may be extended to picture types such as an I picture type, a B picture type, and a P picture type. Hereinafter, a slice is used as an example of a coding segment. Each picture may include multiple slices.

[0175] As described above, the bin string may be determined based on one or more syntax elements, and the initial probabilities of the bin string may be initial probabilities of syntax elements in the one or more syntax elements.

[0176] In one example, the syntax element is converted to a bin string. Thus, the initial probability of the bin string (or the initial probability for entropy coding the syntax element) may be the probability of the first bin to be coded in the syntax element (e.g., the probability of being 0 or 1). The first bin is coded before coding other bins in the bin string. Equation 2 or Equations 3-5 can be used to code the syntax element (or bin string) with the initial probability.

[0177] In one embodiment, a group of syntax elements (e.g., 16 transform coefficient levels in FIG. 10) in a block or sub-block is converted into multiple bin strings (e.g., bin strings (1001)-(1006)). Each bin string (e.g., (1001)) may be associated with a group of syntax elements. The initial probability of a bin string is the probability of the first bin to be coded in the bin string. The first bin is associated with a single syntax element (e.g., the first syntax element in the scan path). For example, the initial probability of bin string (1001) is the probability that bin (1001(0)) is 1 or 0 and is associated with the transform coefficient level at position 0. In the example shown in FIG. 10, a single syntax element (e.g., the transform coefficient level at position 0) may be associated with multiple initial probabilities, e.g., corresponding to (1001)-(1003), respectively. In one embodiment, the independently codable syntax elements of the current coding segment include at least one syntax element that is entropy coded before coding the remaining syntax elements in the syntax element. At least one syntax element may be entropy coded with a probability model or a context model, for example, at least one syntax element may be converted into a bin string and the bin string may be entropy coded.

[0178] The initialization process of the context model may be performed by entropy coding. In one example, without any prior knowledge of the statistical properties of at least one syntax element, each of the probability models may be initialized with a probability state corresponding to a uniform distribution, for example, the initial probability state of each probability model indicates a probability of equal probability.

[0179] In one example, the initial probability of the at least one syntax element is skewed from an equal probability. Another region (e.g., a region in a previously coded picture) that is likely to be statistically correlated with the current coding segment may be determined based on, for example, (i) the location of the current coding segment and (ii) a quantization parameter (QP) of the current coding segment. The prior probability information of the region in the previously coded picture may better reflect the initial probability of the at least one syntax element. Thus, the initial probability information indicating the initial probability of the at least one syntax element may be determined based on the prior probability information of the region, making the initialization more efficient.

[0180] A coding segment is independently codable if it can be coded independently from previously coded data that immediately precedes the coding segment in coding order. An independently codable coding segment can be coded without depending on previously coded data that immediately precedes the independently codable coding segment in coding order. The current coding segment can be a current picture, a current region within the current picture, etc. The area of ​​the region is smaller than the area of ​​the current picture. The region can be a slice, a CTU, etc. Examples of independently codable coding segments include a picture, a slice, a tile, a tile group, a CTU row, a CTU, etc. In one example, the independently codable coding segment is a slice, such as an I slice, a B slice, or a P slice.

[0181] At least one syntax element in the current coding segment may indicate motion information of the current coding segment, residual data of the current coding segment, and / or control information for coding the current coding segment (eg, prediction mode information).

[0182] In one example, the at least one syntax element includes a syntax element for a first block in the current coding segment, where the first block may be coded prior to coding other blocks in the current coding segment.

[0183] In one embodiment, a first block in the current coding segment includes a plurality of sub-blocks, the at least one syntax element includes a syntax element for a first sub-block in the first block, and the first sub-block may be coded prior to coding any other block or sub-block in the current coding segment.

[0184] In one example, the at least one syntax element includes a transform coefficient level in one of the subblocks, as shown in FIG. 10. In one example, the at least one syntax element includes an MVD of a subblock or block. To code (i) a first block or (ii) a first subblock within the first block to be coded in an independently codable coding segment (referred to as a current coding segment, such as a current slice, a current picture, etc.), each of a plurality of previous coding segments may be coded in advance, in which initial probability information indicating initial probabilities of bin strings associated with one or more syntax elements in the first block or the first subblock may be determined based on previous probability information in one of the plurality of previous coding segments. The previous probability information may include previous probabilities of previous bin strings associated with one or more previous syntax elements in a previous block or a previous subblock in a previous coding segment, such as a previously decoded picture or slice. The bin string and the previous bin string are of the same type, for example, the bin string is a bin string (1001) including a significance flag of the first subblock, and the previous bin string includes a significance flag of the previous subblock.

[0185] The initial probability information may include an initial probability state (or an initial context state) indicating an initial probability. The initial probability information (e.g., the initial context state) may be inherited from a previously coded coding segment (e.g., a previously coded slice or a previously coded picture). In order to more efficiently capture the statistical distribution of the current coding segment (e.g., the current slice or the current picture), each context of the syntax element at the start of the current coding segment may inherit a stored context state from a previously coded coding segment (e.g., a previously coded picture).

[0186] In addition to the initial probability information, other parameters for entropy coding may be inherited when the context state of the current coding segment is initialized from one of the previously coded coding segments. The parameters may include an adaptation rate and / or an adaptation weight used to specify a probability model (or a context model) (e.g., when multiple hypothesis probability estimation with adaptive weights as described in equations (3)-(5) is used).

[0187] Instead of using probability information (e.g., context state) at a fixed CTU location as described in FIG. 14, a region from a previous picture can be based on where the start of the current slice is located in the current picture, and the context state at the region from the previous picture can be used to determine the initial probability information. FIG. 15 shows an example of region-based CABAC context state initialization mapping between a current picture (1502) and a reference picture (or previous picture) (1501). A context state from a selected previously coded coding segment (e.g., a previously coded picture) can be determined (e.g., selected) based on where a first block or a first sub-block to be coded is located in the current picture. The selected previous picture (1501) can include multiple regions. Each region can have a selected location where probability information (e.g., context state) of one or more syntax elements at the selected location can be stored and used for prediction. In the example of FIG. 15, for each region, a selected CTU can be determined. For example, the previous picture (1501) includes five regions, such as a top left region including CTU1, a top right region including CTU2, a bottom left region including CTU3, a bottom right region including CTU4, and a center region including CTU5. In one example, the five regions do not overlap. In one example, the five regions can cover the entire previous picture (1501). In one example, the area of ​​the five regions is smaller than the area of ​​the entire previous picture (1501).

[0188] In one example, probability information such as a context state (or probability state) of one or more coded syntax elements (e.g., one or more entropy coded syntax elements) of each of CTU1-5 may be stored in each CTU (or each region). The context state of one or more syntax elements associated with a selected CTU may be stored, for example, after the selected CTU is coded. For a current coding segment (e.g., a current slice), the current coding segment may be mapped to a region of multiple regions in a preceding picture (1501) according to a position of the current coding segment, such as a top-left position of a first coding unit (e.g., a first block) of the current coding segment in a current picture (1502), and an initial context state of the current coding segment may be determined based on an associated context state from a region of the multiple regions.

[0189] Referring to FIG. 15, a current picture (1502) includes multiple independently codable coding segments, such as slices. The current picture (1502) may include multiple regions (e.g., five regions), such as a top left region (1511), a top right region (1512), a bottom left region (1513), a bottom right region (1514), and a center region (1515). In one example, the five regions (1511)-(1515) do not overlap. In one example, the five regions (1511)-(1515) cover the entire picture (1502). An independently codable coding segment, such as a slice, in the current picture (1502) is to be coded. To initialize a context state for an independently codable coding segment in the current picture (1502), a corresponding region or location in a previous picture (1501) may be determined based on the location of the independently codable coding segment in the current picture (1502). In one example, the location of the independently codable coding segment is indicated by the top left position of the first coding unit (e.g., the first block) of the current coding segment. For example, if the independently codable coding segment is in a region (1511) in the current picture (1502), the corresponding region in the previous picture (1501) is the top left region in the previous picture (1501), and the corresponding CTU1 in the top left region in the previous picture (1501) may be selected. Thus, the stored probability information (e.g., stored context state) associated with CTU1 may be used to determine initial probability information of the independently codable coding segment in the region (1511). If the independently codable coding segment is located in another region (e.g., one of (1512)-(1515)), the above description may be applied.

[0190] In this example, the context states of syntax elements at five locations (e.g., top left CTU1, top right CTU2, bottom left CTU3, bottom right CTU4, and center CTU5) are recorded or stored. The locations (e.g., CTU5) may be referenced when a selected CTU (e.g., CTU5) is coded. For example, the context states of syntax elements in CTU5 may be used to initialize the current coding segment in the current picture (1502). One of the locations is selected for CABAC context state initialization according to the location of the current coding block at the start of each slice in the current picture.

[0191] Alternatively, a selection of a location in a previous picture (1501) where the context state is used in the initialization process may be signaled for an independently codable coding segment (e.g., a current slice) in a current picture (1502), e.g., an index pointing to one of multiple candidate locations (e.g., CTU1-CTU5) may be signaled.

[0192] Comparing the initialization processes of Figures 14-15, the initial probability information of Figure 15 is adaptive to the specific information of the independently codable current coding segment. The specific information of the current coding segment may include the QP of the current coding segment, the location of the current coding segment, etc. Thus, the initial probability information is determined from the previous probability information of the area that matches the specific information of the current coding segment (e.g., QP, location, etc.). In contrast, the initial probability information of Figure 14 is obtained from the previous probability information of a fixed location, which is not possible for the specific information of the current coding segment. In various embodiments, the adaptive initialization process of Figure 15 can make the entropy coding process more efficient. For each independently codable coding segment (e.g., each slice), for example, in the current picture (1502), a control flag may be signaled to select a context initialization scheme to be used for each independently codable coding segment (e.g., each slice). In one example, when the flag is zero, the context state of the independently codable coding segment may be initialized using an existing context initialization table, such as a predefined table for the slice type (e.g., one of the predefined initial state tables for I slice type, B slice type, and P slice type). Otherwise (e.g., when the flag is 1), the context state of the independently codable coding segment may be initialized based on a stored context state from a previously coded coding segment (e.g., a previously coded slice), such as by copying the stored context state from the previously coded slice.

[0193] Figure 16 shows an example of a hierarchical temporal structure or hierarchical temporal coding structure for inter-coding. Pictures 0 to 15 (abbreviated as P0 to P15) are shown in Figure 16. The display order is from P0 to P15. The numbers associated with the pictures are in the display order.

[0194] A picture at a lower level (e.g., level 0) is coded before a picture at a higher level (e.g., one of levels 1 to 4). A picture at a level may be used as a reference picture to predict a picture at the same level or at a higher level.

[0195] For a current picture (e.g., (1502)), one or more previously coded pictures may be available as reference pictures. The selection of reference pictures for CABAC context initialization may be based on the position (e.g., level) of the current picture in the hierarchical temporal coding structure. In the example of FIG. 16, P0 and P16 are coded first as level 0 pictures. In one example, P0 is intra-coded and P16 is inter-coded (e.g., based on P0). P8 is coded as a level 1 picture with P0 and / or P16 as reference pictures. For level 2 pictures, for example, P4 may select P0 and P8 as reference pictures, and P12 may select P8 and P16 as reference pictures, taking into account the dependency and distance between the picture to be coded and the respective reference pictures. A similar approach may be applied to level 3 and level 4 pictures. In one example, the reference pictures are selected from coded pictures at a level lower than the level of the current picture to be coded. Thus, pictures at the same level can be coded in parallel and can be discarded (if necessary) without relying on other pictures at the same level as reference pictures.

[0196] In one embodiment, when selecting reference pictures for context initialization, pictures at the same level of a hierarchical temporal coding structure cannot be used as reference pictures for each other. By applying the above constraints, pictures at the same level can be encoded / decoded in parallel (e.g., simultaneously) without depending on each other. For example, since P1 and P3 are at the same level (e.g., level 4), P3 cannot use the context state from P1 for initialization.

[0197] 15 to 16 may be combined. For example, if the current picture (1502) is at a first level (e.g., level 3), the preceding picture (1501) may be restricted to a second level (e.g., one of levels 0 to 2) that is lower than the first level.

[0198] In an embodiment, instead of or in addition to the embodiment described in Figures 15-16, quantization information (e.g., quantization parameter QP) of a previously coded coding segment can be used for context initialization. For example, the QP of the current coding segment can be compared with the QP of the previously coded coding segment, and one of the previously coded coding segments is selected based on the comparison. In one example, one of the previously coded coding segments is associated with a minimum QP difference. A context state of one of the previously coded coding segments is used to determine initial probability information of the current coding segment.

[0199] In one example, the current coding segment is the current picture (1502). Which picture is selected from the previously coded pictures may be determined based on the location of the first block to be coded or the location of the first sub-block to be coded in the current picture (1502). Alternatively, which picture is selected from the previously coded pictures may be determined based on the QP of the current picture (1502) and the QP of the previously coded pictures. In one embodiment, which picture is selected from the previously coded pictures may be determined based on the level of the current picture and the level of the previously coded pictures in the hierarchical temporal structure, as described in FIG. 16. The above embodiments may be combined. For example, P1 (at level 4) can use P2 (at level 3) as a reference picture because level 3 is lower than level 4 and the absolute QP difference |QP difference| between P1 and P2 is the smallest among the absolute QP differences between P1 and P6, P10 and P14.

[0200] For each context state, a finite precision may be defined to describe the probability. For example, an 8-bit integer (or 8-bit state) may be used to indicate a probability state ranging from 0 to 255. Quantization may be applied to selected context states in the coded picture, such as bit-depth reduction. Thus, the stored context states may use less storage space in memory for prediction purposes. In one example, the stored context states may be stored with lower precision using fewer bits (e.g., 4 bits instead of 8 bits), thus retaining only the most significant 4 bits of the original 8-bit information for each context. When used for prediction, the least significant 4 bits may be filled with zeros. With reference to FIG. 15, the syntax elements of CTU1 may be entropy coded with 8-bit precision, and a context state with 8-bit precision may be obtained based on the entropy coding of CTU1. The context state with 8-bit precision may then be stored with 4-bit precision. The context state with 4-bit precision may be used to determine an initial probability of the current coding segment (e.g., a slice in the current picture (1502)).

[0201] FIG. 17 shows a flow chart outlining a process (e.g., encoding process) (1700) according to one embodiment of the present disclosure. The process (1700) may be performed by an apparatus for video / image coding, which may include processing circuitry. Processing circuitry within an apparatus, such as processing circuitry within terminal devices (310), (320), (330), and (340), processing circuitry performing the functions of a video encoder (e.g., (403), (603), (703)), may be configured to perform the process (1700). In some embodiments, the process (1700) is implemented with software instructions, and thus the processing circuitry performs the process (1700) when the processing circuitry executes the software instructions. The process starts at (S1701) and proceeds to (S1710).

[0202] At (S1710), a previous picture may be entropy encoded. Each region of a plurality of regions in the previous picture may be entropy encoded. Previous probability information associated with entropy encoding each region in the previous picture may be determined.

[0203] At (S1720), a region may be selected from among multiple regions in a previous picture based on (i) a location or quantization parameter of an independently encodable coding segment in the current picture, or (ii) syntax information of the independently encodable coding segment. The independently encodable coding segment may be the current picture or a region (e.g., a slice) in the current picture.

[0204] At (S1730), initial probability information for entropy encoding one or more current syntax elements of the block in the independently encodable coding segment may be determined based on the determined prior probability information associated with entropy encoding the selected region. The one or more current syntax elements are initially entropy encoded from among the syntax elements in the independently encodable coding segment.

[0205] The block may be the first block to be entropy encoded from among the blocks in an independently encodable coding segment.

[0206] At (S1740), a bin string associated with the one or more current syntax elements may be entropy encoded into coded bits based on the initial probability information and a probability estimation model for the bin string.

[0207] Syntax information indicating which of the multiple regions is selected may be encoded and included in the bitstream.

[0208] In one example, the syntactic information includes an index that indicates which of the multiple regions is to be selected.

[0209] Next, the process (1700) proceeds to (S1799) and ends.

[0210] Process 1700 may be suitably adapted to various scenarios, and the steps of process 1700 may be adjusted accordingly. One or more of the steps of process 1700 may be adapted, omitted, repeated, and / or combined. Any suitable order may be used to implement process 1700. Additional steps may be added.

[0211] The independently encodable coding segment may be a slice within the current picture.

[0212] In one embodiment, the independently encodable coding segment is the current picture.

[0213] In one example, the multiple regions in the previous picture include coding tree units (CTUs) located at the four corners and the center of the previous picture, respectively. The current picture includes four corner regions and one center region. The CTU located at one of the four corners of the previous picture may be selected based on the block being located at one of the four corner regions of the current picture. The CTU located at the center of the previous picture may be selected based on the block being located at the center region of the current picture. The region includes the CTU located at one of the four corners of the previous picture or the CTU located at the center of the previous picture.

[0214] In one example, one or more parameters in the probability estimation model may be obtained based on a probability estimation model for entropy encoding the selected region. An example of the probability estimation model is described in Equations (2) to (5).

[0215] In one example, the one or more parameters include adaptation rates and adaptation weights associated with a plurality of probability estimates for a bin string, as described in Equations (2)-(5). Each of the adaptation rates associated with each probability estimate indicates a quantization step size between two adjacent probabilities for the bin string (e.g., the difference Δp between two adjacent probability states).

[0216] In one example, a previous picture is selected from a plurality of pictures in a bitstream such that a first level in the hierarchical temporal structure associated with the previous picture is lower than a second level in the hierarchical temporal structure associated with the current picture, each picture in the first level being decoded prior to decoding each of the one or more pictures in the second level.

[0217] In one example, the determined prior probability information is stored at a first quantization precision (e.g., 4-bit precision) lower than a second quantization precision (e.g., 8-bit precision) of the determined prior probability information used for entropy decoding each region of the prior picture. Initial probability information can be determined based on the determined prior probability information stored at the first quantization precision.

[0218] In one example, a flag in the bitstream indicates that the initial probability information is determined based on previous probability information determined from a previous picture.

[0219] In one example, the probability estimation model is a context-adaptive model. The coded bits may be entropy encoded with context-adaptive binary arithmetic coding (CABAC).

[0220] FIG. 18 shows a flow chart outlining a process (e.g., a decoding process) (1800) according to one embodiment of the present disclosure. The process (1800) may be used in a video / image decoder. The process (1800) may be performed by an apparatus for video / image coding, which may include receiving circuitry and processing circuitry. Processing circuitry in an apparatus, such as processing circuitry in terminal devices (310), (320), (330), and (340), processing circuitry performing the functions of a video decoder (410), processing circuitry performing the functions of a video decoder (510), etc., may be configured to perform the process (1800). In some examples, the process (1800) is used in a video encoder (e.g., video encoder (403), video encoder (603)). In one example, the process (1800) is performed by processing circuitry performing the functions of a video encoder (e.g., video encoder (403), video encoder (603)). In some embodiments, the process (1800) is implemented with software instructions, and thus, when the processing circuitry executes the software instructions, the processing circuitry performs the process (1800). The process starts at (S1801) and proceeds to (S1810).

[0221] At (S1810), a bitstream including a current picture and a previous picture is received. The previous picture may include a plurality of regions. Each region of the plurality of regions may be entropy coded.

[0222] At (S1820), prior probability information associated with each entropy coded region of a plurality of regions in a prior picture is determined.

[0223] At (S1830), a region may be selected from among multiple regions in a previous picture based on (i) a location or quantization parameter of an independently decodable coding segment in the current picture, or (ii) syntax information of the independently decodable coding segment. The independently decodable coding segment may be the current picture or a region (e.g., a slice) in the current picture.

[0224] The independently decodable coding segment may be a slice in the current picture. In one example, the independently decodable coding segment is one of an intra slice (I slice), a bi-predictive slice (B slice), and a predictive slice (P slice) in the current picture.

[0225] In one example, the independently decodable coding segment is the current picture. The region may be selected based on syntax information indicating which of the multiple regions is to be selected.

[0226] In one example, the syntactic information includes an index that indicates which of the multiple regions is to be selected.

[0227] In one example, the multiple regions in the previous picture include coding tree units (CTUs) located at the four corners and the center of the previous picture, respectively. The current picture includes four corner regions and one center region. The CTU located at one of the four corners of the previous picture may be selected based on the block being located at one of the four corner regions of the current picture. The CTU located at the center of the previous picture may be selected based on the block being located at the center region of the current picture. The region includes the CTU located at one of the four corners of the previous picture or the CTU located at the center of the previous picture.

[0228] At (S1840), initial probability information for entropy decoding one or more current syntax elements of a block in an independently decodable coding segment may be determined based on previous probability information determined associated with entropy decoding the selected region. The one or more current syntax elements are first entropy decoded from among the syntax elements in the independently decodable coding segment.

[0229] The block may be the first block to be entropy decoded among the blocks in an independently decodable coding segment.

[0230] At (S1850), coded bits associated with the one or more current syntax elements may be entropy decoded into a bin string associated with the one or more current syntax elements based on the initial probability information and a probability estimation model for the coded bits.

[0231] Next, the process (1800) proceeds to (S1899) and ends.

[0232] Process 1800 may be suitably adapted to various scenarios, and the steps of process 1800 may be adjusted accordingly. One or more of the steps of process 1800 may be adapted, omitted, repeated, and / or combined. Any suitable order may be used to implement process 1800. Additional steps may be added.

[0233] In one example, one or more parameters in the probability estimation model may be obtained based on a probability estimation model for entropy decoding of the selected region. An example of the probability estimation model is described in Equation (2) to Equation (5).

[0234] In one example, the one or more parameters include adaptation rates and adaptation weights associated with a plurality of probability estimates for a bin string, as described in Equations (2)-(5). Each of the adaptation rates associated with each probability estimate indicates a quantization step size between two adjacent probabilities for the bin string (e.g., the difference Δp between two adjacent probability states).

[0235] In one example, a previous picture is selected from a plurality of pictures in a bitstream such that a first level in the hierarchical temporal structure associated with the previous picture is lower than a second level in the hierarchical temporal structure associated with the current picture, each picture in the first level being decoded prior to decoding each of the one or more pictures in the second level.

[0236] In one example, the determined prior probability information is stored at a first quantization precision (e.g., 4-bit precision) lower than a second quantization precision (e.g., 8-bit precision) of the determined prior probability information used for entropy decoding each region of the prior picture. Initial probability information can be determined based on the determined prior probability information stored at the first quantization precision.

[0237] In one example, a flag in the bitstream indicates that the initial probability information is determined based on previous probability information determined from a previous picture.

[0238] In one example, the probability estimation model is a context-adaptive model. The coded bits may be entropy decoded with context-adaptive binary arithmetic coding (CABAC).

[0239] The embodiments of the present disclosure may be used separately or in combination in any order. Furthermore, each of the methods (or embodiments), the encoder, and the decoder may be implemented by processing circuitry (e.g., one or more processors, or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0240] The techniques described above may be implemented as computer software using computer readable instructions and physically stored on one or more computer readable media. For example, Figure 19 illustrates a computer system (1900) suitable for implementing certain embodiments of the disclosed subject matter.

[0241] Computer software may be coded using any suitable machine code or computer language that may be subject to assembly, compilation, linking, or similar mechanisms to create code including instructions that may be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., either directly or through interpretation, microcode execution, etc.

[0242] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0243] 19 for the computer system (1900) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (1900).

[0244] The computer system (1900) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users, for example, through tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), olfactory input (not depicted). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from still image cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic images).

[0245] The input human interface devices may include one or more (only one of each is depicted) of a keyboard (1901), a mouse (1902), a trackpad (1903), a touch screen (1910), a data glove (not shown), a joystick (1905), a microphone (1906), a scanner (1907), and a camera (1908).

[0246] The computer system (1900) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (1910), data gloves (not shown), or joystick (1905), which may be haptic feedback devices that do not function as input devices), audio output devices (e.g., speakers (1909), headphones (not depicted)), visual output devices (e.g., touch screens (1910), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without haptic screen input capabilities, some of which may be capable of outputting two-dimensional visual output or even three-dimensional output through such means as stereoscopic output, virtual reality glasses (not depicted), holographic displays, smoke tanks (not depicted)), and printers (not depicted).

[0247] The computer system (1900) may also include human accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (1920) with CD / DVD or similar media (1921), thumb drives (1922), removable hard drives or solid state drives (1923), legacy magnetic media such as tapes and floppy disks (not depicted), specialized ROM / ASIC / PLD based devices such as security dongles (not depicted).

[0248] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0249] The computer system (1900) may also include an interface (1954) to one or more communication networks (1955). The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include Ethernet, WLAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired and wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial including CANBus, etc. Certain networks typically require an external network interface adapter attached to a specific general purpose data port or peripheral bus (1949) (e.g., a USB port on the computer system (1900)), while others are typically built into the core of the computer system (1900) by attachment to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system) as described below. Using any of these networks, the computer system (1900) may communicate with other entities. Such communications may be unidirectional receive only (e.g., broadcast television), unidirectional transmit only (e.g., from a CANbus to a particular CANbus), or bidirectional, for example, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used in each of these networks and network interfaces, as described above.

[0250] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to a core (1940) of the computer system (1900).

[0251] The core (1940) may include one or more central processing units (CPUs) (1941), graphics processing units (GPUs) (1942), specialized programmable processing units in the form of field programmable gate areas (FPGAs) (1943), hardware accelerators for specific tasks (1944), graphics adapters (1950), etc. These devices may be connected through a system bus (1948), along with read-only memory (ROM) (1945), random access memory (1946), internal mass storage (1947) such as an internal hard drive that is not user accessible, SSD, etc. In some computer systems, the system bus (1948) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be either attached directly to the core's system bus (1948) or through a peripheral bus (1949). In one example, a touch screen 1910 may be connected to a graphics adapter 1950. Architectures for peripheral buses include PCI, USB, etc.

[0252] The CPU (1941), GPU (1942), FPGA (1943), and accelerator (1944) may execute certain instructions that may combine to constitute the aforementioned computer code. The computer code may be stored in ROM (1945) or RAM (1946). Temporary data may also be stored in RAM (1946), while persistent data may be stored, for example, in internal mass storage (1947). Rapid storage and retrieval to any of the memory devices may be enabled through the use of cache memory, which may be closely associated with one or more of the CPU (1941), GPU (1942), mass storage (1947), ROM (1945), RAM (1946), etc.

[0253] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0254] As an example, and not by way of limitation, a computer system (1900) having an architecture, specifically a core (1940), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with the user-accessible mass storage introduced above, as well as specific storage of the core (1940) of a non-transitory nature, such as the core internal mass storage (1947), ROM (1945), etc. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1940). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1940), specifically the processor (including a CPU, GPU, FPGA, etc.) therein to perform certain operations or certain parts of certain operations described herein, including defining data structures stored in RAM (1946) and modifying such data structures according to operations defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., an accelerator (1944)), which may operate in place of or in conjunction with software to perform particular processes or particular portions of particular processes described herein. Reference to software may encompass logic, and vice versa, where appropriate. Reference to a computer-readable medium may encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software. Appendix A: Acronyms JEM: Joint exploration model VVC: versatile video coding BMS: benchmark set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplementary Enhancement Information VUI: Video Usability Information GOP: Group of Pictures TU: Transform Unit PU: Prediction Unit CTU: Coding Tree Unit CTB: Coding Tree Block PB: Prediction Block HRD: Hypothetical Reference Decoder SNR Signal Noise Ratio CPU: Central Processing Unit GPU: Graphics Processing Unit CRT: Cathode Ray Tube LCD: Liquid Crystal Display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Areas SSD: Solid-state drive IC: Integrated Circuit CU: Coding Unit CCLM: Combine cross-component linear model

[0255] While this disclosure has described several exemplary embodiments, there are alterations, substitutions, and various substitute equivalents which are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise many systems and methods which, although not explicitly shown or described herein, embody the principles of this disclosure and are within the spirit and scope of this disclosure.

Claims

1. A method of decoding in a decoder receiving a bitstream including a current picture and a previous picture, the previous picture including a plurality of regions, each region of the plurality of regions having been entropy coded; determining prior probability information associated with each entropy coded region of the plurality of regions; selecting a region from among the plurality of regions in the previous picture based on (i) a location or quantization parameter of an independently decodable coding segment in a current picture or (ii) syntax information of the independently decodable coding segment, the decodable coding segment being (i) the current picture or (ii) a region within the current picture; determining initial probability information for entropy decoding one or more current syntax elements of a block in the independently decodable coding segment based on the determined prior probability information associated with the selected entropy coded region, the one or more current syntax elements being initially entropy decoded among the syntax elements in the independently decodable coding segment; and entropy decoding coded bits associated with one or more current syntax elements based on the initial probability information and a probability estimation model for the coded bits into a bin string associated with the one or more current syntax elements.

2. The regions in the preceding picture include coding tree units (CTUs) located at four corners and a center of the preceding picture, respectively; the current picture includes four corner regions and one central region; The step of selecting the region comprises: selecting a CTU located in one of the four corners of the previous picture based on the block being located in one of the four corner regions of the current picture, where the CTU located in one of the four corners is within the selected region; 2. The method of claim 1, comprising: selecting the centrally located CTU of the previous picture based on the location of the block in the central region of the current picture, the centrally located CTU being within the selected region.

3. The method of claim 1 , wherein the independently decodable coding segment is one of an intra slice (I slice), a bi-predictive slice (B slice), and a predictive slice (P slice) in the current picture.

4. the independently decodable coding segment is the current picture; The method of claim 1 , wherein the selecting step comprises selecting a region based on the syntactic information indicating which of the plurality of regions is to be selected.

5. The syntactic information includes an index indicating which of the plurality of regions is to be selected. The method of claim 1 , wherein the selecting step comprises selecting the region based on the index.

6. The method of claim 1 , comprising obtaining one or more parameters in a probability estimation model based on the probability estimation model for entropy decoding of the selected region.

7. 7. The method of claim 6, wherein the one or more parameters include adaptation rates and adaptation weights associated with a plurality of probability estimates for the bin string, each of the adaptation rates associated with a respective probability estimate indicating a quantization step size between two adjacent probabilities for the bin string.

8. 2. The method of claim 1, further comprising: selecting a previous picture from a plurality of pictures in the bitstream such that a first level in a hierarchical temporal structure associated with the previous picture is lower than a second level in a hierarchical temporal structure associated with the current picture, each picture at the first level being decoded before decoding each of one or more pictures at the second level.

9. storing the determined prior probability information at a first quantization precision lower than a second quantization precision of the determined prior probability information used for entropy decoding each region of the prior picture; The method of claim 1 , wherein determining the initial probability information comprises determining the initial probability information based on the stored and determined prior probability information at the first quantization precision.

10. The method of claim 1 , wherein a flag in the bitstream indicates that the initial probability information is determined based on the determined prior probability information from the prior picture.

11. the probability estimation model is a context-adaptive model; The method of claim 1 , wherein the entropy decoding comprises entropy decoding the coded bits with context-adaptive binary arithmetic coding (CABAC).

12. 1. An apparatus for decoding, comprising: An apparatus comprising processing circuitry configured to perform the method of any one of claims 1 to 11.

13. A program which, when executed by at least one processor, causes the at least one processor to perform the method according to any one of claims 1 to 11.

Images