Method, device and storage medium for multiple symbol arithmetic coding - Patents.com

Abstract

A method, apparatus, and computer-readable storage medium for multiple-symbol arithmetic coding in video decoding, the method including the steps of receiving a coded video bitstream, obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1, and performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions, by updating, for each extracted symbol, the array of cumulative distribution functions in response to the extracted symbol based on at least one probability update rate, and continuing arithmetic decoding on the coded video bitstream to extract a next symbol based on the updated array of cumulative distribution functions.

Description

[Technical field]

[0001] Related Applications This application is based on and claims the benefit of priority to U.S. Provisional Application No. 63 / 306,386, filed February 3, 2022, and U.S. Nonprovisional Application No. 17 / 869,964, filed July 21, 2022. Both applications are incorporated by reference in their entirety herein.

[0002] The present disclosure relates to video coding and / or decoding techniques, and more particularly to improved designs and / or implementations of multi-symbol arithmetic coding and / or decoding. [Background technology]

[0003] The Background of the Invention described herein is intended to provide a general overview of the background of the disclosure. To the extent described in the Background of the Invention section, the work of the currently named inventors and other aspects of the description that may not have attained the status of prior art at the time of filing of this application are not admitted, expressly or impliedly, as prior art to the present disclosure.

[0004] Video coding and decoding can be performed using inter-picture prediction with motion compensation. An uncompressed digital video may include a sequence of pictures, each of which has a certain spatial dimension, for example 1920x1080 luma samples, and associated full or subsampled chroma samples. The sequence of pictures may have a fixed or variable picture rate (also called frame rate), for example 60 pictures per second or frames per second. Uncompressed video has specific bit rate requirements for streaming and data processing. For example, a video with a pixel resolution of 1920x1080, a frame rate of 60 per second, and chroma subsampling of 4:2:0 with 8 bits per pixel per color channel requires a bandwidth of about 1.5 Gbit / s. One hour of such video requires more than 600 GByte of storage space.

[0005] One of the goals of video coding and decoding is to reduce redundancy in the compressed input video signal by compression. Compression can be useful to reduce the bandwidth and / or storage requirements mentioned above, in some cases by more than one order of magnitude. Both lossless and lossy compression, as well as combinations of these, can be used. Lossless compression refers to techniques where an exact replica of the original signal can be reconstructed from the compressed original signal through a decoding process. Lossy compression refers to coding / decoding processes where the original video information is not fully preserved during coding and cannot be fully restored during decoding. With lossy compression, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough that the reconstructed signal can be used for the intended application, although some information is lost. In the case of video, lossy compression is widely used for many applications. The amount of acceptable distortion depends on the application. For example, users of certain consumer streaming applications may tolerate more distortion than users of cinema or television broadcast applications. The compression ratio achievable by a particular coding algorithm can be selected or adjusted to reflect different distortion tolerances: higher tolerable distortion typically allows for coding algorithms that result in larger losses and higher compression ratios.

[0006] Techniques from several broad categories and steps may be used in video encoders and decoders, including, for example, motion compensation, Fourier transform, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding, which represents sample values ​​without reference to samples or other data from a previously reconstructed reference picture. Some video codecs spatially subdivide a picture into blocks of samples. If all blocks of samples are coded in intra mode, the picture can be called an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state, and can be used as the first image of a coded video bitstream or video session, or as still images. The samples of the block after intra prediction can then undergo a transform to the frequency domain, and the transform coefficients so generated can be quantized before entropy coding. Intra-prediction represents a technique that minimizes the sample values ​​in the domain before the transform. In some cases, the smaller the DC value and the smaller the AC coefficients after the transform, the smaller the size of the bits required for a given quantization step to represent the block after entropy coding.

[0008] Traditional intra-coding, such as coding known from the MPEG-2 generation of coding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that attempt to code / decode blocks based on surrounding sample data and / or metadata obtained during the encoding and / or decoding of, for example, spatial neighbors, or surrounding sample data and / or metadata that precede the block of intra-coded or decoded data in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. It is noted that, at least in some examples, intra-prediction only uses reference data of the current picture being reconstructed, and not reference data of other reference pictures.

[0009] There may be many different forms of intra prediction. When one or more of such techniques are available for a given video coding technique, the technique used may be referred to as an intra prediction mode. One or more intra prediction modes may be implemented in a particular codec. In some examples, a mode may have sub-modes and / or may be associated with various parameters, and the mode / sub-mode information and intra coding parameters for a block of video may be coded separately or may be included together in a mode codeword. Which codeword is used for a given mode, sub-mode and / or parameter combination may affect the coding efficiency improvement achieved by intra prediction, and therefore may affect the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain modes of intra prediction were introduced in H.264, improved in H.265, and further improved by new coding techniques such as the joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). Typically, intra prediction can use the available values ​​of neighboring samples to form a prediction block. For example, the available values ​​of a certain set of neighboring samples along a given direction and / or line can be replicated in the prediction block. Reference values ​​in the direction used can be coded in the bitstream, or the reference values ​​themselves can be predicted.

[0011] Referring to FIG. 1A, a subset of nine predictor directions specified by the 33 possible intra prediction directions of H.265 (corresponding to the 33 angular modes of the 35 intra modes described in H.265) is shown at the bottom right. The point where the arrows meet (101) represents the sample being predicted. The arrows represent the direction in which the sample 101 is predicted using neighboring samples. For example, the arrow (102) indicates that the sample (101) is predicted from one or more neighboring samples toward the upper right at an angle of 45 degrees from the horizontal. Similarly, the arrow (103) indicates that the sample (101) is predicted from one or more neighboring samples toward the lower left of the sample (101) at an angle of 22.5 degrees from the horizontal.

[0012] Continuing with reference to FIG. 1A, a square block (104) of 4×4 samples is shown at the top left (indicated by a thick dashed line). The square block (104) contains 16 samples, each with an index including "S" and the sample's position in the Y dimension (e.g., row index) and the sample's position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first sample in the X dimension (from the left). Similarly, sample S44 is the fourth sample in the block (104) in both the Y and X dimensions. If the size of the block is 4×4 samples, S44 is at the bottom right. Further examples of reference samples are shown following a similar numbering scheme. The reference samples are indexed with R and the Y position (e.g., row index) and X position (column index) of the reference sample relative to the block (104). In both H.264 and H.265, prediction samples that are adjacent and near the block are used during reconstruction.

[0013] Intra prediction of block 104 may start by duplicating reference sample values ​​from neighboring samples according to a signaled prediction direction. For example, assume that the coded video bitstream includes signaling for block 104 indicating a prediction direction of arrow (102), i.e., predicting samples from one or more prediction samples toward the top right at an angle of 45 degrees from the horizontal. In such a case, predict samples S41, S32, S23, and S14 from the same reference sample R05. Then predict sample S44 from reference sample R08.

[0014] In some instances, particularly where the orientation is not evenly divisible by 45 degrees, values ​​of multiple reference samples may be combined, for example through interpolation, to calculate the reference sample.

[0015] As video coding techniques continue to develop, the number of possible directions has increased. In H.264 (2003), for example, nine different directions are available for intra prediction. This has increased to 33 in H.265 (2013), but JEM / VVC / BMS can support up to 65 directions as of this disclosure. Experimental studies have been performed to help identify the optimal intra prediction direction, but some techniques of entropy coding can be used to encode such an optimal direction for a small number of bits, while accepting some bit penalty for the direction. Furthermore, in some cases, the direction itself can be predicted from nearby directions used in intra prediction of the neighboring blocks being decoded.

[0016] FIG. 1B shows a schematic diagram (180) illustrating the 65 intra prediction directions according to the JEM, with many prediction directions being shown for various ever-evolving coding techniques.

[0017] The manner in which bits representing intra-prediction directions are mapped to prediction directions in a coded video bitstream may vary for different video coding techniques, and may range from simple direct mapping of prediction directions to complex matching schemes that result in intra-prediction modes, codewords, most likely modes, and other techniques. In all cases, however, there may be some directions of intra-prediction that are statistically less likely to occur in the video content than some other directions. Because the goal of video compression is to reduce redundancy, these less likely directions may be represented by more bits than more likely directions if the video coding technique is properly designed.

[0018] Inter-picture prediction (or inter-prediction) may be based on motion compensation, in which sample data from a previously reconstructed picture or part of it (reference picture) may be used to predict a newly reconstructed picture or part of a picture (e.g., a block) after being spatially shifted in a direction indicated by a motion vector (hereafter MV). In some cases, the reference picture may be the same as the picture at the time the reconstruction is being performed. The MV may have two dimensions X and Y, or three dimensions, with the third dimension being an index for the reference picture being used (similar to the temporal dimension).

[0019] In some video compression techniques, a current MV applicable to a region of sample data can be predicted from other MVs, e.g., from other MVs related to other regions of sample data that are spatially adjacent to the region being reconstructed and that precede the current MV in decoding order. In this way, the total amount of data required to code the MV can be substantially reduced as the redundancy of correlated MVs is removed, thereby increasing compression efficiency. For example, when coding an input video signal (called raw video) obtained from a camera, MV prediction can work effectively because there is a statistical probability that a region larger than the region to which one MV is applicable moves in a similar direction in the video sequence, and therefore, in some cases, the region can be predicted using similar motion vectors derived from MVs of nearby regions. As a result, the actual MV of a given region is similar or identical to the MV predicted from the surrounding MVs. After entropy coding, the MV can be represented with fewer bits than would be used if the MV was directly coded without predicting it from one or more nearby MVs. In some examples, MV prediction can be an example of lossless compression of a signal (i.e., MV) derived from the original signal (i.e., sample stream). In other cases, the MV prediction itself may be lossy, for example due to rounding errors when computing the prediction from several surrounding MVs.

[0020] Various MV prediction mechanisms are described in H.265 / HEVC (ITU-T Rec. H.265, “High Efficiency Video Coding”, December 2016). Among the many MV prediction mechanisms described in H.265, a technique hereinafter referred to as “spatial merging” is described below.

[0021] Specifically, referring to Fig. 2, a current block (201) comprises samples that are predictable from a spatially shifted previous block of the same size, as detected by the encoder during the motion search process. Instead of coding the MV directly, the MV can be derived from metadata associated with one or more reference pictures, e.g., the MV can be derived from the most recent (decoding order) reference picture using the MV associated with any one of the surrounding five samples, denoted A0, A1 and B0, B1, B2 (202-206, respectively). In H.265, MV prediction can use predicted values ​​obtained from the same reference picture used by neighboring blocks. Summary of the Invention [Means for solving the problem]

[0022] This disclosure describes various embodiments of methods, apparatus, and computer-readable storage media for video encoding and / or decoding.

[0023] According to one aspect, an embodiment of the present disclosure provides a method for multiple symbol arithmetic coding in video decoding. The method includes a device receiving a coded video bitstream. The device includes a memory storing instructions and a processor in communication with the memory. The method also includes the device obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1, and the device performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions, where for each extracted symbol, the device updates the array of cumulative distribution functions in response to the extracted symbol based on at least one probability update rate, and the device continues arithmetic decoding on the coded video bitstream to extract a next symbol based on the updated array of cumulative distribution functions.

[0024] According to another aspect, an embodiment of the present disclosure provides an apparatus for multiple-symbol arithmetic coding in video decoding, the apparatus including a memory storing instructions and a processor in communication with the memory, the processor being configured, when executing the instructions, to cause the apparatus to perform the above-described method for video decoding and / or encoding.

[0025] In another aspect, an embodiment of the present disclosure provides a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding and / or encoding, cause the computer to perform the above-described method for video decoding and / or encoding.

[0026] These and other aspects, as well as implementations of these aspects, are described in more detail in the drawings, description, and claims.

[0027] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0028] [Figure 1A] 1 shows a schematic diagram of an exemplary subset of intra-prediction directional modes. [Figure 1B] 1 shows a diagram of a typical intra prediction direction. [Diagram 2] 1 illustrates a schematic diagram of a current block and its surrounding spatial merging candidates used for motion vector prediction in one example. [Diagram 3] 1 shows a simplified block diagram schematic of a communication system (300) according to one embodiment. [Figure 4] 1 shows a simplified block diagram of a communication system (400) according to one embodiment. [Diagram 5] FIG. 2 shows a schematic diagram of a simplified block diagram of a video decoder according to one embodiment. [Figure 6] 1 shows a schematic diagram of a simplified block diagram of a video encoder according to an embodiment; [Figure 7]4 shows a block diagram of a video encoder according to another embodiment; [Figure 8] 4 shows a block diagram of a video decoder according to another embodiment; [Figure 9] 1 illustrates a coding block partitioning scheme according to an embodiment of the present disclosure. [Figure 10] 1 illustrates another scheme for coding block partitioning according to an embodiment of the present disclosure. [Figure 11] 1 illustrates another scheme for coding block partitioning according to an embodiment of the present disclosure. [Figure 12] 1 shows an example of dividing a base block into coding blocks according to a typical division method. [Figure 13] An example of the three-way split method is shown below. [Figure 14] 4 shows an example of a quadtree / binary tree coding block division method. [Figure 15] 4 illustrates a scheme for splitting a coding block into multiple transform blocks and a coding order for the transform blocks according to an embodiment of the present disclosure. [Figure 16] 1 illustrates another scheme for splitting a coding block into multiple transform blocks and the coding order of the transform blocks according to an embodiment of the present disclosure. [Figure 17] 1 shows an example of a flow chart used to decode a bin. [Figure 18] 1 shows a flowchart of a method according to an embodiment of the present disclosure. [Figure 19] FIG. 1 shows a schematic diagram of a computer system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof and which show, by way of illustration, specific examples of embodiments. However, it should be noted that the present invention may be embodied in many different forms, and therefore, it is intended that the subject matter covered or claimed be construed as not being limited to any of the embodiments described below. It should also be noted that the present invention may be embodied as a method, device, component, or system. Thus, embodiments of the present invention may take the form of, for example, hardware, software, firmware, or any combination thereof.

[0030] Throughout the specification and claims, terms may have special meanings suggested or implied by the context beyond the scope of their stated meaning. The phrases "in one embodiment" or "in some embodiments" used in this disclosure do not necessarily refer to the same embodiment, and the phrases "in another embodiment" or "in other embodiments" used in this disclosure do not necessarily refer to different embodiments. Similarly, the phrases "in one implementation" or "in some implementations" used in this disclosure do not necessarily refer to the same implementation, and the phrases "in another implementation" or "in other implementations" used in this disclosure do not necessarily refer to different implementations. The intention, for example, is that the claimed subject matter includes a combination of exemplary embodiments / implementations in whole or in part.

[0031] Overall, the usage of a term may be at least partially determined by how it is used in context. For example, terms such as "and," "or," or "and / or" as used in this disclosure may include various meanings that may depend, at least in part, on the context in which the term is used. Typically, when "or" is used to link an enumeration such as A, B, or C, "or" is intended to mean A, B, or C, in this case used in an inclusive sense, and "or" is intended to mean A, B, or C, in this case used in an exclusive sense. In addition, the terms "one or more" or "at least one" as used in this disclosure may be used to describe any feature, configuration, or characteristic in the singular sense or may be used to describe a combination of features, configurations, or characteristics in the plural sense, at least in part depending on the context. Similarly, terms such as "a," "an," or "the" may be used to recognize the use in the singular or to recognize the use in the plural, at least in part depending on the context. In addition to the above, the terms "based on" or "determined" may be understood, again depending at least in part on the context, as not necessarily intended to dictate the recognition of an exclusive set of factors, and may even allow for the existence of other factors not necessarily expressly recited.

[0032] FIG. 3 illustrates a simplified block diagram of a communication system (300) according to an embodiment of the present disclosure. The communication system (300) includes a plurality of terminal devices that can communicate with each other, e.g., via a network (350). For example, the communication system (300) includes a first pair of terminal devices (310) and (320) that are interconnected via the network (350). In the example of FIG. 3, the first pair of terminal devices (310) and (320) may perform a unidirectional transmission of data. For example, the terminal device (310) may code video data (e.g., video data of a stream of video pictures captured by the terminal device (310)) for transmission to another terminal device (320) via the network (350). The encoded video data may be transmitted as one or more coded video bitstreams. The terminal device (320) may receive the coded video data from the network (350), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. One-way data transmission may be implemented, such as in a media presentation application.

[0033] In another example, the communication system (300) includes a second pair of terminal devices (330) and (340) for bidirectional transmission of coded video data, such as may be implemented in videoconferencing applications. In the case of bidirectional transmission of data, by way of example, each of the terminal devices (330) and (340) may code video data (e.g., video data of a stream of video pictures captured by the terminal device) for transmission over the network (350) to the other of the terminal devices (330) and (340). Each of the terminal devices (330) and (340) may also receive coded video data transmitted by the other of the terminal devices (330) and (340), decode the coded video data to recover the video pictures, and display the video pictures corresponding to the recovered video data on an accessible display device.

[0034] In the example of FIG. 3, the terminal devices (310), (320), (330), and (340) may be implemented as a server, a personal computer, and a smartphone, although the scope of application of the underlying principles of the present disclosure is not limited in this manner. The embodiments of the present disclosure may be implemented in desktop computers, laptop computers, tablet computers, media players, wearable computers, dedicated videoconferencing equipment, and / or the like. The network (350) 9 represents any number and type of network that conveys coded video data between the terminal devices (310), (320), (330), and (340), including, for example, wired and / or wireless communication networks. The communication network (350) 9 may exchange data over circuit-switched channels, packet-switched channels, and / or other types of channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of illustration of the present example, unless expressly described in the present application, the architecture and topology of the network (350) may not be important to the operation of the present disclosure.

[0035] Figure 4 shows an arrangement of a video encoder and a video decoder in a video streaming environment as an example application of the disclosed subject matter. The disclosed subject matter may be similarly applicable to other applications of video including, for example, video conferencing, digital television broadcasting, gaming, virtual reality, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0036] The video streaming system may include a video source (401), e.g., a video capture subsystem (413) that may include a digital camera, generating a stream of uncompressed video pictures or images (402). In one example, the stream of video pictures (402) includes samples recorded by a digital camera of the video source 401. The stream of video pictures (402), shown in bold to emphasize its large amount of data when compared to the coded video data (404) (i.e., the encoded video bitstream), may be processed by an electronic device (420) that includes a video encoder (403) connected to the video source (401). The video encoder (403) may include hardware, software, or a combination thereof that enables or implements aspects of the disclosed subject matter, as described in more detail below. The encoded video data (404) (i.e., encoded video bitstream (404)) is shown with thin lines to emphasize its small amount of data when compared to the uncompressed video picture stream (402), which may be stored on the streaming server (405) for later use or directly on a downstream video device (not shown). One or more streaming client subsystems, such as the client subsystems (406) and (408) of FIG. 4, may access the streaming server (405) to obtain copies (407) and (409) of the encoded video data (404). The client subsystem (406) may include a video decoder (410), for example, in an electronic device (430). The video decoder (410) receives and decodes the copy of the encoded video data (407) and generates an output stream of video pictures (411) that is uncompressed and that can be rendered on a display (412) (e.g., a display screen) or other rendering device (not shown). Video decoder 410 may be configured to perform some or all of the various functions described in this disclosure.In some streaming systems, the encoded video data 404, 407, and 409 (e.g., a video bitstream) may be encoded according to a particular video coding / compression standard. Examples of such standards include ITU-T Recommendation H.265. In one example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used with respect to VVC and other video coding standards.

[0037] It is noted that electronic devices 420 and 430 can include other components (not shown). For example, electronic device 420 can include a video decoder (not shown), and similarly, electronic device 430 can include a video encoder (not shown).

[0038] 5 shows a block diagram of a video decoder (510) according to any of the embodiments of the present disclosure described below. The video decoder (510) can be included in an electronic device (530). The electronic device (530) can include a receiver (531) (e.g., a receiving circuit). The video decoder (510) can be used in place of the video decoder (410) in the example of FIG. 4.

[0039] The receiver (531) may receive one or more coded video sequences to be decoded by the video decoder (510). One coded video sequence at a time may be decoded in the same embodiment or in another embodiment, and the decoding of each coded video sequence is independent of the other coded video sequences. Each video sequence may relate to multiple video frames or images. The coded video sequences may be received from a channel (501), which may be a hardware / software coupled to a storage device that stores the coded video data or a streaming source that transmits the coded video data. The receiver (531) may receive the coded video data along with other data, such as coded audio data and / or ancillary data streams, and may forward the other data to corresponding processing circuitry (not shown). The receiver (531) may separate the coded video sequences from the other data. To address network jitter, a buffer memory (515) may be disposed between the receiver (531) and the entropy decoder / parser (520) (hereinafter "parser (520)"). In some applications, the buffer memory (515) may be implemented as part of the video decoder (510). In other applications, the buffer memory (515) may be external to the video decoder (510) and may be separate from the video decoder (510) (not shown). In still other applications, a buffer memory (not shown) may be external to the video decoder (510), for example to handle network jitter, and another additional buffer memory (515) may be present within the video decoder (510), for example to handle playback timing. If the receiver (531) receives data from a store-and-forward device or an isochronous network with sufficient bandwidth and controllability, the buffer memory (515) may not be required, or may be small.For use with a best effort packet network such as the Internet, a sufficiently sized buffer memory (515) may be required, and the size of the buffer memory (515) may be relatively large. Such a buffer memory may be implemented with an optimized size and may be implemented, at least in part, in an operating system or similar element (not shown) external to the video decoder (510).

[0040] The video decoder (510) may include a parser (520) that reconstructs symbols (521) from the coded video sequence. Such categories of symbols include information used to manage the operation of the video decoder (510) and may include information to control a rendering device such as a display (512) (e.g., a display screen) that may or may not be an integral part of the electronic device (530) and may be connected to the electronic device (530) as shown in FIG. 5. The control information for one or more rendering devices may take the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (520) may parse / entropy decode the coded video sequence received by the parser (520). The entropy coding of the coded video sequence may conform to a video coding technique or standard and may follow a variety of principles including variable length coding, Huffman coding, arithmetic coding with or without context-sensitive coding, etc. The parser (520) may extract from the coded video sequence a set of subset parameters for at least one subset of pixels in the video decoder based on at least one parameter corresponding to the subset. A subset may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), etc. The parser (520) may also extract from coded video sequence information such as transform coefficients (e.g., Fourier transform coefficients), quantizer parameter values, motion vectors, etc.

[0041] The parser (520) may generate symbols (521) by performing an entropy decoding / parser operation on the video sequence received from the buffer memory (515).

[0042] The reconstruction of the symbols (521) may involve a number of different processing or functional entities depending on the type of coded video picture or portion thereof (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors. Which entities are involved and how they are involved may be controlled by subgroup control information parsed by the parser (520) from the coded video sequence. For simplicity, the flow of such subgroup control information between the parser (520) and the multiple processing or functional entities described below is not shown.

[0043] Further to the functional blocks already described, the video decoder (510) may be conceptually subdivided into a number of functional units, as described below. In an actual implementation operating within commercial constraints, many of these functional units may interact closely with each other and may be at least partially integrated with each other. However, the following disclosure employs a conceptual subdivision into functional units in order to clearly describe and explain the various features of the disclosed subject matter.

[0044] The first section may include a scalar / inverse transform unit (551), which may receive quantized transform coefficients as symbols (521) from the parser (520) and control information including information indicating which type of inverse transform to use, block size, quantization factors / quantization parameters, quantization scaling matrices, and states. The scalar / inverse transform unit (551) may output blocks comprising sample values ​​that may be input to an aggregater (555).

[0045] In some cases, the scaler / inverse transform DepartmentThe output samples of (551) may be suitable for intra-coded blocks, i.e., blocks that do not use predictive information from previously reconstructed pictures, but may use predictive information from previously reconstructed parts of the current picture. Such predictive information may be provided by an intra-picture predictor (552). In some examples, the intra-picture predictor (552) may generate blocks of the same size and shape as the block being reconstructed using information from surrounding blocks that have already been reconstructed and stored in a current picture buffer (558). The current picture buffer (558) may, for example, buffer a partially reconstructed current picture and / or a fully reconstructed current picture. In some implementations, the aggregator (555) may add, on a sample-by-sample basis, the predictive information generated by the intra-predictor (552) to the output sample information provided by the scaler / inverse transformer (551).

[0046] In other examples, the output samples of the scaler / inverse transform unit (551) may be suitable for an inter-coded block, or even a motion-compensated block. In such examples, the motion compensation prediction unit (553) may access the reference picture memory (557) to obtain samples used for inter-picture prediction. The obtained samples may be motion-compensated according to the symbols (521) related to the block, and then added to the output of the scaler / inverse transform unit (551) (the output of the unit 551 may be referred to as a residual sample or a residual signal) by the aggregator (555) to generate output sample information. The addresses in the reference picture memory (557) from which the motion compensation prediction unit (553) obtains prediction samples may be controlled by a motion vector, which is available to the motion compensation prediction unit (553) in the form of a symbol (521) that may have, for example, X, Y components (shift) and a reference picture component (time). Motion compensation may also include interpolation of sample values ​​obtained from a reference picture memory (557) when sub-sample accurate motion vectors are used, or may involve motion vector prediction mechanisms, etc.

[0047] The output samples of the aggregater (555) may be subjected to various loop filtering techniques in the loop filter unit (556). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also referred to as the coded video bitstream) as symbols (521) from the parser (520) and made available to the loop filter unit (556), but may be sensitive to meta-information obtained when decoding the coded picture or earlier portions of the coded video sequence (decoding order), and may be sensitive to previously reconstructed loop filtered sample values. Several types of loop filters may be included as part of the loop filter unit 556, in various orders, as will be described in more detail below.

[0048] The output of the loop filter unit (556) can be a sample stream that can be output to a rendering device (512) as well as stored in a reference picture memory (557) for later use in inter-picture prediction.

[0049] Once some coded pictures are fully reconstructed, they can be used as reference pictures for subsequent inter-picture prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified (e.g., by the parser (520)) as a reference picture, the current picture buffer (558) can become part of the reference picture memory (557), and a new current picture buffer can be reallocated before starting the reconstruction of the next coded picture.

[0050] The video decoder (510) may perform decoding operations according to a predefined video compression technique employed in a given standard (e.g., ITU-T Recommendation H.265). A coded video sequence is said to be compliant with the syntax defined by the video compression technique or standard used in the sense that the coded video sequence adheres to both the syntax of the video compression technique or standard and the profile described in the document of the video compression technique or standard. In particular, a profile may select a predefined tool from all tools available in the video compression technique or standard as the only tools that can be used according to the profile. To be compliant with a standard, the complexity of the coded video sequence may be within limits defined by the level of the video compression technique or standard. In some examples, the level may limit the maximum picture size, the maximum frame rate, the maximum reconstruction sample rate (e.g., measured in million samples per second), the maximum reference picture size, etc. The limits imposed by the levels may in some instances be further restricted by the specifications of a Hypothetical Reference Decoder (HRD) and metadata for HRD buffer management that is signaled in the coded video sequence.

[0051] In some embodiments, the receiver (531) may receive ancillary (redundant) data along with the encoded video. The ancillary data may be included as part of the coded video sequence. The ancillary data may be used by the video decoder (510) to properly decode the data and / or more accurately reconstruct the original video data. For example, the ancillary data may take the form of temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0052] 6 shows a block diagram of a video encoder (603) according to one embodiment of the present disclosure. The video encoder (603) may be included in an electronic device (620). The electronic device (620) may further include a transmitter (640) (e.g., a transmission circuit). The video encoder (603) may be used in place of the video encoder (403) in the example of FIG. 4.

[0053] The video encoder (603) may receive video samples from a video source (601) (which in the example of FIG. 6 is not part of the electronic device (620)) capable of capturing one or more video images to be coded by the video encoder (603). In another example, the video source (601) may be implemented as part of the electronic device (620).

[0054] The video source (601) may provide a source video sequence to be coded by the video encoder (603) as a digital video sample stream that may be of any suitable bit depth (e.g.: 8-bit, 10-bit, 12-bit, ...), any color space (e.g. BT.601 YCrCb, RGB, XYZ...), and any suitable sampling structure (e.g. YCrCb 4:2:0, YCrCb 4:4:4). In a media provision system, the video source (601) may be a storage device capable of storing pre-prepared video. In a video conferencing system, the video source (601) may be a camera capturing on-site image information as a video sequence. The video data may be provided as a number of individual pictures or images that achieve motion when viewed in sequence. A picture may simply be organized as a spatial array of pixels, where each pixel may comprise one or more samples depending on the sampling structure, color space, etc. being used. The relationship between pixels and samples is easily understood by those skilled in the art. The following description focuses on samples.

[0055] According to some embodiments, the video encoder (603) may code and compress pictures of the source video sequence into the coded video sequence (643) in real-time or subject to any other time constraints required by the application. Enforcing the maintenance of an appropriate coding rate constitutes one of the functions of the controller (650). In some embodiments, the controller (650) may be operatively connected to other functionalities, described below, which the controller (650) controls. For simplicity, the connections are not shown. Parameters set by the controller (650) may include rate control related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, search range for longest motion vector, etc. The controller (650) may be configured to have other appropriate functions for the video encoder (603) optimized for a particular system design.

[0056] In some embodiments, the video encoder (603) may be configured to operate in a coding loop. As a simplistic explanation, in one example, the coding loop may include a source coder (630) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and one or more reference pictures) and a decoder (633) built into the video encoder (603). The decoder (633) reconstructs the symbols and generates sample data as if it were generated by a decoder (located elsewhere), even though the built-in decoder 633 processed the video stream coded by the source coder 630 without entropy coding (any compression between the symbols and the coded video bitstream in entropy coding may be a lossless compression included in the video compression techniques contemplated in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (634). Since decoding the symbol stream will yield bit-accurate results independent of the decoder's location (here or elsewhere), the contents of the reference picture memory (634) will also be bit-accurate compared to the encoder at this location versus the encoder at another location. In other words, the predictive part of the encoder "sees" exactly the same sample values ​​as the reference picture samples as if the decoder "sees" when using prediction during decoding. This fundamental principle of reference picture synchrony (and the resulting drift when synchrony cannot be maintained, e.g., due to channel errors) is used to improve coding quality.

[0057] The operation of the "in-place" decoder (633) may be the same as the operation of a decoder, such as the "elsewhere" video decoder (510) already described in detail above in conjunction with Figure 5. However, and with some reference to Figure 5, because symbols are valid and the encoding / decoding of symbols into a coded video sequence by the entropy coder (645) and parser (520) may be lossless, the entropy decoding portion of the video decoder (510), including the buffer memory (515) and parser (520), may not be fully implemented in the in-place decoder (633) of the encoder.

[0058] At this point, it can be said that any decoder technology other than parsing / entropy decoding, which may only exist in the decoder, must also necessarily exist in the corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter may occasionally focus on the operation of the decoder in relation to the decoding portion of the encoder. Thus, a description of the encoder technology may be omitted, since it is the inverse of the decoder technology described in its entirety. Only certain parts or aspects of the encoder are described in more detail below.

[0059] During operation of some implementations, the source coder (630) may perform motion-compensated predictive coding, which codes an input picture by predicting with reference to one or more previously coded pictures from a video sequence designated as "reference pictures." In this manner, the coding engine (632) codes color channel differences (i.e., residual values) between pixel blocks of the input picture and pixel blocks of one or more reference pictures that can be selected as one or more predictive references for the input picture. The terms "residue" and its adjective form "residual" may be used interchangeably.

[0060] The video decoder (633) at this location may decode the coded video data of pictures that can be designated as reference pictures based on the symbols generated by the source coder (630). It may be useful for the operation of the coding engine (632) to be a lossy process. If the coded video data may be decoded by some video decoder (not shown in FIG. 6), the reconstructed video sequence may generally be a copy of the source video sequence with some errors. The video decoder (633) at this location may repeat the decoding process that may be performed by the video decoder on the reference pictures, such that the reconstructed reference pictures are stored in the reference picture cache (634). In this way, the video encoder (603) may store copies of reconstructed reference pictures at this location that have common content with the reconstructed reference pictures that will be obtained by the far-end (elsewhere) video decoder (without transmission errors).

[0061] The predictor (635) may perform a prediction search for the coding engine (632). That is, for a new picture to be coded, the predictor (635) may search the reference picture memory (634) to obtain sample data (sample data as candidate reference pixel blocks) and to obtain reference picture motion vectors, block shapes, and other specific metadata that can serve as suitable prediction references for the new picture. The predictor (635) may operate using one sample block per pixel block to detect suitable prediction references. In some examples, as determined by the search results obtained by the predictor (635), the input picture may include prediction references obtained from multiple reference pictures stored in the reference picture memory (634).

[0062] For example, the controller (650) may manage the coding operations of the source coder (630), including setting the parameters and subgroup parameters used to encode the video data.

[0063] The output of all of the above functionalities may be subjected to entropy coding in an entropy coder (645), which converts the symbols produced by the various functionalities into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0064] The transmitter (640) may buffer one or more coded video sequences generated by the entropy coder (645) for transmission over a communication channel (660), and may be hardware / software coupled to a storage device that stores the coded video data. The transmitter (640) may also integrate other data to be transmitted, such as coded audio data and / or ancillary data streams (sources not shown), with the coded video data obtained from the video coder (603).

[0065] A controller (650) may manage the operation of the video encoder (603). During coding, the controller (650) may assign to each coded picture a particular coded picture type, which can affect the coding scheme that can be applied to the respective picture. For example, pictures can often be assigned as one of the following picture types:

[0066] An Intra Picture (I-picture) can be described as a picture that can be coded and decoded without using any other picture in the sequence as a basis for prediction. In some video codecs, various types of Intra pictures are possible, including, for example, Independent Decoder Refresh (IDR) pictures. Such variations of I-pictures and their corresponding applications and features will occur to those skilled in the art.

[0067] A predictive picture (P picture) is a picture that can be coded and decoded using intra- or inter-prediction, and at most one motion vector and reference index to predict the sample values ​​of each block.

[0068] A bi-directionally predictive picture (B-picture) is a picture that can be coded and decoded using intra- or inter-prediction and up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multiple-predictive picture can use more than two reference pictures and associated metadata for the reconstruction of a block.

[0069] In general, a source picture may be spatially subdivided into a number of sample coding blocks (e.g., blocks of 4x4, 8x8, 4x8 or 16x16 samples, respectively) and the blocks may be coded one by one. Blocks may be coded predictively with reference to other (already coded) blocks, as determined by the coding assignment applied to the picture corresponding to each block. For example, blocks of an I picture may be coded without prediction or with prediction with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P picture may be coded predictively with spatial prediction or with temporal prediction with reference to one previously coded reference picture. Blocks of a B picture may be coded predictively with spatial prediction or with temporal prediction with reference to one or two previously coded reference pictures. Source pictures or intermediate processing pictures may be subdivided into other types of blocks for other purposes. The division of coding blocks and other types of blocks may or may not follow the same manner, which is described in more detail below.

[0070] The video encoder (603) may perform coding operations in accordance with a pre-defined video coding technique or standard (e.g., ITU-T Recommendation H.265), in which the video encoder (603) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence, and thus the coded video data may conform to a syntax defined by the video coding technique or standard being used.

[0071] In some embodiments, the transmitter (640) may transmit ancillary data along with the coded video. The source coder (630) may include such data as part of the coded video sequence. The ancillary data may comprise temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures or slices, SEI messages, VUI parameter set fragments, etc.

[0072] A video may be captured in time order as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or other correlation) between pictures. For example, a particular picture being coded / decoded, called the current picture, may be divided into blocks. If a block in the current picture is similar to a reference block in a previously coded reference picture in the video that is still buffered, the block in the current picture may be coded by a vector, called a motion vector. The motion vector points towards the reference block in the reference picture, and may have a third dimension to identify the reference picture when multiple reference pictures are used.

[0073] In some embodiments, a bi-prediction technique can be used for inter-picture prediction. This bi-prediction technique uses two reference pictures, such as a first reference picture and a second reference picture, both of which precede the current picture in decoding order (or alternatively may be past or future in display order, respectively) in the video. A block in the current picture can be coded with a first motion vector pointing towards a first reference block in the first reference picture and a second motion vector pointing towards a second reference block in the second reference picture. A combination of the first and second reference blocks can be used together to predict a block.

[0074] Furthermore, the merge mode method can be used for inter-picture prediction to improve coding efficiency.

[0075] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction or intra-picture prediction, is performed on a block-by-block basis. For example, a picture in a sequence of video pictures may be divided into coding tree units (CTUs) for compression, and the CTUs in a picture may have the same size (e.g., 64×64 pixels, 32×32 pixels, or 16×16 pixels). In general, a CTU may include three parallel coding tree blocks (CTBs), i.e., one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree partitioned into one or more coding units (CUs). For example, a 64×64 pixel CTU may be partitioned into one CU of 64×64 pixels or four CUs of 32×32 pixels. Each of one or more of the 32×32 blocks may be further partitioned into four CUs of 16×16 pixels. In some embodiments, during encoding, each CU may be analyzed to determine the prediction type of the CU from various prediction types, such as an inter prediction type or an intra prediction type. A CU may be divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. In general, each PU includes one luma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation of coding (encoding / decoding) is performed in units of prediction blocks. The division of a CU into PUs (or PBs of different color channels) may be performed in various spatial patterns. A luma PB or a chroma PB may include a matrix of sample values ​​(e.g., luma values), for example, 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 samples, etc.

[0076] 7 shows a diagram of a video encoder (703) according to another embodiment of the present disclosure. The video encoder (703) is configured to receive a processed block of sample values ​​(e.g., a predictive block) in a current video picture in a sequence of video pictures and to encode the processed block into a coded picture that is part of a coded video sequence. The example video encoder (703) may be used in place of the example video encoder (403) of FIG. 4.

[0077] For example, the video encoder (703) receives a matrix of sample values ​​for a processing block, such as a prediction block of 8 by 8 samples. The video encoder (703) then determines, for example using rate-distortion optimization (RDO), whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode. If it is determined that the processing block is to be coded in intra mode, the video encoder (703) may use intra prediction methods to encode the processing block into a coded picture, and if it is determined that the processing block is to be coded in inter mode or bi-predictive mode, the video encoder (703) may use inter prediction or bi-predictive methods, respectively, to encode the processing block into a coded picture. In some embodiments, merge mode may be used as a sub-mode of inter-picture prediction that derives motion vectors from one or more motion vector predictors without the aid of coding motion vector components outside the predictor. In some other embodiments, there may be motion vector components applicable to the block of interest. Thus, the video encoder (703) may include components not explicitly shown in FIG. 7, such as a mode decision module that determines the prediction mode of a processing block.

[0078] In the example of FIG. 7, the video encoder (703) includes an inter-encoder (730), an intra-encoder (722), a residual value calculator (723), a switch (726), a residual value encoder (724), an overall controller (721), and an entropy encoder (725), connected together as shown in the example arrangement of FIG.

[0079] The inter encoder (730) is configured to receive a sample of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in previous and subsequent pictures in display order), generate inter prediction information (e.g., a description of redundant information according to an inter coding scheme, motion vectors, merge mode information), and calculate an inter prediction result (e.g., a predicted block) based on the inter prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture that has been decoded based on the coded video information using a decoder 633 (shown as residual decoder 728 in FIG. 7, described in more detail below) incorporated in the example encoder 620 of FIG. 6.

[0080] The intra encoder (722) is configured to receive samples of a current block (e.g., a processing block), compare the block to previously coded blocks in the same picture, generate transformed quantized coefficients, and possibly also generate intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). The intra encoder (722) may calculate an intra prediction result (e.g., a predicted block) based on the intra prediction information and a reference block in the same picture.

[0081] The overall controller (721) may be configured to determine overall control data and control other components of the video encoder (703) based on the overall control data. In one example, the overall controller (721) determines a prediction mode of the block and provides a control signal to the switch (726) based on the prediction mode. For example, if the prediction mode is an intra mode, the overall controller (721) controls the switch (726) to select an intra mode result for use by the residual value calculator (723) and controls the entropy encoder (725) to select intra prediction information and include the intra prediction information in the bitstream, and if the predication mode of the block is an inter mode, the overall controller (721) controls the switch (726) to select an inter prediction result for use by the residual value calculator (723) and controls the entropy encoder (725) to select inter prediction information and include the inter prediction information in the bitstream.

[0082] The residual value calculator (723) may be configured to calculate a difference (residual value data) between the received block and a prediction result for the block selected from the intra-encoder (722) or the inter-encoder (730). The residual value encoder (724) may be configured to encode the residual value data to generate transform coefficients. For example, the residual value encoder (724) may be configured to transform the residual value data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (703) also includes a residual decoder (728). The residual decoder (728) is configured to perform an inverse transform to generate decoded residual value data. The decoded residual value data can be used appropriately by the intra-encoder (722) and the inter-encoder (730). For example, the inter encoder (730) can generate a decoded block based on the decoded residual value data and the inter prediction information, and the intra encoder (722) can generate a decoded block based on the decoded residual value data and the intra prediction information. The decoded block is appropriately processed to generate a decoded picture, which can be buffered in a memory circuit (not shown) and used as a reference picture.

[0083] The entropy encoder (725) may be configured to format a bitstream to include the encoded block to perform entropy coding. The entropy encoder (725) may be configured to include various information in the bitstream. For example, the entropy encoder (725) may be configured to include overall control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual value information, and other suitable information in the bitstream. When coding a block in a merged sub-mode of either an inter mode or a bi-prediction mode, the residual value information may not be present.

[0084] 8 shows a diagram of an example video decoder (810) according to another embodiment of the present disclosure. The video decoder (810) is configured to receive coded pictures that are part of a coded video sequence and to decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (810) may be used in place of the example video decoder (410) of FIG. 4.

[0085] In the example of Figure 8, the video decoder (810) includes an entropy decoder (871), an inter decoder (880), a residual decoder (873), a reconstruction module (874), and an intra decoder (872), connected together as shown in the example arrangement of Figure 8.

[0086] The entropy decoder (871) may be configured to reconstruct from the coded picture certain symbols that represent syntax elements that make up the coded picture. Such symbols may include, for example, prediction information (e.g., intra-mode, inter-mode, bi-predicted mode, merge submode, or another submode) that may identify the mode used to code the block, certain samples or metadata used for prediction by the intra-decoder (872) or inter-decoder (880), and residual information, for example in the form of quantized transform coefficients. In one example, if the prediction mode is an inter-mode or bi-predicted mode, the inter-prediction information is provided to the inter-decoder (880), and if the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra-decoder (872). The residual information may be inverse quantized and provided to the residual decoder (873).

[0087] An inter decoder (880) may be configured to receive the inter prediction information and to generate an inter prediction result based on the inter prediction information.

[0088] The intra decoder (872) may be configured to receive the intra prediction information and to generate a prediction result based on the intra prediction information.

[0089] The residual decoder (873) may be configured to perform inverse quantization to extract dequantized transform coefficients, and process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (873) may also utilize certain control information (to include quantizer parameters (QP)), which may be provided by the entropy decoder (871) (this may be control information with only a small amount of data, so the data path is not shown).

[0090] The reconstruction module (874) may be configured to combine, in the spatial domain, the residual output by the residual decoder (873) with the prediction result (output by the inter-prediction module or the intra-prediction module, as the case may be) to form a reconstructed block that forms part of the reconstructed picture as part of the reconstructed video. It is noted that other suitable operations, such as a deblocking operation, may also be performed to improve video quality.

[0091] It is noted that the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using any suitable technique. In some embodiments, the video encoders (403), (603), and (703) and the video decoders (410), (510), and (810) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (403), (603), and (603) and the video decoders (410), (510), and (810) may be implemented using one or more processors executing software instructions.

[0092] Focusing on block partitioning used for coding and decoding, a typical partitioning may start from a base block and follow a predefined set of rules, a specific pattern, a partitioning tree, or any partitioning structure or scheme. The partitioning may be hierarchical or recursive. After dividing or partitioning the base block according to the example partitioning procedures described below or other procedures or any combination thereof, a final set of partitions or coding blocks may be obtained. Each such partition may be a partition at one of various partitioning levels of a partitioning hierarchy and may be of various shapes. Each partition may be referred to as a coding block (CB). In various partitioning implementations described further below, each resulting CB may be a CB of any allowed size and any allowed partitioning level. Such partitions are referred to as coding blocks because they may form units for which some basic coding / decoding decisions can be made and for which coding / decoding parameter optimization, determination, and signaling can be performed in the encoded video bitstream. The highest or deepest level of the final division represents the depth of the coding block division structure of the tree. The coding block may be a luma coding block or a chroma coding block. The CB tree structure of each color may be referred to as a coding block tree (CBT).

[0093] The coding blocks for all color channels may be collectively referred to as a coding unit (CU). The hierarchical structure used for all color channels may be collectively referred to as a coding tree unit (CTU). The division pattern or structure of the various color channels in the CTU may or may not be the same.

[0094] In some implementations, the partition tree scheme or structure used for the luma channel and the chroma channel may not need to be the same. In other words, the luma channel and the chroma channel may have separate coding tree structures or patterns. Furthermore, whether the same or different coding partition tree structures are used for the luma and chroma channels, and the actual coding partition tree structure used may depend on whether the slice being coded is a P slice, a B slice, or an I slice. For example, in an I slice, the chroma channel and the luma channel may have separate coding partition tree structures or coding partition tree structure modes, while in a P slice or a B slice, the luma channel and the chroma channel may share the same coding partition tree scheme. When separate coding partition tree structures or modes are applied, the luma channel may be partitioned into CBs by a certain coding partition tree structure, and the chroma channel may be partitioned into chroma CBs by another coding partition tree structure.

[0095] In some implementations, a predefined partitioning pattern may be applied to the base block. As shown in FIG. 9, an example of a four-way partitioning tree may start at an initial predefined level (for base block size, e.g., 64×64 block level or other size) and partition the base block down to a predefined lowest level (e.g., 4×4 level). For example, the base block may be subjected to four predefined partitioning options or partitioning patterns shown by 902, 904, 906, and 908, and the partitioning shown by R may be used for recursive partitioning in that the same partitioning options shown in FIG. 9 may be repeated on a smaller scale down to the lowest level (e.g., 4×4 level). In some implementations, other limitations may apply to the partitioning scheme of FIG. 9. In the implementation of FIG. 9, a rectangular partitioning option (e.g., 1:2 / 2:1 rectangular partitioning) may be used, but the rectangular partitioning option cannot be recursive, while the square partitioning option can be recursive. The final set of coding blocks is generated by partitioning according to FIG. 9 with recursion when necessary. The coding tree depth may be further defined to indicate the division depth starting from the root node or block. For example, the coding tree depth of the root node or block (e.g., a 64×64 block) may be set to 0, and the coding tree depth increases by 1 after one division of the root block according to FIG. 9. The maximum or deepest level from the 64×64 base block to the 4×4 minimum division is 4 (starting from level 0) in the above scheme. Such a division scheme may be applied to one or more of the color channels. Each color channel may be divided individually according to the scheme of FIG. 9 (e.g., the division pattern or option may be determined from the predefined pattern for each color channel at each hierarchical level separately). Alternatively, two or more of the color channels may share the same hierarchical pattern tree of FIG. 9 (e.g., the same division pattern or option may be selected from the predefined pattern for two or more color channels at each hierarchical level).

[0096] FIG. 10 shows another example of a predefined partitioning pattern that allows for forming a partitioning tree by recursive partitioning. As shown in FIG. 10, ten example partitioning structures or patterns can be predefined. A root block can originate from a predefined level (e.g., a base block at the 128×128 level or the 64×64 level). The example partitioning structures in FIG. 10 include various 2:1 / 1:2 rectangular partitions and 4:1 / 1:4 rectangular partitions. The partitioning type having three subdivision representations 1002, 1004, 1006, and 1008 in the second row of FIG. 10 may be applied to a "T-type" partition. The "T-type" partitions 1002, 1004, 1006, and 1008 may be referred to as left T-type, upper T-type, right T-type, and lower T-type. In some implementations, none of the rectangular partitions in FIG. 10 may be further subdivided. A coding tree depth may be further defined to indicate the partitioning depth originating from the root node or block. For example, the coding tree depth of the root node or root block (e.g., 128x128 block) can be set to 0, and the coding tree depth increases by 1 after one partition of the root block according to FIG. 10. In some implementations, the recursive partitioning to the next level of the partitioning tree according to the pattern of FIG. 10 can only use the all-square partitions in 1010. In other words, the square partitions in the T-type patterns 1002, 1004, 1006, and 1008 cannot be used for recursive partitioning. The final set of coding blocks is generated using a partitioning procedure according to FIG. 10 with recursion when necessary. Such a scheme can be applied to one or more of the color channels. In some implementations, more flexibility can be added to the use of partitions below the 8x8 level. For example, 2x2 chrominance interpretation can be used in some examples.

[0097] In another implementation of coding block partitioning, a quadtree structure can be used to partition a base block or an intermediate block into quadtree partitions. Such quadtree partitioning can be applied hierarchically and recursively to any square-shaped partition. Depending on various local characteristics of the base block or intermediate block / partition, the possibility of further quadtree partitioning of the base block or intermediate block or partition can be optimized. Furthermore, the quadtree partitioning at picture boundaries can be optimized. For example, an implicit quadtree partitioning can be performed at picture boundaries such that a block continues to be quadtree partitioned until its size fits the picture boundary.

[0098] Other implementations may use hierarchical bisections starting from the base block. In such a scheme, a base block or a mid-level block may be divided into two divisions. The bisections may be either horizontal or vertical. For example, a horizontal bisection may divide a base block or a mid-level block into equal left and right divisions. Similarly, a vertical bisection may divide a base block or a mid-level block into equal top and bottom divisions. Such bisections may be hierarchical and recursive. For each base block or mid-level block, a decision may be made as to whether to continue the bisection scheme, and if so, whether to use a horizontal or vertical bisection. In some implementations, further divisions may be stopped at a predefined minimum division size (either a minimum division size in one dimension or a minimum division size in both dimensions). Alternatively, further divisions may be stopped once a predefined division level or depth starting from the base block is reached. In some implementations, the aspect ratio of the divisions may be limited. For example, the aspect ratio of the divisions may not be less than 1:4 (or more than 4:1). Thus, for a vertical strip-shaped section with a vertical to horizontal aspect ratio of 4:1, it can only be further divided vertically into upper and lower sections, each of which has a vertical to horizontal aspect ratio of 2:1.

[0099] In yet some other examples, a third division scheme as shown in FIG. 13 can be used to divide the base block or any intermediate block. The third division pattern can be implemented vertically as shown in FIG. 13 at 1302 or horizontally as shown in FIG. 13 at 1304. Although the example division ratio in FIG. 13 is shown as 1:2:1 both vertically and horizontally, other ratios may be predefined. In some implementations, two or more different ratios may be predefined. Such a third division scheme can be used to complement a quadtree or binary division structure in that such triple tree division can capture objects in the middle of a block in one continuous division, whereas quadtrees and binary trees always divide along the block middle, which causes objects to be divided into separate divisions. In some implementations, the width and height of a division in a typical triple tree is always a power of two to avoid further transformations.

[0100] The above partitioning schemes can be combined in any manner at various partitioning levels. As an example, the above-mentioned quadtree partitioning scheme and binary partitioning scheme may be combined to partition the base block into a quadtree-binary-tree (QTBT) structure. In such a scheme, the base block or intermediate blocks / partitions may be either quadtree partitioned or binary partitioned according to a predefined set of conditions (if conditions are specified). A specific example is shown in FIG. 14. In the example of FIG. 14, the base block is first quadtree partitioned into four partitions shown at 1402, 1404, 1406 and 1408. Each of the resulting partitions is then either quadtree partitioned into four further partitions at the next level (e.g. 1408), binary partitioned into two further partitions (e.g. either horizontally or vertically as in 1402 or 1406, both of which are symmetrical), or not partitioned (e.g. 1404). As shown in the overall exemplary division pattern in 1410 and the corresponding tree structure / representation in 1420, binary division or quadtree division can be used recursively for square shaped divisions. In 1410 and 1420, solid lines represent quadtree division and dashed lines represent binary division. A flag can be used at each binary division node (non-leaf binary division) to indicate whether the binary division is horizontal or vertical. For example, as shown in 1420, which is consistent with the division structure in 1410, a flag "0" can be said to represent a horizontal binary division and a flag "1" can be said to represent a vertical binary division. For the divisions of quadtree division, there is no need to indicate the division type because in quadtree division, a block or division is always divided both horizontally and vertically to generate four sub-blocks / divisions of equal size. In some implementations, a flag "1" can represent a horizontal binary division and a flag "0" can represent a vertical binary division.

[0101] In some implementations of QTBT, the quadtree and binary partitioning rule sets can be represented by the following predefined parameters and associated corresponding functions: CTU size: The size of the root node of the quadtree (the size of the base block). MinQTSize: The minimum allowable quadtree leaf node size MaxBTSize: The maximum allowable binary tree root node size MaxBTDepth: Maximum allowed binary tree depth MinBTSize: The minimum allowable binary tree leaf node size

[0102] In some implementations of the QTBT partitioning structure, the CTU size can be set as 128×128 luma samples with corresponding 64×64 blocks of chroma samples (when using typical chroma subsampling assumptions), MinQTSize can be set as 16×16, MaxBTSize can be set as 64×64, MinBTSize (both width and height) may be set as 4×4, and MaxBTDepth can be set as 4. A quadtree partition can be applied to the CTU first to generate quadtree leaf nodes. The quadtree leaf nodes can range in size from 16×16 (i.e. MinQTSize), which is the minimum size a quadtree leaf node can have, to 128×128 (i.e. CTU size). If the node is 128×128, it will not be split by the binary tree first since its size exceeds MaxBTSize (i.e. 64×64). Otherwise, the node that does not exceed MaxBTSize can be split by the binary tree. In the example of FIG. 14, the base block is 128x128. According to the default rule set, only quadtree decomposition can be performed on the basic block. The decomposition depth of the base block is zero. Each of the four resulting decompositions is 64x64, which does not exceed MaxBTSize, and each decomposition can be further quadtree or bintree decomposed at level 1. The process proceeds. No further decompositions can be assumed when the bintree depth reaches MaxBTDepth (i.e. 4). No further horizontal decompositions can be assumed when the width of the bintree node is equal to MinBTSize (i.e. 4). Similarly, no further vertical decompositions can be assumed when the height of the bintree node is equal to MinBTSize.

[0103] In some implementations, the above QTBT scheme can be configured to accommodate luma and chroma flexibility with the same QTBT structure, or luma and chroma flexibility with separate QTBT structures. For example, in P and B slices, the luma CTB and chroma CTB in one CTU may share the same QTBT structure. Meanwhile, in an I slice, the luma CTB may be split into CBs with a QTBT structure, and the chroma CTB may be split into chroma CBs with another QTBT structure. This means that a CU can be used to apply to different color channels in an I slice, for example, an I slice may consist of a coding block of a luma component or a coding block of two chroma components, and a CU in a P or B slice may consist of coding blocks of all three color components.

[0104] In other implementations, the QTBT method can be supplemented with the above-mentioned third division method. Such implementations are sometimes referred to as multi-type-tree (MTT) structures. For example, in addition to the bisection of the nodes, one of the third division patterns in Figure 13 can be selected. In some implementations, only square nodes can be subjected to third division. An additional flag can be used to indicate whether the third division is horizontal or vertical.

[0105] The primary motivation for designing two-level and multi-level trees, such as the QTBT implementation and the QTBT implementation complemented by a trichotomous partition, is to reduce complexity. In theory, the complexity over the tree is T D where T is the number of split types and D is the depth of the tree. A tradeoff can be made by using multiple types (T) while reducing the depth (D).

[0106] In some implementations, the CB may be further divided. For example, the CB may be further divided into multiple prediction blocks (PBs) for intraframe or interframe prediction during the coding and decoding processes. In other words, the CB may be further divided into different subdivisions, and separate prediction decisions / configurations may be made. In parallel, the CB may be further divided into multiple transform blocks (TBs) to define the level at which the transformation or inverse transformation of the video data is performed. The manner of dividing the CB into PBs and TBs may be the same or different. For example, each division scheme may be performed using its own procedure, for example, based on various characteristics of the video data. In some implementations, the PB division scheme and the TB division scheme may be unrelated. In other implementations, the PB and TB division schemes and the boundaries may be correlated. In some implementations, for example, the TB may be divided after the PB division, and in particular, each PB may be further divided into one or more TBs after being determined according to the division of the coding blocks. For example, in some implementations, the PB may be divided into one TB, two TBs, four TBs, or other number of TBs.

[0107] In some implementations, the luma and chroma channels may be treated differently when dividing a base block into coding blocks and further into predictive and / or transform blocks. For example, in some implementations, the division of a coding block into predictive and / or transform blocks may be used for a luma channel, whereas the division of a coding block into predictive and / or transform blocks may not be used for one or more chroma channels. Thus, in such implementations, only the transformation and / or prediction of luma blocks may be performed at the coding block level. In another example, the minimum transform block size for a luma channel and one or more chroma channels may be different, e.g., allowing for the division of a coding block of a luma channel into transform blocks and / or predictive blocks that are smaller than the chroma channels. In yet another example, the maximum depth of division of a coding block and / or predictive block into transform blocks may be different for a luma channel and a chroma channel, e.g., allowing for the division of a coding block of a luma channel into transform blocks and / or predictive blocks that are deeper than the chroma channels. In a particular example, a luma coding block may be partitioned into transform blocks of multiple sizes, which may be expressed as a recursive partition going down up to two levels, allowing for transform block shapes such as square, 2:1 / 1:2, and 4:1 / 1:4, and transform block sizes from 4×4 to 64×64, whereas a chroma block may only allow the largest possible transform block specified for the luma block.

[0108] In some implementations of partitioning a coding block into PBs, the depth, shape, and / or other characteristics of the PB partition may depend on whether the PB is intra-coded or inter-coded.

[0109] The division of the coding block (or prediction block) into transform blocks may be performed recursively or non-recursively in various typical manners, including but not limited to quadtree division and predefined pattern division, and may be performed by further considering the transform blocks at the boundaries of the coding block or the prediction block. Generally speaking, the resulting transform blocks may be transform blocks of different division levels, may not be of the same size, and may not be required to be square in shape (e.g., they may be rectangular with some allowable sizes and aspect ratios). Further examples are described in more detail below with respect to Figures 15, 16, and 17.

[0110] In contrast to the above, in another implementation, the CB obtained through any of the above partitioning methods may be used as a basic coding block, i.e., a minimum coding block, for prediction and / or transformation. In other words, no further partitioning is performed for the purpose of inter-prediction / intra-prediction and / or transformation. For example, the CB obtained from the above QTBT method may be directly used as a unit for performing prediction. Specifically, the concept of multiple partition types is eliminated in such a QTBT structure, i.e., the separation of CU, PU, ​​and TU is eliminated, to accommodate the above-mentioned higher flexibility of the CU / CB partition shape. In such a QTBT block structure, the CU / CB can have both a square shape and a rectangular shape. The leaf nodes of such a QTBT are used as units for prediction and transformation processing without any further partitioning. This means that the block sizes of CU, PU, ​​and TU are the same in such an example of a QTBT coding block structure.

[0111] The various CB division schemes described above and further division of the CB into PB and / or TB (not including PB / TB division) may be combined in any manner. The following specific implementations are provided as non-limiting examples.

[0112] A specific implementation of coding block division and transform block division is described below. In such an implementation, a base block can be divided into coding blocks using recursive quadtree division or the above-mentioned predefined division patterns (such as the division patterns in Figs. 9 and 10). For each level, local video data characteristics can determine whether a particular division should be continued with further quadtree division. The obtained CBs may be CBs of different quadtree division levels and may be CBs of different sizes. The decision of whether a picture area should be coded using inter-picture (temporal) prediction or intra-picture (spatial) prediction can be made at the CB level (or at the CU level for all three color channels). Each CB can be further divided into one PB, two PBs, four PBs, or other number of PBs according to a predefined PB division type. Within one PB, the same prediction process can be applied, and related information can be sent to the decoder on a PB-by-PB basis. After obtaining the residual block by applying the prediction process based on the PB division type, the CB can be divided into TBs according to another quadtree structure similar to the coding tree of the CB. In this particular implementation, the CB or TB may be limited to a square shape, but is not required to be limited to this. Furthermore, in this particular example, the PB may be square or rectangular in shape in case of inter prediction, and the PB may be limited to a square in case of intra prediction. The coding block may be divided into, for example, four square-shaped TBs. Each TB may be further divided recursively (using quadtree division) into smaller TBs, which are called Residual Quadtrees (RQTs).

[0113] In the following, another implementation of splitting a base block into CB, PB, and / or TB will be further described. For example, instead of using multiple division unit types such as those shown in FIG. 9 or FIG. 10, a quadtree with nested multi-type tree (e.g., QTBT or QTBT with ternary division described above) using a bisection division structure and a trisection division structure can be used. Separation of CB, PB, and TB (i.e., splitting CB into PB and / or TB, and splitting PB into TB) can be discontinued except when necessary for CBs whose size is too large for the maximum transform length (such CBs may need to be further split). This exemplary division scheme can be designed to accommodate higher flexibility of CB division shape, so that both prediction and transformation can be performed on the CB level without further division. In such coding tree structure, CB can have both square and rectangular shapes. In particular, a coding tree block (CTB) can be first divided by a quadtree structure. Then, the quadtree leaf nodes can be further divided by a nested multi-type tree structure. An example of a nested multi-type tree structure using bisection or trisection is shown in Fig. 11. In particular, the exemplary multi-type tree structure in Fig. 11 includes four split types called vertical bisection (SPLIT_BT_VER) (1102), horizontal bisection (SPLIT_BT_HOR) (1104), vertical trisection (SPLIT_TT_VER) (1106) and horizontal trisection (SPLIT_TT_HOR) (1108). Then, CB corresponds to the leaf of the multi-type tree. In this implementation, this segmentation is used for both prediction and transformation processes without any further splitting, except when CB is too large for the maximum transformation length. This means that in most cases, the block sizes of CB, PB and TB are the same in the quadtree with nested multi-type tree coding block structure. An exception occurs when the maximum transform length that can be accommodated is less than the width or height of the color components of CB. In some implementations, in addition to a binary or trichotomous division, the nested pattern of FIG. 11 can further include a quadtree division.

[0114] One particular example of quadtrees (including quadtree options, bisection options and trisection options) with nested multitype tree coding block structures of block divisions for one base block is shown in Figure 12. In more detail, Figure 12 shows a base block 1200 being quadtree divided into four square divisions 1202, 1204, 1206 and 1208. For each division of the quadtree division, it is determined whether the multitype tree structure and quadtrees of Figure 11 should be further used for further division. In the example of Figure 12, division 1204 is not further divided. A different quadtree division is used for each of divisions 1202 and 1208. In the divider 1202, the upper left, upper right, lower left and lower right subdivisions that have undergone the second level quadtree division are respectively subjected to third level divisions, namely, quadtree, horizontal bisection 1104 in FIG. 11, no division and horizontal bisection 1108 in FIG. 11. Another quadtree division is used in the divider 1208, and the upper left, upper right, lower left and lower right subdivisions that have undergone the second level quadtree division are respectively subjected to third level divisions, namely, vertical bisection 1106 in FIG. 11, no division, no division and horizontal bisection 1104 in FIG. 11. Two of the subdivisions of the third level upper left subdivision in 1208 are further divided according to the horizontal bisection 1104 in FIG. 11 and horizontal bisection 1108 in FIG. 11, respectively. The division 1206 is divided into two divisions using a second level division pattern according to the vertical bisection division 1102 of Figure 11, and the two divisions are further divided at a third level according to the horizontal third division 1108 and vertical bisection division 1102 of Figure 11. A fourth level division is then applied to one of these according to the horizontal bisection division 1104 of Figure 11.

[0115] In the particular example above, the maximum luma transform size may be 64×64, and the maximum supported chroma transform size may be different from the luma (e.g., 32×32). Although the above exemplary CB in FIG. 12 is typically not further divided into smaller PBs and / or TBs, if the width or height of a luma coding block or a chroma coding block exceeds the maximum transform width or height, the luma coding block or the chroma coding block may still be automatically divided in the horizontal and / or vertical directions to satisfy the transform size limitation in that direction.

[0116] In the specific example of dividing the base block into CBs described above, the coding tree scheme can accommodate that luma and chroma can have separate block tree structures, as described above. For example, in P slices and B slices, the luma CTB and chroma CTB in one CTU may share the same coding tree structure. In I slices, for example, the luma and chroma may have separate coding block tree structures. When separate block tree structures are applied, the luma CTB can be divided into luma CBs by a certain coding tree structure, and the chroma CTB is divided into chroma CBs by another coding tree structure. This means that a CU in an I slice can consist of a coding block of a luma component or a coding block of two chroma components, and a CU in a P slice or B slice always consists of coding blocks of all three color components, except when the image is a monochrome image.

[0117] If a coding block is further divided into multiple transform blocks, the transform blocks in the coding block may be ordered in the bitstream according to various orders or may be ordered in the bitstream according to a scanning manner. Implementations of dividing a coding block or a prediction block into transform blocks and coding orders of the transform blocks are described in more detail below. In some implementations, as described above, the transform partitioning can accommodate transform blocks of multiple shapes (e.g., 1:1 (square), 1:2 / 2:1, and 1:4 / 4:1, when the transform block size ranges from, for example, 4×4 to 64×64). In some implementations, when the coding block is 64×64 or smaller, the transform block partitioning can be applied only to the luma component such that for chroma blocks, the transform block size is the same as the coding block size. In other cases, when the width or height of the coding block exceeds 64, both the luma coding block and the chroma coding block can be implicitly divided into transform blocks that are multiples of min(W,64)×min(H,64) and transform blocks that are multiples of min(W,32)×min(H,32), respectively.

[0118] In some implementations of transform block partitioning, for intra-coding and inter-coding blocks, the coding block can be further partitioned into multiple transform blocks using a partition depth up to a predefined number of levels (e.g., 2 levels). The partition depth and partition size of the transform block can be related. For some implementations, the mapping from the transform size of the current depth to the transform size of the next depth is shown in Table 1 as follows: [Table 1]

[0119] Based on the example of the correspondence in Table 1, for a 1:1 square block, the next level transform splitter can produce four 1:1 square sub-transform blocks. The transform splitting can be stopped at, for example, 4×4. Thus, the transform size of the current depth, which is 4×4, corresponds to the same size of 4×4 at the next depth. In the example of Table 1, for a non-square 1:2 / 2:1 block, the next level transform splitter can produce two 1:1 square sub-transform blocks, whereas for a non-square 1:4 / 4:1 block, the next level transform splitter can produce two 1:2 / 2:1 sub-transform blocks.

[0120] In some implementations, further restrictions on transform block partitioning can be applied for the luma components of intra-coding blocks. For example, at each level of transform partitioning, all sub-transform blocks can be restricted to be of equal size. For example, for a 32×16 coding block, the level 1 transform partitioning produces two 16×16+ sub-transform blocks, and the level 2 transform partitioning produces eight 8×8 sub-transform blocks. In other words, it is ensured that the second level partitioning is applied to all first-level sub-blocks to keep the transform units of the same size. An example of transform block partitioning for an intra-coding square block according to Table 1 is shown in FIG. 15 with the coding order indicated by the arrows. Specifically, 1502 denotes a square coding block. The first level partitioning according to Table 1 into four transform blocks of equal size is shown in 1504, with the coding order indicated by the arrows. A second level decomposition of all first level equal sized blocks into 16 equal sized transform blocks according to Table 1 is shown at 1506 with the coding order indicated by the arrows.

[0121] In some implementations, the above restrictions on intra-coding may not apply to the luma components of an inter-coding block. For example, after the first level of transform partitioning, any one of the sub-transform blocks may be further partitioned at an additional level independently of the others. Thus, the resulting transform blocks may or may not be of the same size. An example of partitioning an inter-coding block into transform blocks is shown in FIG. 16 along with their coding order. In the example of FIG. 16, an inter-coding block 1602 is partitioned into two levels of transform blocks according to Table 1. At the first level, the inter-coding block is partitioned into four transform blocks of the same size. Then, only one of the four transform blocks (but not all four) is further partitioned into four sub-transform blocks, resulting in a total of seven transform blocks of two different sizes as shown in 1604. An example of the coding order of these seven transform blocks is shown by the arrows at 1604 in FIG. 16.

[0122] In various embodiments, during a video coding process, data about motion may be sent to an input of an entropy encoder, such as entropy coder 645 shown in FIG. 6 or entropy encoder 725 shown in FIG. 7. The entropy encoder may output a bitstream (i.e., the coded video sequence / bitstream), which may be transmitted to another device via a transmission channel. The transmission channel may include, but is not limited to, a wired or wireless channel, or a combination of both. In various embodiments, during a video decoding process, the coded video sequence / bitstream may be sent to an input of an entropy decoder, such as entropy decoder 871 of FIG. 8. The entropy decoder may output data about motion based on the coded video sequence / bitstream, which may include intra prediction information, residual value information, etc.

[0123] In various embodiments, the entropy coder / decoder may encode / decode according to an arithmetic coding algorithm based on the probability of occurrence of a symbol (or symbol) on the basis of arithmetic coding. In some implementations, the probability of occurrence of a symbol (or symbol) may be dynamically updated during the coding / decoding process. For example, if there are only two symbols available ("a" and "b"), and the probability of occurrence of "a" is denoted as p_a and the probability of occurrence of "b" is denoted as p_b, then p_a+p_b=1 (or any other constant value). Therefore, when "a" appears in the coding / decoding process, p_a may be updated to a larger value and p_b may be updated to a smaller value, since the sum of these values ​​can be made constant. This probability update process may be called a probability transition process in some implementations and a probability state index update process in other implementations.

[0124] In some implementations, for example in HEVC, the entropy coder / decoder may use a context adaptive binary arithmetic coding (CABAC) algorithm, which may be a lossless compression algorithm and may use a lookup table-based probability transition process between 64 different representative probability states.

[0125] In some implementations, syntax elements describing video frame content are coded using binary arithmetic coding to obtain a stream that is coded as a binary bin stream. During CABAC, the initial interval [0,1) may be stretched by an integer multiplier (e.g., 512) and the least probable symbol probability (PLPS) may be expressed as an integer divisor with the quotient rounded up or down. A typical arithmetic coding interval splitting operation may then be performed as an approximation using integer arithmetic at the specified resolution.

[0126] In some implementations, the updated interval length corresponding to the LPS (RLPS) may be calculated as RLPS=R*PLPS, where R is the current interval length value. To save time and increase efficiency, the computationally intensive multiplication operations described above may be replaced with a lookup table populated with pre-calculated multiplication results, so that the updated interval length (ivLpsRange) corresponding to the LPS can be obtained by two indexes pStateIdx and qRangeIdx (i.e., ivlLpsRange=rangeTabLps[pStateIdx][qRangeIdx]).

[0127] During encoding / decoding, the probability value PLPS may be recursively updated each time a new value (binVal) of the bin being encoded / decoded is obtained. For example, at the kth step (i.e., during encoding or decoding of the kth bin), a new value of PLPS may be calculated to be a larger value if binVal is the value of the LPS, or a new value of PLPS may be calculated to be a smaller value if binVal is the value of the Most Probable Symbol (MPS).

[0128] In some implementations, the PLPS may be one of 64 possible values ​​indexed by a 6-bit pStateIdx variable. Updating the probability value may be accomplished by updating the index pStateIdx, which may be performed by looking up a value from a pre-computed table to reduce computational power and / or improve efficiency.

[0129] In some implementations, the range ivlCurrRange representing the state of the coding engine may be quantized to a set of four values ​​before the calculation of the new interval range. The transition state may be implemented using a table containing all 64×4 8-bit pre-calculated values ​​to approximate the value of ivlCurrRange*pLPS(pStateIdx). Also, the decoding decision may be implemented using a pre-calculated look-up table (LUT). After the initial ivlLpsRange is obtained using the LUT, the ivlLpsRange is used to update the ivlCurrRange based on the LUT to calculate the output binVal.

[0130] In some implementations, for example in VCC, the probability may be linearly represented by the probability index pStateIdx. Therefore, all calculations can be done in a formula without LUT operations. To improve the accuracy of the probability estimation, a multi-hypothesis probability update model may be used as shown in Fig. 17. The pStateIdx used for interval subdivision in the binary arithmetic coder is a combination of two probabilities pStateIdx0 and pStateIdx1. The two probabilities are associated with each context model and are updated separately with different adaptation rates. The adaptation rates of pStateIdx0 and pStateIdx1 for each context model are pre-trained based on the statistics of the associated bin. The probability estimate pStateIdx is the average of the estimates obtained from the two hypotheses.

[0131] Figure 17 shows a flow chart used to decode one decision (DecodeDecision), including the normalization process in the arithmetic decoding engine (RenomD, 1740). In some implementations, the input to DecodeDecision may be a context table (ctxTable) and a context index (ctxIdx). The value of the variable ivlLpsRange is derived as in step 1710. Given the current value of ivlCurrRange, the variable qRangeIdx is derived as qRangeIdx=ivlCurrRange>>5. Given qRangeIdx, pStateIdx0, and pStateIdx1 associated with ctxTable and ctxIdx, valMps and ivlLpsRange are derived as follows: pState=pStateIdx1+16*pStateIdx0, valMps=pState>>14 and ivlLpsRange=(qRangeIdx*((valMps?32767-pState:pState)>>9)>>1)+4. The variable ivlCurrRange is set to be ivlCurrRange-ivlLpsRange. If ivlOffset is greater than or equal to ivlCurrRange, then the variable binVal is set equal to 1-valMps, ivlOffset is decreased by ivlCurrRange, and ivlCurrRange is set equal to ivlLpsRange. Otherwise, the variable binVal is set equal to valMps.

[0132] For probability update, during the state transition process, the input to this process is the current pStateIdx0 and pStateIdx1 and the decoded value binVal, and the output of this process is the updated pStateIdx0 and pStateIdx1 of the context variables associated with ctxTable and ctxIdx. At 1730, the variables shift0 and shift1 are derived from the shiftIdx value associated with ctxTable and ctxIdx, i.e., shift0=(shiftIdx>>2)+2 and shift1=(shiftIdx&3)+3+shift0. Depending on the decoded value binVal, the updated values ​​of two variables pStateIdx0 and pStateIdx1 related to ctxTable and ctxIdx are obtained as pStateIdx0=pStateIdx0-(pStateIdx0>>shift0)+(1023*binVal>>shift0) and pStateIdx1=pStateIdx1-(pStateIdx1>>shift1)+(16383*binVal>>shift1).

[0133] In some implementations, for example, VVC CABAC may include a quantization parameter (QP) dependent initialization process that is called at the beginning of each slice. Given an initial value of luma QP for a slice, the initial probability state of the context model, denoted as preCtxState, can be obtained by m=slopeIdx×5-45, n=(offsetIdx<<3)+7 and preCtxState=Clip3(1,127,((m×(QP-32))>>4)+n). In some implementations, slopeIdx and offsetIdx are limited to 3 bits and the entire initialization value is represented with 6 bits of precision. The probability state preCtxState can directly represent the linear domain probabilities. Therefore, preCtxState only requires an appropriate shift operation before input to the arithmetic coding engine, and the mapping from the logarithmic domain to the linear domain and the 256-byte table can be stored / saved in memory by default. pStateIdx0 and pStateIdx1 can be obtained by pStateIdx0=preCtxState<<3 and pStateIdx1=preCtxState<<7.

[0134] In some implementations, the CABAC algorithm may be binary based, which includes two possible symbols (e.g., "0" and "1"). In binary arithmetic coding algorithms, the two possible symbols are sometimes referred to as the least likely symbol (LPS) and the most likely symbol (MPS).

[0135] This disclosure describes various embodiments for performing multi-symbol arithmetic coding and / or decoding that solve at least one problem / challenge associated with binary symbol arithmetic systems to improve efficiency and / or increase flexibility.

[0136] In some implementations, an entropy encoder or decoder may use arithmetic algorithms using an M-ary basis, which includes M possible symbols / symbols, where M may be any integer value from 2 to 16. As a non-limiting example, for M=5, the M-ary basis includes 5 possible symbols / symbols, which may be denoted as "0", "1", "2", "3", and "4".

[0137] An M-ary arithmetic coding engine is used to entropy code the syntax elements. Each syntax element is associated with an alphabet of M elements. As input to an encoder or decoder, a coding context may include a sequence of M-ary symbols along with a set of M probabilities. Each of the M probabilities may correspond to each of the M-ary symbols, which may be represented by a cumulative distribution function (CDF). In some implementations, the cumulative distribution function of the M-ary symbols is C=[c 0 ,c 1 ,…,c (M-2) ,c (M-1) In some implementations, the cumulative distribution function of an M-ary symbol may be represented as an array of M 15-bit integers, where c (M-1) =2 15 and c n / 32768 is the probability of n or fewer symbols, where n is an integer from 0 to M-1.

[0138] In some implementations, the M probabilities (i.e., the array of cumulative distribution functions) may be updated after coding / parsing each syntax element. In some implementations, the M probabilities (i.e., the array of cumulative distribution functions) may be updated after coding / decoding each M-ary symbol. As a non-limiting example, if the array of cumulative distribution functions is [c 0 ,c 1 ,…,c (M-2) ,c (M-1) ], so for M=4, the array of cumulative distribution functions is [c 0 ,c 1 ,c 2 ,c 3] is included.

[0139] In some implementations, the M probabilities (ie, the array of cumulative distribution functions) may be updated using the following formula:

number

[0140] where symbol is the M-ary symbol currently being coded / decoded, α is a probability update rate that adapts based on the number of times the symbol has been coded / decoded (up to a maximum of 32), and m is the index of an element in the CDF. This adaptation of α allows for a fast probability update at the beginning of coding / parsing a syntax element. As a non-limiting example, if M=5 and the currently decoded M-ary symbol is "3", then m∈[0,symbol) is said to be m∈[0,3), where m includes any integer from 0 (inclusive) to 3 (exclusive) (i.e., m includes 0, 1, and 2), and m∈[symbol,M-1) is said to be m∈[3,4), where m includes any integer from 3 (inclusive) to 4 (exclusive) (i.e., m includes only 3).

[0141] In some implementations, the M-ary arithmetic coding process may follow a conventional arithmetic coding engine design, except that only the most significant 9 bits of the 15-bit probability value are input to the arithmetic encoder / decoder. The probability update rate α associated with a symbol is calculated based on the occurrence of the associated symbol when parsing the bitstream, and the value of α is reset at the beginning of a frame or tile using the following formula:

[0142] In some implementations, for example:

number

[0143] In some implementations, a single probability update model may be used for all contexts, which may overlook the fact that the optimal design of the probability update model may differ for different syntaxes and / or that the single probability update model may be less than optimal. In this disclosure, various embodiments that apply syntax-dependent and multi-parameter probability update models are described, which may address at least the above-mentioned problems / challenges and / or may improve coding / decoding performance.

[0144] 18 shows a flow chart of an example method 1800 of multiple symbol arithmetic coding during video decoding / encoding. The flow of the example decoding method begins at S1801 and ends at S1899. Method 1800 may include some or all of steps S1810, in which a device having a memory for storing instructions and a processor in communication with the memory receives a coded video bitstream, S1820, in which the device obtains an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1, and / or S1830, in which the device performs arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions, where for each extracted symbol, the device updates the array of cumulative distribution functions in response to the extracted symbol based on at least one probability update rate, and the device continues arithmetic decoding on the coded video bitstream to extract a next symbol based on the updated array of cumulative distribution functions. In some implementations, the array of cumulative distribution functions is [c 0 ,c 1 ,…,c (M-2) ,c (M-1) ]. As a non-limiting example, when M=4, the array of cumulative distribution functions may be [c 0 ,c 1 ,c 2 ,c 3 ], and each element of the array may be calculated as a 15-bit integer.

[0145] In the embodiments and implementations of the present disclosure, any steps and / or operations may be combined or arranged in any total number or order as desired. Two or more of the steps and / or operations may be performed in parallel. The embodiments and implementations of the present disclosure may be used separately or combined in any order. Furthermore, each of the method (or embodiment), the encoder, and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to a luma block or a chroma block. The term "block" may be interpreted as a prediction block, a coding block, or a coding unit (i.e., CU). The term "block" in this disclosure may also be used to refer to a transform block. In the following, when describing a block size, it may refer to the block width or height, or the maximum width and height, or the minimum width and height, or the area dimension (width x height), or the aspect ratio of the block (width:height or height:width).

[0146] In some implementations of method 1800, the at least one probability update rate comprises a probability update rate and / or the probability update rate comprises a function f(N,M), where N is the number of occurrences of the associated symbol when parsing the coded video bitstream.

[0147] In some implementations of the method 1800, the function f(N,M) is

number

[0148] In some implementations of method 1800, g(M,D) is min(log 2 (M),D).

[0149] In some implementations of the method 1800, default function parameters are initialized for decoding a tile or frame.

[0150] In some implementations of method 1800, the predefined function parameters are obtained from a high level syntax comprising at least one of a video parameter set (VPS), a picture parameter set (PPS), a sequence parameter set (SPS), an adaptation parameter set (APS), a picture header, a frame header, a slice header, a tile header, or a coding tree unit (CTU) header.

[0151] In some implementations of the method 1800, the function f(N,M) is

number

[0152] In some implementations of the method 1800, the function f(N,M) is

number

[0153] In some implementations of method 1800, the at least one probability update rate comprises K probability update rates, where K is an integer greater than one.

[0154] In some implementations of method 1800, each probability update rate comprises a function f(N,M), where N is a number of occurrences of the associated symbol when parsing the coded video bitstream, and / or updating the array of cumulative distribution functions in response to the extracted symbols based on the at least one probability update rate comprises updating K arrays of cumulative distribution functions based on the corresponding K probability update rates, and / or updating the array of cumulative distribution functions as a weighted sum of the K updated arrays of cumulative distribution functions.

[0155] In some implementations of method 1800, K is 2 and / or updating the array of cumulative distribution functions as a weighted sum of the K updated arrays of cumulative distribution functions is a weighted sum of w 1 *p 1 +w 2 *p 2 updating the array of cumulative distribution functions as 1 is the first weight, and p 1 is the updated array of the first cumulative distribution function based on the first probability update rate, and w 2 is the second weight, and p 2 is the updated array of the second cumulative distribution function based on the second probability update rate.

[0156] In some implementations of the method 1800, the first weight and the second weight are equal or the first weight and the second weight are different and are predefined.

[0157] In some implementations of method 1800, one of the K probability update rates is

number

[0158] In some implementations of method 1800, one of the K probability update rates is

number

number

[0159] In some implementations of method 1800, the predefined function parameters are obtained from a high level syntax comprising at least one of a video parameter set (VPS), a picture parameter set (PPS), a sequence parameter set (SPS), an adaptation parameter set (APS), a picture header, a frame header, a slice header, a tile header, or a coding tree unit (CTU) header.

[0160] As a non-limiting example, the probability update rate α may be a function f(count,M) of count and M, where count is the number of occurrences of the associated symbol / context when parsing a bitstream during decoding or when coding data corresponding to a set of syntax elements during encoding, and M represents the number of distinct symbol values ​​of the associated syntax / context. The function f(count,M) may be different for different syntax / contexts.

[0161] In some implementations, the function

number

[0162] In some implementations,

number

[0163] In some implementations, the function

number

[0164] In some implementations, values ​​of the function parameters defined above may be specified in a high level syntax / context, including but not limited to VPS, PPS, SPS, APS, picture / frame header, slice header, tile header, and CTU header.

[0165] In some implementations, different probability update rates α i , i=0,1,...,N-1, while updating the probability as a weighted sum of multiple probabilities (e.g., N probabilities) derived using i is a function f of count and M i (count,M), where count is the number of occurrences of the associated symbol when parsing the bitstream, and M represents the number of distinct symbols of the associated syntax / context.

[0166] In some implementations, the probability is updated as a weighted sum of N probabilities. As a non-limiting example, N is equal to 2, and the two probabilities (p 0 and p 1 ) with different probability update rates α 0 and α 1 In some implementations, the weights are equal. In some implementations, the weights are unequal and are predefined.

[0167] In some implementations, the probability is updated as a weighted sum of multiple probabilities (e.g., N probabilities), where one of the probabilities is

number

[0168] In some implementations, the probability is updated as a weighted sum of multiple probabilities (e.g., N probabilities), where one of the probabilities is

number

number

[0169] In some implementations, the function f is used for different syntaxes / contexts. i (count,M) is different.

[0170] In some implementations, the values ​​of the function parameters and / or weight parameters of the various implementations described above may be specified in a high level syntax / context, including but not limited to the VPS, PPS, SPS, APS, picture / frame header, slice header, tile header, and CTU header.

[0171] The various embodiments and / or implementations described in this disclosure may be used individually or combined in any order. Furthermore, some, all, or any partial or full combination of these embodiments and / or implementations may be implemented as part of an encoder and / or decoder and may be implemented in hardware and / or software. For example, the above may be hard-coded into dedicated processing circuitry (e.g., one or more integrated circuits). In another example, the above may be implemented by one or more processors executing a program stored on a non-transitory computer-readable medium. Furthermore, each of the methods (or embodiments), the encoder, and the decoder may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium. The embodiments of the present disclosure may be applied to luma blocks or chroma blocks, and in the case of chroma blocks, the embodiments may be applied to one or more color components individually or together.

[0172] The techniques described above can be implemented using computer readable instructions and as computer software physically stored on one or more computer readable media. For example, Figure 19 illustrates a computer system (2000) suitable for implementing some embodiments of the disclosed subject matter.

[0173] Computer software may be coded using any suitable machine or computer language, which may be processed by assembly, compilation, linking, or similar mechanisms to generate code comprising instructions which may be executed directly, interpreted, executed by microcode, or the like, by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc.

[0174] The instructions may be executed in various types of computers or components thereof including, for example, a personal computer, a tablet computer, a server, a smart phone, a gaming console, an Internet of Things device, and the like.

[0175] The components illustrated in Figure 20 of the computer system (2000) are exemplary of course and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure, nor should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (2000).

[0176] The computer system (2000) may include several human interface input devices that may be responsive to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, clapping), visual input (e.g., gestures), and olfactory input (not shown). The human interface devices may also be used to capture media that is not necessarily directly associated with conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images captured from a still image camera), and video (e.g., two-dimensional video, three-dimensional video, including stereoscopic video).

[0177] The input human interface devices may include one or more (only one of each is shown) of a keyboard (2001), a mouse (2002), a trackpad (2003), a touch screen (2010), a data glove (not shown), a joystick (2005), a microphone (2006), a scanner (2007), and a camera (2008).

[0178] The computer system (2000) may also include a number of human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (2010), data gloves (not shown) or joystick (2005), although there may also be haptic feedback devices that are not used as input devices), audio output devices (speakers (2009), headphones (not shown), etc.), visual output devices (screens (2010), including CRT screens, LCD screens, plasma screens, OLED screens, etc., each with or without touch screen input capability and with or without haptic feedback capability - some of the above screens may be capable of outputting two-dimensional visual output, or three or more dimensional output through means such as stereoscopic graphic output; virtual reality glasses (not shown), holographic displays and smoke tanks (not shown)), and printers (not shown).

[0179] The computer system (2000) may also include storage devices and associated media that may be directly operated by a human, such as optical media including a CD / DVD ROM / RW (2020) using CD / DVD or similar media (2021), thumb-drives (2022), removable hard drives or solid-state drives (2023), legacy magnetic media such as tapes or floppy disks (not shown), and specialized ROM / ASIC / PLD based devices (not shown) such as security dongles.

[0180] Those skilled in the art will also appreciate that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other transitory signals.

[0181] The computer system (2000) may also include an interface (2054) to one or more communications networks (2055). The networks may be, for example, wireless, wired, optical, etc. Furthermore, the networks may be local, wide area, metropolitan, vehicular, industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable, satellite, and terrestrial television, vehicular and industrial networks including CAN bus, etc. Some networks typically require an external network interface adapter that is attached to some general-purpose data port or peripheral bus (2049) (e.g., a USB port of the computer system (2000)), while other networks are typically built into the core of the computer system (2000) by attaching to the system bus as described below (e.g., a PC computer system may have an Ethernet interface and a smartphone computer system may have a cellular network interface). Any of these networks may be used by the computer system (2000) to communicate with another party. Such communication may be one-way, receive-only (e.g., television broadcast), one-way transmit-only (e.g., CANbus to a specific CANbus device), or two-way, e.g., two-way communication to another computer system using a local or wide area digital network. Several protocols and protocol stacks may be used for each of these networks and network interfaces described above.

[0182] The above-mentioned human interface devices, storage devices that can be directly operated by humans, and network interfaces can be attached to the core (2040) of the computer system (2000).

[0183] The cores (2040) may include one or more central processing units (CPUs) (2041), graphics processing units (GPUs) (2042), specialized programmable processing units in the form of field programmable gate areas (FPGAs) (2043), hardware accelerators for specific tasks (2044), graphics adapters (2050), etc. Such devices may be connected through a system bus (2048) along with read only memory (ROM) (2045), random access memory (2046), and internal mass storage (2047) such as hard drives, SSDs, etc. that are internal and not directly accessible to the user. In some computer systems, the system bus (2048) may be directly accessible in the form of one or more physical plugs that allow expansion with additional CPUs, GPUs, etc. Peripherals may be attached directly to the core's system bus (2048) or through a peripheral bus (2049). In one example, a screen (2010) may be connected to the graphics adapter (2050). Peripheral bus architectures include PCI, USB, etc.

[0184] The CPU (2041), GPU (2042), FPGA (2043) and accelerator (2044) can execute a number of instructions that can be combined to construct the above-mentioned computer code. The computer code can be stored in a ROM (2045) or a RAM (2046). Transient data can also be stored in the RAM (2046), while immutable data can be stored, for example, in an internal mass storage (2047). A cache memory, which can be closely associated with one or more of the CPU (2041), GPU (2042), mass storage (2047), ROM (2045), RAM (2046), etc., can be used to enable fast storage and retrieval from any of the memory devices.

[0185] The computer-readable medium may bear computer code for performing various computer-implemented operations, and the medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those having skill in the computer software arts.

[0186] As a non-limiting example, a computer system having the architecture (2000), and in particular the core (2040), may provide functionality as a result of one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be media associated with mass storage that may be directly manipulated by a user, as described above, or may even be specific storage of a non-transitory nature of the core (2040), such as the core's internal mass storage (2047) or ROM (2045). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the core (2040). The computer-readable media may include one or more memory devices or chips, depending on the particular requirements. The software enables the cores (2040), and in particular the processors (including CPUs, GPUs, FPGAs, etc.) in the cores (2040), to execute the particular processes or particular parts of the particular processes described above, including the definition of data structures stored in RAM (2046) and the modification of such data structures according to the processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of logic being hardwired or otherwise implemented in circuitry (e.g., accelerators (2044)) that may act in place of or cooperate with the software to execute the particular processes or particular parts of the particular processes described in this application. Where appropriate, references to "software" may include logic, and vice versa. Where appropriate, references to "computer-readable medium" may include circuitry (such as integrated circuits (ICs)) that store the software to be executed, circuitry that implements the logic to be executed, or both. Any suitable combination of hardware and software is included in the present disclosure.

[0187] Although a particular invention has been described with reference to exemplary embodiments, this description is not intended to be limiting. Various modifications of the exemplary embodiments and further embodiments of the invention will be apparent to those skilled in the art from this description. Those skilled in the art will readily appreciate that such modifications and various other modifications may be made to the exemplary embodiments illustrated and described in this disclosure without departing from the spirit and scope of the invention. It is therefore contemplated that the appended claims will cover such modifications and that the embodiments may be modified. Certain proportions in the illustrations may be exaggerated, while other proportions may be exaggerated. Therefore, the present disclosure and figures are to be considered illustrative rather than limiting. [Explanation of symbols]

[0188] 300 Communication Systems 310 Terminal Equipment 320 Terminal Equipment 330 Terminal Equipment 340 Terminal Equipment 350 Communication Network 400 Communication Systems 401 Video Source 402 Stream 403 Video Encoder 404 Encoded Video Bitstream 405 Streaming Server 406 Client Subsystem 407 Copying of Encoded Video Data 408 Client Subsystem 409 Copying of Encoded Video Data 410 Video Decoder 411 Output Stream 412 Display 413 Video Imaging Subsystem 420 Electronic equipment 430 Electronic equipment 501 Channel 510 Video Decoder 512 Display device 515 Buffer Memory 520 Parser 521 Symbols 530 Electronic equipment 531 Receiver 551 Scaler / Inverse Converter 552 Intra Prediction Unit 553 Motion compensation prediction unit 555 Collector 556 Loop filter section 557 Reference Picture Memory 558 Current Picture Buffer 601 Video Source 603 Video Encoder 620 Electronic equipment 630 Source Code 632 Coding Engine 633 Built-in Decoder 634 Reference Picture Memory 635 Predictor 640 Transmitter 643 Video Sequence 645 Entropy Coder 650 Controller 660 Communication Channel 703 Video Encoder 721 General Controller 722 Intra Encoder 723 Residual value calculator 724 Residual Value Encoder 725 Entropy Encoder 726 Switch 728 Residual Decoder 730 Interencoder 810 Video Decoder 871 Entropy Decoder 872 Intra Decoder 873 Residual Decoder 874 Reconstruction Module 880 Interdecoder 2000 Computer Systems 2001 Keyboard 2002 Mouse 2003 Trackpad 2005 Joystick 2006 Mike 2007 Scanner 2008 Camera 2009 Audio output device 2010 Touch Screen 2020 CD / DVD ROM / RW 2021 CD / DVD 2022 thumb drive 2023 Solid State Drive 2040 Core 2041 Central Processor 2042 Graphics Processing Unit 2043 Field Programmable Gate Area 2044 Accelerator 2044 Hardware Accelerator 2045 Read Only Memory 2046 Random Access Memory 2047 Mass Storage 2048 System Bus 2049 Peripheral Bus 2050 Graphics Adapter 2054 Interface 2055 Communication Network

Claims

1. 1. A processor-implemented method for multiple-symbol arithmetic coding in video decoding, the method comprising: receiving a coded video bitstream; obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1; performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions; for each extracted symbol, updating the array of cumulative distribution functions in response to the extracted symbol based on K probability update rates, K being an integer greater than 1, each probability update rate having a function f(N,M), N being a number of occurrences of an associated symbol when parsing the coded video bitstream; updating the K arrays of cumulative distribution functions based on the K corresponding probability update rates; updating the array of cumulative distribution functions as a weighted sum of the updated K arrays of cumulative distribution functions. updating the array of cumulative distribution functions; performing arithmetic decoding based on the updated arrangement of cumulative distribution functions by continuing arithmetic decoding on the coded video bitstream to extract a next symbol; A method comprising:

2. A processor-implemented method for multiple-symbol arithmetic coding in video decoding, the method comprising: receiving a coded video bitstream; obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1; performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions; and updating, for each extracted symbol, the array of cumulative distribution functions in response to the extracted symbol based on a probability update rate, the probability update rate being: [0010] where A, B, C, and D are predefined function parameters obtained from a high level syntax comprising at least one of a video parameter set (VPS), a picture parameter set (PPS), a sequence parameter set (SPS), an adaptation parameter set (APS), a picture header, a frame header, a slice header, a tile header, or a coding tree unit (CTU) header, and g(M,D) is a function of M with D as a function parameter, and N is the number of occurrences of an associated symbol when parsing the coded video bitstream; performing arithmetic decoding based on the updated arrangement of cumulative distribution functions by continuing arithmetic decoding on the coded video bitstream to extract a next symbol; A method comprising:

3. g(M,D) is min(log 2 (M), D) The method of claim 2.

4. the predefined function parameters are initialized for decoding a tile or a frame; The method of claim 2.

5. A processor-implemented method for multiple-symbol arithmetic coding in video decoding, the method comprising: receiving a coded video bitstream; obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1; performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions; and updating, for each extracted symbol, the array of cumulative distribution functions in response to the extracted symbol based on a probability update rate, the probability update rate being: [0025] where A, B, C, D, and E are predefined function parameters, and g(M,E) is a function of M with E as a function parameter, and N is the number of occurrences of an associated symbol when parsing the coded video bitstream; performing arithmetic decoding based on the updated arrangement of cumulative distribution functions by continuing arithmetic decoding on the coded video bitstream to extract a next symbol; A method comprising:

6. A processor-implemented method for multiple-symbol arithmetic coding in video decoding, the method comprising: receiving a coded video bitstream; obtaining an array of cumulative distribution functions and corresponding M symbols, where M is an integer greater than 1; performing arithmetic decoding on the coded video bitstream to extract at least one symbol based on the array of cumulative distribution functions; and updating, for each extracted symbol, the array of cumulative distribution functions in response to the extracted symbol based on a probability update rate, the probability update rate being: [0030] where A and B are predefined function parameters, LUT(N) is a lookup table with N as an index, and g(M,B) is a function of M with B as a function parameter, and N is the number of occurrences of an associated symbol when parsing the coded video bitstream; performing arithmetic decoding based on the updated arrangement of cumulative distribution functions by continuing arithmetic decoding on the coded video bitstream to extract a next symbol; A method comprising:

7. K is 2, Updating the array of cumulative distribution functions as a weighted sum of the K updated arrays of cumulative distribution functions w 1 *p 1 +w 2 *p 2 updating the array of cumulative distribution functions as w 1 is the first weight, and p 1 is the updated array of the first cumulative distribution function based on the first probability update rate, and w 2 is the second weight, and p 2 is the updated array of the second cumulative distribution function based on the second probability update rate, The method of claim 1.

8. the first weight and the second weight are equal, or The first weight and the second weight are different and are predefined. The method of claim 7.

9. One of the K probability update rates is [0045] where A, B, C and D are predefined model parameters. The method of claim 1.

10. One of the K probability update rates is [0050] Equipped with Another one of the K probability update rates is [006] Equipped with A, B, C, D and E are predefined model parameters, different from A and E. The method of claim 1.

11. the predefined function parameters are derived from a high level syntax comprising at least one of a video parameter set (VPS), a picture parameter set (PPS), a sequence parameter set (SPS), an adaptation parameter set (APS), a picture header, a frame header, a slice header, a tile header, or a coding tree unit (CTU) header; The method of claim 10.

12. A device configured to perform the method according to any one of claims 1 to 11.

13. A computer program product for causing a processor to carry out the method according to any one of claims 1 to 11.

Images