Method and apparatus for approximating cumulative distribution functions for use in entropy encoding or decoding of data - Patents.com

By using CDFs defined by quadratic polynomials for entropy encoding and decoding, the solution addresses the suboptimal integration of neural networks in hybrid codecs, enhancing efficiency and compatibility in image and video compression.

JP2024543333AInactive Publication Date: 2024-11-21HUAWEI TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2024525753
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-12-23
Publication Date
2024-11-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing hybrid image and video codecs optimize transform, quantization, and entropy coding separately, lacking an integrated approach for efficient encoding and decoding using neural networks, leading to suboptimal compression performance.

Method used

Employing cumulative distribution functions (CDFs) defined by quadratic polynomials for entropy encoding and decoding, with conditions ensuring monotonicity and accuracy, allowing for efficient storage and retrieval across different platforms, and supporting neural network-based encoding frameworks.

Benefits of technology

Enhances the efficiency of entropy encoding and decoding processes, reducing complexity and ensuring compatibility across hardware and software platforms, while maintaining high compression performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024543333000049
    Figure 2024543333000049
  • Figure 2024543333000050
    Figure 2024543333000050
  • Figure 2024543333000051
    Figure 2024543333000051
Patent Text Reader

Abstract

A method and apparatus are provided for approximating a cumulative distribution function (CDF) for each interval using a quadratic polynomial while imposing constraints on the polynomials within the interval and / or on the boundaries between the intervals. In this manner, a CDF approximation is obtained that can be used in a variety of applications, including entropy encoding and decoding of any source data. The constraints correspond to properties of the CDF being approximated.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This disclosure relates to obtaining cumulative distribution functions, eg, for picture or video coding, for use in, for example, entropy encoding and decoding. [Background technology]

[0002] Hybrid image and video codecs have been used for decades to compress image and video data. In such codecs, a signal is typically encoded block-wise by predicting the block and further coding only the difference between the original block and its prediction. In particular, such coding can include transform, quantization, and bitstream generation, and usually includes some kind of entropy coding. Typically, the three components of a hybrid coding method - transform, quantization, and entropy coding - are optimized separately. Modern video compression standards such as High-Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and Essential Video Coding (EVC) also code the predicted residual signal using a transformed representation.

[0003] Recently, neural network architectures have been applied to image and / or video coding. In general, these neural network (NN)-based approaches can be applied to image and video coding in a variety of different ways. For example, several end-to-end optimized image or video coding frameworks have been discussed. Furthermore, deep learning has been used to determine or optimize some parts of the end-to-end coding framework, such as prediction parameter selection or compression, etc. Moreover, several neural network-based approaches have also been discussed for use in hybrid image and video coding frameworks, for example, for implementation as trained deep learning models for intra- or inter-prediction in image or video coding.

[0004] The end-to-end optimized image or video coding applications mentioned above have in common that they generate some feature map data to be communicated between the encoder and the decoder.

[0005] A neural network is a machine learning model that uses one or more layers of nonlinear units that can predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. A corresponding feature map can be provided as an output of each hidden layer. Such corresponding feature map of each hidden layer can be used as an input for a subsequent layer in the network, i.e., a subsequent hidden layer or an output layer. Each layer of the network generates an output from a received input according to the current value of a respective set of parameters. In a neural network that is split between devices, e.g., between an encoder and a decoder, between a device and a cloud, or between different devices, the feature map at the output of the splitting location (e.g., a first device) is compressed and then transmitted to the remaining layers of the neural network (e.g., a second device).

[0006] Additionally, improvements in encoding and decoding using trained network architectures may be desirable. Summary of the Invention

[0007] SUMMARY OF THE PRESENT EMBODIMENTS The present invention provides an apparatus and method for obtaining a cumulative distribution function for entropy encoding and decoding.

[0008] Embodiments of the invention are defined by the features of the independent claims, and further advantageous implementations of the embodiments are defined by the features of the dependent claims.

[0009] According to one embodiment, there is provided a method for entropy coding data into a bitstream, comprising the steps of obtaining a cumulative distribution function, CDF, for at least one interval of a plurality of intervals, said CDF being obtained based on a function defined in said at least one interval by parameters of a quadratic polynomial, said quadratic polynomial satisfying a set of conditions, and applying a probability model based on the CDF obtained in said at least one interval to entropy code said data.

[0010] Obtaining the CDF based on a set of parameters describing a quadratic polynomial provides a reliable and reproducible method for obtaining estimates of the probability model used in entropy coding. The function represents an approximation of the CDF using a quadratic polynomial, while conditions allow a better adaptation of the approximation to some intrinsic properties of the CDF.

[0011] In an exemplary implementation, the set of conditions includes one or more of: the quadratic polynomial in the at least one interval and the polynomials in adjacent intervals provide monotonically non-decreasing functions; the quadratic polynomial has a non-negative derivative within the at least one interval.

[0012] These conditions express certain properties of the approximation given by the function, i.e. monotonicity, curvature, etc. They reflect the characteristics of the CDF and therefore result in an approximation suitable for use as a CDF in entropy coding.

[0013] For example, the function defined by the parameters specifies a predefined CDF having a predefined mean and a predefined standard deviation.

[0014] Providing default CDFs has the advantage that such default CDFs can be efficiently stored or derived when needed, while possible further CDFs can be obtained based on the default CDFs.

[0015] In an exemplary implementation, the CDF is obtained by shifting at least one argument of the function.

[0016] Shifting the arguments makes it possible to use the approximation provided by the function, for example, to obtain additional CDFs with means different from the default mean used in obtaining the function. Only the parameters of the predefined CDFs need to be stored or derived.

[0017] For example, the CDF is obtained by scaling at least one argument of the function.

[0018] Scaling the arguments makes it possible, for example, to use the approximation provided by the function to obtain additional CDFs with standard deviations different from the default standard deviation used in obtaining the function. Only the parameters of the predefined CDFs need to be stored or derived.

[0019] In an exemplary implementation, the entropy coding is an arithmetic encoding.

[0020] Arithmetic encoding is an efficient entropy coding that can contribute to rate reduction. It can advantageously use the CDF approximation obtained above.

[0021] For example, the function in one or more of the intervals is defined by a linear polynomial.

[0022] Using a linear approximation in regions with approximately constant slope may, for example, reduce the number of parameters provided to specify the approximation function.

[0023] In an exemplary implementation, the function in a last one of the intervals is defined by a linear polynomial.

[0024] In the last interval, the CDF approaches the asymptotic value of 1. Thus, the linear approximation reduces the number of parameters without significantly reducing the accuracy of the approximation.

[0025] For example, the function approximates a predefined CDF having a predefined mean and a predefined standard deviation, the function is point-symmetric about the predefined mean, and the function is defined in an interval that includes values ​​higher than the predefined mean.

[0026] By exploiting the symmetry of the CDF, the number of intervals and therefore the number of parameters provided to specify the approximation function is reduced, which lowers the parameter storage requirements.

[0027] In an exemplary embodiment, a look-up table, LUT, provides the parameters for each interval.

[0028] The lookup table provides an efficient implementation for a set of parameters.

[0029] For example, the obtained CDF represents the CDF of a Gaussian distribution.

[0030] The method is easily applied to the Gaussian CDF commonly used in entropy coding.

[0031] In an exemplary embodiment, the parameters are in a fixed-point representation with a predefined precision.

[0032] Such a fixed-point representation together with a given precision makes it possible to provide the same precision and the same fixed-point representation resulting in the same values ​​on different types of platforms (hardware and / or software). The same value of the CDF results in the same mapping of symbols and ranges in the encoder and decoder, which in turn results in compatibility and ensures synchronized entropy encoding and decoding (i.e., deterministic entropy encoding across different software and hardware platforms).

[0033] For example, in the step of obtaining the CDF, intermediate results are adapted to the predetermined precision according to a set of rules.

[0034] Adapting the intermediate results to a predefined precision further ensures compatibility between different platforms as well as synchronized entropy encoding and decoding.

[0035] In an exemplary implementation, for a value x=s-μ-0.5 in the at least one interval, where s represents the symbol being encoded and μ is the mean of the CDF; The CDF is, for x less than 0,

number

number

number

number

number

[0036] Such a combination of shifts and scalings is the sum of the parameters a in the interval i. i , b i , c i We give an efficient implementation for obtaining the CDF from a subset of .

[0037] According to one embodiment, there is provided a method for entropy decoding data from a bitstream, comprising the steps of obtaining a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals, said CDF being obtained based on a function defined in said at least one interval by parameters of a quadratic polynomial, said quadratic polynomial satisfying a set of conditions, and applying a probability model based on the CDF obtained in said at least one interval to entropy decode said data.

[0038] Obtaining the CDF based on a set of parameters describing a quadratic polynomial provides a reliable and reproducible method for obtaining estimates of the probability model used in entropy coding. This function represents an approximation of the CDF using a quadratic polynomial, while conditions allow a better adaptation of the approximation to some intrinsic properties of the CDF.

[0039] In an exemplary implementation, the set of conditions includes one or more of: the quadratic polynomial in the at least one interval and the polynomials in adjacent intervals provide monotonically non-decreasing functions; the quadratic polynomial has a non-negative derivative within the at least one interval.

[0040] These conditions express certain properties of the approximation given by the function: monotonicity, curvature, etc. They reflect the characteristics of the CDF and therefore result in an approximation suitable for use as a CDF in entropy coding.

[0041] For example, the function defined by the parameters specifies a predefined CDF having a predefined mean and a predefined standard deviation.

[0042] Providing default CDFs has the advantage that such default CDFs can be efficiently stored or derived when needed, while possible further CDFs can be obtained based on the default CDFs.

[0043] In an exemplary implementation, the CDF is obtained by shifting at least one argument of the function.

[0044] Shifting the arguments makes it possible to use the approximation provided by the function, for example, to obtain additional CDFs with means different from the default mean used in obtaining the function. Only the parameters of the predefined CDFs need to be stored or derived.

[0045] For example, the CDF is obtained by scaling at least one argument of the function.

[0046] Scaling the arguments makes it possible, for example, to use the approximation provided by the function to obtain additional CDFs with standard deviations different from the default standard deviation used in obtaining the function. Only the parameters of the predefined CDFs need to be stored or derived.

[0047] In an exemplary implementation, the entropy decoding is arithmetic decoding.

[0048] Arithmetic decoding is an efficient entropy coding that can contribute to rate reduction. It can advantageously use the CDF approximation obtained above.

[0049] For example, the function in one or more of the intervals is defined by a linear polynomial.

[0050] Using a linear approximation in regions with approximately constant slope may, for example, reduce the number of parameters provided to specify the approximation function.

[0051] In an exemplary implementation, the function in a last interval of the plurality of intervals is defined by a linear polynomial.

[0052] In the last interval, the CDF approaches the asymptotic value of 1. Thus, the linear approximation reduces the number of parameters without significantly reducing the accuracy of the approximation.

[0053] For example, the function approximates a predefined CDF having a predefined mean and a predefined standard deviation, the function is point-symmetric about the predefined mean, and the function is defined in an interval that includes values ​​higher than the predefined mean.

[0054] By exploiting the symmetry of the CDF, the number of intervals and therefore the number of parameters provided to specify the approximation function is reduced, which lowers the parameter storage requirements.

[0055] In an exemplary embodiment, a look-up table, LUT, provides the parameters for each interval.

[0056] The lookup table provides an efficient implementation for a set of parameters.

[0057] For example, the obtained CDF represents the CDF of a Gaussian distribution.

[0058] The method is easily applied to the Gaussian CDF, which is commonly used in entropy decoding.

[0059] In an exemplary embodiment, the parameters are in a fixed-point representation with a predefined precision.

[0060] Such a fixed-point representation together with a given precision makes it possible to provide the same precision and the same fixed-point representation resulting in the same values ​​on different types of platforms (hardware and / or software). The same value of the CDF results in the same mapping of symbols and ranges in the encoder and decoder, which results in compatibility and ensures synchronized entropy encoding and decoding (i.e., deterministic entropy encoding across different software and hardware platforms).

[0061] For example, in the step of obtaining the CDF, intermediate results are adapted to the predetermined precision according to a set of rules.

[0062] Adapting the intermediate results to a predefined precision further ensures compatibility between different platforms as well as synchronized entropy encoding and decoding.

[0063] In an exemplary implementation, for a value x=s-μ-0.5 in the at least one interval, where s represents the symbol being encoded and μ is the mean of the CDF; The CDF is, for x less than 0,

number

number

number

number

number

[0064] Such a combination of shifts and scalings is the sum of the parameters a in the interval i. i , b i , c i We give an efficient implementation for obtaining the CDF from a subset of .

[0065] According to one embodiment, a method is provided for obtaining a set of parameters specifying a cumulative distribution function, CDF, for entropy coding, the method comprising the steps of obtaining, for an interval in a plurality of predefined intervals, a subset of the set of parameters that defines a quadratic polynomial within the interval, the quadratic polynomial approximating the CDF, the polynomial satisfying a set of conditions.

[0066] Obtaining a set of parameters that describe a quadratic polynomial allows for a reliable and reproducible method for obtaining an approximation of the CDF, while the conditions allow for a better fit of the approximation to some intrinsic properties of the CDF.

[0067] In an exemplary implementation, the set of conditions includes one or more of: the quadratic polynomial and the polynomials in adjacent intervals provide monotonically non-decreasing functions; the quadratic polynomials have non-negative derivatives within each of the intervals.

[0068] These conditions express certain properties of the CDF approximated by the function, i.e. monotonicity, curvature, etc. They reflect the characteristics of the CDF and therefore result in an approximation suitable for use as a CDF in entropy coding.

[0069] For example, the function defined by the parameters specifies a predefined CDF based on a predefined mean and a predefined standard deviation.

[0070] Using a default CDF has the advantage that such a default CDF can be efficiently stored or derived when needed, while possible further CDFs can be obtained based on the default CDF.

[0071] In an exemplary implementation, the function in one or more of the intervals is defined by a linear polynomial.

[0072] Using a linear approximation in regions with approximately constant slope may, for example, reduce the number of parameters provided to specify the approximation function.

[0073] For example, the function in a last interval of the plurality of intervals is defined by a linear polynomial.

[0074] In the last interval, the CDF approaches the asymptotic value of 1. Thus, the linear approximation reduces the number of parameters without significantly reducing the accuracy of the approximation.

[0075] In an exemplary embodiment, the function approximates a predefined CDF having a predefined mean and a predefined standard deviation, the function is point-symmetric about the predefined mean, and the function is defined in an interval that includes values ​​higher than the predefined mean.

[0076] By exploiting the symmetry of the CDF, the number of intervals and therefore the number of parameters provided to specify the approximation function is reduced, which lowers the parameter storage requirements.

[0077] For example, the parameters for each interval are stored in a look-up table, LUT.

[0078] The lookup table provides an efficient implementation for a set of parameters.

[0079] In an exemplary implementation, the CDF represents the CDF of a Gaussian distribution.

[0080] The method is easily applied to the Gaussian CDF, which is commonly used in entropy coding.

[0081] For example, the parameters are in a fixed-point representation with a predefined precision.

[0082] Such a fixed-point representation together with a given precision makes it possible to provide the same precision and the same fixed-point representation resulting in the same values ​​on different types of platforms (hardware and / or software). The same value of the CDF results in the same mapping of symbols and ranges in the encoder and the decoder. This in turn results in compatibility and ensures synchronized entropy encoding and decoding (i.e., deterministic entropy encoding across different software and hardware platforms).

[0083] In an exemplary implementation, in the obtaining step, a cost function is optimized, the cost function including the difference between the second order polynomial and the CDF.

[0084] Optimizing the cost function provides an efficient implementation for obtaining a subset of parameters within the interval.

[0085] For example, the cost function includes a penalty that emphasizes the difference between the second order polynomial and the CDF at the boundaries of the interval.

[0086] Such a cost function increases the influence of regions close to the interval boundaries, which allows for efficient processing of conditions that must be satisfied at the interval boundaries.

[0087] In an exemplary embodiment, the cost function includes a penalty that is added when the apex of the parabola defined by the second degree polynomial lies within the interval or to the left of the interval.

[0088] Such a penalty ensures that the function is monotonically increasing in the interval and therefore respects the properties of the CDF being approximated.

[0089] In an exemplary implementation, a computer program stored on a non-transitory medium and including code instructions, when executed on one or more processors, causes the one or more processors to perform method steps according to any of the methods described above.

[0090] According to one embodiment, there is provided an apparatus for entropy coding data into a bitstream, comprising a processing circuit configured to obtain a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals, the CDF being obtained based on a function defined in the at least one interval by parameters of a quadratic polynomial, the quadratic polynomial satisfying a set of conditions, and to apply a probability model to entropy code the data based on the CDF obtained in the at least one interval.

[0091] According to one embodiment, there is provided an apparatus for entropy decoding data from a bitstream, comprising a processing circuit configured to obtain a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals, the CDF being obtained based on a function defined in the at least one interval by parameters of a quadratic polynomial, the quadratic polynomial satisfying a set of conditions, and to apply a probability model to entropy decode the data based on the CDF obtained in the at least one interval.

[0092] According to one embodiment, an apparatus is provided for obtaining a set of parameters specifying a cumulative distribution function, CDF, for entropy coding, the apparatus including a processing circuit configured to obtain, for an interval within a plurality of predefined intervals, a subset of a set of parameters defining a quadratic polynomial within the interval, the quadratic polynomial approximating the CDF, the polynomial satisfying a set condition.

[0093] According to one embodiment, there is provided a computer readable storage medium for storing a bitstream generated by an entropy encoding method according to any one of the embodiments and / or implementations described herein or an apparatus for entropy encoding according to any one of the embodiments and / or implementations described herein. The computer readable storage medium may be a computer readable recording medium or may be referred to as a computer readable storage medium, and may be, for example, a non-transitory computer readable recording medium, such as a non-transitory computer readable storage medium.

[0094] The present apparatus provides the advantages of the methods described above.

[0095] The present invention can be implemented in hardware (HW) and / or software (SW) or any combination thereof. Furthermore, HW-based implementations may be combined with SW-based implementations.

[0096] The embodiments and implementations described herein make it possible, for example, to provide a low-complexity fixed-point approximation of the ordinary cumulative distribution function.

[0097] The embodiments and implementations described herein may be used to code still / video images and audio signals (e.g., multi-channel audio signals), including but not limited to, entropy encoding and decoding data related to still / video images and audio signals, respectively. For example, the embodiments and implementations described herein may also be used (in the same or corresponding manner) to obtain an activation function of or for a neural network (instead of obtaining a CDF for entropy encoding and / or decoding), where the activation function is obtained based on a function defined in at least one interval by parameters of a quadratic polynomial, and the quadratic polynomial satisfies a set of conditions. The neural network may be used, for example, to code or compress / decompress still / video images and audio signals (e.g., multi-channel audio signals), or to generally process still / video images or audio signals (e.g., for classification, denoising, or other purposes), or the neural network may be used for data encryption or other purposes.

[0098] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0099] In the following, embodiments of the invention are described in more detail with reference to the accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram illustrating the encoding of a message using an alphabet with two symbols using arithmetic coding. [Diagram 2] FIG. 2 is a schematic diagram of the first step of arithmetic (range) coding of a message, using an alphabet with eight symbols. [Diagram 3] FIG. 3 is a schematic diagram of the second step of the arithmetic (range) coding of the message, using an alphabet with 8 symbols. [Figure 4] FIG. 4 is a schematic diagram of the third step of the arithmetic (range) coding of a message, using an alphabet with eight symbols. [Diagram 5] FIG. 5 is a schematic diagram of the fourth step of the arithmetic (range) coding of a message using an alphabet with eight symbols. [Figure 6] FIG. 6 is a schematic diagram of the final step of the arithmetic (range) coding of a message using an alphabet with eight symbols. [Figure 7A] FIG. 7a is a schematic diagram showing the minimum and maximum values ​​of a range during a coding step, expressed as binary numbers. [Figure 7B] FIG. 7b is a schematic diagram showing the minimum and maximum values ​​of the range during the coding step, expressed as binary numbers. [Figure 7C] FIG. 7c is a schematic diagram showing the minimum and maximum values ​​of the range during the coding step, expressed as binary numbers. [Figure 7D] FIG. 7d is a schematic diagram showing the minimum and maximum values ​​of the range during the coding step, expressed as binary numbers. [Figure 8] FIG. 8 is a schematic diagram illustrating an exemplary arithmetic decoding. [Figure 9] FIG. 9 shows exemplary CDFs of normal distributions with different means and standard deviations. [Figure 10] FIG. 10 shows an exemplary approximation of the error function by a second order polynomial. [Figure 11] FIG. 11 is a schematic diagram of obtaining a second order polynomial in an interval from a lookup table to approximate a CDF. [Figure 12A] FIG. 12a is a schematic diagram illustrating the conditions satisfied by a second order polynomial approximating the CDF. [Figure 12B] FIG. 12b is a schematic diagram illustrating the conditions satisfied by a second order polynomial approximating the CDF. [Figure 13] FIG. 13 exemplarily shows a floating-point format and a fixed-point format. [Figure 14A] FIG. 14a is a flow chart of one exemplary implementation for obtaining a set of parameters that define a second order polynomial. [Figure 14B] FIG. 14b is a flow chart of one exemplary implementation for obtaining a set of parameters that define a second order polynomial. [Figure 15] FIG. 15 is a schematic diagram of an entropy encoder and decoder using the CDF. [Figure 16] FIG. 16 is a schematic diagram showing channels processed by layers of a neural network. [Figure 17] FIG. 17 is a schematic diagram illustrating an autoencoder type of neural network. [Figure 18A] FIG. 18a is a schematic diagram showing one exemplary network architecture for the encoder and decoder side including the hyperprior model. [Figure 18B] FIG. 18b is a schematic diagram showing a general network architecture for the encoder side including the hyperprior model. [Figure 18C] FIG. 18c is a schematic diagram showing a general network architecture for the decoder side including the hyperprior model. [Figure 19] FIG. 19 is a schematic diagram showing one exemplary network architecture for the encoder and decoder side including the hyperprior model. [Figure 20] FIG. 20 is a block diagram illustrating the use of arithmetic encoding of some embodiments in picture encoding. [Figure 21]FIG. 21 is a block diagram illustrating the use of arithmetic encoding of some embodiments in picture decoding. [Figure 22] FIG. 22 is a block diagram illustrating an example video coding system configured to implement embodiments of the present disclosure. [Diagram 23] FIG. 23 is a block diagram illustrating another example decoding system configured to implement embodiments of the present disclosure. [Figure 24] FIG. 24 is a block diagram illustrating an exemplary encoding or decoding device. [Diagram 25] FIG. 25 is a block diagram illustrating another exemplary encoding or decoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0100] In the following description, reference is made to the accompanying drawings which form a part of this disclosure and which show, by way of illustration, certain aspects of embodiments of the invention or in which embodiments of the invention may be used. It is understood that embodiments of the invention may be used in other ways and may include structural or logical changes not shown in the drawings. The following detailed description is therefore not to be taken in a limiting sense and the scope of the present invention is defined by the appended claims.

[0101] For example, it is understood that disclosure related to a described method also applies to a corresponding device or system configured to perform the method, and vice versa. For example, when one or more particular method steps are described, a corresponding apparatus can include one or more units for performing the described one or more method steps (e.g., one unit performing one or more steps, or multiple units performing one or more of the steps, respectively), even if such one or more units are not explicitly described or shown in the drawings. On the other hand, for example, when a particular apparatus is described based on one or more units, e.g., functional units, a corresponding method can include one step of performing a function of the one or more units (e.g., one step of performing a function of the one or more units, or multiple steps performing one or more of the functions, respectively), even if such one or more steps are not explicitly described or shown in the drawings. Furthermore, it is understood that the features of the various exemplary embodiments and / or aspects described herein can be combined with each other, unless otherwise specified.

[0102] Video coding generally refers to the processing of a sequence of pictures to form a video or a video sequence. Instead of the term picture, the terms frame or image may be used as synonyms in the field of video coding. Video coding comprises two parts: video encoding and video decoding. Video encoding is performed at the source side and typically involves processing the original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed at the destination side and typically involves the reverse processing compared to the encoder to reconstruct the video picture. An embodiment referring to "coding" of a video picture (or pictures in general, as will be explained later) shall be understood to relate to both "encoding" and "decoding" of a video picture. The encoding and decoding parts together are also referred to as Coding and DECoding.

[0103] In the case of lossless video coding, the original video picture can be reconstructed, i.e. the reconstructed video picture has the same quality as the original video picture (assuming there are no transmission errors or other data losses during storage or transmission). In the case of lossy video coding, further compression, e.g. by quantization, is performed to reduce the amount of data representing the video picture that cannot be fully reconstructed at the decoder. In other words, the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.

[0104] The following provides an overview of some of the technical terminology and framework used in the embodiments in which the present disclosure may be employed.

[0105] Arithmetic encoding Entropy coding is typically used as a lossless coding. Arithmetic coding is a class of entropy coding that encodes a message as a binary real number in an interval (range) that represents the message. Here, the term message refers to a sequence of symbols. The symbols are selected from a predefined alphabet of symbols. For example, the alphabet may consist of two values ​​0 and 1. A message using such an alphabet is then a sequence of bits. The symbols (0 and 1) may occur in a message with different frequencies from each other. In other words, the symbol probabilities may be non-uniform. In fact, the more non-uniform the distribution, the higher the compression achievable by entropy codes in general, and arithmetic codes in particular. Arithmetic coding utilizes an a priori known probability model that specifies a symbol probability for each symbol of the alphabet.

[0106] The alphabet need not be binary. Rather, the alphabet may consist of, for example, eight values ​​from 0 to 7. In general, any alphabet having any size may be used. Typically, the alphabet is given by the range of values ​​of the coded data.

[0107] The interval representing the message is obtained by dividing the initial range according to the probabilities of the alphabet symbols with which the message is coded.

[0108] For example, let the current interval be initially the initial interval [0,1]. For each symbol of the message, the following two steps are performed: 1) Subdivide the current interval into subintervals, one for each possible alphabet symbol. The size of a symbol subinterval is proportional to the estimated probability that the symbol will be the next symbol in the message, according to a probability model (of the symbol source). 2) Select the subinterval that corresponds to the next actually occurring symbol in the message, and make the selected subinterval the new current interval.

[0109] As a third step, enough bits are output to distinguish the current interval from all other possible intervals. This step can be performed already during the encoding in steps 1 and 2, or it can be performed after encoding the entire message. The length of the interval obtained after repeating steps 1) and 2) for all symbols of the message is obviously equal to the product of the probabilities of the individual symbols, which is also the probability of a particular sequence of symbols in the message.

[0110] Theoretically, an arithmetic coder would recursively divide the interval from 0 to 1 to encode messages of arbitrary length, resulting in smaller and smaller intervals. In practice, the system is limited by a finite bit depth - only discrete values ​​are representable. Thus, the smaller the interval, the more precise arithmetic would be required. Furthermore, no output would be produced until the entire message has been read. A solution to both of these problems could be to output some bits as soon as they are known, and then double the length of the current interval for each output bit, so that it reflects only the (yet) unknown part of the interval. In practice, arithmetic can be done by storing the current interval in a sufficiently long integer, rather than a floating point or exact rational number.

[0111] A variant of the arithmetic coder that has been improved for practical use is called a range coder, which does not use the interval [0,1], but a finite range of integers, for example from 0 to 255. This range is divided according to the probabilities of the alphabetic symbols. In order to describe all alphabetic symbols according to their probabilities, this range can be renormalized if the remaining range becomes too small.

[0112] In terms of the terminology used herein, the current interval is given by its minimum value (denoted as LOW) and its maximum value (denoted as HIGH). The length of the interval is denoted as RANGE. In general, HIGH=LOW+RANGE, and RANGE is expressed as the number of (distinguishable) sub-intervals of the smallest size. The minimum range for encoding the symbols is the BOTTOM. This minimum range ensures that the least likely symbols still have at least one valid range. In other words, the BOTTOM is a design parameter that can be determined based on the alphabet symbols and their probabilities, and ensures that the range can be further divided into distinguishable intervals corresponding to all the alphabet symbols.

[0113] The HIGH position indicates how many bits are needed to cover the initial maximum range. The BOTTOM position indicates how many bits are needed to cover the minimum range for encoding another symbol. The amount of bits between the HIGH position and the default TOP position corresponds to the smallest fraction of bits that will be streamed (inserted) into the bitstream and become the coded bits.

[0114] Figure 1 shows diagrammatically one exemplary procedure of arithmetic encoding. The message to be coded is provided in an alphabet with two possible symbols A and B. The symbols of the alphabet {A,B} have probabilities P(A)=2 / 3 and P(B)=1 / 3. The message to be encoded is (read) AABA.

[0115] Step 0 is the length Range corresponding to HIGH here. 0 This shows the initial interval with 1010 because initially, LOW=0 (low0=0 at step 0). Range 0 In the lower half of the Range 0 The leading bits of the binary representation of the numbers in are 11011, while the leading bits of the numbers in 0 In the upper half of the initial range, the leading bits are 01012. In other words, the current interval of any step that is in the upper half of the initial range will have a 0 as its first leading bit, while the current interval that is in the lower half of the initial range will have a 1 as its first leading bit. Now, for example, assume that any number (code value) in the initial range can be represented in 8 bits, so the range is 0 to 255, so HIGH=255.

[0116] Step 1: Range 0 is divided according to probability, and the first symbol A 0 In this example, the initial interval is 1 / 3 of the total range size, 1022 (symbol B 0 ), and size 2 / 3 1021 (corresponding to symbol A 0 Dividing according to probability means that the initial interval is divided into a number of subintervals equal to the number of symbols in the alphabet (here two symbols A and B), and the sizes of the intervals are proportional to the respective probabilities of the symbols represented by the intervals. 0 The upper sub-interval Range 1 =P(A 0 )*Range0 is then selected as the current interval for the next step.

[0117] In step 2, symbol A 0 Range 1 1020 is divided according to probability. The next message symbol is A. Message A 0 A 1 Describes 1031, remaining range 2 =P(A 1 )*Range 1 is encoded with bit 0. 0 Since the current maximum range is completely within the upper half of step 0, the addition of bit 0 to the encoded bitstream is performed. In this particular, exemplary embodiment, bits are added to the bitstream as soon as possible and then renormalization 1040 is performed to double the resolution. The current maximum range is now the upper half of the initial range of step 0. The upper half of this current maximum range is assigned to bit 0, and the lower half of the current maximum range is assigned to bit 1.

[0118] Message A in Step 3 0 A 1 B 2 1041 is Range 3 still cannot be unambiguously encoded because it overlaps with bit 1 (the corresponding lower half of the current maximum range) as well as bit 0 (the corresponding upper half of the current maximum range), so a carry of 1050 is performed. Nothing is streamed (no bits are included in the bitstream). Range 3 is divided in step 4 according to the probabilities of the symbols in the alphabet.

[0119] Now the message to be coded is A 0 A 1 B 2 A 3 The result is 1051. Range 4still overlaps with both possible bits, and there are no further symbols to be encoded, so finalization 1060 is performed in steps 5 and 6. This involves several renormalizations being performed to create the unambiguous code 0011 for the message AABA.

[0120] 2 through 7 show an exemplary range encoder in which the eight symbols {0,1,2,3,4,5,6,7} of the alphabet 2020 have probabilities according to a probability density function PDF 2030 for a standard normal distribution. The symbols are mapped onto the range by a cumulative distribution function CDF 2040. The encoded message is 4420. The maximum starting range HIGH for encoding is represented in this example by 8 bits corresponding to a code value of 255. The minimum range for coding the symbols is BOTTOM=16 in this example. This minimum range ensures that even the least likely symbols "0" and "7" still have a valid range of at least 1 when the cumulative distribution function is applied. For example, a normal distribution results in the following number of minimum intervals per 8 symbols: 1,1,2,3,4,2,2,1.

[0121] Figure 2 shows the initial range 2010 0 =HIGH=255=1111 1111.b, where ".b" is the binary representation of a number, is mapped onto the symbols of the alphabet 2020 according to the Gaussian probability density function PDF 2030. Range 0 The partitioning of is obtained from the cumulative distribution function CDF 2040. This means that the first symbol of the message to be encoded, i.e., “4”, falls within the lower endpoint 2050. 1 =71=0100 0111.b and the excluded upper endpoint 2060 Low 1 +Range1 =199=1100 0111.b, which means that the range 1 = 128. total=const=16 2070 means a working precision of 1 / 16. This means that the subrange assigned to the least likely symbol is 1 / total = 1 / 16 of the current range. The total 2070 is determined by the cumulative distribution function.

[0122] The binary representation of this interval is shown in Fig. 7a with indication of the HIGH, TOP and BOTTOM positions. The HIGH position 740 indicates the number of bits required to cover the initial maximum range (8 in this case). The BOTTOM position 760 indicates the number of bits required to cover the minimum range for coding another symbol (4 in this case). The amount of bits between the HIGH position 740 and the predefined TOP position 750 corresponds to the smallest part of the bits that are streamed into the bit stream and then become the coded bits. For the arithmetic encoding described above with reference to Fig. 1, there is only one bit between the HIGH position 740 and the TOP position 750. For the range coding of this example, there are two bits between the HIGH position 740 and the TOP position 750. In Fig. 7a (as well as Figs. 7b to 7d), the minimum value of the interval 720a to 720d and the maximum value of the interval 710a to 720d are represented in binary numbers. This may in fact correspond to two registers used for the purpose of storing the current interval and, therefore, the current result of the encoding.

[0123] In FIG. 3, for mapping the symbols of the alphabet 2020 to ranges, the cumulative distribution function CDF 3040 is 1 Range starting from =71 1= 128. The next symbol in the message is "4". This results in a new current interval 350-360, which represents the message 44. This interval is applied to the new lower bound (current minimum) 720b, as shown in FIG. 7b. 2 =106=0110 1010.b, and the new upper bound (current maximum value) is 710b Low 2 +Range 2 =170=1010 1010.b. New Range 2 3020 is equal to 64.

[0124] Figure 4 and Figure 7c show the next steps in the encoding procedure. 2 Since 3020 is still greater than BOTTOM, the range is again divided according to the cumulative distribution function 4040. This means that for symbol 2, 3 =111=0110 1111.b low value 720c and Range 3 = 4 4010. Thus, message 442 3 From Low 3 +Range 3 =115=0111 0011.b is represented by the range up to 710c.

[0125] Since the two bits 730 between the HIGH and TOP positions are equal, they can be output as coded bits into the stream 731. See Figure 7d.

[0126] Figure 5 shows the Range 3 BOTTOM=4 4010 is smaller than BOTTOM=16, so a proper mapping of the symbols to the range is not possible and, therefore, a renormalization procedure is necessary. 4 This indicates that all bit representations are shifted left until the new Range is equal to or greater than BOTTOM. 4=(4<<2)=16 610. The new upper limit 710e is obtained from the current upper limit 710d after two left shifts. The same shifts are performed on the already 4 =188=1011 1100.b Low except for two coded bits that are part of 720d 3 This applies to.

[0127] Message 442 is now encoded with any value between 444=01 1011 1100.b and 460=01 1100 1100.b.

[0128] Figure 6 shows the encoding of the last symbol "0". The probability distribution is 5 =188=1011 1100.b 650, and Range 5 = 1620. Since there are no further symbols to be encoded, 5 and Range 5 describes the range interval of the trailing bits, which (together with the coded bits) represent the encoded message. In general, any value from the interval can be used (included in the bitstream) to represent the interval and, therefore, also the coded message (sequence of symbols). Thus, the trailing bits can be chosen arbitrarily from this final range interval. In this example, Range 5 =1 620 is a single value for the following bits, i.e., Low 5 =188=1011 1100.b 650. Thus, message 4420 is encoded by appending the trailing bit to the coded bit, resulting in coded value 444=01 1011 1100.b

[0129] Arithmetic decoding Figure 8 shows an example decoder decoding process. The decoder receives coded values ​​(bits) sequentially. The received coded values ​​820 are 0 =HIGH=255 810. The probability distribution function is known at the decoder side, and therefore the mapping of symbols 220 to ranges according to the cumulative distribution function 830 is known. The decoder does not know the inverse of this mapping, and therefore the decision of the symbol requires a searching process. The decoder makes a first guess 840 about the encoded symbol by selecting the most likely symbol "4", calculates the Low value 841 of the range corresponding to this symbol, and checks if the received coded value 820 is higher than this Low value 841.

[0130] Because the received coded value 820 is less than the low value 841 of the first guess 840, the next guess 850 is a symbol that maps to a lower value in the range, i.e., one of the symbols "0", "1", "2", or "3". Selecting a low value approximately in the middle of the remaining interval of symbols leads to a faster decoding process since fewer steps are required to obtain the correct symbol. In this example, the next low value 851 selected for comparison corresponds to the symbol "2".

[0131] A test for the symbol "2" results in the received coded value being higher than the low value of the range 851, which encodes the symbol "2". Thus, the received coded value can represent the symbols "2" or "3". A final check 860 reveals that the low value of the range 861, which corresponds to "3", is higher than coded value 820. Thus, the received coded value 820 is decoded as the symbol "2".

[0132] Obtaining a cumulative distribution function As shown in Figures 2 to 4 and 8 and the corresponding description, a cumulative distribution function is estimated for encoding and / or decoding symbols. For example, such a distribution function may be a normal (or Gaussian) distribution. However, the present invention is not limited to a normal distribution, and other probability distributions may be used, such as a Laplacian distribution, a cosine distribution, a geometric distribution, etc.

[0133] The probability density function (PDF) of a normal distribution with mean μ and standard deviation σ is:

number

[0134] The corresponding cumulative distribution function (CDF) F(x) has the form

number

number

[0135] An example CDF for a normal distribution is shown in Figure 9. The four CDFs are 2 ), in particular, the CDFs have variances of 0.2, 0.5, 1, and 5. Three of these example CDFs 900, 910, 920 have mean μ=0, and the remaining CDF 930 has mean μ=−2.

[0136] In this disclosure, the CDF is approximated interval-wise using polynomials within the intervals and / or with constraints on the boundaries between the intervals. In this way, a CDF approximation is obtained, which can be used in a variety of applications, including arithmetic encoding and decoding of arbitrary source data. The constraints can correspond to properties of the approximated CDF, such as monotonicity or continuity.

[0137] For example, a nonlinear function such as a CDF is approximated in intervals that define subranges of the domain of the definition of the nonlinear function. The approximation may be obtained from a function that is parameterized and whose parameters are defined piece-wise in said intervals. This is exemplarily shown in FIG. 10, where an error function that can be used to represent the CDF of a Gaussian distribution is approximated piece-wise in four intervals with x>0 by a polynomial of degree 1 or higher. The parameters of the polynomials can then be stored and used later to obtain (define) an approximated CDF for a particular application.

[0138] FIG. 11 exemplarily illustrates obtaining a CDF from a set of parameters for application of entropy coding.

[0139] The CDF 1150 is obtained for at least one interval 1160. The at least one interval 1160, included in the plurality of intervals 1160 to 1162, corresponds to a subrange of the value range of the CDF (x range in FIG. 9). The intervals in the plurality of intervals may be of the same size or of different sizes. It may be advantageous to divide the value range into a predefined number of intervals, such as 20, 40, 50, or 100, or any other predefined number. The value range of the CDF may be infinite on one or both sides of the value range of some CDFs. In such a case, the first and / or the last interval may approximate an asymptotic part of the value range. In one exemplary embodiment, all intervals apart from the asymptotic interval are of the same length. Furthermore, some CDFs, such as Gaussian CDFs, are symmetric. Therefore, it may be sufficient to approximate half of the CDF (e.g., starting from the mean and increasing) and obtain the remaining part by mirroring vertically and horizontally. For example, the last interval may have a different size than the remaining intervals, containing values ​​up to a predefined large number corresponding to the maximum number of symbols in the encoding.

[0140] The CDF 1150 is obtained based on a function 1130 defined in at least one interval 1160. The function 1130 is defined in at least one interval 1160 by parameters 1120 of a quadratic polynomial. For example, k x 2 +b k x+c k is a function that takes three parameters a k , b k , and c k A quadratic polynomial satisfies a set of conditions: k , b k , and c k Any parameter in a can be zero. k Zero or parameter a k and b k If is zero, then the approximating polynomial has a lower degree, and therefore the approximation is not necessarily a second-order approximation.

[0141] As mentioned above, the obtained CDF can represent the CDF of a Gaussian distribution. However, as already indicated, the present disclosure is not limited to Gaussian distributions. There may be additional CDFs whose forms may be difficult to generate by a single functional definition. Laplace, Gibbs, Maxwell-Boltzmann, Beta, Arcsine, Bates, Logits-normal, Irwin-Hall, Kumaraswamy, logit metalog, Marchenko-Pastur, raised cosine, Wigner semicircle, continuous Bernoulli, wrapped Cauchy, wrapped asymmetric Laplace, noncentral chi-squared, Dagum, Cauchy, exponentially modified Gaussian These include Gaussian, Gumbel, Holtsmark, hyperbolic, Landau, logistic, skew normal, Student's-T, Tracy-Widom, Voigt, and the CDF of other distributions.

[0142] Examples for polynomials that meet or violate the conditions are shown in Figures 12a and 12b. Figure 12a shows a CDF 1200 that is approximated by polynomials 1210, 1211, and 1212 in three intervals A, B, and C. In this example, the polynomials are second order polynomials with parameters a, b, and c that differ from one another.

[0143] For example, the set of conditions includes that the quadratic polynomial 1211 and the polynomial 1212 in adjacent (neighboring) intervals provide a monotonically non-decreasing function defining the CDF. The adjacent intervals of the current interval 1160 are the preceding interval 1161 (before) and the successive interval 1162 (after).

[0144] For example, the set of conditions includes that the quadratic polynomial 1211 has a non-negative derivative within the interval. This ensures the monotonic and non-decreasing properties of the approximation. As can be seen in Fig. 12a, discontinuities may still exist at the interval boundaries. However, these discontinuities are subject to the set of conditions.

[0145] The polynomials in Fig. 12a have non-negative derivatives within their respective intervals, and the functions defined by these three example polynomials are monotonically non-decreasing. In contrast, the two example polynomials 1220 and 1221 in intervals D and C in Fig. 12b, which approximate the CDF 1201, do not form a monotonically non-decreasing function in region 1230, and the apex of the quadratic polynomial 1221 is within the interval. Thus, the polynomials have negative derivatives 1240 within the interval. Non-negative derivatives and / or decreasing functions may result in overlapping ranges for different symbols in the entropy coding, and thus non-unique mappings of symbols and ranges.

[0146] A probability model based on the CDF obtained in at least one interval is applied in entropy coding the data into a bitstream. In other words, an estimate for the probability model is provided by the obtained CDF. The entropy coding may be an arithmetic encoding as described above in the arithmetic encoding section with reference to Figs. 2 to 7. In particular, the obtained CDF is applied to define a mapping of symbols 2020 to ranges 2010 in the arithmetic encoder and decoder. The interval of the approximation function may be different from the interval used for mapping symbols 2020 to ranges 2010 in the arithmetic encoding as shown in Fig. 2. The interval for which the parameters of the approximation function are defined may be smaller or larger than the interval used for mapping in the encoding depending on the accuracy of the obtained CDF.

[0147] The function 1130, defined by parameters 1120, can specify a default CDF with a default mean and a default standard deviation. For a normal distribution, the default CDF has a default mean μ 0 = 0 and the default standard deviation σ 0 = 1. However, this disclosure is not limited to these example values.

[0148] This function can provide the CDF without applying further operations. However, in some implementations, obtaining the CDF can further include an operation of scaling and / or shifting 1140 at least one argument of the function. The CDF can be obtained by shifting at least one argument of the function that approximates a predefined CDF. The argument of the function is, for example, a random variable of the approximated CDF. Such a shift can be performed for one or more of the intervals.

[0149] For example, the function here is the default CDF of a Gaussian distribution with zero mean and variance 1. The shift is taken to be the target mean μT The shifted argument (x value) of the function can be calculated as x=s-μ T −0.5, where the symbol s to be encoded has a target mean μ T 2, an additional shift of 0.5 is applied to obtain a lower bound on the interval in the mapping of symbols 2020 to CDF 2040.

[0150] A scaling may be applied to at least one argument of the function to obtain the CDF in at least one interval, which may be performed in addition to or instead of a shift in one or more of the intervals.

[0151] For example, the scaling is done by using a target standard deviation σ T and the default standard deviation σ 0 Scaling can be applied to align the scaled variables, e.g.,

number

[0152] The function may be defined in one or more of the intervals by a linear polynomial. There may be regions where the linear approximation does not significantly reduce the accuracy of the CDF. For example, this may be the case in intervals close to the mean or in intervals where the CDF is approximately 0 or 1. In these exemplary intervals, the slope of the function only shows a small influence of the value x in the value range. For example, the function in the last interval of the intervals is defined by a linear polynomial. In said last interval, the CDF to be approximated may approach its limit value 1. Thus, the number of parameters specifying the polynomial of the last interval may be reduced.

[0153] If the function approximates a predefined CDF as described above, the function may be point-symmetric with respect to a predefined mean. In other words, the function f(x) may be point-symmetric with respect to the point (μ,f(μ)). This is the case of the Gaussian CDF, exemplarily shown in FIG. 9. The function may be defined in an interval that includes values ​​higher than the predefined mean. For example, a function specifying a reference normal CDF with mean μ=0 and standard deviation σ=1 is defined for x>0. Based on this function, the point symmetry may be exploited to obtain a CDF for x>0. This reduces the number of intervals in which the function approximates the predefined CDF, and therefore the number of parameters to be used and stored may be reduced.

[0154] A look-up table (LUT) may provide the parameters for each of the intervals. However, this disclosure is not limited to LUTs providing the parameters. The parameters may be stored in any format. FIG. 11 provides, by way of example, such a LUT 1110 that includes a subset of parameters 1120 for each interval k. In the exemplary case, when the last interval is specified by a linear polynomial, the parameters a N =0 may not be included in the set of parameters. This reduces the number of parameters to be stored.

[0155] The parameters may be provided in a fixed-point representation having a predefined precision. A diagram of the fixed-point format is shown in FIG. 13. A number x may be written in fixed-point representation as: A number x may be written in fixed-point representation as:

number

[0156] A parameter in fixed-point format may be represented by an integer and a predefined precision, for example 3.14=314 / 100 may be represented by the integer 314 and a precision factor of 1 / 100.

[0157] In an exemplary implementation, a 32-bit integer may be used for each parameter, in which case 40 x 3 x 32 bits are needed to define the above function when the function is defined in 40 intervals by a quadratic polynomial (i.e., 3 parameters per polynomial).

[0158] Such fixed-point representation provides the same precision and the same fixed-point representation that, together with a predefined precision, results in the same values ​​on different types of platforms. To ensure the same results obtained by processing parameters on different platforms, the precision of intermediate results can be adapted to the predefined precision. This can include a set of predefined rules for reducing the precision of intermediate results. For example, the rules can include rules for rounding and / or clipping numbers in fixed-point format. Such rules should be observed in any processor architecture where a CDF is obtained for entropy coding, which then provides compatibility and ensures synchronized entropy coding and decoding.

[0159] Such intervalwise approximation by a quadratic polynomial as a basis for the CDF used in entropy coding eliminates the need for (platform-dependent) implementations of special functions such as error functions. By using simple operations such as scaling and shifting, the same values ​​of the CDF can be obtained on different hardware and / or software platforms.

[0160] However, the present disclosure is not limited to parameters in fixed-point format. For example, the floating-point format shown in FIG. 13 may be used. A floating-point number x with base b is defined as follows.

Number

[0161] In one exemplary implementation, the CDF is obtained for the value x = s - μ - 0.5 within at least one interval. Here, s represents the symbol to be encoded, and μ is the average of the CDF to be obtained. The CDF is obtained by the following formula for x less than zero.

Number

Number

[0162] The multiplication factor M is related to the default precision 1 / M,

Number

[0163] The CDF is given by the following formula for x greater than zero:

number

number

[0164] In the corresponding decoding, the CDF is obtained similarly to the encoding in order to properly reconstruct the encoded data from the bitstream. Thus, for at least one interval 1160 of a number of intervals 1160 to 1162, a CDF 1150 is obtained. The intervals are defined similarly to the encoding. Thus, the intervals may be predetermined, for example defined by a standard or signaled to the decoder. In the at least one interval 1160, a function 1130 is defined by parameters 1120 of a quadratic polynomial. The CDF 1150 is obtained based on said function 1130. The obtaining may include scaling and / or shifting 1140 of arguments of said function.

[0165] The function 1130 may be to approximate a predefined CDF. The predefined CDF may have a predefined mean and a predefined standard deviation. For example, the predefined CDF may approximate a predefined mean μ 0 = 0 and the default standard deviation σ 0 = 1. The obtained CDF may represent the CDF of a Gaussian distribution, however, as already indicated, the present disclosure is not limited to Gaussian distributions.

[0166] Data is entropy decoded from the bitstream by applying a probability model based on the CDF 1150 obtained in at least one interval 1160. The entropy decoding may be arithmetic decoding as described above in the arithmetic decoding section with reference to Figure 8. The range of the arithmetic decoder is defined according to the CDF.

[0167] One exemplary scheme for entropy encoding and decoding is shown in FIG. 15. Data 1520 to be encoded into a bitstream 1550 may be obtained, for example, from an encoder portion 1510 of an autoencoding neural network, which is described in detail in the section "Autoencoders and Unsupervised Learning." The CDF 1530 used in the entropy encoding 1540 may be obtained as described above. In the decoding, the CDF 1560 is obtained as described above. For correct decoding 1570, the CDF 1560 applied in the decoder should be identical to the CDF 1530 applied in the encoder, which is achieved by the method described above. Data 1580 is decoded. This may be a latent space representation, which may be further processed, for example, by a decoding portion 1590 of the autoencoding neural network.

[0168] The encoding and decoding as described above may be applied, for example, in a variational autoencoder, a hybrid coder, or a hybrid AI coder. Some exemplary implementations of such types of encoders and decoders that can use the obtained CDF are described in the following sections.

[0169] The quadratic polynomials satisfy a set of conditions, including, for example, that the quadratic polynomials 1211 and 1212 in adjacent (neighboring) intervals provide a monotonically non-decreasing function that defines the CDF.

[0170] For example, the set of conditions includes that the quadratic polynomial 1211 has a non-negative derivative within the interval. This ensures the monotonic and non-decreasing properties of the approximation of the CDF, as seen in the example of Figure 12a. The discontinuity is subject to the set of conditions.

[0171] The function defined by the parameters can provide the CDF without applying further operations. In some implementations, obtaining the CDF can further include scaling and / or shifting 1140 at least one argument of the function. The CDF can be obtained by shifting at least one argument of the function that approximates a predefined CDF. Such shifting can be performed for one or more of the intervals.

[0172] For example, a function, a, in the interval k k x 2 +b k x+c k The x-values ​​are the target mean μ of the CDF to be obtained. T The shifted x value of the function can be calculated based on x=s-μ T −0.5, where the symbol s to be encoded has a target mean μ T2, an additional shift of 0.5 is applied to obtain a lower bound on the interval in the mapping of symbols 2020 to CDF 2040.

[0173] To obtain the CDF, a scaling may be applied to at least one argument of the function, which may be performed in addition to or as an alternative to a shift in one or more of the intervals.

[0174] For example, the scaling can be done by using the target standard deviation σ of the fitted CDF. T and the default standard deviation σ 0 is applied to align the scaled variables

number

[0175] 9 shows CDFs that may be obtained by scaling and / or shifting from a standard normal CDF 900. To obtain CDFs 910 and 920 from the standard normal CDF 900, a scaling of the argument x is performed to match the respective target standard deviations. To obtain CDF 930, a scaling and shift of the argument x is performed.

[0176] The function may be defined in one or more of the intervals by a linear polynomial. For example, in intervals close to the mean or where the CDF is approximately 0 or 1, a linear approximation may not significantly reduce the accuracy of the CDF. In these exemplary intervals, the slope of the function shows only a small effect of the value x in the value range. In other words, the polynomial ax 2The parameter a in +bx+c is small compared to the parameters b and / or c. The function in the last interval of the intervals may be defined by a linear polynomial. In said last interval, the CDF to be approximated can approach its limit value 1. Thus, the number of parameters can be reduced. The number of parameters in each interval can be limited to 3 in the general case and to 2 in some cases (e.g., only "a and b" or only "a and c").

[0177] As with the encoding, the number of intervals can be reduced in case of point-symmetric functions. A given CDF can be point-symmetric about its given mean, i.e., the point (μ,CDF(μ)). Correspondingly, a function f(x) approximating the given CDF is point-symmetric about the point (μ,f(μ)). The function can be defined in an interval that includes values ​​higher than the given mean above. For example, a function specifying a normal CDF with mean μ=0 is defined for x>0. For a given mean value μ 0 The function f(x) for values ​​of x smaller than f(μ 0 +x)=-f(μ 0 -x).

[0178] Similar to the encoding, the parameters may be provided to the LUT 1110 for each interval. The LUT may include three columns (one column for each of the three parameters a, b, and c), or two columns with parameters of a parabolic polynomial (some parameters of the quadratic polynomial may be skipped or omitted in certain cases, e.g., one column for each of two parameters, e.g., only "a and b" or only "a and c" as described above). The parameters may be provided in a fixed-point format with a predefined precision. The fixed-point format is described above with reference to FIG. 13.

[0179] Such fixed-point representation, together with a predefined precision, provides the same precision and the same fixed-point representation on different types of platforms, e.g., during encoding and decoding on different platforms, resulting in the same values ​​on different platforms. The precision of intermediate results may be adapted to the predefined precision. This may include a set of predefined rules for reducing the precision of intermediate results, which is the same for all platforms. For example, the rules may include rules for rounding and / or clipping numbers in fixed-point format. Such rules should be observed in any processor architecture where a CDF is obtained for entropy decoding, which then provides compatibility and ensures synchronized entropy encoding and decoding.

[0180] In one exemplary implementation, the CDF used in entropy decoding is obtained similarly to the encoding for the value x=s-μ-0.5 within at least one interval, where s represents the symbol to be encoded and μ is the mean of the CDF to be obtained. For x less than zero, the CDF is given by the following formula:

number

number

[0181] The multiplication factor M is related to the given precision 1 / M,

number

[0182] The CDF is given by the following formula for x greater than zero:

number

number

[0183] As mentioned above, the CDF used for entropy coding is specified by a set of parameters in a number of intervals, the number of intervals corresponding to sub-ranges of the value range of the CDF. For intervals in the number of intervals, a subset of parameters is obtained. The subset of parameters defines a quadratic polynomial in the interval. The polynomial approximates the CDF. This is exemplarily shown in FIG. 11, where a quadratic polynomial 1130 is defined by a subset of parameters 1120.

[0184] The CDF specified by the set of parameters may be applied to, but is not limited to, entropy coding of still images / video images, audio signals (e.g., multi-channel audio signals), entropy coding and decoding of data related to still images / video images, and audio signals, respectively.

[0185] In the approximation, a set of conditions are satisfied. As explained above, examples of the conditions are shown in Figures 12a and 12b. For example, the set of conditions includes that the quadratic polynomial 1211 and the polynomial 1212 in the adjacent interval provide a monotonically non-decreasing function that defines the CDF. For example, the set of conditions includes that the quadratic polynomial 1211 has a non-negative derivative in the interval.

[0186] The example polynomials of FIG. 12a have non-negative derivatives within their respective intervals, and the functions defined by these three example polynomials are monotonically non-decreasing.

[0187] The function 1130, defined by parameters 1120, can specify a default CDF with a default mean and a default standard deviation. For a normal distribution, the default CDF has a default mean μ 0 = 0 and the default standard deviation σ 0 = 1. However, this disclosure is not limited to these example values.

[0188] The CDF that is approximated may be a Gaussian CDF, however, the present disclosure is not limited to a Gaussian CDF and any other CDF may be approximated, including the examples listed above.

[0189] The function may be defined in one or more of the intervals by a linear polynomial. There may be regions where the linear approximation does not significantly reduce the accuracy of the CDF. For example, this may be the case for intervals close to the mean or intervals where the CDF is approximately 0 or 1. In these exemplary intervals, the slope of the function shows only a small influence of the value x in the value range. In an exemplary implementation, the function in the last interval of the intervals is defined by a linear polynomial. In said last interval, the approximated CDF may approach its limit value 1, i.e., the last interval includes all remaining values ​​of the value range higher than the right boundary of the preceding interval. Thus, the number of parameters specifying the polynomial of the last interval may be reduced.

[0190] If the function approximates a predefined CDF as described above, the function may be point-symmetric with respect to a predefined mean. In other words, the function f(x) may be point-symmetric with respect to the point (μ,f(μ)). This is the case for the Gaussian CDF, exemplarily shown in FIG. 9. The function may be defined in an interval that includes values ​​higher than the above-defined mean. Based on this function, the point symmetry may be exploited to obtain a CDF with mean μ=0 for x<0. This may reduce the number of intervals in which the function approximates the predefined CDF, and therefore the number of parameters to be used and stored may be reduced.

[0191] The parameters 1120 for each interval are stored in a look-up table (LUT) 1110. However, this disclosure is not limited to LUTs providing the parameters. The parameters may be stored in any format. Figure 11 provides, by way of example, such a LUT 1110, which includes a subset of parameters 1120 for each interval k.

[0192] The parameters may be provided in a fixed-point format with a predefined precision, which is described above with reference to FIG.

[0193] The derivation of one exemplary set of parameters is shown in FIG. 14a. The x-value range of the CDF to be fitted is determined by the step size step x In this exemplary implementation, each x i For the CDF value y to be approximated, i =CDF(x i ) is obtained S1420. In the following step S1430, the interval i is obtained. The parameters may be obtained by curve fitting, such as a least squares fit or any other fitting method.

[0194] Obtaining the parameters may include optimizing a cost function. The optimization may be performed once during the initial design of the LUT. If the LUT is provided by a standard, the optimization may be performed once for said standard. Such a cost function includes the difference between a second order polynomial and a CDF. One exemplary cost function is given in FIG. 14b. This exemplary cost function F_err(x,a,b,c) is the squared difference of the CDF to be approximated and the second order polynomial ax 2 S1460 depends on +bx+c.

[0195] Such a cost function may further include a penalty, which emphasizes the difference between the quadratic polynomial and the CDF at the boundaries of the interval. One exemplary implementation for such a penalty may be the bound corrector multiplier S1450 in FIG. 14b. This bound corrector is multiplied with the squared error in S1480 to correct the interval bound x i and x i+1 Highlight the squared error in

[0196] Thus, the boundary correction multiplier in one exemplary implementation may have the following form:

number

[0197] The cost function may further include a penalty, which is added if the apex of the parabola defined by the quadratic polynomial is within the above interval or to the left of the above interval.

[0198] One example implementation for such a penalty may be the roof corrector S1470 of Figure 14b. Such a roof corrector adds a high penalty to negative derivatives, which occur when the apex of the parabola is inside or to the left of a certain interval. The roof corrector may have the following form:

number

[0199] One exemplary cost function including the boundary correction multiplier and the roof correction is obtained in S1480 of FIG. 14b as F_err=bound_corrector*err 2 +roof_corrector After repeating steps S1410 to S1430 for each of the N intervals, a set of parameters is obtained in S1440.

[0200] In obtaining the parameters, the initial states of the parameters (a, b, c) may be used along with the values ​​for the linear approximation.

[0201] Artificial neural network Artificial neural networks (ANNs) or connectionist systems are computing systems loosely inspired by the biological neural networks that make up animal brains. Such systems “learn” to perform tasks by considering examples, typically without being programmed with task-specific rules. For example, in image recognition, they can learn to identify images containing cats by analyzing example images that have been manually labeled as “cat” or “no cat” and using the results to identify cats in other images. They do this without prior knowledge of cats, e.g., that they have fur, tails, whiskers, and cat-like faces. Instead, they automatically generate discriminative characteristics from the examples they process.

[0202] ANNs are based on a collection of connected units or nodes called artificial neurons, which loosely model the neurons in a biological brain. Each connection can send a signal to other neurons, like a synapse in a biological brain. An artificial neuron that receives a signal can then process it and send a signal to the neurons connected to it.

[0203] In an ANN implementation, the "signals" at the connections are real, and the output of each neuron is calculated by some nonlinear function of the sum of its inputs. The connections are called edges. Neurons and edges typically have weights that adjust as learning progresses. The weights increase or decrease the strength of the signal at the connection. Neurons may have a threshold so that a signal is sent only if the aggregate signal exceeds the threshold. Typically, neurons are aggregated into layers. Different layers may perform different transformations on their inputs. Signals travel from the first layer (input layer) to the last layer (output layer), possibly after traversing the layers multiple times.

[0204] The original goal of the ANN approach was to solve problems in the same way that the human brain does. Over time, attention has shifted to performing specific tasks, leading to a deviation from biology. ANNs have been used in a variety of tasks, including computer vision, speech recognition, machine translation, social network filtering, playing board and video games, medical diagnosis, and even activities traditionally thought of as the preserve of humans, such as painting.

[0205] The name "convolutional neural network" (CNN) indicates that the network uses a mathematical operation called convolution, which is a special kind of linear operation. Convolutional networks are neural networks that use convolution instead of the common matrix multiplication in at least one of their layers.

[0206] FIG. 16 shows a schematic of the general concept of processing by a neural network such as a CNN. A convolutional neural network consists of an input layer and an output layer, as well as multiple hidden layers. The input layer is the layer where the input (such as a portion of an image as shown in FIG. 16) is provided for processing. The hidden layers of a CNN typically consist of a series of convolutional layers that convolve using multiplication or other dot products. The result of the layers is one or more feature maps (f.map in FIG. 16), sometimes also referred to as channels. There may be subsampling involved in some or all of the layers. As a result, the feature maps become smaller, as shown in FIG. 16. The activation function in a CNN is usually a Rectified Linear Unit (ReLU) layer, followed by additional convolutions such as pooling layers, fully connected layers, and normalization layers, called hidden layers because their inputs and outputs are masked by the activation function and the final convolution. The layers are colloquially called convolutions, but this is only by convention. Mathematically, it is technically a sliding dot product or cross-correlation. This is important to the index in the matrix in that it affects how the weights are determined at a particular index point.

[0207] As shown in Figure 16, when programming a CNN to process images, the input is a tensor with shape (number of images) x (image width) x (image height) x (image depth). It should be known that image depth can be composed of the channels of an image. After passing through the convolutional layers, the image is abstracted into a feature map with shape (number of images) x (feature map width) x (feature map height) x (feature map channels). The convolutional layers in a neural network should have the following attributes: A convolution kernel defined by width and height (hyperparameters); The number of input and output channels (hyperparameters); The depth of the convolutional filter (input channels) is equal to the number of channels (depth) of the input feature map.

[0208] In the past, traditional multi-layer perceptron (MLP) models have been used for image recognition. However, due to the full connectivity between nodes, they suffer from high dimensionality and do not scale well with higher resolution images. A 1000x1000 pixel image with RGB color channels has 3 million weights, which is too high to be efficiently processed at a scale with full connectivity. Also, such network architectures do not take into account the spatial structure of the data, treating input pixels that are far apart the same as pixels that are close to each other. This ignores the locality of reference in the image data, both computationally and semantically. Thus, the full connectivity of neurons is useless for purposes such as image recognition, which are dominated by spatially local input patterns.

[0209] Convolutional neural networks are biologically inspired variants of multi-layer perceptrons specifically designed to emulate the behavior of the visual cortex. These models mitigate the challenges posed by MLP architectures by exploiting the strong spatially local correlations present in natural images. The convolutional layer is the core building block of CNNs. The parameters of the layer consist of a set of learnable filters (kernels, as mentioned above) that have a small receptive field but extend through the full depth of the input volume. During the forward pass, each filter is convolved across the width and height of the input volume, computing the dot product between the filter's entries and the input, and generating a 2D activation map for that filter. As a result, the network learns filters that activate when it detects a certain type of feature at a certain spatial location in the input.

[0210] Stacking the activation maps for all filters along the depth dimension forms the complete output volume of the convolutional layer. Every entry in the output volume can therefore also be interpreted as the output of a neuron that looks at a small region in the input and shares parameters with neurons in the same activation map. A feature map, or activation map, is the output activation for a given filter. Feature map and activation have the same meaning. In some papers it is called an activation map because it is a mapping that corresponds to the activation of different parts of the image, and it is also called a feature map because it is also a mapping of where certain features are found in the image. High activation means that a feature has been found.

[0211] Another important concept in CNNs is pooling, a form of nonlinear downsampling. There are several nonlinear functions to implement pooling, among which max pooling is the most common. It partitions the input image into a set of non-overlapping rectangles, and for each such subregion, it outputs the maximum value.

[0212] Intuitively, the exact location of a feature is less important than its rough location relative to other features. This is the idea behind the use of pooling in convolutional neural networks. Pooling layers serve to gradually reduce the spatial size of the representation, reducing the number of parameters in the network, the memory footprint, and the amount of computation, and thus also to control overfitting. It is common to periodically insert pooling layers between successive convolutional layers in CNN architectures. The pooling operation provides another form of translation invariance.

[0213] Pooling layers operate independently on every depth slice of the input and are spatially resizable. The most common form is a pooling layer with a filter of size 2×2 applied with a stride of 2 at every depth slice in the input by 2 along both the width and height, discarding 75% of the activations. In this case, each max operation exceeds the number of 4. The depth dimension remains unchanged. In addition to max pooling, the pooling unit can use other functions, such as average pooling or l2-norm pooling. Average pooling was often used historically, but has recently fallen out of favor in practice, compared to max pooling, which often performs better. Due to the aggressive reduction in the size of the representation, there is a recent trend towards using smaller filters or discarding the pooling layer altogether. "Region of interest" pooling (also known as ROI pooling) is a variant of max pooling where the output size is fixed and the input rectangle is a parameter. Pooling is a key component of convolutional neural networks for object detection based on the Faster R-CNN architecture.

[0214] The ReLU mentioned above stands for rectified linear unit and is an application of a non-saturating activation function. It effectively removes negative values ​​from the activation map by setting them to zero. This increases the non-linearity of the decision function and the entire network without affecting the receptive fields of the convolutional layers. Other functions are also used to increase the non-linearity, for example saturated hyperbolic tangent and sigmoid functions. ReLU is often preferred over other functions because it trains neural networks several times faster without a significant penalty to generalization accuracy.

[0215] After several convolutional and max pooling layers, high-level reasoning in neural networks is performed via fully connected layers. Neurons in fully connected layers have connections to all activations of the previous layer, as in normal (non-convolutional) artificial neural networks. Their activations can therefore be computed as affine transformations using matrix multiplications followed by bias offsets (vector addition of learned or fixed bias terms).

[0216] The “loss layer” (which contains the computation of the loss function) specifies how training penalizes deviations between predicted (output) labels and true labels, and is usually the final layer of a neural network. Various loss functions suitable for different tasks can be used: Softmax loss is used to predict a single class out of K mutually exclusive classes; Sigmoid cross entropy loss is used to predict K independent probability values ​​in [0,1]; Euclidean loss is used to regress to real-valued labels.

[0217] In summary, Figure 16 shows the data flow in a typical convolutional neural network. First, the input image passes through a convolutional layer and is abstracted into a feature map containing several channels, corresponding to the number of filters in the set of learnable filters of this layer. The feature map is then subsampled, for example using a pooling layer, which reduces the dimensionality of each channel in the feature map. Next, the data comes to another convolutional layer, which may have a different number of output channels. As mentioned above, the number of input and output channels are hyperparameters of the layer. To establish the connectivity of the network, these parameters need to be synchronized between two connected layers, such that the number of input channels of the current layer is equal to the number of output channels of the previous layer. For the first layer, which processes input data, e.g., an image, the number of input channels is usually equal to the number of channels of the data representation. For example, 3 channels for an RGB or YUV representation of an image or video, or 1 channel for a grayscale image or video representation.

[0218] Estimating the activation functions of neural network layers As mentioned above, neural networks may include activation layers. The output of such activation layers may be determined by applying so-called activation functions. Some activation functions, such as Sigmoid / Logistic, Hyperbolic Tangent, Exponential Linear Units, Softmax, Scaled Exponential Linear Units, and others, may require deterministic behavior on different platforms. Estimates of such nonlinear activation functions within at least one interval may be obtained similarly as described above for the CDF. Obtaining the CDF may be performed for other functions, such as activation functions applied in processing by neural networks. Constraints applied in the approximation may correspond to properties of the functions to be approximated, such as monotonicity, continuity, etc. In other words, the embodiments described herein used to obtain the CDF for entropy encoding and / or decoding may be equally applied to obtain activation functions used in neural networks.

[0219] The activation function is approximated in multiple intervals. For each interval, a subset of parameters is obtained. The subset of parameters defines a quadratic polynomial in said interval. The polynomial approximates the activation function in said interval. The quadratic polynomial satisfies a set of conditions. The set of conditions is adapted to the properties of the function to be approximated, e.g. monotonically increasing / decreasing, positive / negative derivatives, discontinuities, etc.

[0220] An estimate of an activation function to be applied to one or more layers of the neural network is obtained in at least one interval, the at least one interval being included in the plurality of intervals and corresponding to a subrange of the value range of the activation function. The estimate of the activation function is obtained based on a function defined in the at least one interval. The function 1130 is defined in the at least one interval by parameters of a quadratic polynomial. The quadratic polynomial satisfies a set of conditions.

[0221] Autoencoders and Unsupervised Learning An autoencoder is a type of artificial neural network used to learn efficient data coding in an unsupervised way. A schematic diagram is shown in Figure 17. The goal of an autoencoder is to learn a representation (encoding) of a set of data, typically for dimensionality reduction, by training the network to ignore the signal “noise”. Along with the reduction side, a reconstruction side is trained, where the autoencoder tries to generate a representation from the reduced encoding that is as close as possible to its original input, hence the name. In the simplest case, given one hidden layer, the encoder stage of an autoencoder takes an input x and maps it to h.

number

[0222] This image h is often called the code, latent variable, or latent representation. Here, σ is an element-wise activation function, such as a sigmoid function or a rectified linear unit. W is a weight matrix and b is a bias vector. The weights and biases are often initialized randomly and then updated iteratively during training by backpropagation. The decoder stage of the autoencoder then maps h to a reconstruction x' of the same shape as x.

number

[0223] Variational autoencoder models make strong assumptions about the distribution of the latent variables. They use a variational approach for latent representation learning, which results in an additional loss component and a specific estimator for the training algorithm called the Stochastic Gradient Variation Bayes (SGVB) estimator. The data is represented by a directed graphical model p θ (x|h), and the encoder determines the posterior distribution p θ Approximation q for (x|h) Φ We assume that we are learning (x|h), where θ and Φ denote the parameters of the encoder (recognition model) and decoder (generative model), respectively. The probability distribution of the latent vectors in a VAE typically matches the probability distribution of the training data much more closely than a standard autoencoder. The objective of a VAE has the following form:

number

[0224] Here, D KL stands for Kullback-Leibler divergence. The priors for the latent variables are usually determined by the centrally isotropic multivariate Gaussian p θ It is set that (h) = N(0,I). In general, the shapes of the variational and likelihood distributions are chosen so that they are factored Gaussian distributions.

number

number

[0225] Recent advances in the field of artificial neural networks, and convolutional neural networks in particular, have enabled researchers to interest in applying neural network-based techniques to image and video compression tasks. For example, end-to-end optimized image compression using networks based on variational autoencoders has been proposed.

[0226] Data compression is therefore considered a fundamental and well-studied problem in engineering and is commonly formulated with the goal of designing a code for a given discrete data ensemble that has minimal entropy. The solution relies heavily on knowledge of the probabilistic structure of the data and, therefore, the problem is closely related to probabilistic source modeling. However, since all practical codes must have finite entropy, continuous-valued data (such as a vector of image pixel intensities) must be quantized into a finite set of discrete values, which introduces errors.

[0227] In this context, known as a lossy compression problem, we must trade off two competing costs: the entropy of the discretized representation (rate), and the error resulting from quantization (distortion). Different compression applications, such as data storage or transmission over limited-capacity channels, require different rate-distortion tradeoffs.

[0228] Joint optimization of rate and distortion is difficult. Without further constraints, the general problem of optimal quantization in a high-dimensional space is intractable. For this reason, most existing image compression methods work by linearly transforming the data vector into an appropriate continuous-valued representation, quantizing its elements independently, and then encoding the resulting discrete representation using a reversible entropy code. This scheme is called transform coding, due to the central role of the transform.

[0229] For example, JPEG uses the discrete cosine transform on blocks of pixels, and JPEG 2000 uses multi-scale orthogonal wavelet decomposition. Typically, the three components of a transform coding method - the transform, the quantizer, and the entropy code - are optimized separately (often through manual parameter tuning). Modern video compression standards such as HEVC, VVC, and EVC also use transformed representations to code the residual signal after prediction. Several transforms have been used for that purpose, such as the discrete cosine and sine transforms (DCT, DST) and the low-frequency non-separable manually optimized transform (LFNST).

[0230] Variational image compression The variable autoencoder (VAE) framework can be considered as a nonlinear transform coding model. The transform process can be divided into four main parts. This is illustrated in Figure 18a, which shows the VAE framework.

[0231] The transformation process can be divided into four main parts. Figure 18a illustrates the VAE framework. In Figure 18a, the Encoder 101 maps the input image x to a latent representation (denoted as y) via a function y=f(x). This latent representation is also referred to as a part or point in the "latent space" in the following. The function f() is a transformation function that transforms the input signal x into a more compressible representation y. The Quantizer 102 converts the latent representation y into a quantized latent representation

number

[0232] A latent space can be understood as a compressed representation of data where similar data points are close to each other in the latent space. Latent spaces are useful for learning data features and finding simpler representations of data for analysis. The quantized latent representation of the hyperprior, T, ŷ, and the side information,

number

number

[0233] In Fig. 18a, component AE105 is an arithmetic encoding module that converts samples of the quantized latent representation ŷ and side information ẑ into a binary representation bitstream 1. The samples of ŷ and ẑ may comprise, for example, integer or floating-point numbers. One purpose of the arithmetic encoding module is to convert (through a process of binarization) the sample values ​​into a string of binary digits that are then included in the encoded image or in a bitstream that may comprise further portions corresponding to further side information.

[0234] Arithmetic decomposition (AD) 106 is the process of undoing the binarization process, where the binary digits are converted back to sample values. Arithmetic decoding is provided by arithmetic decoding module 106.

[0235] Note that the present disclosure is not limited to this particular framework. Furthermore, the present disclosure is not limited to image or video compression, and may be applied to object detection, image generation, and recognition systems as well.

[0236] In Fig. 18a there are two sub-networks connected to each other. A sub-network in this context is a logical division between parts of an overall network. For example, in Fig. 18a the modules 101, 102, 104, 105 and 106 are called the "Encoder / Decoder" sub-network. The "Encoder / Decoder" sub-network is responsible for encoding (generation) and decoding (parsing) the first bitstream "bitstream1". The second network in Fig. 18a comprises the modules 103, 108, 109, 110 and 107 and is called the "HyperEncoder / Decoder" sub-network. The second sub-network is responsible for generating the second bitstream "bitstream2". The purpose of the two sub-networks is different.

[0237] The first sub-network is responsible for: Transforming an input image x into its latent representation y (which is easier to compress than x) 101. Quantizing the latent representation y into a quantized latent representation y^102. Compressing the quantized latent representation ŷ using the AE via the arithmetic encoding module 105 to obtain the bitstream “bitstream1”. Parsing the bitstream 1 via AD using an arithmetic decoding module 106. · Reconstructing 104 a reconstructed image (x̂) using the analyzed data.

[0238] The purpose of the second sub-network is to obtain statistical properties of the samples of "Bitstream 1" (e.g., mean, variance, and correlation between samples of Bitstream 1) so that the first sub-network can compress Bitstream 1 more efficiently. The second sub-network generates a second bitstream, "Bitstream 2", that includes the above information (e.g., mean, variance, and correlation between samples of Bitstream 1).

[0239] The second network includes an encoding section, which converts the quantized latent representation y^ into side information z 103, quantizes the side information z into quantized side information z^, and encodes (e.g., binarizes) 109 the quantized side information z^ into a bitstream 2. In this example, binarization is performed by arithmetic encoding (AE). The decoding section of the second network includes arithmetic decoding (AD) 110, which converts the input bitstream 2 into decoded quantized side information z^'. Since arithmetic encoding and arithmetic decoding are lossless compression methods, z^' may be identical to z^. The decoded quantized side information z^' is then converted 107 into decoding side information y^', which represents a statistical property of y^ (e.g., the mean value of the samples of y^, or the variance of the sample values, etc.). The decoded latent representation y' is then provided to the arithmetic encoder 105 and arithmetic decoder 106 described above, which control the probability model of y.

[0240] FIG. 18a illustrates one example of a VAE (Variational Autoencoder), the details of which may vary in different implementations. For example, in a particular implementation, there may be additional components to more efficiently obtain statistical characteristics of samples of bitstream 1. In one such implementation, there may be a context modeler that targets extracting cross-correlation information of bitstream 1. The statistical information provided by the second sub-network may be used by the AE (Arithmetic Encoder) 105 and AD (Arithmetic Decoder) 106 components.

[0241] Figure 18a shows the encoder and decoder in a single diagram. As will be apparent to those skilled in the art, the encoder and decoder may, and very often are, integrated into different devices.

[0242] FIG. 18b shows the encoder and FIG. 18c shows the decoder component of the VAE framework in isolation. As input, the encoder receives a picture according to some embodiments. The input picture may include one or more channels, such as color channels or other types of channels, e.g., depth channels or motion information channels. The outputs of the encoder (as shown in FIG. 18b) are bitstream 1 and bitstream 2. Bitstream 1 is the output of the first sub-network of the encoder and bitstream 2 is the output of the second sub-network of the encoder.

[0243] Similarly, in Fig. 18c, two bitstreams, Bitstream1 and Bitstream2, are received as input and a reconstructed (decoded) image, ẑ, is produced at the output. As mentioned above, the VAE can be divided into different logical units performing different actions. This is illustrated in Figs. 18b and 18c, whereby Fig. 18b shows the components involved in encoding a signal, such as video, and provides encoding information. This encoded information is then received for encoding, for example, by the decoder component shown in Fig. 18c. Note that the encoder and decoder components denoted with the numerals 12x and 14x may correspond in function to the components described above in Fig. 18a and denoted with the numeral 10x.

[0244] In particular, as can be seen in Figure 18b, the encoder comprises an encoder 121 which converts an input x into a signal y, which is then provided to a quantizer 322. The quantizer 122 provides information to an arithmetic encoding module 125 and to a hyper-encoder 123. The hyper-encoder 123 provides the bitstream 2 already mentioned above to a hyper-decoder 147, which in turn provides information to an arithmetic encoding module 105 (125).

[0245] The output of the arithmetic encoding module is bitstream 1. Bitstream 1 and bitstream 2 are the output of the encoding of the signal and are then provided (sent) to the decoding process. Unit 101 (121) is called "encoder", but it is also possible to call the complete sub-network described in Fig. 18b "encoder". The process of encoding generally means a unit (module) that converts an input into an encoded (e.g. compressed) output. From Fig. 18b it can be seen that unit 121 can in fact be considered as the core of the entire sub-network, since it converts the input x into y, which is a compressed version of x. The compression in encoder 121 can be achieved, for example, by applying a neural network, or in general any processing network with one or more layers. In such a network, the compression can be performed by cascading processes, including downsampling to reduce the size and / or number of the channels of the input. Thus, the encoder can be called, for example, a neural network (NN)-based encoder, etc.

[0246] The remaining parts in the figure (quantization unit, hyperencoder, hyperdecoder, arithmetic encoder / decoder) are all parts responsible for improving the efficiency of the encoding process or converting the compressed output y into a series of bits (bitstream). Quantization may be provided to further compress the output of the NN encoder 121 by lossy compression. The AE 125 in combination with the hyperencoder 123 and hyperdecoder 127 used to construct the AE 125 can perform binarization, which may further compress the quantized signal by lossless compression. Therefore, it is also possible to call the whole sub-network of Fig. 18b an "encoder".

[0247] Most deep learning (DL) based image / video compression systems reduce the dimensionality of a signal before converting it into binary digits (bits). For example, in the VAE framework, a nonlinear transformation, an encoder maps an input image x to y, where y has a smaller width and height than x. Since y has a smaller width and height and thus a smaller size, the dimensionality of the signal is reduced, and therefore it is easier to compress the signal y. Note that in general, an encoder does not necessarily need to reduce the size in both (or generally all) dimensions. Rather, some example implementations may provide an encoder that reduces the size in only one dimension (or, generally, a subset thereof).

[0248] In J. Balle, L. Valero Laparra, and EP Simoncelli (2015) "Density Modeling of Images Using a Generalized Normalization Transformation", arXiv e-prints, 4th International Conference for Learning Representations, 2016 (hereafter "Balle"), the authors propose a framework for end-to-end optimization of image compression models based on nonlinear transformations. They optimize for the mean squared error (MSE) but use a more flexible transformation built from a cascade of linear convolutions and nonlinearities. In particular, they use a Generalized Divided Normalization (GDN) combined nonlinearity, inspired by a model of neurons in the biological visual system and proven effective in Gaussianizing image densities. This cascade transformation is followed by a uniform scalar quantization (i.e., each element is rounded to the nearest integer), which effectively implements a parametric form of vector quantization on the original image space. The compressed image is reconstructed from these quantized values ​​using an approximate parametric nonlinear inverse transform.

[0249] Such an example of a VAE framework is shown in FIG. 19 and utilizes six downsampling layers marked with 401 to 406. The network architecture includes a hyperprior model. a ,g s ) shows the image autoencoder architecture, and the right side (h a ,h s ) corresponds to an autoencoder that implements a hyperprior. A factorized-prior model is a model that implements the analytic transformation g a and the composite transformation g s We use the same architecture for both. Q stands for quantization, and AE and AD stand for arithmetic encoder and decoder, respectively. The encoder converts the input image x into g a to generate a response y (latent representation) with a spatially varying standard deviation. a The encoding of involves multiple convolutional layers with subsampling and generalized divisive normalization (GDN) as the activation function.

[0250] The response is h a to summarize the distribution of standard deviations in z. z is then quantized, compressed, and transmitted as side information. The encoder uses the quantized vector z to

number

[0251] Layers that include downsampling are indicated with a downward arrow in the layer description. The layer description "Conv Nx5x5 / 2↓" means that the layer is a convolutional layer, has N channels, and the convolution kernel is of size 5×5. As mentioned above, 2↓ means that a downsampling of a factor of 2 is performed in this layer. Downsampling by a factor of 2 results in one of the dimensions of the input signal being reduced by half at the output. In FIG. 19, 2↓ indicates that both the width and height of the input image are reduced by a factor of 2. Since there are 6 downsampling layers, if the width and height of the input image 414 (also denoted with x) are given by w and h, the output signal z^ 413 has a width and height equal to w / 64 and h / 64, respectively. The modules denoted AE, AD are arithmetic encoder, arithmetic decoder, which are described with reference to FIG. 18a to FIG. 18c. The arithmetic encoder and decoder are specific implementations of entropy coding. AE and AD may be replaced by other means of entropy coding. In information theory, entropy coding is a lossless data compression method used to convert the value of a symbol into a binary representation, which is a revertible process. Also, the "Q" in the figure corresponds to the quantization operation also described above in connection with FIG. 19 and further explained in the "Quantization" section. Also, the quantization operation and the corresponding quantization unit as part of the component 413 or 415 may not necessarily be present and / or may be replaced with another unit.

[0252] Also shown in Figure 19 is a decoder including upsampling layers 407 to 412. A further layer 420 is provided between upsampling layers 411 and 410 in the input processing order which is implemented as a convolutional layer but which does not provide upsampling on the received input. A corresponding convolutional layer 430 is also shown for the decoder. Such a layer may be provided in the NN to perform operations on the input that do not change the size of the input but do change certain characteristics. However, it is not necessary to provide such a layer.

[0253] Viewed in the processing order of the bitstream 2 through the decoder, the upsampling layers are performed in reverse order, i.e., from upsampling layer 412 to upsampling layer 407. Each upsampling layer is shown here to provide upsampling with an upsampling ratio of 2, denoted by ↑. Of course, it is not necessary that all upsampling layers have the same upsampling ratio, and other upsampling ratios such as 3, 4, 8, etc. may also be used. Layers 407 to 412 are implemented as convolutional layers (conv). In particular, the upsampling layers may be intended to provide an inverse operation on the input to that of the encoder, so that the size of the upsampling layers may apply a deconvolution operation to the received input, so that its size is increased by a factor corresponding to the upsampling ratio. However, the present disclosure is not generally limited to deconvolution, and upsampling may be performed in any other manner, such as by bilinear interpolation between two adjacent samples, by nearest neighbor sample copying, etc.

[0254] In the first sub-network, several convolutional layers (401 to 403) are followed by generalized divisive normalization (GDN) on the encoder side and inverse GDN (IGDN) on the decoder side. In the second sub-network, the activation function applied is ReLu. Note that the present disclosure is not limited to such implementation and that in general, other activation functions may be used instead of GDN or ReLu.

[0255] Cloud solutions for machine tasks Video Coding for Machines (VCM) is another computer science direction that is popular today. The main idea behind this approach is to transmit a coded representation of image or video information intended for further processing by computer vision (CV) algorithms, such as object segmentation, detection and recognition. In contrast to traditional image and video coding, which is intended for human perception, the quality characteristic is not the reconstructed quality but the performance of a computer vision task, e.g. object detection accuracy.

[0256] Video coding for machines is also called collaborative intelligence and is a relatively new paradigm for efficient deployment of deep neural networks across mobile-cloud infrastructures. By splitting the network between mobile and cloud, it is possible to distribute the computational workload so that the overall energy and / or latency of the system is minimized. In general, collaborative intelligence is a paradigm in which the processing of a neural network is distributed between two or more different computational nodes, e.g., devices, but in general, any functionally defined node. Here, the term "node" does not mean a node of the neural network described above. Rather, a (computational) node here refers to a separate device / module (physically or at least logically) that implements parts of a neural network. Such devices can be different servers, different end-user devices, a mix of servers and / or user devices and / or clouds and / or processors, etc. In other words, computational nodes can be considered as nodes that belong to the same neural network and communicate with each other to transfer coded data within / for the neural network. For example, one or more layers may run on a first device and one or more layers may run in another device to be able to perform complex calculations. However, the distribution may also be finer and a single layer may run on multiple devices. In this disclosure, the term "plurality" refers to more than one. In some existing solutions, part of the neural network function is run on a device (such as a user device or an edge device) or multiple such devices, and then the output (feature map) is passed to the cloud. The cloud is a collection of processing or computing systems located outside the device on which the neural network part operates. The concept of collaborative intelligence has also been extended to model training.In this case, data flows both ways: from cloud to mobile during backpropagation in training, and from mobile to cloud during the forward pass in training as well as inference.

[0257] Some works have presented semantic image compression by encoding deep features and then reconstructing the input image from them. Compression based on uniform quantization was presented, followed by Context-Based Adaptive Arithmetic Encoding (CABAC) from H.264. In some scenarios, it may be more efficient to send the output of the hidden layer (deep feature map) from the mobile part to the cloud, rather than sending the compressed natural image data to the cloud and then using the reconstructed image to perform object detection. Efficient compression of feature maps is useful for image and video compression and reconstruction for both human perception and machine vision. Entropy coding methods, e.g., arithmetic encoding, are popular approaches to compression of deep features (i.e., feature maps).

[0258] Today, video content contributes to over 80% of Internet traffic, and the proportion is expected to further increase. Therefore, it is important to build an efficient video compression system and generate higher quality frames with a given bandwidth budget. In addition, most video-related computer vision tasks, such as video object detection or video object tracking, are sensitive to the quality of the compressed video, and efficient video compression can benefit other computer vision tasks. Meanwhile, techniques in video compression are also useful for action recognition and model compression. However, in the past decades, video compression algorithms have relied on hand-crafted modules, such as block-based motion estimation and discrete cosine transform (DCT), to reduce redundancy in video sequences, as mentioned above. Although each module is well designed, the entire compression system has not been optimized end-to-end. It is desirable to further improve the video compression performance by jointly optimizing the entire compression system.

[0259] End-to-end image or video compression DNN-based image compression methods can take advantage of large-scale end-to-end training and highly nonlinear transformations that are not used in traditional approaches. However, it is non-trivial to directly apply these techniques to build end-to-end learning systems for video compression. First, learning how to generate and compress motion information tailored for video compression remains an open problem. Video compression methods rely heavily on motion information to reduce temporal redundancy in video sequences.

[0260] A straightforward solution is to use learning-based optical flow to represent the motion information. However, current learning-based optical flow approaches aim to generate a flow field that is as accurate as possible. Accurate optical flow is often not optimal for a particular video task. In addition, the amount of data for optical flow is significantly increased compared to the motion information in traditional compression systems, and directly applying existing compression approaches to compress the optical flow values ​​significantly increases the number of bits required to store the motion information. Second, it is unclear how to build a DNN-based video compression system by minimizing a rate-distortion-based objective for both the residual information and the motion information. Rate-distortion optimization aims to achieve a higher quality reconstructed frame (i.e., less distortion) given the number of bits (or bitrate) for compression. RDO is critical for video compression performance. To take advantage of the power of end-to-end training for learning-based compression systems, an RDO strategy is needed to optimize the entire system.

[0261] In "DVC: An End-to-end Deep Video Compression Framework" by Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, and Zhiyong Gao, Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pages 11006-11015, the authors propose an end-to-end deep video compression (DVC) model that jointly learns motion estimation, motion compression, and residual coding.

[0262] Such an encoder is shown in FIG. 6A. In particular, FIG. 6A shows the overall structure of an end-to-end trainable video compression framework. To compress the motion information, a CNN is specified to convert the optical flow into a corresponding representation suitable for better compression. Specifically, an autoencoder type network is used to compress the optical flow. A motion vector (MV) compression network is shown in FIG. 6B. The network architecture is somewhat similar to the ga / gs in FIG. 4. In particular, the optical flow is fed into a series of convolution operations and nonlinear transformations, including GDN and IGDN. The number of output channels for convolution (deconvolution) is 128, except for the last deconvolution layer, which is equal to 2. Given an optical flow with a size of M×N×2, the MV encoder generates a motion representation with a size of M / 16×N / 16×128. The motion representation is then quantized, entropy coded, and sent to the bitstream. The MV decoder receives the quantized representation and uses the MV encoder to reconstruct the motion information.

[0263] FIG. 6C shows the structure of the motion compensation unit, where the previous reconstructed frame x t-1 Using the reconstructed motion information, a warping unit generates a warped frame (usually with the help of an interpolation filter, such as a bilinear interpolation filter). A separate CNN with three inputs then generates a predicted picture. The architecture of the motion compensation CNN is also shown in Figure 6C.

[0264] The residual information between the original frame and the predicted frame is encoded by a residual encoder network. A highly nonlinear neural network is used to convert the residual into a corresponding latent representation. Compared with the discrete cosine transform in traditional video compression systems, this approach can better utilize the power of nonlinear transform and achieve higher compression efficiency.

[0265] From the above overview, considering different parts of the video framework including motion estimation, motion compensation, and residual coding, we can see that CNN-based architectures can be applied to both image and video compression. Entropy coding is a common method used for data compression, which has been widely adopted by the industry, and is also applicable to feature map compression for either human perception or computer vision tasks.

[0266] Implementation in image coding One possible development can be seen in FIGS.

[0267] Figure 20 shows a schematic block diagram of an exemplary video encoder 20 configured to implement the techniques of the present application. In the example of Figure 20, the video encoder 20 includes an input 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoding picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output 272 (or an output interface 272). The entropy coding 270 may implement the arithmetic encoding method or device described above.

[0268] The mode selection unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a splitting unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 as shown in FIG. 20 may be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0269] The encoder 20 may be configured to receive, for example via an input 201, a picture 17 (or picture data 17), for example a picture of a sequence of pictures forming a video or a video sequence. The received picture or picture data may also be a pre-processed picture 19 (or pre-processed picture data 19). For simplicity, the following description refers to the picture 17. The picture 17 is also called the current picture, or the picture to be coded (particularly in video coding, to distinguish the current picture from other pictures, for example previously encoded and / or decoded pictures of the same video sequence, i.e. the video sequence which also contains the current picture).

[0270] A (digital) picture is or can be considered as a two-dimensional array or matrix of samples with intensity values. The samples in the array may also be called pixels (short for picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, typically three color components are used; that is, a picture may be represented by or contain three sample arrays. In an RGB format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance and chrominance format or color space, e.g. YCbCr, which contains a luminance component denoted by Y (sometimes L is also used instead) and two chrominance components denoted by Cb and Cr. The luminance (or luma for short) component Y represents brightness or gray level intensity (e.g., as in a grayscale picture), while the two chrominance (or chroma for short) components Cb and Cr represent chromaticity or color information components. Thus, a picture in YCbCr format comprises a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). A picture in RGB format may be converted or translated into YCbCr format and vice versa, this process also known as color conversion or translation. If the picture is monochrome, the picture may comprise only a luminance sample array. Thus, a picture may be, for example, an array of luma samples in monochrome format, or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0271] An embodiment of the video encoder 20 may comprise a picture division unit (not shown in FIG. 20) configured to divide a picture 17 into multiple (typically non-overlapping) picture blocks 203. These blocks may also be called root blocks, macroblocks (H.264 / AVC), or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265 / HEVC and VVC). The picture division unit may be configured to use the same block size for all pictures of a video sequence and a corresponding grid that defines the block size, or to vary the block size between pictures or between a subset or group of pictures, and to divide each picture into corresponding blocks. The abbreviation AVC stands for Advanced Video Coding.

[0272] In a further embodiment, the video encoder may be configured to directly receive block 203 of picture 17, e.g., one, several, or all of the blocks forming picture 17. Picture block 203 may also be referred to as a current picture block or a picture block to be coded.

[0273] Similar to the image 17, the picture block 203 is or can be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values), again with smaller dimensions than the image 17. In other words, the block 203 may comprise, for example, one sample array (e.g., a luma ray for a monochrome picture 17, or a luma or chroma array for a color picture), or three sample arrays (e.g., a luma and two chroma arrays for a color picture 17), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (also axes) of the block 203 defines the size of the block 203. Thus, the block may be, for example, an M×N (M columns×N rows) array of samples, or an M×N array of transform coefficients.

[0274] An embodiment of video encoder 20 as shown in FIG. 20 may be configured to encode picture 17 on a block-by-block basis, eg, encoding and prediction is performed for each block 203.

[0275] An embodiment of video encoder 20 as shown in FIG. 20 may be further configured to divide and / or encode a picture using slices (also referred to as video slices). A picture may be divided into or encoded using one or more slices (typically non-overlapping), and each slice may comprise one or more blocks (e.g., CTUs).

[0276] An embodiment of video encoder 20 as shown in FIG. 20 may be further configured to partition and / or encode a picture using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles). A picture may be partitioned into or encoded using one or more tile groups (typically non-overlapping), and each tile group may comprise, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape, and may comprise one or more blocks (e.g., CTUs), e.g., full blocks or partial blocks.

[0277] 21 shows an example of a video decoder 30 configured to implement the techniques of the present application. The video decoder 30 is configured to receive encoding picture data 21 (e.g., encoding bitstream 21), e.g., encoded by encoder 20, to obtain a decoded picture 331. The encoding picture data or bitstream comprises information for decoding the encoding picture data, e.g., data representing picture blocks of an encoding video slice (and / or tile group or tile), and associated syntax elements.

[0278] The entropy decoding unit 304 is configured to parse the bitstream 21 (or, generally, the encoding picture data 21) and, e.g., perform entropy decoding on the encoding picture data 21 to obtain, e.g., quantization coefficients 309 and / or decoded coding parameters (not shown in FIG. 21 ), e.g., any or all of inter prediction parameters (e.g., reference picture indexes and motion vectors), intra prediction parameters (e.g., intra prediction modes or indices), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to the encoding schemes described with respect to the entropy coding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter prediction parameters, intra prediction parameters and / or other syntax elements to the mode application unit 360 and provide other parameters to other units of the decoder 30. The video decoder 30 may receive syntax elements at a video slice level and / or a video block level. In addition to or as an alternative to slices and their respective syntax elements, tile groups and / or tiles and their respective syntax elements may be received and / or used. The entropy decoding may implement any of the arithmetic decoding methods or apparatus described above.

[0279] The reconstruction unit 314 (e.g., adder or summer 314) may be configured to add the reconstructed residual block 313 to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, for example by adding sample values ​​of the reconstructed residual block 313 and sample values ​​of the prediction block 365.

[0280] An embodiment of video decoder 30 such as that shown in FIG. 21 may be configured to divide and / or decode pictures using slices (also referred to as video slices), where a picture may be divided into or decoded using one or more slices (typically non-overlapping), and each slice may comprise one or more blocks (e.g., CTUs).

[0281] An embodiment of video decoder 30 as shown in FIG. 21 may be configured to partition and / or decode a picture using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where a picture may be partitioned into or decoded using one or more (typically non-overlapping) tile groups, where each tile group may comprise, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape and may comprise one or more blocks (e.g., CTUs), e.g., full or partial blocks.

[0282] Other variations of the video decoder 30 may be used to decode the encoded picture data 21. For example, the decoder 30 may generate an output video stream without using a loop filtering unit 320. For example, a non-transform-based decoder 30 may inverse quantize the residual signal directly for a given block or frame without using the inverse transform processing unit 312. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0283] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after the interpolation filtering, the motion vector derivation, or the loop filtering, further operations such as clipping or shifting may be performed on the processing result of the interpolation filtering, the motion vector derivation, or the loop filtering.

[0284] One or more blocks in the exemplary scheme of the hybrid encoder described above with reference to Figures 20 and 21 may be implemented by neural network processing. For example, the inter prediction unit 244 and / or the intra prediction unit 254 may involve neural network processing.

[0285] Hardware and software implementation Several further implementations in hardware and software are described below.

[0286] Any of the encoding devices described below with reference to Figures 22 through 25 may provide means for performing entropy encoding of data into a bitstream. Processing circuitry within any of these example devices is configured to obtain an estimate of the cumulative distribution function and to apply a probability model based on the obtained CDF in at least one interval according to the method described above to entropy encode the data.

[0287] Any of the decoding devices of Figures 22 to 25 may include a processing circuit adapted to perform a decoding method, which comprises obtaining an estimate of a cumulative distribution function (CDF) for at least one of a plurality of predefined intervals, where the CDF is obtained based on a function defined in the at least one interval by parameters of a quadratic polynomial, where the quadratic polynomial satisfies a set of conditions, and applying a probability model based on the obtained CDF in the at least one interval to entropy decode the data.

[0288] Any of the devices in any of Figures 22 through 25, or another device, may include processing circuitry configured to obtain a subset of a set of parameters that define a quadratic polynomial in an interval, the quadratic polynomial approximating the CDF, where the polynomial satisfies a set condition for the interval in a plurality of predefined intervals.

[0289] In the following embodiments of video coding system 10, video encoder 20 and video decoder 30 are described based on FIGS. 22 to 25 with reference to FIGS. 20 and 21 above, or other encoders and decoders, such as neural network-based encoders and decoders.

[0290] 22 is a schematic block diagram illustrating an example coding system, such as a video, image, audio, and / or other coding system (or short coding system), that may utilize the techniques of the present application. The video encoder 20 (or short encoder 20) and the video decoder 30 (or short decoder 30) of the video coding system 10 represent examples of devices that may be configured to perform techniques according to various examples described in the present application. For example, video coding and decoding may be distributed and employ neural networks that may apply the above-mentioned bitstream parsing and / or bitstream generation to transfer feature maps between distributed computational nodes (two or more).

[0291] As shown in FIG. 22, coding system 10 comprises a source device 12 configured to provide encoded picture data 21 to, for example, a destination device 14 for decoding encoded picture data 13.

[0292] The source device 12 comprises an encoder 20, and may additionally, i.e. optionally, comprise a picture source 16, a pre-processor (or pre-processing unit) 18, such as a picture pre-processor 18, and a communication interface or unit 22.

[0293] Image source 16 may comprise or be any type of image capture device, e.g., a camera for capturing real-world images, and / or any type of image generation device, e.g., a computer graphics processor for generating computer animated images, or any type of other device for obtaining and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Picture source may be any type of memory or storage device that stores any of the aforementioned pictures.

[0294] As distinct from pre-processing unit 18 and the processing performed by pre-processing unit 18 , the picture or picture data 17 is also referred to as a raw picture or raw picture data 17 .

[0295] The pre-processor 18 is configured to receive the (raw) picture data 17 and perform pre-processing on the picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19. The pre-processing performed by the pre-processor 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise removal. It may be understood that the pre-processing unit 18 may be any component. Note that the pre-processing may also employ a neural network (such as any of those in Figures 1 to 7) that uses presence indicator signaling.

[0296] Video encoder 20 is configured to receive pre-processed picture data 19 and to provide encoded picture data 21 .

[0297] The communications interface 22 of the source device 12 may be configured to receive the encoded picture data 21 and to transmit the encoded picture data 21 (or any further processed version thereof) via the communications channel 13 to another device, such as the destination device 14 or any other device, for storage or direct reconstruction.

[0298] Destination device 14 comprises a decoder 30 (eg, a video decoder 30), and may additionally, i.e. optionally, comprise a communications interface or unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0299] The communications interface 28 of the destination device 14 is configured to receive the encoded picture data 21 (or any further processed version thereof), for example directly from the source device 12 or from any other source, for example a storage device, for example an encoded picture data storage device, and to provide the encoded picture data 21 to a decoder 30.

[0300] Communications interface 22 and communications interface 28 may be configured to transmit or receive encoding picture data 21 or encoding data 13 over a direct communications link between source device 12 and destination device 14, e.g., a direct wired or wireless connection, or over any type of network, e.g., a wired or wireless network, or any combination thereof, or over any type of private and public network, or any combination thereof.

[0301] The communications interface 22 may be configured, for example, to package the encoded picture data 21 into an appropriate format, e.g., packets, and / or to process the encoded picture data using any type of transmission encoding or processing for transmission over a communications link or network.

[0302] The communications interface 28, forming a counterpart to the communications interface 22, may for example be configured to receive transmitted data and process the transmitted data using any kind of corresponding transmission decoding or processing and / or de-packaging to obtain the encoded picture data 21.

[0303] Both communication interface 22 and communication interface 28 may be configured as unidirectional communication interfaces, as indicated by the arrow pointing from source device 12 to destination device 14 relative to communication channel 13 in Figure 22, or as bidirectional communication interfaces, and may be configured, for example, to send and receive messages, for example, to set up connections, to acknowledge and exchange communications links and / or any other information related to data transmission, for example, encoding picture data transmission. Decoder 30 is configured to receive encoding picture data 21 and provide decoding picture data 31 or decoding pictures 31.

[0304] The post-processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also referred to as reconstructed picture data), e.g., the decoded picture 31, to obtain post-processed picture data 33, e.g., the post-processed picture 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color correction, cropping, or resampling, or any other processing to prepare the decoded picture data 31 for display by, for example, a display device 34.

[0305] A display device 34 of destination device 14 is configured to receive the post-processed picture data 33, for example, to display the picture to a user or viewer. The display device 34 may be or include any type of display for presenting the reconstructed picture, such as, for example, an integrated or external display or monitor. The display may include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any type of other display.

[0306] 22 illustrates source device 12 and destination device 14 as separate devices, an embodiment of the device may also include functionality of both, i.e., source device 12 or corresponding functionality, and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality, and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or by any combination thereof.

[0307] As will be clear to a person skilled in the art based on the description, the presence and (exact) division of functions or features of different units within the source device 12 and / or destination device 14 as shown in FIG. 22 may vary depending on the actual device and application.

[0308] The encoder 20 (e.g., video encoder 20) or the decoder 30 (e.g., video decoder 30), or both the encoder 20 and the decoder 30, may be implemented via processing circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video coding, or any combination thereof. The encoder 20 may be implemented via processing circuitry 46 to embody various modules, including a neural network or portions thereof. The decoder 30 may be implemented via processing circuitry 46 to embody any coding system or subsystem described herein. The processing circuitry may be configured to perform various operations, such as those described below. When the techniques are implemented in part in software, a device may store instructions for the software on a suitable non-transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Either the video encoder 20 and the video decoder 30 may be integrated as part of a combined encoder / decoder (codec) within a single device, for example, as shown in FIG. 23.

[0309] Source device 12 and destination device 14 may comprise any of a wide range of devices, including any type of handheld or fixed device, e.g., a notebook or laptop computer, a mobile phone, a smartphone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiver device, a broadcast transmitter device, and the like, and may use no operating system or any type of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

[0310] In some cases, the video coding system 10 shown in FIG. 22 is just one example, and techniques of the present application may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include data communication between encoding and decoding devices. In other examples, data may be retrieved from local memory, streamed over a network, etc. A video coding device may encode data and store it in memory, and / or a video decoding device may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data in memory and / or retrieve data from memory and decode it.

[0311] 24 is a schematic diagram of a video coding apparatus 8000 according to an embodiment of the present invention. The video coding apparatus 8000 is suitable for implementing the disclosed embodiments as described herein. In one embodiment, the video coding apparatus 8000 may be a decoder, such as the video decoder 30 of FIG. 22, or an encoder, such as the video encoder 20 of FIG. 22.

[0312] The video coding device 8000 comprises an ingress port 8010 (or input port 8010) and a receiver unit (Rx) 8020 for receiving data, a processor, logic unit, or central processing unit (CPU) 8030 for processing data, a transmitter unit (Tx) 8040 and an egress port 8050 (or output port 8050) for transmitting data, and a memory 8060 for storing data. The video coding device 8000 may also comprise optical-electrical (OE) and electrical-optical (EO) components coupled to the ingress port 8010, the receiver unit 8020, the transmitter unit 8040, and the egress port 8050 for egress or ingress of optical or electrical signals.

[0313] The processor 8030 is implemented by hardware and software. The processor 8030 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 8030 communicates with the ingress port 8010, the receiver unit 8020, the transmitter unit 8040, the egress port 8050, and the memory 8060. The processor 8030 includes a neural network-based codec 8070. The neural network-based codec 8070 implements the above-disclosed embodiments. For example, the neural network-based codec 8070 implements, processes, prepares, or provides various coding operations. The inclusion of the neural network-based codec 8070 thus provides a substantial improvement in the functionality of the video coding device 8000 and results in the transformation of the video coding device 8000 into different states. Alternatively, the neural network based codec 8070 may be implemented as instructions stored in memory 8060 and executed by the processor 8030.

[0314] The memory 8060 may comprise one or more disks, tape drives, and solid state drives and may be used as an overflow data storage device to store programs when such programs are selected for execution and to store instructions and data read during program execution. The memory 8060 may be, for example, volatile and / or non-volatile, and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0315] FIG. 25 is a simplified block diagram of an apparatus that may be used as either or both of source device 12 and destination device 14 from FIG. 22, according to an example embodiment.

[0316] The processor 9002 in the device 9000 may be a central processing unit. Alternatively, the processor 9002 may be any other type of device or devices capable of manipulating or processing information, now existing or later developed. Although the disclosed implementations may be implemented with a single processor as shown, e.g., processor 9002, advantages in speed and efficiency may be achieved using two or more processors.

[0317] The memory 9004 in the device 9000 may be a read-only memory (ROM) device or a random access memory (RAM) device in one implementation. Any other suitable type of storage device may be used as the memory 9004. The memory 9004 may include code and data 9006 that is accessed by the processor 9002 using the bus 9012. The memory 9004 may further include an operating system 9008 and application programs 9010, which include at least one program that enables the processor 9002 to perform the methods described herein. For example, the application programs 9010 may include applications 1 through N, which further include a video coding application that performs the methods described herein.

[0318] The apparatus 9000 may also include one or more output devices, such as a display 9018. The display 9018, in one example, may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 9018 may be coupled to the processor 9002 via the bus 9012.

[0319] Although shown here as a single bus, the bus 9012 of the device 9000 may be comprised of multiple buses. Additionally, the secondary storage device may be directly coupled to other components of the device 9000 or may be accessed over a network, and may comprise a single integrated unit such as a memory card, or multiple units such as multiple memory cards. Thus, the device 9000 may be implemented in a wide variety of configurations.

[0320] It should be noted that although embodiments of the present invention have been described primarily in terms of video coding, embodiments of coding system 10, encoder 20, and decoder 30 (and, correspondingly, system 10), as well as other embodiments described herein, may also be configured for still image processing or coding, i.e., processing or coding of individual pictures independent of any preceding or successive pictures, as in video coding. In general, when image processing coding is limited to a single picture 17, only inter prediction units 244 (encoder) and 344 (decoder) may not be available. All other functions (also called tools or techniques) of the video encoder 20 and the video decoder 30 may be equally used for still image processing, such as residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, partitioning 262 / 362, intra prediction 254 / 354, and / or loop filtering 220, 320, as well as entropy encoding 270 and entropy decoding 304.

[0321] The functions described herein with reference to, for example, the embodiments of the encoder 20 and the decoder 30, and the encoder 20 and the decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium, including any medium that facilitates transfer of a computer program from one place to another, for example according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0322] By way of example, and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where a disk often reproduces data magnetically, while a disc reproduces data optically with a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0323] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the above structures, or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated within a composite codec. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0324] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units including one or more processors as described above, along with suitable software and / or firmware.

[0325] Although operations are shown in the figures in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown, or in sequential order, or that all of the shown operations be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, it should be understood that the separation of various system modules and components in the above-described embodiments does not require such separation in all embodiments, and that the program components and systems described may generally be integrated together in a single software product or packaged in multiple software products.

[0326] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0327] In summary, a method and apparatus are provided for approximating the Cumulative Distribution Function (CDF) for each interval using a quadratic polynomial while imposing constraints on the polynomials within the interval and / or on the boundaries between the intervals. In this way, a CDF approximation is obtained that can be used in a variety of applications, including entropy encoding and decoding of arbitrary source data. The constraints correspond to properties of the CDF being approximated.

Claims

1. 1. A method for entropy encoding data into a bitstream, comprising the steps of: obtaining a cumulative distribution function, CDF, for at least one of the plurality of intervals; The CDF is obtained based on a function defined in the at least one interval by parameters of a second order polynomial; and The second order polynomial satisfies a set of conditions: Steps and applying a probability model based on the CDF obtained in the at least one interval to entropy code the data; A method comprising:

2. The set of conditions is: said quadratic polynomial in said at least one interval and said polynomials in adjacent intervals provide monotonically non-decreasing functions; the second order polynomial having a non-negative derivative within the at least one interval; Including one or more of the following: The method of claim 1.

3. The function defined by the parameters specifies a predefined CDF having a predefined mean and a predefined standard deviation. The method according to claim 1 or 2.

4. The CDF is obtained by shifting at least one argument of the function.

4. The method according to claim 1 .

5. The CDF is obtained by scaling at least one argument of the function.

5. The method according to any one of claims 1 to 4.

6. The entropy coding is an arithmetic coding.

6. The method according to any one of claims 1 to 5.

7. the function in one or more of the plurality of intervals is defined by a linear polynomial; 7. The method according to any one of claims 1 to 6.

8. the function in a last one of the intervals is defined by a linear polynomial; 8. The method according to any one of claims 1 to 7.

9. the function approximates a predefined CDF having a predefined mean and a predefined standard deviation; the function is point-symmetric with respect to the predetermined mean, and the function is defined in an interval that includes values ​​higher than the predetermined mean; 9. The method according to any one of claims 1 to 8.

10. A look-up table, LUT, provides the parameters for each interval.

10. The method according to any one of claims 1 to 9.

11. The obtained CDF represents the CDF of a Gaussian distribution.

11. The method according to any one of claims 1 to 10.

12. The parameters are in a fixed-point representation with a predefined precision.

12. The method according to any one of claims 1 to 11.

13. In the step of obtaining the CDF, intermediate results are adapted to the predetermined precision according to a set of rules. The method of claim 12.

14. for a value x=s-μ-0.5 in the at least one interval, where s represents the symbol being encoded and μ is the mean of the CDF; The CDF is, for x less than 0, [0010] Obtained by a i , b i , and c i are the parameters of the second order polynomial in said at least one interval, Parameters [0025] is the standard deviation σ of the CDF for x less than 0 left , and scaling the value x by a small predefined parameter ε>0; and The multiplication factor M is related to the predetermined precision, 1 / M, [0030] It is defined as follows: total is related to the number of bits used in entropy coding, and s max is the maximum number of symbols to be encoded, The CDF is, for x greater than zero, [0045] Obtained by a i , b i , and c i are the parameters of the second order polynomial in said at least one interval, Parameters [0050] is the standard deviation σ of the CDF for x greater than 0 left , and scaling the value x by a small default parameter ε>0.

14. The method according to claim 12 or 13.

15. 1. A method for entropy decoding data from a bitstream, comprising the steps of: obtaining a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals; The CDF is obtained based on a function defined in the at least one interval by parameters of a second order polynomial; and The second order polynomial satisfies a set of conditions: Steps and applying a probability model based on the CDF obtained in the at least one interval to entropy decode the data; A method comprising:

16. The set of conditions is: said quadratic polynomial in said at least one interval and said polynomials in adjacent intervals provide monotonically non-decreasing functions; the second order polynomial having a non-negative derivative within the at least one interval; Including one or more of the following: The method of claim 15.

17. The function defined by the parameters specifies a predefined CDF having a predefined mean and a predefined standard deviation.

17. The method according to claim 15 or 16.

18. The CDF is obtained by shifting at least one argument of the function.

18. The method according to any one of claims 15 to 17.

19. The CDF is obtained by scaling at least one argument of the function.

19. The method according to any one of claims 15 to 18.

20. The entropy decoding is arithmetic decoding.

20. The method of any one of claims 15 to 19.

21. the function in one or more of the plurality of intervals is defined by a linear polynomial; 21. The method according to any one of claims 15 to 20.

22. the function in a last interval of the plurality of intervals is defined by a linear polynomial; 22. The method according to any one of claims 15 to 21.

23. the function approximates a predefined CDF having a predefined mean and a predefined standard deviation; the function is point-symmetric with respect to the predetermined mean, and the function is defined in an interval that includes values ​​higher than the predetermined mean; 23. The method according to any one of claims 15 to 22.

24. A look-up table, LUT, provides the parameters for each interval.

24. The method according to any one of claims 15 to 23.

25. The obtained CDF represents the CDF of a Gaussian distribution.

25. The method of any one of claims 15 to 24.

26. The parameters are in a fixed-point representation with a predefined precision.

26. The method of any one of claims 15 to 25.

27. In the step of obtaining the CDF, intermediate results are adapted to the predetermined precision according to a set of rules.

27. The method of claim 26.

28. for a value x=s-μ-0.5 in the at least one interval, where s represents the symbol being encoded and μ is the mean of the CDF; The CDF is, for x less than 0, [006] Obtained by a i , b i , and c i are the parameters of the second order polynomial in said at least one interval, Parameters [0070] is the standard deviation σ of the CDF for x less than 0 left, and scaling the value x by a small predefined parameter ε>0; and The multiplication factor M is related to the predetermined precision, 1 / M, [0080] It is defined as follows: total is related to the number of bits used in entropy coding, and s max is the maximum number of symbols to be encoded, The CDF is, for x greater than zero, [0097] Obtained by a i , b i , and c i are the parameters of the second order polynomial in said at least one interval, Parameters [0010] is the standard deviation σ of the CDF for x greater than 0 left , and scaling the value x by a small default parameter ε>0.

28. The method of claim 26 or 27.

29. 1. A method for obtaining a set of parameters specifying a cumulative distribution function, CDF, for entropy coding, comprising the steps of: For intervals at multiple predefined intervals, obtaining a subset of the set of parameters that defines a second order polynomial within the interval; the second order polynomial approximating the CDF; The polynomial satisfies a set of conditions: Step, A method comprising:

30. The set of conditions is: said second order polynomial and the polynomials in adjacent intervals provide monotonically non-decreasing functions; the second order polynomials having non-negative derivatives within each of the intervals; Including one or more of the following:

30. The method of claim 29.

31. The function defined by the parameters specifies a predefined CDF based on a predefined mean and a predefined standard deviation.

31. The method of claim 29 or 30.

32. the function in one or more of the plurality of intervals is defined by a linear polynomial; 32. The method of any one of claims 29 to 31.

33. the function in a last interval of the plurality of intervals is defined by a linear polynomial; 33. A method according to any one of claims 29 to 32.

34. the function approximates a predefined CDF having a predefined mean and a predefined standard deviation; the function is point symmetric about the predetermined mean, and the function is defined as an interval that includes values ​​higher than the predetermined average, 34. A method according to any one of claims 29 to 33.

35. The parameters for each interval are stored in a look-up table, LUT.

35. A method according to any one of claims 29 to 34.

36. The CDF represents the CDF of a Gaussian distribution.

36. The method of any one of claims 29 to 35.

37. The parameters are in a fixed-point representation with a predefined precision.

37. A method according to any one of claims 29 to 36.

38. In the obtaining step, a cost function is optimized; The cost function includes the difference between the second order polynomial and the CDF.

38. The method of any one of claims 29 to 37.

39. the cost function includes a penalty that emphasizes the difference between the second order polynomial and the CDF at the boundaries of the interval; 39. The method of claim 38.

40. The cost function includes a penalty, a penalty is added when the apex of the parabola defined by the second degree polynomial is within the interval or to the left of the interval; 40. The method of claim 38 or 39.

41. A computer program stored on a non-transitory medium and including code instructions, When executed on one or more processors, causes the one or more processors to perform the steps of the method of any one of claims 1 to 40. Computer program.

42. 1. An apparatus for entropy encoding data into a bitstream, comprising a processing circuit, comprising: The processing circuitry includes: Obtaining a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals; The CDF is obtained based on a function defined in the at least one interval by parameters of a second order polynomial; and The second order polynomial satisfies a set of conditions: applying a probability model based on the CDF obtained in the at least one interval to entropy code the data; It is configured as follows: Device.

43. 1. An apparatus for entropy decoding data from a bitstream, comprising a processing circuit, The processing circuitry includes: Obtaining a cumulative distribution function, CDF, for at least one of a plurality of predefined intervals; The CDF is obtained based on a function defined in the at least one interval by parameters of a second order polynomial; The second order polynomial satisfies a set of conditions: applying a probability model based on the CDF obtained in the at least one interval to entropy decode the data; It is configured as follows: Device.

44. 1. An apparatus for entropy encoding data from a bitstream, comprising: A processing circuit configured to carry out the method according to any one of claims 1 to 14.

13. An apparatus comprising:

45. 1. An apparatus for obtaining a set of parameters specifying a cumulative distribution function, CDF, for entropy coding, comprising: The apparatus includes a processing circuit; The processing circuitry determines, for an interval within a plurality of predefined intervals, obtaining a subset of a set of parameters defining a second order polynomial within the interval; the second order polynomial approximating the CDF; The polynomial satisfies a set condition, It is configured as follows: Device.

46. 1. An apparatus for entropy decoding data from a bitstream, comprising: A processing circuit configured to carry out the method of any one of claims 15 to 28.

13. An apparatus comprising:

47. 1. An apparatus for obtaining a set of parameters specifying a cumulative distribution function, CDF, for entropy coding, comprising: A processing circuit configured to carry out the method of any one of claims 15 to 28.

13. An apparatus comprising:

48. Storing a bitstream generated by the entropy coding method according to any one of claims 1 to 14 or the entropy coding device according to claim 43 or 44. A computer-readable storage medium.

Citation Information

Patent Citations

  • Image encoding and decoding method and device based on wavelet transform

    CN112235583A

  • Linear encoder for image / video processing

    JP2021510885A