Adaptive selection of entropy coding parameters

Abstract

Methods and apparatus for encoding data into and decoding data from a code stream are described. There is provided a decoding method comprising: receiving a code stream comprising encoded data of an input signal and a first parameter; analyzing the code stream to obtain the first parameter; acquiring an entropy decoding parameter according to the first parameter; at least a portion of the input signal is reconstructed from the entropy coding parameters and the encoded data. Due to the self-adaptability of entropy decoding parameters, the entropy decoder can optimally work at a low bit rate, so that the bit rate is saved; and an amplitude limiting effect does not exist at a high bit rate, so that the quality of a reconstructed signal is improved.

Description

[0001] The present invention relates to entropy coding and decoding. In particular, the present invention relates to adaptive selection of entropy coding parameters. Background Art

[0002] Video coding (video encoding and decoding) is widely used in digital video applications such as broadcast digital television, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat and video conferencing, DVD and Blu-ray Discs, video content acquisition and editing systems, video recording on mobile devices, and camcorders for security applications.

[0003] Even short videos require a large amount of video data to depict, which can cause difficulties when the data is streamed or otherwise transmitted over a communication network with limited bandwidth capacity. With limited network resources and the growing demand for higher video quality, there is a need for improved compression and decompression techniques that can increase compression ratios with little impact on image quality. Encoding and decoding of videos can be performed by standard video encoders and decoders, such as those compatible with H.264 / AVC, HEVC (H.265), VVC (H.266), or other video decoding technologies. In addition, video decoding or portions thereof can be performed by neural networks.

[0004] Entropy decoding has been widely used in the encoding or decoding of any still image or other source signal (such as the characteristic channel of a neural network). The input alphabet of the entropy encoder is limited, and the size of the input alphabet should be known on both the encoder and decoder sides. A decoder with a larger input alphabet size supports encoding a wider range of symbols, but is less efficient than the same decoder with a smaller input alphabet. Due to this effect, it is best to use the smallest possible alphabet. In traditional methods, the entropy decoding parameters, specifically the input alphabet size, are predefined and used for all possible input signals, which will cause a limiting effect under high bit rate conditions and an unreasonable waste of bits under low bit rate conditions. As a result, the reconstruction quality and decoding efficiency will be reduced. Summary of the invention

[0005] Embodiments of the present invention provide an apparatus and method for entropy encoding data into a bitstream and entropy decoding data from the bitstream.

[0006] Embodiments of the invention are defined by the features of the independent claims, and further advantageous realizations of these embodiments are defined by the features of the dependent claims.

[0007] According to a first aspect, an embodiment of the present application provides a decoding method implemented by a decoder, the decoding method comprising: receiving a bit stream including encoded data of an input signal and a first parameter; parsing the bit stream to obtain the first parameter; obtaining an entropy decoding parameter based on the first parameter; and reconstructing at least a portion of the input signal based on the entropy decoding parameter.

[0008] In conventional methods, entropy decoding parameters are usually predefined, for example, the alphabet size M is usually predefined by selecting once according to the expected tensor range (or potential tensor range) and using the predefined alphabet size M for all cases. Since the size of the input alphabet of the entropy encoder is the same as the size of the output alphabet of the entropy decoder, the alphabet size M here represents the size of the input alphabet of the entropy encoder, or the size of the output alphabet of the entropy decoder. In this case, if the actual tensor range is wider than the expected tensor range, the input alphabet size determined according to the expected tensor range will not be appropriate, and the decoded tensor values ​​need to be clipped. Such clipping can damage the signal, especially when the decoded tensor range is very different from the alphabet size. In this case, the damage to the decoded tensor is nonlinear distortion, which causes unpredictable errors in the reconstructed signal, so the quality of the reconstructed signal may be significantly affected. In one implementation, a very large alphabet size may be chosen and used in all cases, but increasing the alphabet size penalizes compression efficiency at low bit rates, and using a large alphabet size will result in a significant increase in bit rate without improving reconstruction quality.

[0009] In an embodiment of the present application, the decoder can obtain entropy decoding parameters (specifically, the alphabet size) according to the parameters carried in the code stream. Since the parameters carried in the code stream can be changed, the encoder can adaptively adjust the entropy coding parameters by changing the parameters carried in the code stream. Therefore, under high bit rate conditions, the limiting effect can be avoided, and the rate overhead caused by the unreasonable large alphabet size under low bit rate conditions can also be avoided. In other words, due to the adaptability of the entropy decoding parameters (specifically the alphabet size), the best operation of the entropy decoder may be performed at a low bit rate (corresponding to a narrow range of decoded values), which saves bit rate; and there is no limiting effect at a high bit rate (corresponding to a wide range of decoded values), which improves the quality of the reconstructed signal.

[0010] It should be noted here that "entropy decoder" can be used as a synonym for "entropy decoding algorithm", including encoding algorithms and decoding algorithms. The entropy encoder can be a module that is part of the encoder; the entropy decoder can be another module that is part of the decoder. The parameters of the entropy decoder and the entropy decoder should be synchronized in order to work correctly, so the term "entropy decoder parameters" or "entropy decoding parameters" refers to the parameters of both the entropy encoder and the entropy decoder. In other words, "entropy decoding parameters" can be equivalent to "entropy encoder and entropy decoder parameters". The entropy encoder encodes the symbols of the alphabet into one or more bits in the code stream, and the entropy decoder decodes one or more bits in the code stream into symbols of the alphabet. On the entropy encoder side, the alphabet represents the input alphabet, and on the entropy decoder side, the alphabet represents the output alphabet. The size of the input alphabet on the entropy encoder side is equal to the size of the output alphabet on the entropy decoder side.

[0011] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vectors or any other type of media data.

[0012] In one possible embodiment, the entropy decoding parameter includes at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0013] Three possible modes of derivation of the alphabet size based on the codestream are considered:

[0014] (1) Explicit indication with predefined prediction value;

[0015] (2) Derived from the quantization parameter (β);

[0016] (3) having an explicit indication of the prediction value that depends on the quantization parameter;

[0017] The technique can be applied to any type of decoder that uses entropy decoding in a pipeline.

[0018] In a possible embodiment, the first parameter is the size of the alphabet; wherein obtaining the entropy decoding parameter according to the first parameter includes: using the first parameter as the size of the alphabet.

[0019] In this embodiment, the alphabet size is indicated directly in the code stream, for example using fixed length decoding, exp-Golomb decoding or some other decoding algorithm. Typical values ​​of M can be 256, 512, 1024. For example, for an indication of 1024 with a fixed length code, 11 bits (1024 10=100000000002), for the indication log2(1024)–9=1, only 1 bit is needed if only values ​​512 and 1024 are supported, and 2 bits are needed if 4 different alphabet sizes are supported, such as 512, 1024, 2048, 4096. Therefore, direct indication of M will use more bits. But for some exotic cases (such as the alphabet size M is not a power of 2), direct indication of M may help.

[0020] In one possible embodiment, the first parameter is p, ie, an entropy decoding parameter, including the size of the alphabet M, where M is a function of p.

[0021] In a possible embodiment, obtaining the entropy decoding parameter according to the first parameter includes: M=f –1 (p); where f –1 (p) is the inverse function of f(M), f(M)=p.

[0022] In this embodiment, the output p of some reversible function f(M) is indicated in the code stream instead of M itself. This p can be indicated by fixed length decoding, exp-Golomb decoding or some other decoding algorithm. Therefore, at the decoder side, M is derived from p. Specifically, M is derived from M=f –1 (p) is derived. The benefit of the above embodiment is that any optimal alphabet size selected on the encoder side can be indicated, so the flexibility of indicating the alphabet size is increased. In some embodiments, p is greater than or equal to 0, but in other embodiments, p can also be negative. For example, the value p can be in the range of [0, 5] and indicated using 3 bits. The function f(M) can be negotiated in advance between the encoder side and the decoder side.

[0023] In a possible embodiment, M satisfies one of the following: M = k^p, where k is a natural number; or M = k^(p+C), where k is a natural number and C is an integer; or M = k^(a*p+C), where k is a natural number and a and C are constants; or M = a*p+b, where a and b are constants; or M = p^2. It should be noted that in any embodiment, A^B means A B .

[0024] In one possible embodiment, p = log2(M) - 9, M = f –1 (p) = 2^(p+9), where f –1 (p) is the inverse function of f(M), where f(M) = log2(M)–9.

[0025] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0026] In one possible embodiment, p is indicated using a 0th order exp-Golomb code.

[0027] In one possible embodiment, the alphabet size is indicated in a parameter set part or the like in a code stream, for example, in a picture parameter set part of the code stream.

[0028] In a possible embodiment, the first parameter includes at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor.

[0029] In the above embodiments, the alphabet size can be derived from some other parameters. In an exemplary implementation, the alphabet size is derived from a quantization parameter or a rate control parameter, or the alphabet size can be derived from image resolution, video resolution, frame rate, density of pixels in a 3D object, etc. In a trainable codec, the alphabet size can be derived from some parameters of the loss function used during training, such as a weighting factor for rate / distortion, or some parameters that affect the selection of the gain vector g. The loss function may include rate and distortion components, such as peak signal-to-noise ratio (PSNR), multi-scale structural similarity index (MS-SSIM), video multimethod assessment fusion (VMAF) or some other quality indicators. For example, the loss function can be: loss = β * distortion + bit number, where the distortion is measured by PSNR or MS-SSIM or VMAF, the bit number is the number of bits used, and β is a weighting parameter that controls the ratio between bit rate and reconstruction quality, which can also be called a rate control parameter. It may also be a quantization parameter, such as a quantization parameter (QP) in conventional codecs (such as JPEG, HEVC, VVC).

[0030] The benefit of the above embodiment is that, since the quantization parameters or rate control parameters already exist in the bitstream and are used for other processes, the decoder side can use these parameters to derive the alphabet size M, so there is no need for additional indication of information specifically used to indicate the alphabet size M, thereby saving bit rate.

[0031] In a possible embodiment, obtaining an entropy decoding parameter according to a first parameter includes: determining a target sub-range in which the first parameter is located, wherein the allowable range of values ​​of the first parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the first parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; using the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculating the value of the entropy decoding parameter based on one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0032] In the above embodiment, the rate control parameter is represented as β, and the range of beta (β) is divided into K intervals (K sub-ranges), as shown below:

[0033] [β_0, β_1), [β_1, β_2)……[β_(K–1), β_K)

[0034] Each interval / sub-range corresponds to an alphabet size value Mi. It should be noted that for a specific codec value β, there will be an allowed range; for example, for some codec β, it may be allowed in the range [–∞, ∞], and for other codec β, it may only be allowed in the range [0, ∞]. In the context of this embodiment, the original large range of allowed β values ​​is divided into several sub-ranges, and for each sub-range, there is a specific value of the alphabet size. After obtaining the parameter β, the decoder can select a target interval based on the β value obtained from the bitstream. Specifically, the decoder determines that β_i≤β≤β_(i+1), then selects the interval [β_i,β_(i+1)] as the target interval, and the decoder side derives the alphabet size value Mi corresponding to the target interval as the alphabet size value M. In some embodiments, each βi in the β range {βi} may correspond to an alphabet size value Mi, and the alphabet size value M corresponding to a specific β is calculated based on one or more values ​​Mi corresponding to βi adjacent to β. It should be noted that the value used to calculate M can be the timely value Mi of the nearest neighbor corresponding to the target interval, or it can be linear or bilinear or some other interpolation from two or more Mi corresponding to βi adjacent to β, or some other interpolation from two or more Mi corresponding to intervals adjacent to the target interval.

[0035] In one possible embodiment, the first parameter is D, and the entropy decoding parameter includes the size of the alphabet M, where M is obtained based on P and D, where P is a prediction value that can be derived by the decoder.

[0036] In the above embodiment, the alphabet size can be derived based on the predicted value P and the first parameter indicated in the code stream. Therefore, when the code stream is received, the decoder derives the predicted value P based on the predefined parameters and parses the first parameter from the code stream, and then the alphabet size M can be derived based on the predicted value P and the first parameter. The benefit of the above embodiment is that since only the difference between P and M is indicated in the code stream, the additional bits used are reduced compared to indicating M in the code stream. In addition, the difference between P and M can be selected based on content or bit rate, and the flexibility of indicating the alphabet size is also increased. Therefore, this embodiment provides flexibility in alphabet size selection, and the additional bits used for indication are minimal. In some rare cases, when the alphabet size predicted from β does not work well, the encoder can still indicate the difference between M and P. This will use some bits, but can solve the serious problem of clipping effect.

[0037] In a possible embodiment, obtaining the entropy decoding parameter according to the first parameter includes: M=s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P), s(M,P)=D.

[0038] In one possible embodiment, s(M,P) includes: s(M,P)=log k (M)–log k (P), where k is a natural number; or s(M,P) = log k (P)–log k (M), where k is a natural number; or

[0039] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or s(M,P)=a*log k (P)–b*log k (M)–c, where k is a natural number and a, b, and c are constants; or s(M,P)=a*M+b*P+c, where a, b, and c are constants.

[0040] In one possible embodiment, M=2^(D+log2(P)), D=s(M,P)=log2(M)−log2(P).

[0041] It should be noted that the reversible function D = s(M, P) can be considered as D = s P (M), M = s –1(D,P) can be considered as M=s –1 P (D), where P can be any fixed number, or in other words, P is a constant coefficient.

[0042] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0043] In one possible embodiment, D is indicated using a 0th order exp-Golomb code.

[0044] In a possible embodiment, P may be derived according to at least one parameter other than the first parameter carried in the code stream.

[0045] In a possible embodiment, at least one parameter other than the first parameter includes at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor.

[0046] In one possible embodiment, P is derived based on at least one parameter, including: obtaining a rate control parameter beta (β) from a bitstream; determining a target subrange in which the obtained β is located; wherein the allowable range of the value of the rate control parameter β is [β_0, β_K], the allowable range [β_0, β_K] includes multiple subranges, and the target subrange is one of the multiple subranges; each of the multiple subranges includes at least one β value, and each of the multiple subranges corresponds to a P value; selecting a value corresponding to the target subrange as the value of P; or calculating the value of P based on one or more values ​​corresponding to one or more subranges adjacent to the target subrange.

[0047] In a possible embodiment, the decoding method further includes: parsing the bitstream to obtain a flag, wherein the flag is used to indicate whether the bitstream directly carries the entropy decoding parameter.

[0048] In the above embodiment, a flag may be introduced into the code stream to indicate switching between the three embodiments, in which case the flag may require two bits. In another possible embodiment, a flag may be used to indicate switching between two embodiments, in which case only one bit is required.

[0049] In a possible embodiment, when the flag is equal to the first value, it indicates that the bitstream carries entropy decoding parameters, in which case the first parameter is the entropy decoding parameter, or the first parameter is a transformation result of the entropy decoding parameter; or when the flag is equal to the second value, it indicates that the bitstream does not carry entropy decoding parameters, and the entropy decoding parameters can be derived by the decoder.

[0050] This scheme provides a balance between bit saving and flexibility: in most cases, where a derived entropy parameter is appropriate, only one bit is used to indicate this, and in some specific cases it is possible to explicitly indicate the entropy parameter from the other side.

[0051] In one possible embodiment, when the flag is equal to the third value, it indicates that the code stream carries the difference between M and P or the transformation result of the difference between M and P. In this case, the first parameter is the difference between M and P or the transformation result of the difference between M and P, where M is the size of the input alphabet and P is the predicted value that the decoder can derive.

[0052] In a possible embodiment, the entropy decoder is an arithmetic decoder, or a range decoder, or an asymmetric numerical system (ANS) decoder.

[0053] In one possible embodiment, reconstructing at least a portion of an input signal based on entropy decoding parameters includes: obtaining at least one probability model, wherein the probability model of an output symbol is used to indicate the probability of each possible value of the output symbol; performing entropy decoding on one or more bits in a code stream by using at least one probability model and entropy decoding parameters to obtain one or more output symbols; and reconstructing at least a portion of the input signal based on the one or more output symbols.

[0054] In one possible embodiment, the method further includes: updating the probability model. For example, the probability model is updated after each output symbol; therefore, each output symbol has its own probability distribution of possible values. It should be noted that the probability model can also be called a probability distribution.

[0055] In one possible embodiment, the probability model depends on the entropy decoding parameters. For example, the symbol probability is distributed according to the normal distribution N(μ,σ), where N(μ,σ) means that the mean is equal to μ and the variance is equal to σ 2Gaussian distribution. However, the actual probability model (also referred to as the mathematical model or theoretical model), such as the quantization histogram, depends on the alphabet size and probability precision in the entropy decoding engine or entropy decoder. That is, the entropy decoding parameters may affect the histogram structure inside the entropy decoder. Basically, the alphabet size is the number of possible symbol values, so if (for example) the alphabet size is equal to 4, larger values ​​(such as the value "7") cannot be encoded / decoded. The histogram used in the entropy decoder consists of the quantized probability of each symbol value: for example, the alphabet is {0, 1, 2, 3}, and the corresponding probabilities are {7 / 16, 7 / 16, 1 / 16, 1 / 16}, each probability is non-zero, and the sum of the probabilities is equal to 1; moreover, each probability is greater than the minimum probability (probability precision) supported by the entropy decoding engine (in this example, 1 / 16). If the probability of some symbols is lower than the minimum probability supported by the entropy decoding engine, the probability of at least some symbols needs to be adjusted to ensure that the probability of each symbol is greater than the minimum probability supported by the entropy decoding engine.

[0056] According to a second aspect, an embodiment of the present application provides a decoding method for entropy decoding a bit stream, the method comprising: receiving a bit stream including decoded data of an input signal; parsing the bit stream to obtain a flag, the flag being used to indicate whether the bit stream carries an entropy decoding parameter; obtaining the entropy decoding parameter according to the flag; and reconstructing at least a portion of the input signal according to the entropy decoding parameter.

[0057] In the above embodiment, a flag may be introduced into the code stream to indicate switching between the three embodiments, in which case the flag may require two bits. In another possible embodiment, a flag may be used to indicate switching between two embodiments, in which case only one bit is required. This scheme provides a balance between bit saving and flexibility: in most cases, only one bit is used for indication when the derived entropy parameter is appropriate, and in some specific cases, it is possible to explicitly indicate the entropy parameter from the other side.

[0058] In one possible embodiment, the entropy decoding parameters include at least one of: the size of the alphabet of the entropy decoder, the size of the alphabet being: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder.

[0059] In a possible embodiment, when the flag is equal to a first value, it indicates that the bitstream carries entropy decoding parameters, or the bitstream carries a transformation result of entropy decoding parameters; or when the flag is equal to a second value, it indicates that the bitstream does not carry entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder.

[0060] In a possible embodiment, when the flag is equal to the third value, it indicates that the bitstream carries the difference between M and P, or the bitstream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder.

[0061] In a possible embodiment, obtaining an entropy decoding parameter according to a flag includes: when the flag is equal to a first value, parsing a bitstream to obtain a first parameter; when the first parameter is an entropy decoding parameter, using the first parameter as the entropy decoding parameter; or when the first parameter is a transformation result of the entropy decoding parameter, obtaining the entropy decoding parameter according to the first parameter.

[0062] In a possible embodiment, the transformation result of the entropy decoding parameter is p=f(M), where M is the entropy decoding parameter, and f(M) includes: f(M)=log k (M), k is a natural number; or f(M) = a*log k (M)–C, k is a natural number, a and C are predefined constants; or f(M)=a*M+R, where a and R are predefined constants; or f(M)=sqrt(M); wherein obtaining the entropy decoding parameter according to the first parameter includes: M=f –1 (p); where f –1 (p) is the inverse function of f(M).

[0063] In a possible embodiment, the first parameter is p=log2(M)−9.

[0064] In a possible embodiment, obtaining entropy decoding parameters according to a flag includes: when the flag is equal to a second value, parsing the bitstream to obtain a second parameter, the second parameter including at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; and deriving the entropy decoding parameters according to the second parameter.

[0065] In a possible embodiment, deriving an entropy decoding parameter based on a second parameter includes: determining a target sub-range in which the second parameter is located; wherein the allowable range of values ​​of the second parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the second parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; using the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculating the value of the entropy decoding parameter based on one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0066] In a possible embodiment, obtaining entropy decoding parameters according to a flag includes: when the flag is equal to a third value, parsing the bitstream to obtain a third parameter, wherein the third parameter is a difference between M and P, or the third parameter is a transformation result of the difference between M and P, wherein M is an entropy decoding parameter, and P is a prediction value that can be derived by a decoder; deriving P according to at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; and obtaining the entropy decoding parameters according to the third parameter and P.

[0067] In a possible embodiment, the transformation result of the difference between M and P is D=s(M, P), where s(M, P) is a reversible function; wherein s(M, P) includes:

[0068] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0069] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0070] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0071] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0072] s(M,P)=a*log k (P)–b*log k (M)–c, where k is a natural number and a, b, and c are constants; or

[0073] s(M,P)=a*M+b*P+c, where a, b and c are constants;

[0074] Wherein, the entropy decoding parameter is obtained according to the third parameter, including: M = s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P).

[0075] According to a third aspect, an embodiment of the present application provides an encoding method implemented by an encoder, the method comprising: encoding an input signal and a first parameter into a bit stream, wherein the first parameter is used to obtain an entropy decoding parameter; and sending the bit stream to a decoder.

[0076] In an embodiment of the present application, the decoder can obtain entropy decoding parameters (specifically, the alphabet size) according to the parameters carried in the code stream. Since the parameters carried in the code stream can be changed, the encoder can adaptively adjust the entropy coding parameters by changing the parameters carried in the code stream. Therefore, under high bit rate conditions, the limiting effect can be avoided, and the rate overhead caused by the unreasonable large alphabet size under low bit rate conditions can also be avoided. In other words, due to the adaptability of the entropy decoding parameters (specifically the alphabet size), the best operation of the entropy decoder may be performed at a low bit rate (corresponding to a narrow range of decoded values), which saves bit rate; and there is no limiting effect at a high bit rate (corresponding to a wide range of decoded values), which improves the quality of the reconstructed signal.

[0077] In one possible embodiment, the entropy decoding parameters include at least one of: the size of the alphabet of the entropy decoder, the size of the alphabet being: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder.

[0078] In one possible embodiment, the first parameter is the size of the alphabet.

[0079] In a possible embodiment, the first parameter is p, p is a transformation result of M, and M is an entropy decoding parameter.

[0080] In a possible embodiment, p=f(M), where f(M) is a reversible function.

[0081] In one possible embodiment, f(M) includes:

[0082] f(M)=a*log k (M)–C, where k is a natural number, and a and C are predefined constants; or

[0083] f(M) = a*M+b, where a and b are constants; or

[0084] f(M)=sqrt(M).

[0085] In one possible embodiment, p=log2(M)−9.

[0086] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0087] In one possible embodiment, the first parameter includes at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor, wherein the first parameter is used by an entropy decoder to derive an entropy decoding parameter.

[0088] In a possible embodiment, the first parameter is D obtained according to P and M, wherein M is an entropy decoding parameter and P is a prediction value that can be derived by a decoder.

[0089] In a possible embodiment, D=s(M, P), where s(M, P) is a reversible function.

[0090] In one possible embodiment, s(M,P) includes:

[0091] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0092] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0093] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0094] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0095] s(M,P)=a*log k (P)–b*log k (M)–c, where k is a natural number and a, b, and c are constants; or

[0096] s(M,P)=a*M+b*P+c, where a, b and c are constants.

[0097] In one possible embodiment, D=s(M,P)=log2(P)−log2(M).

[0098] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0099] In a possible embodiment, the encoding method further includes: encoding a flag into the bitstream, wherein the flag is used to indicate whether the bitstream directly carries the entropy decoding parameter.

[0100] In a possible embodiment, when the flag is equal to a first value, it indicates that the bitstream carries entropy decoding parameters, the first parameter is the entropy decoding parameter, or the first parameter is a transformation result of the entropy decoding parameter; or when the flag is equal to a second value, it indicates that the bitstream does not carry entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder.

[0101] In a possible embodiment, when the flag is equal to the third value, it indicates that the bitstream carries the difference between M and P, or the bitstream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder.

[0102] In one possible embodiment, several possible schemes are provided for the alphabet selection at the encoder side.

[0103] In one possible embodiment, the method further includes: obtaining a minimum value and a maximum value of a potential space element of an entropy encoder; wherein the potential space element is a result of input signal processing; and obtaining a size of the alphabet, as shown below:

[0104] M=ceil(max{y}–min{y}) or M=2^(ceil(log2(max{y}–min{y}))),

[0105] Where ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, min{y} represents the minimum value of the latent space element, and M represents the size of the alphabet.

[0106] In this embodiment, the alphabet size is selected to be larger than the smallest possible number of the decoded value range. For example, first obtain the minimum and maximum values ​​of the tensor y, and the alphabet size is selected as follows:

[0107] M = ceil(max{y}–min{y}),

[0108] For most entropy decoders, the alphabet size should be a power of 2, in which case the alphabet size can be chosen to be M = 2^(ceil(log2(max{y}–min{y}))). It should be noted that in some cases, for example, when the modules of all y values ​​are less than 1, additional scaling operations can be performed before entropy decoding.

[0109] In one possible embodiment, the method further includes: obtaining at least two values ​​near M0, wherein M0=ceil(max{y}–min{y}) or M0=2^(ceil(log2(max{y}–min{y}))); calculating the loss function of at least two values; selecting the value with the smallest loss function among the at least two values ​​as the size of the alphabet; wherein ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, and min{y} represents the minimum value of the latent space element.

[0110] The loss function may include rate and distortion components, for example, the loss function may be as follows: Loss = β * distortion + number of bits, where distortion is the number of bits used, measured using peak signal-to-noise ratio (PSNR), multi-scale structural similarity index (MS-SSIM), video multimethod assessment fusion (VMAF) or some other quality metric, number of bits is the number of bits used, and β is a weighting parameter that controls the ratio between bit rate and reconstruction quality, β may also be referred to as a rate control parameter. In this approach, clipping sometimes occurs, but the bit rate saved due to the use of a smaller alphabet compensates for the slight increase in distortion.

[0111] According to the fourth aspect, an embodiment of the present application provides an encoding method implemented by an encoder, the method comprising: encoding an input signal and a flag into a bit stream, wherein the flag is used to indicate whether the bit stream directly carries entropy decoding parameters; and sending the bit stream to a decoder.

[0112] In the above embodiment, a flag may be introduced into the code stream to indicate switching between the three embodiments, in which case the flag may require two bits. In another possible embodiment, a flag may be used to indicate switching between two embodiments, in which case only one bit is required. This scheme provides a balance between bit saving and flexibility: in most cases, only one bit is used for indication when the derived entropy parameter is appropriate, and in some specific cases, it is possible to explicitly indicate the entropy parameter from the other side.

[0113] In one possible embodiment, the entropy decoding parameters include at least one of: the size of the alphabet of the entropy decoder, the size of the alphabet being: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder.

[0114] In a possible embodiment, when the flag is equal to a first value, it indicates that the bitstream carries entropy decoding parameters, or the bitstream carries a transformation result of entropy decoding parameters; or when the flag is equal to a second value, it indicates that the bitstream does not carry entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder.

[0115] In a possible embodiment, when the flag is equal to the third value, it indicates that the bitstream carries the difference between M and P, or the bitstream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder.

[0116] In a possible embodiment, the method further includes: when the flag is equal to the first value, encoding a first parameter into a bitstream, wherein the first parameter is an entropy decoding parameter or the first parameter is a transformation result of the entropy decoding parameter.

[0117] In a possible embodiment, the transformation result of the entropy decoding parameter is p=f(M), where M is the entropy decoding parameter, and f(M) may be as follows:

[0118] f(M)=log k (M), where k is a natural number; or

[0119] f(M)=a*log k (M)–C, where k is a natural number, and a and C are predefined constants; or

[0120] f(M) = aM + b, where a and b are constants; or

[0121] f(M)=sqrt(M).

[0122] In a possible embodiment, the first parameter is p=log2(M)−9.

[0123] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0124] In one possible embodiment, p is indicated using a 0th order exp-Golomb code.

[0125] In one possible embodiment, the method further includes: when the flag is equal to a third value, encoding a third parameter into the bitstream, wherein the third parameter is the difference between M and P, or the third parameter is a transformation result of the difference between M and P, wherein M is an entropy decoding parameter and P is a predicted value that can be derived by the decoder.

[0126] In a possible embodiment, the transformation result of the difference between M and P is D=s(M, P), where s(M, P) is a reversible function; where s(M, P) includes:

[0127] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0128] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0129] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0130] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0131] s(M,P)=a*log k (P)–b*log k (M)–c, where k is a natural number and a, b, and c are constants; or

[0132] s(M,P)=a*M–b*P+c, where a, b and c are constants;

[0133] Wherein, the entropy decoding parameter is obtained according to the third parameter, including: M = s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P).

[0134] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0135] In one possible embodiment, D is indicated using a 0th order exp-Golomb code.

[0136] According to the fifth aspect, an embodiment of the present application provides a decoding device, including: a receiving unit, used to: receive a code stream including encoded data of an input signal; a parsing unit, used to: parse the code stream to obtain a first parameter; an acquisition unit, used to: obtain an entropy decoding parameter based on the first parameter; a reconstruction unit, used to: reconstruct at least a part of the input signal based on the entropy decoding parameter.

[0137] These devices provide the advantages of the methods described above.

[0138] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vectors or any other type of media data.

[0139] In one possible embodiment, the entropy decoding parameter includes at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0140] In a possible embodiment, the first parameter is the size of the alphabet; wherein the acquisition unit is further configured to: use the first parameter as the size of the alphabet.

[0141] In one possible embodiment, the first parameter is p, ie, an entropy decoding parameter, including the size of the alphabet M, where M is a function of p.

[0142] In a possible embodiment, the acquisition unit is further used to: acquire M, M=f –1 (p); where f –1 (p) is the inverse function of f(M), where f(M)=p.

[0143] In a possible embodiment, the acquisition unit is further used to: determine a target sub-range in which the first parameter is located, wherein the allowable range of values ​​of the first parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the first parameter, and each of the multiple sub-ranges corresponds to a value of an entropy decoding parameter; use the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculate the value of the entropy decoding parameter based on one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0144] According to the sixth aspect, an embodiment of the present application provides a decoding device, including: a functional unit that implements the encoding method of the second aspect or any possible embodiment of the second aspect.

[0145] These devices provide the advantages of the methods described above.

[0146] According to the seventh aspect, an embodiment of the present application provides a coding device, comprising: a coding unit, configured to encode an input signal and a first parameter into a bitstream, wherein the first parameter is used to obtain an entropy decoding parameter; and a sending unit, configured to send the bitstream to a decoder. The coding device also includes other functional units, configured to implement the coding method in any possible embodiment of the third aspect.

[0147] According to the eighth aspect, an embodiment of the present application provides a coding device, comprising: a coding unit, configured to: encode an input signal and a flag into a bitstream; the flag is used to indicate whether the bitstream directly carries entropy decoding parameters; and a sending unit, configured to send the bitstream to a decoder. The coding device also includes other functional units, configured to implement the coding method in any possible embodiment of the fourth aspect.

[0148] According to the ninth aspect, an embodiment of the present application provides a decoding device, including: a processing circuit, used to execute the decoding method described in the first aspect or any possible embodiment of the first aspect.

[0149] According to the tenth aspect, an embodiment of the present application provides a decoding device, including: a processing circuit, used to execute the decoding method described in the second aspect or any possible embodiment of the second aspect.

[0150] According to the eleventh aspect, an embodiment of the present application provides a coding device, including: a processing circuit, used to execute the coding method described in the third aspect or any possible embodiment of the third aspect.

[0151] According to the twelfth aspect, an embodiment of the present application provides a coding device, including: a processing circuit, used to execute the coding method described in the fourth aspect or any possible embodiment of the fourth aspect.

[0152] According to the thirteenth aspect, an embodiment of the present application provides a decoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein, when the program is executed by the one or more processors, the program causes the decoder to execute the method described in the first aspect or any possible embodiment of the first aspect.

[0153] According to the fourteenth aspect, an embodiment of the present application provides a decoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein, when the program is executed by the one or more processors, the program causes the decoder to execute the method described in the second aspect or any possible embodiment of the second aspect.

[0154] According to the fifteenth aspect, an embodiment of the present application provides a codec, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein, when the program is executed by the one or more processors, the program causes the decoder to execute the method described in the third aspect or any possible embodiment of the third aspect.

[0155] According to the sixteenth aspect, an embodiment of the present application provides an encoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein, when the program is executed by the one or more processors, the program causes the decoder to execute the method described in the fourth aspect or any possible embodiment of the fourth aspect.

[0156] According to the seventeenth aspect, an embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions. When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute the method described in the first aspect or any possible embodiment of the first aspect.

[0157] According to the eighteenth aspect, an embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions. When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute the method described in the second aspect or any possible embodiment of the second aspect.

[0158] According to the nineteenth aspect, an embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions. When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute the method described in the third aspect or any possible embodiment of the third aspect.

[0159] According to the twelfth aspect, an embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions. When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute the method described in the fourth aspect or any possible embodiment of the fourth aspect.

[0160] According to the twenty-first aspect, an embodiment of the present application provides a non-transitory storage medium, comprising a code stream encoded by the method described in the third aspect or any possible embodiment of the third aspect.

[0161] According to the twenty-second aspect, an embodiment of the present application provides a non-transitory storage medium, comprising a code stream encoded by the method described in the fourth aspect or any possible embodiment of the fourth aspect.

[0162] According to the twenty-third aspect, an embodiment of the present application provides a computer program stored in a non-transitory medium and including code instructions. When the computer program is executed on one or more processors, the computer program causes one or more processors to perform the steps of the method described in any one of the above aspects or any possible embodiment of the above aspects.

[0163] According to the twenty-fourth aspect, an embodiment of the present application provides a system for sending a code stream, including: at least one storage medium, used to store at least one code stream generated by the encoding method described in the third aspect or any possible embodiment of the third aspect, the fourth aspect or any possible embodiment of the fourth aspect; a video streaming device, used to obtain the code stream from one storage medium in the at least one storage medium, and send the code stream to a terminal device, wherein the video streaming device includes a content server or a content distribution server.

[0164] In a possible embodiment, it also includes: one or more processors for performing encryption processing on at least one code stream to obtain at least one encrypted code stream; at least one storage medium for storing the encrypted code stream; or one or more processors for converting a code stream in a first format into a code stream in a second format; at least one storage medium for storing the code stream in the second format.

[0165] In a possible embodiment, it also includes: a receiver for receiving a first operation request; one or more processors for determining a target code stream in at least one storage medium in response to the first operation request; and a transmitter for sending the target code stream to a terminal side device.

[0166] In one possible embodiment, one or more processors are also used to: encapsulate the code stream to obtain a transmission stream in a first format; the transmitter is also used to: send the transmission stream in the first format to a terminal side device for display; or send the transmission stream in the first format to a storage space for storage.

[0167] The present invention may be implemented by hardware (HW) and / or software (SW) or any combination thereof. In addition, an implementation based on HW may be combined with an implementation based on SW.

[0168] One or more embodiments will be described in detail in the drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0169] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0170] Figure 1 A diagram of the channels processed by a neural network layer.

[0171] Figure 2 Schematic diagram of an autoencoder type of neural network.

[0172] Figure 3 Schematic diagram of an exemplary network architecture including the encoder and decoder sides of the hyper-prior model.

[0173] Figure 4 Schematic diagram of an exemplary network architecture including the encoder and decoder sides of the hyper-prior model.

[0174] Figure 5 A block diagram of the architecture of a cloud-based solution for machine-based tasks, such as machine vision tasks.

[0175] Fig. 6A Block diagram of the neural network-based end-to-end video compression framework.

[0176] Figure 6B A block diagram illustrating some exemplary details of a neural network application for motion field compression.

[0177] Figure 6C A block diagram illustrating some exemplary details of a neural network application for motion compensation.

[0178] Figure 7 Schematic diagram of the general scheme of the entropy decoder.

[0179] Figure 8 Schematic diagram of the general scheme using entropy decoding in an autoencoder based decoder.

[0180] Fig. 9 Schematic diagram of a general scheme of an autoencoder based decoder with an entropy decoder and a gain unit.

[0181] Fig.10 Schematic diagram of the rate-distortion curve showing an abnormal decrease in PSNR at high rates.

[0182] Fig.11 Schematic diagram of dividing the β range into intervals.

[0183] Fig.12 A schematic diagram of the decoding method.

[0184] Fig.13 A schematic diagram of the decoding method.

[0185] Fig.14 A schematic diagram of the encoding method.

[0186] Fig.15 A schematic diagram of an exemplary method for determining the alphabet size of an entropy encoder.

[0187] Fig.16 A schematic diagram of an exemplary method for determining the alphabet size of an entropy encoder.

[0188] Fig.17 A schematic diagram of an exemplary method for determining the alphabet size of an entropy encoder.

[0189] Fig.18 A schematic diagram of the encoding method.

[0190] Fig.19 Schematic diagram of encoding channels of input data into sub-streams and concatenating the sub-streams into a bitstream for a multi-core encoder.

[0191] Fig. 20 A schematic diagram of an exemplary encoder for implementing the technology of the present application.

[0192] Fig.21 A schematic diagram of an example of a decoder for implementing the technology of the present application.

[0193] Fig. 22 FIG. 4 is a schematic diagram of an example of a decoding system for implementing an embodiment of the present invention.

[0194] Fig.23 A schematic diagram of an example of an encoding device or a decoding device.

[0195] Fig.24 A schematic diagram of an example of a decoding device.

[0196] Fig.25 A schematic diagram of an example of a decoding system or an encoding device or a decoding device.

[0197] Fig.26 A schematic diagram of an exemplary structure of a content providing system 3100 that implements content distribution services.

[0198] Fig. 27 A schematic diagram of an exemplary structure of a terminal service. DETAILED DESCRIPTION

[0199] In the following description, reference is made to the accompanying drawings that form a part of the present invention, which illustrate specific aspects of embodiments of the present invention or specific aspects in which embodiments of the present invention may be used by way of illustration. It should be understood that embodiments of the present invention may be used in other aspects and include structural or logical changes not shown in the accompanying drawings. Therefore, the following detailed description is not intended to be limiting, and the scope of the present invention is defined by the appended claims.

[0200] For example, it should be understood that what is disclosed with reference to a described method may also be true for a corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units for performing the described one or more method steps (e.g., one unit that performs one or more steps; or multiple units, each unit performing one or more of the multiple steps), even if the one or more units are not explicitly described or shown in the drawings. In addition, for example, if a specific device is described in terms of one or more units (e.g., functional units), the corresponding method may include a step for performing the function of the one or more units (e.g., one step that performs the function of the one or more units, or multiple steps, each step performing the function of one or more of the multiple units), even if the one or more steps are not explicitly described or shown in the drawings. In addition, it should be understood that, unless otherwise specified, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0201] In the following, an overview of some used technical terms and frameworks in which embodiments of the present invention may be used is provided.

[0202] Video decoding generally refers to the processing of a sequence of images that form a video or video sequence. In the field of video decoding, the terms "frame" and "picture / image" can be used as synonyms. Video decoding includes two parts: video encoding and video decoding. Video encoding is performed at the source end and generally includes processing (e.g., by compression) the original video image to reduce the amount of data required to represent the video image (to obtain more efficient storage and / or transmission). Video decoding is performed at the destination end and generally includes inverse processing relative to the encoder to reconstruct the video image. The "decoding" of the video image (or generally referred to as an image, which will be explained below) involved in the embodiment should be understood as the "encoding" and "decoding" of the video image. The combination of the encoding part and the decoding part is also referred to as codec (encoding and decoding).

[0203] Artificial Neural Networks

[0204] An artificial neural network (ANN) or connectionist system is a computing system vaguely inspired by the biological neural networks that make up animal brains. These systems "learn" to perform tasks by considering examples, and are usually not programmed with task-specific rules. For example, in image recognition, these systems might learn to recognize images that include cats by analyzing example images that have been manually labeled as "cat" or "no cat" and using the results to recognize cats in other images. These systems do not have any prior knowledge about cats, such as whether these cats have fur, tails, whiskers, and cat-like faces. Instead, these systems automatically generate identifying features from the examples they are processed.

[0205] ANNs are based on a collection of connected units or nodes called artificial neurons, which loosely mimic neurons in biological brains. Each connection, like a synapse in a biological brain, can send signals to other neurons. An artificial neuron receives a signal, processes it, and can send signals to neurons connected to it.

[0206] In the ANN implementation, the "signals" at the connections are real numbers, and the output of each neuron is calculated as some nonlinear function of the sum of its inputs. These connections are called edges. Neurons and edges usually have weights that are adjusted as learning occurs. Weights can increase or decrease the strength of the signal on a connection. Neurons can have a threshold so that they are only sent if the aggregate signal exceeds that threshold. Typically, neurons are grouped into layers. Different layers can perform different transformations on their inputs. Signals travel from the first layer (input layer) to the last layer (output layer), possibly traversing the layers multiple times during this time.

[0207] The original goal of the ANN approach was to solve problems in the same way as the human brain. Over time, the focus shifted to performing specific tasks, leading to a deviation from biology. ANNs have been used for a variety of tasks, including computer vision, speech recognition, machine translation, social network filtering, board and video games, medical diagnosis, and even activities traditionally considered exclusive to humans, such as painting.

[0208] The name "convolutional neural network" (CNN) indicates that the network uses a mathematical operation called convolution. Convolution is a specialized linear operation. A convolutional network is a neural network that uses convolution instead of general matrix multiplication in at least one of its multiple layers.

[0209] Figure 1Schematic diagram showing the general concept of processing by a neural network such as a CNN. A convolutional neural network consists of an input layer, an output layer, and multiple hidden layers. The input layer is where the input (e.g. Figure 1 The hidden layers of a CNN are usually composed of a series of convolutional layers, which perform convolutions using multiplication or other dot products. The result of the layer is one or more feature maps ( Figure 1 Some or all layers may involve subsampling. As a result, feature maps may become smaller, such as Figure 1 As shown. The activation function in a CNN is typically a rectified linear unit (ReLU) layer, followed by additional convolutional (e.g. pooling) layers, fully connected layers, and normalization layers. These convolutions are called hidden layers because the inputs and outputs of these convolutions are masked by the activation function and the final convolution. By convention, the layers are colloquially called convolutions. Mathematically speaking, a convolution is technically a sliding dot product or cross correlation. This has important implications for the indexing in the matrix because the convolution affects how the weights are determined at a specific index point.

[0210] When programming a CNN to process images, such as Figure 1 As shown, the input is a tensor with the shape of (number of images) × (image width) × (image height) × (image depth). It should be known that the image depth can be composed of the channels of the image. After passing through the convolution layer, the image is abstracted into a feature map with the shape of (number of images) × (feature map width) × (feature map height) × (feature map channels). The convolution layer in the neural network should have the following properties: The convolution kernel is defined by width and height (hyperparameters). The number of input channels and output channels (hyperparameters). The depth of the convolution filter (input channel) should be equal to the number of channels (depth) of the input feature map.

[0211] The convolutional layer is the core building block of a CNN. The parameters of this layer consist of a set of learnable filters (the kernels mentioned above) that have a small receptive field but extend to the full depth of the input volume. During the forward pass, each filter is convolved over the width and height of the input volume, computing the dot product between the filter entries and the input and generating a 2D activation map for that filter. Thus, the network learns filters that activate when they detect certain specific types of features at a certain spatial location in the input.

[0212] Another important concept of CNN is pooling, which is a form of nonlinear downsampling. There are several nonlinear functions that can implement pooling, of which max pooling is the most common. Max pooling divides the input image into a set of non-overlapping rectangles and outputs the maximum value for each such sub-region. Pooling layers are used to gradually reduce the spatial size of the representation to reduce the number of parameters, memory footprint, and computation in the network, thereby also controlling overfitting. In CNN architectures, it is common to regularly insert pooling layers between consecutive convolutional layers. The pooling operation provides another form of translation invariance.

[0213] Pooling layers operate independently on each depth slice of the input and spatially align these depth slices. The most common form is a pooling layer with a filter of size 2×2, which applies a stride of 2 to each depth slice of the input, along both the width and height, and discards 75% of the activations. Due to the large reduction in the size of the representation, there has been a recent trend to use smaller filters or to discard pooling layers entirely. "Region of Interest" pooling (also known as ROI pooling) is a variant of max pooling where the output size is fixed and the input rectangle is a parameter. Pooling is an important component of convolutional neural networks and is used for object detection based on the Fast R-CNN architecture.

[0214] The above ReLU is short for Rectified Linear Unit, which applies a non-saturating activation function. ReLU sets negative values ​​to zero, effectively removing negative values ​​from the activation map. ReLU increases the nonlinear nature of the decision function and the overall network without affecting the receptive field of the convolutional layer.

[0215] After several convolutional and max pooling layers, high-level reasoning in a neural network is done with fully connected layers. Neurons in a fully connected layer have connections to all activations in the previous layer, as in a regular (non-convolutional) artificial neural network. Therefore, their activations can be calculated as an affine transformation, a matrix multiplication followed by a bias shift (a vector addition of a learned or fixed bias term).

[0216] The "loss layer" (including the calculation of the loss function) specifies how the training penalizes deviations between the predicted labels (outputs) and the true labels, and is usually the last layer of the neural network. A variety of loss functions suitable for different tasks can be used. Softmax loss is used to predict a single class among K mutually exclusive classes. Sigmoid cross entropy loss is used to predict K independent probability values ​​in [0, 1]. Euclidean loss is used to regress to real-valued labels.

[0217] In conclusion, Figure 1The data flow in a typical convolutional neural network is shown. First, the input image passes through a convolutional layer and is abstracted into a feature map including several channels, corresponding to multiple filters in the learnable filter set of the layer. The feature map is then subsampled using a pooling layer, etc., which reduces the dimension of each channel in the feature map. Next, the data reaches another convolutional layer, which can have a different number of output channels. As mentioned above, the number of input channels and output channels are hyperparameters of the layer. In order to establish the connection of the network, these parameters need to be synchronized between the two connected layers so that the number of input channels of the current layer should be equal to the number of output channels of the previous layer. For the first layer that processes the input data (such as an image), the number of input channels is usually equal to the number of channels represented by the data, such as 3 channels for RGB or YUV representation of images or videos, or 1 channel for grayscale image or video representation.

[0218] Autoencoders and Unsupervised Learning

[0219] An autoencoder is an artificial neural network that is used to learn efficient data decoding in an unsupervised manner. Figure 2 As shown. The purpose of an autoencoder is to learn a representation (encoding) of a set of data by training the network to ignore the signal "noise", usually for dimensionality reduction. In addition to the dimensionality reduction side, a reconstruction side is also learned, where the autoencoder tries to generate a representation as close to its original input as possible from the simplified encoding, hence the name. In the simplest case, given one hidden layer, the encoder stage of the autoencoder takes the input x and maps it to h

[0220] h=σ(Wx+b).

[0221] This image h is often called a code, latent variable, or latent representation. Here, σ is an element-wise activation function, such as a sigmoid function or a rectified linear unit. W is a weight matrix and b is a bias vector. The weights and biases are usually initialized randomly and then iteratively updated via backpropagation during training. Afterwards, the decoder stage of the autoencoder maps h to a reconstruction x′ of the same shape as x:

[0222] x′=σ′(W′h′+b′)

[0223] Here, σ′, W′, and b′ of the decoder may be independent of the corresponding σ, W, and b of the encoder.

[0224] Recent advances in the field of artificial neural networks, especially convolutional neural networks, have made researchers interested in applying neural network-based techniques to image and video compression tasks. For example, end-to-end optimized image compression using a variational autoencoder-based network was proposed.

[0225] Data compression is therefore considered a fundamental and intensively studied problem in engineering, usually with the goal of designing a code with minimum entropy for a given set of discrete data. This approach relies heavily on knowledge of the probabilistic structure of the data, and the problem is therefore closely related to probabilistic source modeling. However, since all practical codes must have finite entropy, continuous-valued data (such as a vector of image pixel intensities) must be quantized into a finite set of discrete values, which introduces errors.

[0226] In this case, the lossy compression problem, two competing costs must be balanced: the entropy of the discrete representation (rate) and the error introduced by quantization (distortion). Different compression applications, such as data storage or transmission over channels of limited capacity, require different trade-offs between rate and distortion.

[0227] For example, JPEG uses discrete cosine transform on pixel blocks, and JPEG 2000 uses multi-scale orthogonal wavelet decomposition. Typically, the three components of the transform coding method (transform, quantizer, and entropy coding) are optimized separately (usually through manual parameter tuning). Modern video compression standards (such as HEVC, VVC, and EVC) also use transform representations to decode the predicted residual signal. Several transforms such as discrete cosine transform (DCT), discrete sine transform (DST), and low frequency non-separable manually optimized transform (LFNST) are used for this purpose.

[0228] Variational Image Compression

[0229] The variational auto-encoder (VAE) framework can be considered as a nonlinear transform decoding model. Figure 3 For example, Figure 3 The VAE framework is shown: the encoder 101 maps the input image x to a latent representation (represented by y) through the function y=f(x). In the following, this latent representation may also be referred to as a part or point of the "latent space". The function f() is a transformation function that transforms the input signal x into a more compressible representation y. The quantizer 102 Transform the latent representation y into a quantized latent representation with (discrete) values Where Q represents the quantizer function. The entropy model or super encoder / decoder (also called super prior) 103 estimates the quantized latent representation distribution to obtain the minimum rate achievable by lossless entropy source decoding.

[0230] The latent space can be understood as a compressed representation of the data, where similar data points are closer in the latent space. The latent space is very useful for learning data features and finding simpler representations of the data for analysis. Quantized latent representation T and hyperprior 3 and side information The quantized latent representation is included in the bitstream 2 (binarized) using arithmetic encoding (AE). In addition, a decoder 104 is provided which transforms the quantized latent representation into a reconstructed image. Signal is an estimate of the input image x. We hope that x is as close as possible In other words, the reconstruction quality is as high as possible. However, The higher the similarity between x and x, the more side information is needed to send. Side information includes Figure 3 The bitstream 1 and bitstream 2 shown are generated by the encoder and sent to the decoder. Generally speaking, the greater the amount of side information, the higher the reconstruction quality. However, a large amount of side information means a low compression ratio. Therefore, Figure 3 One goal of the system described in is to balance the quality of reconstruction and the amount of side information transmitted in the bitstream.

[0231] exist Figure 3 , component AE 105 is an arithmetic coding module that converts the quantized latent representation Samples and side information The samples are converted into binary representation code stream 1. For example, Samples and The samples may include integers or floating point numbers. One purpose of the arithmetic coding module is to convert the sample values ​​into a binary digital string (through a binarization process) (which is then included in the codestream, which may include other parts corresponding to the encoded image or other side information).

[0232] Arithmetic decoding (AD) 106 is the process of restoring the binarization process, wherein the binarized numbers are converted back to sample values. Arithmetic decoding is provided by the arithmetic decoding module 106 .

[0233] It should be noted that the present invention is not limited to this specific framework. In addition, the present invention is not limited to image or video compression, and can also be applied to object detection, image generation and recognition systems.

[0234] exist Figure 3In this example, there are two subnets cascaded to each other. In this context, a subnet is a logical division between parts of the overall network. For example, in Figure 3 In FIG. 1 , modules 101 , 102 , 104 , 105 and 106 are referred to as the “encoder / decoder” subnet. The “encoder / decoder” subnet is responsible for encoding (generating) and decoding (parsing) the first bitstream “bitstream 1”. Figure 3 The second network in the embodiment includes modules 103, 108, 109, 110 and 107 and is called the "super encoder / decoder" subnetwork. The second subnetwork is responsible for generating the second bitstream "Bitstream 2".

[0235] The first subnet is responsible for:

[0236] Transform (101) the input image x to its latent representation y (which makes it easier to compress x),

[0237] Quantize the latent representation y (102) to the quantized latent representation

[0238] Arithmetic coding module 105 uses AE to compress the quantized latent representation To get the code stream "code stream 1",

[0239] · Parsing the code stream 1 via AD using the arithmetic decoding module 106, and

[0240] Reconstruct the image using the parsed data reconstruction (104)

[0241] The purpose of the second subnet is to obtain the statistical properties of the samples of "stream 1" (such as the mean, variance and correlation between the samples of stream 1) so that the first subnet can compress stream 1 more efficiently. The second subnet generates a second stream "stream 2", which includes information (such as the mean, variance and correlation between the samples of stream 1).

[0242] The second network includes an encoding part, which includes converting the quantized latent representation Transform (103) into side information z, and quantize the side information z into quantized side information And the quantized side information The encoding (e.g., binarization) (109) is performed into the bitstream 2. In this example, the binarization is performed by arithmetic encoding (AE). The decoding part of the second network includes arithmetic decoding (AD) 110, which transforms the input bitstream 2 into decoded quantized side information. Possibly The same as arithmetic coding and decoding operations are lossless compression methods. Then, the decoded quantized side information is transformed (107) into the decoded side information express The statistical properties of The mean of the samples, or the variance of the sample values, etc.). Then, the decoded latent representation is provided to the arithmetic encoder 105 and the arithmetic decoder 106 to control The probability model of .

[0243] Figure 3 An example of a VAE (Variational Autoencoder) is described, the details of which may vary in different implementations.

[0244] Most deep learning (DL) based image / video compression systems reduce the dimensionality of the signal before converting it into a binary number (bit). For example, in the VAE framework, the encoder is a nonlinear transformation that maps the input image x to y, where y has a smaller width and height than x. Since y has a smaller width and height, it is smaller in size, the dimensionality (size) of the signal is reduced, and therefore, it is easier to compress the signal y. It should be noted that, in general, the encoder does not necessarily need to reduce the size in two (or usually all) dimensions. Instead, some exemplary implementations may provide an encoder that reduces the size in only one dimension (or usually a subset).

[0245] Such an example of a VAE framework is Figure 4 As shown, it uses 6 downsampling layers, labeled 401 to 406. The network architecture includes a hyper-prior model. a , g s ) shows the image autoencoder architecture, the right side (h a ,h s ) corresponds to the autoencoder that implements the hyper-prior. The decomposed prior model uses the same architecture for both analysis and synthesis transformations g a and g s Q represents quantization, AE and AD represent arithmetic encoder and arithmetic decoder respectively. The encoder puts the input image x into g a , producing a response y (latent representation) with spatially varying standard deviation. Encoding g a It consists of multiple convolutional layers with subsampling and generalized divisive normalization (GDN) as the activation function.

[0246] The response is fed to h a, thereby summarizing the standard deviation distribution in z. Then z is quantized, compressed, and sent as side information. The encoder then uses the quantized vector To estimate That is, the spatial distribution of the standard deviation (used to obtain the probability value (or frequency value) of arithmetic encoding (AE)), and use it to compress and send the quantized image representation (or latent representation). The decoder first recovers the Then, it uses h s Get This provides it with the correct probability estimate to successfully recover Then, it will Feed to g s to obtain the reconstructed image.

[0247] In the layer descriptions, layers that include downsampling are indicated by a downward-pointing arrow. The description of a layer "Conv N×5×5 / 2↓" means that the layer is a convolutional layer with N channels and a kernel size of 5×5. As mentioned above, 2↓ means that downsampling by a factor of 2 is performed in this layer. Downsampling by a factor of 2 causes one of the dimensions of the input signal to be reduced by half at the output. Figure 4 In the example, 2↓ indicates that the width and height of the input image are both reduced by half. Since there are 6 downsampling layers, if the width and height of the input image 414 (also represented by x) are w and h, the output signal The width and height of are equal to w / 64 and h / 64 respectively. The modules denoted by AE and AD are the arithmetic encoder and arithmetic decoder, refer to Figure 3 Explain. Arithmetic encoder and arithmetic decoder are specific implementations of entropy decoding. AE and AD can be replaced by other entropy decoding methods. In information theory, entropy coding is a lossless data compression scheme for converting the value of a symbol into a binary representation, which is a recoverable process. In addition, the "Q" in the figure corresponds to a quantization operation, and the quantization operation and the corresponding quantization unit as part of component 413 or 415 do not necessarily exist and / or can be replaced by another unit.

[0248] Cloud solutions for machine tasks

[0249] Video coding for machine (VCM) is another popular direction in computer science today. The main idea behind this approach is to send a decoded representation of image or video information for further processing by computer vision (CV) algorithms, such as object segmentation, detection, and recognition. In contrast to traditional image and video decoding for human perception, the quality features are the performance of computer vision tasks, such as object detection accuracy, rather than reconstruction quality. Figure 5 This is shown.

[0250] Machine video decoding, also known as collaborative intelligence, is a relatively new paradigm for efficiently deploying deep neural networks in mobile cloud infrastructure. By dividing the network between mobile devices and the cloud, the computing workload can be distributed so that the total energy and / or latency of the system is minimized. In general, collaborative intelligence is a paradigm in which the processing of a neural network is distributed between two or more different computing nodes (e.g., devices, but generally any functionally defined nodes). Here, the term "node" does not refer to the neural network node described above. Instead, the (computing) node here refers to a (physically or at least logically) independent device / module that implements each part of the neural network. These devices can be different servers, different end-user devices, different smart vehicles or on-board devices, a combination of servers and / or user devices and / or clouds and / or processors, etc. In other words, computing nodes can be considered to be nodes that belong to the same neural network and communicate with each other to transmit decoded data within / for the neural network. For example, in order to be able to perform complex calculations, one or more layers can be executed on a first device and one or more layers can be executed on another device. However, the distribution can also be more refined, and a single layer can be executed on multiple devices. In the present invention, the term "multiple" refers to two or more. In some prior art solutions, part of the neural network function is executed in a device (user device or edge device, etc.) or multiple such devices, and then the output (feature map) is passed to the cloud. The cloud is a collection of processing systems or computing systems located outside the device that runs part of the neural network. The concept of collaborative intelligence is also extended to model training. In this case, data flows in both directions: from the cloud to the mobile during backpropagation in training and inference, and from the mobile to the cloud during forward propagation in training and inference.

[0251] Some works propose semantic image compression: deep features are encoded and then the input image is reconstructed from these features. Compression based on uniform quantization is shown, followed by context-based adaptive arithmetic coding (CABAC) of H.264. In some scenarios, it may be more efficient to send the output of the hidden layer (deep feature map) from the mobile part to the cloud, rather than sending compressed natural image data to the cloud and performing object detection using the reconstructed image. Efficient compression of feature maps is beneficial for image and video compression and reconstruction, both for human perception and machine vision. Entropy decoding methods such as arithmetic decoding are popular methods for compression of deep features (i.e., feature maps).

[0252] Today, video content contributes more than 80% of Internet traffic, and this proportion is expected to rise. Therefore, it is crucial to build an efficient video compression system and generate higher quality frames under a given bandwidth budget. In addition, most video-related computer vision tasks (such as video object detection or video object tracking) are sensitive to the quality of compressed video, and efficient video compression may bring benefits to other computer vision tasks. At the same time, video compression technology also helps action recognition and model compression.

[0253] End-to-end image or video compression

[0254] DNN-based image compression methods can take advantage of large-scale end-to-end training and highly nonlinear transformations, which are not used in traditional methods. However, it is not easy to directly apply these techniques to build an end-to-end learning system for video compression. First, learning how to generate and compress motion information tailored for video compression remains an open problem. Video compression methods heavily rely on motion information to reduce temporal redundancy in video sequences.

[0255] A simple solution is to use learning-based optical flow to represent motion information. However, current learning-based optical flow methods aim to generate flow fields as accurately as possible. Accurate optical flow is often not the best choice for a specific video task. In addition, compared with the motion information in traditional compression systems, the amount of optical flow data increases significantly, and directly applying existing compression methods to compress optical flow values ​​will significantly increase the number of bits required to store motion information. Secondly, it is not yet known how to build a DNN-based video compression system by minimizing the rate-distortion-based residual and motion information objectives. The purpose of rate-distortion optimization (RDO) is to achieve higher quality (i.e., less distortion) of the reconstructed frame when the number of bits (or bitrate) of compression is given. RDO is very important for video compression performance. In order to take advantage of the end-to-end training capabilities of learning-based compression systems, RDO strategies are needed to optimize the entire system.

[0256] In "DVC: An End-to-end Deep Video Compression Framework" by Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, Zhiyong Gao (Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 11006–11015), the authors propose an end-to-end deep video compression (DVC) model that jointly learns motion estimation, motion compression, and residual decoding.

[0257] This encoder is Fig. 6A shown. Specifically, Fig. 6A The overall structure of the end-to-end trainable video compression framework is shown. In order to compress motion information, a CNN is specified to transform the optical flow into a corresponding representation suitable for better compression. Specifically, an autoencoder style network is used to compress the optical flow. The motion vector (MV) compression network is shown in Figure 2. Figure 6B As shown in Figure 2. Figure 4 The ga / gs in are somewhat similar. Specifically, the optical flow is fed into a series of convolution operations and nonlinear transformations including GDN and IGDN. The number of output channels of the convolution (deconvolution) except the last deconvolution layer is 128, and the number of output channels of the last deconvolution layer is equal to 2. Given an optical flow of size M×N×2, the MV encoder will generate a motion representation of size M / 16×N / 16×128. The motion representation is then quantized, entropy decoded and sent to the bitstream. The MV decoder receives the quantized representation and reconstructs the motion information using the MV encoder.

[0258] Figure 6C Here, the previously reconstructed frame x is used. t–1 The warping unit generates the warped frame (usually with the help of an interpolation filter, such as a bilinear interpolation filter) using the reconstructed motion information. Then, a separate CNN with three inputs generates the predicted image. The architecture of the motion compensated CNN is also as Figure 6C shown.

[0259] The residual information between the original frame and the predicted frame is encoded by the residual encoder network. A highly nonlinear neural network is used to transform the residual into the corresponding latent representation. Compared with the discrete cosine transform in traditional video compression systems, this method can better utilize the power of nonlinear transformation and achieve higher compression efficiency.

[0260] From the above overview, it can be seen that CNN-based architectures can be applied to image and video compression, thereby considering different parts of the video framework, including motion estimation, motion compensation, and residual decoding. Entropy decoding is a popular method for data compression, widely adopted by the industry, and is also suitable for feature map compression for human perception or computer vision tasks.

[0261] In the case of lossless video decoding, the reconstructed video image has the same quality as the original video image (assuming no transmission errors or other data loss during storage or transmission). In the case of lossy video decoding, further compression is performed by quantization and the like to reduce the amount of data required to represent the video image, and the video image cannot be fully reconstructed at the decoding end, that is, the quality of the reconstructed video image is lower or worse than the quality of the original video image.

[0262] Arithmetic Coding

[0263] Entropy decoding is often used as lossless decoding. Arithmetic decoding is a type of entropy decoding that encodes a message as a binary real number within an interval (range) representing the message. In this article, the term message refers to a sequence of symbols. Symbols are selected from a predefined symbol alphabet. For example, the alphabet can consist of two values ​​0 and 1. A message using such an alphabet is a bit sequence. Symbols (0 and 1) can appear in a message at different frequencies from each other. In other words, the symbol probability can be uneven. In fact, the more uneven the distribution, the higher the compression that can be achieved by general entropy decoding (specifically arithmetic decoding). Arithmetic decoding uses a priori known probability model to represent the symbol probability of each symbol in the alphabet. The alphabet does not need to be binary. Instead, for example, the alphabet can consist of M values ​​(0 to M–1). In general, an alphabet of any size can be used. Typically, the alphabet is given by the value range of the decoded data.

[0264] A variant of the arithmetic decoder that has been improved for practical use is called a range decoder, which does not use the interval [0, 1), but a finite range of integers, for example from 0 to 255. The range is divided according to the probability of the alphabet symbols. If the remaining range becomes too small, the range can be renormalized so that all alphabet symbols are described according to their probabilities.

[0265] One of the main types of entropy decoders assigns a unique code to each unique symbol that appears in the input. These entropy encoders compress data by replacing each fixed-length input symbol with a corresponding variable-length output codeword. Simple static codes are useful for data streams with some specific entropy characteristics. These static codes include universal codes (such as Elias gamma decoding or Fibonacci decoding) and Golomb codes (such as unary decoding or Rice decoding). For general data streams, the code can be constructed according to the following rule: the length of each codeword is approximately proportional to the negative logarithm of the probability of occurrence of the codeword. Therefore, the most common symbols use the shortest code. According to the constructed code table, the decoder compresses the data by replacing each fixed-length input symbol with the corresponding variable-length prefix-free output codeword. An example of this decoding is Huffman decoding. The main problem with this decoding is that even if the probability of each input symbol is close to 1, each input symbol requires at least one bit. With the development of arithmetic decoders, the asymmetric numeral system (ANS) series of entropy decoding techniques were invented. These decoders offer a combination of the compression ratio of arithmetic decoding with a processing cost similar to Huffman decoding.

[0266] The entropy decoder encodes the symbols of size M of the input alphabet A into symbols of size R of the alphabet B by using the number of output symbols inversely proportional to the probability of the decoded symbols. In general, a symbol a in alphabet A is i The probability p i It refers to the symbol a i The probability of appearing in any sequence of symbols in the alphabet A. In other words, the probability p i means that the received symbol y is equal to a i The unequal probabilities of different symbols in the alphabet give the possibility of compression. If all symbols in the alphabet have the same probability pi = 1 / M, where M is the size of the alphabet A, then compression is impossible.

[0267] The general scheme of entropy decoder is as follows Figure 7 In most cases, the output alphabet is {0, 1}, the size of the output alphabet is usually equal to 2, the symbols of the output binary alphabet are called bits, and the bit sequence corresponding to the decoded symbol sequence in the input alphabet is called a code stream. Figure 7As shown, the output symbols of the entropy encoder are called code streams, which are the input of the entropy decoder. The output of the entropy decoder uses the same alphabet as the input of the entropy encoder. The output of the entropy decoder can be called decoded symbols. In addition, the alphabet represents a set of symbols, and the alphabet size represents the number of symbols in the input alphabet. Here, the input symbols are represented as 0, 1, 2...M-1, so there are a total of M different symbols in the input alphabet. It should be noted that in this application, the alphabet size M always refers to the size of the input alphabet.

[0268] In autoencoder based decoding schemes, an entropy decoder is used to compress the latent space symbols. The distribution estimation can be done in advance (pre-trained histograms) or it can be performed using some additional information from the codestream and / or information from neighboring latent objects. Figure 8 The general scheme of using entropy decoding in an autoencoder-based decoder is described: an input signal x is converted into a feature (or latent) tensor y, where "x" represents the input signal corresponding to image data, and "y" represents the latent space tensor. The conversion process here can be called feature extraction, and the latent space tensor includes latent space elements, which will be quantized and used as input to the entropy encoder. In some possible embodiments, the latent space elements will be composed of Fig. 9 The gain unit shown is processed, then quantized, and then used as input to the entropy encoder. The tensor y includes real numbers (rather than integers) (e.g., floating-point values), and the range of these numbers is not known in advance. The entropy decoder can only use a limited input alphabet, so the tensor y is converted to an integer tensor containing values ​​from 0 to M–1 Where M is the alphabet size of the entropy decoder. This conversion from a continuous set of values ​​to a discrete set is called quantization. The conversion may include clamping, rounding, and scaling operations. In one exemplary implementation, y may first be clamped to the range Then it is rounded, and then by adding The resulting tensor is converted to the range [0, M-1]. As long as the real tensor y is converted to an integer tensor All values ​​are in the range [0, M–1], then the tensor The data can be encoded into a bitstream by an entropy decoder. On the decoder side, the entropy decoder decodes the tensor from the bitstream This tensor is further transformed into a reconstructed signal by the synthesis part of the autoencoder. It should be noted that the same parameters of the entropy decoder, specifically the same input alphabet size, are used for all possible input signals {x}. Therefore, it can be said that in the traditional approach, the entropy decoding parameters are predefined.

[0269] like Fig. 9As shown, in order to control the quantization error and the bitstream size (bitrate), a gain unit can be added to the decoding scheme. It should be noted here that the larger the bitrate, the better the data can be compressed, and the lower the bitrate, the lower the data can be compressed. The process of adjusting the compression system parameters to achieve the desired ratio between bitrate and quality (or to achieve the desired bitrate) is called rate control. The parameter used during rate control is called the rate control parameter β. The gain unit is used for rate control. The gain unit is a part of the neural network that multiplies the latent space tensor y with the gain vector g: y⊙g, where the i-th channel of the tensor y is multiplied by the element g[i] of the vector g. The size of the vector g is equal to the number of channels in the latent tensor y, so each channel is multiplied by its own scaling factor g[i]. If a lower bitrate is required, a smaller gain vector g is selected, and if a higher bitrate is required, a larger vector g is used. In this case, before quantization, the latent tensor y is multiplied by the gain vector g to obtain the actual tensor y g Typically, for higher bit rates, a vector g is chosen that includes larger values, and y g becomes much wider than the range of y, in which case the predefined alphabet size M, chosen once according to the expected y values, becomes inadequate for these y g Value. y based on the alphabet size M g Clipping can severely corrupt the signal, resulting in poor reconstruction quality. In some exemplary implementations, the bit rate can be controlled by a scalar parameter β, which can also be referred to as a rate control parameter β, and there is a predefined scheme or pre-trained function g(β) to obtain the gain vector g for each value of β. Generally, larger values ​​of β result in higher bit rates. For high bit rates, y g The range of can be much larger than the original value of y. In one possible embodiment, for a system without a gain unit, the rate control parameter β can be specified as a rate-distortion weighting factor in the loss function during model training. For example, the loss function can be as follows: loss = β * distortion + number_of_bits, where β represents the rate control parameter and number_of_bits is the number of bits used or the number of bits perpixel (bpp). In this case, the model is trained to have a specific ratio (a rate point) between distortion and bit rate.

[0270] Fig.10Figure 1 shows the rate-distortion curve of one of the decoders with gain units based on the autoencoder. The horizontal axis represents the bit rate (in bits per pixel (bpp)) and the vertical axis represents the peak signal-to-noise ratio (PSNR). As can be noticed here, as the bit rate (horizontal axis) increases above 0.5bpp, the PSNR (vertical axis) of the reconstructed signal does not increase anymore but decreases. This is due to the larger y g Limiting error occurs.

[0271] One possible solution is to choose a very large alphabet size and use it in all cases, but there are some conditions where increasing the alphabet size can reduce compression efficiency, e.g. low bitrates do not require a large alphabet size, and using a large alphabet size can significantly increase the bitrate without improving the reconstruction quality.

[0272] In conventional methods, entropy decoding parameters are usually predefined, for example, the alphabet size M is usually predefined by selecting once according to the expected tensor range (or potential tensor range) and using the predefined alphabet size M for all cases. In this case, if the actual tensor range is wider than the expected tensor range, the input alphabet size determined according to the expected tensor range will not be appropriate, and the decoded tensor values ​​need to be clipped. Such clipping will damage the signal, especially when the decoded tensor range is very different from the alphabet size. In this case, the damage to the decoded tensor is nonlinear distortion, which leads to unpredictable errors in the reconstructed signal, so the quality of the reconstructed signal may be significantly affected. In one implementation, an oversized alphabet size can be selected and used in all cases, but increasing the alphabet size will penalize the compression efficiency under low bit rate conditions, and using a large alphabet size will result in a significant increase in bit rate, but will not improve the reconstruction quality.

[0273] In order to solve the above problems, the embodiment of the present application provides the selection of content / bit rate adaptive entropy decoding parameters, specifically, the entropy decoding parameter can be the input alphabet size, so that the clipping effect at high rate can be avoided, and the rate overhead will not be caused by the unreasonable large alphabet size at low rate. Due to the adaptability of the entropy decoding parameter (specifically the alphabet size), the entropy decoder can work optimally at low rate (narrow range of decoded values), which achieves bit rate savings; and there is no clipping effect at high rate (wide range of decoded values), which improves the quality of the reconstructed signal.

[0274] The basic idea of ​​this scheme is the bitrate / content adaptability of the entropy decoding parameters (specifically the alphabet size). In order to perform entropy decoding correctly, all parameters between the encoder and the decoder should be aligned, so basically two problems need to be solved:

[0275] 1. How to choose the appropriate alphabet size on the encoder side?

[0276] 2. How does the decoder side derive the alphabet size (chosen on the encoder side)?

[0277] For the alphabet selection on the encoder side, several possible schemes are provided.

[0278] In one possible implementation, the alphabet size can be selected to be the smallest possible number that is larger than the decoded value range. For example, first obtain the minimum and maximum values ​​of the tensor y, and the alphabet size is selected as follows:

[0279] M = ceil(max{y}–min{y}),

[0280] Where ceil(x) is the smallest integer greater than x. For most entropy decoders, the alphabet size should be a power of 2, in which case the alphabet size can be chosen as M = 2^(ceil(log2(max{y}–min{y}))). Where {y} refers to the latent space element in the latent space, where the latent space element is the result of the input signal processing. Sometimes, the process of transforming the input signal into the latent space tensor y is called feature extraction. Typically, the input signal (e.g., an input image) is converted to a latent space (feature space), and the latent space elements are quantized and then encoded with an entropy encoder. In addition, the latent space can be additionally processed (e.g., multiplied by a gain vector) before quantization. It should be noted that in some cases, for example, when the modules of all y values ​​are less than 1, additional scaling operations can be performed before entropy decoding.

[0281] In another possible implementation, the alphabet size can be selected based on a rate-distortion optimization process. First, several values ​​of M around M_0 = ceil(max{y}–min{y}) are tried, and then the loss function is calculated for all these values. The alphabet size Mi for which the loss function is the smallest is selected. The loss function may include rate and distortion components, such as PSNR, multi-scale structural similarity index (MS-SSIM), video multimethod assessment fusion (VMAF), or some other quality metric. In this approach, clipping sometimes occurs, but the bitrate saved by using a smaller alphabet compensates for the slight increase in distortion. For example, the loss function can be: loss = β*distortion + number of bits, where distortion is measured by PSNR or MS-SSIM or VMAF, number of bits is the number of bits used, and β is a weighting parameter that controls the ratio between bitrate and reconstruction quality. β can also be called a rate control parameter.

[0282] For the alphabet size derivation on the decoder side, several possible schemes are provided.

[0283] Example 1

[0284] In one possible implementation, the alphabet size may be explicitly indicated in the code stream. In one embodiment, the alphabet size may be indicated directly in the code stream, for example using fixed length decoding, exp-Golomb decoding, or some other decoding algorithm. Typical values ​​of M may be 256, 512, 1024. For example, for an indication of 1024 with a fixed length code, 11 bits (1024 10 =100000000002), for the indication log2(1024)–9=1, only 1 bit is needed if only values ​​512 and 1024 are supported, and 2 bits are needed if 4 different alphabet sizes are supported, such as 512, 1024, 2048, 4096. Therefore, direct indication of M will use more bits. But for some exotic cases (such as the alphabet size M is not a power of 2), direct indication of M may help.

[0285] In another embodiment, the output p of some reversible function f(M) may be indicated in the code stream instead of M itself, and the output p may be referred to as the first indication information. The p may be indicated by fixed-length decoding, exp-Golomb decoding or some other decoding algorithm. Therefore, at the decoder side, M is derived according to the first indication information. Specifically, M is derived according to M=f –1 (p) is derived. An example of such a reversible function f(M) can be as follows:

[0286] 1. f(M) = log k (M), where k is a natural number, for example, k may be equal to 2;

[0287] 2. f(M) = log k (M)–C, where C is an integer used as a prediction value, for example, C can be equal to 9;

[0288] 3. f(M) = M + R, where R is an integer used as the prediction value;

[0289] 4. f(M) = sqrt(M);

[0290] A preferred method is to indicate that p=f(M)=log2(M)−9.

[0291] In some implementations, p is greater than or equal to 0, but in other implementations, p may also be negative. For example, the value p may be in the range [0, 5], indicated using 3 bits. The function f(M) is negotiated in advance between the encoder side and the decoder side.

[0292] In one possible embodiment, the alphabet size is indicated in a parameter set part or the like in a code stream, for example, in a picture parameter set part of the code stream.

[0293] The advantage of the above embodiment 1 is that any optimal alphabet size selected at the encoder side can be indicated, so the flexibility of indicating the alphabet size is increased. The only disadvantage is that some bits are required for indication, so the code stream size is slightly increased.

[0294] Example 2

[0295] In one possible embodiment, the alphabet size can be derived from some other parameters. In an exemplary implementation, the alphabet size is derived from a quantization parameter or a rate control parameter, or the alphabet size can be derived from image resolution, video resolution, frame rate, density of pixels in a 3D object, etc. In a trainable codec, the alphabet size can be derived from some parameters of the loss function used during training, such as a weighting factor for rate / distortion, or some parameters that affect the selection of the gain vector g. It can also be a quantization parameter, such as a quantization parameter (QP) in conventional codecs (such as JPEG, HEVC, VVC). For example, the loss function can be: loss = β * distortion + bit rate, where β is a weighting factor.

[0296] In an exemplary implementation, the rate control parameter is denoted as β, and the range of beta (β) is divided into K intervals (K sub-ranges) as follows:

[0297] [β_0, β_1), [β_1, β_2)……[β_(K–1), β_K)

[0298] Each interval / sub-range corresponds to an alphabet size Mi. Note that β_0 can be equal to –∞ and β_K can be equal to +∞. For a particular codec value β, there will be an allowed range; for example, for some codecs β may be allowed in the range [–∞,∞], and for other codecs β may only be allowed in the range [0,∞]. Therefore, there is a large range of supported beta (β) values. In the context of this embodiment, the original large range of allowed β values ​​is divided into several sub-ranges, and for each sub-range, there is a specific value of the alphabet size. Fig.11 A particular division of the β values ​​into intervals is shown.

[0299] In this case, after obtaining the parameter β, the decoder can select the target interval according to the β value obtained from the bitstream. Specifically, the decoder determines that β_i≤β≤β_(i+1), then selects the interval [β_i,β_(i+1)] as the target interval, and the decoder side derives the alphabet size Mi corresponding to the target interval as the input alphabet size M.

[0300] In some embodiments, each βi within the β range {βi} may correspond to an alphabet size Mi, and the alphabet size M corresponding to a particular β is calculated based on one or more Mi corresponding to the βi adjacent to β. It should be noted that the value used to calculate M may be the timely value Mi of the nearest neighbor corresponding to the target interval, or it may be linear or bilinear or some other interpolation from two or more Mi corresponding to the βi adjacent to β, or some other interpolation from two or more Mi corresponding to the interval adjacent to the target interval.

[0301] The advantage of the above-mentioned embodiment 2 is that, since the decoder side can use the quantization parameter or the rate control parameter (which already exists in the bitstream and is used for other processes) to derive the alphabet size M, there is no need for additional indication of information specifically used to indicate the alphabet size M, so bit rate can be saved. The disadvantage of embodiment 2 is the lack of flexibility, so if for some reason the derived alphabet size is not optimal, the encoder and decoder must use it, although the compression efficiency is lower.

[0302] Example 3

[0303] In one possible implementation, the alphabet size can be derived based on the prediction value P and the second indication information, which is indicated in the bitstream and is used to indicate the difference between P and M. The prediction value P can be derived by the decoder according to one of the techniques described in the above embodiment 2, such as quantization parameters, rate control parameters, parameters of the loss function used during the training of the trainable codec, or some parameters that affect the selection of the gain vector g. The parameters used to derive the prediction value P are selected by the encoder and can also be predefined by the standard. Therefore, when the bitstream is received, the decoder derives the prediction value P according to the predefined parameters, and parses the second indication information from the bitstream, and then the alphabet size M can be derived based on the prediction value P and the second indication information.

[0304] In one embodiment, the difference between P and M can be indicated directly in the bitstream, for example using fixed length decoding, exp-Golomb decoding, or some other decoding algorithm. In an alternative embodiment, the output D of some reversible function s(M,P) can be indicated in the bitstream. This D can be indicated using fixed length decoding, exp-Golomb decoding, or some other decoding algorithm. In this case, M is indicated at the decoder side according to M=s –1 (D, P) is derived. An example of the reversible function s(M, P) can be as follows:

[0305] 1.s(M,P)=log k (M)–log k (P), where k is a natural number, for example, k may be equal to 2;

[0306] 2.s(M,P)=log k (P)–log k (M), where k is a natural number, for example, k may be equal to 2;

[0307] 3.s(M,P)=log k (M)–log k (P)–C, where C is an integer;

[0308] 4.s(M,P)=log k (P)–log k (M)–C, where C is an integer;

[0309] 5.s(M,P)=a*log k (P)+b*log k (M)–c, where a, b, and c are constants;

[0310] 6.s(M,P)=a*M+b*P+c, where a, b and c are constants.

[0311] It should be noted here that, in any embodiment, A*B means A multiplied by B.

[0312] A preferred method is to indicate that D = s(M, P) = log2(P) - log2(M).

[0313] Correspondingly, M satisfies one of the following:

[0314] or

[0315] or

[0316] or

[0317] or

[0318] Among them, a, b and c are predefined constants;

[0319] M=a1*D+b1*P+c1, where a1, b1 and c1 are constants.

[0320] In most cases, k=2.

[0321] Since only the difference between P and M is indicated in the code stream, the additional bits used are reduced compared to indicating M in the code stream. In addition, the difference between P and M can be selected according to the content or bit rate, and the flexibility of indicating the alphabet size is increased. Therefore, the above-mentioned embodiment 3 combines the advantages of embodiment 1 and embodiment 2: flexibility in selecting the alphabet size is provided, and the additional bits used for indication are minimal. In some rare cases, when the alphabet size predicted from β does not work well, the encoder can still indicate the difference between M and P. This will use some bits, but can solve the serious problem of clipping effect.

[0322] In one possible implementation, a flag may be introduced into the code stream to indicate the switch between Embodiment 1, Embodiment 2 and Embodiment 3, in which case the flag may require two bits. In another possible implementation, the flag may be used to indicate the switch between Embodiment 1 and Embodiment 2, in which case only one bit is required. This scheme provides a balance between bit saving and flexibility: in most cases, when the derived entropy parameter is appropriate, only one bit is used for indication, and in some specific cases, it is possible to explicitly indicate the entropy parameter from the other side.

[0323] In a possible implementation, when the flag is equal to the first value, it indicates that embodiment 1 will be used, and the bitstream carries the entropy decoding parameters or the transformation results of the entropy decoding parameters. When the flag is equal to the second value, it indicates that embodiment 2 will be used, and the bitstream does not carry the entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder. When the flag is equal to the third value, it indicates that embodiment 3 will be used, and the bitstream carries the difference between M and P or the transformation results of the difference between M and P, where M is the size of the input alphabet and P is the predicted value that can be derived by the decoder.

[0324] In addition to the above embodiments, alternative indication modes may also be considered. The alphabet size M may be derived from a predefined value using an interpolation or extrapolation process. For example, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code. In one possible embodiment, p is indicated using a 0th order exp-Golomb code.

[0325] The above embodiments can be applied to different entropy decoders, such as an arithmetic decoder, a range decoder, an asymmetric numerical system (ANS) decoder, etc.

[0326] In some possible implementations, more entropy decoding parameters may be adaptively selected at least according to the content or bit rate. For example, the parameters of entropy decoding may also include: the minimum symbol probability supported by the entropy decoder or the probability accuracy supported by the entropy decoder, or the renormalization period of the entropy decoder. In some embodiments, the renormalization period may be 8 bits, 16 bits, etc. It should be noted here that "entropy decoder" may be used as a synonym for "entropy decoding algorithm", including encoding algorithms and decoding algorithms. The entropy encoder is a module that is part of the encoder; the entropy decoder is another module that is part of the decoder. The parameters of the entropy decoder and the entropy decoder should be synchronized in order to work correctly, so the term "entropy decoder parameters" or "entropy decoding parameters" refers to the parameters of both the entropy encoder and the entropy decoder. In other words, "entropy decoding parameters" may be equivalent to "entropy encoder and entropy decoder parameters". The entropy encoder encodes the symbols of the alphabet into one or more bits in the code stream, and the entropy decoder decodes one or more bits in the code stream into symbols of the alphabet. On the entropy encoder side, the alphabet represents the input alphabet, while on the entropy decoder side, the alphabet represents the output alphabet. The size of the input alphabet on the entropy encoder side is equal to the size of the output alphabet on the entropy decoder side.

[0327] Fig.12 A flowchart of an exemplary decoding method implemented by a decoding device, the method comprising:

[0328] 1201: Receive a code stream including encoded data of an input signal and a first parameter;

[0329] 1202: Parse the code stream to obtain the first parameter.

[0330] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vector or any other type of media data; the encoded data refers to the encoded result of the input signal, and the encoded data consists of multiple bits; the entropy decoding parameters may include: the size of the entropy decoder alphabet, where the size of the alphabet is: the size of the input alphabet of the entropy encoder, or the size of the output alphabet of the entropy decoder; the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0331] 1203: Obtain an entropy decoding parameter according to the first parameter;

[0332] 1204: Reconstruct at least a portion of the input signal based on the entropy decoding parameters and the encoded data.

[0333] In an embodiment of the present application, the decoder can obtain entropy decoding parameters (specifically, the alphabet size) according to the parameters carried in the code stream. Since the parameters carried in the code stream can be changed, the encoder can adaptively adjust the entropy coding parameters by changing the parameters carried in the code stream. Therefore, under high bit rate conditions, the limiting effect can be avoided, and the rate overhead caused by the unreasonable large alphabet size under low bit rate conditions can also be avoided. In other words, due to the adaptability of the entropy decoding parameters (specifically the alphabet size), the best operation of the entropy decoder may be performed at a low bit rate (corresponding to a narrow range of decoded values), which saves bit rate; and there is no limiting effect at a high bit rate (corresponding to a wide range of decoded values), which improves the quality of the reconstructed signal.

[0334] In one possible embodiment, reconstructing at least a portion of an input signal according to an entropy decoding parameter includes:

[0335] Acquire at least one probability model, wherein the probability model of the output symbol is used to indicate the probability of each possible value of the output symbol; perform entropy decoding on one or more bits in the code stream by using the at least one probability model and entropy decoding parameters to obtain one or more output symbols; reconstruct at least a portion of the input signal according to the one or more output symbols.

[0336] In one possible embodiment, the method further includes: updating the probability model. For example, the probability model is updated after each output symbol; therefore, each output symbol has its own probability distribution of possible values. It should be noted that the probability model can also be called a probability distribution.

[0337] In one possible embodiment, the probability model is selected based on the entropy decoding parameters. For example, the symbol probability is distributed according to the normal distribution N(μ,σ), where N(μ,σ) means that the mean is equal to μ and the variance is equal to σ 2 Gaussian distribution. But the actual probability model (also referred to as mathematical model or theoretical model), such as quantization histogram, depends on the alphabet size and probability precision in the entropy decoding engine or entropy decoder. The probability precision can be the minimum probability supported by the entropy decoding engine. That is, the entropy decoding parameters may affect the histogram structure inside the entropy decoder. Basically, the alphabet size is the number of possible symbol values, so if (for example) the alphabet size is equal to 4, larger values ​​(such as the value "7") cannot be encoded / decoded.

[0338] The histogram used in the entropy decoder consists of the quantized probability of each symbol value: for example, the alphabet is {0, 1, 2, 3}, and the corresponding probability is {7 / 16, 7 / 16, 1 / 16, 1 / 16}, each probability is non-zero, and the sum of the probabilities is equal to 1; moreover, each probability is greater than the minimum probability (probability accuracy) supported by the entropy decoding engine (1 / 16 in this example). If the probability of some symbols is lower than the minimum probability supported by the entropy decoding engine, it is necessary to adjust the probability of at least some symbols to ensure that the probability of each symbol is greater than the minimum probability supported by the entropy decoding engine. For example, the alphabet size is 8: {0, 1, 2, 3, 4, 5, 6, 7}, if two symbol values ​​are equal to the probability of 7 / 16, such as {7 / 16, 7 / 16, 1 / 16, 1 / 16, 0 / 16, 0 / 16, 0 / 16, 0 / 16}, because the probability should be greater than or equal to 1 / 16. So, for example, in this model, the probability of symbols "0" and "1" must be reduced from 7 / 16 to 5 / 16, and the probability of each symbol adjusted to {5 / 16, 5 / 16, 1 / 16, 1 / 16, 1 / 16, 1 / 16, 1 / 16, 1 / 16}. Basically, this is one of the explanations why entropy decoders with larger alphabets are less efficient. If there are many different possible symbol values, each of them should have a probability no less than the minimum probability supported by the entropy decoder. So, even if the probability of a symbol is very large (like 0.99999...), in the quantization histogram it will be only 1-(M-1)*p min , where M is the size of the alphabet, p min is the minimum probability supported by the entropy decoder. Therefore, the maximum probability in the model depends on the alphabet size and the minimum probability supported by the entropy decoder: max =1-(M-1)*p min In practice, p min cannot be small, as it is related to computational precision. So, for example, if and the alphabet size M is equal to 128, then the maximum possible probability will be equal to This probability is not that great and in some cases is not enough.

[0339] In a possible embodiment, the first parameter is the size of the alphabet; wherein obtaining the entropy decoding parameter according to the first parameter includes: using the first parameter as the size of the alphabet.

[0340] In another possible embodiment, the first parameter is the output p of a reversible function f(M), rather than M itself. For example, the first parameter is p=f(M). In this case, the entropy decoding parameter is obtained as follows: M=f –1 (p); where f –1 (p) is the inverse function of f(M).

[0341] In one possible embodiment, f(M) may be as follows:

[0342] f(M)=log k (M), where k is a natural number; or

[0343] f(M)=log k (M)–C, where k is a natural number and C is an integer; or

[0344] f(M)=M+R, where R is an integer; or

[0345] f(M)=sqrt(M).

[0346] In one possible embodiment, p=f(M)=log2(M)−9.

[0347] Correspondingly, where M satisfies one of the following:

[0348] M = k^p, where k is a natural number; or

[0349] M = k^(p+C); where k is a natural number and C is an integer; or

[0350] M = k^(a*p+b); where k is a natural number, and a and b are constants; or

[0351] M = a*p+b, where a and b are constants; or

[0352] M=p^2.

[0353] It should be noted that in any embodiment, A^B means A B .

[0354] In one possible embodiment, p = log2(M) - 9, M = f –1(p) = 2^(p+9), where f –1 (p) is the inverse function of f(M), where f(M) = log2(M)–9.

[0355] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0356] In one possible embodiment, p is indicated using a 0th order exp-Golomb code.

[0357] In one possible embodiment, the alphabet size is indicated in a parameter set part or the like in a code stream, for example, in a picture parameter set part of the code stream.

[0358] In one possible embodiment, the first parameter may be some other parameters, such as rate control parameters, image resolution, video resolution, frame rate, density of pixels in a 3D object, some parameters of the loss function used during the trainable codec, such as a weighting factor for rate / distortion, or some parameters that affect the selection of the gain vector g. The loss function may include rate and distortion components, such as peak signal-to-noise ratio (PSNR), multi-scale structural similarity index (MS-SSIM), video multimethod assessment fusion (VMAF) or some other quality indicators. For example, the loss function may be: loss = β * distortion + number of bits, where the distortion is measured by PSNR or MS-SSIM or VMAF, the number of bits is the number of bits used, and β is a weighting parameter that controls the ratio between bit rate and reconstruction quality, which may also be referred to as a rate control parameter. The first parameter may also be a quantization parameter, such as a quantization parameter (QP) in conventional codecs (such as JPEG, HEVC, VVC). In this case, the entropy decoding parameters may be derived by the decoder side based on the above-mentioned other parameters.

[0359] The benefit of the above embodiment is that, since the quantization parameters or rate control parameters already exist in the bitstream and are used for other processes, the decoder side can use these parameters to derive the alphabet size M, so there is no need for additional indication of information specifically used to indicate the alphabet size M, thereby saving bit rate.

[0360] In a possible embodiment, obtaining an entropy decoding parameter according to a first parameter includes: determining a target sub-range in which the first parameter is located; wherein the allowable range of values ​​of the first parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the first parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; using the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculating the value of the entropy decoding parameter according to one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0361] In one possible embodiment, the first parameter is D, and the entropy decoding parameter includes the size of the alphabet M, where M is obtained based on P and D, where P is a prediction value that can be derived by the decoder.

[0362] In one possible embodiment, the first parameter may be the difference between M and P, where M is the size of the input alphabet and P is a prediction value that the decoder may derive using one of the techniques described in Embodiment 2 above.

[0363] The benefit of the above embodiment is that, since only the difference between P and M is indicated in the code stream, the additional bits used are reduced compared to indicating M in the code stream. In addition, the difference between P and M can be selected based on content or bit rate, and the flexibility of indicating the size of the alphabet is increased. Therefore, this embodiment provides flexibility in the selection of the alphabet size, and the additional bits used for indication are minimal. In some rare cases, when the alphabet size predicted from β does not work well, the encoder can still indicate the difference between M and P. This will use some bits, but can solve the serious problem of clipping effects.

[0364] In a possible embodiment, the first parameter is a value obtained by processing the difference between M and P. For example, the first parameter is D=s(M, P), where s(M, P) is a reversible function; where s(M, P) can be:

[0365] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0366] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0367] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0368] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0369] s(M,P)=a*M–b*P+c, where a, b and c are constants.

[0370] In one possible embodiment, D=s(M,P)=log2(P)−log2(M).

[0371] In this case, the entropy decoding parameters can be obtained as follows: M = s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P).

[0372] It should be noted that the reversible function D = s(M, P) can be considered as D = s P (M), M = s –1 (D,P) can be considered as M=s –1 P (D), where P can be any fixed number, or in other words, P is a constant coefficient.

[0373] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0374] In one possible embodiment, D is indicated using a 0th order exp-Golomb code.

[0375] In a possible embodiment, P may be derived according to at least one parameter other than the first parameter carried in the code stream.

[0376] In a possible embodiment, at least one parameter other than the first parameter includes at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor.

[0377] In one possible embodiment, P is derived based on at least one parameter, including: obtaining a rate control parameter beta (β) from a bitstream; determining a target sub-range in which the obtained β is located; wherein there is an allowable range [β_0, β_K] of the value of the rate control parameter β, the allowable range [β_0, β_K] is divided into multiple sub-ranges, and the target sub-range is one of the multiple sub-ranges; each of the multiple sub-ranges includes at least one value of β, and each of the multiple sub-ranges corresponds to a value of P; selecting a value corresponding to the target sub-range as the value of P; or calculating the value of P based on one or more values ​​corresponding to one or more sub-ranges adjacent to the target sub-range.

[0378] In a possible embodiment, the entropy decoder is an arithmetic decoder, or a range decoder, or an asymmetric numerical system (ANS) decoder.

[0379] Optionally, the method further comprises:

[0380] 1205: Parse the bitstream to obtain a flag, wherein the flag is used to indicate whether the bitstream directly carries entropy decoding parameters.

[0381] In the above embodiment, a flag may be introduced into the code stream to indicate switching between the three embodiments, in which case the flag may require two bits. In another possible embodiment, a flag may be used to indicate switching between two embodiments, in which case only one bit is required.

[0382] In a possible embodiment, when the flag is equal to the first value, it indicates that the bitstream carries entropy decoding parameters, in which case the first parameter is the entropy decoding parameter, or the first parameter is a transformation result of the entropy decoding parameter; or when the flag is equal to the second value, it indicates that the bitstream does not carry entropy decoding parameters, and the entropy decoding parameters can be derived by the decoder.

[0383] This scheme provides a balance between bit saving and flexibility: in most cases, where a derived entropy parameter is appropriate, only one bit is used to indicate this, and in some specific cases it is possible to explicitly indicate the entropy parameter from the other side.

[0384] In one possible embodiment, when the flag is equal to the third value, it indicates that the code stream carries the difference between M and P or the transformation result of the difference between M and P. In this case, the first parameter is the difference between M and P or the transformation result of the difference between M and P, where M is the size of the input alphabet and P is the predicted value that the decoder can derive.

[0385] Fig.13 A flowchart of an exemplary decoding method implemented by a decoding device, the method comprising:

[0386] 1301: Receive a code stream including decoded data and a flag of an input signal;

[0387] 1302: Parse the bitstream to obtain a flag, where the flag is used to indicate whether the bitstream directly carries entropy decoding parameters;

[0388] 1303: Obtain entropy decoding parameters according to the flag;

[0389] 1304: Reconstruct at least a portion of an input signal based on the entropy decoding parameters and the encoded data.

[0390] In the above embodiment, a flag may be introduced into the code stream to indicate switching between the three embodiments, in which case the flag may require two bits. In another possible embodiment, a flag may be used to indicate switching between two embodiments, in which case only one bit is required. This scheme provides a balance between bit saving and flexibility: in most cases, only one bit is used for indication when the derived entropy parameter is appropriate, and in some specific cases, it is possible to explicitly indicate the entropy parameter from the other side.

[0391] In one possible embodiment, the entropy decoding parameter includes at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0392] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vectors or any other type of media data.

[0393] In one possible embodiment, when the flag is equal to the first value, it indicates that the code stream carries entropy decoding parameters, or the code stream carries the transformation result of the entropy decoding parameters. It should be noted that the transformation result of the entropy decoding parameters refers to the result obtained by processing the entropy decoding parameters, such as a value. When the flag is equal to the second value, it indicates that the code stream does not carry the entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder. In this case, the flag is used to indicate the switch between the above-mentioned embodiment 1 and embodiment 2, and only one bit is required.

[0394] This scheme provides a balance between bit saving and flexibility: in most cases, where a derived entropy parameter is appropriate, only one bit is used to indicate this, and in some specific cases it is possible to explicitly indicate the entropy parameter from the other side.

[0395] Optionally, the flag can be used to indicate the switching between the above-mentioned embodiment 1, embodiment 2 and embodiment 3. In this case, the flag has three possible values, requiring 2 bits. When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P, or the code stream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder. It should be noted that the transformation result of the difference between M and P refers to the processing result of the difference between M and P.

[0396] In a possible embodiment, obtaining an entropy decoding parameter according to a flag includes: when the flag is equal to a first value, parsing a bitstream to obtain a first parameter; when the first parameter is an entropy decoding parameter, using the first parameter as the entropy decoding parameter; or when the first parameter is a transformation result of the entropy decoding parameter, obtaining the entropy decoding parameter according to the first parameter.

[0397] In a possible embodiment, the transformation result of the entropy decoding parameter is p=f(M), where M is the entropy decoding parameter, and f(M) includes: f(M)=log k (M), where k is a natural number; or f(M) = log k (M)–C, where k is a natural number and C is an integer; or f(M)=M+R, where R is an integer; or f(M)=sqrt(M); wherein obtaining the entropy decoding parameter according to the first parameter includes: M=f –1 (p); where f –1 (p) is the inverse function of f(M).

[0398] Correspondingly, M satisfies one of the following:

[0399] M=k p , where k is a natural number; or

[0400] M=k p+C , where k is a natural number and C is an integer; or

[0401] M = ap + b, where a and b are constants; or

[0402] M=p 2 .

[0403] In one possible embodiment, k=2.

[0404] In a possible embodiment, the first parameter is p=log2(M)−9.

[0405] In a possible embodiment, obtaining entropy decoding parameters according to a flag includes: when the flag is equal to a second value, parsing the bitstream to obtain a second parameter, the second parameter including at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; and deriving the entropy decoding parameters according to the second parameter.

[0406] In a possible embodiment, deriving an entropy decoding parameter based on a second parameter includes: determining a target sub-range in which the second parameter is located, wherein the allowable range of values ​​of the second parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the second parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; using the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculating the value of the entropy decoding parameter based on one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0407] In a possible embodiment, obtaining entropy decoding parameters according to a flag includes: when the flag is equal to a third value, parsing the bitstream to obtain a third parameter, wherein the third parameter is a difference between M and P, or the third parameter is a transformation result of the difference between M and P, wherein M is an entropy decoding parameter, and P is a prediction value that can be derived by a decoder; deriving P according to at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; and obtaining the entropy decoding parameters according to the third parameter and P.

[0408] In a possible embodiment, the transformation result of the difference between M and P is D=s(M, P), where s(M, P) is a reversible function; where s(M, P) includes:

[0409] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0410] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0411] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0412] s(M,P)=logk (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0413] s(M,P)=a*log k (P)+b*log k (M)–c, where a, b, and c are constants;

[0414] s(M,P)=a*M–b*P+c, where a, b and c are constants;

[0415] Wherein, the entropy decoding parameter is obtained according to the third parameter, including: M = s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P). It should be noted here that A*B means A multiplied by B.

[0416] In a possible embodiment, M satisfies one of the following:

[0417] or

[0418] or

[0419] or

[0420] or

[0421]

[0422] M=a1*D+b1*P+c1, where a1, b1 and c1 are constants.

[0423] Fig.14 A flowchart of an exemplary encoding method implemented by an encoding device, the method comprising:

[0424] 1401: Encode an input signal and a first parameter into a bitstream, wherein the first parameter is used to obtain an entropy decoding parameter.

[0425] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vector or any other type of media data; the entropy decoding parameter includes at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0426] 1402: Send the code stream to the decoder.

[0427] In one possible embodiment, the first parameter is the size of the alphabet.

[0428] In a possible embodiment, the first parameter is p, p is a transformation result of M, and M is an entropy decoding parameter.

[0429] In a possible embodiment, p=f(M), where f(M) is a reversible function.

[0430] In one possible embodiment, f(M) includes:

[0431] f(M)=log k (M), where k is a natural number; or

[0432] f(M)=log k (M)–C, where k is a natural number and C is an integer; or

[0433] f(M) = a*M+b, where a and b are constants; or

[0434] f(M)=sqrt(M).

[0435] In one possible embodiment, p=log2(M)−9.

[0436] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0437] In one possible embodiment, the first parameter includes at least one of the following: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor, wherein the first parameter is used by an entropy decoder to derive an entropy decoding parameter.

[0438] In a possible embodiment, the first parameter is D obtained according to P and M, wherein M is an entropy decoding parameter and P is a prediction value that can be derived by a decoder.

[0439] In a possible embodiment, D=s(M, P), where s(M, P) is a reversible function.

[0440] In one possible embodiment, s(M,P) includes:

[0441] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0442] s(M,P)=logk (P)–log k (M), where k is a natural number; or

[0443] s(M,P)=log k (M)–log k (P)–C, where k is a natural number and C is an integer; or

[0444] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or

[0445] s(M,P)=a*M–b*P+c, where a, b and c are constants.

[0446] In one possible embodiment, D=s(M,P)=log2(P)−log2(M).

[0447] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0448] In a possible embodiment, the encoding method further includes:

[0449] A flag is encoded into a bitstream, wherein the flag is used to indicate whether the bitstream directly carries entropy decoding parameters.

[0450] In a possible embodiment, when the flag is equal to a first value, it indicates that the bitstream carries entropy decoding parameters, the first parameter is the entropy decoding parameter, or the first parameter is a transformation result of the entropy decoding parameter; or when the flag is equal to a second value, it indicates that the bitstream does not carry entropy decoding parameters, but the entropy decoding parameters can be derived by the decoder.

[0451] In a possible embodiment, when the flag is equal to the third value, it indicates that the bitstream carries the difference between M and P, or the bitstream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder.

[0452] In one possible embodiment, several possible schemes are provided for the alphabet selection at the encoder side.

[0453] In a possible embodiment, before encoding the first parameter into the bitstream, the encoding method further includes:

[0454] The size of the alphabet of the entropy encoder is determined according to at least one of the bit rate of the image data or the decoded value.

[0455] Fig.15is a flow chart of an exemplary method for determining an alphabet size for an entropy encoder, the method comprising:

[0456] 1501: Get the minimum and maximum values ​​of the latent space elements;

[0457] 1502: Get the size of the input alphabet as follows:

[0458] M=ceil(max{y}–min{y})

[0459] Where ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, min{y} represents the minimum value of the latent space element, and M represents the size of the alphabet.

[0460] Fig.16 is a flow chart of an exemplary method for determining an alphabet size for an entropy encoder, the method comprising:

[0461] 1601: Get the minimum and maximum values ​​of the latent space elements;

[0462] 1602: Get the size of the input alphabet as follows:

[0463] M=2^(ceil(log2(max{y}–min{y}))),

[0464] Where ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, min{y} represents the minimum value of the latent space element, and M represents the size of the alphabet. For most entropy decoders, the alphabet size should be a power of 2, in which case the alphabet size can be chosen to be M = 2^(ceil(log2(max{y}–min{y}))). It should be noted that in some cases, for example, when the modules of all y values ​​are less than 1, additional scaling operations can be performed before entropy decoding.

[0465] Fig.17 is a flow chart of an exemplary method for determining an alphabet size for an entropy encoder, the method comprising:

[0466] 1701: Obtain at least two values ​​near M_0, where M_0 = ceil(max{y}–min{y}), or M_0 = 2^(ceil(log2(max{y}–min{y})));

[0467] 1702: Calculate the loss function of at least two values;

[0468] 1703: Select the value with the smallest loss function among at least two values ​​as the size of the input alphabet.

[0469] Among them, ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, and min{y} represents the minimum value of the latent space element.

[0470] For example, the loss function can be: Loss = β * distortion + number of bits, where distortion is measured using PSNR or MS-SSIM or VMAF, number of bits is the number of bits used, and β is a weighting parameter that controls the ratio between bitrate and reconstruction quality, β can also be called a rate control parameter. In this approach, clipping sometimes occurs, but the bitrate saved due to using a smaller alphabet compensates for the slight distortion increase.

[0471] Fig.18 A flowchart of an exemplary encoding method implemented by an encoding device, the method comprising:

[0472] 1801: Encode the input signal and the flag into a bitstream, wherein the flag is used to indicate whether the bitstream directly carries the entropy decoding parameter;

[0473] 1802: Send the code stream to the decoder.

[0474] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vectors or any other type of media data; the entropy decoding parameters include at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder.

[0475] In a possible embodiment, the flag is used to indicate the switch between the above-mentioned embodiment 1 and embodiment 2, in which case only one bit is required. When the flag is equal to the first value, it indicates that the bitstream carries the entropy decoding parameter, or the bitstream carries the transformation result of the entropy decoding parameter; or when the flag is equal to the second value, it indicates that the bitstream does not carry the entropy decoding parameter, but the entropy decoding parameter can be derived by the decoder.

[0476] This scheme provides a balance between bit saving and flexibility: in most cases, where a derived entropy parameter is appropriate, only one bit is used to indicate this, and in some specific cases it is possible to explicitly indicate the entropy parameter from the other side.

[0477] Optionally, the flag may be used to indicate switching between the above-mentioned embodiments 1, 2 and 3. In this case, the flag has three possible values ​​and requires 2 bits. When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P, or the code stream carries the transformation result of the difference between M and P, where M is the entropy decoding parameter and P is the predicted value that can be derived by the decoder.

[0478] In a possible embodiment, when the flag is equal to a first value, a first parameter is encoded into a bitstream, wherein the first parameter is an entropy decoding parameter or the first parameter is a transformation result of an entropy decoding parameter.

[0479] In a possible embodiment, the transformation result of the entropy decoding parameter is p=f(M), where M is the entropy decoding parameter, and f(M) may be as follows:

[0480] f(M)=log k (M), where k is a natural number; or

[0481] f(M)=log k (M)–C, where k is a natural number and C is an integer; or

[0482] f(M) = aM + b, where a and b are constants; or

[0483] f(M)=sqrt(M).

[0484] In a possible embodiment, the first parameter is p=log2(M)−9.

[0485] In a possible embodiment, p is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0486] In one possible embodiment, p is indicated using a 0th order exp-Golomb code.

[0487] In a possible embodiment, the method further includes: when the flag is equal to a third value, encoding a third parameter into the bitstream, wherein the third parameter is a difference between M and P, or the third parameter is a transformation result of the difference between M and P.

[0488] In a possible embodiment, the transformation result of the difference between M and P is D=s(M, P), where s(M, P) is a reversible function; where s(M, P) includes:

[0489] s(M,P)=log k (M)–log k (P), where k is a natural number; or

[0490] s(M,P)=log k (P)–log k (M), where k is a natural number; or

[0491] s(M,P)=log k (M)–log k(P)–C, where k is a natural number and C is an integer; or

[0492] s(M,P)=log k (P)–log k (M)–C, where k is a natural number and C is an integer; or s(M,P)=a*log k (P)–b*log k (M)–c, where a, b, and c are constants;

[0493] s(M,P)=a*M–b*P+c, where a, b and c are constants;

[0494] Wherein, the entropy decoding parameter is obtained according to the third parameter, including: M = s –1 (D,P); where s –1 (D,P) is the inverse function of s(M,P).

[0495] In a possible embodiment, D is indicated using one of the following codes: a binarization code; or a unary code; or a truncated unary code; or an exp-Golomb code.

[0496] In one possible embodiment, D is indicated using a 0th order exp-Golomb code.

[0497] An embodiment of the present application provides a decoding device, including: a receiving unit, used to: receive a code stream including encoded data of an input signal; a parsing unit, used to: parse the code stream to obtain a first parameter; an acquisition unit, used to: obtain an entropy decoding parameter according to the first parameter; and a reconstruction unit, used to: reconstruct at least a portion of the input signal according to the entropy decoding parameter.

[0498] These devices provide the advantages of the methods described above.

[0499] In one possible embodiment, the input signal includes video data, image data, point cloud data, motion flow or motion vectors or any other type of media data.

[0500] In one possible embodiment, the entropy decoding parameter includes at least one of the following: the size of the alphabet of the entropy decoder, the size of the alphabet is: the size of the input alphabet of the entropy decoder, or the size of the output alphabet of the entropy decoder; or the minimum symbol probability supported by the entropy decoder; or the renormalization period of the entropy decoder. In some embodiments, the renormalization period can be 8 bits, 16 bits, etc.

[0501] In a possible embodiment, the first parameter is the size of the alphabet; wherein the acquisition unit is further configured to: use the first parameter as the size of the alphabet.

[0502] In one possible embodiment, the first parameter is p, ie, an entropy decoding parameter, including the size of the alphabet M, where M is a function of p.

[0503] In a possible embodiment, the acquisition unit is further used to: acquire M, M=f –1 (p); where f –1 (p) is the inverse function of f(M), where f(M)=p.

[0504] In a possible embodiment, the acquisition unit is further used to: determine a target sub-range in which the first parameter is located, wherein the allowable range of values ​​of the first parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the first parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; use the value of the entropy decoding parameter corresponding to the target sub-range as the value of the entropy decoding parameter; or calculate the value of the entropy decoding parameter based on one or more values ​​of the entropy decoding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

[0505] An embodiment of the present application provides a decoding device, including: a functional unit that implements any of the above encoding methods.

[0506] The embodiment of the present application provides a coding device, comprising: a coding unit, configured to encode an input signal and a first parameter into a bitstream, wherein the first parameter is used to obtain an entropy decoding parameter; and a sending unit, configured to send the bitstream to a decoder. The coding device also includes other functional units to implement any one of the above coding methods.

[0507] An embodiment of the present application provides a decoding device, including: a processing circuit, configured to execute any one of the above decoding methods.

[0508] An embodiment of the present application provides a coding device, including: a processing circuit, configured to execute any one of the above coding methods.

[0509] An embodiment of the present application provides a decoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein when the program is executed by the one or more processors, the program causes the decoder to perform any one of the above-mentioned decoding methods.

[0510] An embodiment of the present application provides an encoder, comprising: one or more processors; a non-transitory computer-readable storage medium coupled to the one or more processors, the storage medium storing a program for execution by the one or more processors, wherein when the program is executed by the one or more processors, the program causes a decoder to perform any one of the above-mentioned encoding methods.

[0511] An embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions. When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute any one of the above-mentioned encoding methods.

[0512] An embodiment of the present application provides a non-transitory computer-readable medium carrying computer instructions, characterized in that when the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or one or more processors to execute any one of the above-mentioned decoding methods.

[0513] An embodiment of the present application provides a non-transitory storage medium, comprising a code stream encoded by any one of the above encoding methods.

[0514] An embodiment of the present application provides a computer program stored in a non-transitory medium and including code instructions. When the computer program is executed on one or more processors, the computer program causes the one or more processors to perform the steps of any one of the above-mentioned encoding methods.

[0515] An embodiment of the present application provides a computer program stored in a non-transitory medium and including code instructions. When the computer program is executed on one or more processors, the computer program causes the one or more processors to perform any step of the above-mentioned decoding method.

[0516] An embodiment of the present application provides a system for sending a code stream, including: at least one storage medium, used to store at least one code stream generated by the encoding method described in the third aspect or any possible embodiment of the third aspect, the fourth aspect or any possible embodiment of the fourth aspect; a video streaming device, used to obtain the code stream from one of the at least one storage medium and send the code stream to a terminal device, wherein the video streaming device includes a content server or a content distribution server.

[0517] In a possible embodiment, it also includes: one or more processors for performing encryption processing on at least one code stream to obtain at least one encrypted code stream; at least one storage medium for storing the encrypted code stream; or one or more processors for converting a code stream in a first format into a code stream in a second format; at least one storage medium for storing the code stream in the second format.

[0518] In a possible embodiment, it also includes: a receiver for receiving a first operation request; one or more processors for determining a target code stream in at least one storage medium in response to the first operation request; and a transmitter for sending the target code stream to a terminal side device.

[0519] In one possible embodiment, one or more processors are also used to: encapsulate the code stream to obtain a transmission stream in a first format; the transmitter is also used to: send the transmission stream in the first format to a terminal side device for display; or send the transmission stream in the first format to a storage space for storage.

[0520] In a possible embodiment, an exemplary method for storing a code stream is provided, the method comprising:

[0521] Obtain a bit stream according to any one of the above encoding methods;

[0522] The code stream is stored in a storage medium.

[0523] Optionally, the method further comprises:

[0524] Perform encryption processing on the code stream to obtain an encrypted code stream;

[0525] The encrypted code stream is stored in a storage medium.

[0526] It will be appreciated that any known encryption method may be used.

[0527] Optionally, the method further comprises:

[0528] The code stream is segmented to obtain a plurality of code stream segments; and the plurality of code stream segments are stored in a storage medium.

[0529] Optionally, the method further comprises:

[0530] At least one backup of the code stream is obtained, and the at least one backup is stored in a storage medium. It should be understood that the at least one backup of the code stream can be stored in a storage medium different from the storage medium storing the original code stream.

[0531] Optionally, the method further comprises:

[0532] Receiving multiple code streams generated according to any one of the above encoding methods;

[0533] Allocating address information or identification information to multiple code streams respectively;

[0534] The code streams are stored in corresponding positions according to the address information or identification information corresponding to the plurality of code streams.

[0535] Optionally, the method further comprises:

[0536] Classifying the code streams to obtain at least two code streams, wherein the at least two code streams include a first code stream and a second code stream;

[0537] The first code stream is stored in a first storage space, and the second code stream is stored in a second storage space.

[0538] Optionally, the method further comprises:

[0539] The video streaming device sends a code stream to the terminal device, wherein the video streaming device may be a content server or a content distribution server.

[0540] In a possible embodiment, an exemplary system for storing a code stream is provided, the system comprising:

[0541] A receiver, configured to receive a code stream generated by any one of the above encoding methods;

[0542] A processor, used for performing encryption processing on the code stream to obtain an encrypted code stream;

[0543] A computer-readable storage medium is used to store an encrypted code stream.

[0544] Optionally, the system includes several storage media, which can be deployed in different locations. Multiple code streams can be stored in different storage media in a distributed manner. For example, the several storage media include: a first storage medium for storing a first code stream; a second storage medium for storing a second code stream.

[0545] Optionally, the system includes a video streaming device, wherein the video streaming device may be a content server or a content distribution server, and the video streaming device is used to obtain a code stream from one of the storage media and send the code stream to the terminal device.

[0546] In a possible embodiment, an exemplary method for converting a bitstream format is provided, the method comprising:

[0547] Receiving a code stream in a first format generated by any one of the above encoding methods;

[0548] Converting a bitstream in a first format into a bitstream in a second format;

[0549] The code stream in the second format is stored in a storage medium.

[0550] Optionally, the method further comprises:

[0551] In response to the access request of the terminal side device, the stored second format code stream is sent to the terminal side device.

[0552] In a possible embodiment, an exemplary system for converting a code stream format is provided, the system comprising:

[0553] A receiver, configured to receive a code stream in a first format generated by any one of the above encoding methods;

[0554] A processor, configured to convert a code stream in a first format into a code stream in a second format;

[0555] The processor is further configured to store the code stream in the second format into a storage medium;

[0556] The storage medium is used to store the code stream in the second format;

[0557] The transmitter is used to respond to the access request of the terminal side device and send the stored second format code stream to the terminal side device.

[0558] In a possible embodiment, an exemplary method for processing a code stream is provided, the method comprising:

[0559] Receiving a transport stream including a video stream and an audio stream, wherein the video stream is generated by any one of the above encoding methods;

[0560] Demultiplex the transport stream, demultiplex the video stream and the audio stream;

[0561] Decoding the video stream using a video decoder to obtain video data;

[0562] The audio stream is decoded by using an audio decoder to obtain audio data.

[0563] Optionally, the method further comprises:

[0564] Synchronize audio data and video data;

[0565] Output the synchronization result to the player for playback.

[0566] Optionally, the method further comprises:

[0567] Decode the code stream to obtain video data or image data;

[0568] performing at least one of brightness mapping, chrominance mapping, resolution adjustment, or format conversion on the video data or the image data,

[0569] Sends video data or image data to a display.

[0570] In a possible embodiment, an exemplary method for sending a code stream according to a user operation request is provided, the method comprising:

[0571] Receiving a first operation request from a terminal device, wherein the first operation request is used to request to play a target video;

[0572] In response to the first operation request, determining a code stream corresponding to the target video in the storage medium, wherein the code stream corresponding to the target video is a code stream generated according to any one of the above encoding methods;

[0573] Send the target code stream to the end-side device.

[0574] Optionally, the method further comprises:

[0575] Encapsulating the code stream to obtain a transport stream in a first format;

[0576] Sending the transport stream in the first format to the terminal side device for display; or

[0577] The transport stream in the first format is sent to the storage space for storage.

[0578] In a possible embodiment, an exemplary system for sending a code stream according to a user operation request is provided, and the system includes:

[0579] A storage medium, used for storing a code stream, wherein the code stream is a code stream generated according to any one of the above encoding methods;

[0580] A receiver, configured to receive a first operation request;

[0581] The processor is configured to determine a target code stream in a storage medium in response to the first operation request;

[0582] The transmitter is used to send the target code stream to the terminal side device.

[0583] Optionally, the processor is further configured to:

[0584] The code stream is encapsulated to obtain a transport stream in a first format; the system also includes a transmitter, which is used to:

[0585] Sending the transport stream in the first format to the terminal side device for display; or

[0586] The transport stream in the first format is sent to the storage space for storage.

[0587] In a possible embodiment, an exemplary method for downloading a code stream is provided, the method comprising:

[0588] Acquire a code stream from a storage medium, wherein the code stream is generated according to any one of the above encoding methods;

[0589] Decode the code stream to obtain the streaming media file;

[0590] Divide the streaming media file into a plurality of streaming media segments;

[0591] Download multiple streaming media segments separately.

[0592] In a possible embodiment, an exemplary system for downloading a code stream is provided, the system comprising:

[0593] An acquisition unit, configured to acquire a code stream from a storage medium, wherein the code stream is generated according to any one of the above encoding methods;

[0594] A decoder is used to decode the code stream to obtain a streaming media file;

[0595] A processor, for dividing a streaming media file into a plurality of streaming media segments;

[0596] The processor is configured to download a plurality of streaming media segments respectively. However, the present invention is not limited to any one of these exemplary embodiments.

[0597] Arithmetic decoding can be performed in parallel, for example by a multi-core decoder. In addition, only parts of arithmetic decoding can be performed in parallel. The method of arithmetic decoding can be implemented as range decoding.

[0598] The arithmetic decoding of the present invention can be easily applied to the encoding of feature maps of neural networks or to the encoding and decoding of traditional images (still images or videos). Neural networks can be used for any purpose, in particular for encoding and decoding of images (still images or moving images), or encoding and decoding of image-related data (such as motion flow or motion vectors or other parameters). Neural networks can also be used for computer vision applications, such as image classification, depth detection, segmentation map determination, identified object recognition, etc.

[0599] Entropy decoding can be performed in parallel, for example by a multi-core decoder. In addition, only parts of the entropy decoding can be performed in parallel. Fig.19 An exemplary scheme of a parallel (e.g., multi-core) encoder 620 is shown. Each of the input data channels 610 can be encoded into a single substream including decoded bits 630 to 633 and tail bits 640 to 643. The length of the substream 650 is indicated. In a parallel processing implementation, the code stream consists of several substreams, which are cascaded in the last step. Each substream needs to be finalized. This is because the substreams are encoded independently of each other, so that the encoding (and decoding) of one substream does not require the encoding (or decoding) of another substream or substreams first.

[0600] An input data channel may refer to a channel obtained by a neural network processing certain data. For example, the input data may be a feature channel such as an output channel of a neural network or a potential representation channel. In an exemplary implementation, the neural network is a deep neural network and / or a convolutional neural network, etc. The neural network can be trained to process images (still images or moving images). The processing can be used for image encoding and reconstruction, or for computer vision, such as object recognition, classification, segmentation, etc. In general, the present invention is not limited to any particular type of task or neural network. On the contrary, the present invention is applicable to encoding any type of data from multiple channels, which are generally understood to be any data source. In addition, the channel can be provided by preprocessing of the source data.

[0601] Implementation in image decoding

[0602] Fig. 20 and Fig.21 One possible deployment is shown in FIG.

[0603] Fig. 20 Schematic block diagram of an exemplary encoder 20 for implementing the technology of the present application is shown. Fig. 20 In the example of , the encoder 20 includes an input terminal 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, a dequantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter 220, a decoded picture buffer (decoded picture buffer, DPB) 230, a mode selection unit 260, an entropy coding unit 270 and an output terminal 272 (or an output interface 272). The entropy decoding unit 270 can implement the arithmetic decoding method or device as described above.

[0604] The mode selection unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a partitioning unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown in the drawing). Fig. 20 The illustrated encoder 20 may also be referred to as a hybrid encoder or an encoder according to a hybrid video / image codec.

[0605] The encoder 20 can be used to receive an image 17 (or image data 17 or point cloud data, motion stream or other type of media data) via an input terminal 201, etc., such as an image of an image sequence forming a video or a video sequence. The received image or image data can also be a preprocessed image 19 (or preprocessed image data 19). For simplicity, it is referred to as image 17 in the following description. Image 17 can also be referred to as a current image or an image to be decoded (specifically, in video decoding, it is to distinguish the current image from other images (such as previously encoded images and / or decoded images) in the same video sequence (i.e., a video sequence that also includes the current image)).

[0606] A (digital) image is or can be considered as a two-dimensional array or matrix of samples with intensity values. The samples in the array can also be called pixels (pixel / pel) (short for picture elements). The number of samples in the horizontal and vertical directions (or axes) of the array or image defines the size and / or resolution of the image. Three color components are usually used to represent colors, that is, an image can be represented by three sample arrays or an image can include three sample arrays. In the RGB format or color space, the image includes corresponding red, green and blue sample arrays. However, in video decoding, each pixel is usually represented in a brightness and chrominance format or color space, for example, the color space is YCbCr, which includes a brightness component represented by Y (sometimes also represented by L) and two chrominance components represented by Cb and Cr. The brightness component Y represents the brightness or grayscale intensity (for example in a grayscale image), while the two chrominance components Cb and Cr represent the chrominance or color information components. Therefore, an image in the YCbCr format includes a brightness sample array (Y) of brightness sample values ​​and two chrominance sample arrays (Cb and Cr) of chrominance values. An image in RGB format can be converted or transformed into YCbCr format and vice versa, a process also referred to as color conversion or transformation. If the image is black and white, the image may include only a luma sample array. Thus, the image may be, for example, a luma sample array in black and white format or a luma sample array and two corresponding chroma sample arrays in 4:2:0, 4:2:2 and 4:4:4 color formats.

[0607] An embodiment of the encoder 20 may include an image segmentation unit ( Fig. 20The image segmentation unit may be used to segment the image 17 into a plurality of (typically non-overlapping) image blocks 203 (not shown). These blocks may also be referred to as root blocks or macroblocks (H.264 / AVC standard) or as coding tree blocks (CTB) or coding tree units (CTU) (H.265 / HEVC and VVC standards). The image segmentation unit may be used to use the same block size for all images in a video sequence and a corresponding grid defining the block size, or to vary the block size between images or subsets or groups of images, and to segment each image into corresponding blocks. The abbreviation AVC stands for Advanced Video Coding.

[0608] In other embodiments, the encoder 20 may be configured to directly receive a block 203 of the image 17, for example, one, several or all blocks constituting the image 17. The image block 203 may also be referred to as a current image block or an image block to be decoded.

[0609] Although the size of the image block 203 is smaller than that of the image 17, like the image 17, the image block 203 is also or can also be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values). In other words, the block 203 may include, for example, one sample array (e.g., a brightness array in the case of the black and white image 17), three sample arrays (e.g., one brightness array and two chrominance arrays in the case of the color image 17), or any other number and / or type of arrays, depending on the color format applied. The number of samples of the block 203 in the horizontal and vertical directions (or axes) defines the size of the block 203. Therefore, a certain image block may be an M×N (M columns×N rows) sample array, or an M×N transform coefficient array, etc.

[0610] exist Fig. 20 In the illustrated embodiment of the encoder 20 , the encoder 20 may be configured to perform encoding on the image 17 block by block, for example, performing encoding and prediction on each block 203 .

[0611] Fig. 20 The illustrated embodiment of the encoder 20 may also partition and / or encode an image using slices (also referred to as video slices), wherein an image may be partitioned or encoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTUs).

[0612] Fig. 20The illustrated embodiment of the encoder 20 may also segment and / or encode an image using tile groups (also referred to as video tile groups) and / or blocks (also referred to as video blocks), wherein one or more tile groups (usually non-overlapping) may be used to segment or encode an image, each tile group may include one or more blocks (e.g., CTU) or one or more blocks, etc., wherein each block may be a rectangular shape, etc., and may include one or more blocks (e.g., CTU), such as complete or partial blocks.

[0613] Fig.21 3 is an example of a decoder 30 for implementing the technology of the present application. The decoder 30 is used to receive the coded image data 21 (e.g., the coded code stream 21) encoded by the encoder 20, for example, to obtain a decoded image 331. The coded image data or code stream includes information for decoding the coded image data, such as data representing image blocks of coded slices (and / or partition groups or partitions or sub-images) and related syntax elements.

[0614] The entropy decoding unit 304 is used to parse the bitstream 21 (or generally referred to as the encoded image data 21) and perform entropy decoding on the encoded image data 21 to obtain quantization coefficients 309 and / or decoded decoding parameters ( Fig.21 The entropy decoding unit 304 may be used to perform a decoding algorithm or scheme corresponding to the encoding scheme described by the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may also be used to provide the inter-frame prediction parameters, intra-frame prediction parameters and / or other syntax elements to the mode application unit 360, and to provide other parameters to other units of the decoder 30. The decoder 30 may receive syntax elements at the video slice level and / or at the video block level. In addition to or as an alternative to the slices and the corresponding syntax elements, block groups and / or blocks and corresponding syntax elements may also be received and / or used. Entropy decoding may implement any of the above-mentioned arithmetic decoding methods or devices.

[0615] The reconstruction unit 314 (eg, adder or summer 314 ) may be configured to add the reconstructed residual block 313 to the prediction block 365 to obtain the reconstructed block 315 in the sample domain, eg, by adding sample values ​​of the reconstructed residual block 313 to sample values ​​of the prediction block 365 .

[0616] Fig.21The embodiment of the decoder 30 shown in the figure can be used to segment and / or decode a picture using slices (also called video slices). A picture can be segmented into one or more (usually non-overlapping) slices or decoded using one or more (usually non-overlapping) slices, each of which can include one or more blocks (e.g., CTUs).

[0617] In an embodiment, Fig.21 The decoder 30 shown may be used to segment and / or decode an image using block groups (also referred to as video block groups) and / or blocks (also referred to as video blocks). An image may be segmented into one or more block groups (usually non-overlapping) or decoded using one or more block groups (usually non-overlapping); each block group may include one or more blocks (e.g., CTU) or one or more blocks, etc.; each block may be rectangular, etc., and may include one or more complete or partial blocks (e.g., CTU), etc.

[0618] Other variations of the decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without the loop filter unit 320. For example, for certain blocks or frames, a non-transform based decoder 30 may directly quantize the residual signal without the inverse transform processing unit 312. In another implementation, the decoder 30 may combine the inverse quantization unit 310 and the inverse transform processing unit 312 into a single unit.

[0619] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current link can be further processed and then output to the next link. For example, after interpolation filtering, motion vector derivation or loop filtering, the processing result of interpolation filtering, motion vector derivation or loop filtering can be further subjected to operations such as clipping or shifting.

[0620] Implementation in hardware and software

[0621] Some further implementations in hardware and software will be described below.

[0622] Reference above Figure 22 to Figure 25 Any encoding device described may provide components for performing the above encoding method and decoding method. Specifically, the processing circuit in any one of these exemplary devices is used to perform the above encoding method and decoding method.

[0623] In the following embodiment of the decoding system 10, according to Fig. 22 and Fig.23 , combined with the above Fig. 20 and Fig.21The encoder 20 and the decoder 30 may be described in conjunction with other encoders and decoders such as encoders and decoders based on neural networks.

[0624] Fig. 22 1 is a schematic block diagram of an exemplary decoding system 10 (eg, a video decoding system 10 or an image decoding system 10 that can use the techniques of the present application). The encoder 20 and the decoder 30 of the decoding system 10 represent examples of devices that can be used to perform techniques according to various examples described in the present application.

[0625] like Fig. 22 As shown, the decoding system 10 includes a source device 12 for providing the encoded image data 21 to a destination device 14 or the like for decoding the encoded image data 13 .

[0626] Source device 12 includes encoder 20 and may additionally (ie, optionally) include image source 16, preprocessor (preprocessing unit) 18 (eg, image preprocessor 18), and communication interface or communication unit 22. Source device 12 may be a cloud server, a content server, or a content distribution server.

[0627] The image source 16 may include or may be: any type of image capture device, such as a camera for capturing real-world images; and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images; or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage device that stores any of the above images.

[0628] In order to distinguish them from the pre-processing unit 18 and the processing performed by the pre-processing unit 18 , the image or the image data 17 may also be referred to as a raw image or raw image data 17 .

[0629] The preprocessor 18 is used to receive (raw) image data 17 and perform preprocessing on the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. The preprocessing performed by the preprocessor 18 may include trimming, color format conversion (e.g., from RGB to YCbCr), color correction or denoising, etc. It can be understood that the preprocessing unit 18 may be an optional component.

[0630] The encoder 20 is used to receive the pre-processed image data 19 and provide encoded image data 21 (for example, Fig. 20 described in further detail).

[0631] The communication interface 22 in the source device 12 may be used to receive the encoded image data 21 and send the encoded image data 21 (or any other processed version) to another device such as the destination device 14 or any other device via the communication channel 13 for storage or direct reconstruction.

[0632] The destination device 14 includes a decoder 30 and may additionally (ie, optionally) include a communication interface or communication unit 28 , a post-processing unit 32 (or post-processing units 32 ), and a display device 34 .

[0633] The communication interface 28 of the destination device 14 is used to receive the encoded image data 21 or (or any other processed version), for example directly from the source device 12, or from any other source (such as a storage device, such as an encoded image data storage device), and to provide the encoded image data 21 to the decoder 30.

[0634] The communication interface 22 and the communication interface 28 can be used to send or receive the encoded image data 21 or the encoded data 13 via a direct communication link (e.g., a direct wired or wireless connection) between the source device 12 and the destination device 14, or via any type of network (e.g., a wired network, a wireless network, any combination of wired and wireless networks, any type of private and public networks, or any type of combination of private and public networks).

[0635] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or use any type of transmission coding or processing to process the encoded image data for transmission on a communication link or a communication network or a transmission medium. For example, the communication interface 22 may be used to encapsulate the encoded image data to obtain a transmission stream in a first format, and send the transmission stream to a terminal side device for display; or send the transmission stream in the first format to a storage space for storage.

[0636] For example, the communication interface 28 (corresponding to the communication interface 22 ) may be used to receive the transmitted data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation process to obtain the encoded image data 21 .

[0637] Both the communication interface 22 and the communication interface 28 can be used as Fig. 22 The unidirectional communication interface represented by the arrow of the communication channel 13 pointing from the source device 12 to the destination device 14 can be used as a bidirectional communication interface, and can be used to send and receive messages, establish connections, confirm and exchange any other information related to the communication link and / or data transmission (such as encoded image data transmission), etc.

[0638] The decoder 30 is used to receive the encoded image data 21 and provide decoded image data 31 or decoded image 31 (for example, according to the above Fig.21 described in further detail).

[0639] The post-processor 32 of the destination device 14 is used to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain post-processed image data 33, such as the post-processed image 33. The post-processing performed by the post-processing unit 32 may include color format conversion (e.g., from YCbCr to RGB), color correction, cropping or resampling, or any other processing, for example, to prepare the decoded image data 31 for display through a display device 34, etc.

[0640] The display device 34 of the destination device 14 is used to receive the post-processed image data 33 to display the image, for example, to a user or viewer. The display device 34 can be or include any type of display for presenting the reconstructed image, such as an integrated or external display or monitor. For example, the display can include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, liquid crystal on silicon (LcoS), a digital light processor (DLP), or any other type of display.

[0641] although Fig. 22 The source device 12 and the destination device 14 are shown as separate devices, but embodiments of the device may also include both devices or the functions of both devices, that is, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions. In such embodiments, the source device 12 or the corresponding functions and the destination device 14 or the corresponding functions may be implemented by the same hardware and / or software, by separate hardware and / or software, or any combination thereof.

[0642] From the above description, it is obvious to those skilled in the art that Fig. 22 The presence and the (precise) functional division of the different units or functions of the source device 12 and / or the destination device 14 shown in FIG. 1 may vary depending on the actual device and application.

[0643] The encoder 20 or the decoder 30, or both the encoder 20 and the decoder 30, may be configured as follows: Fig.23The processing circuit shown is implemented, and the processing circuit is, for example, one or more microprocessors, digital signal processors (digital signal processors, DSP), application-specific integrated circuits (application-specific integrated circuits, ASIC), field-programmable gate arrays (field-programmable gate arrays, FPGA), discrete logic, hardware, video decoding dedicated processors or any combination thereof. The encoder 20 can be implemented by the processing circuit 46 to achieve the combination of Fig. 20 The decoder 30 may be implemented by processing circuit 46 to implement the various modules discussed in conjunction with encoder 20 and / or any other encoder system or subsystem described herein. Fig.21 The various modules discussed in the decoder 30 of the present invention and / or any other decoder system or subsystem described herein. The processing circuitry may be used to perform the various operations described herein. Fig.25 As shown, if these techniques are partially implemented in software, the device can store the software instructions in a suitable non-transitory computer-readable storage medium, and these instructions can be executed in hardware by one or more processors to perform the techniques of the present invention. Either of the encoder 20 and the decoder 30 can be integrated into a single device as part of a combined encoder / decoder (CODEC), such as Fig.23 shown.

[0644] Source device 12 and destination device 14 may include any of a variety of devices, including any type of handheld or fixed device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video game console, a video streaming device (such as a content service server or a content distribution server), a broadcast receiver device, a broadcast transmitter device, etc., and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 may be used for wireless communication. Therefore, source device 12 and destination device 14 may be wireless communication devices.

[0645] In some cases, Fig. 22The video decoding system 10 shown is exemplary only, and the techniques of the present application can be applied to decoding settings (e.g., video / image encoding or video / image decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, the data can be retrieved from a local memory, streamed over a network, etc. The encoding device can encode the data and store the data to a memory, and / or the decoding device can retrieve and decode the data from the memory. In some examples, the encoding and decoding are performed by devices that do not communicate with each other but simply encode the data to a memory and / or retrieve and decode the data from a memory.

[0646] For ease of description, this document describes embodiments of the present invention with reference to high-efficiency video coding (HEVC) or versatile video coding (VVC) (next generation video coding standard) reference software developed by the ITU-T video coding experts group (VCEG) and the ISO / IEC motion picture experts group (MPEG) video coding joint collaboration team (JCT-VC). Those of ordinary skill in the art should understand that embodiments of the present invention are not limited to HEVC or VVC.

[0647] Fig.24 4 is a schematic diagram of a decoding device 400 (video decoding device or image decoding device) according to an embodiment of the present invention. The decoding device 400 is suitable for implementing the disclosed embodiments described herein. In one embodiment, the decoding device 400 may be a decoder (e.g. Fig. 22 Decoder 30) or encoder (such as Fig. 22 encoder 20).

[0648] The decoding device 400 includes: an input port 410 (or input port 410) and a receiving unit (Rx) 420 for receiving data; a processor, a logic unit or a central processing unit (CPU) 430 for processing data; a transmitting unit (Tx) 440 and an output port 450 (or output port 450) for transmitting data; and a memory 460 for storing data. The decoding device 400 may also include an optical-to-electrical (OE) component and an electrical-to-optical (EO) component coupled to the input port 410, the receiver unit 420, the transmitter unit 440 and the output port 450 for inputting or outputting optical or electrical signals.

[0649] The processor 430 is implemented by hardware and software. The processor 430 can be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the input port 410, the receiving unit 420, the sending unit 440, the output port 450, and the memory 460. The processor 430 includes a decoding module 470. The decoding module 470 implements the disclosed embodiments described above. For example, the decoding module 470 performs, processes, prepares, or provides various decoding operations. Therefore, the inclusion of the decoding module 470 provides substantial improvements to the functionality of the video decoding device 400 and affects the transformation of the video decoding device 400 to different states. Alternatively, the decoding module 470 is implemented with instructions stored in the memory 460 and executed by the processor 430.

[0650] Memory 460 may include one or more disks, tape drives, and solid-state hard disks, and may be used as an overflow data storage device to store programs when such programs are selected for execution, as well as to store instructions and data read during program execution. For example, memory 460 may be volatile and / or non-volatile, and may be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (TCAM), and / or a static random-access memory (SRAM).

[0651] Fig.25 is a simplified block diagram of an apparatus 500 according to an exemplary embodiment, which can be used as Fig. 22 Either or both of the source device 12 and the destination device 14 in .

[0652] The processor 502 in the apparatus 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices capable of manipulating or processing information now existing or later developed. Although the disclosed implementations may be implemented by a single processor (e.g., the processor 502), more than one processor may be used to increase speed and efficiency.

[0653] The memory 504 in the apparatus 500 may be a read only memory (ROM) device or a random access memory (RAM) device in one implementation. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 accessed by the processor 502 using the bus 512. The memory 504 may also include an operating system 508 and an application 510, which includes at least one program that causes the processor 502 to perform the methods described herein. For example, the application 510 may include application 1 to application N, which also includes a video decoding application that performs the methods described herein, including encoding and decoding using arithmetic decoding as described above.

[0654] The apparatus 500 may also include one or more output devices, such as a display 518. In one example, the display 518 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0655] Although described here as a single bus, the bus 512 of the device 500 may be composed of multiple buses. In addition, the auxiliary storage device 514 may be directly coupled to other components of the device 500, or may be accessed through a network, and may include a single integrated unit (e.g., a memory card) or multiple units (e.g., multiple memory cards). Therefore, the device 500 can be implemented in a variety of configurations.

[0656] It should be noted that embodiments of the decoding system 10, encoder 20 and decoder 30 (and correspondingly, system 10) and other embodiments described herein can be used for video, still image processing or decoding, that is, independent of the processing or decoding of any previous or consecutive images in video decoding. Typically, in the case where the image processing decoding is limited to a single image 17, only the inter-frame prediction unit 244 (encoder) and 344 (decoder) may be unavailable. All other functions (also called tools or techniques) of the encoder 20 and decoder 30 can also be used for still image processing, such as residual calculation 204 / 304, transformation 206, quantization 208, inverse quantization 210 / 310, (inverse) transformation 212 / 312, segmentation 262 / 362, intra-frame prediction 254 / 354 and / or loop filtering 220, 320, entropy coding 270 and entropy decoding 304.

[0657] Embodiments of encoders 20, decoders 30, etc., and the functions described herein in connection with encoders 20, decoders 30, etc., may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes via a communication medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, corresponding to tangible media such as data storage media, or communication media including any medium that facilitates, for example, the transmission of a computer program from one place to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media or (2) communication media, such as signals or carrier waves. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present invention. A computer program product may include a computer-readable medium.

[0658] Fig. 27 31 is a block diagram of a content providing system 3100 for implementing a content distribution service. The content providing system 3100 includes a capture device 3102, a terminal device 3106, and optionally a display 3126. The capture device 3102 communicates with the terminal device 3106 via a communication link 3104. The communication link may include the above-mentioned communication channel 13. The communication link 3104 includes but is not limited to WIFI, Ethernet, cable, wireless (3G / 4G / 5G), USB, or any combination thereof.

[0659] Capture device 3102 generates data, and can encode data by the coding method shown in the above embodiment.Or, capture device 3102 can distribute data to a streaming server (not shown in the figure), and the server encodes the data and sends the encoded data to terminal device 3106. Capture device 3102 includes but is not limited to a camera, a smart phone or a tablet computer, a computer or a notebook computer, a video conferencing system, a PDA, a vehicle-mounted device, or any combination thereof, etc. For example, capture device 3102 may include source device 12 as described above. When data includes video, the video encoder 20 included in capture device 3102 can actually perform video encoding processing. When data includes audio (i.e., voice), the audio encoder included in capture device 3102 can actually perform audio encoding processing. For some actual scenes, capture device 3102 distributes by multiplexing coded video and audio data together. For other practical scenes, for example, in a video conferencing system, coded audio data and coded video data are not multiplexed. Capture device 3102 distributes coded audio data and coded video data to terminal device 3106 respectively.

[0660] In the content providing system 3100, the terminal device 310 receives and reproduces the encoded data. The terminal device 3106 may be a device having data receiving and recovery capabilities, such as a smart phone or tablet computer 3108, a computer or laptop computer 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a set top box (STB) 3116, a video conferencing system 3118, a video surveillance system 3120, a personal digital assistant (PDA) 3122, a vehicle device 3124 or any combination thereof, or a device capable of decoding the above-mentioned encoded data. For example, the terminal device 3106 may include the destination device 14 as described above. When the encoded data includes video, the video decoder 30 included in the terminal device preferentially performs video decoding. When the encoded data includes audio, the audio decoder included in the terminal device preferentially performs audio decoding processing.

[0661] For a terminal device with a display, such as a smartphone or tablet 3108, a computer or laptop 3110, a network video recorder (NVR) / digital video recorder (DVR) 3112, a TV 3114, a personal digital assistant (PDA) 3122, or a vehicle-mounted device 3124, the terminal device can feed the decoded data to the display of the terminal device. For a terminal device without a display (such as an STB 3116, a video conferencing system 3118, or a video surveillance system 3120), an external display 3126 is connected to receive and display the decoded data.

[0662] Each device in the system can use the image encoding device or the image decoding device shown in the above-mentioned embodiments when performing encoding or decoding.

[0663] Figure Y is a diagram of the structure of an example of the terminal device 3106. After the terminal device 3106 receives the stream from the capture device 3102, the protocol processing unit 3202 analyzes the transmission protocol of the stream. The protocol includes, but is not limited to, real-time streaming protocol (RTSP), hypertext transfer protocol (HTTP), HTTP live streaming protocol (HLS), MPEG-DASH, real-time transport protocol (RTP), real-time messaging protocol (RTMP), or any combination thereof.

[0664] After the protocol processing unit 3202 processes the stream, a stream file is generated. The file is output to the demultiplexing unit 3204. The demultiplexing unit 3204 can divide the multiplexed data into coded audio data and coded video data. As described above, for some actual scenarios, such as in a video conferencing system, the coded audio data and the coded video data are not multiplexed. In this case, the coded data is sent to the video decoder 3206 and the audio decoder 3208 without passing through the demultiplexing unit 3204.

[0665] By demultiplexing, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 3206 includes the video decoder 30 as described in the above embodiment, and decodes the video ES to generate a video frame by the decoding method shown in the above embodiment, and feeds the data to the synchronization unit 3212. The audio decoder 3208 decodes the audio ES to generate an audio frame, and feeds the data to the synchronization unit 3212. Alternatively, before the video frame is fed to the synchronization unit 3212, the video frame can be stored in a buffer (not shown in Figure Y). Similarly, before the audio frame is fed to the synchronization unit 3212, the audio frame can be stored in a buffer (not shown in Figure Y).

[0666] The synchronization unit 3212 synchronizes the video frames and the audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 3212 synchronizes the presentation of the video and audio information. The information may be encoded in the syntax using timestamps regarding the representation of the decoded audio and video data and timestamps regarding the delivery of the data stream itself.

[0667] If subtitles are included in the stream, the subtitle decoder 3210 decodes the subtitles, synchronizes the subtitles with the video frames and the audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 3216.

[0668] The present invention is not limited to the above-mentioned system, and the image encoding device or the image decoding device in the above-mentioned embodiments can be combined with other systems, such as an automobile system.

[0669] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection can be referred to as a computer-readable medium. For example, if the instruction is sent from a website, server or other remote source via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio, microwave, etc.), the definition of the medium includes coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave, etc.). However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other transient media, but refer to non-transient tangible storage media. The disks and optical disks used herein include compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks and blue-ray discs, and the disks usually reproduce data magnetically, while the optical discs reproduce data optically with lasers. The above combinations should also be included in the scope of computer-readable media.

[0670] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided in dedicated hardware and / or software modules for encoding and decoding, or may be combined in a combined codec. In addition, these techniques may be fully implemented in one or more circuits or logic elements.

[0671] The technology of the present invention can be implemented in a variety of devices or apparatuses, including wireless handsets, cloud servers, application servers, integrated circuits (ICs) or a group of ICs (e.g., chipsets). The present invention describes various components, modules, or units to emphasize the functional aspects of the devices used to perform the disclosed technology, but these components, modules, or units do not necessarily require implementation by different hardware units. Instead, as described above, the various units can be combined in a codec hardware unit, or by a collection of interoperable hardware units including one or more processors as described above in combination with appropriate software.

Claims

1. A decoding method, characterized in that: The decoding method is implemented by a decoder, and the decoding method includes: receiving a code stream including encoded data of an input signal and a first parameter; Parsing the code stream to obtain the first parameter; Acquire an entropy decoding parameter according to the first parameter; At least a portion of the input signal is reconstructed based on the entropy coding parameters and the encoded data.

2. The decoding method according to claim 1, characterized in that: The entropy decoding parameter includes at least one of the following: the size of an alphabet of an entropy decoder, the size of the alphabet being: the size of an input alphabet of an entropy encoder, or the size of an output alphabet of an entropy decoder; or the minimum symbol probability supported by the entropy decoder; or The renormalization period of the entropy decoder.

3. The decoding method according to claim 2, characterized in that: When the first parameter is the size of the alphabet, obtaining the entropy decoding parameter according to the first parameter includes: The first parameter is used as the size of the alphabet.

4. The decoding method according to claim 1 or 2, characterized in that: The first parameter is p, and the entropy decoding parameter includes the size of the alphabet M, where M is a function of p.

5. The decoding method according to claim 4, characterized in that: Acquiring the entropy decoding parameter according to the first parameter includes: M=f –1 (p); Among them, f –1 (p) is the inverse function of f(M), where f(M)=p.

6. The decoding method according to claim 5, characterized in that: M satisfies one of the following: M=k a*p+C , where k is a natural number, and a and C are constants; or M = a*p+b, where a and b are constants; or M=p 2 。 7. The decoding method according to claim 6, characterized in that: p=log2(M)–9, M=f -1 (p) = 2 p+9 , where f –1 (p) is the inverse function of f(M), where f(M) = log2(M)–9.

8. The decoding method according to any one of claims 4 to 7, characterized in that: p is indicated using one of the following codes: - Binarization code; or - a unary code; or - truncate the unary code; or -exp-Golomb code.

9. The decoding method according to claim 8, characterized in that: p is indicated using a 0th order exp-Golomb code.

10. The decoding method according to claim 1 or 2, characterized in that: The first parameter includes at least one of: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor.

11. The decoding method according to claim 10, characterized in that: Acquiring the entropy decoding parameter according to the first parameter includes: Determine a target sub-range in which the first parameter is located, wherein the allowable range of values ​​of the first parameter includes multiple sub-ranges, the target sub-range is one of the multiple sub-ranges, each of the multiple sub-ranges includes at least one value of the first parameter, and each of the multiple sub-ranges corresponds to a value of the entropy decoding parameter; using the value of the entropy coding parameter corresponding to the target sub-range as the value of the entropy coding parameter; or The value of the entropy coding parameter is calculated based on one or more values ​​of the entropy coding parameter corresponding to one or more sub-ranges adjacent to the target sub-range.

12. The decoding method according to claim 1, characterized in that: The first parameter is D, and the entropy decoding parameter includes the size of the alphabet M, where M is obtained based on P and D, and where P is the predicted value derived by the decoder.

13. The decoding method according to claim 12, characterized in that: Acquiring the entropy decoding parameter according to the first parameter includes: M=s –1 (D,P); Among them, s –1 (D,P) is the inverse function of s(M,P), where s(M,P)=D.

14. The decoding method according to claim 13, characterized in that: M satisfies one of the following: Where a, b, and c are predefined constants; or M=a1*D+b1*P+c1, where a1, b1 and c1 are predefined constants.

15. The decoding method according to claim 14, characterized in that: M=2 D+log 2 (P) ,D=s(M,P)=log2(M)–log2(P)。 16. The decoding method according to any one of claims 12 to 15, characterized in that: D is indicated using one of the following codes: - Binarization code; or - a unary code; or - truncate the unary code; or -exp-Golomb code.

17. The decoding method according to any one of claims 12 to 16, characterized in that: P is derived according to at least one parameter other than the first parameter carried in the code stream.

18. The decoding method according to claim 17, characterized in that: The at least one parameter other than the first parameter comprises at least one of: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor.

19. The decoding method according to claim 18, characterized in that: P is derived according to the at least one parameter and comprises: Obtaining a rate control parameter beta (β) from the bitstream; Determine a target sub-range in which the acquired β is located; wherein the allowable range of the value of the rate control parameter β is [β_0, β_K], the allowable range [β_0, β_K] is divided into a plurality of sub-ranges, the target sub-range is one of the plurality of sub-ranges; each of the plurality of sub-ranges includes at least one β value, and each of the plurality of sub-ranges corresponds to a P value; Selecting a value corresponding to the target sub-range as the value of P; or The value of P is calculated based on one or more values ​​corresponding to one or more sub-ranges adjacent to the target sub-range.

20. The decoding method according to any one of claims 1 to 19, characterized in that: The decoding method further comprises: The code stream is parsed to obtain a flag, wherein the flag is used to indicate whether the code stream directly carries the entropy decoding parameter.

21. The decoding method according to claim 20, characterized in that: When the flag is equal to the first value, it indicates that the bitstream carries the entropy decoding parameter. In this case, the first parameter is the entropy decoding parameter or the first parameter is a transformation result of the entropy decoding parameter. or When the flag is equal to the second value, it indicates that the bitstream does not carry the entropy decoding parameter, and the entropy decoding parameter is derived by the decoder.

22. The decoding method according to claim 20 or 21, characterized in that: When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P or the transformation result of the difference between M and P, and the first parameter is the difference between M and P or the transformation result of the difference between M and P, wherein M is the size of the input alphabet, and P is the predicted value derived by the decoder.

23. The decoding method according to any one of claims 2 to 22, characterized in that: The entropy decoder is an arithmetic decoder, or a range decoder, or an asymmetric numerical system (ANS) decoder.

24. The decoding method according to any one of claims 1 to 23, characterized in that: Reconstructing at least a portion of the input signal based on the entropy decoding parameters and the encoded data comprises: Obtaining at least one probability model, wherein the probability model of an output symbol is used to indicate a probability of each possible value of the output symbol; entropy decoding one or more bits of the coded data using the at least one probability model and the entropy decoding parameters to obtain one or more output symbols; The at least a portion of the input signal is reconstructed from the one or more output symbols.

25. The decoding method according to claim 24, characterized in that: The probability model depends on the entropy coding parameters.

26. A decoding method, characterized in that: The decoding method is implemented by a decoder, and the decoding method includes: receiving a code stream including coded data and a flag of an input signal; Parsing the bitstream to obtain the flag, where the flag is used to indicate whether the bitstream carries an entropy decoding parameter; Acquire the entropy decoding parameter according to the flag; At least a portion of the input signal is reconstructed based on the entropy coding parameters and the encoded data.

27. The decoding method according to claim 26, characterized in that: The entropy decoding parameter includes at least one of the following: the size of an alphabet of an entropy decoder, the size of the alphabet being: the size of an input alphabet of an entropy encoder, or the size of an output alphabet of an entropy decoder; or the minimum symbol probability supported by the entropy decoder; or The renormalization period of the entropy decoder.

28. The decoding method according to claim 26 or 27, characterized in that: When the flag is equal to the first value, it indicates that the bitstream carries the entropy decoding parameter, or the bitstream carries the transformation result of the entropy decoding parameter; or When the flag is equal to the second value, it indicates that the bitstream does not carry the entropy decoding parameter, but the entropy decoding parameter is derived by the decoder.

29. The decoding method according to claim 28, characterized in that: When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P, or the code stream carries the transformation result of the difference between M and P, wherein M is the entropy decoding parameter and P is the predicted value derived by the decoder.

30. The decoding method according to claim 28, characterized in that: Acquiring the entropy decoding parameter according to the flag includes: When the flag is equal to the first value, the code stream is parsed to obtain a first parameter; when the first parameter is the entropy decoding parameter, Using the first parameter as the entropy decoding parameter; or In the case where the first parameter is the transformation result of the entropy decoding parameter, Obtain entropy decoding parameters according to the first parameters.

31. The decoding method according to claim 28, characterized in that: Acquiring the entropy decoding parameter according to the flag includes: When the flag is equal to the second value, parsing the bitstream to obtain a second parameter, the second parameter comprising at least one of: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; The entropy decoding parameter is derived according to the second parameter.

32. The decoding method according to claim 29, characterized in that: Acquiring the entropy decoding parameter according to the flag includes: When the flag is equal to the third value, parsing the bitstream to obtain a third parameter, wherein the third parameter is the difference between M and P, or the third parameter is a transformation result of the difference between M and P, wherein M is the entropy decoding parameter, and P is a prediction value derived by the decoder; deriving P based on at least one of: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor; Obtain entropy decoding parameters according to the third parameter and P.

33. A coding method, characterized in that: The encoding method is implemented by an encoder, and the encoding method includes: Encoding an input signal and a first parameter into a bitstream, wherein the first parameter is used to obtain an entropy decoding parameter; The code stream is sent to the decoder.

34. The encoding method according to claim 33, characterized in that The entropy decoding parameter includes at least one of the following: the size of an alphabet of an entropy decoder, the size of the alphabet being: the size of an input alphabet of an entropy encoder, or the size of an output alphabet of an entropy decoder; or the minimum symbol probability supported by the entropy decoder; or The renormalization period of the entropy decoder.

35. The encoding method according to claim 34, characterized in that The first parameter is the size of the alphabet.

36. The encoding method according to claim 33 or 34, characterized in that: The first parameter is p, where p is the transformation result of M, and M is the entropy decoding parameter.

37. The encoding method according to claim 36, characterized in that p=f(M), where f(M) is a reversible function.

38. The encoding method according to claim 37, characterized in that f(M) includes: f(M)=a*log k (M)+b, where k is a natural number, and a and b are constants; or f(M) = a*M+b, where a and b are constants; or f(M)=sqrt(M).

39. The encoding method according to claim 38, characterized in that p = log2(M)–9.

40. The decoding method according to any one of claims 4 to 7, characterized in that: p is indicated using one of the following codes: - Binarization code; or - a unary code; or - truncate the unary code; or -exp-Golomb code.

41. The encoding method according to claim 33 or 34, characterized in that: The first parameter comprises at least one of: a rate control parameter, a quantization parameter (QP), an image resolution, a video resolution, a frame rate, a pixel density in a 3D object, or a rate-distortion weighting factor, wherein the first parameter is used by the entropy decoder to derive the entropy decoding parameter.

42. The encoding method according to claim 33 or 34, characterized in that: The first parameter is D obtained according to P and M, wherein M is the entropy decoding parameter and P is the prediction value derived by the decoder.

43. The encoding method according to claim 42, characterized in that D=s(M,P), where s(M,P) is a reversible function.

44. The encoding method according to claim 43, characterized in that s(M,P) includes: s(M,P)=a*log k (P)+b*log k (M)–c, where k is a natural number and a, b, and c are constants; or s(M,P)=a*M+b*P+c, where a, b and c are constants.

45. The encoding method according to claim 44, characterized in that D=s(M,P)=log2(P)–log2(M).

46. ​​The encoding method according to any one of claims 42 to 45, characterized in that D is indicated using one of the following codes: - Binarization code; or - a unary code; or - truncate the unary code; or -exp-Golomb code.

47. The encoding method according to any one of claims 33 to 46, characterized in that The encoding method further comprises: A flag is encoded into the bitstream, wherein the flag is used to indicate whether the bitstream directly carries the entropy decoding parameter.

48. The encoding method according to claim 47, characterized in that When the flag is equal to the first value, it indicates that the bitstream carries the entropy decoding parameter, the first parameter is the entropy decoding parameter or the first parameter is a transformation result of the entropy decoding parameter; or When the flag is equal to the second value, it indicates that the bitstream does not carry the entropy decoding parameter, but the entropy decoding parameter is derived by a decoder.

49. The encoding method according to claim 47 or 48, characterized in that When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P, or the code stream carries the transformation result of the difference between M and P, wherein M is the entropy decoding parameter and P is the predicted value derived by the decoder.

50. The encoding method according to any one of claims 33 to 49, characterized in that: The method further comprises: Obtaining a minimum value and a maximum value of a latent space element of the entropy encoder, wherein the latent space element is a result of processing the input signal; Get the size of the alphabet as follows: M=ceil(max{y}–min{y}) or M=2^(ceil(log2(max{y}–min{y}))), Wherein, ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, min{y} represents the minimum value of the latent space element, and M represents the size of the alphabet.

51. The encoding method according to any one of claims 33 to 49, characterized in that The method further comprises: Get at least two values ​​near M0, where M0=ceil(max{y}–min{y}) or M0=2^(ceil(log2(max{y}–min{y}))); Calculating a loss function of the at least two values; Selecting the value with the smallest loss function among the at least two values ​​as the size of the alphabet; Wherein, ceil(x) is the smallest integer greater than x, max{y} represents the maximum value of the latent space element, and min{y} represents the minimum value of the latent space element.

52. A coding method, characterized in that The encoding method is implemented by an encoder, and the encoding method includes: Encoding an input signal and a flag into a bitstream, wherein the flag is used to indicate whether the bitstream directly carries entropy decoding parameters; The code stream is sent to the decoder.

53. The encoding method according to claim 33, characterized in that: The entropy decoding parameter includes at least one of the following: the size of an alphabet of an entropy decoder, the size of the alphabet being: the size of an input alphabet of an entropy encoder, or the size of an output alphabet of an entropy decoder; or the minimum symbol probability supported by the entropy decoder; or The renormalization period of the entropy decoder.

54. The encoding method according to claim 52 or 53, characterized in that When the flag is equal to the first value, it indicates that the bitstream carries the entropy decoding parameter, or the bitstream carries the transformation result of the entropy decoding parameter; or When the flag is equal to the second value, it indicates that the bitstream does not carry the entropy decoding parameter, but the entropy decoding parameter is derived by a decoder.

55. The encoding method according to any one of claims 52 to 54, characterized in that When the flag is equal to the third value, it indicates that the code stream carries the difference between M and P, or the code stream carries the transformation result of the difference between M and P, wherein M is the entropy decoding parameter and P is the predicted value derived by the decoder.

56. The encoding method according to any one of claims 52 to 55, characterized in that The method further comprises: When the flag is equal to the first value, a first parameter is encoded into the bitstream, wherein the first parameter is the entropy decoding parameter or the first parameter is a transformation result of the entropy decoding parameter.

57. The encoding method according to any one of claims 52 to 55, characterized in that The method further comprises: When the flag is equal to the third value, a third parameter is encoded into the bitstream, wherein the third parameter is the difference between M and P, or the third parameter is a transformation result of the difference between M and P, wherein M is the entropy decoding parameter and P is the predicted value derived by the decoder.

58. A decoding device, characterized in that: include: Processing circuitry for performing the steps of a method according to any one of claims 1 to 25 or any one of claims 26 to 32.

59. A coding device, characterized in that include: Processing circuitry for performing the steps of a method according to any one of claims 33 to 51 or any one of claims 52 to 57.

60. A decoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium coupled to the one or more processors, wherein the storage medium stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the program causes the decoder to perform a method according to any one of claims 1 to 25 or any one of claims 26 to 32.

61. An encoder, characterized in that include: one or more processors; A non-transitory computer-readable storage medium coupled to the one or more processors, wherein the storage medium stores a program executed by the one or more processors, wherein when the program is executed by the one or more processors, the program causes the encoder to perform a method according to any one of claims 33 to 51 or any one of claims 52 to 57.

62. A non-transitory computer-readable medium carrying computer instructions, characterized in that: When the computer instructions are executed by a computer device or one or more processors, the computer instructions cause the computer device or the one or more processors to perform the method according to any one of claims 1 to 57.

63. A non-transitory storage medium, characterized in that: Comprising a code stream encoded by a method according to any one of claims 33 to 51 or any one of claims 52 to 57.

64. A computer program, characterized in that The computer program is stored in a non-transitory medium and comprises code instructions which, when executed on one or more processors, cause the one or more processors to perform the steps of the method according to any one of claims 1 to 57.

65. A system for sending code streams, characterized in that: include: at least one storage medium for storing at least one code stream generated by the method according to any one of claims 33 to 51 or any one of claims 52 to 57; A video streaming device is used to obtain a code stream from one of the at least one storage medium and send the code stream to a terminal device, wherein the video streaming device includes a content server or a content distribution server.

66. The system according to claim 65, characterized in that Also includes: One or more processors, configured to perform encryption processing on at least one code stream to obtain at least one encrypted code stream; The at least one storage medium is used to store the encrypted code stream; or The one or more processors are configured to convert a code stream in a first format into a code stream in a second format; The at least one storage medium is used to store the code stream in the second format.

67. The system according to claim 65 or 66, characterized in that Also includes: A receiver, configured to receive a first operation request; The one or more processors are configured to determine a target bitstream in the at least one storage medium in response to the first operation request; A transmitter is used to send the target code stream to the terminal side device.

68. The system according to claim 67, characterized in that The one or more processors are further configured to: Encapsulating the code stream to obtain a transport stream in a first format; The transmitter is also used for: Sending the transport stream in the first format to a terminal device for display; or The transport stream in the first format is sent to the storage space for storage.

Images