Image coding method and system based on generalized Gaussian model

Abstract

The invention discloses an image coding method and system based on a generalized Gaussian model, and the related method comprises a coding part and a decoding part, the coding part comprises transformation, quantization and entropy coding, the decoding part comprises entropy decoding, inverse quantization and inverse transformation, and for image lossy coding, the inverse quantization and inverse transformation are carried out. Introducing a generalized Gaussian model to carry out modeling on probability distribution of the feature map in the encoding process, and carrying out entropy encoding and entropy decoding on the feature map according to generalized Gaussian model parameters obtained through modeling; for image lossless coding, a generalized Gaussian model is introduced to carry out modeling on probability distribution of pixels or lossless transformation coefficients in image lossless compression, and entropy coding and entropy decoding are carried out according to generalized Gaussian model parameters obtained through modeling. According to the scheme provided by the invention, the generalized Gaussian model is introduced into an image coding scheme, so that more accurate probability estimation can be carried out, the code rate required by image compression is saved, the compression performance is improved, and relatively low complexity is maintained.

Description

Technical Field

[0001] The present invention relates to the field of image coding technology, and in particular to an image coding method and system based on a generalized Gaussian model. Background Art

[0002] In applications such as image storage and transmission, it is usually necessary to encode the image to reduce storage capacity and transmission bandwidth. Image coding includes two steps: encoding and decoding. Coding usually includes three steps: transformation, quantization and entropy coding. The first step is to transform the image from the pixel domain to the feature domain to obtain feature elements; the second step is quantization to discretize the feature elements to reduce the amount of information they contain; the third step is entropy coding to entropy encode the quantized feature elements to obtain a bitstream. Decoding corresponds to the encoding operation, and the reconstructed image is obtained through entropy decoding, inverse quantization and inverse transformation in sequence. The end-to-end image lossy coding method uses a neural network to construct each module and uses a rate-distortion function for end-to-end optimization.

[0003] A typical coding framework for end-to-end image lossy coding method is as follows: Figure 1 At the encoding end, the original image x is input into the analysis transformation (corresponding to the transformation described above) to obtain a three-dimensional feature map y. On the one hand, the feature map is output through the side information extraction module to output the side information z. The quantized side information Enter the probability estimation module to output the quantized feature map The probability distribution parameter ψ y , where z is also a three-dimensional feature map, and the number of feature elements it contains is usually less than y. On the other hand, the feature map y is input into the quantization module to obtain the quantized feature map The entropy coding module is based on the estimated probability distribution parameter ψ y Calculate the corresponding cumulative distribution function and quantize the feature map Each characteristic element in is entropy encoded to obtain a compressed code stream. Also according to its probability distribution parameter ψ z The constructed cumulative distribution function is entropy encoded and written into the bitstream. At the decoding end, first decode Then according to the side information Output the probability distribution of the feature map to be decoded. The entropy decoding module solves the value of each feature element in turn according to the probability distribution of each feature element, thereby decoding and obtaining the feature map The feature map The input image reconstruction module performs synthesis transformation and outputs the reconstructed image In addition to using side information to model the conditional probability of feature maps, other methods can also be used to model the feature maps. Perform conditional probability modeling, such as autoregressive models.

[0004] In entropy coding or decoding The cumulative distribution function required by the entropy encoder needs to be dynamically calculated. The more complex the probability model selected, the higher the computational complexity. The following introduces two commonly used solutions in the prior art.

[0005] The first type of scheme: image coding scheme based on Gaussian model.

[0006] In this scheme, a simpler parameterized probability model (i.e., Gaussian model) is used to quantize the feature map Probabilistic modeling. The Gaussian model uses two parameters, mean μ and scale σ, to describe the probability distribution, and its cumulative distribution function is: Where x is the input of the cumulative distribution function and the erf function (error function) is defined as A represents the input of the erf function, π is the symbol of pi, d is the symbol of the derivative, e is a natural constant, and t represents the integral variable. During encoding and decoding, the probability estimation module in the end-to-end image compression algorithm calculates the feature map The probability distribution parameters μ and σ are used to construct the cumulative distribution function for entropy coding. The parameters μ, σ and feature map The same size.

[0007] In one implementation, the feature map is dynamically calculated based on the μ and σ parameter values ​​and the form of the Gaussian cumulative distribution function The cumulative distribution function table corresponding to each element in is input into the feature map in the entropy coding or entropy decoding module Encode or decode each element in . This method requires calculating the erf function, which introduces additional complexity; it introduces floating point calculation errors that are not conducive to cross-platform encoding and decoding consistency; and if the feature map The range of values ​​is very large, and dynamically constructing a cumulative distribution function table for each element will consume a lot of memory / video memory.

[0008] In another implementation, a series of discrete parameter values ​​are sampled from the range of the parameters μ and σ, and a set of cumulative distribution function lookup tables are pre-built and stored using these parameter values ​​for sharing at the codec end. During encoding and decoding, the parameters μ and σ output by the probability estimation module are mapped to the index I of the corresponding cumulative distribution function table, and the entropy coding module compares the feature map with the pre-stored cumulative distribution function lookup table and the index I. Encode or decode each element in. Among them, the index I and the feature map This method consumes some storage, but does not require dynamic construction of the cumulative distribution function table and does not introduce floating point calculation errors. Usually, the number of pre-stored lookup tables is significantly less than the feature map when encoding and decoding the image. The number of elements in the image is small, so the memory / video memory consumed will also be small.

[0009] Because Gaussian distribution is simple and efficient, it is widely used in end-to-end image compression schemes. For example, the following three articles:

[0010] Article 1: D.Minnen, J.Ballé, and G.Toderici, "Joint autoregressive and hierarchical priors for learned image compression," in Advances in neuralinformation processing systems, 2018, pp.10794-10803.

[0011] Article 2: J.Ballé, J.Nick, and D.Minnen. "Integer networks for datacompression with latent-variable models." International Conference on LearningRepresentations.2018.

[0012] Article 3: H.Sun, L.Yu and J.Katto, "Learned Image Compression with Fixed-point Arithmetic," 2021Picture Coding Symposium(PCS), Bristol, United Kingdom, 2021, pp.1-5, doi:10.1109 / PCS50896.2021.9477496.

[0013] The second type of scheme: image coding scheme based on mixed Gaussian model.

[0014] A probability model with higher degrees of freedom, that is, more parameters, can fit the probability distribution of the feature map more accurately, but the complexity will also be higher.

[0015] The mixed Gaussian model uses the weighted sum of multiple Gaussian models to describe the probability distribution, and its cumulative distribution function is: Where Φ is the cumulative distribution function of the standard Gaussian distribution, n is the number of Gaussian models, and p l is the weight of the lth Gaussian model, μ l ,σ l is the parameter of the lth Gaussian model. During encoding and decoding, the probability estimation module in end-to-end image compression calculates the feature map The probability distribution parameters of are used to construct the cumulative distribution function for entropy coding. l ,μ l ,σ l}, l = 1...n and feature map The same size.

[0016] In one implementation, according to the parameter p l ,μ l ,σ l Dynamically calculate feature maps in the form of Gaussian cumulative distribution function The cumulative distribution function table corresponding to each element in is input into the feature map in the entropy encoding and decoding module In another implementation, a set of cumulative distribution function lookup tables of Gaussian distribution are pre-built and shared at the encoding and decoding ends, as described in the above-mentioned solution 1. During encoding and decoding, the parameter μ output by the probability estimation module is l ,σ l is the index I mapped to the corresponding cumulative distribution function table l To obtain the cumulative distribution function of the lth Gaussian component, the cumulative distribution function table of each Gaussian component is obtained in this way, and then the cumulative distribution function table of each Gaussian component is weightedly summed to obtain the cumulative distribution function table of the mixed Gaussian model.

[0017] The mixed Gaussian model can provide more accurate probability modeling than the Gaussian model, but its complexity is also higher. For example, the following two articles introduce related solutions:

[0018] Article 4: Z.Cheng, H.Sun, M.Takeuchi and J.Katto, "Learned Image CompressionWith Discretized Gaussian Mixture Likelihoods and Attention Modules," in IEEE / CVF Conference on Computer Vision and Pattern Recognition(CVPR), Seattle, WA, USA, 2020, pp.7936-7945, doi:10.1109 / CVPR42600.2020.00796.

[0019] Article 5: [4] D. He, Z. Yang, Y. Chen, Q. Zhang, H. Qin, Y. Wang. Post-training quantization for cross-platform learned image compression. arXiv preprintarXiv:2202.07513.

[0020] Among the above two types of schemes, the image coding scheme based on the Gaussian model is simple and efficient, but the compression performance needs to be improved. The image coding scheme based on the mixed Gaussian model can accurately perform probability modeling, but the complexity is relatively high; therefore, it is necessary to develop an image coding scheme that can perform more accurate probability estimation, maintain low complexity, and improve compression performance. Summary of the invention

[0021] The purpose of the present invention is to provide an image encoding method and system based on a generalized Gaussian model to solve the technical problems existing in the existing solutions.

[0022] The objective of the present invention is achieved through the following technical solutions:

[0023] An image coding method based on a generalized Gaussian model, comprising:

[0024] When applied to lossy image coding, a generalized Gaussian model is introduced to model the probability distribution of feature maps in the coding process, and entropy coding and entropy decoding are performed on the feature maps according to the generalized Gaussian model parameters obtained by modeling.

[0025] When applied to lossless coding of images, a generalized Gaussian model is introduced to model the probability distribution of image pixels or feature maps generated by lossless transformation of the image during the encoding process. According to the generalized Gaussian model parameters obtained by the modeling, entropy encoding and entropy decoding are performed on the image pixels or feature maps generated by lossless transformation.

[0026] An image coding system based on a generalized Gaussian model, the system is used to implement the aforementioned method, comprising: an end-to-end image lossy coding model applied to image lossy coding, and an end-to-end image lossless coding model applied to image lossless coding;

[0027] The end-to-end image lossy coding model is used to introduce a generalized Gaussian model to model the probability distribution of feature maps in the encoding process, and perform entropy encoding and entropy decoding on the feature maps according to the generalized Gaussian model parameters obtained by the modeling;

[0028] An end-to-end image lossless coding model, wherein a generalized Gaussian model is introduced to model the probability distribution of image pixels or feature maps generated by lossless transformation during the image encoding process, and entropy encoding and entropy decoding are performed on image pixels or feature maps generated by lossless transformation according to the generalized Gaussian model parameters obtained by the modeling.

[0029] It can be seen from the technical solution provided by the present invention that introducing the generalized Gaussian model into the image coding scheme can perform more accurate probability estimation, save the bit rate required for image compression, improve compression performance, and maintain low complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0031] Figure 1 A schematic diagram of an end-to-end image coding method based on deep learning provided as the background technology of the present invention;

[0032] Figure 2 A schematic diagram of an image encoding method based on a generalized Gaussian model provided by an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of the cumulative distribution function of the generalized Gaussian distribution provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0035] First, the terms that may be used in this article are explained as follows:

[0036] The terms "include", "comprises", "contains", "has" or other descriptions with similar semantics should be interpreted as non-exclusive inclusion. For example, including certain technical feature elements (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or products, etc.) should be interpreted as including not only certain technical feature elements explicitly listed, but also other technical feature elements known in the art that are not explicitly listed.

[0037] The term "consisting of..." means excluding any technical feature elements not explicitly listed. If this term is used in a claim, it will make the claim closed, so that it does not contain technical feature elements other than the technical feature elements explicitly listed, except for the conventional impurities related to them. If this term only appears in a clause of a claim, it only limits the elements explicitly listed in the clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0038] The following is a detailed description of an image coding method and system based on a generalized Gaussian model provided by the present invention. The contents not described in detail in the embodiments of the present invention belong to the prior art known to professionals in the field. If no specific conditions are specified in the embodiments of the present invention, the conventional conditions in the field or the conditions recommended by the manufacturer are followed. If the manufacturer of the instruments used in the embodiments of the present invention is not specified, they are all conventional products that can be purchased commercially.

[0039] Embodiment 1

[0040] The embodiment of the present invention provides an image coding method based on a generalized Gaussian model, which is similar to the solution described in the background technology, and also includes two parts: coding and decoding. In the image lossy coding, the coding part includes transformation, quantization and entropy coding, and the decoding part includes entropy decoding, inverse quantization and inverse transformation. Figure 2 As shown. Image lossless coding includes: lossless coding of image pixels, the coding part includes entropy coding, and the decoding part includes entropy decoding; or lossless coding of feature maps generated by lossless transformation of the image, the coding part includes transformation and entropy coding, and the decoding part includes entropy decoding and inverse transformation. The method provided by the present invention introduces a generalized Gaussian model for modeling probability distribution, which can take into account both complexity and accuracy of probability estimation, save bit rate, and improve compression performance; it is applicable to lossless coding and lossy coding, specifically:

[0041] (1) When applied to lossy image coding, a generalized Gaussian model is introduced to the feature map in the encoding process. and The probability distribution of is modeled, and the feature map is and Perform entropy encoding and entropy decoding.

[0042] In the embodiment of the present invention, the feature map in the encoding process includes two parts, one of which is a quantized feature map y obtained by quantizing the feature map y obtained by lossy transformation in the encoding part. The other part is the edge information extracted from the feature graph y and They can all be collectively referred to as feature graphs. Modeling probability distributions using generalized Gaussian models, including: The set of characteristic elements for each channel Using a set of parameters of the generalized Gaussian model {β k ,μ k ,α k}Modeling probability distribution, k is the index of the channel, β, μ, α correspond to shape, mean, and scale respectively; for the quantized feature map Conditional Entropy Model Prediction Using Generalized Gaussian Model The parameters of the generalized Gaussian model corresponding to each characteristic element in , and then the generalized Gaussian model is used to model the probability distribution.

[0043] Based on the above method, the quantized feature map is obtained With side information After obtaining the generalized Gaussian model parameters, their characteristic elements can be entropy encoded and decoded respectively.

[0044] (2) Lossless coding of images requires entropy coding of image pixels or feature maps generated by lossless transformation of images. When applied to lossless coding of images, a generalized Gaussian model is introduced to model the probability distribution of image pixels or feature maps generated by lossless transformation of images during the encoding process. The generalized Gaussian model parameters of each pixel can be obtained using methods such as side information or autoregressive models, which are not limited here. Based on the generalized Gaussian model parameters obtained by modeling, entropy coding and entropy decoding are performed on the image pixels or feature maps generated by lossless transformation of images.

[0045] In an embodiment of the present invention, a conditional entropy model based on a generalized Gaussian model is used to perform probabilistic modeling on feature maps generated by pixels or lossless transformation in image lossless compression, and conditional entropy models of different complexities are realized by limiting the diversity of generalized Gaussian model parameters predicted by the conditional entropy model.

[0046] Afterwards, entropy encoding and entropy decoding are also performed based on the generalized Gaussian model parameters obtained through modeling.

[0047] Considering that the processes involved in lossy image coding and lossless image coding can be implemented by referring to conventional technologies, they will not be described in detail.

[0048] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the method provided by the embodiment of the present invention is described in detail with reference to specific embodiments below.

[0049] 1. Overall introduction of the plan.

[0050] This section first introduces the generalized Gaussian probability model, the conditional entropy model based on the generalized Gaussian model, and the related entropy coding and entropy decoding schemes; then, on this basis, the related lossy and lossless coding schemes are introduced.

[0051] 1. Generalized Gaussian probability model and corresponding entropy coding and decoding schemes.

[0052] In the embodiment of the present invention, the generalized Gaussian model uses three parameters, shape β, mean μ and scale α, to describe the probability distribution, and its cumulative distribution function is: like Figure 3 As shown, x is the input of the cumulative distribution function, sgn is the sign function, P is the incomplete gamma function under regularization, Where a and b are the inputs of the incomplete gamma function under regularization, e is a natural constant, d is the derivative sign, and t is the integral variable. is the gamma function.

[0053] When the scale parameter α = 2, the generalized Gaussian model degenerates into a Gaussian model. The generalized Gaussian model can be used to perform probabilistic modeling on data and optimize the three parameters of shape β, mean μ and scale α to fit the data distribution.

[0054] In an end-to-end image lossy coding scheme, a generalized Gaussian probability model can be used to construct the probability distribution of feature map elements. For example, for side information The set of characteristic elements for each channel k is the index of the channel, and a set of parameters of the generalized Gaussian model {β k ,μ k ,α k}Modeling probability distribution. In the actual encoding and decoding process, the obtained parameters are used to construct the corresponding cumulative distribution function table for each channel All elements in are entropy encoded and decoded. The constructed cumulative distribution function table can be pre-stored in the model parameters.

[0055] 2. Conditional entropy model based on generalized Gaussian model and corresponding entropy coding and entropy decoding schemes.

[0056] In the image coding scheme, generalized Gaussian can be applied to the conditional entropy model, that is, using methods such as side information or autoregressive model to quantize image pixels or feature maps (this feature map can be the feature map after quantization when the image is lossy encoded) The conditional entropy model based on the generalized Gaussian model predicts the pixels or feature maps of the image to be encoded (this feature map can be the quantized feature map when the image is lossy encoded) through side information or autoregressive models. It can also be the parameters of the generalized Gaussian model corresponding to each element of the feature map generated by lossless transformation, and then use the generalized Gaussian model to probabilistically model the image pixels or feature map elements. It is possible to predict only part of the parameters of the generalized Gaussian model and keep the other part fixed, without any restriction. For example, the mean parameters of the generalized Gaussian model corresponding to all pixels or feature elements can be fixed to zero, and the conditional entropy model only predicts the shape and scale parameters.

[0057] In the actual encoding and decoding process, after obtaining the generalized Gaussian model parameters corresponding to the image pixels or feature elements, these parameters are used to construct a cumulative distribution function table to perform entropy encoding and decoding. The present invention provides two implementation methods.

[0058] (1) In one implementation, after obtaining the generalized Gaussian model parameters corresponding to each pixel or feature element, a cumulative distribution function table is constructed through the cumulative distribution function of the generalized Gaussian to perform entropy encoding and decoding. The specific operation is as follows:

[0059] (1.1) Entropy coding: Combine the parameters of the generalized Gaussian model corresponding to each pixel or feature element, and the cumulative distribution function of the generalized Gaussian model, and construct a corresponding cumulative probability distribution table for each pixel or feature element in a certain length interval; then use an entropy encoder (for example, an arithmetic encoder) to entropy encode the pixels or feature elements to be encoded in sequence into the bitstream using the cumulative probability distribution table corresponding to the pixels or feature elements. The interval length can be set according to the value range of the quantized pixel or feature element to be encoded; the feature elements described in this section are quantized feature maps. The elements in , or the elements in the feature map produced by a lossless transformation.

[0060] (1.2) Entropy decoding: Use the same process as entropy coding to construct a cumulative probability distribution table corresponding to pixels or feature elements, and use an entropy decoder (e.g., an arithmetic decoder) to use the cumulative probability distribution table corresponding to pixels or feature elements to decode pixels or feature elements from the bitstream in sequence.

[0061] (2) The method in (1) above needs to dynamically construct a cumulative distribution function table during entropy encoding and decoding, which will introduce some computational complexity and video memory overhead. In another implementation method, before deploying the codec, a series of parameter samples are sampled from the value range of the generalized Gaussian model parameters, and a cumulative distribution function lookup table corresponding to these parameter samples is constructed and stored at the encoding and decoding ends. During entropy encoding and decoding, the generalized Gaussian model parameters corresponding to the pixels or feature elements are first mapped to the sampled parameter samples to obtain the index of the corresponding cumulative distribution function table. The entropy codec uses the cumulative distribution function lookup table and the index to encode and decode the pixels or feature elements. The specific operations are as follows:

[0062] (2.1) Lookup table construction: Before deploying the codec, sample N parameter samples {β n ,μ n ,α n}, n=1...N; β n ,μ n ,α n represents the nth parameter sample; for each parameter sample, the corresponding cumulative probability distribution table is constructed using the cumulative distribution function of the generalized Gaussian model, and finally the lookup table {CDF n}, n = 1…N; where CDF n is the cumulative probability distribution table corresponding to the nth sample. These lookup tables are stored in the encoder and decoder and shared between the encoder and decoder.

[0063] (2.2) Entropy coding: For each pixel or feature element, the parameters of the generalized Gaussian model are mapped to the nearest parameter sample, and the index I of the cumulative probability distribution table of the corresponding parameter sample is obtained. Then, the entropy encoder is used to calculate the cumulative probability distribution table according to the index I and the lookup table {CDF n} Carry out entropy coding on pixels or feature elements and write them into the bitstream;

[0064] (2.3) Entropy decoding: Use the same process as entropy coding to obtain the index of the cumulative probability distribution table corresponding to each pixel or feature element, and use the entropy decoder to calculate the cumulative probability distribution table according to the index and the lookup table {CDF n}Decode pixels or feature elements from the code stream one by one.

[0065] 3. End-to-end image lossy coding scheme based on generalized Gaussian model.

[0066] A typical framework for end-to-end image lossy compression is Figure 1 As shown. The generalized Gaussian model can be directly applied to modeling side information The conditional entropy model based on the generalized Gaussian model can be applied to the quantized feature map of the model The probability distribution of .

[0067] (1) Structural plan.

[0068] When the image is lossy encoded, the corresponding image encoding method is implemented through an end-to-end image lossy coding model.

[0069] (1.1) Side Information Probabilistic modeling.

[0070] Side Information Usually a three-dimensional feature map, for side information The set of characteristic elements for each channel k is the index of the channel, and a set of parameters of the generalized Gaussian model {β k ,μ k ,α k}Modeling probability distributions.

[0071] (1.2) Quantized feature map Probabilistic modeling.

[0072] Quantized feature map Usually it is a three-dimensional feature map, which can be probabilistically modeled using a conditional entropy model based on a generalized Gaussian model. By limiting the diversity of the generalized Gaussian model parameters predicted by the conditional entropy model, models of different complexity can be implemented. For example, limiting the quantized feature map All elements in the model share a shape parameter, and the conditional entropy model only needs to predict the mean and scale parameters. Below are three conditional entropy models based on the generalized Gaussian model with different complexities.

[0073] (1.2.1) Model-level shape parameters: constraining quantized feature maps All feature elements in the same graph share a shape parameter β0, and the quantized feature map is predicted by the conditional entropy model. The mean and scale parameter of each feature element.

[0074] (1.2.2) Channel-level shape parameters: constraining quantized feature maps All feature elements of each channel share a shape parameter, and the quantized feature map is predicted by the conditional entropy model. The mean and scale parameter of each feature element.

[0075] (1.2.3) Element-level shape parameters: quantized feature maps Each feature element has a separate shape parameter, and the quantized feature map is predicted by the conditional entropy model. Shape, mean, and scale parameters for each feature element.

[0076] (2) Entropy coding and entropy decoding.

[0077] Get the opposite side information Quantized feature map After the parameters of the generalized Gaussian model of probability modeling are obtained, the above scheme in Section 1 is used to The characteristic elements in are entropy encoded and decoded according to the scheme in Section 2 above. The characteristic elements in are entropy encoded and decoded.

[0078] (3) Model training plan.

[0079] In the embodiment of the present invention, the end-to-end image lossy coding model is optimized using an unconstrained rate-distortion loss function L1, and the loss function L1 is in the following form:

[0080] L1=R1+λD;

[0081] Among them, R1 is the bit rate, which is usually measured by bpp, i.e. bits per pixel; D is the distortion, which is usually measured by MSE, i.e. mean square error, etc., and λ is the Lagrange multiplier, which is used to control the trade-off between bit rate and distortion. The calculation scheme of bit rate and distortion can be implemented with reference to conventional technology, and the present invention will not elaborate on it.

[0082] During the training process, since we need to deal with the problem of non-differentiable quantization, we usually add uniform noise to estimate the bit rate. The bit rate term used in training is as follows:

[0083]

[0084] in, is obtained by adding uniform noise in the interval [-0.5, 0.5) to the unquantized feature map y (that is, the feature map obtained by transformation), F is the cumulative distribution function of the generalized Gaussian model, for An element in i ,μ i ,α i are the corresponding generalized Gaussian model parameters.

[0085] Since adding uniform noise during training is different from the quantization rounding operation used during testing, there will be inconsistency between training and testing. In order to reduce the impact of inconsistency between training and testing on the generalized Gaussian model, the present invention proposes an improved training method, including two techniques, one of which is a scale parameter lower bound dependent on the shape parameter, and the other is a joint gradient rectification of the shape and scale parameters.

[0086] (3.1) A lower bound on the scale parameter that depends on the shape parameter.

[0087] Since some small scale parameters can cause large inconsistencies in training and testing, setting a lower bound for the scale parameter during training can effectively alleviate the negative impact of the inconsistency in training and testing. In addition, different shape parameters correspond to different ranges of scale parameters that cause large inconsistencies in training and testing, so it is proposed to set a lower bound of the scale parameter that depends on the shape parameter.

[0088] Set the scale parameter lower bound α β , the setting method is as follows:

[0089]

[0090] When the scale parameter is small enough that the probability of 0 F(0.5)-F(-0.5) is close to 1, it is meaningless to continue to reduce the scale parameter, so the lower bound is set as shown in the above formula. Setting a smaller lower bound cannot fully remove the inconsistency between training and testing, and setting a larger lower bound will risk making the estimated distribution unable to approach the true distribution.

[0091] (3.2) Joint gradient rectification of shape and scale parameters.

[0092] Under the same scale parameter, the larger the shape parameter, the smaller the entropy of the corresponding variable. Setting a lower bound on the scale parameter will cause the gradient update direction of the shape parameter to be wrong. For example, when the actual scale parameter of the parameter to be estimated is very small, setting a lower bound on the scale parameter makes it impossible for the scale parameter to continue to decrease, which will cause the shape parameter to continue to increase and deviate from its original range. To this end, a joint gradient rectification method of shape and scale parameters is proposed to correct the gradient update direction of shape and scale parameters.

[0093] The specific gradient rectification method is as follows:

[0094] If the current scale parameter α>α β , the gradients of shape and scale parameters are updated normally;

[0095] If the current scale parameter α≤α β , the gradients of updated scale and shape parameters are;

[0096]

[0097]

[0098] in, is the sign of the partial derivative, η,ζ are the intermediate parameters calculated when updating the gradients of the scale and shape parameters.

[0099] 4. End-to-end image lossless coding scheme based on generalized Gaussian model.

[0100] (1) Structural plan.

[0101] Lossless image coding requires entropy encoding and decoding of image pixels or feature maps generated by lossless transformation. A conditional entropy model based on a generalized Gaussian model is used to perform probability modeling on the image pixels or feature maps generated by lossless transformation, and side information or autoregressive models are used to predict the generalized Gaussian model parameters of the image pixels or feature maps generated by lossless transformation. The obtained generalized Gaussian model parameters are used to perform entropy encoding and entropy decoding on the image pixels or feature maps generated by lossless transformation. Models of different complexities can be implemented by limiting the diversity of generalized Gaussian model parameters predicted by the conditional entropy model. For example, the shape parameters of the generalized Gaussian model corresponding to all pixels or feature elements generated by lossless transformation can be fixed, and the conditional entropy model only predicts the mean and scale parameters. For details, please refer to the previous introduction. The main difference is that the quantized feature maps involved in the previous introduction are Replace with pixels or feature elements produced by lossless transformation.

[0102] (2) Entropy coding and entropy decoding.

[0103] After obtaining the parameters of the generalized Gaussian model for probabilistic modeling of the feature map generated by the image pixels or lossless transformation, the image pixels or the feature map generated by the lossless transformation are entropy encoded and decoded according to the scheme introduced in Section 2 above.

[0104] (3) Model training plan.

[0105] The optimization target of the end-to-end image lossless coding model only includes the bit rate, and its loss function L2 is: L2=R2; wherein R2 is the bit rate. The bit rate calculation scheme can be implemented with reference to conventional technology, and the present invention will not elaborate on it.

[0106] Lossless compression usually does not need to deal with the problem of non-differentiable quantization during training. The bit rate can be estimated directly on the feature map generated by the pixel or lossless transformation. The bit rate during training is:

[0107]

[0108] Among them, x is the feature map generated by pixel or lossless transformation, x i is an element of x, β i ,μ i ,α i For x i The corresponding generalized Gaussian model parameters, β, μ, and α, correspond to shape, mean, and scale respectively.

[0109] 2. Example explanation.

[0110] 1. Image lossy compression scheme based on generalized Gaussian model.

[0111] Example 1:

[0112] Using the conditional entropy model based on the generalized Gaussian model, applying side information or autoregressive model to predict the parameters of the generalized Gaussian model corresponding to the feature graph, the feature graph is calibrated using the scheme described in Section 2 of the first part above. Perform entropy encoding and entropy decoding.

[0113] Example 2:

[0114] Use the conditional entropy model based on the generalized Gaussian model to predict the feature map using side information or autoregressive models The corresponding parameters μ, β, α of the generalized Gaussian model.

[0115] During entropy coding, the feature map to be encoded is updated as In the right When performing entropy coding, the mean parameter of the generalized Gaussian model is fixed to 0, and the β, α pairs obtained by prediction are obtained using the scheme described in Section 2 of the first part. Perform entropy coding.

[0116] During entropy decoding, the mean parameter of the generalized Gaussian model is fixed to 0, and the β, α pairs obtained by prediction are obtained using the scheme described in Section 2 of the first part. Perform entropy decoding. Get Afterwards, rebuild for

[0117] 2. Image lossless compression scheme based on generalized Gaussian model.

[0118] Example 3:

[0119] Using a conditional entropy model based on a generalized Gaussian model, after applying side information or an autoregressive model to predict the parameters of the generalized Gaussian model corresponding to the image pixels or the feature map generated by the lossless transformation, the image pixels or the feature map generated by the lossless transformation are entropy encoded and decoded using the scheme described in Section 2 of the first part above.

[0120] 3. Performance description.

[0121] The technology of the present invention is applied to multiple end-to-end image compression lossy models (Shallow-JPEG, Shallow-2layer, MS-hyper, Charm, ELIC) to compare compression performance, as shown in Table 1. In Table 1, BD-rate (RGB-BDBR) is used to measure the coding performance gain, a negative value indicates the percentage of bit rate saving, and a positive value indicates the percentage of bit rate increase; Params indicates the size of the entropy model parameter; KMACs / pixel, the number of thousand multiplication and addition operations per pixel, indicates the computational complexity of the multiplication and addition operation; Params and KMACs / pixel can be used to measure the model complexity, the lower the complexity, the lower the complexity; GM indicates the application of Gaussian model; GMM indicates the application of Gaussian mixture model (including a mixture of 3 Gaussian models); GGM-m indicates the generalized Gaussian model of model-level shape parameters; GGM-c indicates the generalized Gaussian model of channel-level shape parameters; GGM-e indicates the generalized Gaussian model of element-level shape parameters.

[0122] Table 1: Performance comparison of the solution of Example 2 of the present invention and two types of solutions in the prior art on several end-to-end image lossy compression models

[0123]

[0124] As can be seen from Table 1, when the aforementioned Example 2 of the present invention uses model-level shape parameters, compared with the first type of solution introduced in the background technology, the same complexity can be maintained and the compression performance can be improved. When the aforementioned Example 2 of the present invention uses element-level shape parameters, compared with the second type of solution introduced in the background technology, the complexity can be reduced and the compression performance can be improved.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0126] Embodiment 2

[0127] The embodiment of the present invention provides an image coding system based on a generalized Gaussian model, which is mainly used to implement the method provided in the above embodiment. The system mainly includes: an end-to-end image lossy coding model applied to image lossy coding, and an end-to-end image lossless coding model applied to image lossless coding;

[0128] The end-to-end image lossy coding model is used to introduce a generalized Gaussian model to model the probability distribution of a feature map in the coding process, and perform entropy coding and entropy decoding on the feature map according to the generalized Gaussian model parameters obtained by the modeling;

[0129] The end-to-end image lossless coding model is used to introduce a generalized Gaussian model to model the probability distribution of image pixels or feature maps generated by lossless transformation during the image encoding process, and entropy encoding and entropy decoding are performed on the image pixels or feature maps generated by lossless transformation according to the generalized Gaussian model parameters obtained by the modeling.

[0130] Considering that the specific scheme contents involved in the above model have been introduced in detail in the previous embodiments, they will not be repeated here.

[0131] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image coding method based on a generalized Gaussian model, characterized in that: include: When applied to lossy image coding, a generalized Gaussian model is introduced to model the probability distribution of feature maps in the coding process, and entropy coding and entropy decoding are performed on the feature maps according to the generalized Gaussian model parameters obtained by modeling. When applied to lossless coding of images, a generalized Gaussian model is introduced to model the probability distribution of image pixels or feature maps generated by lossless transformation of the image during the encoding process. According to the generalized Gaussian model parameters obtained by the modeling, entropy encoding and entropy decoding are performed on the image pixels or feature maps generated by lossless transformation.

2. The image coding method based on the generalized Gaussian model according to claim 1, characterized in that: The introduction of the generalized Gaussian model to model the probability distribution of the feature graph in the encoding process includes: The feature map in the encoding process includes two parts. One part is the quantization of the feature map y obtained by the lossy transformation in the encoding part. The other part is the edge information extracted from the feature graph y For side information Modeling probability distributions using generalized Gaussian models, including: The set of characteristic elements for each channel Using a set of parameters of the generalized Gaussian model {β k ,μ k ,α k }Modeling probability distribution, k is the index of the channel, β, μ, α correspond to shape, mean, and scale respectively; For the quantized feature map The conditional entropy model of the generalized Gaussian model is used to predict the parameters of the generalized Gaussian model corresponding to each characteristic element, and then the generalized Gaussian model is used to model the probability distribution; wherein, conditional entropy models of different complexities are realized by limiting the diversity of the generalized Gaussian model parameters predicted by the conditional entropy model.

3. The image coding method based on generalized Gaussian model according to claim 2, characterized in that: The conditional entropy models of different complexities are realized by limiting the diversity of the generalized Gaussian model parameters predicted by the conditional entropy model, including: Model-level shape parameters: constraining quantized feature maps All feature elements in the same graph share a shape parameter β0, and the quantized feature map is predicted by the conditional entropy model. The mean and scale parameter of each feature element; Channel-level shape parameter: constraining quantized feature maps All feature elements of each channel share a shape parameter, and the quantized feature map is predicted by the conditional entropy model. The mean and scale parameter of each feature element; Element-wise shape parameters: quantized feature maps Each feature element has a separate shape parameter, and the quantized feature map is predicted by the conditional entropy model. Shape, mean, and scale parameters for each feature element.

4. The image coding method based on generalized Gaussian model according to claim 1, characterized in that: Also includes: For image lossy coding, the corresponding image coding method is implemented by an end-to-end image lossy coding model, which is optimized using an unconstrained rate-distortion loss function L1. The loss function L1 is as follows: L1=R1+λD; Among them, R1 is the code rate, D is the distortion, and λ is the Lagrange multiplier; During the training process, uniform noise is added for bit rate estimation, and the following two techniques are combined for training: The first technique is to set the scale parameter lower bound α depending on the shape parameter. β , the setting method is as follows: Among them, β, μ, and α correspond to shape, mean, and scale respectively, and all three are parameters of the generalized Gaussian model; F is the cumulative distribution function; The second technique is: joint gradient rectification of shape and scale parameters, including: if the current scale parameter α>α β , the gradients of the shape and scale parameters are updated normally; if the current scale parameter α≤α β , the gradients of updated scale and shape parameters are; in, is the symbol of partial derivative, η, Intermediate parameters calculated when updating the gradients of scale and shape parameters.

5. The image coding method based on generalized Gaussian model according to claim 1, characterized in that: The introduction of the generalized Gaussian model to model the probability distribution of the feature map generated by the image pixels or the image through lossless transformation during the encoding process includes: A conditional entropy model based on a generalized Gaussian model is used to model the probability distribution of image pixels or feature maps generated by lossless transformation during the encoding process of the image. Conditional entropy models of different complexities are achieved by limiting the diversity of generalized Gaussian model parameters predicted by the conditional entropy model.

6. The image coding method based on generalized Gaussian model according to claim 1, characterized in that: Also includes: For image lossless coding, the corresponding image coding method is implemented by an end-to-end image lossless coding model, the optimization target of which only includes the bit rate, and its loss function L2 is: L2 = R2; wherein R2 is the bit rate; During training, the bitrate is estimated on the image pixels or feature maps generated by lossless transformation. The bitrate is in the form of: Among them, x is the feature map generated by image pixels or lossless transformation, x i is an element of x, β i ,μ i ,α i For x i The corresponding generalized Gaussian model parameters, β, μ, and α, correspond to shape, mean, and scale respectively.

7. The image coding method based on generalized Gaussian model according to claim 2, characterized in that: Entropy encoding and entropy decoding of feature maps include: For side information The set of characteristic elements for each channel Combined with the parameters of its generalized Gaussian model {β k ,μ k ,α k }, construct the corresponding cumulative distribution function table for each channel All elements in are entropy encoded and decoded; wherein the constructed cumulative distribution function table is pre-stored in the model parameters.

8. The image coding method based on generalized Gaussian model according to claim 1, characterized in that: Also includes: The feature map in the lossy coding of the image includes quantizing the feature map y obtained by the lossy transformation in the coding part, and obtaining the quantized feature map For the quantized feature map As well as the image pixels or feature maps generated by lossless transformation during image lossless encoding, entropy encoding and entropy decoding are performed in the following manner: Combine the parameters of the generalized Gaussian model corresponding to each image pixel or feature element, and the cumulative distribution function of the generalized Gaussian model, to construct a cumulative probability distribution table corresponding to each image pixel or feature element; then use the entropy encoder to entropy encode the image pixels or feature elements in sequence into the bitstream using the cumulative probability distribution table corresponding to the image pixels or feature elements; where the feature element is the quantized feature map The elements in , or the elements in the feature map produced by a lossless transformation; The same process as entropy coding is used to construct a cumulative probability distribution table corresponding to pixels or feature elements, and an entropy decoder is used to sequentially decode pixels or feature elements from the bitstream using the cumulative probability distribution table corresponding to pixels or feature elements.

9. The image coding method based on generalized Gaussian model according to claim 2, characterized in that: Also includes: The feature map in the lossy coding of the image includes quantizing the feature map y obtained by the lossy transformation in the coding part, and obtaining the quantized feature map For the quantized feature map As well as the image pixels or feature maps generated by lossless transformation during image lossless encoding, entropy encoding and entropy decoding are performed in the following manner: Before deploying the encoder and decoder, N parameter samples {β n ,μ n ,α n }, n=1...N; β n ,μ n ,α n represents the nth sample, β, μ, and α correspond to shape, mean, and scale respectively; For each parameter sample, the corresponding cumulative probability distribution table is constructed using the cumulative distribution function of the generalized Gaussian model, and finally the lookup table {CDF n }, n = 1…N, and stored in the encoder and decoder; where CDF n is the cumulative probability distribution table corresponding to the nth sample; For each pixel or feature element corresponding to the parameter of the generalized Gaussian model, map it to the nearest parameter sample, obtain the index I of the cumulative probability distribution table of the corresponding parameter sample, and then use the entropy encoder to calculate the cumulative probability distribution table according to the index I and the lookup table {CDF n } Carry out entropy coding on pixels or feature elements and write them into the bitstream; The same process as entropy coding is used to obtain the index of the cumulative probability distribution table corresponding to each pixel or feature element, and the entropy decoder is used to calculate the cumulative probability distribution table according to the index and the lookup table {CDF n }Decode pixels or feature elements from the code stream one by one.

10. An image coding system based on a generalized Gaussian model, characterized in that: The system is used to implement the method according to any one of claims 1 to 9, comprising: an end-to-end image lossy coding model applied to image lossy coding, and an end-to-end image lossless coding model applied to image lossless coding; The end-to-end image lossy coding model is used to introduce a generalized Gaussian model to model the probability distribution of feature maps in the encoding process, and perform entropy encoding and entropy decoding on the feature maps according to the generalized Gaussian model parameters obtained by the modeling; An end-to-end image lossless coding model, wherein a generalized Gaussian model is introduced to model the probability distribution of image pixels or feature maps generated by lossless transformation during the image encoding process, and entropy encoding and entropy decoding are performed on image pixels or feature maps generated by lossless transformation according to the generalized Gaussian model parameters obtained by the modeling.

Images