Image encoding, decoding and compression method based on deep Gaussian process regression

Abstract

The present invention discloses an image encoding method based on deep Gaussian process regression, comprising: using an encoding convolutional neural network to obtain the bottleneck layer multi-channel features of the image to be encoded as the first feature map; quantizing each feature in the first feature map to an integer to obtain a second feature map; based on the autoregressive model and the hyper-prior model of deep Gaussian process regression, for each channel of the second feature map, weighted combination of mixed Gaussian distribution encoding features of multiple Gaussian distributions to generate a feature binary code stream; encoding the hyper-prior information obtained by the hyper-prior model into a hyper-prior binary code stream; merging the hyper-prior binary code stream with the feature binary code stream to obtain a binary code stream of the compressed image. Using a non-parametric deep Gaussian process regression method for autoregressive modeling, using the posterior distribution of the deep Gaussian process regression output as the mean of the mixed Gaussian model, the uncertainty of the mean estimate can be flexibly obtained, thereby obtaining a more accurate mean estimate.

Description

Technical Field

[0001] The present invention relates to the field of image compression, and in particular to an image encoding, decoding and compression method based on deep Gaussian process regression. Background Art

[0002] In recent years, deep neural networks have been widely used in the field of image compression. In the end-to-end image compression methods that have emerged in recent years, deep neural networks have replaced the transformation, quantization, and entropy coding modules in traditional image compression methods. The current mainstream end-to-end image compression method uses the architecture of variational autoencoders. In this architecture, the accuracy of modeling the entropy model of the bottleneck layer features has attracted widespread attention because it greatly affects the size of the bit rate.

[0003] Early entropy models assumed that the elements of the bottleneck layer features were independent of each other. After searching the literature on the prior art, it was found that J. Ballé et al. proposed a non-parametric, fully decomposable entropy model in the paper "End-to-end Optimized Image Compression" published at the 2017 International Conference on Learning Representation. Subsequently, in order to utilize the dependencies between the elements of the bottleneck layer features, a context-based entropy model was proposed. There are two main methods for establishing entropy models based on context. The first is an entropy model based on three-dimensional context, which considers the correlation of elements within and between channels of the bottleneck layer features. The second is an entropy model based on two-dimensional context. Since it only considers the correlation of elements within the bottleneck layer feature channel, compared with the entropy model based on three-dimensional context, channels can be parallelized during encoding and decoding, which greatly reduces the computational and time overhead, and therefore has attracted widespread interest.

[0004] In the entropy model based on 2D context, there are two main methods to use context for modeling. The first is the autoregressive method, which can only use the decoded elements in the bottleneck layer features due to causality. The second is the hyperprior model proposed by J.Ballé et al. in the paper "Variational Image Compression With a Scale Hyperprior" published at the 2018 International Conference on Learning Representation. It uses convolutional neural networks to obtain global context information, but requires the storage of additional bitstreams. The current mainstream entropy model based on 2D context uses these two complementary methods at the same time, such as the entropy model proposed by Minnen et al. in the paper "Joint Autoregressive and Hierarchical Priors for Learned Image Compression" published at the 2018 Conference and Workshop on Neural Information Processing Systems. However, existing entropy models based on two-dimensional context use certain parameters to estimate the distribution of elements in the bottleneck layer features, such as the mean in the mixed Gaussian distribution, that is, the entropy model estimates the distribution of the bottleneck layer features based on the mixed Gaussian distribution. Specifically, for each feature, the mean of the mixed Gaussian distribution is used as the predicted value for the feature, and the residual is estimated according to the mixed Gaussian distribution with a mean of zero and a variance of the estimated variance to estimate the probability value for encoding. Therefore, when the mean estimate is not accurate, a larger residual will be generated, resulting in a smaller corresponding probability value and an increase in the bit rate. At present, the mixed Gaussian distribution estimates the mean based on a linear combination of the means of a certain number of Gaussian distributions, rather than estimating the distribution of the mean. Therefore, it is more susceptible to the influence of the input image, so that the estimated value of the bottleneck layer feature will be inaccurate and deviate, making the estimated probability used for encoding far away from the peak of the Gaussian distribution, resulting in an increase in the bit rate of the compressed image and a deterioration in the rate-distortion performance. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide an image encoding, decoding and compression method based on deep Gaussian process regression.

[0006] According to one aspect of the present invention, there is provided an image encoding method based on deep Gaussian process regression, comprising:

[0007] Using a coding convolutional neural network to obtain the bottleneck layer multi-channel features of the image to be encoded as the first feature map;

[0008] quantizing each feature in the first feature map into an integer to obtain a second feature map;

[0009] Based on the autoregressive model and the hyper-prior model of deep Gaussian process regression, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream;

[0010] Encoding the super-a priori information obtained by the super-a priori model into a super-a priori binary code stream;

[0011] The super a priori binary code stream and the characteristic binary code stream are combined to obtain a binary code stream of a compressed image.

[0012] Preferably, the mixed Gaussian distribution encoding features include weight, mean and variance.

[0013] Preferably, obtaining the mean value includes:

[0014] The autoregressive model based on deep Gaussian process regression outputs a posterior distribution;

[0015] The posterior distribution is the distribution of the mean of a mixture of Gaussian distributions encoding the second feature map;

[0016] Sampling from the distribution of the mean to obtain a mean of a mixed Gaussian distribution for encoding the second feature map;

[0017] The acquisition of the variance and weight includes:

[0018] Based on the super-prior information output by the super-prior model, a variance and a weight of a mixed Gaussian distribution for encoding the second feature map are generated.

[0019] Preferably, the weighted combination of mixed Gaussian distribution encoding features of multiple Gaussian distributions to generate a feature binary code stream includes:

[0020] Performing probability estimation on the second feature map according to the mean, variance and weight to obtain an estimated probability;

[0021] Performing arithmetic coding according to the estimated probability;

[0022] The characteristic binary code stream is obtained.

[0023] Preferably, the autoregressive model based on deep Gaussian process regression includes:

[0024] M independent autoregressive sub-models based on deep Gaussian process regression;

[0025] Each of the autoregressive sub-models uses a two-dimensional context N of different sizes 1 ×N 1,…,N M ×N M , 0<N 1 <…<N M , and perform autoregressive modeling.

[0026] Preferably, the autoregressive sub-model comprises:

[0027] Autoregressive sequence generation module, the two-dimensional context size of the mth (m=1,2,…,M) autoregressive sub-model based on deep Gaussian process regression is N m ×N m , for the kth channel of the second feature map, select the autoregressive target feature y i,j,k (feature of row i and column j in channel k), scan and select row by row Constructing the sequence for regression

[0028] The preprocessing module processes the autoregressive sequence through a deep neural network composed of a cascade of multiple fully connected layers and Leaky ReLU activation functions. Perform whitening and dimensionality reduction preprocessing to obtain the preprocessed autoregressive sequence

[0029] The deep Gaussian process regression module is used to process the preprocessed autoregressive sequence. Use deep Gaussian process to perform regression and get the output As y i，j，k The unbiased estimate of .

[0030] Preferably, the deep Gaussian process regression module includes a variational inference submodule and a sampling estimation submodule, wherein:

[0031] In the variational inference submodule, let the number of deep Gaussian process layers be L. For the lth layer, replace the D l-1 Output vector As input, calculate D of the lth layer l Output vector The posterior distribution of include:

[0032] For each output vector Introducing the posterior distribution Gaussian distribution Auxiliary vector of The expectation of the Gaussian distribution and variance The value of is obtained by random initialization and then training optimization;

[0033] break down The joint posterior distribution of The distribution Gaussian distribution

[0034] according to and calculate The posterior distribution of The calculation process is get Gaussian distribution

[0035] In the sampling estimation submodule, the deep Gaussian process is sampled in order from the first layer, and for the l-1th layer, according to D l-1 Output vector The posterior distribution of Sampling, get the sample value As the input of the lth layer; until the posterior distribution of the output of the last layer of the deep Gaussian process is obtained As the autoregressive target feature y i,j,k An unbiased estimate of the mean of a mixture of Gaussian distributions.

[0036] Preferably, the super a priori model comprises:

[0037] Using a super-prior coding convolutional neural network to obtain super-prior information on the second feature map;

[0038] quantizing the super a priori information to obtain quantized super a priori information, establishing a non-parametric distribution estimate for the quantized super a priori information, and performing arithmetic coding on the quantized super a priori information using the distribution estimate to obtain a super a priori binary code stream;

[0039] arithmetically decoding the super-prior binary code stream using the non-parametric distribution estimate to obtain restored super-prior information;

[0040] The recovered super-prior information is decoded by a super-prior decoding convolutional neural network to obtain reconstructed super-prior information as the variance and weight of the mixed Gaussian distribution of the second feature map.

[0041] According to a second aspect of the present invention, there is provided an image decoding method based on deep Gaussian process regression, which is used for decoding a binary code stream obtained by the above method, comprising:

[0042] Decomposing the binary code stream to obtain a characteristic binary code stream and a super a priori binary code stream;

[0043] Reconstructing super-prior information from the super-prior binary code stream using a super-prior model;

[0044] Obtaining a mean, variance and weight of a mixed Gaussian distribution for decoding the second feature map using the reconstructed hyper-prior information and the posterior distribution output by the autoregressive model based on deep Gaussian process regression;

[0045] Binary-decoding the characteristic binary code stream according to the mean, variance and weight of the mixed Gaussian distribution to obtain the second characteristic map;

[0046] The second feature map is decoded through a convolutional neural network to obtain a reconstructed image.

[0047] According to a third aspect of the present invention, there is provided an image compression method based on deep Gaussian process regression, comprising:

[0048] The above-mentioned image encoding method based on deep Gaussian process regression is used on the compressed image to obtain a binary code stream;

[0049] The binary code stream is decompressed using the above-mentioned image decoding method based on deep Gaussian process regression to obtain a reconstructed image.

[0050] Preferably, the parameters based on the convolutional neural network and the deep Gaussian process regression model are obtained by rate-distortion optimization:

[0051] min R+λ·D

[0052] Among them, λ is the selected Lagrangian coefficient used to balance distortion and bit rate, distortion It is based on the image to be compressed x on the training image set c and reconstructed image The mean square error of the bit rate R is calculated based on the mixed Gaussian distribution obtained by M deep Gaussian process regression. For a given image x c ,in:

[0053]

[0054] are the mean, variance, and weight of the mixed Gaussian distribution when encoding the features of the i-th row and j-th column in the k-th channel, is the scaling factor used to combine the autoregressive sub-models based on deep Gaussian process regression, is the prior distribution of the auxiliary variable, is the distribution estimate of the nth dimension of the quantized hyper-prior information, Represents the interval defined arrive of uniform distribution.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The image encoding, decoding and compression method based on deep Gaussian process regression in the embodiment of the present invention adopts a non-parametric deep Gaussian process regression method for autoregressive modeling, and uses the posterior distribution of the deep Gaussian process regression output as the mean of the mixed Gaussian model, which can estimate the mean more flexibly, thereby obtaining a more accurate mean estimation.

[0057] The image encoding, decoding and compression method based on deep Gaussian process regression in the embodiment of the present invention can save 3.6% of the bit stream overhead and improve the rate-distortion performance compared with the benchmark masked product neural network while maintaining the same reconstructed image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0059] Figure 1 A flowchart of image encoding based on deep Gaussian process regression in one embodiment of the present invention;

[0060] Figure 2 is a flow chart of a model based on deep Gaussian process regression in one embodiment of the present invention;

[0061] Figure 3 is a flowchart of a deep Gaussian process regression module in one embodiment of the present invention;

[0062] Figure 4 This is a flowchart of image decoding based on deep Gaussian process regression in one embodiment of the present invention;

[0063] Figure 5 The process of image compression based on deep Gaussian process regression in one embodiment of the present invention;

[0064] Figure 6 This is a flow chart of the sub-model training process based on deep Gaussian process regression of the present invention. DETAILED DESCRIPTION

[0065] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0066] The present invention provides an embodiment, an image encoding method based on deep Gaussian process regression, comprising:

[0067] S100 uses a coding convolutional neural network to obtain the bottleneck layer multi-channel features of the image to be encoded as the first feature map;

[0068] S200 quantizes each feature in the first feature map into an integer to obtain a second feature map;

[0069] S300, based on the autoregressive model and the hyper-prior model of deep Gaussian process regression, weightedly combines the mixed Gaussian distribution encoding features of multiple Gaussian distributions for each channel of the second feature map to generate a feature binary code stream;

[0070] S400 encodes the super-a priori information obtained by the super-a priori model into a super-a priori binary code stream;

[0071] S500 combines the super a priori binary code stream with the characteristic binary code stream to obtain a binary code stream of a compressed image.

[0072] Based on the above embodiments, further optimization is performed to obtain a preferred embodiment, such as Figure 1 As shown, it is a flowchart of image encoding based on deep Gaussian process regression of the preferred embodiment, including:

[0073] S101 Bottleneck layer multi-channel feature extraction: The image to be encoded is encoded through a convolutional neural network to obtain three-dimensional bottleneck layer multi-channel features;

[0074] S102 Uniform Quantization: Round off the multi-channel features of the bottleneck layer to integers;

[0075] S103 Deep Gaussian Process Regression: For each channel of the multi-channel features of the bottleneck layer, an independent autoregressive model based on deep Gaussian process regression is established; each autoregressive model based on deep Gaussian process regression includes M autoregressive sub-models based on deep Gaussian process regression, each of which uses two-dimensional contexts N of different sizes. 1 ×N 1 ,…,N M ×N M , 0<N 1 <…<N M , and perform autoregressive modeling.

[0076] Each channel of the second feature map is obtained from the input image using a convolution filter with different parameters (each layer of the convolutional neural network uses a convolution filter with the same receptive field size but different parameter values ​​for each input channel and output channel), so the statistical distribution of the features in each channel of the second feature map is different. In deep Gaussian process regression, multiple autoregressive sub-models with different context sizes (that is, the model structures of each autoregressive sub-model are the same, but the autoregressive orders are different) are used to process the above differences to improve the accuracy of model fitting.

[0077] S104 Super prior information extraction: Super prior information is obtained by using super prior coding convolutional neural network for multi-channel features of bottleneck layer;

[0078] S105 Super prior information encoding: quantizing and arithmetically encoding the super prior information, establishing a non-parametric distribution estimate, and using the distribution estimate to arithmetically encode the quantized super prior information to obtain a super prior binary code stream;

[0079] S106 super-prior information decoding: performing arithmetic decoding on the super-prior information binary code stream to obtain restored super-prior information; applying a super-prior decoding convolutional network to the restored super-prior information to obtain reconstructed super-prior information;

[0080] S107 Bottleneck layer multi-channel feature coding: using the posterior distribution of the output of the autoregressive model based on deep Gaussian process regression as the mean of the Gaussian mixture model for encoding the quantized bottleneck layer multi-channel features; using the reconstructed super prior information to obtain the variance and weight of the Gaussian mixture model for encoding the quantized bottleneck layer multi-channel features; performing arithmetic coding on the quantized bottleneck layer multi-channel features according to the mean, variance and weight of the Gaussian mixture model to obtain a bottleneck layer multi-channel feature binary code stream;

[0081] S108 code stream synthesis: merge the super-prior information binary code stream and the bottleneck layer multi-channel feature code stream to obtain the compressed image binary code stream.

[0082] The present invention provides a preferred embodiment of the autoregressive sub-model based on deep Gaussian process regression, such as Figure 2 As shown in the figure, it is a schematic diagram of the structure of the model, including:

[0083] Autoregressive sequence generation module, the two-dimensional context size of the mth (m=1,2,…,M) autoregressive sub-model based on deep Gaussian process regression is N m ×N m , for the kth channel of the second feature map, select the autoregressive target feature y i,j,k (feature of row i and column j in channel k), scan and select row by row Constructing the sequence for regression

[0084] The preprocessing module processes the autoregressive sequence through a deep neural network composed of a cascade of multiple fully connected layers and Leaky ReLU activation functions. Perform whitening and dimensionality reduction preprocessing to obtain the preprocessed autoregressive sequence According to the universal approximation theorem, in theory, multiple layers (greater than or equal to 2 layers) of fully connected layers can fit any function with arbitrary precision. In actual network design, nonlinearity is introduced through the Leaky ReLU activation function commonly used in image processing to improve the fitting ability of deep neural networks.

[0085] The deep Gaussian process regression module is used to process the preprocessed autoregressive sequence. Use deep Gaussian process to perform regression and get the output As y i,j,k The estimated value of .

[0086] Furthermore, the deep Gaussian process regression module has the following structure: Figure 3 As shown, including:

[0087] Variational inference submodule, assuming that the number of deep Gaussian process layers is L, for the lth layer (l≤L), D of the l-1th layer is l-1 Output vector As input (when l = 1, the input is the sequence of preprocessed regression Calculate D of the lth layer l Output vector The posterior distribution of The specific operation is

[0088] For each output vector Introducing the posterior distribution Gaussian distribution Auxiliary vector of The expectation of the Gaussian distribution and variance The value of is obtained by random initialization and then training optimization;

[0089] break down The joint posterior distribution of The distribution Gaussian distribution

[0090] according to and calculate The posterior distribution of The calculation process is therefore Gaussian distribution

[0091] The sampling estimation submodule samples the deep Gaussian process from the first layer in order between layers, and for the l-1th layer, according to D l-1 Output vector The posterior distribution of Sampling, get the sample value As the input of the lth layer (when l = 1, the input is the sequence of preprocessed regression ), and finally obtain the sampling of the posterior distribution of the last layer output of the deep Gaussian process As the autoregressive target feature y i,j,k An unbiased estimate of the mean distribution of the Gaussian mixture distribution. The unbiased estimate is the posterior distribution of the output of the deep Gaussian process regression as the mean of the Gaussian mixture model, that is, according to the autoregressive submodel of the deep Gaussian process regression, the obtained mean is statistically unbiased for the output value obtained for a given input.

[0092] In another preferred embodiment, the deep Gaussian process in the deep Gaussian process regression module is simplified to a single-layer Gaussian process (L=1).

[0093] The image encoding method based on deep Gaussian process regression in this embodiment uses the posterior distribution of the output of the last layer of the deep Gaussian process as the mean of the Gaussian mixture model of the multi-channel features of the bottleneck layer after quantization, which has better flexibility and accuracy of mean estimation.

[0094] Based on the same inventive concept, another embodiment of the present invention provides an image decoding method based on deep Gaussian process regression, as shown in the flowchart: Figure 4 As shown, including:

[0095] S201 code stream decomposition: decompose the compressed image binary code stream to obtain the super prior information binary code stream and the bottleneck layer multi-channel feature binary code stream;

[0096] S202 super-prior information decoding: performing arithmetic decoding on the super-prior information binary code stream to obtain restored super-prior information; applying a super-prior decoding convolutional network to the restored super-prior information to obtain reconstructed super-prior information;

[0097] S203 Deep Gaussian Process Regression: The structure and parameter values ​​of this module are similar to Figure 1 The deep Gaussian process regression in the illustrated embodiment is exactly the same. Specifically, firstly, the undecoded quantized bottleneck layer multi-channel features are initialized with 0 elements; an independent autoregressive model based on deep Gaussian process regression is established for each channel of the quantized bottleneck layer multi-channel features; each autoregressive model based on deep Gaussian process regression includes M autoregressive sub-models based on deep Gaussian process regression, each of which uses two-dimensional contexts N of different sizes. 1 ×N 1 ,…,N M ×N M , 0<N 1 <…<N M, perform autoregressive modeling; the structure of each autoregressive sub-model based on deep Gaussian process regression is as follows Figure 2 As shown, it includes an autoregressive sequence generation module, a preprocessing module and a deep Gaussian process regression module; wherein the structure of the deep Gaussian process regression module is as follows Figure 3 As shown, it includes a variational inference module and a sampling estimation module.

[0098] In another preferred embodiment, the deep Gaussian process in the deep Gaussian process regression module is simplified to a single-layer Gaussian process (L=1).

[0099] S204 Decoding of bottleneck layer multi-channel features: using the posterior distribution of the output of the autoregressive model based on deep Gaussian process regression as the mean of the Gaussian mixture model for decoding the quantized bottleneck layer multi-channel features; using the reconstructed super-prior information to obtain the variance and weight of the Gaussian mixture model for decoding the quantized bottleneck layer multi-channel features; performing binary decoding on the quantized bottleneck layer multi-channel feature binary code stream according to the mean, variance and weight of the Gaussian mixture model, and progressively obtaining the reconstructed quantized bottleneck layer multi-channel features. Progressively obtaining means that the elements of the quantized bottleneck layer multi-channel features are reconstructed one by one in the scanning order. Specifically, for each channel of the quantized bottleneck layer multi-channel features, the elements in the channel are scanned in the order from the upper left to the lower right in the channel.

[0100] S205 Image reconstruction: The reconstructed quantized bottleneck layer multi-channel features are decoded using a convolutional neural network to obtain a reconstructed image.

[0101] The image decoding method based on deep Gaussian process regression in this embodiment is used for Figure 1 The image coding method based on deep Gaussian process regression is shown in FIG. Similarly, the posterior distribution of the output of the last layer of the deep Gaussian process is used as the mean of the Gaussian mixture model of the multi-channel features of the quantized bottleneck layer, making the estimation of the mean more accurate and flexible.

[0102] According to the same inventive concept, another embodiment provides an image compression method based on deep Gaussian process regression. Figure 5 As shown, it is a flow chart of the method, comprising:

[0103] S301 obtains the image to be compressed;

[0104] S302 compresses the image to be compressed based on convolutional neural network and deep Gaussian process regression to obtain a compressed image binary code stream. Figure 1 The image coding method based on deep Gaussian process regression is shown to compress the image to be compressed.

[0105] S303 decompresses the compressed image binary code stream based on convolutional neural network and deep Gaussian process regression to obtain a reconstructed image. Figure 2 The image decoding method based on deep Gaussian process regression is shown to decompress the compressed image binary code stream.

[0106] Based on the same inventive concept, in other embodiments of the present invention, a training method for an image compression model based on deep Gaussian process regression is provided, and the process is as follows: Figure 6 In this embodiment, the training set uses 20,000 images from the Flicker dataset. The trained image compression model can be used for actual encoding and decoding. Figure 6 As shown, including:

[0107] S401 Bottleneck layer multi-channel feature extraction: The images in the Flicker dataset are cropped into 256×256 size and input into the encoding convolutional neural network to obtain the bottleneck layer multi-channel features;

[0108] S402 Super prior information extraction: Input the multi-channel features of the bottleneck layer into the super prior coding convolutional neural network to obtain super prior information;

[0109] S403 Super prior information can be quantized: Since the gradient of the quantization operation is almost 0 everywhere, in order to allow the gradient of the quantization operation of the super prior information to be back-propagated during training, uniformly distributed quantization noise is added to the super prior information to simulate actual quantization;

[0110] S404 super-prior information reconstruction: inputting the super-prior information after adding quantization noise into the super-prior decoding convolutional neural network to obtain reconstructed super-prior information;

[0111] S405 Super prior information bit rate estimation: Assuming that each element in the super prior information is independent, a non-parametric distribution estimation is used to fit the probability distribution of each element in the super prior information to obtain the super prior information bit rate estimation;

[0112] S406 Feature derivative quantization: The bottleneck layer multi-channel features are rounded and quantized to integers. In order to allow the gradient of the quantization operation on the bottleneck layer multi-channel features during training to be back-propagated, the gradient of the quantized bottleneck layer multi-channel features to the bottleneck layer multi-channel features before quantization is set to 1;

[0113] S407 Deep Gaussian Process Regression: Establish an independent autoregressive model based on deep Gaussian process regression for each channel of the quantized bottleneck layer multi-channel features; each autoregressive model based on deep Gaussian process regression includes M autoregressive sub-models based on deep Gaussian process regression, each using two-dimensional context N of different sizes. 1 ×N 1,…,N M ×N M , 0<N 1 <…<N M , perform autoregressive modeling;

[0114] S408 feature bit rate estimation: using a sample of the posterior distribution of the output of the autoregressive model based on deep Gaussian process regression as the mean of the Gaussian mixture model of the quantized bottleneck layer multi-channel features; using the reconstructed hyper-prior information to obtain the variance and weight of the Gaussian mixture model of the quantized bottleneck layer multi-channel features; obtaining the bit rate estimation of the quantized bottleneck layer multi-channel features according to the mean, variance and weight of the Gaussian mixture model;

[0115] S409 total bit rate estimation: adding the bit rate estimation of the super prior information and the bit rate estimation of the bottleneck layer multi-channel features to obtain a total bit rate estimation;

[0116] S410 Image reconstruction: The quantized bottleneck layer multi-channel features are decoded using a convolutional neural network to obtain a reconstructed image.

[0117] Among them, the autoregressive sub-model based on deep Gaussian process regression is as follows: Figure 2 and Figure 3 As shown, no further details are given here.

[0118] Furthermore, the stochastic gradient descent algorithm is used to learn the above image compression model, and its loss function is defined as:

[0119]

[0120] Among them, λ is the selected Lagrangian coefficient used to balance distortion and bit rate (the larger λ is, the smaller the distortion is after the model converges, and the larger the bit rate is. By changing λ, image compression models at different bit rates can be obtained). It is based on the image to be compressed x on the training image set c and reconstructed image The mean square error of the bit rate R is calculated based on the mixed Gaussian distribution obtained by M deep Gaussian process regression. For a given image x c The specific calculation process is

[0121]

[0122] here, are the expectation, variance, and weight of the mixed Gaussian distribution when encoding the features of the i-th row and j-th column in the k-th channel, is the scaling factor used to combine the autoregressive sub-models based on deep Gaussian process regression, is the prior distribution of the auxiliary variable, is the distribution estimate of the nth dimension of the quantized hyper-prior information, Represents the interval defined arrive of uniform distribution.

[0123] Implementation effect:

[0124] In the above embodiment, the Flicker dataset is used to train the image compression model based on deep Gaussian process regression. The sampling rate of each channel of the multi-channel feature of the bottleneck layer is set to 1 / 16, the distortion term in the loss function is set to the mean square error, and λ is set to 0.001, 0.004, 0.012 and 0.035 respectively, corresponding to four image compression models with different bit rates. The Adam optimizer is first used at a learning rate of 10 -4 We train for 750,000 steps and then reduce the learning rate to 5×10 -5 After 250,000 training steps, the model converged and training stopped.

[0125] In the above embodiment, the image encoders of the four image compression models based on deep Gaussian process regression obtained by training are used to compress 24 images in the Kodak test set. When the number of layers of the deep Gaussian process is set to 1, the average bit rates of the 24 compressed images are 0.1079bpp (bits per pixel), 0.2879bpp, 0.5247bpp and 0.9210bpp respectively; when the number of layers of the deep Gaussian process is set to 2, the average bit rates of the 24 compressed images are 0.0940bpp, 0.2711bpp, 0.5194bpp and 0.9008bpp respectively; when the number of layers of the deep Gaussian process is set to 3, the average bit rates of the 24 compressed images are 0.0938bpp, 0.2725bpp, 0.5172bpp and 0.9071bpp respectively.

[0126] In the above embodiment, the image encoders of the four image compression models based on deep Gaussian process regression obtained by training are used to decompress the binary code stream generated by compressing 24 images of the Kodak test set to obtain reconstructed images. When the number of layers of the deep Gaussian process is set to 1, the average peak signal-to-noise ratio (PSNR) of the 24 reconstructed images is 27.6683dB (decibel), 30.9539dB, 33.6554dB and 36.7882dB respectively; when the number of layers of the deep Gaussian process is set to 2, the average peak signal-to-noise ratio of the 24 reconstructed images is 27.2620dB, 30.8822dB, 33.7075dB and 36.7512dB respectively; when the number of layers of the deep Gaussian process is set to 3, the average peak signal-to-noise ratio of the 24 reconstructed images is 27.2691dB, 30.8667dB, 33.7406dB and 36.7921dB respectively.

[0127] In addition, compared with the method of using masked convolution to autoregress the multi-channel features of the bottleneck layer, the image compression method based on deep Gaussian process regression provided by the present invention has better rate-distortion performance. Specifically, when the number of layers of the deep Gaussian process is set to 2, 3.6% of the bitstream overhead can be saved under the same reconstructed image distortion.

[0128] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. An image coding method based on deep Gaussian process regression, characterized in that: include: Using a coding convolutional neural network to obtain the bottleneck layer multi-channel features of the image to be encoded as the first feature map; quantizing each feature in the first feature map into an integer to obtain a second feature map; Based on the autoregressive model and the hyper-prior model of deep Gaussian process regression, for each channel of the second feature map, a mixed Gaussian distribution encoding feature of multiple Gaussian distributions is weightedly combined to generate a feature binary code stream; Encoding the super-a priori information obtained by the super-a priori model into a super-a priori binary code stream; Combining the super a priori binary code stream with the characteristic binary code stream to obtain a binary code stream of a compressed image; The mixed Gaussian distribution encoding features include weight, mean and variance; The acquisition of the mean value includes: The autoregressive model based on deep Gaussian process regression outputs a posterior distribution; The posterior distribution is the distribution of the mean of a mixture of Gaussian distributions encoding the second feature map; Sampling from the distribution of the mean to obtain a mean of a mixed Gaussian distribution for encoding the second feature map; The acquisition of the variance and weight includes: Based on the super-prior information output by the super-prior model, generating a variance and a weight of a mixed Gaussian distribution for encoding the second feature map; The weighted combination of the mixed Gaussian distribution encoding features of multiple Gaussian distributions to generate a feature binary code stream includes: Performing probability estimation on the second feature map according to the mean, variance and weight to obtain an estimated probability; Performing arithmetic coding according to the estimated probability; Obtaining the characteristic binary code stream; The super prior model includes: Using a super-prior coding convolutional neural network to obtain super-prior information on the second feature map; quantizing the super a priori information to obtain quantized super a priori information; Establishing a non-parametric distribution estimate for the quantized super-prior information, and using the distribution estimate to perform arithmetic coding on the quantized super-prior information to obtain a super-prior binary code stream; arithmetically decoding the super-prior binary code stream using the non-parametric distribution estimate to obtain restored super-prior information; The recovered super-prior information is decoded by a super-prior decoding convolutional neural network to obtain reconstructed super-prior information as the variance and weight of the mixed Gaussian distribution of the second feature map.

2. The image coding method based on deep Gaussian process regression according to claim 1, characterized in that: The autoregressive model based on deep Gaussian process regression includes: M independent autoregressive sub-models based on deep Gaussian process regression; Each of the autoregressive sub-models uses a two-dimensional context of different sizes N1×N1,…,N M ×N M , 0 <N1<…<N M , and perform autoregressive modeling.

3. The image coding method based on deep Gaussian process regression according to claim 2, characterized in that: The autoregressive sub-model comprises: Autoregressive sequence generation module, the mth, m = 1, 2, ..., M, the two-dimensional context size of the autoregressive sub-model based on deep Gaussian process regression is N m ×N m , for the kth channel of the second feature map, select the autoregressive target feature y i,j,k ,y i,j,k is the feature of the i-th row and j-th column in the k-th channel, and is selected by scanning row by row Constructing the sequence for regression The preprocessing module processes the autoregressive sequence through a deep neural network composed of a cascade of multiple fully connected layers and Leaky ReLU activation functions. Perform whitening and dimensionality reduction preprocessing to obtain the preprocessed autoregressive sequence The deep Gaussian process regression module is used to process the preprocessed autoregressive sequence. Use deep Gaussian process to perform regression and get the output As y i,j,k The unbiased estimate of .

4. The image coding method based on deep Gaussian process regression according to claim 3, characterized in that: The deep Gaussian process regression module includes a variational inference submodule and a sampling estimation submodule, wherein: In the variational inference submodule, let the number of deep Gaussian process layers be L. For the lth layer, replace the D l-1 Output vector As input, calculate D of the lth layer l Output vector The posterior distribution of include: For each output vector Introducing the posterior distribution Gaussian distribution Auxiliary vector of The expectation of the Gaussian distribution and variance The value of is obtained by random initialization and then training optimization; break down The joint posterior distribution of The distribution Gaussian distribution according to and calculate The posterior distribution of The calculation process is get Gaussian distribution In the sampling estimation submodule, the deep Gaussian process is sampled in order from the first layer, and for the l-1th layer, according to D l-1 Output vector The posterior distribution of Sampling, get the sample value As the input of the lth layer; until the posterior distribution of the output of the last layer of the deep Gaussian process is obtained As the autoregressive target feature y i,j,k An unbiased estimate of the mean of a mixture of Gaussian distributions.

5. An image decoding method based on deep Gaussian process regression, used to decode the binary code stream obtained in any one of claims 1 to 4, characterized in that: include: Decomposing the binary code stream to obtain a characteristic binary code stream and a super a priori binary code stream; Reconstructing super-prior information from the super-prior binary code stream using a super-prior model; Obtaining a mean, variance and weight of a mixed Gaussian distribution for decoding the second feature map using the reconstructed hyper-prior information and the posterior distribution output by the autoregressive model based on deep Gaussian process regression; Binary-decoding the characteristic binary code stream according to the mean, variance and weight of the mixed Gaussian distribution to obtain the second characteristic map; The second feature map is decoded through a convolutional neural network to obtain a reconstructed image.

6. An image compression method based on deep Gaussian process regression, characterized in that: include: The image to be compressed is subjected to the image encoding method based on deep Gaussian process regression as described in any one of claims 1 to 4 to obtain a binary code stream; The binary code stream is decompressed using the image decoding method based on deep Gaussian process regression as described in claim 5 to obtain a reconstructed image.

7. The image compression method based on deep Gaussian process regression according to claim 6, characterized in that: The parameters of the convolutional neural network-based and deep Gaussian process regression-based autoregressive models are obtained through rate-distortion optimization: min R+λ·D Among them, λ is the selected Lagrangian coefficient used to balance distortion and bit rate, distortion It is based on the image to be compressed x on the training image set c and reconstructed image The obtained mean square error, the bit rate R is calculated based on the mixed Gaussian distribution obtained by M autoregressive sub-models based on deep Gaussian process regression and the hyper-prior model. For a given image x c ,in: are the mean, variance, and weight of the mixed Gaussian distribution when encoding the features of the i-th row and j-th column in the k-th channel, is the scaling factor used to combine the autoregressive sub-models based on deep Gaussian process regression, is the prior distribution of the auxiliary variable, p zn (ψ n ) is the distribution estimate of the nth dimension of the quantized hyper-prior information, Represents the interval defined arrive of uniform distribution.

Images