A Deep Learning-Based Image Compression Method Based on Importance Scale and Gaussian Mixture Model

Abstract

This invention relates to a deep learning image compression method based on importance scaling and Gaussian mixture models, belonging to the field of image processing technology. An importance scaling network is constructed to process the latent representation obtained by the master encoder, achieving adaptive bit allocation of content and ensuring effective extraction of key feature information, thereby obtaining an importance-enhanced latent representation. A Gaussian mixture model is introduced into both the content stream and the model stream. This model incorporates multiple Gaussian distributions, yielding a more accurate probability distribution to guide entropy encoding and decoding for more accurate reconstruction of the latent representation or neurogrammatical information. Content-adaptive optimization is performed on the latent representation, and a more reasonable and flexible probability distribution model is obtained by predicting using Gaussian mixture distributions. This reduces redundant information in the latent representation or neurogrammatical information, resulting in a visually better reconstructed image while also improving the rate-distortion performance of the deep learning image compression model.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and relates to a deep learning image compression method based on importance scale and Gaussian mixture model. Background Technology

[0002] Image compression refers to the compression and encoding of raw image data using a series of coding methods to obtain the smallest possible compressed bitstream while maintaining the visual quality of the image, facilitating subsequent storage and transmission. Traditional image compression methods include Joint Photographic Experts Group (JPEG), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC). These methods reduce spatial redundancy and improve coding efficiency through intra / inter-frame prediction, transform, quantization, and entropy coding. Each module is manually designed, but current traditional image compression methods have reached a bottleneck in improving compression performance. In recent years, with the continuous development of deep learning, deep neural networks have been widely applied in image processing, and various deep learning-based image compression methods have been proposed. Deep learning-based image compression can process large amounts of image data through training and learning, extracting image feature information, and has powerful capabilities in processing high-dimensional data. Furthermore, compared to traditional image compression methods, deep learning image compression methods can jointly optimize various modules in the compression process, such as transformation, quantization, and entropy coding, considering the block-based problem holistically and achieving better compression performance. However, current deep learning-based image compression methods rely too heavily on the model's data generalization ability, neglecting the adaptiveness of image content. Moreover, existing methods typically use a single Gaussian model to predict probability distributions, which, due to their fixed shape, results in an inaccurate description of the probability distribution model. Redundancy remains in the latent information or neurogrammatical information after image transformation, affecting data compression performance. Existing deep learning image compression models improve image content adaptability through online updates, but this method may increase computational overhead. Furthermore, these methods neglect the refinement of the latent representation within the entire compression model. The latent representation refers to deeper feature information of the image. By optimizing existing models, different bit rates can be allocated to different regions of the latent representation, enabling deep learning image compression models to achieve content adaptation at a finer granular level. Meanwhile, the single Gaussian model is extended to a Gaussian mixture model to predict a more accurate and flexible probability distribution model to guide entropy encoding and decoding, reduce redundant information, and obtain more accurate reconstructed latent representations or neurogrammatical information. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a deep learning image compression method based on importance scale and Gaussian mixture model.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A deep learning-based image compression method based on importance scale and Gaussian mixture model includes the following steps:

[0006] S1. The image to be processed is analyzed and transformed by the main encoder to obtain the latent representation;

[0007] S2. The latent representation is passed through a preset importance scale network to adaptively allocate bits for different regions of content to obtain an importance-enhanced latent representation;

[0008] S3. Divide the importance-enhanced latent representation into a content flow latent representation and a model flow latent representation in the channel dimension;

[0009] S4. By processing the content stream using a priori model, a context model, and a preset Gaussian mixture model, the probability distribution of the potential representation is obtained to acquire the reconstructed potential representation.

[0010] S5. Obtain the neural syntax of the latent representation of the model stream using a neural syntax generator; obtain the probability distribution of the neural syntax through the preset Gaussian mixture model to guide entropy encoding and decoding to obtain neural syntax information; feed the neural syntax information into a weight generator to obtain the kernel parameters of the last transposed convolutional layer of the main decoder.

[0011] S6. The reconstructed latent representation and the kernel parameters of the last transposed convolutional layer of the decoder are synthesized and transformed by the main decoder to obtain the reconstructed image.

[0012] Furthermore, in S1: the main encoder includes four two-dimensional zero-padding 2D layers (ZeroPad2d), four convolutional layers, and three generalized divisive normalization (GDN) layers.

[0013] Furthermore, in S2: the importance scale network includes a convolutional layer and three residual blocks; an adaptive bit allocation strategy is adopted, and for regions with larger potential values ​​in the region, a higher bit rate is allocated to extract more feature information, thereby obtaining an importance-enhanced potential representation.

[0014] Furthermore, in S4: the super-prior model includes a super-prior encoder and a super-prior decoder; the super-prior encoder includes three groups of convolutional layers; the super-prior decoder includes three transposed convolutional layers; the convolutional layers and the transposed convolutional layers have rectified linear unit (ReLU) functions.

[0015] Furthermore, in S4: the context model contains three convolutional layers, which are connected by a Leaky Rectified Linear Unit (LeakyReLU) function; the context information of the latent representation is obtained through the context model.

[0016] Furthermore, in S4: the Gaussian mixture model consists of multiple Gaussian distributions with different means, variances, and weights, and the Gaussian mixture model is expressed as:

[0017]

[0018] Where w represents the weight value of each Gaussian distribution, μ and σ represent the mean and variance of the Gaussian distribution, and K represents the total number of Gaussian models. Represents the potential representation of reconstruction. Quantitative information representing auxiliary information.

[0019] Furthermore, in S5: the neural syntax generator includes a multi-scale network, a global average pooling layer, and a fully connected layer; the latent representation of the model flow extracts features through the multi-scale network, and after global average pooling, they are concatenated into a neural syntax feature vector for use by the subsequent weight generator.

[0020] Furthermore, in S5: the weight generator comprises three fully connected layers, which are connected by a Leaky Rectified Linear Unit (LeakyReLU) function.

[0021] Furthermore, in S6: the main decoder contains five transposed convolutional layers, which are connected by an inverse generalized divisive normalization layer (GDN); after the entire deep learning image compression model is trained, the parameters of all transposed convolutional layers except the last transposed convolutional layer are fixed values; the kernel parameters of the last transposed convolutional layer are generated in real time by the weight generator during decoding.

[0022] Furthermore, the total number of Gaussian models is K = 4.

[0023] The beneficial effects of this invention are as follows: it achieves adaptive bit allocation of content, allocates a higher bit rate to information-dense regions, ensures the effective extraction of key feature information, and thus obtains a latent representation with enhanced importance; compared with a single Gaussian model, the Gaussian mixture model is more flexible; it performs content adaptive optimization of the latent representation, and obtains a more reasonable and flexible probability distribution model by using Gaussian mixture distribution prediction, reducing redundant information in the latent representation or neural grammar, and improving the rate-distortion performance of the deep learning image compression model while obtaining reconstructed images with better visual quality.

[0024] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0026] Figure 1 This is a schematic diagram of a deep learning image compression model based on importance scale and Gaussian mixture model according to an embodiment of the present invention;

[0027] Figure 2 This is a schematic diagram of the importance scale network in an embodiment of the present invention. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0030] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0031] Please see Figure 1 This is a schematic diagram of a deep learning image compression model based on importance scale and Gaussian mixture model according to an embodiment of the present invention.

[0032] This embodiment provides a deep learning-based image compression method based on importance scale and Gaussian mixture model, including:

[0033] Step 1: The image to be processed is first analyzed and transformed by the main encoder to obtain the corresponding latent representation. The specific structure of the main encoder consists of four two-dimensional zero-padding 2D layers (ZeroPad2d), four convolutional layers, and three generalized divisive normalization (GDN) layers.

[0034] Step 2, please refer to Figure 2 The diagram below illustrates the importance scale network in an embodiment of the present invention. The main body of the importance scale network consists of a convolutional layer and three residual blocks. The features are then mapped to the range [-1, 1] using the hyperbolic tangent function (tanh) and the softsign function (softsign) in sequence. The latent representation obtained from the main encoder is used as input, and the importance-enhanced latent representation is obtained through the importance scale network. The importance scale network achieves adaptive bit rate allocation for feature content. For regions with larger latent values ​​in any channel, a higher bit rate is allocated to ensure that more feature information is extracted.

[0035] Step 3: The importance-enhanced latent representation obtained by the main encoder and importance scale network is divided into content stream and model stream in the channel dimension and processed separately.

[0036] Step 4: Process the latent representation of the content stream by using a super-prior model, a context model, and a pre-defined Gaussian mixture model to obtain a more accurate probability distribution to guide entropy encoding and decoding, thereby obtaining a more accurate reconstructed latent representation. This includes the following steps:

[0037] Step 4.1: The latent representation of the content stream is quantized, entropy encoded, and decoded using a super-prior encoder and decoder to obtain quantized information of auxiliary information for subsequent probability distribution modeling. The super-prior model consists of a super-prior encoder and a super-prior decoder. The super-prior encoder consists of three convolutional layers, each followed by a Rectified Linear Unit (ReLU) function. The purpose is to extract features from the input data through the convolutional layers and introduce non-linear characteristics using the ReLU function, thereby enhancing the model's expressive and learning capabilities. The super-prior decoder consists of three transposed convolutional layers arranged sequentially, each followed by a ReLU function. This structural design aims to upsample the input features through the transposed convolutional layers and introduce non-linear characteristics using the ReLU function, thereby enhancing the decoder's expressive and reconstruction capabilities.

[0038] Step 4.2: The latent representation is processed through a context model to obtain contextual information, providing a basis for subsequent entropy parameter estimation of the probability distribution and improving the accuracy of parameter estimation. This context model consists of three convolutional layers, each followed by a Leaky Rectified Linear Unit (LeakyReLU) function. The purpose is to effectively extract features through convolutional layers and introduce non-linear characteristics using the LeakyReLU function, thereby enhancing the model's expressive power and its ability to capture contextual information. In this way, the context model can provide a more accurate and reliable basis for subsequent entropy parameter estimation.

[0039] Step 4.3: The Gaussian mixture model consists of multiple Gaussian distributions with different means and variances, and different weights are assigned to each type of Gaussian distribution. Using multiple different types of Gaussian distributions can characterize a more complex probability distribution model, ensuring that the obtained information is accurate enough. The more accurate probability distribution is then used to guide entropy encoding and decoding, reducing redundant information and obtaining a more accurate reconstructed latent representation. The formula for the Gaussian mixture model is as follows:

[0040]

[0041] Where w represents the weight value of each Gaussian distribution, μ and σ represent the mean and variance of the Gaussian distribution, and K represents the total number of Gaussian models. Reconstructing the latent representation, The quantitative information representing auxiliary information is selected based on a comprehensive consideration of image compression effect and convergence speed, and a Gaussian mixture model with K=4 is chosen.

[0042] Step 5: For the latent representation of the model stream, obtain the kernel parameters of the last transposed convolutional layer of the main decoder using a neural syntax generator, a weight generator, and a pre-defined Gaussian mixture model. This specifically includes the following steps:

[0043] Step 5.1 The neural syntax generator consists of a multi-scale network, a global average pooling layer, and a fully connected layer. The multi-scale network fully extracts feature information at different scales from the latent representation in the model flow. Then, the global average pooling layer maps the features obtained at each scale to a one-dimensional latent vector and connects them. This connected feature vector is called the neural syntax, which provides feature information for the subsequent weight generator.

[0044] Step 5.2: Quantize, entropy encode, and decode the neural syntax, and use a more accurate probability distribution predicted by a Gaussian mixture model to guide the entropy encoding and decoding to obtain more accurate neural syntax information; the Gaussian mixture model used in this step is the same as in step 4.3.

[0045] Step 5.3: The main body of the weight generator consists of three fully connected layers, each followed by a LeakyReLU function as a non-linear activation, forming a multi-layer fully connected neural network. The decoded neural syntax information is passed as input to the weight generator, which then predicts the kernel parameters needed by the main decoder in the last transposed convolutional layer. Because this process is based on the dynamic changes in the input information, the transformation parameters at the decoding end are also dynamically adjusted to flexibly adapt to and process diverse input images, ensuring that the model can generate matching outputs when faced with different images.

[0046] Step 6: The main decoder consists of five transposed convolutional layers, with an embedded inverse GDN (Inverse Generalized Divisive Normalization) layer. After training the entire deep learning image compression model, the parameters of all layers except the last transposed convolutional layer are fixed. However, the kernel parameters of the last transposed convolutional layer are generated in real-time by the weight generator during decoding, thus highly dependent on the input image. The reconstructed latent representation obtained from the content stream and the kernel parameters of the last transposed convolutional layer obtained from the model stream are synthesized and transformed by the main decoder to obtain a reconstructed image with better visual effects.

[0047] For ease of description and concise expression, the above division of functional units and modules will be used as an example. In practical applications, the above functions can be assigned to different functional units or modules according to requirements. That is, the internal structure of the model can be divided into different functional units or modules to achieve all or part of the functions described above.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A deep learning-based image compression method based on importance scale and Gaussian mixture model, characterized in that, Includes the following steps: S1. The image to be processed is analyzed and transformed by the main encoder to obtain the latent representation; S2. The latent representation is passed through a preset importance scale network to adaptively allocate bits for different regions of content to obtain an importance-enhanced latent representation; S3. Divide the importance-enhanced latent representation into a content flow latent representation and a model flow latent representation in the channel dimension; S4. By processing the content stream using a priori model, a context model, and a preset Gaussian mixture model, the probability distribution of the potential representation is obtained to acquire the reconstructed potential representation. S5. Obtain the neural syntax of the latent representation of the model stream using a neural syntax generator; obtain the probability distribution of the neural syntax through the preset Gaussian mixture model to guide entropy encoding and decoding to obtain neural syntax information; feed the neural syntax information into a weight generator to obtain the kernel parameters of the last transposed convolutional layer of the main decoder. S6. The reconstructed latent representation and the kernel parameters of the last transposed convolutional layer of the decoder are synthesized and transformed by the main decoder to obtain the reconstructed image.

2. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S1: the main encoder includes four two-dimensional zero-padding 2D layers (ZeroPad2d), four convolutional layers, and three generalized divisive normalization (GDN) layers.

3. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S2: the importance scale network includes a convolutional layer and three residual blocks; an adaptive bit allocation strategy is adopted, which allocates a higher bit rate to regions with larger potential values ​​in the region to extract more feature information and obtain an importance-enhanced potential representation.

4. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S4: the super-prior model includes a super-prior encoder and a super-prior decoder; the super-prior encoder includes three groups of convolutional layers; the super-prior decoder includes three transposed convolutional layers; the convolutional layers and the transposed convolutional layers have rectified linear unit (ReLU) functions.

5. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In step S4: the context model contains three convolutional layers, which are connected by a Leaky Rectified Linear Unit (LeakyReLU) function; the context information of the latent representation is obtained through the context model.

6. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S4: the Gaussian mixture model consists of multiple Gaussian distributions with different means, variances, and weights, and the Gaussian mixture model is expressed as follows: Where w represents the weight value of each Gaussian distribution, μ and σ represent the mean and variance of the Gaussian distribution, and K represents the total number of Gaussian models. Represents the potential representation of reconstruction. Quantitative information representing auxiliary information.

7. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S5: the neural syntax generator includes a multi-scale network, a global average pooling layer, and a fully connected layer; the latent representation of the model flow extracts features through the multi-scale network, and after global average pooling, they are concatenated into a neural syntax feature vector for use by the subsequent weight generator.

8. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S5: the weight generator contains three fully connected layers, which are connected by a leaky Rectified Linear Unit (LeakyReLU) function.

9. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 1, characterized in that, In S6: the main decoder contains five transposed convolutional layers, which are connected by an inverse generalized divisive normalization layer (GDN); after the entire deep learning image compression model is trained, the parameters of all transposed convolutional layers except the last one are fixed values; the kernel parameters of the last transposed convolutional layer are generated in real time by the weight generator during decoding.

10. The deep learning image compression method based on importance scale and Gaussian mixture model according to claim 6, characterized in that: The total number of Gaussian models is K = 4.

Images