An image compression method based on a multi-source entropy model with non-uniform channels

Abstract

The present invention discloses an image compression method based on a multi-source entropy model with non-uniform channels. The image compression method includes: obtaining latent feature representation information y after an encoder processes picture information x; dividing the latent feature representation information y into five non-uniform feature segments y i , and feeding it together with the prior information Hyper_Params provided by the hyperprior network into a multi-source entropy model MSEM (Multisource Entropy Model) module to obtain the generated feature segment information #imgabs0# and δ i , δ i is the mean μ and variance σ of the feature segment y i ; obtaining the conditional probability distribution of the feature segment information #imgabs1# based on the mean μ and variance σ of the feature segment y i . After processing the i-th block of the feature segment, the generated feature segment information #imgabs2# Hyper_Params and y i+1 are fed into the MSEM i+1 module for cyclic processing. The number of channels fed to subsequent modules is reduced, thereby reducing the computational amount of the model and improving the efficiency of the model.

Description

Technical Field

[0001] The present invention relates to the field of image compression, and in particular, to an image compression method based on a multi-source entropy model with non-uniform channels. Background Art

[0002] Currently, the research on deep learning-based image compression method models has advanced by leaps and bounds. Compared with traditional image compression methods, it can achieve a lower compression ratio while maintaining higher reconstruction quality. As the most important part of the deep learning network framework, the entropy model mainly performs effective bit allocation, minimizing the number of encoding bits while maintaining the quality of the decoded image. The entropy model usually uses a deep learning model to learn the conditional probability distribution of the latent representation by understanding the correlation and statistical characteristics between pixel values in the image, helping to establish a probability model and achieve efficient compression. The performance of the entropy model directly affects the efficiency of the entire image compression network model. Therefore, many researchers focus on the research of the entropy model. For example, He et al. proposed using a checkerboard convolutional autoregressive context model, which shows excellent characteristics in terms of parallelism and complexity. Minnen et al. adopted a context model along the channel dimension to further enhance the parallel performance of the network.

[0003] Most research methods use only one context model to estimate the conditional probability, which inevitably leads to relatively general results. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide an image compression method based on a multi-source entropy model with non-uniform channels that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, there is provided an image compression method based on a multi-source entropy model with non-uniform channels, the image compression method comprising:

[0006] Obtaining the latent feature representation information y after the picture information x passes through the encoder;

[0007] Dividing the latent feature representation information y into five non-uniform feature segments y i , and feeding it together with the prior information Hyper_Params provided by the hyperprior network into the multi-source entropy model MSEM (Multisource Entropy Model) module to obtain the generated feature segment information and δ i , δ i being the mean μ and variance σ of the feature segment y i ;

[0008] According to the feature segment y iThe conditional probability distribution of the feature segment information is obtained from the mean μ and variance σ

[0009] After processing the i-th feature segment, the generated feature segment information Hyper_Params and y i+1 are fed into the MSEM i+1 module for loop processing.

[0010] Optionally, the multi-source entropy model MSEM specifically includes: a channel feature module, a spatial feature module, and a CWP module.

[0011] Optionally, the channel feature module specifically includes:

[0012] Convolutional layers of 1×1, 3×3, and 5×5 and three GELU layer activation functions. Convolution kernels of different sizes 1×1, 3×3, and 5×5 can capture feature information at different scales in the image;

[0013] 1×1 convolution is used for feature fusion; 3×3 and 5×5 convolutions can extract richer spatial channel features;

[0014] The three GELU layer activation functions include three residual RBS. The residual block, by introducing a skip connection, enables the network to more easily learn the residual between the input and output;

[0015] The result is subjected to a sigmoid operation, the output information of the two parts is multiplied to obtain k, and finally k is connected with the input information through a skip connection to obtain the output result.

[0016] Optionally, the spatial feature module specifically includes:

[0017] By learning the spatial distribution between pixels in the input feature information to capture the correlation between adjacent pixels, which provides a basis for the prediction of pixel values;

[0018] BPDiv is a module for dividing base points and non-base points. Entroy_Bpoint is an entropy parameter network used to generate the parameters of the conditional Gaussian entropy model of the base points. The parameters include the mean and variance, and the non-base points are predicted based on the base point information.

[0019] Optionally, the capturing the correlation between adjacent pixels by learning the spatial distribution between pixels in the input feature information specifically includes:

[0020] After the input information yi passes through the BPDiv module, it is divided into base points and non-base points

[0021] For base points Model the information after the Cat operation with the prior information Hyper_Params provided by the hyperprior network to obtain Params_Bpoint, and feed Params_Bpoint into the SPCM module to predict the non-basepoint information to obtain Params_NonBpoint;

[0022] Perform an addition operation on Params_Bpoint and Params_NonBpoint to obtain the output result.

[0023] Optionally, the CWP module specifically includes:

[0024] Adopt a multi-path structure, enabling the model to extract image features at different scales, capture input feature information more comprehensively, and enhance the expressive ability of the entropy model;

[0025] UpSample↑ is an upsampling layer. The first path consists of Conv1×1, Conv3×3, and a Conv3×3 convolutional layer, capturing the local spatial information of the input features and extracting edge, detail, and texture structure features in the input feature information;

[0026] Introduce a sliding window attention mechanism in the second path. By dividing the input image into multiple windows of a fixed size and independently calculating self-attention within each window, the model can focus on local features in the image, helping the model to concentrate on important regional features in the image;

[0027] The third path consists of a MaxPool layer and a Conv3×3 convolutional layer;

[0028] Merge the feature information of different scales and different abstraction levels extracted from the three paths, comprehensively considering the local details and global features of the input feature information.

[0029] A multi-source entropy model image compression method based on non-uniform channels provided by the present invention. The image compression method includes: obtaining the latent feature representation information y after the picture information x passes through the encoder; dividing the latent feature representation information y into five uneven feature segments y i , and feeding it together with the prior information Hyper_Params provided by the hyperprior network into the multi-source entropy model MSEM (Multisource Entropy Model) module to obtain the generated feature segment information and δ i , δ i is the mean μ and variance σ of the feature segment y i ; obtain the conditional probability distribution of the feature segment information based on the mean μ and variance σ of the feature segment y i ; The feature segment information generated after processing the i-th feature segment Hyper_Params and y i+1 are fed into the MSEM i+1 module for loop processing. The number of channels fed into subsequent modules is reduced, thereby reducing the computational amount of the model and improving the efficiency of the model.

[0030] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are hereinafter specifically exemplified. Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0032] Figure 1 It is a multi-source entropy model framework diagram provided by an embodiment of the present invention;

[0033] Figure 2 It is a channel feature module provided by an embodiment of the present invention;

[0034] Figure 3 It is a CWP module provided by an embodiment of the present invention;

[0035] Figure 4 It is a spatial feature module provided by an embodiment of the present invention;

[0036] Figure 5 It is a flowchart of the spatial feature module provided by an embodiment of the present invention. Detailed Embodiments

[0037] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0038] The terms "including" and "having" and any variations thereof in the embodiments of the specification and claims of the present invention and the drawings are intended to cover non-exclusive inclusion. For example, including a series of steps or units.

[0039] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0040] As Figure 1 shown, the present invention proposes a multi-source entropy model network framework based on non-uniform channels.

[0041] The meanings represented by the letters in the figure: ga and gs represent the encoder and decoder; ha and hs represent the encoder and decoder of the hyperprior network, and AE&AE is the arithmetic encoder-decoder.

[0042] Specifically, it includes:

[0043] (1) The input image information x obtains the latent feature representation information y after passing through the encoder.

[0044] (2) The latent feature representation information y is divided into five uneven feature segments y i , and together with the prior information Hyper_Params provided by the hyperprior network, they are fed into the MSEM (Multisource Entropy Model) module to obtain the generated feature segment information and δ i , δ i is the mean μ and variance σ of the feature segment y i .

[0045] (3) According to the mean μ and variance σ of the feature segment y i , the conditional probability distribution of the feature segment information is obtained

[0046] (4) After processing the i-th block of feature segments, the generated feature segment information Hyper_Params and y i+1 are fed into the MSEM i+1 module together, and so on.

[0047] MSEM is composed of three parts: the channel feature module g cf , the spatial feature module g pf and the CWP module (composed of CNN, W-MSA and Pooling). When processing the feature segment y i , these three modules are designed to operate in parallel, and finally the extracted information is aggregated.

[0048] On the one hand, in the deep learning image compression model, the learned latent variable information shows a tendency of uneven distribution, where a large amount of information is concentrated in a few channels. However, after introducing the channel feature module, this information will show a certain serial nature. The earlier processed channel groups contain more abundant information because they are given priority in the encoding and decoding processes. We use the information to continuously predict subsequent channels (such as Figure 1 the orange, purple, red, and yellow curves in

[0049] y = ga(x) (1)

[0050]

[0051] Channel Feature Module:

[0052] The channel feature module uses the correlation information between channels of the input feature information to improve the efficiency and quality of image compression. By capturing the correlation between channels, it helps to extract information that improves the accuracy of probability estimation. The framework of the entire module is as Figure 2 shown. The entire module consists of two parts. One part consists of convolutional layers of 1×1, 3×3, and 5×5 and three GELU layers (activation functions). Convolutional kernels of different sizes (1×1, 3×3, 5×5) can capture feature information of different scales in the image. The 1×1 convolution performs feature fusion; the 3×3 and 5×5 convolutions can extract richer spatial channel features. The other part consists of three residual blocks (RBS). The residual blocks introduce skip connections, enabling the network to more easily learn the residuals between the input and output, which helps to alleviate the vanishing gradient problem in deep networks. Then, a sigmoid operation is performed on the result (represented by the σ symbol in the figure). After that, the output information of the two parts is multiplied to obtain k. Finally, k is connected with the input information through a skip connection to obtain the output result. The combination of such multi-scale convolutional layers and residual blocks enables the module to extract and fuse rich image features, which is beneficial to alleviating the problem of information loss existing in the training process of the model and improving the performance of the model.

[0053] Spatial Feature Module:

[0054] The spatial feature module captures the correlation between adjacent pixels by learning the spatial distribution between pixels in the input feature information. This correlation provides a basis for predicting pixel values, reduces the amount of information required in the encoding process, and achieves efficient compression. The overall framework of this module is as Figure 4As shown, BPDiv is the module for dividing base points and non-base points. Entroy_Bpoint is the entropy parameter network, which is used to generate the parameters (mean + variance) of the conditional Gaussian entropy model for base points. SPCM (Spatial Prediction Context Module) predicts non-base point information based on base point information. The input information is y i After passing through the BPDiv module, it is divided into base points and non-base points As Figure 5 shown (the green part is the segmented base point information the yellow part is the non-base point information predicted based on the base point confidence ). This design can reduce the complexity of the model and make the model easier to train. Then, the information after the Cat operation on the base points and the prior information Hyper_Params provided by the hyperprior network is modeled to obtain Params_Bpoint. Params_Bpoint is fed into the SPCM module to predict non-base point information to obtain Params_NonBpoint. Finally, Params_Bpoint and Params_NonBpoint are added together to obtain the output result. In the SPCM module (the specific process is as Figure 4 shown), by introducing a sliding window attention mechanism in this module, the model pays more attention to information related to base points when predicting non-base point information, so as to enhance the local perception ability of the model and pay more attention to information related to base points when predicting non-base point information, so as to enhance the local perception ability of the model, thereby improving the accuracy and efficiency of prediction. The whole process is described as follows:

[0055]

[0056] Params_NonBpoint = SPCM(Params_Bpoint) (7)

[0057] output = Params_NonBpoint + Params_Bpoint (8)

[0058] CWP module:

[0059] The CWP module adopts a multi-path structure, enabling the model to extract image features at different scales, thereby capturing input feature information more comprehensively and increasing the expressive ability of the entropy model. The entire model framework is as Figure 3As shown. UpSample↑ is the upsampling layer. The first path consists of Conv1×1, Conv3×3, and Conv3×3 convolutional layers, aiming to capture the local spatial information of the input features to effectively extract structural features such as edges, details, and textures in the input feature information. In the second path, a sliding window attention mechanism is introduced. By dividing the input image into multiple windows of a fixed size and independently calculating self-attention within each window, the computational complexity is significantly reduced, enabling the model to focus on local features in the image, helping the model to concentrate on important regional features in the image, and enhancing the ability to express key information. The third path consists of a MaxPool layer and a Conv3×3 convolutional layer, aiming to reduce the computational complexity while retaining the main feature information to maintain the integrity of the image features. Finally, the feature information of different scales and different abstraction levels extracted from the three paths is merged to comprehensively consider the local details and global features of the input feature information.

[0060] Advantageous effects: The present invention is based on an uneven-channel multi-source entropy model (Unevenly Multi-source Entropy Model), which is mainly composed of a channel feature module, a spatial feature module, and a CWP module (composed of a CNN, a W-MSA, and a pooling layer). The channel feature module helps improve the information for accurate probability estimation by capturing the correlation between channels. The spatial feature module captures the correlation between adjacent pixels by learning the spatial distribution between pixels in the input feature information, which provides a basis for predicting pixel values, thereby reducing the amount of information required during the encoding process and achieving efficient compression. The CWP module can extract image features at different scales, thus capturing the input feature information more comprehensively and increasing the expression ability of the entropy model. The three modules independently and parallelly process the feature segment information, which is beneficial for better feature extraction while improving efficiency. In addition, the model adopts an uneven-channel allocation method to divide the potential representation feature information into five uneven feature segments to reduce the number of channels fed to the subsequent modules, thereby reducing the computational amount of the model and improving the efficiency of the model. Compared with other advanced models in three public datasets (Kodak, Tecnick, CLIC), the model has better visual effects in terms of detail restoration and structural features, and also has advantages in various evaluation indicators of image compression.

[0061] The above specific implementation manners have further elaborated on the purpose, technical solutions, and advantageous effects of the present invention. It should be understood that the above are only the specific implementation manners of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-source entropy model image compression method based on non-uniform channels, characterized in that, The described image compression method includes: Obtaining the latent feature representation information y after the picture information x passes through the encoder; Divide the potential feature representation information y into five uneven feature segments y i , and feed it together with the prior information Hyper_Params provided by the hyperprior network into the multi-source entropy model MSEM (Multisource Entropy Model) module to obtain the generated feature segment information and δ i , δ i is the mean μ and variance σ of the feature segment y i ; According to the mean μ and variance σ of the feature segment y i obtain the conditional probability distribution of the feature segment information The feature segment information generated after processing the i-th feature segment Hyper_Params and y i+1 are fed into the MSEM i+1 module for loop processing; The multi-source entropy model MSEM specifically includes: a channel feature module, a spatial feature module, and a CWP module; The channel feature module specifically includes: 1×1, 3×3, and 5×5 convolutional layers and three GELU layer activation functions. Convolutional kernels of different sizes 1×1, 3×3, and 5×5 can capture feature information of different scales in the image; 1×1 convolution is used for feature fusion; 3×3 and 5×5 convolutions can extract richer spatial channel features; The three GELU layer activation functions include three residual RBS. The residual block makes the network able to more easily learn the residual between the input and the output by introducing skip connections; Perform a sigmoid operation on the result, multiply the output information of the two parts to get k, and finally make a skip connection between k and the input information to get the output result; The spatial feature module specifically includes: Capturing the correlation between adjacent pixels by learning the spatial distribution between pixels in the input feature information, and the correlation provides a basis for the prediction of pixel values; BPDiv is the module for dividing base points and non-base points, and Entroy_Bpoint is the entropy parameter network, which is used to generate the parameters of the conditional Gaussian entropy model of the base points. The parameters include the mean and variance, and the non-base point information is predicted through the base point information; The CWP module specifically includes: Adopting a multi-path structure, enabling the model to extract image features at different scales, capturing the input feature information more comprehensively, and increasing the expression ability of the entropy model; UpSample↑ is the upsampling layer. The first path consists of Conv1×1, Conv3×3, and a Conv3×3 convolutional layer, capturing the local spatial information of the input features and extracting the edge, detail, and texture structure features in the input feature information; Introduce a sliding window attention mechanism in the second path. By dividing the input image into multiple windows of a fixed size and independently calculating self-attention within each window, the model can focus on the local features in the image to help the model concentrate on the important region features in the image; The third path consists of a MaxPool layer and a Conv3×3 convolutional layer; Merge the feature information of different scales and different abstraction levels extracted from the three paths, and comprehensively consider the local details and global features of the input feature information.

2. The multi-source entropy model image compression method based on non-uniform channels according to claim 1, wherein, The capturing the correlation between adjacent pixels by learning the spatial distribution between pixels in the input feature information specifically includes: Input information y i After passing through the BPDiv module, it is divided into base points and non-base points For the base point Model the information after performing a Cat operation on the prior information Hyper_Params provided by the hyper-prior network to obtain Params_Bpoint, and feed Params_Bpoint into the SPCM module to predict non-base point information to obtain Params_NonBpoint; Performing an addition operation on Params_Bpoint and Params_NonBpoint to obtain the output result.

Images