Progressive image encoding, decoding, compression method based on autoregressive entropy modeling

Abstract

The application provides a progressive image coding, decoding and compression method based on autoregressive entropy modeling, comprising the following steps: obtaining a multi-channel feature map of an image to be coded, super-prior feature, super-prior binary code stream and super-prior information; grouping the multi-channel feature map in space, quantizing each feature dead zone of each spatial group to obtain quantized features of a highest quality enhancement layer; obtaining initial code words and initial quantized features, aligning the code words, and sequentially obtaining quantized features of L quality layers with low quality; calculating code words of a basic layer, and encoding to obtain a binary code stream of the basic layer; layer by layer, calculating code words, and encoding to obtain binary code streams of enhancement layers; and merging the super-prior binary code stream, the binary code stream of the basic layer and the binary code streams of the L enhancement layers to form a binary code stream. The application can adapt to real image communication scenes with dynamic bandwidth changes and heterogeneous access terminals with a single binary code stream, improve user experience, and reduce the overall bandwidth demand of network transmission.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a progressive image encoding, decoding, and compression method based on autoregressive entropy modeling. Background Technology

[0002] With the rapid development of deep learning, end-to-end image compression has become increasingly popular. It utilizes deep neural networks to jointly optimize various image compression modules, achieving superior compression performance. However, in real-world scenarios, different terminals, heterogeneous networks, and different users have varying bitrate requirements, necessitating models capable of adapting to different bitrates. Subsequently, researchers explored variable bitrate models, where a single model can achieve multiple bitrates; however, their bitstreams are not scalable, still placing a burden on storage.

[0003] Progressive compression encodes an image into a single bitstream, which can be divided into a base layer and several enhancement layers. The base layer is used to recover a coarse-grained image, while the enhancement layers are used to progressively refine the image quality. A literature search of existing technologies revealed that Lu et al.'s "Progressive Neural Image Compression With Nested Quantization And Latent Ordering" presented at the 2021 IEEE International Conference on Image Processing (ICIP) and Lee et al.'s "DPICT: Deep Progressive Image Compression Using Trit-Planes" presented at the 2022 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) are considered the earliest two progressive image compression coding schemes based on convolutional neural networks (CNNs), which achieve progressive image compression coding by adjusting the quantization step size and employing a super-prior model for entropy modeling. The "Context-Based Trit-Plane Coding for Progressive Image Compression" presented by Lee et al. at the 2023 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) is the first scheme to use a priori model and already encoded information as context for entropy modeling.

[0004] However, the performance of the above-mentioned model that uses the super-prior model for entropy modeling alone is poor. At the same time, although the performance of the model that uses context is improved, its encoding and decoding speed is slow because of the way it uses context. Therefore, the progressive coding scheme based on the autoregressive model must be further improved. Summary of the Invention

[0005] This invention addresses the shortcomings of current asymptotic coding schemes: poor performance of advanced prior models and high complexity of asymptotic coding methods utilizing contextual information. It provides an autoregressive entropy modeling-based asymptotic image encoding, decoding, and compression method.

[0006] A first aspect of the present invention provides a progressive image coding method based on autoregressive entropy modeling, comprising:

[0007] Obtain multi-channel feature maps of the image to be encoded;

[0008] Based on the multi-channel feature map, the prior features, the prior binary code stream, and the prior information are obtained.

[0009] The multi-channel feature map is divided into K spatial groups according to each channel, where K is a positive integer;

[0010] For each feature of each spatial group of the multi-channel feature map, prediction is made based on prior information and the quantized features of that channel. Dead zone quantization is performed given the dead zone and other quantization interval sizes to obtain the quantized features of the highest quality enhancement layer.

[0011] The quantization interval of the dead zone quantization is scaled up and down with a fixed scale factor. The initial codeword and initial quantization feature are obtained through dead zone quantization. The quantization feature is obtained by aligning the codeword and is used as the quantization feature of the current quality layer. The quantization features of L low quality layers are obtained in sequence, where: L is a positive integer. The lowest quality layer with the largest quantization interval is the base layer, and the remaining L-1 quality layers are the first to the L-1 enhancement layers in descending order of quantization interval.

[0012] The codewords of the basic layer are calculated based on the quantization characteristics of the basic layer. The probability is estimated based on the quantization interval. The codewords of the basic layer are then encoded to obtain the binary code stream of the basic layer.

[0013] Following the order of the first to the Lth enhancement layers, codewords are calculated layer by layer based on the quantization features of the corresponding enhancement layer, the estimated probability is calculated, and the codewords are encoded to obtain L binary code streams of enhancement layers;

[0014] The aforementioned super-prior binary code stream, basic layer binary code stream, and L enhancement layer binary code streams are merged to form a binary code stream.

[0015] Optionally, for each feature in each spatial group of the multi-channel feature map, prediction is performed based on prior information and the quantized features of that channel. Dead-zone quantization is then performed given the dead zone and other quantization interval sizes to obtain the codewords and quantized features of the highest quality enhancement layer, including:

[0016] Contextual information is extracted based on the quantized spatial grouping of the current channel and the quantized features of the current spatial grouping. This information is then fused with prior information to obtain the predicted value μ of the current feature y in the highest quality enhancement layer. L ;

[0017] Calculate the prediction residual of the current feature y in the highest quality enhancement layer.

[0018] For prediction residuals Perform dead-zone quantization based on the codeword mapping function. Obtain the initial codewords for the highest quality enhancement layer. Specifically:

[0019]

[0020] Here, the dead zone size p in dead zone quantization. L =1, other quantization interval sizes q L =1, sign(·) returns the sign of the input, and max(·,·) returns the larger of the two inputs;

[0021] For the initial codeword According to the reconstruction function Obtain the quantization characteristics of the highest quality enhancement layer

[0022]

[0023] Here, the reconstruction function is

[0024]

[0025] Among them, |·| returns the absolute value of the input.

[0026] Optionally, the step of sequentially scaling the quantization interval of dead-zone quantization with a fixed scale factor, obtaining initial codewords and initial quantization features through dead-zone quantization, aligning the codewords to obtain quantization features as the quantization features of the current quality layer, and sequentially obtaining L quantization features of lower quality layers includes:

[0027] Following the order of quality from high to low, starting from the Lth quality layer, codewords and quantization features are obtained sequentially. Specifically, for the lth (l = L, ..., 1)th quality layer, the steps are as follows:

[0028] Based on the predicted μ of the (l+1)th mass layerl+1 With dead zone size p l Other quantization interval sizes q l Dead-zone quantization is performed using a codeword mapping function. Obtain the initial codeword

[0029]

[0030] For the initial codeword According to the reconstruction function Obtain initial quantization features

[0031]

[0032] Contextual information is extracted based on the quantized spatial grouping of the current channel and the quantized features of the current spatial grouping. This information is then fused with prior information to obtain the predicted value μ of the current feature y in the l-th enhancement layer. l ;

[0033] Initial quantization features Transform to quantization interval q l Uniform quantization, based on the predicted μ of the (l+1)th quality layer l+1 Obtain the uniform quantization feature ξ l :

[0034]

[0035] Among them, the codeword mapping function of uniform quantization β(·) is the forward conversion function between dead-zone quantization and uniform quantization;

[0036] Aligning codewords to obtain the quantization features of the l-th quality layer

[0037]

[0038] Where β -1 (·) is the inverse conversion function between dead zone quantization and uniform quantization.

[0039] Optionally, the step of calculating the codewords of the basic layer based on the quantization characteristics of the basic layer, estimating the probability based on the quantization interval, and encoding the codewords of the basic layer to obtain the basic layer binary code stream includes:

[0040] Based on the quantization characteristics of the basic layer Calculate the basic layer codeword c 0 :

[0041]

[0042] Calculate the quantization interval of the basic layer codeword [a0 ,b 0 ],in

[0043]

[0044]

[0045] Based on the Gaussian probability density function, the basic layer codeword c 0 Perform probability estimation

[0046]

[0047] in

[0048] According to the probability p(c) 0 Arithmetic encoding is performed to obtain the basic layer binary code stream.

[0049] Optionally, the step of calculating codewords layer by layer according to the quantization features of the corresponding enhancement layer in the order of the 1st to the Lth enhancement layers, calculating the estimated probability, and encoding the codewords to obtain the binary code stream of the enhancement layer includes:

[0050] For any l-th (l=1,…,L) enhancement layer, based on the quantization characteristics of the (l-1)-th layer... Calculate the enhancement layer codeword c l :

[0051]

[0052] Calculate the quantization interval of the enhancement layer codeword [a 0 ,b 0 ],in

[0053]

[0054]

[0055] Based on Gaussian probability density function, the enhancement layer codeword c l Perform probability estimation

[0056]

[0057] in

[0058]

[0059]

[0060] According to the probability p(c) l Arithmetic encoding is performed to obtain the binary code stream of the l-th enhancement layer.

[0061] Preferably, the sizes of other quantization intervals for any l-th and (l+1)-th quality layers are scaled by a fixed scale factor s:

[0062] q l =s·q l+1

[0063] The dead zone size p of the l-th mass layer l And other quantization interval sizes q l They are respectively:

[0064] p l =2s L-l -1

[0065] q l =s L-l

[0066] Preferably, a forward transformation function β(x) is used to convert the quantization features of dead-zone quantization into the corresponding quantization features of uniform quantization:

[0067]

[0068] Using the inverse transformation function β -1 (x) Reconstruct the quantization features of uniformly quantized codewords into those of dead-zone quantization:

[0069]

[0070] A second aspect of the present invention provides an image progressive decoding method based on autoregressive entropy modeling, used to decode the encoding obtained by the above decoding method, comprising:

[0071] The obtained bitstream is decomposed into a priori binary bitstream, a basic layer binary bitstream, and L enhancement layer binary bitstreams;

[0072] The prior information is obtained by reconstructing the prior binary code stream through a priori decoding neural network.

[0073] Contextual information is extracted from the quantized spatial groupings of the current channel and the decoded quantized features of the current spatial groupings. This information is then fused with prior information to obtain the predicted value μ of the current feature y in the base layer. 0 The basic layer codeword c is obtained by decoding the basic layer binary codestream by combining the probability estimates of the basic layer dead zone and other quantization intervals. 0 According to the reconstruction function Obtain the quantization features of the base layer

[0074]

[0075] Following the order of quality layers from low to high, starting with the first enhancement layer, the codewords are decoded and quantized features are obtained sequentially for reconstruction. For the l-th enhancement layer (l = 1, ..., L), the codewords are decoded from the binary codestream of the l-th enhancement layer. The specific steps are as follows:

[0076] Based on the quantization representation decoded by the (l-1)th enhancement layer Predicted Gaussian mean μ l-1 and the quantization interval length q l-1 Obtain the quantization feature of the (l-1)th enhancement layer within the quantization interval length q l-1 Uniform quantization representation of η l :

[0077]

[0078] When l=1, the first enhancement layer is obtained based on the quantization characteristics, Gaussian mean, and quantization interval length of the base layer, i.e., the superscript... 0 Refers to the basic layer.

[0079] Based on the obtained η l The predicted Gaussian mean μ l and the quantization interval length q l-1 Obtain the quantization feature of the l-th enhancement layer within the quantization interval length q. l-1 Uniform quantization representation below:

[0080]

[0081] Obtain the quantization feature of the l-th enhancement layer in a dead zone of size p l-1 Other quantization interval sizes q l-1 Dead zone quantization representation:

[0082]

[0083] Using the predicted Gaussian distribution parameters of the l-th enhancement layer, combined with the estimated conditional probabilities of the l-th enhancement layer dead zone and other quantization intervals, the l-th enhancement layer bitstream is decoded to obtain the l-th enhancement layer codeword c. l ;

[0084] Based on the obtained l-th enhancement layer codeword c l The obtained Predicted Gaussian mean μ l And the dead zone size p l Other quantization interval sizes q l The representation of the quantization feature of the l-th enhancement layer under dead zone quantization is obtained:

[0085]

[0086] The quantized features obtained from decoding are input into a decoding neural network to obtain multiple reconstructed images of varying quality from low to high.

[0087] The third invention provides a progressive image compression method based on autoregressive entropy modeling, comprising:

[0088] Obtain the image to be compressed;

[0089] The progressive image coding method based on the above autoregressive entropy modeling obtains the compressed image binary bitstream;

[0090] The progressive image decoding method based on the above-mentioned autoregressive entropy modeling decompresses the compressed image binary code stream to obtain the reconstructed image.

[0091] A fourth aspect of the present invention provides a progressive image compression system, the progressive image compression system comprising an encoder and a decoder, including:

[0092] The encoder uses the aforementioned autoregressive entropy modeling progressive image coding method to encode the image to be compressed and obtain the compressed image binary bitstream.

[0093] The decoder uses the aforementioned autoregressive entropy modeling progressive image decoding method to decode the compressed image binary stream and obtain the reconstructed image.

[0094] A fifth aspect of the present invention provides a progressive image compression apparatus, the progressive image compression apparatus comprising a memory and a processor, the memory storing a program executable by the processor, the processor executing the program and performing any of the following methods:

[0095] -The above-mentioned progressive image coding method based on autoregressive entropy modeling, or,

[0096] -The above-mentioned autoregressive entropy modeling progressive image decoding method, or,

[0097] -The above-mentioned progressive image compression method based on autoregressive entropy modeling.

[0098] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0099] The autoregressive entropy modeling-based progressive image encoding, decoding, and compression method described above can generate arbitrarily truncated single binary bitstreams, corresponding to decoded images of different qualities at different bitrates. This allows for seamless switching in scenarios with dynamically changing network bandwidth during image transmission, ensuring continuity of image transmission and improving user experience. Furthermore, it can provide arbitrarily truncated single binary bitstreams adapted to access terminals with different resolutions, eliminating the need to switch between multiple bitstream versions of different qualities and reducing the overall bandwidth requirements of network transmission.

[0100] The autoregressive entropy modeling progressive image encoding, decoding, and compression method described above in this invention can save 8.05% of the bitstream overhead and improve rate-distortion performance compared to the current progressive coding standard HEVC, while maintaining the same image reconstruction quality.

[0101] This invention can be seamlessly applied to existing deep learning image compression models, supports context-based autoregressive entropy modeling, and can quickly achieve quantization of different quality layers by changing the quantization interval through the scale factor without retraining. It ensures a one-to-one mapping between each quality layer under different scale factors, does not introduce redundancy, and further reduces the coding bitrate and improves rate-distortion performance compared with existing methods.

[0102] This invention presents a progressive image compression method based on autoregressive entropy modeling. This method is built upon existing deep learning image compression models (neural networks). Its beneficial effect on rate-distortion performance lies in its ability, as mentioned above, to avoid introducing redundancy during progressive encoding, thereby reducing the encoding bitrate. Therefore, it can improve the rate-distortion performance of progressive image compression on natural images of different scenarios and resolutions. Thus, this invention has strong practical application value, especially in real-world image communication scenarios. It can adapt to real-world scenarios with dynamically changing bandwidth and heterogeneous access terminals, reducing overall bandwidth requirements and improving user experience. Attached Figure Description

[0103] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0104] Figure 1 This is a flowchart of a progressive image coding method for autoregressive entropy modeling according to an embodiment of the present invention;

[0105] Figure 2 This is a flowchart of a progressive image coding method for autoregressive entropy modeling in a preferred embodiment of the present invention.

[0106] Figure 3 This is a flowchart of a progressive image decoding method based on autoregressive entropy modeling in one embodiment of the present invention;

[0107] Figure 4 This is a flowchart of a progressive compression method according to an embodiment of the present invention. Detailed Implementation

[0108] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but should not be construed as limiting the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These modifications and improvements all fall within the scope of protection of the present invention.

[0109] To address the shortcomings of current asymptotic coding schemes—namely, the poor performance of super-prior models and the high complexity of context-based asymptotic coding methods—this invention proposes a context-based asymptotic coding method. This method utilizes an autoregressive model for entropy modeling, generating different entropy parameters for each layer. Then, a pre-defined scaling factor is used to scale the feature maps, resulting in feature maps corresponding to different code rates and qualities. To achieve lower complexity, "codeword alignment" is employed to address the issue of varying entropy parameters across layers due to different quality context inputs, which pre-defines the code table. Without any post-processing modules, the performance is comparable to current mainstream context-based asymptotic coding schemes, with a significantly reduced time complexity. Detailed explanations follow.

[0110] Reference Figure 1 The diagram shows a flowchart of a progressive image coding method based on autoregressive entropy modeling according to an embodiment of the present invention. This embodiment provides a progressive image coding method based on autoregressive entropy modeling, including steps S1-S8. Specifically:

[0111] S1. Obtain the multi-channel feature map of the image to be encoded;

[0112] S2. Based on the multi-channel feature map, obtain the prior features, the prior binary code stream, and the prior information;

[0113] S3. Divide the multi-channel feature map into K spatial groups according to each channel, where K is a positive integer;

[0114] S4. For each feature of each spatial group of the multi-channel feature map, predict based on the prior information and the quantized features of that channel, perform dead zone quantization given the dead zone and other quantization interval sizes, and obtain the quantized features of the highest quality enhancement layer.

[0115] S5. The quantization interval of the dead zone quantization is scaled up and down with a fixed scale factor. The initial codeword and initial quantization feature are obtained through dead zone quantization. The quantization feature is obtained by aligning the codeword and is used as the quantization feature of the current quality layer. The quantization features of L low quality layers are obtained in sequence (relative to the quantization feature of the highest quality enhancement layer in S4). Where: L is a positive integer. The lowest quality layer with the largest quantization interval is the base layer. The remaining L-1 quality layers are the first to the L-1 enhancement layers in descending order of quantization interval.

[0116] S6. Calculate the codewords of the basic layer based on the quantization characteristics of the basic layer, estimate the probability based on the quantization interval, and encode the codewords of the basic layer to obtain the binary code stream of the basic layer.

[0117] S7. Following the order of the first to the Lth enhancement layers, calculate the codewords layer by layer based on the quantization features of the corresponding enhancement layer, calculate the estimated probability, and encode the codewords to obtain the binary code stream of the enhancement layer.

[0118] S8. Merge the super-prior binary code stream, the basic layer binary code stream, and L enhancement layer binary code streams to form a binary code stream.

[0119] In the above embodiment, the highest quality enhancement layer in S4 is not among the 1st to L-1st enhancement layers, but can be considered as the Lth enhancement layer. That is, the quality order in progressive coding is from the highest quality enhancement layer (Lth), the L-1st enhancement layer, ..., the 1st enhancement layer, and the base layer. The quantization features of the highest quality enhancement layer in S4 can be directly obtained from the multi-channel feature map without codeword alignment. In addition, in the above embodiment, the highest quality enhancement layer must exist in progressive coding, that is, there will always be one enhancement layer and one base layer.

[0120] This embodiment forms a bitstream by merging a base layer and L enhancement layers. According to the encoding method, a single base layer, or a base layer and any of the first l (l = 1, 2, ..., L) enhancement layers, can provide decoded images of different qualities. Furthermore, quality can be enhanced by receiving and decoding more enhancement layers without needing to decode the entire bitstream each time. Therefore, it is possible to generate arbitrarily truncated single binary bitstreams corresponding to decoded images of different qualities at different bitrates. This is suitable for seamless switching in scenarios where network bandwidth changes dynamically during image transmission, ensuring the continuity of image transmission and improving user experience.

[0121] Figure 2 This is a flowchart of a progressive image coding method based on autoregressive entropy modeling according to a preferred embodiment of the present invention. In this preferred embodiment, the progressive image coding method based on autoregressive entropy modeling may include steps S110-S180.

[0122] S110, Image feature extraction: Obtain the image to be encoded, and obtain a multi-channel feature map of the image to be encoded through an encoding neural network;

[0123] S120, Extraction of super-prior information: The multi-channel feature map is processed by a super-prior coding neural network to obtain super-prior features, quantized and encoded to obtain a super-prior binary code stream, and reconstructed by a super-prior decoding neural network to obtain super-prior information;

[0124] S130, Feature map channel grouping: Divide the multi-channel feature map into K spatial groups according to each channel, where K is a positive integer;

[0125] For example, in one embodiment, each channel is divided into two spatial groups.

[0126] S140, Quantization of the highest quality enhancement layer: For each feature of each spatial group of the multi-channel feature map, prediction is made based on the prior information and the quantized features of that channel. Dead zone quantization is performed given the dead zone and the size of other quantization intervals to obtain the quantized features of the highest quality enhancement layer.

[0127] S150, Quantization of other quality layers: The quantization interval of the dead zone quantization is scaled up and down with a fixed scale factor. The initial codeword and initial quantization feature are obtained through dead zone quantization. The quantization feature is obtained by aligning the codeword and is used as the quantization feature of the current quality layer. The quantization features of L lower quality layers are obtained in sequence, where: L is a positive integer. The lowest quality layer with the largest quantization interval is the base layer. The remaining L-1 quality layers are the first to the L-1 enhancement layers in descending order of quantization interval.

[0128] S160, Basic Layer Encoding: Calculate the codewords of the basic layer based on the quantization characteristics of the basic layer, estimate the probability based on the quantization interval, and encode the codewords of the basic layer to obtain the basic layer binary code stream.

[0129] S170, Enhancement layer coding: In the order of the 1st to the Lth enhancement layers, calculate the codewords layer by layer based on the quantization features of the corresponding enhancement layer, calculate the estimated probability, and encode the codewords to obtain the binary code stream of the enhancement layer;

[0130] S180, stream merging: Merge the super-prior binary stream, the basic layer binary stream, and L enhancement layer binary streams to form a binary stream.

[0131] In this embodiment, codeword alignment technology is used for the formation of the base layer and enhancement layer, and the aligned codewords and intervals are used for encoding. This enables the introduction of autoregressive entropy modeling in progressive encoding, improving rate distortion performance. At the same time, it supports parallel processing of spatial context. By dividing the spatial group into two groups, two-stage encoding can be adopted. The first spatial group is encoded using super-prior, and the second stage uses autoregressive entropy modeling encoding, which improves parallelism and speeds up processing.

[0132] This embodiment is based on the fact that a single bitstream can be truncated into a basic layer and L enhancement layers to form decoded images of different qualities, which can be adapted to access terminals with different resolutions.

[0133] To perform progressive coding, in one possible implementation, S140 in the above embodiment can be implemented by the following method, specifically:

[0134] S141. Extract contextual information based on the quantized spatial grouping of the current channel and the quantized features of the current spatial grouping, and fuse it with the prior information to obtain the predicted value μ of the current feature y in the highest quality enhancement layer. L ;

[0135] S142. Calculate the prediction residual of the current feature y in the highest quality enhancement layer.

[0136] S143, Regarding the predicted residuals Perform dead-zone quantization based on the codeword mapping function. Obtain the initial codewords for the highest quality enhancement layer. Specifically:

[0137]

[0138] Here, the dead zone size p in dead zone quantization. L =1, other quantization interval sizes q L =1, sign(·) returns the sign of the input, and max(·,·) returns the larger of the two inputs;

[0139] S144. The initial codeword According to the reconstruction function Obtain the quantization characteristics of the highest quality enhancement layer

[0140]

[0141] Here, the reconstruction function is

[0142]

[0143] Among them, |·| returns the absolute value of the input.

[0144] Because autoregressive entropy modeling leads to the problem of parameterized distribution of changes between different quality layers, conditional probability cannot be modeled. Therefore, in the above embodiment, steps S141-S144 are proposed to solve this problem, first quantifying the highest quality layer.

[0145] In one possible implementation, S150 in the above embodiments can be implemented by the following method, specifically:

[0146] S151, Based on the predicted μ of the (l+1)th quality layer l+1 With dead zone size p l Other quantization interval sizes q l Dead-zone quantization is performed using a codeword mapping function. Obtain the initial codeword

[0147]

[0148] S152, the initial codeword According to the reconstruction function Obtain initial quantization features

[0149]

[0150] S153. Extract contextual information based on the quantized spatial grouping of the current channel and the quantized features of the current spatial grouping, and fuse it with the prior information to obtain the predicted value μ of the current feature y in the l-th enhancement layer. l ;

[0151] S154, Initial quantization features Transform to quantization interval q l Uniform quantization, based on the predicted μ of the (l+1)th quality layer l+1 Obtain the uniform quantization feature ξ l :

[0152]

[0153] Among them, the codeword mapping function of uniform quantization β(·) is the forward conversion function between dead-zone quantization and uniform quantization:

[0154]

[0155] S155. Align the codewords to obtain the quantization features of the l-th quality layer.

[0156]

[0157] Where β -1 (·) is the inverse conversion function between dead-zone quantization and uniform quantization:

[0158]

[0159] The above S150-S155 describes the quantization method used during encoding. Based on the initial codewords obtained from the highest quality layer, the position of the codewords in each layer is gradually adjusted. This enables embedded quantization, thereby achieving progressive coding, and also allows for conditional probability modeling, pre-setting code tables, and accelerating encoding and decoding speeds.

[0160] In some embodiments, in step S4 above, the sizes of other quantization intervals of any l-th and (l+1)-th quality layers are scaled by a fixed scale factor s:

[0161] q l =s·q l+1

[0162] The dead zone size p of the l-th mass layer l And other quantization interval sizes q l They are respectively:

[0163] p l =2s L-l -1

[0164] q l =s L-l .

[0165] The aforementioned fixed scaling factor s is generally selected as a positive integer. For example, in a specific embodiment, to make the quality of the quality layers as close as possible, a value of 2 can be selected. Using a fixed scaling factor s for scaling ensures that the quantization interval of the higher quality layer is embedded in the lower quality layer, thereby enabling the rapid determination of quantization features and quantization intervals, and ensuring a one-to-one correspondence between the codewords of each layer.

[0166] In one possible implementation, S160 in the above embodiments can be implemented by the following method, specifically:

[0167] S161. Based on the quantization characteristics of the basic layer Calculate the basic layer codeword c 0 :

[0168]

[0169] S162. Calculate the quantization interval of the basic layer codeword [a 0 ,b 0 ],in

[0170]

[0171]

[0172] S163. Based on Gaussian probability density function, the basic layer codeword c 0 Perform probability estimation

[0173]

[0174] in

[0175] S164, According to probability p(c 0 Arithmetic encoding is performed to obtain the basic layer binary code stream.

[0176] In the above S160, the basic layer codeword encoding part is involved. Steps S161-S164 are used to obtain the aligned basic layer codewords and their corresponding quantization intervals to obtain the basic layer probability estimate and perform encoding. This can avoid codeword redundancy in the basic layer encoding and realize progressive encoding.

[0177] In one possible implementation, S170 in the above embodiments can be implemented by the following method, specifically:

[0178] S171. For any l-th (l = 1, ..., L) enhancement layer, based on the quantization characteristics of the (l-1)-th layer... Calculate the enhancement layer codeword c l :

[0179]

[0180] S172. Calculate the quantization interval of the enhancement layer codeword [a 0 ,b 0 ],in

[0181]

[0182]

[0183] S173, Based on Gaussian probability density function, the enhancement layer codeword c l Perform probability estimation

[0184]

[0185] in

[0186]

[0187]

[0188] S174, According to probability p(c l Arithmetic encoding is performed to obtain the binary code stream of the l-th enhancement layer.

[0189] This part iteratively executes the encoding of the enhancement layers. The S171-S174 method is used to obtain the aligned codewords of each enhancement layer, and the quantization range of the aligned codewords is calculated. Entropy encoding is performed based on the conditional probability of the codewords already encoded in the lower quality layers, thereby ensuring that the bitstream can be arbitrarily truncated and removing redundancy in the encoding of each layer.

[0190] The progressive image coding method based on autoregressive entropy modeling in the above embodiments of the present invention can be applied to existing deep learning image compression models. It supports context-based autoregressive entropy modeling, and can quickly achieve quantization of different quality layers by changing the quantization interval through the scale factor without retraining. It ensures a one-to-one mapping between each quality layer under different scale factors, does not introduce redundancy, and further reduces the coding bitrate and improves rate-distortion performance compared with existing methods.

[0191] Based on the same inventive concept, another embodiment of the present invention provides a progressive image decoding method based on autoregressive entropy modeling, which can be used to decode the results obtained by the aforementioned progressive image encoding method based on autoregressive entropy modeling. The flowchart is as follows. Figure 3 As shown, the process includes: decomposing the obtained binary bitstream into a super-prior bitstream and multiple quality layers of bitstream, then using an autoregressive entropy model to decode each layer of bitstream to obtain a multi-layer reconstructed quantized representation, and obtaining multiple reconstructed images through a decoding neural network.

[0192] Specifically, in one possible implementation, autoregressive entropy modeling of progressive image decoding includes the following steps:

[0193] S210. Decompose the obtained bitstream into a priori binary bitstream, a basic layer binary bitstream, and L enhancement layer binary bitstreams.

[0194] S220. Obtain the prior information by reconstructing the prior binary code stream through a priori decoding neural network;

[0195] S221. Extract contextual information based on the quantized spatial grouping of the current channel and the decoded quantized features of the current spatial grouping, and fuse it with the prior information to obtain the predicted value μ of the current feature y in the lowest quality enhancement layer. 0 ;

[0196] S222. Decode the basic layer binary codestream by combining the probability estimates from the basic layer dead zone and other quantization intervals to obtain the basic layer codeword c. 0 ,

[0197] S223, Based on the reconstruction function Obtain the quantization features of the base layer

[0198]

[0199] S230. Following the order of quality layers from low to high, starting from the first enhancement layer, decode the codewords and obtain quantization features for reconstruction in sequence. The l-th (l = 1, ..., L) enhancement layer is obtained by decoding the binary codestream of the l-th enhancement layer. The specific steps are as follows:

[0200] S231 is based on the quantized representation decoded from the (l-1)th enhancement layer. Predicted Gaussian mean μ l-1 and the quantization interval length q l-1 Obtain the quantization features of the (l-1)th layer within the quantization interval length q l-1 Uniform quantization representation of η l :

[0201]

[0202] When l=1, the first enhancement layer is obtained based on the quantization characteristics, Gaussian mean, and quantization interval length of the base layer, i.e., the superscript... 0 Refers to the basic layer.

[0203] S232 based on the obtained η l The predicted Gaussian mean μ l and the quantization interval length q l-1 Obtain the quantization feature of the l-th enhancement layer within the quantization interval length q l-1 Uniform quantization representation below:

[0204]

[0205] S233. Obtain the quantization feature of the l-th enhancement layer in a dead zone of size p. l-1 Other quantization interval sizes q l-1 Dead zone quantization representation:

[0206]

[0207] S234 uses the predicted Gaussian distribution parameters of the l-th enhancement layer, combined with the conditional probability estimated from the dead zone of the l-th enhancement layer and other quantization intervals, to decode the l-th enhancement layer bitstream and obtain the l-th layer codeword c. l ;

[0208] S240, Based on the obtained l-th enhancement layer codeword c l The obtained Predicted Gaussian mean μ l And the size of the dead zone p l Other quantization interval sizes q l The representation of the quantization feature of the l-th enhancement layer under dead zone quantization is obtained:

[0209]

[0210] S250. Input the quantized features obtained from decoding into the decoding neural network to obtain multiple reconstructed images of varying quality from low to high.

[0211] The compression method in the above embodiments of the present invention is based on Figure 1 , 2A progressive image coding method based on autoregressive entropy modeling and Figure 3 The method of progressive image decoding based on autoregressive entropy modeling is used to achieve this. For parts not described in detail above, please refer to the corresponding descriptions in the above embodiments, which will not be repeated here.

[0212] The compression method in the above embodiments of the present invention can improve the accuracy of bitrate estimation and improve the rate-distortion performance of image compression on natural images of different resolutions collected in different scenarios. It can flexibly estimate the feature map distribution according to different image content and different computational complexity constraints to achieve bitrate estimation and improve rate-distortion performance.

[0213] Based on the same inventive concept, another embodiment of the present invention provides an image compression method for compressing 24 images with a resolution of 512×768 from the Kodak test set. The flowchart of this method is as follows. Figure 4 As shown, it includes:

[0214] S310, Acquisition Module: Acquires the image to be compressed;

[0215] S320, Encoder: Encode the image to be compressed according to steps S110-S180 to obtain the compressed image binary code stream.

[0216] For example, the parameters are as follows:

[0217] Each input image is processed by an encoding neural network composed of convolutional neural networks to obtain a multi-channel feature map with a size of 192*32*48. The multi-channel feature map is divided into two groups, each with a size of 384*16*24. Then, for each spatial group, progressive encoding is performed using the autoregressive entropy modeling encoding method proposed in this paper, which can achieve a quality layer of 4 layers. The dead zone size and other quantization steps are {1, 1}, {3, 2}, {7, 4}, and {15, 8} from the highest quality enhancement layer to the base layer. Each spatial group has 4 quantization representations, corresponding to 4 target bitrate points, with a size of 384*16*24. The spatial groups are merged to obtain 4 quantized features with a size of 192*32*48. Finally, a decoding neural network composed of convolutional neural networks is used to obtain 4 reconstructed images with quality from coarse to fine.

[0218] S330, Decoder: Decompress the compressed image binary bitstream according to S210-S250 to obtain the reconstructed image.

[0219] The parameters of the decoder are exactly the same as those of the encoder.

[0220] Based on the above technical solution, in a specific embodiment, the Kodak dataset is compressed using an image encoder in a progressive image compression system that incorporates the progressive coding method proposed in the above embodiments. The Kodak dataset contains 24 images with a resolution of 512*768 or 768*512. Dead-zone quantization is used, and the progressive coding layers are set to 4. The lengths of the 4-layer quantization dead zone and other quantization intervals are [1,1], [3,2], [7,4], and [15,8]. This generates a priori bitstream and a feature map bitstream. The feature map bitstream can be further decomposed into a base layer and three enhancement layer bitstreams. The average lengths of the four-layer bitstream are 0.1322, 0.3383, 0.7614, and 1.4167 bpp, respectively.

[0221] Based on the above technical solution, in a specific embodiment, the image decoder in the progressive image compression system proposed in the above embodiments is used to decompress the binary bitstream generated by the Kodak dataset to obtain the reconstructed image. The same quantization step size configuration as in the encoder is adopted to obtain the decoded image, wherein the average peak signal-to-noise ratio (PSNR) of all images on the Kodak dataset is 28.0838, 31.7904, 36.088, and 39.4969 from low to high in each layer. Comparing the experimental results of the proposed method with the current progressive coding compression standard HEVC, an average bitrate saving of 8.05% is achieved while maintaining the same reconstruction quality.

[0222] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0223] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0224] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0225] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0226] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A progressive image coding method based on autoregressive entropy modeling, characterized in that, include: Obtain multi-channel feature maps of the image to be encoded; Based on the multi-channel feature map, the prior features, the prior binary code stream, and the prior information are obtained. The multi-channel feature map is divided into K spatial groups according to each channel, where K is a positive integer; For each feature of each spatial group of the multi-channel feature map, prediction is made based on prior information and the quantized features of that channel. Dead zone quantization is performed given the dead zone and other quantization interval sizes to obtain the quantized features of the highest quality enhancement layer. The quantization interval of dead-zone quantization is scaled sequentially using a fixed scaling factor. Initial codewords and initial quantization features are obtained through dead-zone quantization. The codewords are aligned to obtain the quantization features used as the quantization features of the current quality layer. This process is repeated to obtain L quantization features for lower quality layers, where L is a positive integer. The lowest quality layer with the largest quantization interval is the base layer, and the rest... The quality layers are ordered from largest to smallest quantization interval as number 1 to number 2. One enhancement layer; The codewords of the basic layer are calculated based on the quantization characteristics of the basic layer. The probability is estimated based on the quantization interval. The codewords of the basic layer are then encoded to obtain the binary code stream of the basic layer. Following the order of the first to the Lth enhancement layers, codewords are calculated layer by layer based on the quantization features of the corresponding enhancement layer, the estimated probability is calculated, and the codewords are encoded to obtain L binary code streams of enhancement layers; The above-mentioned super-prior binary code stream, basic layer binary code stream, and L enhancement layer binary code streams are merged to form a binary code stream; The quantization interval of dead-zone quantization is scaled sequentially using a fixed scale factor. Initial codewords and initial quantization features are obtained through dead-zone quantization. The codewords are aligned to obtain quantization features as the quantization features of the current quality layer. This process is repeated to obtain L quantization features for lower quality layers, including: Following the order of quality from high to low, starting from the Lth quality layer, codewords and quantization features are obtained sequentially, where for the Lth quality layer... One quality layer, The specific steps are as follows: According to the Prediction of individual quality layers Based on the size of the dead zone Other quantization interval sizes Dead-zone quantization is performed using a codeword mapping function. Obtain the initial codeword : ； For the initial codeword According to the reconstruction function Obtain initial quantization features : ； Contextual information is extracted based on the quantized spatial groupings of the current channel and the quantized features of the current spatial groupings. This information is then fused with prior information to obtain the current features. In the Predicted values ​​of each enhancement layer ; Initial quantization features Transform to quantization range The uniform quantization, according to the first Prediction of individual quality layers Obtain uniform quantization features : ； Among them, the codeword mapping function of uniform quantization , This is the forward conversion function for dead-zone quantization and uniform quantization. Align codewords to obtain the first Quantitative characteristics of each quality layer : ； in This is the inverse conversion function for dead-zone quantization and uniform quantization.

2. The progressive image coding method based on autoregressive entropy modeling according to claim 1, characterized in that, For each feature of each spatial group of the multi-channel feature map, prediction is performed based on prior information and the quantized features of that channel. Dead-zone quantization is then performed given the dead zone and other quantization interval sizes to obtain the codewords and quantized features of the highest quality enhancement layer, including: Contextual information is extracted based on the quantized spatial groupings of the current channel and the quantized features of the current spatial groupings. This information is then fused with prior information to obtain the current features. Predicted values ​​at the highest quality enhancement layer ; Calculate current features Predicted residuals in the highest quality enhancement layer ; For prediction residuals Perform dead-zone quantization based on the codeword mapping function. To obtain the initial codewords for the highest quality enhancement layer. , specifically: ； Here, the dead zone size is quantized as a dead zone. Other quantization interval sizes , Return the input symbol. Return the larger of the two inputs; For the initial codeword According to the reconstruction function Obtain the quantization characteristics of the highest quality enhancement layer : ； Here, the reconstruction function is: ； in, Returns the absolute value of the input.

3. The progressive image coding method based on autoregressive entropy modeling according to claim 1, characterized in that, Using forward transformation function Convert the quantization features of dead-zone quantization into the corresponding quantization features of uniform quantization: ； Using reverse conversion function Reconstruct the quantization features of uniformly quantized codewords into those of dead-zone quantization: 。 4. The progressive image coding method based on autoregressive entropy modeling according to claim 1, characterized in that, The step of calculating the codewords of the basic layer based on the quantization characteristics of the basic layer, estimating the probability based on the quantization interval, and encoding the codewords of the basic layer to obtain the basic layer binary code stream includes: Based on the quantization characteristics of the basic layer Calculate the basic layer codeword : ； Calculate the quantization range of the basic layer codeword ,in ； Based on Gaussian probability density function for basic layer codewords Perform probability estimation ； in ; According to probability Arithmetic encoding is performed to obtain the basic layer binary code stream.

5. The progressive image coding method based on autoregressive entropy modeling according to claim 1, characterized in that, The process of calculating codewords layer by layer according to the order of enhancement layers 1 to L, calculating estimated probabilities, and encoding codewords to obtain the binary code stream of the enhancement layers includes: For any i The enhancement layer, according to the first Quantization characteristics of the layer Calculate the enhancement layer codeword : ； Calculate the quantization range of the enhancement layer codeword ,in ； Enhancement layer codewords based on Gaussian probability density function Perform probability estimation ； in ； According to probability Perform arithmetic encoding to obtain the first... An enhancement layer binary code stream.

6. The progressive image coding method based on autoregressive entropy modeling according to claim 1, characterized in that, Any number The and the first The sizes of other quantization intervals for each quality layer are scaled by a fixed scaling factor S: ； No. Dead zone size of each mass layer And other quantization interval sizes They are respectively: 。 7. A progressive image decoding method based on autoregressive entropy modeling, used to decode the binary bitstream obtained according to any one of claims 1-6, characterized in that, include: The obtained bitstream is decomposed into a priori binary bitstream, a basic layer binary bitstream, and L enhancement layer binary bitstreams; The prior information is obtained by reconstructing the prior binary code stream through a priori decoding neural network. Contextual information is extracted from the quantized spatial groupings of the current channel and the decoded quantized features of the current spatial groupings, and then fused with prior information to obtain the current features. Predicted values ​​at the base layer The basic layer codewords are obtained by decoding the basic layer binary codestream by combining the probability estimates of the basic layer dead zone and other quantization intervals. According to the reconstruction function Obtain the quantization features of the base layer : ； Following the order of quality layers from low to high, starting with the first enhancement layer, the codewords are decoded and quantized features are reconstructed sequentially. For the [missing information - likely a specific layer or layer], the codewords are decoded and quantized features are reconstructed sequentially. An enhancement layer, From the first The enhancement layer binary code stream is obtained by decoding the following steps: According to the Quantization representation of each enhancement layer decoded Predicted Gaussian mean and the length of the quantization interval , obtained the The quantization features of each enhancement layer are within the quantization interval length. Uniform quantization representation under : ； when At that time, the first enhancement layer is obtained based on the quantization characteristics, Gaussian mean, and quantization interval length of the base layer, i.e., the superscript. 0 Refers to the basic layer; According to the obtained Predicted Gaussian mean and the length of the quantization interval , obtained the The quantization features of each enhancement layer are within the quantization interval length. Uniform quantization representation below: ； Obtain the Each enhancement layer quantization feature has a dead zone size of [value missing]. Other quantization interval sizes Dead zone quantization representation: ； Using the predicted first The Gaussian distribution parameters of the first enhancement layer, combined with the first... The conditional probability of estimating the dead zone and other quantization intervals of the enhancement layer is related to the conditional probability of the first layer dead zone. Decode the enhancement layer bitstream to obtain the first enhancement layer bitstream. one enhancement layer codeword ; According to the obtained number one enhancement layer codeword The obtained Predicted Gaussian mean and the size of the dead zone Other quantization interval sizes , obtained the Representation of enhancement layer quantization features under dead-zone quantization: ； The quantized features obtained from decoding are input into a decoding neural network to obtain multiple reconstructed images of varying quality from low to high.

8. A progressive image compression method based on autoregressive entropy modeling, characterized in that, include: Obtain the image to be compressed; A compressed image binary bitstream is obtained based on the image encoding method according to any one of claims 1-6; The image decoding method according to claim 7 is used to decompress the compressed image binary code stream to obtain a reconstructed image.

9. A progressive image compression system, comprising an encoder and a decoder, characterized in that, include: Encoder: The encoder uses the image encoding method according to any one of claims 1-6 to encode the image to be compressed to obtain a compressed image binary bitstream; Decoder: The decoder decodes the compressed image binary code stream based on the image decoding method of claim 7 to obtain the reconstructed image.

10. A progressive image compression apparatus, the image compression apparatus comprising a memory and a processor, the memory storing a program executable by the processor, characterized in that, The processor may execute any of the following methods when executing the program: The image encoding method according to any one of claims 1-6, or, The image decoding method according to claim 7, or, The image compression method according to claim 8.

Images