Variable bit rate image compression method, system, apparatus, terminal and storage medium

Abstract

The application provides a variable code rate image compression method, system, device, terminal and storage medium, comprising: obtaining an initial feature map of a to-be-encoded image through a first decomposition transform neural network; quantizing the initial feature map through a dead zone quantizer to obtain a quantized feature map; performing entropy coding on the quantized feature map and super-prior information by using an entropy model to obtain a compressed code stream; performing entropy decoding on the compressed code stream, and recovering the quantized super-prior information and the quantized feature map by using the entropy model; performing inverse quantization on the quantized feature map to obtain a reconstructed feature map; obtaining a reconstructed image through a first synthesis transform neural network; and adjusting parameters of quantization and inverse quantization in the encoding process according to a target code rate or a target distortion, so that the code rate of the compressed code stream is close to the target code rate, or the distortion of the reconstructed image is close to the target distortion. The application provides an accurate code rate control scheme, so that the code rate of the compressed code stream is better adapted to dynamic network bandwidth changes, and has extremely strong practical application value.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to an end-to-end optimized variable bitrate image compression scheme with bitrate control function. Specifically, it is a variable bitrate image compression method, system, device, terminal and storage medium. Background Technology

[0002] End-to-end optimization is a widely used technique in image compression. It utilizes convolutional neural networks to model compression as a rate-distortion optimization problem, enabling joint optimization of various modules. Variable bitrate schemes can output different quality and bitrate image or video sequences from an input image or video sequence using a single model. A literature search of existing technologies revealed that Dumas et al., in their 2018 presentation at the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), titled "Autoencoder Based Image Compression: Can the Learning be Quantization Independent?", proposed an optimization scheme based on a uniform quantizer by simultaneously optimizing the transform network, quantization step size, and entropy model parameters. This method achieves variable bitrate image encoding by adjusting the quantization step size with a fixed transform. Choi et al., in their 2019 presentation at the IEEE / CVF International Conference on Computer Vision (ICCV), proposed a conditional autoencoder, "Variable Rate Deep Image Compression With a Conditional Autoencoder," which achieves adaptive transformation for different target bit rates. This method employs a uniform quantizer with a variable quantization step size, realizing both adaptive transformation and variable bit rate encoding with a variable quantization step size. Zhou et al., in their 2020 presentation at the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), proposed a variable bit rate encoding method using a dead-zone quantizer. However, in this method, the transformation network and quantization step size are not optimized simultaneously; instead, they are implemented in stages by first training the transformation network and then manually adjusting the quantization step size.

[0003] Furthermore, these methods lack explicit modeling of the relationship between the quantization step size and the bitrate of the compressed bitstream or the distortion of the reconstructed image, making it impossible to calculate the quantization step size from the target bitrate or target distortion. Therefore, given a target bitrate or target distortion, these methods require a binary search at existing preset bitrate points to obtain a suitable quantization step size. Additionally, the staged implementation of the dead-zone quantizer does not fully utilize the advantages of joint transform and quantization optimization in end-to-end optimization, and rate-distortion performance needs improvement. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a variable bitrate image compression method, system, apparatus, and storage medium.

[0005] According to one aspect of the present invention, a variable bitrate image compression method is provided, comprising:

[0006] Forward mapping: The image to be encoded is passed through the first decomposition transformation neural network to obtain the initial feature map of the image;

[0007] Quantization: The initial feature map is quantized using a dead-zone quantizer to obtain a quantized feature map of the image;

[0008] Entropy coding: The quantized feature map and the quantized prior information involved in the entropy model are entropy encoded using the entropy model to obtain a compressed bitstream;

[0009] Entropy decoding: The compressed bitstream is entropy decoded, and the quantized prior information and quantized feature map are recovered sequentially using the entropy model;

[0010] Inverse quantization: The recovered quantized feature map is inverse quantized to obtain the reconstructed feature map of the image;

[0011] Reverse mapping: The reconstructed feature map is passed through a first synthetic transform neural network to obtain a reconstructed image;

[0012] Bitrate control: Based on the target bitrate or target distortion, the parameters of quantization and dequantization in the encoding process are adjusted so that the bitrate of the compressed bitstream is close to the target bitrate, or the distortion of the reconstructed image is close to the target distortion.

[0013] Preferably, the first decomposition transformation neural network includes:

[0014] A downsampling subnetwork, implemented by a convolutional neural network, transforms the input image into a hidden representation;

[0015] A reversible coding subnetwork transforms the hidden representation into an initial feature map; the reversible coding subnetwork is obtained by arranging reversible coding units hierarchically, the hierarchical arrangement containing I layers, and the i-th layer (1≤i≤I) containing 2... i-1 There are 1 reversible coding unit, each with two outputs. The two outputs of the i-th layer reversible coding unit serve as the inputs of two reversible coding units in the (i+1)-th layer. For the unique reversible coding unit in the 1st layer, its input is an implicit representation; while for the 2-input reversible coding unit provided by the i-th layer... I The outputs can be merged to obtain the initial feature map.

[0016] Preferably, the process performed in the reversible encoding unit includes:

[0017] Eigenvalue decomposition: Dividing the input into two sub-signals;

[0018] Reversible encoding: For two sub-signals x1 and x2, the reversible encoding process includes:

[0019]

[0020]

[0021] in, and All of them are convolutional neural networks.

[0022] Preferably, the quantization involves adjusting the quantization step size of the dead-zone quantizer to output a quantized feature map that meets the target bit rate or target distortion requirements; for any element y at any position in the initial feature map, the quantization output is...

[0023]

[0024] When y is positive, sign(y) is 1; when y is negative, sign(y) is -1; when y is zero, sign(y) is 0; max(a, b) returns the larger of a and b. Returns the largest integer not greater than y; q is the quantization step size, and z is the dead zone ratio.

[0025] Preferably, the entropy encoding includes: estimating the Gaussian distribution of elements on the quantized feature map using an entropy model, performing arithmetic encoding on the quantized feature map, and performing arithmetic encoding on the quantized prior information using a constructed distribution to obtain a compressed bitstream.

[0026] Preferably, the entropy decoding includes: performing arithmetic decoding on the quantized prior information using a constructed distribution, inputting it into an entropy model to obtain the codeword distribution of the quantized feature map, performing arithmetic decoding on the compressed bitstream, and recovering the quantized feature map.

[0027] Preferably, the entropy model includes:

[0028] Advanced prior information extraction module: The initial feature map of the image is passed through a second decomposition and transformation network to obtain advanced prior information;

[0029] Super-prior quantization module: For the super-prior information, quantized super-prior information is generated through uniform quantization;

[0030] Advanced prior information reconstruction module: The quantized advanced prior information is processed through a second synthetic transform neural network to obtain reconstructed advanced prior information;

[0031] Context modeling module: Enables the encoded and decoded quantized feature maps to obtain context information through a context extraction neural network;

[0032] Distribution estimation module: merges the context information with the reconstructed prior information, and through the information fusion neural network, outputs two outputs with the same size as the initial feature map, which are respectively used as the mean and standard deviation of the Gaussian distribution, thereby obtaining the distribution of each element on the quantized feature map;

[0033] The entropy coding is performed based on the prior information extraction module, the prior information quantization module, the prior information reconstruction module, the reconstructed prior information, and the distribution estimation module.

[0034] The entropy decoding is performed based on the prior information reconstruction module, the context modeling module, and the distribution estimation module.

[0035] Preferably, the inverse quantization is the reverse process of the quantization, using the same quantization step size and dead zone ratio as the quantization, for any element at any position in the recovered quantized feature map. The dequantization output is

[0036]

[0037] Where q is the quantization step size and z is the dead zone ratio.

[0038] Preferably, the first synthetic transform neural network includes:

[0039] Reversible decoding subnetwork: The reversible decoding subnetwork transforms the reconstructed feature map into a reconstructed hidden representation; the reversible decoding subnetwork is obtained by arranging reversible decoding units hierarchically, the hierarchical arrangement contains I layers, and the i-th layer (1≤i≤I) contains 2I-i+2 There are 2 reversible decoding units; each reversible decoding unit has two inputs and one output, and the outputs of the two reversible decoding units in the i-th layer are used as the inputs of one reversible decoding unit in the (i+1)-th layer; by dividing the reconstructed feature map into 2 I The components yield 2I-1 reversible decoding units of the first layer. I There are 1 input; and the output of the only reversible decoding unit in the I-th layer is the reconstructed implicit representation.

[0040] Upsampling subnetwork: Implemented by a convolutional neural network, it transforms the reconstructed latent representation into a reconstructed image;

[0041] Preferably, the process performed in the reversible decoding unit includes:

[0042] Reversible decoding: for two input signals and The reversible decoding process includes:

[0043]

[0044]

[0045] in, and Both are convolutional neural networks, where θ and μ are the trainable tests of the two convolutional neural networks, respectively;

[0046] Feature decomposition: merging and The output signal is obtained.

[0047] Preferably, the parameters of the first decomposition transform neural network, the first synthesis transform neural network, and the entropy model are obtained through iterative learning based on a training image set, including:

[0048] Obtain the training image set: Use the training image set as the images to be encoded;

[0049] Forward mapping: The image to be encoded is passed through the first decomposition transformation neural network to obtain the initial feature map of the image;

[0050] Quantization proxy: Quantizes the feature map using a dead-zone quantizer to obtain a noisy feature map of the image. For any element y at any position in the initial feature map, the output of the quantization proxy is...

[0051]

[0052] Where n is the range of values ​​in The random uniform noise, and β(y; q, z) satisfies

[0053]

[0054] Where q is the quantization step size and z is the dead zone ratio;

[0055] Entropy estimation: Input the noisy feature map into the entropy model, and output the Gaussian distribution of the elements at each position on the noisy feature map. Then, the elements at any position... The sign probability is

[0056]

[0057] Where μ and σ are the mean and variance of the predicted Gaussian distribution, respectively; furthermore, let any element in the noisy prior information in the entropy model be denoted as . The symbol probability calculated by constructing the distribution is: The empirical entropy of the encoded noisy feature map and the noisy prior information is

[0058] Dequantization: Dequantize the noisy feature map to obtain a reconstructed feature map of the image. For any element at any position in the noisy feature map... The dequantization output is

[0059]

[0060] Where q is the quantization step size and z is the dead zone ratio;

[0061] Reverse mapping: The reconstructed feature map is passed through the first synthetic transform neural network to obtain the reconstructed image;

[0062] Distortion estimation: Let the image to be encoded be X, with size H×W, and the reconstructed image be... Reconstruction distortion

[0063] Gradient update: By pre-fixing the dead zone ratio and providing N different quantization step sizes, N sets of empirical entropy and reconstruction distortion can be obtained, denoted as […]. and N Lagrange multipliers λ1, λ2, ..., λ are predetermined for it. N During training, a quantization step size is randomly selected each time, resulting in the loss function for that iteration. The neural network parameters are then updated via backpropagation.

[0064] Preferably, the bitrate control includes:

[0065] Bitrate calculation: Input the dimensions H×W of the image to be encoded and the number of bits s in the compressed bitstream. The bitrate of the compressed bitstream is calculated as follows:

[0066] Distortion calculation: Let the image to be encoded be X, with dimensions H×W, and the reconstructed image be... Reconstruction distortion

[0067] Quantization step size-bitrate modeling: By performing quantization, entropy coding, and bitrate calculation three times with different quantization step sizes on the initial feature map, three sets of bitrates for quantization step sizes and compressed bitstreams are obtained, and a quantization step size-bitrate model is fitted.

[0068]

[0069] Where q is the quantization step size, R is the code rate, and α, β and γ are three parameters to be fitted.

[0070] Quantization step size-distortion modeling: By performing multiple quantization, dequantization, inverse mapping, and distortion calculations on the initial feature map with different quantization step sizes, multiple sets of distortion values ​​for the reconstructed image are obtained, and a quantization step size-distortion model is fitted.

[0071]

[0072] Where q is the quantization step size, D is the bit rate, and ζ, η and ι are three parameters to be fitted;

[0073] Quantization step size decision: given target bit rate Or target distortion The corresponding quantization step size is inferred through the quantization step size-bit rate model or the quantization step size-distortion model. or

[0074] According to a second aspect of the present invention, a variable bitrate image compression system is provided, comprising:

[0075] The forward mapping module passes the image to be encoded through a first decomposition transform neural network to obtain the initial feature map of the image;

[0076] The quantization module quantizes the feature map using a dead-zone quantizer to obtain a quantized feature map of the image.

[0077] An entropy coding module uses an entropy model to entropy code the quantized feature map and the quantized prior information involved in the entropy model to obtain a compressed bitstream.

[0078] The entropy decoding module performs entropy decoding on the compressed bitstream and uses the entropy model to sequentially recover the quantized prior information and quantized feature map.

[0079] The dequantization module dequantizes the quantized feature map to obtain the reconstructed feature map of the image.

[0080] The reverse mapping module reconstructs the feature map through a first synthetic transform neural network to obtain a reconstructed image;

[0081] The bitrate control module adjusts the quantization and dequantization parameters during the encoding process according to the target bitrate or target distortion, so that the bitrate of the compressed bitstream is close to the target bitrate, or the distortion of the reconstructed image is close to the target distortion.

[0082] According to a third aspect of the present invention, an image compression apparatus is provided, which performs image compression using a variable bit rate image compression method as described in any one of the claims.

[0083] According to a fourth aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to perform any of the methods described herein, or to run the system, or to run the apparatus.

[0084] According to a fifth aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods described herein, or to run the system, or to run the apparatus.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] The variable bitrate image encoding method and system in this invention can approximate the orthogonal transformation in local space through forward and reverse mapping. Quantization and dequantization based on dead zone quantizer can improve rate distortion performance in variable bitrate scenarios, while bitrate control can provide an accurate bitrate control scheme, enabling the compressed bitstream bitrate to better adapt to dynamic network bandwidth changes, which has great practical application value.

[0087] The variable bitrate image encoding method and system in this embodiment of the invention have a novel arrangement of the first, decomposition-synthesis-transformation neural network and the reversible encoding unit, which is a hierarchical arrangement and each reversible encoding unit has a feature decomposition module / process.

[0088] The variable bitrate image encoding method and system in this embodiment of the invention, in terms of bitrate control, the neural network implemented by the technical solution of this embodiment can approximate the orthogonal transformation on the local space, thereby improving the average performance of variable bitrate encoding; and the required number of parameters is less, which can effectively reduce the storage overhead of the model required to implement bitrate control. Attached Figure Description

[0089] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0090] Figure 1 This is a flowchart of a variable bitrate image compression method according to an embodiment of the present invention;

[0091] Figure 2 This is a flowchart of a variable bitrate image compression system according to another embodiment of the present invention;

[0092] Figure 3 This is a flowchart of the entropy encoding module and entropy decoding module in a preferred embodiment of the present invention;

[0093] Figure 4 This is a flowchart of the bitrate control module in a preferred embodiment of the present invention. Detailed Implementation

[0094] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0095] This invention provides an embodiment, see [link to embodiment]. Figure 1 A variable bitrate image compression method, comprising:

[0096] S100, Forward Mapping: The image to be encoded is passed through the first decomposition transformation neural network to obtain the initial feature map of the image;

[0097] S200, quantization: The initial feature map in S100 is quantized by a dead-zone quantizer to obtain the quantized feature map of the image;

[0098] S300, Entropy Coding: Entropy coding is performed on the quantized feature map in S200 and the quantized prior information involved in the entropy model to obtain a compressed bitstream;

[0099] S400, Entropy Decoding: Entropy decoding is performed on the compressed bitstream of S300, and the quantized prior information and quantized feature map of S300 are recovered sequentially using the entropy model.

[0100] S500, Inverse Quantization: Inverse quantization is performed on the quantized feature map of S200 to obtain the reconstructed feature map of the image;

[0101] S600, Reverse Mapping: The reconstructed feature map in S500 is passed through the first synthetic transform neural network to obtain the reconstructed image;

[0102] S700, Rate Control: Adjusts the quantization and dequantization parameters during the encoding process according to the target bit rate or target distortion, so that the bit rate of the compressed bit stream of S300 is close to the target bit rate, or the distortion of the reconstructed image of S600 is close to the target distortion.

[0103] This embodiment provides an accurate bitrate control scheme, which enables the compressed bitrate to better adapt to dynamic network bandwidth changes, and has great practical application value.

[0104] In an embodiment of the present invention, a rate control scheme with a given target bitrate is implemented on the Kodak dataset.

[0105] In a preferred embodiment, S100 is implemented, and the specific implementation steps are as follows:

[0106] Forward mapping: The image to be encoded, with a size of H×W, is passed through the first decomposition transform neural network to obtain the initial feature map of the image. The first decomposition transform neural network consists of a downsampling sub-network and a reversible encoding sub-network connected sequentially. The downsampling sub-network consists of four convolutional layers and a GDN (Generalized Divisive Normalization) layer. Each convolutional layer has a kernel size of 5×5, a stride of 2, and output channels of 192, 192, 192, and 320 respectively. There is a GDN layer between every two convolutional layers. The image to be encoded, with a size of H×W, becomes an initial feature map after passing through the downsampling sub-network. The implicit representation. The reversible coding subnetwork contains three levels of reversible coding modules, containing 1, 2, and 4 reversible coding units respectively. The i-th (0≤i≤2) unit of the j-th layer (0≤j≤2) is used. j -1) The reversible coding unit is denoted as Its input is denoted as The output contains two elements: and That is, the inputs of the 2i-th and 2i+1-th reversible coding units in the j-th layer. For each reversible coding unit, it first... Dividing along the channel dimension yields two entities of size [missing information]. implicit representation and Next, the calculation process for the output is as follows:

[0107]

[0108]

[0109] in and All are composed of convolutional neural networks with the same structure, including three convolutional layers: each convolutional layer has an output channel count of 1. The convolutional kernel size is 3, the stride is 1, and the first two convolutional layers are followed by ReLU nonlinear units. After passing through three levels of reversible encoding modules, eight kernels of size 3 are obtained. The implicit representation of can be obtained by concatenating it along the channel dimension to obtain a size of . The initial feature map.

[0110] The forward mapping in this embodiment can approximate the orthogonal transformation in the local space, thereby improving the average performance of variable code rate coding.

[0111] In a preferred embodiment of the present invention, S200 is implemented, using a quantization step size of Furthermore, a dead-zone quantizer with a dead-zone ratio of 1 quantizes the initial feature map to obtain a quantized feature map. Specifically:

[0112] By adjusting the quantization step size of the dead-zone quantizer, the output quantized feature map satisfies the target bit rate or target distortion requirements. For an element y at any position in the initial feature map, the quantization output is...

[0113]

[0114] When y is positive, sign(y) is 1; when y is negative, sign(y) is -1; when y is zero, sign(y) is 0; max(a, b) returns the larger of a and b. Returns the largest integer not greater than y; q is the quantization step size, and z is the dead zone ratio.

[0115] This embodiment implements quantization to adjust the quantization step size in variable bit rate coding scenarios, and achieves the purpose of controlling the bit rate by adjusting the quantization step size, thereby improving the rate-distortion performance in variable bit rate scenarios.

[0116] In a preferred embodiment of the present invention, S300 is performed for entropy coding. An entropy model is used to estimate the distribution of elements on the quantized feature map. Arithmetic coding is then performed on the quantized feature map. A constructed distribution is used to perform arithmetic coding on the quantized prior information to obtain a compressed bitstream. Specifically, this includes the following six steps:

[0117] S31, Prior Information Extraction: The initial feature map of the image is passed through a second decomposition transformation network to obtain prior information. This second decomposition transformation network consists of three convolutional layers and a LeakyReLU layer. The first convolutional layer has a 3×3 kernel size, a stride of 1, and 192 output channels. The latter two convolutional layers have a 5×5 kernel size, a stride of 2, and 192 output channels each. A LeakyReLU layer is located between each pair of the three convolutional layers.

[0118] S32, Super-prior quantization: For super-prior information, quantized super-prior information is generated through uniform quantization;

[0119] S33, Reconstruction of Prior Information: The quantized prior information is passed through a second synthesis transform neural network to obtain reconstructed prior information. The second decomposition and synthesis network consists of three deconvolutional layers and a LeakyReLU layer. The first two deconvolutional layers have a kernel size of 5×5, a stride of 2, and output channels of 320 and 480 respectively. The last deconvolutional layer has a kernel size of 3×3, a stride of 1, and output channels of 640. A LeakyReLU layer is located between every two of the three deconvolutional layers.

[0120] S34, Context Modeling: Enables the encoded quantized feature map to obtain context information through a context extraction neural network;

[0121] S35, Distribution Estimation: Contextual information is merged with the reconstructed prior information, and then passed through an information fusion neural network. Two output terms with the same size as the initial feature map are used as the mean and standard deviation of a Gaussian distribution, respectively, thus obtaining the distribution of each element on the quantized feature map. The information fusion neural network consists of three convolutional layers and a LeakyReLU layer. Each convolutional layer has a 1×1 kernel size, a stride of 1, and output channels of 1280, 1067, and 640, respectively.

[0122] S36, Arithmetic Coding: Arithmetic coding is performed on the quantized feature map by estimating the distribution of elements on the quantized feature map, and arithmetic coding is performed on the quantized prior information by constructing the distribution, resulting in a compressed bitstream.

[0123] In a preferred embodiment of the present invention, step S400 is implemented to perform entropy decoding. An arithmetic decoding of the quantized prior information is performed using a constructed distribution, and this distribution is input into the entropy model to obtain the codeword distribution of the quantized feature map. Arithmetic decoding is then performed on the compressed bitstream to recover the quantized feature map. Specifically, the steps include:

[0124] S40, Super-prior arithmetic decoding: The quantized super-prior information is arithmetically decoded using a constructed distribution to obtain the quantized super-prior information;

[0125] S41, Reconstruction of Prior Information: The quantized prior information is passed through the second synthetic transform neural network to obtain the reconstructed prior information;

[0126] S42, Context Modeling: Enables the encoded quantized feature map to obtain context information through a context extraction neural network;

[0127] S43, Distribution Estimation: The contextual information is combined with the reconstructed prior information, and two output terms with the same size as the initial feature map are output through the information fusion neural network. These two output terms are used as the mean and standard deviation of the Gaussian distribution, respectively, thus obtaining the distribution of each element on the quantized feature map.

[0128] S44, Arithmetic Decoding: Arithmetic decoding of the quantized feature map is performed using the estimated distribution of elements on the quantized feature map.

[0129] In a preferred embodiment of the present invention, an entropy model is described. Specifically, the entropy model includes:

[0130] Advanced prior information extraction module: The initial feature map of the image is passed through a second decomposition and transformation network to obtain advanced prior information;

[0131] Super-prior quantization module: For the super-prior information, quantized super-prior information is generated through uniform quantization;

[0132] Advanced prior information reconstruction module: The quantized advanced prior information is processed through a second synthetic transform neural network to obtain reconstructed advanced prior information;

[0133] Context modeling module: Enables the encoded and decoded quantized feature maps to obtain context information through a context extraction neural network;

[0134] Distribution estimation module: merges the context information with the reconstructed prior information, and through the information fusion neural network, outputs two outputs with the same size as the initial feature map, which are respectively used as the mean and standard deviation of the Gaussian distribution, thereby obtaining the distribution of each element on the quantized feature map;

[0135] Entropy coding is performed based on the aforementioned prior information extraction module, prior information quantization module, prior information reconstruction module, reconstructed prior information, and distribution estimation module.

[0136] Entropy decoding is performed based on the aforementioned prior information reconstruction module, context modeling module, and distribution estimation module.

[0137] In a preferred embodiment of the present invention, S500 is performed for inverse quantization. Inverse quantization is the reverse process of quantization, using the same quantization step size and dead zone ratio as quantization, for elements at any position in the quantization feature map. The dequantized output is

[0138]

[0139] Where q is the quantization step size and z is the dead zone ratio.

[0140] In this embodiment, after inverse quantization, a value of [value] is obtained. The reconstructed feature map. Inverse quantization in this embodiment can improve rate-distortion performance in variable bit-rate scenarios.

[0141] In a preferred embodiment of the present invention, step S500 is performed, where the reconstructed feature map is passed through a first synthetic transform neural network to obtain a reconstructed image. Specifically, the decomposition transform neural network is composed of a reversible decoding subnetwork and an upsampling subnetwork connected sequentially. The reversible decoding subnetwork first divides the reconstructed feature map along the channel dimension, obtaining 8 segments of size [missing information]. implicit representation The reversible decoding subnetwork also contains three levels of reversible encoding modules, arranged in reverse order to the reversible encoding subnetwork, containing 4, 2, and 1 reversible decoding units respectively. The l-th (0≤l≤2) unit of the k-th layer (0≤k≤2) is... 2-k -1) The reversible coding unit is denoted as Its input is and The output is The computation process of each reversible decoding module consists of two steps. The first step is:

[0142]

[0143]

[0144] in and In contrast to the reversible coding subnetwork in the decomposition transform neural network and The same. The second step is to... and Merging is performed on the channel to obtain... This is the output of the reversible encoding unit. After passing through three levels of reversible decoding modules, a size of [size missing] can be obtained. The reconstructed implicit representation is obtained. The upsampling subnetwork consists of four deconvolutional layers and an iGDN (inverse generalized divisive normalization) layer. Each deconvolutional layer has a kernel size of 5×5, a stride of 2, and output channels of 192, 192, 192, and 3 respectively. An iGDN layer is located between every two convolutional layers. The output of the upsampling subnetwork is the reconstructed image of size H×W.

[0145] In a preferred embodiment of the present invention, S700 is implemented to perform bitrate control: based on the target bitrate This involves adjusting the quantization and dequantization parameters during the encoding process to make the compressed bitstream's bitrate close to the target bitrate, or to make the distortion of the reconstructed image close to the target distortion. This step includes the following five sub-steps:

[0146] S71, Quantization: The initial feature map is quantized three times using a dead-zone quantizer with quantization steps of q0, q1, and q2 and a dead-zone ratio of 1, to obtain quantized feature maps.

[0147] S72, z: Entropy coding is performed on the three feature maps respectively, and the number of codewords in the compressed code stream after encoding are s0, s1, and s2 respectively.

[0148] S73, Bitrate Calculation: Given an input image size of H×W and a compressed bitstream size of s, the compressed bitstream bitrate can be calculated. Through calculation, the bitrates of the above three compressed bitstreams are denoted as R0, R1, and R2, respectively.

[0149] S74, Quantization step size-bit rate modeling: fitting q0, q1, q2 with R0, R1, R2 And by determining the parameters α, β, and γ, the quantization step size-code rate model is obtained as follows:

[0150] S75, Quantization Step Size Decision: Target bitrate Inputting the quantization step size-bitrate model, we obtain the quantization step size corresponding to the target bitrate.

[0151] Tests on the Kodak dataset show that when the target bitrate is set to 0.25, 0.50, 0.75, 1.00, and 1.25 bpp, the average absolute error between the compressed bitrate and the target bitrate is 0.0118 bpp.

[0152] In this embodiment, regarding bitrate control, the neural network implemented by the technical solution of this embodiment can approximate the orthogonal transformation in the local space, thereby improving the average performance of variable bitrate coding; and it requires fewer parameters, which can effectively reduce the storage overhead of the model required to implement bitrate control.

[0153] Based on the same inventive concept, embodiments of the present invention also provide a variable bitrate image compression system, see [link to relevant documentation]. Figure 2The system comprises a forward mapping module, a quantization module, an entropy encoding module, an entropy decoding module, an inverse quantization module, an inverse mapping module, and a rate control module. Specifically, the forward mapping module passes the image to be encoded through a first decomposition transform neural network to obtain an initial feature map of the image; the quantization module quantizes the feature map using a dead-zone quantizer to obtain a quantized feature map of the image; the entropy encoding module uses an entropy model to entropy-encode the quantized feature map and the quantized prior information involved in the entropy model to obtain a compressed bitstream; the entropy decoding module performs entropy decoding on the compressed bitstream and uses the entropy model to sequentially recover the quantized prior information and the quantized feature map; the inverse quantization module inverse-quantizes the quantized feature map to obtain a reconstructed feature map of the image; the inverse mapping module passes the reconstructed feature map through a first synthesis transform neural network to obtain a reconstructed image; and the rate control module adjusts the quantization and inverse quantization parameters during the encoding process according to the target bitrate or target distortion, so that the bitrate of the compressed bitstream is close to the target bitrate, or the distortion of the reconstructed image is close to the target distortion.

[0154] The specific details of each module / unit in the above examples of the present invention (see [link]). Figure 3 and Figure 4 The implementation techniques of the variable bit rate image compression method in the above embodiments can be referred to, and will not be repeated here.

[0155] Based on the same inventive concept, embodiments of the present invention also provide an image compression apparatus that performs image compression using any of the variable bitrate image compression methods described in any one of the claims.

[0156] Based on the same inventive concept, embodiments of the present invention also include a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to describe any of the methods described above, or to run the system, or to run the device.

[0157] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods described above, or to run the system, or to run the apparatus.

[0158] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A variable bitrate image compression method, characterized in that, include: Forward mapping: The image to be encoded is passed through the first decomposition transformation neural network to obtain the initial feature map of the image; Quantization: The initial feature map is quantized using a dead-zone quantizer to obtain a quantized feature map of the image; Entropy coding: The quantized feature map and the quantized prior information involved in the entropy model are entropy encoded using the entropy model to obtain a compressed bitstream; Entropy decoding: The compressed bitstream is entropy decoded, and the quantized prior information and quantized feature map are recovered sequentially using the entropy model; Inverse quantization: The recovered quantized feature map is inverse quantized to obtain the reconstructed feature map of the image; Reverse mapping: The reconstructed feature map is passed through a first synthetic transform neural network to obtain a reconstructed image; Bitrate control: Based on the target bitrate or target distortion, the parameters of quantization and dequantization in the encoding process are adjusted so that the bitrate of the compressed bitstream is close to the target bitrate, or the distortion of the reconstructed image is close to the target distortion. The bitrate control includes: Bitrate calculation: Input the size of the image to be encoded Given the number of bits S in the compressed bitstream, the bitrate of the compressed bitstream is calculated as follows: ; Distortion calculation: Let the image to be encoded be... Size is Reconstruct the image as , reconstruction distortion ; Quantization step size-bitrate modeling: By performing quantization, entropy coding, and bitrate calculation three times with different quantization step sizes on the initial feature map, three sets of bitrates for quantization step sizes and compressed bitstreams are obtained, and a quantization step size-bitrate model is fitted. ， in, To quantize the step size, For bitrate, , and There are three parameters to be fitted; Quantization step size-distortion modeling: By performing multiple quantization, dequantization, inverse mapping, and distortion calculations on the initial feature map with different quantization step sizes, multiple sets of distortion values ​​for the reconstructed image are obtained, and a quantization step size-distortion model is fitted. ， in, To quantize the step size, For bitrate, , and There are three parameters to be fitted; Quantization step size decision: given target bit rate Or target distortion The corresponding quantization step size is inferred through the quantization step size-bit rate model or the quantization step size-distortion model. or .

2. The variable bitrate image compression method according to claim 1, characterized in that, The first decomposition transformation neural network includes: A downsampling subnetwork, implemented by a convolutional neural network, transforms the input image into a hidden representation; A reversible coding subnetwork transforms the hidden representation into an initial feature map; the reversible coding subnetwork is obtained by hierarchically arranging reversible coding units, and the hierarchical arrangement includes... layer, and the first layer( ) contains There are 1 reversible coding unit, each of which has two outputs, and the 1st... The two outputs of the layer reversible coding unit are used as the first The inputs of two reversible coding units in a layer are, for the only reversible coding unit in layer 1, an implicit representation; while for the layer 2... The layer provides The outputs can be merged to obtain the initial feature map.

3. The variable bitrate image compression method according to claim 2, characterized in that, The process performed in the reversible encoding unit includes: Eigenvalue decomposition: Dividing the input into two sub-signals; Reversible encoding: For the two sub-signals... and The reversible encoding process includes: ； in, and All of them are convolutional neural networks.

4. The variable bitrate image compression method according to claim 1, characterized in that, The quantization process involves adjusting the quantization step size of the dead-zone quantizer to output a quantized feature map that meets the target bit rate or target distortion requirements; for any element at any position in the initial feature map... Quantization output is ， Among them, when When it is a positive number, When it is 1, When it is negative, =-1, when When it is zero, =0; return The larger one; Returns no greater than The largest integer; To quantize the step size, This represents the dead zone ratio.

5. The variable bitrate image compression method according to claim 1, characterized in that, The entropy coding includes: estimating the Gaussian distribution of elements on the quantized feature map using an entropy model, performing arithmetic coding on the quantized feature map, and performing arithmetic coding on the quantized prior information using a constructed distribution to obtain a compressed bitstream.

6. The variable bit rate image compression method according to claim 1, characterized in that, The entropy decoding includes: performing arithmetic decoding on the quantized prior information using a constructed distribution, inputting it into an entropy model to obtain the codeword distribution of the quantized feature map, performing arithmetic decoding on the compressed bitstream, and recovering the quantized feature map.

7. The variable bitrate image compression method according to claim 1, characterized in that, The inverse quantization is the reverse process of the quantization, using the same quantization step size and dead zone ratio as the quantization. This applies to elements at any position in the recovered quantized feature map. The dequantized output is: ； in, To quantize the step size, This represents the dead zone ratio.

8. The variable bit rate image compression method according to claim 2, characterized in that, The first synthetic transform neural network includes: Reversible decoding subnetwork: The reversible decoding subnetwork transforms the reconstructed feature map into a reconstructed hidden representation; the reversible decoding subnetwork is obtained by a hierarchical arrangement of reversible decoding units, and the hierarchical arrangement includes layer, and the first layer( ) contains There are 1 reversible decoding unit; each reversible decoding unit has two inputs and one output, and the 1st... The outputs of the two reversible decoding units in the first layer are used as the outputs of the second layer. The input to a reversible decoding unit in the layer; by dividing the reconstructed feature map into Each component yields the first layer. One reversible decoding unit The first input; and the second The output of the single reversible decoding unit of the layer is the reconstructed implicit representation; parameter reuse is achieved between the reversible decoding subnetwork and the reversible coding subnetwork; Upsampling subnetwork: Implemented by a convolutional neural network, it transforms the reconstructed hidden representation into a reconstructed image.

9. A variable bit rate image compression method according to claim 8, characterized in that, The process performed in the reversible decoding unit includes: Reversible decoding: for two input signals and The reversible decoding process includes: ； in, and All are convolutional neural networks, among which and These are the trainable tests for two convolutional neural networks; Feature decomposition: merging and The output signal is obtained.

10. A variable bitrate image compression method according to claim 1, characterized in that, The parameters of the first decomposition transformation neural network, the first synthesis transformation neural network, and the entropy model are obtained through iterative learning based on the training image set, including: Obtain the training image set: Use the training image set as the images to be encoded; Forward mapping: The image to be encoded is passed through the first decomposition transformation neural network to obtain the initial feature map of the image; Quantization proxy: Quantizes the feature map using a dead-zone quantizer to obtain a noisy feature map of the image. For any element at any position in the initial feature map... Quantization proxy output is ； in, For the range of values ​​within Random uniform noise, and satisfy ； in, To quantize the step size, Dead zone ratio; Entropy estimation: Input the noisy feature map into the entropy model, and output the Gaussian distribution of the elements at each position on the noisy feature map. Then, the elements at any position... The sign probability is ； in, and Let be the estimated mean and variance of the Gaussian distribution, respectively; furthermore, let any element in the noisy prior information of the entropy model be denoted as . The symbol probability is obtained by constructing a distribution. The empirical entropy of the encoded noisy feature map and the noisy prior information is: ; Dequantization: Dequantize the noisy feature map to obtain a reconstructed feature map of the image. For any element at any position in the noisy feature map... The dequantized output is ； in, To quantize the step size, Dead zone ratio; Reverse mapping: The reconstructed feature map is passed through the first synthetic transform neural network to obtain the reconstructed image; Distortion estimation: Let the image to be encoded be... The size is Reconstruct the image as , reconstruction distortion ; Gradient update: By pre-fixing the dead zone ratio and giving... With different quantization step sizes, we can obtain Group experience entropy and reconstruction distortion are denoted as follows: Predetermine for it Lagrange multipliers During training, a quantization step size is randomly selected each time, resulting in the loss function for that iteration. The neural network parameters are then updated via backpropagation.

11. A variable bitrate image compression system, characterized in that, include: The forward mapping module passes the image to be encoded through the first decomposition transformation neural network to obtain the initial feature map of the image; The quantization module quantizes the feature map using a dead-zone quantizer to obtain a quantized feature map of the image. An entropy coding module uses an entropy model to entropy code the quantized feature map and the quantized prior information involved in the entropy model to obtain a compressed bitstream. The entropy decoding module performs entropy decoding on the compressed bitstream and uses the entropy model to sequentially recover the quantized prior information and quantized feature map. The dequantization module dequantizes the quantized feature map to obtain a reconstructed feature map of the image. The reverse mapping module reconstructs the feature map through a first synthetic transform neural network to obtain a reconstructed image; A bitrate control module adjusts the quantization and dequantization parameters during the encoding process according to the target bitrate or target distortion, so that the bitrate of the compressed bitstream is close to the target bitrate, or the distortion of the reconstructed image is close to the target distortion. The bitrate control includes: Bitrate calculation: Input the size of the image to be encoded Given the number of bits S in the compressed bitstream, the bitrate of the compressed bitstream is calculated as follows: ; Distortion calculation: Let the image to be encoded be... The size is Reconstruct the image as , reconstruction distortion ; Quantization step size-bitrate modeling: By performing quantization, entropy coding, and bitrate calculation three times with different quantization step sizes on the initial feature map, three sets of bitrates for quantization step sizes and compressed bitstreams are obtained, and a quantization step size-bitrate model is fitted. ， in, To quantize the step size, For bitrate, , and There are three parameters to be fitted; Quantization step size-distortion modeling: By performing multiple quantization, dequantization, inverse mapping, and distortion calculations on the initial feature map with different quantization step sizes, multiple sets of distortion values ​​for the reconstructed image are obtained, and a quantization step size-distortion model is fitted. ， in, To quantize the step size, For bitrate, , and There are three parameters to be fitted; Quantization step size decision: given target bit rate Or target distortion The corresponding quantization step size is inferred through the quantization step size-bit rate model or the quantization step size-distortion model. or .

12. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to perform the method of any one of claims 1-10, or to run the system of claim 11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program can be used to perform the method of any one of claims 1-10, or to run the system of claim 11.

Images