Learning image compression method based on side information autoregression

Abstract

The invention discloses a learning image compression method based on side information autoregression, and the method comprises the steps: taking a natural image x as input, obtaining a compact potential representation y, and directly coding y when the image x is actually compressed; the potential representation y is used as input, side information z is obtained, and the side information z is used for providing prior guidance for entropy coding of the potential representation y; a side information autoregression module is used for estimating probability distribution of side information z, and a compressed bit stream of the side information z is formed after entropy coding; the quantized side information z is sent to a hyper-synthesis transformation module to obtain hyper-prior information potentially representing y; estimating the probability distribution of the potential representation y by using the context of the potential representation y and the hyper-prior information, and forming a compressed bit stream of the potential representation y after entropy coding; and the quantized potential representation y is sent to a synthesis transformation module to obtain a final reconstructed image # imgabs0 #. The reconstructed image obtained by the method is more excellent in the aspect of structural similarity objective indexes, and texture details of the reconstructed image obtained in the aspect of subjective quality are reconstructed better.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision and data compression, and is used in industries such as image compression and transmission. Specifically, the invention relates to a learning image compression method based on side information autoregression. Background Art

[0002] Due to the rapid development of the Internet and computers, a huge amount of image data is constantly generated. This poses a major challenge to the transmission and storage of images. Therefore, people have been looking for more efficient image compression methods to maintain higher image quality at the lowest possible bit rate. Traditional image coding schemes, such as JPEG, AVC, HEVC, and VVC, have achieved remarkable performance, but their modules are not optimally integrated, and each component relies on manual design. This greatly limits the performance of traditional image compression codecs.

[0003] In recent years, significant progress has been made in learning-based image compression (LIC) models (proposed by Johannes Ballé, Li, Minnen, Hu, et al.). In particular, some advanced methods (proposed by Minnen, Cheng, Gao, Zou, Chen, He, Jiang, Li, et al.) have surpassed traditional image compression standards such as VVC. The end-to-end optimization capability of neural networks enables them to integrate and coordinate internal components under a unified goal, making learning-based image compression techniques gradually become a better choice. Currently, most image compression models are built on the variational autoencoder framework and follow a fixed processing flow: transform, quantize, and entropy coding. In the transform stage, the original image is mapped to a latent representation and further quantized into discrete values. The entropy coding model is responsible for estimating the probability distribution of these discrete latent representations, and then using entropy coding tools to losslessly compress them into a bitstream. In this process, how to use the entropy model to more accurately estimate the conditional probability distribution of the latent representation is a major challenge.

[0004] State-of-the-art learned image compression models model redundant information within latent representations by incorporating hyper-prior modules (Johannes Ballé et al.) and context modules (Minnen et al.). These additional modules are able to model the correlations between latent representations using conditional probabilities.

[0005] Specifically, the researchers use a hyper-prior module to extract side information from the latent representation, which is then used as a hyper-prior in an entropy model to estimate the probability distribution of the latent representation.

[0006] On the other hand, the context module captures the correlation between spatial and channel dimensions in the latent representation and uses previously decoded features and hyper-priors to predict the mean and variance of the Gaussian distribution of the feature currently being decoded.

[0007] In recent years, researchers have proposed a variety of methods (Koyuncu, Liu, He, Qian, Jiang, Li, etc.) to improve the context module, but less attention has been paid to the hyper-prior module. The most original fully factored density model (FFDM) for modeling the probability distribution of side information is still widely used. By visualizing the side information of the FFDM-based model (Figure), we find that the vast majority of channels in the side information are invalid, and only a few channels contain valid information. We believe this is because FFDM relies on a simple assumption that all images share the same distribution within the same side information channel, and the features of different channels are considered independent. This assumption causes the estimated side information entropy to be significantly larger than the actual entropy. During the joint training process of the end-to-end model, the model tends to allocate more bits to the potential representation using a more accurate Gaussian density model, while very little bitrate is allocated to the side information. As Figure 1 and 2 As shown in Figure 2, a visualization of Cheng’s side information is shown. Taking Cheng as an example, among the 192 side information channels, only five channels carry information. It is worth noting that among the remaining 187 channels, each channel holds the same value in different images (e.g. Figure 1 This means that the side information cannot retain enough useful information and cannot provide effective help in estimating the probability distribution of the latent representation. Summary of the invention

[0008] In the invention, we found a common problem in existing learning-based image compression models, that is, the potential of side information is not fully utilized. To overcome the shortcomings of this technology, the present invention provides a learning image compression method based on side information autoregression, which can prompt the neural network to allocate more bit rate to the side information during the end-to-end learning image compression training process. This can save richer contextual information in the side information, better guide the probability estimation of the potential representation, and improve the overall performance of the entropy module of the compression model. Experimental results show that our component has a significant reduction in the potential representation bit rate at the expense of a small increase in the side information bit rate. Our side information autoregression can be used as a plug-and-play module to replace the side information module of the existing model. Under the condition of comparable coding performance, the reconstructed image obtained by the present invention performs better in the objective indicator of structural similarity, and the texture details of the reconstructed image obtained in terms of subjective quality are better reconstructed.

[0009] The technical solution of the present invention is: the learning image compression method based on side information autoregression comprises the following steps:

[0010] S1, takes the natural image x as input and sends it to the analysis and transformation module to obtain a more compact potential representation y, which can be used to eliminate the redundant information in the image and reduce the number of features that need to be encoded when compressing the image (the feature is each value in the tensor). When actually compressing the image x, y can be directly encoded;

[0011] S2, taking the latent representation y as input and sending it to the super analysis transformation module to obtain the side information z, which can be used to provide prior guidance for the entropy coding of the latent representation y;

[0012] S3, using the side information autoregressive module to estimate the probability distribution of the side information z, which will be used for entropy coding of the quantized side information z, and after entropy coding, a compressed bit stream of the side information z is formed;

[0013] S4, the quantized side information z is fed into the super synthesis transformation module to obtain the super prior information of the potential representation y;

[0014] S5, using the context and super prior information of the potential representation y to estimate the probability distribution of the potential representation y, which will be used for entropy coding of the quantized potential representation y, and after entropy coding, a compressed bit stream of the potential representation y is formed;

[0015] S6, the quantized potential representation y is sent to the synthesis transformation module to obtain the final reconstructed image

[0016] The present invention provides a learning image compression method based on side information autoregression, which can prompt the neural network to allocate more bit rate to the side information during the end-to-end learning image compression training process. This can enable the side information to store richer contextual information, better guide the probability estimation of the potential representation, and improve the overall performance of the entropy module of the compression model. Experimental results show that our component has a significant reduction in the potential representation bit rate at the expense of a slight increase in the side information bit rate. Our side information autoregression can be used as a plug-and-play module to replace the side information module of the existing model. Under the condition of comparable coding performance, the reconstructed image obtained by the present invention performs better in the objective indicator of structural similarity, and the texture details of the reconstructed image obtained in terms of subjective quality are better reconstructed. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a visualization of the side information of the Cheng and Cheng+ours models (λ=0.045).

[0018] Figure 2 Schematic diagram of the average bit rate per channel of side information on the Kodak dataset (λ=0.045).

[0019] Figure 3A comparison chart of the original model and our proposed model.

[0020] Figure 4 This is the overall framework diagram of the learning image compression based on side information autoregression (SIA) of the present invention.

[0021] Figure 5 Visualization of the latent representation, hyper-prior, and regularized latent representation for Cheng and Cheng+ours models on Kadak8.

[0022] Figure 6 Comparison of performance and bitrate distribution on the Kodak dataset for three baseline models and the model improved by our method, including Cheng, STF-CNN and MLIC++.

[0023] Figure 7 Rate-distortion curves and ablation experiment results for three baseline models and our improved model based on them.

[0024] Figure 8 Reconstructing subjective quality maps for image compression. DETAILED DESCRIPTION

[0025] The technical solution of the present invention is: this learning image compression method based on side information autoregression, such as Figure 4 As shown, the following steps are included:

[0026] Take the natural image x as input and send it to the analysis and transformation module to obtain a more compact potential representation y, which can be used to eliminate redundant information in the image and reduce the number of features that need to be encoded when compressing the image (the feature is each value in the tensor). When actually compressing the image x, you can directly encode y;

[0027] The latent representation y is taken as input and sent to the super analysis transformation module to obtain the side information z, which can be used to provide prior guidance for the entropy coding of the latent representation y;

[0028] The probability distribution of the side information z is estimated using the side information autoregressive module, which will be used for entropy coding of the quantized side information z to form a compressed bit stream of the side information z after entropy coding;

[0029] The quantized side information z is fed into the super-synthetic transformation module to obtain the super-prior information of the potential representation y;

[0030] The probability distribution of the latent representation y is estimated using the context and super prior information of the latent representation y, which will be used for entropy coding of the quantized latent representation y to form a compressed bit stream of the latent representation y after entropy coding;

[0031] The quantized potential representation y is fed into the synthetic transformation module to obtain the final reconstructed image

[0032] The overall network framework diagram of the network model is as follows: Figure 4 As shown in Figure 2, similar to existing image compression models, it usually includes an analysis transformation network (g a , used to remove redundant information in the image and obtain a compact potential representation y), a synthetic transformation network (g s , used to reconstruct the latent representation y back to the original image), a hyperanalysis transformation network (h a , used to extract side information from the compact latent representation y, which can be used to help estimate the hyper-prior information of y), a hyper-synthetic transformer network (h s , transforming the side information into the hyper-prior information of y), as well as a latent representation entropy model (used to estimate the probability distribution of the latent representation) and a side information entropy model (used to estimate the probability distribution of the side information).

[0033] In step 3, in order to encourage the neural network to allocate the bit rate more reasonably (the number of bits occupied by each value of each pixel in the image), it is necessary to use a more accurate side information entropy model. Therefore, we propose a plug-and-play side information autoregression (SIA) module as our side information entropy model. This module can be seamlessly integrated with any image compression model based on side information, replacing the original FFDM. Specifically: In order to more accurately model the probability distribution of side information, we use a channel-by-channel autoregression method to split the side information z into L (this is a hyperparameter) slices along the channel dimension (that is, split a tensor z into multiple tensors {z0,z1,…,z L-1}), each slice {z0,z1,…,z L-1} contains S (this is a hyperparameter) channels.

[0034] These slices are encoded sequentially, and the previously decoded slices serve as the channel context information of the current decoded slice. Specifically, for the first slice z0, we still use FFDM to model its probability distribution (Based on the probability distribution we can determine the entropy of the variables contained in these slices, from which we can use various entropy coding schemes to encode these variables into binary bit streams):

[0035]

[0036] in represents the quantized z0 (that is, quantizing the original continuous value into a discrete value), Q represents the quantization operation, and ψ is the parameter of FFDM (it refers to the weight value in the neural network). represents unit uniform distribution, * represents the convolution operation of the function, is the probability distribution of the obtained side information, which will be used to encode the side information into a binary bit stream as mentioned above. Divide into two halves evenly along the channel dimension (That is, a tensor Split into two tensorz along the channel μ ,z σ ), z μ ,z σ It will be used to compress and encode the remaining slices. When compressing and encoding the remaining L-1 slices {z1,…,z L-1}, we use a mean-scaled Gaussian probability distribution to model these slices (as mentioned earlier, the probability here is also used to determine the entropy of the variables contained in these slices, based on which we can use various entropy coding schemes to encode these variables into binary bit streams).

[0037] To this end, we designed L-1 groups of side information mean modules and side information scale modules (this refers to the neural network) To predict the mean and variance (two parameters of the Gaussian probability distribution) of each feature of the current decoded slice.

[0038]

[0039] It refers to the set of all slices with slice numbers from 1 to i-1, that is μ i That is, the mean of the Gaussian distribution of each feature mentioned above, σ i That is, the variance of the Gaussian distribution of each feature. The representative mean is μ i , with variance σ i Gaussian distribution, is a unit uniform distribution, and * represents the convolution operation of the function. To obtain the Gaussian probability distribution of the i-th slice, as mentioned above, this probability distribution can be used in entropy coding to encode the eigenvalues ​​into a binary code stream.

[0040] This design exploits the rich contextual information of previously decoded slices at the same or nearby locations to adaptively estimate the mean and variance for each feature. It is composed of three layers of convolutional layers and two layers of LeakyReLU activation functions stacked alternately. The input of this network is the previously decoded side information, and the output is the mean and variance of the currently decoded side information. Using this variance, we can estimate an accurate probability distribution for each currently decoded side information feature.

[0041] In steps 2 and 4, in order to better extract / utilize richer and more compact side information, h a Modules and h s The module is no longer simply composed of convolutions, but of g a Modules and g s The module is composed of components similar to the original model, such as the window attention layer, residual block and GDN.

[0042] In addition, since we adjusted the probability modeling method of side information, the overall loss function of the network needs to be adjusted as follows:

[0043]

[0044] in is the code rate of the potential representation y That is to say, the number of bits of y that encode the average of each value in the RGB three channels of each pixel of the original image x. is the code rate of the potential representation y That is to say, the number of bits z that encodes the average of each value in the RGB three channels of each pixel of the original image x. Distortion value, that is, the original image x and the reconstructed image MES (an indicator for measuring image differences, which will be described in detail later). λ is the rate-distortion factor, which is used to measure the balance between bit rate and image distortion. Under the end-to-end unified training of the neural network, the larger λ is, the smaller the distortion is, the larger the bit rate is, and vice versa.

[0045] In summary, the bitrate estimation of the side information is divided into two steps. The first slice of the side information still uses the FFDM method, using an increasing neural network with values ​​ranging from 0 to 1 to fit the probability distribution of each feature. The remaining L-1 slices are fitted using the Gaussian density model.

[0046] Experimental Section

[0047] The present invention verifies the side information autoregression effects of three models, including Cheng, STF-cnn and MLIC++, on the Pytorch 2.2.2 and CompressAI 1.2.6 platforms.

[0048] Following the method of Minnen et al., the hyperparameter S is set to 320, but the number of channels of side information is set to 224, which means that L is set to 7. Considering the non-convex nature of the neural network, we set the second term of the loss function to Multiply by 0.1. Following the settings in the original papers of each model, we selected a subset of the ImageNet dataset to train Cheng+ours, 300K images from the OpenImage dataset to train STF-cnn+ours, and 100k images larger than 512×512 from ImageNet, COCO 2017, DIV2K, and Flickr2K to train (MLIC++)+ours.

[0049] Experimental dataset:

[0050] The ImageNet dataset is mainly used for computer vision tasks such as image recognition, classification, detection and segmentation. It contains more than 14 million images, divided into 21,841 categories, and is organized based on the WordNet hierarchy. Each category contains hundreds to thousands of high-definition images. It was created by Professor Fei-Fei Li's team at Stanford University in 2009 and is a key dataset for deep learning to achieve breakthroughs in the field of image recognition.

[0051] The OpenImages dataset is used for tasks such as image recognition, classification, object detection, and segmentation. It contains about 9 million images, annotated with 6,000 categories, with strong image diversity and a large amount of annotation information, including image-level labels, bounding boxes, and segmentation masks. It was released by the Google team and is an open source, diverse, and large-scale image dataset.

[0052] The COCO 2017 dataset is mainly used for tasks such as object detection, segmentation, human key point recognition, and caption generation. It contains about 120,000 images with more than 80 categories labeled. It provides detailed annotation information, including the bounding box of the object, segmentation mask, and category label. Released by the Microsoft team, it is an important benchmark dataset in the field of computer vision.

[0053] The DIV2K dataset is mainly used for image super-resolution tasks. It contains 1,000 high-definition images that are downsampled to low-resolution versions. The images are of high quality and rich in details, making them suitable for developing and evaluating super-resolution algorithms. It is provided by the NTIRE 2017 and 2018 Super-Resolution Challenges and is a standard dataset in the field of super-resolution.

[0054] The Flickr2K dataset is also used for image super-resolution tasks. It contains about 2,500 high-definition images. The images come from Flickr and cover a variety of scenes and themes. It is suitable for research and testing super-resolution algorithms and is used by academic research and industry. It is a publicly available image dataset.

[0055] Evaluation Metrics:

[0056] MSE (Mean Squared Error) is a method to measure the difference between the reconstructed image and the original image. It calculates the average of the square of the error of each pixel. The smaller the MSE value, the closer the reconstructed image is to the original image and the smaller the error. The formula is as follows: Where m and n are the image width and height, I and K represent the original image and the reconstructed image respectively.

[0057] PSNR (Peak Signal-to-Noise Ratio): Peak signal-to-noise ratio is an image quality evaluation indicator based on MSE. It measures the ratio of the maximum possible signal strength to the noise strength in decibels (dB). The larger the PSNR value, the better the image quality. Generally speaking, a PSNR value higher than 30dB indicates acceptable image quality, while a PSNR value higher than 50dB indicates very good image quality. The formula is as follows: Where Max represents the maximum possible pixel value of the image.

[0058] Experimental setup:

[0059] The models are optimized using Adam and the mean square error (MSE) is used as the quality metric. Other hyperparameters, such as the lambda value, number of iterations, learning rate decay, batch size, and patch size, are consistent with the original papers of each benchmark model and will not be described in detail here. In particular, due to GPU memory limitations, the batch size when training (MLIC++)+ours is set to 4. We used a single 3080Ti and AMD A10-7870K Radeon R7 for training all models, with the operating system being Ubuntu and the hard disk size being 5T.

[0060] Comparative experiment:

[0061] We perform computations on the widely used Kodak dataset (e.g. Figure 7 All models are evaluated by the average rate-distortion performance (PeakSignal-to-Noise Ratio, PSNR) on the Cheng+ours model. It can be seen that our method significantly improves the performance of the original model. In particular, on the Cheng+ours model, the PSNR is improved by 0.65dB (λ=0.03) and 0.59dB (λ=0.045) compared with the baseline model. At the same time, we visualize the potential representation y of Cheng and Cheng+ours, as shown in Figure 5As shown in . In Cheng+ours, since the side information z contains richer information, it can provide a more accurate Gaussian distribution mean and scale for the latent representation y. Therefore, the normalized latent representation is closer to uniform noise, and its absolute value is closer to 0. Taking the fifth λ value as an example, our autoregressive side information trades a small increase in the side information bit rate for a significant reduction in the latent bit rate (as shown in Figure 6 ). We Figure 8 The figure shows some subjective comparisons between the original model and our improved model. It can be seen that the edges and contours of our model are clearer than those of the original model.

[0062] To demonstrate the performance of our proposed component, the following ablation experiments are conducted.

[0063] We separately trained the hyperanalysis transformer network h with the enhancement a and the super-synthetic transformation network h s The STF-cnn model (denoted as STF-cnn+EN) has a rate-distortion curve as follows Figure 7 The right figure shows that at low bit rates, the performance of the model is significantly improved, which fully demonstrates the enhanced super-analysis transformation network h a and the super-synthetic transformation network h s At the same time, it can be observed that at high bitrates, the performance of STF-cnn+EN is close to that of STF-cnn, while it is farther away from that of STF-cnn+ours, which further proves the effectiveness of SIA.

[0064] Complexity analysis:

[0065] Table 1 shows the complexity and BD-RATE of Cheng, STF-cnn, MLIC++ and our improved model. The complexity includes MACs (multiply-accumulate operations), FLOPs (floating-point operations) and the number of parameters (the input image size is 512×512). It can be seen that the complexity of Cheng+ours and STF-cnn+ours is reduced compared with the baseline model, while the complexity of (MLIC++)+ours is only slightly increased compared with the original model. In other words, our model reduces the computational cost while improving the compression performance, or only brings a slight increase. In addition, we also show the complexity of SIA and FFDM (Table 2). Although the complexity of SIA is higher than that of FFDM, it is negligible for the entire model. For example, Cheng's MACs is 282.5G, while SIA's MACs is only 86.7M.

[0066] Table 1. Comparison of complexity and BD-Rate of different models, all models are trained with the largest λ. Complexity includes MACs (G), FLOPs (G), and Params (M). For BD-Rate (%) comparison of PSNR (dB), the benchmark is the respective baseline model. .

[0067]

[0068] Table 2. Comparison of FFDM and SIA complexity.

[0069]

Claims

1. A learning image compression method based on side information autoregression, characterized in that: The following steps are involved: S1 takes the natural image x as input and sends it to the analysis and transformation module to obtain a compact potential representation y, which is used to eliminate the redundant information in the image and reduce the number of features that need to be encoded when compressing the image. The feature is each value in the tensor. When compressing the image x, y is directly encoded; S2 takes the latent representation y as input and sends it to the super-analysis transformation module to obtain the side information z, which is used to provide prior guidance for the entropy coding of the latent representation y; S3 uses the side information autoregressive module to estimate the probability distribution of the side information z, which will be used for entropy coding of the quantized side information z to form a compressed bit stream of the side information z after entropy coding; S4 sends the quantized side information z to the super-synthetic transformation module to obtain the super-prior information of the potential representation y; S5 estimates the probability distribution of the potential representation y using the context and super prior information of the potential representation y, which will be used for entropy coding of the quantized potential representation y to form a compressed bit stream of the potential representation y after entropy coding; S6 sends the quantized potential representation y into the synthetic transformation module to obtain the final reconstructed image 2. The learning image compression method based on side information autoregression according to claim 1, characterized in that: The overall network framework of the network model includes an analysis transformation network g a , used to remove redundant information in the image and obtain a compact potential representation y; a synthetic transformation network g s , used to reconstruct the latent representation y back to the original image; a hyperanalysis transformation network h a , used to extract side information from the compact potential representation y to help estimate the super prior information of y; a super synthetic transformation network h s , which transforms the side information into the hyper-prior information of y, as well as a latent representation entropy model to estimate the probability distribution of the latent representation, and a side information entropy model to estimate the probability distribution of the side information.

3. The learning image compression method based on side information autoregression according to claim 2, characterized in that: In S3, in order to encourage the neural network to allocate bit rate, the number of bits occupied by each value of each pixel in the image is calculated using a more accurate side information entropy model; a plug-and-play side information autoregressive SIA module is proposed as the side information entropy model; in order to more accurately model the probability distribution of side information, a channel-by-channel autoregressive method is used to split the side information z into L slices along the channel dimension, that is, a tensor z is split into multiple tensors {z0,z1,…,z L-1 }), each slice {z0,z1,…,z L-1 } contains S channels.

4. The learning image compression method based on side information autoregression according to claim 3, characterized in that: The slices are encoded sequentially, and the previously decoded slices are used as the channel context information of the current decoded slice; for the first slice z0, FFDM is used to model its probability distribution The entropy of the variables contained in these slices is determined according to the probability distribution, and various entropy coding schemes are used to encode these variables into binary bit streams: in represents the quantized z0, Q represents the quantization operation; ψ is the parameter of FFDM; represents unit uniform distribution, * represents the convolution operation of the function, is the probability distribution of the obtained side information, which is used to encode the side information into a binary bit stream; Divide into two halves evenly along the channel dimension Put one Split into two tensorz along the channel μ ,z σ ;z μ ,z σ It will be used to compress and encode the remaining slices; when compressing and encoding the remaining L-1 slices {z1,…,z L-1 }, a mean-scaled Gaussian probability distribution is used to model the slices.

5. The learning image compression method based on side information autoregression according to claim 4, characterized in that: Design L-1 groups of side information mean module and side information scale module To predict the mean and variance of each feature of the current decoded slice; It refers to the set of all slices with slice numbers from 1 to i-1, that is μ i That is, the mean of the Gaussian distribution of each feature mentioned above, σ i That is, the variance of the Gaussian distribution of each feature; The representative mean is μ i , with variance σ i Gaussian distribution, is a unit uniform distribution, * represents the convolution operation of the function; To obtain the Gaussian probability distribution of the i-th slice, as mentioned above, this probability distribution is used in entropy coding to encode the eigenvalues ​​into a binary code stream.

6. The method for learning image compression based on side information autoregression according to claim 5, characterized in that: The mean and variance are estimated for each feature using rich contextual information from previously decoded slices at the same or nearby locations. It is composed of three layers of convolutional layers and two layers of LeakyReLU activation functions stacked alternately; the network input is the previously decoded side information, and the output is the mean and variance of the currently decoded side information; using the variance, an accurate probability distribution is estimated for each currently decoded side information feature.

7. The method for learning image compression based on side information autoregression according to claim 6, characterized in that: h a Modules and h s The module consists of the original model g a Modules and g s The module is composed of similar components, including the window attention layer, residual block and GDN in the original model; To adjust the probability modeling method of side information, the overall loss function of the network needs to be adjusted as follows: in, is the code rate of the potential representation y The number of bits y that encode the average of each value in the RGB three channels of each pixel of the original image x; is the code rate of the potential representation y ( and ), the number of bits that encode the average of each value z in the RGB three channels of each pixel of the original image x; Distortion value, that is, the original image x and the reconstructed image MES; λ is the rate-distortion factor, which is used to measure the balance between bit rate and image distortion. Under the end-to-end unified training of the neural network, the larger λ is, the smaller the distortion is and the larger the bit rate is, and vice versa. The bit rate estimation of the side information is divided into two steps. The first slice of the side information still uses the FFDM method, using an increasing neural network with values ​​ranging from 0 to 1 to fit the probability distribution of each feature; the remaining L-1 slices are fitted using a Gaussian density model.

Images