A multi-rate deep image compression system and method for mixed context

Abstract

The present invention relates to a mask-based multi-rate deep image compression system and method for use in mixed contexts. The system comprises a deep image encoding and decoding module, a latent vector importance analysis module, and a latent space encoding module. The present invention utilizes a single deep encoding and decoding model to achieve multi-rate and mixed context adaptation. Compared to current methods that train multiple models for rate and context adaptation, this method saves significant storage and communication resources. Furthermore, when performing image compression encoding, the present invention uses a single deep encoder to generate the latent vector, and achieves rate and context adaptation through latent space encoding, resulting in more real-time image compression and transmission.

Description

Technical Field

[0001] The present invention relates to the technical field of image compression, and in particular to a mask-based multi-rate deep image compression system and method applied in a mixed context. Background Art

[0002] Images are one of the most important sources of information transmitted on the internet. The transmission of visual information consumes a large amount of communication resources, making image compression crucial for efficient image transmission. Deep image compression (DIC), which leverages deep neural networks to achieve image compression, has attracted widespread attention. Compared with traditional image compression algorithms (such as BPG, JPEG200, and JPEG), DIC offers several advantages: First, whereas traditional image compression algorithms optimize each module in the codec individually, DIC employs an end-to-end learning approach to globally optimize the compression system, achieving overall optimality. Second, whereas traditional image compression algorithms employ manually designed feature representations for compression, DIC learns from large amounts of data based on specific application scenarios. Consequently, DIC exhibits superior performance in these scenarios.

[0003] In practical communication systems, channels are time-varying, so rapidly adapting to channel variations and providing image compression at different bitrates is a crucial challenge for practical DIC algorithms. Currently, most DIC algorithms use a single deep compression model for compression at a single bitrate. These deep compression models have a large number of parameters, and their training and storage consume significant computational and storage resources. Training different compression models for different bitrates is clearly undesirable. Therefore, practical DIC algorithms should train a single compression model to achieve multi-bitrate compression. Context can be defined as how the image receiver uses the compressed reconstructed image, such as for visual recording, image classification, or entertainment. Image compression in mixed contexts requires that the compressed reconstructed image not only possess high-quality visual quality but also achieve superior performance in downstream artificial intelligence (AI) tasks. Practical image compression methods in mixed contexts should train a single compression model for bitrate adaptation. Furthermore, flexible and configurable mechanisms should be available to enable gradual transitions between different contextual objectives within resource constraints.

[0004] Most current DIC compression methods are primarily applied in a single context, typically image reconstruction or image classification. However, there are relatively few multi-rate DIC methods for mixed contexts. Multi-rate DIC architectures for mixed contexts can be categorized as encoder-based or latent vector-based. Encoder-based DIC architectures require the generation of new latent vectors each time the rate or context changes, incurring additional computational overhead and processing latency. Summary of the Invention

[0005] In view of the problems existing in the prior art, the purpose of the present invention is to provide a mask-based multi-rate deep image compression method applied in a mixed context, aiming to simultaneously meet the mixed context goals of image classification and reconstruction.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A mask-based multi-rate deep image compression system for mixed contexts, comprising a deep image encoding and decoding module, a latent vector importance analysis module, and a latent space encoding module;

[0008] The deep image encoding and decoding module includes a depth encoder, a quantizer, an entropy encoder, an entropy decoder and a depth decoder; wherein the depth encoder and the depth decoder are composed of a deep convolutional neural network that has been trained and converged, and the parameters of the convolutional neural network are trained in an end-to-end manner;

[0009] The depth encoder is used to input the original image and generate a latent space potential vector;

[0010] The quantizer is used to input the to-be-sent vector output by the latent space coding module and perform scalar quantization processing on each element of the to-be-sent vector;

[0011] The entropy encoder is used to input the quantized latent vector and perform entropy encoding on the quantized latent vector to generate a final transmitted code stream, which is sent to the entropy decoder via a channel;

[0012] The entropy decoder is used to receive a code stream, perform entropy decoding on the code stream, and reconstruct a latent vector;

[0013] The depth decoder is used to input the potential vector reconstructed by the entropy decoder, perform image reconstruction, and output a reconstructed image of the original image;

[0014] The latent vector importance analysis module includes a semantic importance analysis unit, a reconstruction importance analysis unit and an importance map fusion unit;

[0015] The semantic importance analysis unit is used to input the original image, analyze the importance of each pixel of the original image to the image classification result, and map each plane position of the latent space potential vector to generate a semantic importance map of the latent space potential vector;

[0016] A reconstruction importance analysis unit, configured to input a latent vector, analyze the importance of each plane position of the latent vector to image reconstruction, and generate a reconstruction importance map of the latent vector in the latent space;

[0017] An importance map fusion unit, configured to input a semantic importance map of a latent vector and a reconstruction importance map to generate an importance map of a latent vector in a mixed context;

[0018] The latent space encoding module includes a latent space mask generation unit and an encoding unit;

[0019] The latent space mask generation unit is used to input the target code rate and latent vector importance map of the channel feedback, calculate the number of elements of the maximum latent vector allowed to be transmitted under the current channel resources, and then calculate the latent vector importance map Figure 2 The threshold of the value and the importance map Figure 2 The value is converted into a latent space mask vector;

[0020] The encoding unit is used to input a latent vector and a latent space mask vector, select elements from the latent vector according to the latent space mask vector, and concatenate the selected elements with the mask to form a vector to be sent.

[0021] The depth encoder and depth decoder are iteratively trained in an end-to-end manner, and the objective function of the training process is:

[0022]

[0023] Among them, E, D represent the depth encoder and decoder, is the bit rate of the latent vector, is the semantic feature matching loss term, is the reconstruction loss item;

[0024]

[0025]

[0026] Among them, x, y are the original image and the reconstructed image, (H, W, 3) are the dimensions of x, y, ξ(· is the semantic feature extraction function, H ξ ,W ξ ,C ξ are the dimensions of the semantic feature vectors.

[0027] The process of iterative training of the deep encoder and deep decoder network models is as follows:

[0028] Step S1: Sample an input image x from the training set and generate a latent vector z through a deep encoder. Calculate the variance of each plane position of the vector z along the channel direction to obtain the variance vector latent vector v. Use the relative bit rate R(z) to estimate the bit rate of the compressed vector. The bit rate loss term is calculated as follows:

[0029]

[0030] Step S2: Generate a random binary mask vector M of the same dimension as the vector v, and transform the potential vector z according to M according to formula (5) to obtain the vector Pair Vector Add noise to simulate the quantization process, according to the vector after noise addition And the mask vector M is used to obtain the reconstructed image y. The reconstruction loss and semantic feature matching loss are calculated as shown in equations (2) and (3);

[0031]

[0032] in, for The element z i,j,l is the element of z, M i,j is the element of the mask vector M;

[0033] Step S3, the final loss term is as follows:

[0034]

[0035] Step S4: solve the loss term for each parameter of the depth encoder and depth decoder The Adam optimizer is used to update the parameters.

[0036] The semantic importance analysis unit uses a classifier based on a deep convolutional neural network to classify the input image, and uses a gradient-based class activation mapping method to calculate the importance of each pixel to the classification result. Then, based on the spatial invariance of the convolutional neural network, the pixel importance is mapped to the latent space to obtain a latent vector semantic importance map.

[0037] The reconstruction importance analysis unit calculates the variance of all elements at each plane position of the potential vector. The larger the variance, the greater the entropy of the position, and the more important all elements at the position are to the reconstruction. The variance value of each position is normalized to obtain a reconstruction importance map.

[0038] A mask-based multi-rate deep image compression method for mixed contexts, implemented using the system described above, comprises the following steps:

[0039] Step 1: Pass the original image x through the deep encoder to generate the latent vector z;

[0040] Step 2: Input the original image x and the latent vector z into the latent vector importance analysis module to generate a latent vector importance map I;

[0041] Step 3: The latent space encoding module generates a latent space mask vector M based on the latent vector importance map I output by the latent vector importance analysis module and the target bit rate r, and transforms the latent vector z into a vector The vector Splice with the mask vector M to generate the vector W to be sent;

[0042] Step 4: The quantizer quantizes the vector W to be transmitted, and the entropy encoder uses an adaptive arithmetic coding method to quantize the vector W. Perform entropy coding to generate a bit stream, and send the bit stream to the entropy decoder through the channel;

[0043] Step 5: The entropy decoder uses an adaptive arithmetic decoding algorithm to perform entropy decoding on the received code stream and reconstruct the potential vector

[0044] Step 6: The deep decoder reconstructs the latent vector Generate the reconstructed original image y.

[0045] The step 2 is specifically as follows:

[0046] Step 2-1: The original image x is input into the semantic importance analysis unit to perform semantic importance analysis on the latent vector to obtain a latent space latent vector semantic importance map, denoted as vector e. An adaptive binarization algorithm is used to binarize vector e into vector B.

[0047] Step 2-2: Input the latent vector z into the importance analysis unit, calculate the variance vector v of the latent vector composed of the variance of the C element values ​​at each plane position of the latent vector z, and use v as the reconstructed importance map;

[0048] Step 2-3: The importance map fusion unit generates a latent vector importance map I according to the following formula:

[0049] I=(1+e)B+sigmoid(v)1-B(7).

[0050] The step 3 is specifically as follows:

[0051] Step 3-1: The latent space mask generation unit takes the Sth largest number in vector I as the binarization threshold T based on the target bit rate r and the importance map I. Based on T, I is binarized into a mask M, where S is the number of elements allowed to be sent at the target bit rate divided by the number of channels, rounded up.

[0052] Step 3-2: The encoding unit transforms the potential vector z according to the following formula to generate the vector And form the vector W to be sent with the mask vector M;

[0053]

[0054] in, for The element z i,j,l is the element of z, M i,j is an element of the mask vector M. After adopting the above solution, the present invention has the following beneficial effects:

[0055] 1. The present invention is applicable to image compression coding in mixed contexts. The image reconstructed at the receiving end satisfies both the reconstruction and classification contexts, thereby improving the utility of the reconstructed image.

[0056] 2. This invention uses a single deep encoding and decoding model to achieve multi-bitrate and mixed-context adaptation, which saves a lot of storage and communication resources compared to the current method of training multiple models for bitrate and context adaptation;

[0057] 3. When performing image compression encoding, the proposed method uses a deep encoder to generate a latent vector through a single pass. This latent space encoding achieves bitrate and context adaptation, enabling more real-time image compression and transmission. Furthermore, this bitrate and context adaptation approach is more suitable for multicast and broadcast scenarios, saving significant computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of the overall framework of the mask-based progressive depth image compression method applied in a mixed context according to the present invention;

[0059] Figure 2 This is a schematic diagram of the deep encoding and decoding network model training process in the present invention;

[0060] Figure 3 This is the implementation case deep encoding and decoding network model diagram;

[0061] The present invention will be described in further detail below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] like Figure 1 As shown, the present invention discloses a mask-based multi-rate deep image compression system applied in a mixed context, which includes a deep image encoding and decoding module, a latent vector importance analysis module and a latent space encoding module.

[0063] The deep image encoding and decoding module includes a deep encoder, quantizer, entropy encoder, entropy decoder, and deep decoder. The deep encoder and deep decoder are composed of a deep convolutional neural network that has been trained to convergence. The parameters of the convolutional neural network are trained in an end-to-end manner.

[0064] The depth encoder is used to input the original image and generate the latent space potential vector. Specifically, in this embodiment, the depth encoder downsamples the original image with a dimension of (H, W, 3) by 8 times to obtain a dimension of In order to adapt to different channel conditions and different contexts, the latent space potential vector generated by the deep encoder needs to be sent to the latent space encoding module for further encoding.

[0065] The quantizer is used to input the to-be-sent vector output by the latent space encoding module and perform scalar quantization on each element of the to-be-sent vector to reduce the entropy of the compressed vector for transmission over the communication channel. In this embodiment, the quantizer is a scalar quantizer with a quantization level of 5 and a quantization center of {-2, -1, 0, 1, 2}.

[0066] The entropy encoder is used to input the quantized latent vector and perform entropy encoding on the quantized latent vector to generate a final transmit bitstream, which is then sent via a channel to the entropy decoder. In this embodiment, the entropy encoder is an adaptive arithmetic encoder. Entropy encoding can remove redundancy in the compressed vector and improve coding efficiency. This process is lossless to the information in the quantized latent vector.

[0067] The entropy decoder is used to receive the bit stream, perform entropy decoding on the bit stream, and reconstruct the latent vector. In this embodiment, the entropy decoder is an adaptive arithmetic decoder.

[0068] The depth decoder is configured to input the latent vector reconstructed by the entropy decoder, perform image reconstruction, and output a reconstructed image of the original image. Specifically, in this embodiment, the depth decoder performs an 8x upsampling on the reconstructed latent vector to generate a reconstructed image.

[0069] The latent vector importance analysis module includes a semantic importance analysis unit, a reconstruction importance analysis unit and an importance map fusion unit.

[0070] The semantic importance analysis unit is used to input the original image, analyze the importance of each pixel of the original image to the image classification result, and map each plane position of the latent space potential vector to generate a semantic importance map of the latent space latent vector. In this embodiment, the semantic importance analysis unit uses VGG16 to classify the input image and uses the gradient-based class activation mapping (CAM) method Grad-CAM++ to calculate the importance of each pixel to the classification result. Then, based on the spatial invariance of the convolutional neural network, the pixel importance is mapped to the latent space to obtain a semantic importance map of the latent vector.

[0071] The reconstruction importance analysis unit is configured to input a latent vector and analyze the importance of each plane position of the latent vector to image reconstruction, thereby generating a reconstruction importance map of the latent vector in the latent space. In this embodiment, the reconstruction importance analysis unit calculates the variance of all elements at each plane position of the latent vector. A larger variance indicates greater entropy at that position and more importance of all elements at that position to reconstruction. The variance value at each position is normalized to generate the reconstruction importance map.

[0072] The importance map fusion unit is used to input the semantic importance map of the latent vector and the reconstruction importance map to generate the latent vector importance map in the mixed context.

[0073] The latent space encoding module includes a latent space mask generation unit and an encoding unit.

[0074] The latent space mask generation unit is used to input the target code rate and latent vector importance map of the channel feedback, calculate the number of elements of the maximum latent vector allowed to be transmitted under the current channel resources, and then calculate the latent vector importance map Figure 2 The threshold of the value and the importance map Figure 2 The value is converted into a latent space mask vector.

[0075] The encoding unit is used to input the latent vector and the latent space mask vector, and select elements from the latent vector according to the latent space mask vector to form a vector to be sent.

[0076] The depth encoder and depth decoder are iteratively trained in an end-to-end manner, and the objective function of the training process is:

[0077]

[0078] Among them, E, D represent the depth encoder and decoder, is the bit rate of the latent vector, is the semantic feature matching loss term, is the reconstruction loss term, the commonly used reconstruction loss term is the mean square error,

[0079]

[0080]

[0081] Among them, x, y are the original image and the reconstructed image, (H, W, 3) are the dimensions of x, y, ξ(·) is the semantic feature extraction function, H ξ ,W ξ ,C ξ are the dimensions of the semantic feature vectors.

[0082] like Figure 2As shown in the figure, the ImageNet dataset is used as the training set, the input images are resized to 256×256, the Adam optimizer is used, the learning rate is 2e-4, and the batch size is 1. The iterative training process of the deep encoder and deep decoder network models is as follows:

[0083] Step S1: Sample an input image x from the training set and generate a latent vector z through a deep encoder. Calculate the variance of each plane position of the vector z along the channel direction to obtain the variance vector latent vector v. Use the relative bit rate to estimate the bit rate of the compressed vector. The bit rate loss term is calculated as follows:

[0084]

[0085] Step S2: Generate a random binary mask vector M of the same dimension as the vector v, and transform the potential vector according to formula (5) to obtain the vector Pair Vector Noise is added to simulate the quantization process, and the deep decoder inputs the noisy vector Get the reconstructed image y, and calculate the reconstruction loss and semantic feature matching loss as shown in equations (2) and (3);

[0086]

[0087] in, for The element z i,j,l is the element of z, M i,j is the element of the mask vector M;

[0088] Step S3, the final loss term is as follows:

[0089]

[0090] Step S4: solve the loss term for each parameter of the depth encoder and depth decoder The Adam optimizer is used to update the parameters.

[0091] Continue to refer to Figure 1 Combined with Figure 3 As shown, based on the above system, the present invention also discloses a mask-based multi-rate deep image compression method applied in a mixed context, which includes the following steps:

[0092] Step 1: Pass the original image x through the deep encoder to generate the latent vector z;

[0093] Step 2: Input the original image x and the latent vector z into the latent vector importance analysis module to generate a latent vector importance map I;

[0094] Step 2-1: The original image x is input into the semantic importance analysis unit to perform semantic importance analysis on the latent vector to obtain a latent space latent vector semantic importance map, denoted as vector e. The vector e is binarized into vector B using an adaptive binarization algorithm.

[0095] Step 2-2: Input the latent vector z into the importance analysis unit, calculate the variance vector v of the latent vector composed of the variance of the C element values ​​at each plane position of the latent vector z, and use v as the reconstructed importance map;

[0096] Step 2-3: The importance map fusion unit generates a latent vector importance map I according to the following formula:

[0097] I=(1+e)B+sigmoid(v)1-B(6)

[0098] Step 3: The latent space encoding module generates a latent space mask vector M based on the latent vector importance map I output by the latent vector importance analysis module and the target bit rate r, and transforms the latent vector z into a vector according to formula (5). The mask vector M and Splice to generate the vector W to be sent;

[0099] Step 3-1: The latent space mask generation unit obtains S according to the target bit rate r and the importance map I according to formula (7), and takes the S-th largest number in the vector I as the binarization threshold T;

[0100]

[0101] Where P is the number of elements in the mask vector M, S is the number of 1 positions in the mask vector M, AAC(·) is the adaptive algorithm used to calculate the bit of each pixel in the input vector, o is the vector composed of the quantized elements extracted from z when all elements in the mask vector M are 1, and u is the vector composed of the quantized elements extracted from z when all elements in the mask vector M are 0.

[0102] Step 3-2: The encoding unit transforms the potential vector according to formula (5) to generate the vector And combine with the mask vector M to form the vector W to be sent;

[0103] Step 4: The quantizer quantizes the vector W to be transmitted into a vector The entropy encoder uses an adaptive arithmetic coding method to encode the vector Perform entropy coding to generate a bit stream, and send the bit stream to the entropy decoder through the channel;

[0104] Step 5: The entropy decoder uses an adaptive arithmetic decoding algorithm to entropy decode the received code stream and reconstruct the potential vector

[0105] Step 6: The deep decoder reconstructs the latent vector Generate the reconstructed original image y.

[0106] In summary, the present invention has the following beneficial effects:

[0107] 1. The present invention is applicable to image compression coding in mixed contexts. The image reconstructed at the receiving end satisfies both the reconstruction and classification contexts, thereby improving the utility of the reconstructed image.

[0108] 2. This invention uses a single deep encoding and decoding model to achieve multi-bitrate and mixed-context adaptation, which saves a lot of storage and communication resources compared to the current method of training multiple models for bitrate and context adaptation;

[0109] 3. When performing image compression encoding, the proposed method uses a deep encoder to generate a latent vector through a single pass. This latent space encoding achieves bitrate and context adaptation, enabling more real-time image compression and transmission. Furthermore, this bitrate and context adaptation approach is more suitable for multicast and broadcast scenarios, saving significant computing resources.

[0110] The above description is merely an embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any minor modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A mask-based multi-rate deep image compression system for mixed contexts, characterized by: The system includes a depth image encoding and decoding module, a latent vector importance analysis module and a latent space encoding module; The deep image encoding and decoding module includes a depth encoder, a quantizer, an entropy encoder, an entropy decoder and a depth decoder; wherein the depth encoder and the depth decoder are composed of a deep convolutional neural network that has been trained and converged, and the parameters of the convolutional neural network are trained in an end-to-end manner; The depth encoder is used to input the original image and generate a latent space potential vector; The quantizer is used to input the to-be-sent vector output by the latent space coding module and perform scalar quantization processing on each element of the to-be-sent vector; The entropy encoder is used to input the quantized latent vector and perform entropy encoding on the quantized latent vector to generate a final transmitted code stream, which is sent to the entropy decoder via a channel; The entropy decoder is used to receive a code stream, perform entropy decoding on the code stream, and reconstruct a latent vector; The depth decoder is used to input the potential vector reconstructed by the entropy decoder, perform image reconstruction, and output a reconstructed image of the original image; The latent vector importance analysis module includes a semantic importance analysis unit, a reconstruction importance analysis unit and an importance map fusion unit; The semantic importance analysis unit is used to input the original image, analyze the importance of each pixel of the original image to the image classification result, and map it to each plane position of the latent space potential vector to generate a semantic importance map of the latent space potential vector; A reconstruction importance analysis unit, configured to input a latent vector, analyze the importance of each plane position of the latent vector to image reconstruction, and generate a reconstruction importance map of the latent vector in the latent space; An importance map fusion unit, configured to input a semantic importance map of a latent vector and a reconstruction importance map to generate an importance map of a latent vector in a mixed context; The latent space encoding module includes a latent space mask generation unit and an encoding unit; A latent space mask generation unit is configured to input the target bit rate and the latent vector importance map fed back by the channel, calculate the maximum number of elements of the latent vector allowed to be transmitted under the current channel resources, calculate the threshold for binarizing the latent vector importance map, and binarize the importance map into a latent space mask vector; An encoding unit, configured to input a latent vector and a latent space mask vector, select elements from the latent vector according to the latent space mask vector, and concatenate the selected elements with the mask to form a vector to be sent; The depth encoder and depth decoder are iteratively trained in an end-to-end manner, and the objective function of the training process is: （1） Among them, E, D represent the depth encoder and decoder, is the bit rate of the latent vector, is the semantic feature matching loss term, is the reconstruction loss item; Among them, x and y are the original image and the reconstructed image, (H, W, 3) are the dimensions of x and y, is the semantic feature extraction function, are the dimensions of the semantic feature vectors.

2. The mask-based multi-rate deep image compression system for mixed contexts according to claim 1, characterized in that: The process of iterative training of the deep encoder and deep decoder network models is as follows: Step S1: Sample an input image x from the training set and generate a latent vector z through a deep encoder. Calculate the variance of each plane position of the vector z along the channel direction to obtain the variance vector latent vector v. Use the relative bit rate R(z) to estimate the bit rate of the compressed vector. The bit rate loss term is calculated as follows: （4） Step S2: Generate a random binary mask vector M of the same dimension as the vector v, and transform the potential vector z according to M according to formula (5) to obtain the vector , for the vector Add noise to simulate the quantization process, according to the vector after noise addition And the mask vector M is used to obtain the reconstructed image y. The reconstruction loss and semantic feature matching loss are calculated as shown in equations (2) and (3); (5) in, for Elements, is the element of z, is an element of the mask vector M, and C is one of the dimensions of the latent space potential vector; Step S3, the final loss term is as follows: (6) Step S4: solve the loss term for each parameter of the depth encoder and depth decoder The Adam optimizer is used to update the parameters.

3. The mask-based multi-rate deep image compression system for mixed contexts according to claim 1, characterized in that: The semantic importance analysis unit uses a classifier based on a deep convolutional neural network to classify the input image, and uses a gradient-based class activation mapping method to calculate the importance of each pixel to the classification result. Then, based on the spatial invariance of the convolutional neural network, the pixel importance is mapped to the latent space to obtain a latent vector semantic importance map.

4. The mask-based multi-rate deep image compression system for mixed contexts according to claim 1, characterized in that: The reconstruction importance analysis unit calculates the variance of all elements at each plane position of the potential vector. The larger the variance, the greater the entropy of the position, and the more important all elements at the position are to the reconstruction. The variance value of each position is normalized to obtain a reconstruction importance map.

5. A mask-based multi-rate deep image compression method for mixed contexts, characterized by: The method is implemented by the system according to any one of claims 1 to 4, and the method comprises the following steps: Step 1: Pass the original image x through the deep encoder to generate the latent vector z; Step 2: Input the original image x and the latent vector z into the latent vector importance analysis module to generate a latent vector importance map I; Step 3: The latent space encoding module calculates the latent vector importance map I and the target bit rate according to the latent vector importance analysis module output. r Generate a latent space mask vector M, transforming the potential vector z into a vector , the vector Splice with the mask vector M to generate the vector to be sent ; Step 4: The quantizer will transmit the vector After quantization, the entropy encoder uses an adaptive arithmetic coding method to quantize the vector Perform entropy coding to generate a bit stream, and send the bit stream to the entropy decoder through the channel; Step 5: The entropy decoder uses an adaptive arithmetic decoding algorithm to perform entropy decoding on the received code stream and reconstruct the potential vector ; Step 6: The deep decoder reconstructs the latent vector Generate the reconstructed original image y.

6. The mask-based multi-rate deep image compression method in a mixed context according to claim 5, characterized in that: The step 2 is specifically as follows: Step 2-1: The original image x is input into the semantic importance analysis unit to perform latent vector semantic importance analysis to obtain a latent space latent vector semantic importance map, denoted as vector e. An adaptive binarization algorithm is used to binarize vector e into vector B. Step 2-2: Input the latent vector z into the importance analysis unit, calculate the variance vector v of the latent vector composed of the variance of the C element values ​​at each plane position of the latent vector z, and use v as the reconstructed importance map; Step 2-3: The importance map fusion unit generates a latent vector importance map I according to the following formula: （7）。 7. The mask-based multi-rate deep image compression method in a mixed context according to claim 5, characterized in that: The step 3 is specifically as follows: Step 3-1: The latent space mask generation unit generates the target bit rate r And the importance map I, and take the Sth largest number in vector I as the binarization threshold T , binarize I into a mask M according to T, where S is the number of elements allowed to be sent at the target bit rate divided by the number of channels and rounded up; Step 3-2: The encoding unit transforms the potential vector z according to the following formula to generate the vector and form the to-be-sent vector W with the mask vector M; (5) in, for Elements, is the element of z, is an element of the mask vector M, and C is one of the dimensions of the latent space potential vector.

Images