A generative image compression method based on vector quantization

Abstract

The application provides a generative image compression method based on vector quantization, and belongs to the technical field of image and video compression. The method comprises the following steps: obtaining a continuous latent representation of an input image through an analysis transformation module; then performing vector quantization processing to obtain discrete indexes of the continuous latent representation and corresponding quantized features; constructing a continuous index probability distribution; predicting a conditional probability distribution of the discrete indexes through a conditional autoregressive entropy model; calculating a coding rate based on the continuous index probability distribution and the conditional probability distribution; reconstructing an image based on the quantized features and constructing a distortion loss; constructing a rate-distortion loss function and jointly training an image compression model comprising the analysis transformation module, the vector quantization module, the conditional autoregressive entropy model and a synthesis transformation module to obtain a trained image compression model; and compressing the input image by using the trained image compression model. The application can realize rate-distortion joint optimization and collaborative improvement of compression efficiency and reconstruction quality.

Description

Technical Field

[0001] This invention belongs to the field of image and video compression technology, and particularly relates to a generative image compression method based on vector quantization. Background Technology

[0002] With the continuous growth of digital media data, achieving high compression efficiency and high reconstruction quality in image compression methods under the constraints of limited storage resources and transmission bandwidth has become an important research direction in this field. Existing image compression standards (such as BPG and VVC) are mainly based on artificially designed signal processing frameworks, and their rate-distortion performance has certain limitations under different data distributions and compression ratios.

[0003] In recent years, deep learning-based image compression methods have achieved some improvements in compression performance by jointly modeling bit rate and distortion through end-to-end optimization. Furthermore, generative image compression methods, by introducing prior models, can improve the perceptual quality of reconstructed images under low bit rate conditions.

[0004] Existing learning-based compression methods typically employ a joint optimization framework that includes an encoder, a quantization module, and an entropy model. The coding rate is estimated using the cross-entropy between the latent representation distribution and the predicted distribution of the entropy model. However, in scalar quantization methods, since the quantization operation is non-differentiable, additive noise or pass-through estimation methods are usually used for approximation. These methods struggle to effectively model the correlations between features and are prone to structural information loss at low bit rates.

[0005] In contrast, vector quantization methods, which jointly represent features through codebook mapping, have a potential advantage in preserving structural information. However, in existing technologies, the discrete index generation operation involved in vector quantization is non-differentiable, making it difficult to effectively constrain the rate-related loss of the encoder, thus resulting in insufficient rate-distortion optimization.

[0006] Therefore, how to implement a differentiable rate modeling mechanism within the vector quantization framework to support joint optimization of the encoder and entropy model has become a pressing technical problem in this field. Summary of the Invention

[0007] The purpose of this invention is to provide a generative image compression method based on vector quantization, so as to solve the technical problems of non-differentiable bitrate modeling and difficulty in achieving joint rate-distortion optimization under the vector quantization compression framework in the prior art.

[0008] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0009] A generative image compression method based on vector quantization, the method comprising the following steps:

[0010] Step S1: Input the input image into the analysis and transformation module for downsampling, nonlinear transformation, and merging operations to obtain a continuous latent representation;

[0011] Step S2: The continuous latent representation is processed by vector quantization through the vector quantization module to obtain the discrete index and corresponding quantization features of the continuous latent representation;

[0012] Step S3: Construct a continuous index probability distribution;

[0013] Step S4: Predict the conditional probability distribution of the discrete index using a conditional autoregressive entropy model;

[0014] Step S5: Calculate the coding rate based on the continuous index probability distribution and the conditional probability distribution;

[0015] Step S6: Reconstruct the image based on quantization features and construct the distortion loss;

[0016] Step S7: Construct the rate-distortion loss function and jointly train the image compression model, which includes the analysis transformation module, vector quantization module, conditional autoregressive entropy model, and synthesis transformation module, to obtain the trained image compression model;

[0017] Step S8: Compress the input image using the trained image compression model.

[0018] Further, step S1 includes the following steps:

[0019] Step S11: The input image is processed by multi-level downsampling units to obtain feature representations at multiple different scales;

[0020] Step S12: Multiple feature representations at different scales are nonlinearly transformed by the feature extraction unit at their corresponding resolutions, and then merged to obtain a continuous latent representation.

[0021] Further, step S2 includes the following steps:

[0022] Step S21: Construct a codebook vector set;

[0023] Step S22: Calculate any position in the continuous latent representation corresponding feature vector The distance to each codebook vector in the codebook vector set is used to obtain the distance tensor;

[0024] Step S23: Determine any position in the continuous latent representation based on the minimum distance criterion. Codebook Index This leads to the discrete index of the continuous latent representation. ;

[0025] Step S24: Construct quantization features based on the codebook index of the continuous latent representation.

[0026] Further, step S3 includes the following steps:

[0027] Step S31: Calculate any position in the continuous latent representation corresponding feature vector Distance to each codebook vector in the codebook vector set ;

[0028] Step S32: Generate the corresponding similarity based on the distance between the continuous latent representation feature vector and the codebook vector;

[0029] Step S33: Normalize the similarity and construct a continuous index probability distribution.

[0030] Further, step S4 includes the following steps:

[0031] Step S41: Construct a global order relationship for the quantized features to obtain a global order vector;

[0032] Step S42: Construct the causal constraint matrix from the global order vector;

[0033] Step S43: The conditional autoregressive entropy model performs conditional probability prediction based on the causal constraint matrix.

[0034] Further, step S41 includes the following steps:

[0035] Step S411: Sort the spatial locations within each scale to obtain the scale order;

[0036] Step S412: Determine the order between scales based on the hierarchical relationship between different scales;

[0037] Step S413: Perform a unified mapping between the intra-scale order and the inter-scale order to obtain the global order vector.

[0038] Further, step S6 includes the following steps:

[0039] Step S61: Input the quantized features into the synthesis transformation module for decoding, restore the spatial structure and reconstruct the image;

[0040] Step S62: Based on the input image With reconstructed images Calculate the distortion loss.

[0041] Furthermore, the rate-distortion loss function in step S7 is expressed as follows:

[0042]

[0043] in, For rate-distortion loss function; Indicates the input image. Represents the reconstructed image; This parameter represents the trade-off between coding rate and reconstruction quality. Indicates the coding rate; It indicates distortion and loss.

[0044] Furthermore, the image compression model is divided into an encoder and a decoder to achieve image compression and reconstruction. The process of compressing an input image using a trained image compression model includes the following steps:

[0045] Encoding end processing: Input image By analyzing the transformation module, a continuous latent representation is obtained. The continuous latent representation is vector-quantized using a vector quantization module to obtain a discrete index. Predict the conditional probability distribution of discrete indices based on the conditional autoregressive entropy model. Based on the conditional probability distribution, the discrete index is entropy encoded using an arithmetic encoder to generate a compressed bitstream.

[0046] Decoding end processing: The received bitstream is entropy-decoded using an arithmetic decoder to recover the discrete index under the constraints of the conditional autoregressive entropy model. Based on the discrete index, the corresponding quantization feature is searched from the codebook. The quantized features are input into the synthesis and transformation module to obtain the reconstructed image. .

[0047] Furthermore, to achieve flexible adjustment of the coding rate, at the encoding end, only a portion of the prefixes in the discrete index sequence are entropy encoded and transmitted; for the untransmitted portion, the decoding end performs position-by-position prediction of the discrete index under causal constraints based on the conditional autoregressive entropy model to complete the index completion; image reconstruction is performed based on the completed discrete index sequence; by adjusting the length of the portion of the prefixes used for entropy encoding and transmission in the discrete index sequence, continuous adjustment between the coding rate and reconstruction quality can be achieved without retraining the model.

[0048] Compared with the prior art, the present invention has the following beneficial technical effects:

[0049] 1) This invention introduces a soft-softening mechanism, enabling rate-related losses to participate in network training, thus achieving joint optimization of rate-distortion under vector quantization; under low bit rate conditions, quantization with the help of a high-quality learnable codebook helps to improve reconstruction quality by preserving image structural information; thus achieving a synergistic improvement in compression efficiency and reconstruction quality.

[0050] 2) By introducing a differentiable probability modeling mechanism and a structured dependency modeling strategy, this invention effectively solves the problems of non-differentiability of traditional vector quantization and insufficient context modeling. While ensuring the trainability of the model, it improves the accuracy of probability modeling and compression performance.

[0051] 3) This invention does not require complex extensions to the model structure, resulting in good implementation efficiency; it supports bitrate adjustment within a certain range, improving system flexibility. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the overall process of the generative image compression method based on vector quantization of the present invention.

[0054] Figure 2 This is a schematic diagram of the differentiable vector quantization process of the present invention.

[0055] Figure 3 This is a schematic diagram of the sorting vector and attention mask generation process of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention proposes a generative image compression method based on vector quantization, such as... Figure 1 As shown, the method includes the following steps:

[0058] Step S1: Input the input image into the analysis and transformation module for downsampling, nonlinear transformation and merging operations to obtain a continuous latent representation.

[0059] Specifically, the input image is represented as ;in, Indicates the height of the input image; Indicates the width of the input image; This refers to the number of input image channels. In this invention, the number of input image channels... .

[0060] The analysis and transformation module is implemented by a multi-layer neural network including multi-level downsampling units and feature extraction units. The input image is reduced in spatial resolution by multi-level downsampling units, and then nonlinearly transformed by feature extraction units to map the input image to a low-dimensional continuous feature space. After merging, a continuous latent representation is obtained.

[0061] Step S11: The input image is processed by multi-level downsampling units to obtain feature representations at multiple different scales.

[0062] The first-scale feature is obtained by downsampling by 8 times; the second-scale feature is obtained by downsampling by 16 times; and the third-scale feature is obtained by downsampling by 32 times. The number of downsampling scales is not limited to the above three, and this invention does not impose any limitation on this.

[0063] Step S12: Multiple feature representations at different scales are nonlinearly transformed by the feature extraction unit at their corresponding resolutions, and then merged to obtain a continuous latent representation.

[0064] Nonlinear transformation can acquire structural and semantic information under different receptive fields. The nonlinear transformation includes the following steps: expanding the feature representation at each scale in spatial dimensions, converting it from a two-dimensional spatial structure to a one-dimensional sequence; the expansion operation is a process of rearranging the dimensions of the feature tensor, only changing the arrangement of the data without introducing additional learnable parameters; concatenating or rearranging the one-dimensional feature sequences at different scales according to a preset method, i.e., merging, to obtain a unified sequence representation; thus obtaining a continuous latent representation. The operation of the analysis and transformation module is represented as follows:

[0065]

[0066] in, Represents a continuous potential representation. ; Indicates batch size, This indicates the length of the sequence after spatial location is expanded. Indicates feature dimension; This indicates the analysis and transformation module.

[0067] Therefore, the continuous latent representation It integrates image feature information at different scales, providing a multi-scale contextual basis for subsequent quantization and probabilistic modeling.

[0068] Step S2: The continuous latent representation is processed by vector quantization through the vector quantization module to obtain the discrete index and corresponding quantization features of the continuous latent representation.

[0069] like Figure 2 As shown, step S2 includes the following steps:

[0070] Step S21: Construct a codebook vector set, represented as follows:

[0071]

[0072] in, Represents a set of codebook vectors; Indicates the codebook size; Indicates the first Each codebook vector.

[0073] Step S22: Calculate any position in the continuous latent representation corresponding feature vector The distance to each codebook vector in the codebook vector set is used to obtain the distance tensor.

[0074] Specifically, any position in a continuous latent representation corresponding feature vector The distance to each codebook vector in the codebook vector set is calculated as follows:

[0075]

[0076] in, Represents the location of consecutive potential representations corresponding feature vector With the The distance between codebook vectors; This represents the square of the Euclidean distance.

[0077] The distance tensor is obtained by calculating the distance between the feature vector corresponding to each position in the continuous latent representation and each codebook vector in the codebook vector set. .

[0078] Step S23: Determine any position in the continuous latent representation based on the minimum distance criterion. Codebook Index This leads to the discrete index of the continuous latent representation. .

[0079] Any position in a continuous latent representation Codebook Index Determined in the following manner:

[0080]

[0081] in, Represents the location of consecutive potential representations The codebook index; This indicates that the positions of consecutive potential representations are... corresponding feature vector With the Distance between codebook vectors The smallest codebook vector index.

[0082] By finding the codebook index with the minimum distance to each position in a continuous latent representation, the discrete index tensor can be obtained. .

[0083] Step S24: Construct quantization features based on the codebook index of the continuous latent representation.

[0084] Specifically, for any position in a continuous latent representation Its corresponding codebook index is The quantization feature at that position is obtained by looking up a table in the codebook vector set, and is represented as:

[0085]

[0086] in, Represents the location of consecutive potential representations Quantized feature vectors; The index in the codebook vector set is The codebook vector.

[0087] By finding the quantization features at each location, the complete quantization feature tensor can be obtained. .

[0088] Through the above processing, the process of transforming continuous latent representations is realized. To discrete index and quantitative features The mapping.

[0089] The above process is essentially as follows: using the discrete index as the lookup address, the corresponding codebook vector is selected from the pre-constructed codebook vector set to replace the original continuous feature, thereby realizing the discretized representation of the feature.

[0090] Step S3: Construct a continuous index probability distribution.

[0091] In step S2, the discrete index is obtained through minimum distance matching. This process is a discontinuous operation, such that the discrete index is obtained with respect to the continuous latent representation. The discrete index is not differentiable. However, in the subsequent coding rate modeling process (corresponding to step S5), cross-entropy calculation is required based on the probability distribution of the discrete index to estimate the number of bits required for coding. Since the discrete index is not differentiable, the coding rate cannot generate effective gradient constraints on the continuous latent representation, making it difficult to achieve rate-distortion joint optimization among the encoder, vector quantization, and entropy model. Therefore, it is necessary to approximate the discrete index by constructing a continuous probability distribution to support differentiable modeling of the coding rate.

[0092] Step S31: Calculate any position in the continuous latent representation corresponding feature vector Distance to each codebook vector in the codebook vector set .

[0093] Step S32: Generate the corresponding similarity based on the distance between the continuous latent representation feature vector and the codebook vector.

[0094] Specifically, for continuous potential representation positions corresponding feature vector With the The distance between each codebook vector is negatively evaluated to generate the corresponding similarity, thus realizing the transformation from "distance metric" to "similarity metric", as shown below:

[0095]

[0096] in, Represents the location of consecutive potential representations corresponding feature vector With the The similarity between the codebook vectors. The similarity is used to reflect the degree of matching between the feature vectors of the continuous latent representation and each codebook vector.

[0097] Step S33: Normalize the similarity and construct a continuous index probability distribution.

[0098] Specifically, the use of similarity The function is normalized to construct a continuous index probability distribution, as shown below:

[0099]

[0100] in, Represents the location of consecutive potential representations corresponding feature vector With the The probability distribution of continuous indices of codebook vectors; express function; This represents a temperature parameter used to control the smoothness of the probability distribution of consecutive indices. Represents the location of consecutive potential representations corresponding feature vector With the The similarity of codebook vectors; Indicates the codebook size.

[0101] The continuous index probability distribution of the continuous latent representation is obtained by calculating the continuous index probability distribution of the feature vector corresponding to each position in the continuous latent representation and the continuous index probability distribution of each codebook vector in the codebook vector set. .

[0102] Furthermore, when the temperature parameter When the temperature parameter is small, the probability distribution of continuous indices approaches the one-heat distribution; when the temperature parameter is small... When the value is large, the probability distribution of consecutive indices tends to be a smooth distribution.

[0103] Step S4: Predict the conditional probability distribution of the discrete index using a conditional autoregressive entropy model.

[0104] The continuous index probability distribution has been constructed in step S3. To perform probabilistic modeling of discrete indices, this invention constructs a conditional autoregressive entropy model for quantified features. Perform conditional probability prediction to obtain the conditional probability distribution of the discrete index. As a probability distribution of continuous indices The prediction.

[0105] The conditional autoregressive entropy model quantifies features. As input, its core lies in: by constructing sequential relationships and generating causal constraints accordingly, the direction of information dependence between different positions in the probabilistic modeling process can be controlled.

[0106] Step S41: Construct a global order relationship for the quantized features to obtain a global order vector.

[0107] For length of Quantitative characteristics This constructs a global order relationship to describe the encoding sequence at each position. Specifically:

[0108] like Figure 3 As shown in the middle left section, the construction of global order relations includes the following steps:

[0109] Step S411: Sort the spatial locations within each scale to obtain the scale order.

[0110] Step S412: Determine the order between scales based on the hierarchical relationship between different scales.

[0111] Step S413: Perform a unified mapping between the intra-scale order and the inter-scale order to obtain a global order vector, represented as follows: .

[0112] Step S42: Construct the causal constraint matrix from the global order vector.

[0113] like Figure 3 As shown in the right half, based on the global order vector Construct information dependency constraints between locations. Define the causal constraint matrix:

[0114]

[0115] Causal constraint matrix The elements are:

[0116]

[0117] in, Represents the global order vector. The encoding order of each position; Represents the global order vector. The encoding order of each position; the larger the encoding order, the later the position is in the encoding process. : indicates that in the prediction of the first When the position is specified, the first position is allowed. Information about each location; : indicates that in the prediction of the first When in position 1, the use of position 2 is not allowed. Information about each location.

[0118] The above construction clearly defines that any position depends only on the position preceding it in the global order.

[0119] Step S43: The conditional autoregressive entropy model performs conditional probability prediction based on the causal constraint matrix.

[0120] The causal constraint matrix In conditional autoregressive entropy models, this is used to limit the scope of information interaction between features. When performing feature interaction or context modeling, only features that satisfy the conditional autoregressive entropy model are allowed to interact. The location participates in the information calculation of the current location, thereby ensuring that the model's dependencies conform to the global order relationship. Input quantized features The order of the elements remains unchanged, and the information flow is restricted by the aforementioned causal constraint matrix, thereby achieving causal modeling.

[0121] Under the aforementioned causal constraints, the conditional autoregressive entropy model employs a sequence modeling network to predict the positional conditional probabilities of discrete indices, thereby obtaining the conditional probability distribution. The conditional autoregressive entropy model makes predictions position by position in a global order.

[0122] Under the aforementioned causal constraints, a sequence modeling network is used to predict the conditional probability of discrete indices. For discrete indices... any position Its conditional probability distribution is expressed as:

[0123]

[0124] in: Represents discrete index Location The feature belongs to the first The probability of each codebook vector; This represents the accessible context information determined by the causal constraint matrix; , The number of codebook vectors; This represents a conditional probability estimate.

[0125] The prediction results for all locations are summarized to obtain the conditional probability distribution tensor:

[0126]

[0127] The sequence modeling network is a neural network that performs probabilistic modeling based on contextual information. In one embodiment, the sequence modeling network may employ an attention-based structure.

[0128] Furthermore, although the above probability predictions can be computationally parallelized using masking techniques in the attention mechanism, due to the limitations of the causal constraint matrix, they are equivalent to an autoregressive modeling process performed position-by-position according to the global order.

[0129] Step S5: Calculate the coding rate based on the continuous index probability distribution and the conditional probability distribution.

[0130] The continuous index probability distribution constructed in step S3 and the conditional probability distribution predicted in step S4 These are used together for coding rate modeling. Unlike coding rate modeling methods based on discrete index statistics, this invention constructs a continuous index probability distribution, making the coding rate differentiable with respect to the continuous latent representation.

[0131] The coding rate is calculated based on the continuous index probability distribution and the conditional probability distribution. The coding rate is calculated as follows:

[0132]

[0133] in, Indicates the coding rate, used to measure the bit overhead required to transmit discrete indices; It represents the mathematical expectation.

[0134] Through continuous index probability distribution Used to participate in coding rate calculation, thereby making the coding rate relative to the continuous latent representation. Differentiability, specifically manifested as:

[0135]

[0136] in, Indicates the coding rate.

[0137] It should be noted that the continuous index probability distribution is only used for code rate modeling during the training phase; during the inference phase, the discrete index from step S2 is still used for actual encoding and reconstruction. Therefore, this embodiment achieves continuous optimization during the training phase without changing the actual encoding process.

[0138] The above continuous index probability distribution processing establishes a differentiable relationship between the coding rate and the continuous latent representation, enabling the coding rate to effectively constrain the analysis transformation; it realizes end-to-end rate-distortion joint optimization of the analysis transformation module, vector quantization module, conditional autoregressive entropy model, and synthesis transformation module; under low bit rate conditions, it is beneficial to maintain the structural information of latent features and improve the quality of reconstructed images.

[0139] The conditional probability distribution Used as an entropy coding model to model the probability distribution of discrete indices; the probability distribution of continuous indices As an index approximate discrete distribution, it is obtained from the continuous latent representation through a differentiable mapping and is used to construct a differentiable coding rate estimate, thereby establishing a differentiable relationship between the coding rate and the continuous latent representation, and realizing end-to-end optimization.

[0140] During the training phase, discrete indexes are not... Perform the actual encoding and decoding operations, the coding rate For encoding The estimated rate overhead is used to optimize the coding rate through gradient descent.

[0141] During the inference phase, it is necessary to... The actual encoding is a bitstream transmitted to the decoding end, where it is reconstructed from the bitstream. Firstly, at the encoding end, based on conditional probability distribution... Using an arithmetic encoder (AE) to convert discrete indexes Encode into a bitstream; at the decoding end, predict the conditional probability distribution position by position according to the order in the global order vector constructed in S41. And in conjunction with an arithmetic decoder (AD), discrete indices are recovered from the bitstream. .

[0142] Step S6: Reconstruct the image based on quantization features and construct the distortion loss.

[0143] Step S61: Input the quantized features into the synthesis transformation module for decoding, restore the spatial structure and reconstruct the image.

[0144] The synthesis transformation module is implemented by a multi-layer neural network, including a sequence rearrangement unit, a multi-level upsampling unit, and an image reconstruction unit, which is used to gradually restore the quantized features in the form of a one-dimensional sequence into a two-dimensional spatial structure.

[0145] First, the quantized features are inversely rearranged to restore the sequence representation to two-dimensional features corresponding to multiple scales. Then, spatial resolution is gradually restored through multi-level upsampling, and features at different scales are fused. Finally, the fused features are mapped to pixel space to obtain the reconstructed image.

[0146]

[0147] in, Indicates the reconstructed image. , This indicates the synthesis and transformation module.

[0148] The synthesis transformation module and the analysis transformation module are structurally dual.

[0149] Step S62: Based on the input image With reconstructed images Calculate distortion loss .

[0150] Distortion loss includes pixel-level reconstruction error, perceptual loss, and adversarial loss, as shown below:

[0151]

[0152] in, Distortion loss is used to measure the difference between the reconstructed image and the original image; Indicates pixel-level reconstruction error. ; Indicates perceived loss; Indicates resistance to loss; , , , where represents the weighting ratio of pixel-level reconstruction error, perceptual loss, and adversarial loss, respectively.

[0153] Step S7: Construct the rate-distortion loss function and jointly train the image compression model, which includes the analysis transformation module, vector quantization module, conditional autoregressive entropy model, and synthesis transformation module, to obtain the trained image compression model.

[0154] The coding rate constructed in step S5 and the distortion loss constructed in step S6 A rate-distortion loss function is constructed to achieve a dynamic trade-off between model compression ratio and reconstruction quality. The rate-distortion loss function is expressed as follows:

[0155]

[0156] in, Let be the rate-distortion loss function, and represent the rate-distortion joint optimization objective term; Indicates the input image. This represents the reconstructed image obtained through synthetic transformation. This parameter represents the trade-off between coding rate and reconstruction quality. Indicates the coding rate; It indicates distortion and loss.

[0157] The model includes all learnable parameters from the analysis transformation module, vector quantization module, synthesis transformation module, and conditional autoregressive entropy model described in S1-S4. Based on the rate-distortion loss function, all model parameters can be optimized and updated through backpropagation, enabling the model to balance compression ratio and reconstruction quality.

[0158] Step S8: Compress the input image using the trained image compression model.

[0159] After completing the training in step S7, an image compression model is obtained, which includes an analysis transformation module, a vector quantization module, a conditional autoregressive entropy model, and a synthesis transformation module.

[0160] During the inference phase, the trained image compression model is used to compress the input image. The image compression model is divided into an encoder and a decoder to achieve image compression and reconstruction, specifically including the following processes:

[0161] Encoding end processing: Input image By analyzing the transformation module, a continuous latent representation is obtained. The continuous latent representation is vector-quantized using a vector quantization module to obtain a discrete index. Predict the conditional probability distribution of discrete indices based on the conditional autoregressive entropy model. According to the conditional probability distribution, the discrete index is entropy encoded using an arithmetic encoder (AE) to generate a compressed bitstream, which can be used for storage or transmission.

[0162] Decoding end processing: The received bitstream is entropy decoded using an arithmetic decoder (AD) to recover the discrete index under the constraints of the conditional autoregressive entropy model. Based on the discrete index, the corresponding quantization feature is searched from the codebook. The quantized features are input into the synthesis and transformation module to obtain the reconstructed image. .

[0163] In one embodiment, to achieve flexible adjustment of the coding rate, at the encoding end, only a portion of the prefixes in the discrete index sequence are entropy encoded and transmitted; for the untransmitted portion, the decoding end performs position-by-position prediction of the discrete index under causal constraints based on the conditional autoregressive entropy model to complete the index completion; and image reconstruction is performed based on the completed discrete index sequence.

[0164] By adjusting the length of the partial prefix used for entropy coding and transmission in the discrete index sequence, continuous adjustment between coding rate and reconstruction quality can be achieved without retraining the model.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A generative image compression method based on vector quantization, characterized in that, The method includes the following steps: Step S1: Input the input image into the analysis and transformation module for downsampling, nonlinear transformation, and merging operations to obtain a continuous latent representation; Step S2: The continuous latent representation is processed by vector quantization through the vector quantization module to obtain the discrete index and corresponding quantization features of the continuous latent representation; Step S3: Construct a continuous index probability distribution; Step S4: Predict the conditional probability distribution of the discrete index using a conditional autoregressive entropy model; Step S5: Calculate the coding rate based on the continuous index probability distribution and the conditional probability distribution; Step S6: Reconstruct the image based on quantization features and construct the distortion loss; Step S7: Construct the rate-distortion loss function and jointly train the image compression model, which includes the analysis transformation module, vector quantization module, conditional autoregressive entropy model, and synthesis transformation module, to obtain the trained image compression model; Step S8: Compress the input image using the trained image compression model.

2. The generative image compression method based on vector quantization according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: The input image is processed by multi-level downsampling units to obtain feature representations at multiple different scales; Step S12: Multiple feature representations at different scales are nonlinearly transformed by the feature extraction unit at their corresponding resolutions, and then merged to obtain a continuous latent representation.

3. The generative image compression method based on vector quantization according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Construct a codebook vector set; Step S22: Calculate any position in the continuous latent representation corresponding feature vector The distance to each codebook vector in the codebook vector set is used to obtain the distance tensor; Step S23: Determine any position in the continuous latent representation based on the minimum distance criterion. Codebook Index This leads to the discrete index of the continuous latent representation. ; Step S24: Construct quantization features based on the codebook index of the continuous latent representation.

4. The generative image compression method based on vector quantization according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Calculate any position in the continuous latent representation corresponding feature vector Distance to each codebook vector in the codebook vector set ; Step S32: Generate the corresponding similarity based on the distance between the continuous latent representation feature vector and the codebook vector; Step S33: Normalize the similarity and construct a continuous index probability distribution.

5. The generative image compression method based on vector quantization according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Construct a global order relationship for the quantized features to obtain a global order vector; Step S42: Construct the causal constraint matrix from the global order vector; Step S43: The conditional autoregressive entropy model performs conditional probability prediction based on the causal constraint matrix.

6. The generative image compression method based on vector quantization according to claim 5, characterized in that, Step S41 includes the following steps: Step S411: Sort the spatial locations within each scale to obtain the scale order; Step S412: Determine the order between scales based on the hierarchical relationship between different scales; Step S413: Perform a unified mapping between the intra-scale order and the inter-scale order to obtain the global order vector.

7. The generative image compression method based on vector quantization according to claim 1, characterized in that, Step S6 includes the following steps: Step S61: Input the quantized features into the synthesis transformation module for decoding, restore the spatial structure and reconstruct the image; Step S62: Based on the input image With reconstructed images Calculate the distortion loss.

8. The generative image compression method based on vector quantization according to claim 1, characterized in that, The rate-distortion loss function in step S7 is expressed as follows: in, For rate-distortion loss function; Indicates the input image. Represents the reconstructed image; This represents a parameter that balances coding rate and reconstruction quality. Indicates the coding rate; It indicates distortion and loss.

9. The generative image compression method based on vector quantization according to claim 1, characterized in that, Image compression models are divided into encoder and decoder ends to achieve image compression and reconstruction. The process of compressing an input image using a trained image compression model includes the following steps: Encoding end processing: Input image By analyzing the transformation module, a continuous latent representation is obtained. ; The continuous latent representation is vector-quantized using a vector quantization module to obtain a discrete index. ; Predict the conditional probability distribution of discrete indices based on the conditional autoregressive entropy model. Based on the conditional probability distribution, the discrete index is entropy encoded using an arithmetic encoder to generate a compressed bitstream. Decoding end processing: The received bitstream is entropy-decoded using an arithmetic decoder to recover the discrete index under the constraints of the conditional autoregressive entropy model. ; Based on the discrete index, the corresponding quantization feature is searched from the codebook. The quantized features are input into the synthesis and transformation module to obtain the reconstructed image. .

10. The generative image compression method based on vector quantization according to claim 1, characterized in that, To achieve flexible adjustment of the coding rate, at the encoding end, only a portion of the prefixes in the discrete index sequence are entropy encoded and transmitted; for the untransmitted portion, the decoding end performs position-by-position prediction of the discrete index under causal constraints based on the conditional autoregressive entropy model to complete the index completion; image reconstruction is performed based on the completed discrete index sequence; by adjusting the length of the portion of the prefixes used for entropy encoding and transmission in the discrete index sequence, continuous adjustment between the coding rate and reconstruction quality can be achieved without retraining the model.

Images