Image encoding, decoding and compression method and system based on vector quantization entropy modeling
By using a lattice vector quantizer and relaxed boundary entropy modeling, the problem of training instability in image compression is solved, achieving more efficient image compression and improving rate-distortion performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2024-08-15
- Publication Date
- 2026-04-21
AI Technical Summary
In existing deep learning-based image compression methods, vector quantizers suffer from difficulties in convergence during training and inconsistencies between training and testing, and their high complexity leads to poor rate-distortion performance.
A lattice vector quantizer is used to quantize the features, and entropy modeling is performed on the quantized features through relaxed boundaries to reduce complexity, solve the problem of inconsistency between training and testing, and improve rate distortion performance.
Under the premise of the same reconstructed image quality, compared with scalar quantization methods and the latest vector quantization methods, it saves 6.03% and 2.48% of bitstream overhead, respectively, improves rate-distortion performance, and achieves efficient and stable image compression.
Smart Images
Figure CN118984392B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing, specifically relating to an image encoding, decoding, and compression method and system based on vector quantization entropy modeling in a deep learning-based image compression method. Background Technology
[0002] In recent years, deep learning-based image compression methods have flourished. Deep learning-based image compression is mainly implemented through transform coding, which consists of three parts: transform, quantization, and entropy coding, all optimized end-to-end. Specifically, the image to be encoded first obtains three-dimensional features through a nonlinear deep neural network, these features are then quantized, and subsequently, an entropy model is used to estimate the distribution of the quantized features and entropy-encode them. Because quantization is incompatible with gradient backpropagation during image compression model training, quantization has been less studied in deep learning-based image compression. However, it is clear that efficient quantization methods can improve the rate-distortion performance of image compression.
[0003] Scalar quantizers are widely used in deep learning-based image compression. The assumption behind their use is that powerful nonlinear transformations can accommodate cumbersome scalar quantization. A literature review revealed that Ballé et al. first used scalar quantizers in their paper "End-to-end Optimized Image Compression" presented at the 2017 International Conference on Learning Representation. However, this assumption is flawed because scalar quantizers have been shown to be suboptimal even for sources with independent dimensions. In contrast, vector quantizers offer a more efficient way to partition the space. However, very few existing techniques consider using vector quantizers in deep learning-based image compression.
[0004] In their 2017 Conference and Workshop on Neural Information Processing Systems, Agustsson et al. proposed a soft-to-hard vector quantization method. However, this method can lead to inconsistencies between training and testing, as well as training instability, resulting in lower rate-distortion performance compared to scalar quantizers. In their 2022 IEEE Conference on Computer Vision and Pattern Recognition, Zhu et al. employed a soft-to-hard cascaded vector quantization method to improve rate-distortion performance in their paper "Unified Multivariate Gaussian Mixture for Efficient Neural Image Compression." However, this method is more complex and suffers from codebook collapse due to the co-training of the codebook and model. In their paper "VQAC: Lattice Vector Quantization Coupled With Spatially Adaptive Companding for Efficient Learned Image Compression" presented at the 2023 IEEE Conference on Computer Vision and Pattern Recognition, Zhang et al. proposed using rhombic lattice vector quantization. However, they still employed a soft-to-hard approach and implemented it using two scalar quantizers.
[0005] In summary, due to the use of a soft-to-hard training approach, existing vector quantization methods in deep learning-based image compression suffer from difficulties in convergence during training and inconsistencies between training and testing. Summary of the Invention
[0006] To address the aforementioned problems, this invention proposes an image encoding, decoding, and compression method and system based on vector quantization entropy modeling. It uses a lattice vector quantizer to quantize features and employs relaxed boundaries to model the entropy of the quantized features, thereby reducing complexity, making training more stable, and resolving the inconsistency between training and testing, thus improving the rate-distortion performance of image compression.
[0007] A first aspect of the present invention provides an image coding method for vector quantization entropy modeling, comprising:
[0008] A first feature map of the image to be encoded is obtained through an encoding network, wherein the first feature map is a three-dimensional feature map, and the dimensions include channel dimension, height dimension and width dimension;
[0009] Obtain the binary code stream of the super-prior features of the first feature map and the corresponding super-prior information;
[0010] The first feature map is quantized using a lattice vector quantizer to obtain a quantized first feature map;
[0011] The first feature map of quantization is projected onto the integer coefficient domain of the lattice vector quantizer to obtain the second feature map;
[0012] The distribution of the second feature map is parameterized and modeled as a Gaussian distribution with independent dimensions. The mean and variance of the Gaussian distribution are predicted by combining the prior information and the spatial context model.
[0013] The integer coefficient domain of the lattice vector quantizer is partitioned using a relaxed boundary. Based on the predicted mean and variance of the Gaussian distribution, the second feature map is probability estimated. Arithmetic coding is then performed based on the estimated probability to obtain the feature binary code stream.
[0014] The feature binary code is merged with the prior feature binary code stream to obtain the compressed image binary code stream.
[0015] Optionally, obtaining the prior binary code stream of the first feature map and the corresponding prior information includes:
[0016] A super-prior coding network is used to obtain a super-prior feature map from the first feature map;
[0017] Each feature in the super-prior feature map is rounded to an integer to obtain the quantized super-prior feature map.
[0018] An independent nonparametric distribution estimate is established for each channel of the quantized prior feature map;
[0019] Arithmetic coding of the super-prior second feature map is performed based on the non-parametric distribution estimation to obtain the super-prior feature binary code stream;
[0020] The reconstructed hyperprior feature map is obtained by arithmetic decoding of the hyperprior binary code stream using the nonparametric distribution estimation.
[0021] The reconstructed prior feature map is processed by a priori decoding network to obtain prior information.
[0022] Optionally, quantizing the first feature map using a lattice vector quantizer to obtain a quantized first feature map includes:
[0023] Construct an n-dimensional lattice based on a pre-defined arrangement.
[0024] The feature vector in the i1th row and j1st column of the first feature map with height H1, width W1, and number of channels M1. Divide into M1 / n vectors The k1th vector
[0025] For any vector Select the nearest lattice point in the lattice Λ based on the second norm. As a quantized vector, the specific calculation process is as follows:
[0026]
[0027] The quantized M1 / n vectors By concatenating the data, we obtain the first quantized feature map. The eigenvector of the i1th row and j1st column is
[0028] The first feature map is obtained by obtaining the feature vectors of all H1 rows and W1 columns. The first feature map of quantization The height and width are the same as the height and width of the first feature map.
[0029] Optionally, the construction of an n-dimensional lattice include:
[0030] Obtain a set of linearly independent n-dimensional basis {v1, v2, ..., v n The values of each basis are related to the specific arrangement of the crystal lattice. Common crystal lattice arrangements include cubic arrangement, rhombic arrangement, etc.
[0031] Let the set of linearly independent n-dimensional basis {v1,v2,…,v... n Linear combinations are performed (where the combination coefficients are integers) such that all the n-dimensional vectors resulting from these linear combinations fill the entire space, resulting in a lattice Λ, which can be represented as a set:
[0032]
[0033] Among them, u=(u1,u2,…,u n ) T V is an integer vector. Λ =(v1,v2,…,vn ) T It is the generating matrix corresponding to the lattice Λ arrangement, composed of linearly independent n-dimensional basis {v1,v2,…,v... n}composition.
[0034] Optionally, projecting the quantized first feature map onto the integer coefficient domain of the lattice vector quantizer to obtain the second feature map includes:
[0035] The k1th feature vector in the i1th row and j1st column of the quantized first feature map The generation matrix V of the lattice Λ in the lattice vector quantizer Λ The inverse matrix is projected onto the integer coefficient space. n is the dimension of the crystal lattice, and the specific calculation method is as follows:
[0036]
[0037] in
[0038] The projected vector The feature vector in the i2th row and j2nd column of the second feature map is formed by splicing the vectors together. Obtain the second feature map
[0039] Optionally, the step of parameterizing and modeling the distribution of the second feature map as a Gaussian distribution with independent dimensions, and predicting the mean and variance of the Gaussian distribution by combining the prior information and the spatial context model, includes:
[0040] The second feature map is divided into two spatial groups in a chessboard pattern. The sum of the height and width coordinates in the first spatial group is odd, and the sum of the height and width coordinates in the second spatial group is even.
[0041] Generate an all-zero tensor according to the size of the first spatial group and concatenate it with the prior information. Input it into a neural network consisting of multiple convolutional layers and ReLU activation functions to obtain the mean and variance of the Gaussian distribution of the first spatial group, where the ReLU activation function returns a value larger than zero from the input value.
[0042] The first spatial group is input into a convolutional layer to obtain the context information of the second spatial group. The prior information is concatenated with the context information of the second spatial group and input into the neural network (a neural network composed of multiple convolutional layers and ReLU activation functions) to obtain the mean and variance of the Gaussian distribution of the second spatial group.
[0043] The mean and variance of the Gaussian distribution of the first spatial group and the mean and variance of the Gaussian distribution of the second spatial group are combined to obtain the mean and variance of the Gaussian distribution of the second feature map.
[0044] Optionally, the step of partitioning the integer coefficient domain of the lattice vector quantizer using relaxed boundaries, performing probability estimation on the second feature map based on the predicted mean and variance of the Gaussian distribution, and performing arithmetic encoding based on the estimated probability to obtain the feature binary code stream includes:
[0045] The integer coefficient domain of the lattice vector quantizer is divided by a hypercube of volume 1, which fills the entire space and is centered on an integer vector, such that the second feature map has a feature vector in the i2th row and j2th column that is k2th. It is the center of its hypercube; where 1≤i2≤H2, 1≤j2≤W2, 1≤k2≤M2 / n, H2, W2 and M2 are the height, width and number of channels of the second feature map, respectively, and n is the dimension of the lattice;
[0046] The boundary of the hypercube is used as the k2th feature vector in the i2th row and j2nd column of the estimated second feature map. Based on the probability boundary and the mean and variance of the predicted Gaussian distribution, estimate the k2-th feature vector in the i2-th row and j2-th column of the second feature map. probability The specific calculation method is as follows:
[0047]
[0048] in and They are respectively The mean and variance of a Gaussian distribution;
[0049] Based on the estimated probability, the k2nd feature vector in the i2th row and j2nd column of the second feature map is... Perform arithmetic encoding to obtain the bitstream of the k2th feature vector in the i2th row and j2th column of the second feature map;
[0050] Merge all feature vector bitstreams to obtain the feature binary bitstream.
[0051] A second aspect of the present invention provides an image decoding method based on vector quantization entropy modeling, for correspondingly decoding the encoding obtained by the above-described image encoding method, comprising:
[0052] Obtain a compressed image binary bitstream, and decompose the compressed image binary bitstream to obtain a feature binary bitstream and a priori feature binary bitstream;
[0053] The arithmetic decoding of the hyperprior binary code stream is performed using nonparametric distribution estimation to obtain the hyperprior feature map;
[0054] The super-prior feature map is processed by a super-prior decoding network to obtain super-prior information;
[0055] The mean and variance of the Gaussian distribution for each spatial group used to decode the second feature map are predicted by combining the aforementioned prior information and the spatial context model.
[0056] The integer coefficient domain of the lattice vector quantizer is divided using a relaxed boundary. The second feature map is then arithmetically decoded based on the mean and variance of the Gaussian distribution. The second feature map is obtained by spatially grouping the data in the order of spatial grouping.
[0057] The second feature map is inversely projected onto the feature domain to obtain the reconstructed first feature map;
[0058] The reconstructed first feature map is passed through a decoding network to obtain a reconstructed image.
[0059] Optionally, the projection feature vector of the second feature map in the i2th row and j2th column is... The generation matrix V of the lattice Λ in the lattice vector quantizer Λ Inverse projection onto feature space The specific calculation method is as follows:
[0060]
[0061] The vector after inverse projection The feature vector in the i2th row and j2nd column of the reconstructed first feature map is formed by splicing the vectors together. Obtain the first feature map of the reconstruction. Where 1≤i2≤H2, 1≤j2≤W2, 1≤k2≤M2 / n, H2, W2 and M2 are the height, width and number of channels of the second feature map, respectively, and n is the dimension of the lattice.
[0062] A third aspect of the present invention provides an image compression method using vector quantization entropy modeling, comprising:
[0063] Obtain the image to be compressed;
[0064] The image encoding method based on the above vector quantization entropy model is used to encode the image to be compressed to obtain a compressed image binary code stream;
[0065] The image decoding method based on the above-described vector quantization entropy model decodes the compressed image binary code stream to obtain the reconstructed image.
[0066] Optionally, the parameters of the encoding network, decoding network, super-prior encoding network, super-decoding neural network, vector quantization entropy modeling, and spatial context model are obtained through rate-distortion optimization.
[0067]
[0068] Among them, distortion The image to be compressed (x) and the reconstructed image are... Mean squared error, λ>0 is a preset Lagrange coefficient used to balance distortion and bit rate. The bit rate R is calculated based on the Gaussian distribution of the second feature map and the nonparametric distribution of the super-prior feature map.
[0069]
[0070] Where 1≤i2≤H2, 1≤j2≤W2, 1≤k2≤M2 / n, H2, W2, and M2 are the height, width, and number of channels of the second feature map, respectively, and n is the dimension of the lattice. These represent the feature vectors encoded in the i2th row, j2th column, and k2th column of the second feature map. The mean and variance of the Gaussian distribution. It is a priori feature map Distribution estimation of the l-th dimension, yes A uniform distribution on the surface. This refers to summing over each dimension of the prior feature map.
[0071] A fourth aspect of the present invention provides an image compression system, the image compression system comprising an encoder and a decoder, comprising:
[0072] Obtain the image to be compressed;
[0073] The encoder uses the image encoding method described above, which models vector quantization entropy, to encode the image to be compressed, thereby obtaining a compressed image binary code stream.
[0074] The decoder uses the image decoding method modeled by vector quantization entropy described above to decompress the compressed image binary code stream and obtain the reconstructed image.
[0075] A fifth aspect of the present invention provides an image compression apparatus, the image compression apparatus comprising a memory and a processor, the memory storing a program executable by the processor, the processor executing the program and performing any of the following methods:
[0076] -The above-mentioned image coding method using vector quantization entropy modeling, or,
[0077] -The above-mentioned image decoding method based on vector quantization entropy modeling, or,
[0078] -The above-mentioned image compression method based on vector quantization entropy modeling.
[0079] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:
[0080] The image encoding, decoding, and compression method provided by this invention using vector quantization entropy modeling utilizes a lattice vector quantizer to efficiently quantize features, with a complexity approaching that of a scalar quantizer. The codebook has an infinite number of codewords and does not produce the codebook collapse problem.
[0081] The image encoding, decoding, and compression method based on vector quantization entropy modeling provided by this invention does not adopt a soft-to-hard training approach, but rather performs entropy modeling based on relaxed boundaries. It achieves stable convergence during training and does not have the problem of inconsistency between training and testing.
[0082] The image encoding, decoding, and compression method based on vector quantization entropy modeling provided by this invention can save 6.03% and 2.48% of bitstream overhead, respectively, while maintaining the same image reconstruction quality, compared to scalar quantization methods and the latest vector quantization methods, thus improving rate-distortion performance.
[0083] The image compression method based on vector quantization entropy modeling in this invention can improve the rate-distortion performance of natural images at different resolutions acquired in different scenarios. This invention has strong practical application value, especially in actual image compression, where it can perform efficient, stable, and low-complexity vector quantization on features with different statistical properties, thereby improving rate-distortion performance. Attached Figure Description
[0084] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0085] Figure 1 This is a flowchart of an image coding method for vector quantization entropy modeling according to an embodiment of the present invention.
[0086] Figure 2 This is a flowchart of an image coding method for vector quantization entropy modeling in a preferred embodiment of the present invention.
[0087] Figure 3 This is a flowchart of an image decoding method using vector quantization entropy modeling, as described in a preferred embodiment of the present invention.
[0088] Figure 4 This is a block diagram of image compression using vector quantization entropy modeling in one embodiment of the present invention.
[0089] Figure 5 This is a flowchart of an image compression method using vector quantization entropy modeling, as described in a preferred embodiment of the present invention. Detailed Implementation
[0090] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.
[0091] Reference Figure 1 The diagram shows a flowchart of an image coding method for vector quantization entropy modeling according to an embodiment of the present invention. This embodiment provides an image coding method for vector quantization entropy modeling, including steps S100-S700. Specifically:
[0092] S100, the first feature map of the image to be encoded is obtained through the encoding network;
[0093] In this step, the image to be encoded is first determined, and then the first feature map of the image is obtained through an encoding network. Specifically, the encoding network can be a deep neural network.
[0094] The first feature map is a three-dimensional feature map of the image to be encoded. The dimensions of the three-dimensional feature map include the channel dimension, the height dimension, and the width dimension.
[0095] S200, obtain the binary code stream of the super-prior features of the first feature map and the corresponding super-prior information;
[0096] In this step, the first feature map is sequentially passed through a super-prior coding network to obtain a super-prior first feature map. The super-prior feature map is then quantized into integers to obtain a quantized super-prior feature map. Furthermore, an independent non-parametric distribution estimate is established for each channel of the quantized super-prior feature map, and this non-parametric distribution estimate is used to perform arithmetic coding on the quantized super-prior feature map to obtain a super-prior feature binary code stream. The reconstructed super-prior feature map is obtained from the super-prior feature binary code stream, and the super-prior information is obtained using a super-prior decoding network.
[0097] S300, the first feature map is quantized using a lattice vector quantizer to obtain the quantized first feature map;
[0098] In this step, the mass of each spatial location in the first feature map is uniformly divided into several vectors according to the size of the lattice dimension. Each vector is quantized using a lattice vector quantizer, and the resulting vectors are stitched together to obtain the quantized first feature map.
[0099] S400: Project the quantized first feature map onto the integer coefficient domain of the lattice vector quantizer to obtain the second feature map;
[0100] In this step, the inverse matrix of the lattice generation matrix in the lattice vector quantizer is used to project the quantized first feature map onto the integer coefficient domain to obtain the second feature map.
[0101] S500 parameterizes the distribution of the second feature map as a Gaussian distribution with independent dimensions, and predicts the mean and variance of the Gaussian distribution by combining prior information and spatial context model.
[0102] In this step, the spatial context model is implemented by a deep neural network, which divides the second feature map into two spatial groups in a chessboard pattern. For each spatial group, the super-prior information and the spatial context model are combined to predict the mean and variance of the Gaussian distribution.
[0103] S600 uses a relaxed boundary to partition the integer coefficient field of the lattice vector quantizer. Based on the predicted mean and variance of the Gaussian distribution, it performs probability estimation on the second feature map and performs arithmetic coding based on the estimated probability to obtain the feature binary code stream.
[0104] In this step, the integer coefficient domain of the lattice vector quantizer is divided using a hypercube with a volume of 1. Based on the boundary of the hypercube and the mean and variance of the predicted Gaussian distribution, the second feature map is probabilistically estimated and arithmetically encoded to obtain the feature binary code stream.
[0105] S700 merges the feature binary code with the prior feature binary code stream to obtain the compressed image binary code stream.
[0106] In the above embodiments of the present invention, lattice vector quantization of three-dimensional features can be performed efficiently, and relaxed boundaries are used to encode the three-dimensional features. In actual image compression, more efficient vector quantization can be achieved with a complexity close to that of scalar quantization, thus improving rate-distortion performance. The lattice vector quantizer is used to efficiently quantize features, with a complexity close to that of a scalar quantizer. The codebook has an infinite number of codewords and does not produce codebook collapse problems.
[0107] In some embodiments, performing the above-described S200 to obtain the prior binary code stream of the first feature map and the corresponding prior information may include:
[0108] M101, a super-prior coding network is used to obtain a super-prior feature map from the first feature map;
[0109] M102 rounds each feature in the prior feature map to an integer to obtain the quantized prior feature map.
[0110] M103 establishes an independent nonparametric distribution estimate for each channel of the quantized prior feature map;
[0111] M104, based on the non-parametric distribution estimation, performs arithmetic coding on the super-prior second feature map to obtain the super-prior feature binary code stream;
[0112] M105 uses non-parametric distribution estimation to perform arithmetic decoding on the prior binary code stream to obtain the reconstructed prior feature map.
[0113] M106 uses a priori decoding network to obtain prior information from the reconstructed priori feature map.
[0114] The above embodiments of the present invention, by employing the above steps, can better obtain the required prior information, providing conditions for the final encoding and decoding.
[0115] Figure 2 This is a flowchart of an image coding method for vector quantization entropy modeling according to a preferred embodiment of the present invention. In this preferred embodiment, the lattice vector quantization image coding method may include steps S101-S109, specifically:
[0116] S101, Feature extraction: The image to be encoded is passed through an encoding network to obtain a first feature map. The first feature map is a three-dimensional feature map, and the dimensions include channel dimension, height dimension and width dimension.
[0117] S102, Super-prior feature extraction and quantization: The three-dimensional features of the first feature map are passed through a super-prior coding network to obtain a super-prior feature map; the super-prior features are rounded and quantized into integers;
[0118] S103, Super-prior feature encoding and decoding: Establish an independent non-parametric distribution estimate for each channel of the quantized super-prior feature map, and use this non-parametric distribution estimate to perform arithmetic encoding on the super-prior features to obtain the super-prior feature binary code stream; perform arithmetic decoding on the super-prior feature binary code stream to obtain the reconstructed super-prior feature map;
[0119] S104, Obtaining Super Prior Information: Obtain super prior information by passing the reconstructed super prior feature map through a super prior decoding network;
[0120] S105, Lattice Vector Quantization: Quantizes the vector at each spatial location of a three-dimensional feature using a lattice vector quantizer;
[0121] S106, Feature Projection: The three-dimensional features are projected onto the integer coefficient domain using the inverse matrix of the generator matrix of the lattice in the lattice vector quantizer to obtain the projected three-dimensional features.
[0122] S107, Entropy modeling based on relaxed boundaries: The distribution of the projected three-dimensional features is parameterized and modeled as a Gaussian distribution with independent dimensions. The mean and variance of the Gaussian distribution are predicted by combining the prior information and the spatial context model. The integer coefficient field of the lattice vector quantizer is divided by a hypercube with a volume of 1.
[0123] S108, Feature Encoding: Based on the predicted mean and variance of the Gaussian distribution and the boundary division of the integer coefficient domain of the lattice vector quantizer, the projected three-dimensional features are probabilistically estimated, and arithmetic encoding is performed based on the estimated probabilities to obtain the feature binary code stream.
[0124] S109, Bitstream Synthesis: The feature binary bitstream is merged with the prior feature binary bitstream to obtain the compressed image binary bitstream.
[0125] In the above embodiments of the present invention, lattice vector quantizers and entropy modeling based on relaxed boundaries can more efficiently perform vector quantization of three-dimensional features, thereby improving the rate-distortion performance of image compression. Entropy modeling based on relaxed boundaries does not employ a soft-to-hard training method, but rather performs entropy modeling based on relaxed boundaries. This results in stable convergence during training and eliminates the problem of inconsistency between training and testing.
[0126] To perform lattice vector quantization and entropy modeling more efficiently, in one possible implementation, S105 in the above embodiment can be achieved by the following method, specifically:
[0127] S1051, for a given n-dimensional lattice The feature vector of the 3D feature in the i-th row and j-th column (1≤i≤H, 1≤j≤W, where H and W are the height and width of the 3D feature, respectively). (M is the number of channels in the three-dimensional feature) is uniformly divided into M / n vectors. in
[0128] S1052, based on the L2 norm, M / n vectors Each quantization in the lattice is the nearest lattice point in the lattice Λ. We obtain M / n quantized vectors The specific calculation process is as follows:
[0129]
[0130] S1053, quantized M / n vectors The features are spliced together to form the feature vector in the i-th row and j-th column of the quantized three-dimensional feature. This yields the quantized three-dimensional features.
[0131] In one possible implementation, S106 in the above embodiment can be implemented by the following method:
[0132] S1061, quantized three-dimensional features The feature vector in row i, column j, and k is... The generation matrix V of the lattice Λ in the lattice vector quantizer Λ The inverse matrix is projected onto the integer coefficient space. The specific calculation method is as follows:
[0133]
[0134] in
[0135] S1062, the projected vector The features are stitched together to form the feature vector in the i-th row and j-th column of the projected 3D features. Obtaining projected 3D features
[0136] In one possible implementation, S107 in the above embodiment can be implemented by the following method:
[0137] S1071, the projected 3D features are divided into two spatial groups in a chessboard pattern. The sum of the height and width coordinates in the first spatial group is odd, and the sum of the height and width coordinates in the second spatial group is even.
[0138] S1072 concatenates the prior information with a zero tensor of the same size as the first spatial group, and inputs it into a neural network consisting of a cascade of convolutions with a kernel size of 1×1 and ReLU activation functions to obtain the mean and variance of the Gaussian distribution of the first spatial group; of course, other kernel sizes are also applicable, such as 3×3, 5×5, etc., and are not limited to 1×1.
[0139] In step S1073, the first spatial group is input into a single convolutional layer to obtain the contextual information of the second spatial group. The prior information and the contextual information of the second spatial group are concatenated and then input into a neural network consisting of multiple convolutional layers and cascaded ReLU activation functions. This yields the mean and variance of the Gaussian distribution of the second spatial group. The neural network can be composed of multiple cascaded convolutional layers with 1×1 kernels and ReLU activation functions; other kernel sizes are also applicable, such as 3×3, 5×5, etc., and are not limited to 1×1. This step reduces computational cost, avoids overfitting, and enhances the expressive power of neural network neurons.
[0140] S1074, the mean and variance of the Gaussian distribution of the first spatial group and the mean and variance of the Gaussian distribution of the second spatial group are combined to obtain the mean and variance of the Gaussian distribution of the projected three-dimensional features.
[0141] S1075, the integer coefficient field of the lattice vector quantizer is divided by a hypercube of volume 1. All hypercubes fill the entire space and are centered on integer vectors, such that the i-th row, j-th column, and k-th feature vector of the projected 3D feature is obtained. The center of its hypercube;
[0142] S1076, the boundary of the hypercube is used as the k-th feature vector in the i-th row and j-th column of the estimated second feature map. Based on the probability boundary and the mean and variance of the predicted Gaussian distribution, estimate the k-th feature vector in the i-th row and j-th column of the second feature map. The probability is calculated as follows:
[0143]
[0144] in and They are respectively The mean and variance of the Gaussian distribution.
[0145] The image encoding method using vector quantization entropy modeling in this embodiment utilizes a lattice vector quantizer to quantize features and combines it with relaxed boundaries for entropy modeling. It features low complexity, high stability, and an infinitely large codebook, thus providing a more efficient vector quantization method.
[0146] Based on the same inventive concept, another embodiment of the present invention provides an image decoding method using vector quantization entropy modeling, used to decode the encoding obtained by the image encoding method using vector quantization entropy modeling in any of the above embodiments. For example... Figure 3 As shown, the image decoding method based on vector quantization entropy modeling includes:
[0147] S201, Bitstream decomposition: Decompose the compressed image binary bitstream to obtain the feature binary bitstream and the prior feature binary bitstream;
[0148] S202, Super-prior feature decoding: Perform arithmetic decoding on the super-prior feature binary code stream to obtain the reconstructed super-prior feature map;
[0149] S203, Obtaining Super Prior Information: Input the reconstructed super prior feature map into the super prior decoding network to obtain super prior information;
[0150] S204, Entropy Modeling Based on Relaxed Boundaries: The structure and parameter values of this module are related to... Figure 2The entropy modeling based on relaxed boundaries in the illustrated embodiments is exactly the same;
[0151] For example, the projected 3D features are first initialized with an all-zero tensor. The projected 3D features are then spatially divided into two groups in a chessboard pattern. In the first group, the sum of the height and width coordinates is odd, while in the second group, the sum is even. The distribution of each spatial group is parameterized and modeled as a Gaussian distribution with independent dimensions. The mean and variance of this Gaussian distribution are predicted by combining the prior information and the spatial context model. The integer coefficient domain of the lattice vector quantizer is partitioned using relaxed boundaries. Based on the predicted mean and variance of the Gaussian distribution, a probability estimate is performed for each spatial group.
[0152] S205, Projection Feature Decoding: Based on the predicted mean and variance of the Gaussian distribution and the integer coefficient domain boundary of the lattice vector quantizer, the projected three-dimensional features are obtained by spatially grouping in the order of spatial grouping.
[0153] S206, Inverse Projection of Projected Features: Inversely project the projected 3D features onto the feature domain to obtain a 3D feature map, i.e., the reconstructed first feature map, where the feature domain is the real number domain in which the first feature map is located.
[0154] S207, Image Reconstruction: Input the reconstructed first feature map into the decoding network to obtain the reconstructed image.
[0155] In one possible implementation, S206 in the above embodiment can be implemented by the following method, specifically:
[0156] S2061, project the i-th row, j-th column, and k-th projection feature vector of the three-dimensional feature. The generation matrix V of the lattice Λ in the lattice vector quantizer Λ Inverse projection onto feature space The specific calculation method is as follows:
[0157]
[0158] S2062, the vector after inverse projection By splicing, a feature vector in the i-th row and j-th column of the three-dimensional feature is formed. Obtaining 3D features
[0159] The image decoding method for vector quantization entropy modeling in this embodiment is used to correspond to Figure 2 The image coding method using vector quantization entropy modeling, as shown, utilizes a lattice vector quantizer to quantize features and combines this with relaxed boundaries for entropy modeling, thus achieving a more efficient quantization method. For details not explained above, please refer to... Figure 2 The corresponding descriptions in the illustrated embodiments will not be repeated here.
[0160] Based on the same inventive concept, in another embodiment, a vector quantization entropy modeling image compression method is also provided, comprising:
[0161] M201, Obtain the image to be compressed;
[0162] M202, based on the image encoding method of any of the above embodiments, encodes the image to be compressed to obtain a compressed image binary bitstream;
[0163] M203, based on the image decoding method of any of the above embodiments, decodes the compressed image binary code stream to obtain a reconstructed image.
[0164] The image encoding and decoding methods described above are the same as the vector quantization entropy modeling image encoding and decoding methods in the above embodiments, and will not be repeated here.
[0165] The aforementioned image encoding and decoding methods can be executed using an encoder and a decoder. In one specific embodiment, 24 images with a resolution of 512×768 from the Kodak test set are compressed. The flowchart of this vector quantization entropy modeling image compression method is as follows. Figure 4 As shown, it includes:
[0166] S301, Image Acquisition Module: Sequentially acquires the images to be compressed;
[0167] S302, Encoder: Encode the image to be compressed according to steps S101-S109 to obtain the compressed image binary code stream.
[0168] For example, the parameters are as follows: the height H of the 3D feature is 32, the width W is 48, and the number of channels is M = 192; the specific values of the preset parameters for entropy modeling based on relaxed boundaries are: the dimension of the lattice Λ is n = 2, and the generation matrix of the lattice Λ is...
[0169] S303, Decoder: Decompresses the compressed image binary stream according to S201-S207 to obtain the reconstructed image.
[0170] The height, width, number of channels, and entropy modeling preset parameters based on relaxed boundaries for the three-dimensional features are exactly the same as those for the encoder.
[0171] The compression method in the above embodiments of the present invention is based on Figure 1 , 2 Image coding methods based on vector quantization entropy modeling and Figure 3The image decoding method shown is implemented using vector quantization entropy modeling. For parts not described in detail above, please refer to the corresponding descriptions in the above embodiments, which will not be repeated here.
[0172] The compression method in the above embodiments of the present invention can improve the performance of vector quantization and improve the rate-distortion performance of image compression on natural images of different resolutions collected in different scenarios. It can perform efficient, stable and low-complexity vector quantization on features with different statistical characteristics, thereby improving rate-distortion performance.
[0173] The image compression method described above can be improved by training a model. For example, the training set consists of 20,000 natural images of different resolutions collected in various natural scenes. The trained model can then be used for actual image encoding and decoding.
[0174] Specifically, such as Figure 5 As shown, in a preferred embodiment, the image compression method may include:
[0175] S401, Feature Extraction: Randomly crop the images in the training set into images with a resolution of 256×256, and train 16 images in one step to input into the encoding network to obtain three-dimensional features. The encoding network consists of a convolutional layer, a GDN activation function, a convolutional layer, a GDN activation function, a convolutional layer, a GDN activation function, and a convolutional layer cascaded in sequence. The three-dimensional features have a height and width of 16 and a channel number of 192.
[0176] The image resolution, number of images trained in one step, structure of the encoding network, and three-dimensional feature parameters mentioned in this embodiment can be adjusted according to actual conditions in other embodiments and are not limited to those described in this step. The same applies to the parameters in the following steps.
[0177] S402, Feature Noise Addition: To avoid the inability to backpropagate gradients due to lattice vector quantization, noise is added to each vector at each spatial location of the 3D feature in a uniformly distributed region of the lattice vector quantizer to simulate the quantization operation.
[0178] S403, Feature Projection: This involves projecting each vector at each spatial location of the noisy 3D feature using the generation matrix of the lattice Λ in the lattice vector quantizer. The inverse matrix is projected onto the integer coefficient field to obtain the projected three-dimensional features;
[0179] S404, Super Prior Feature Extraction and Noise Addition: The three-dimensional features are input into the super prior coding network to obtain super prior features. In order to avoid the gradient not being able to backpropagate due to the quantization operation, noise with a uniform distribution from -0.5 to 0.5 is added to the super prior features to simulate the quantization operation. The super prior coding network consists of three concatenated convolutional layers.
[0180] S405, Super Prior Feature Code Rate Estimation: Establish an independent non-parametric distribution estimate for each channel of the super prior feature, and use this non-parametric distribution estimate to obtain the super prior feature code rate estimate;
[0181] S406, Acquisition of super-prior information: Input the noisy super-prior features into the super-prior decoding network to obtain super-prior information, wherein the super-prior decoding network consists of three concatenated convolutional layers;
[0182] S407, Entropy Modeling Based on Relaxed Boundaries: The structure of this module is similar to... Figure 2 The entropy modeling based on relaxed boundaries in the illustrated embodiment is exactly the same. The distribution of the projected three-dimensional features is parameterized and modeled as a Gaussian distribution with independent dimensions. The mean and variance of the Gaussian distribution are predicted by combining the prior information and the spatial context model.
[0183] In this embodiment, a hypercube with a volume of 1 is used to divide the integer coefficient domain of the lattice vector quantizer; the specific values of the preset parameters are: lattice dimension n = 2, and the generation matrix of lattice Λ.
[0184] S408, Feature Code Rate Estimation: Based on the predicted mean and variance of the Gaussian distribution and the boundary division of the integer coefficient domain of the lattice vector quantizer, the code rate of the projected three-dimensional features is estimated to obtain the feature code rate estimate.
[0185] S409, Total Bitrate Estimation: Add the prior feature bitrate estimate and the feature bitrate estimate to obtain the total bitrate estimate;
[0186] S410, Inverse Projection of Projected Features: This involves transforming each vector at each spatial location of the projected 3D feature using the generation matrix V of the lattice vector quantizer within the lattice Λ. Λ By inversely projecting onto the feature domain, the reconstructed, noisy 3D features are obtained, where the generation matrix of the lattice Λ is...
[0187] S411, Image Reconstruction: The reconstructed, noisy 3D features are input into the decoding network to obtain the reconstructed image. The decoding neural network consists of a convolutional layer, a GDN activation function, a convolutional layer, a GDN activation function, a convolutional layer, a GDN activation function, and a convolutional layer cascaded in sequence.
[0188] In some possible implementations, the above image compression method can be used as an image compression model, which can be further optimized using stochastic gradient descent, with a rate-distortion loss function as follows:
[0189]
[0190] Among them, distortion The image to be compressed (x) and the reconstructed image are... Mean squared error, λ>0, is a preset Lagrange coefficient used to balance distortion and bit rate (one λ value corresponds to one image compression model). Bit rate R is calculated based on the Gaussian distribution of the projected 3D features and the nonparametric distribution of the prior features.
[0191]
[0192] in and These are the predicted projected 3D features, specifically the feature vector in the i-th row, j-th column, and k-th position. The mean and variance of the Gaussian distribution. It is a priori feature The distribution estimate of the nth dimension, yes A uniform distribution on the surface.
[0193] Application implementation results:
[0194] In the above embodiment, the image compression model modeled by vector quantization entropy was trained using the Flicker dataset. λ was set to 0.0016, 0.0032, 0.0075, 0.015, 0.03, and 0.045, corresponding to six different bitrate image compression models. The Adam optimizer was used for 1,000,000 optimization steps, with the initial 950,000 steps having a learning rate of 10. -4 It then dropped to 10 -5 .
[0195] In the above embodiment, an image encoder using an image compression model modeled with six trained vector quantization entropies was used to compress 24 images with a resolution of 512×768 from the Kodak test set. The average bitrates of the 24 compressed images were 0.1443 bpp, 0.2334 bpp, 0.3366 bpp, 0.4840 bpp, 0.6842 bpp, and 0.9062 bpp, respectively.
[0196] In the above embodiment, an image encoder using an image compression model modeled with six trained vector quantization entropies compresses 24 images with a resolution of 512×768 from the Kodak test set to obtain reconstructed images. The average peak signal-to-noise ratios (PSNR) of the 24 reconstructed images are 29.01dB, 30.79dB, 32.30dB, 34.02dB, 35.84dB, and 37.42dB, respectively.
[0197] Compared to methods using baseline neural networks, the vector quantization entropy modeling image compression method provided in the above embodiments of the present invention exhibits better rate-distortion performance. Specifically, compared to scalar quantization methods and the latest vector quantization methods, it can save 6.03% and 2.48% of bitstream overhead, respectively, while maintaining the same reconstructed image quality. The vector quantization entropy modeling image compression method of the present invention can improve image compression rate-distortion performance on natural images of different resolutions acquired in different scenarios. The present invention has strong practical application value, especially in actual image compression, where it can perform efficient, stable, and low-complexity vector quantization on features with different statistical properties, thereby improving rate-distortion performance.
[0198] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0199] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0200] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0201] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0202] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.
Claims
1. A method of image coding with vector quantization entropy modeling, characterized by, include: A first feature map of the image to be encoded is obtained through an encoding network, wherein the first feature map is a three-dimensional feature map, and the dimensions include channel dimension, height dimension and width dimension; Obtain the binary code stream of the super-prior features of the first feature map and the corresponding super-prior information; The first feature map is quantized using a lattice vector quantizer to obtain a quantized first feature map; The first feature map of quantization is projected onto the integer coefficient domain of the lattice vector quantizer to obtain the second feature map; The distribution of the second feature map is parameterized and modeled as a Gaussian distribution with independent dimensions. The mean and variance of the Gaussian distribution are predicted by combining the prior information and the spatial context model. The integer coefficient domain of the lattice vector quantizer is divided using a relaxed boundary. The second feature map is probabilistically estimated based on the predicted mean and variance of the Gaussian distribution. Arithmetic coding is then performed based on the estimated probability to obtain the feature binary code stream. The feature binary code is merged with the prior feature binary code stream to obtain the compressed image binary code stream.
2. The image coding method of claim 1, wherein, The step of obtaining the prior binary code stream of the first feature map and the corresponding prior information includes: A super-prior coding network is used to obtain a super-prior feature map from the first feature map; Each feature in the super-prior feature map is rounded to an integer to obtain the quantized super-prior feature map. An independent nonparametric distribution estimate is established for each channel of the quantized prior feature map; Arithmetic coding of the super-prior second feature map is performed based on the non-parametric distribution estimation to obtain the super-prior feature binary code stream; The reconstructed hyperprior feature map is obtained by arithmetic decoding of the hyperprior binary code stream using the nonparametric distribution estimation. The reconstructed prior feature map is processed by a priori decoding network to obtain prior information.
3. The image coding method of claim 1, wherein, The step of quantizing the first feature map using a lattice vector quantizer to obtain a quantized first feature map includes: Construct a based on a pre-defined arrangement 3D lattice ; height , width , and the number of channels of the first feature map are evenly divided into , the first column of the first feature map , the first column of the first feature map , the first column of the first feature map , the first column of the first feature map For any vector Selecting the lattice based on the second norm The nearest lattice point , as the quantized vector; , , ; For the first Line number The first of the columns 1 eigenvector; concatenate the quantized vectors to obtain a quantized first feature map obtaining all rows column-wise feature vectors to obtain a quantized first feature map , a height and a width of the quantized first feature map are the same as a height and a width of the first feature map.
4. The image coding method of claim 3, wherein, The preset arrangement mode is used to construct a lattice , comprising: Obtain a set of linearly independent Victory ; The set of linearly independent vectors of dimension d are combined linearly such that all linear combinations of the vectors span the entire space, resulting in a lattice , which is specifically represented as a set: ； wherein, is an integer vector, is a generating matrix of the arrangement, consisting of linearly independent vectors, of dimension n, consisting of linearly independent vectors, is the dimension of the lattice.
5. The image coding method of claim 1, wherein, The step of projecting the quantized first feature map onto the integer coefficient domain of the lattice vector quantizer to obtain the second feature map includes: The first feature map of the quantization Line 1 The first of the columns eigenvectors The lattice in the lattice vector quantizer Generating matrix The inverse matrix is projected onto the integer coefficient space. , , , , The dimension of the crystal lattice; , , These represent the height, width, and number of channels of the first feature map, respectively, while M represents the number of channels in the three-dimensional feature map. The projected vector splicing together to form the second feature map Line number Column eigenvectors Obtain the second feature map . 6.The image coding method of vector quantization entropy modeling according to claim 1, wherein, The step of parameterizing and modeling the distribution of the second feature map as a Gaussian distribution with independent dimensions, and predicting the mean and variance of the Gaussian distribution by combining the prior information and the spatial context model, includes: The second feature map is divided into two spatial groups in a chessboard pattern. The sum of the height and width coordinates in the first spatial group is odd, and the sum of the height and width coordinates in the second spatial group is even. Generate an all-zero tensor according to the size of the first spatial group and concatenate it with the prior information. Input it into a neural network consisting of multiple convolutional layers and ReLU activation functions to obtain the mean and variance of the Gaussian distribution of the first spatial group, where the ReLU activation function returns a value larger than zero from the input value. The first spatial group is input into a convolutional layer to obtain the context information of the second spatial group. The prior information is concatenated with the context information of the second spatial group and input into the neural network described above to obtain the mean and variance of the Gaussian distribution of the second spatial group. The mean and variance of the Gaussian distribution of the first spatial group are combined with the mean and variance of the Gaussian distribution of the second spatial group to obtain the mean and variance of the Gaussian distribution of the second feature map.
7. The image coding method of claim 1, wherein, The process involves partitioning the integer coefficient domain of the lattice vector quantizer using relaxed boundaries, performing probability estimation on the second feature map based on the predicted mean and variance of the Gaussian distribution, and then performing arithmetic encoding based on the estimated probabilities to obtain the feature binary code stream, including: The integer coefficient domain of the lattice vector quantizer is divided by a hypercube of volume 1, the hypercube filling the entire space and centered on an integer vector, such that the second feature map's... Line number The first of the columns eigenvectors is the center of its hypercube; where, , , , , and These are the height, width, and number of channels of the second feature map, respectively. The dimension of the crystal lattice; The boundary of the hypercube is used as the first feature map for estimation. Line number The first of the columns eigenvectors The probability boundary is estimated based on the mean and variance of the predicted Gaussian distribution, and the second feature map is then used to estimate the probability boundary. Line number The first of the columns eigenvectors probability ; Based on the estimated probability, the second feature map... Line number The first of the columns eigenvectors Perform arithmetic encoding to obtain the second feature map. Line number The first of the columns The bitstream of 1 feature vector; The bitstreams of all feature vectors in the second feature map are merged to obtain the feature binary bitstream.
8. A method of image decoding for vector quantization entropy modeling, for decoding the encoding obtained by the image encoding method according to any one of claims 1-7, characterized in that, include: Obtain a compressed image binary bitstream, and decompose the compressed image binary bitstream to obtain a feature binary bitstream and a priori feature binary bitstream; The arithmetic decoding of the hyperprior binary code stream is performed using nonparametric distribution estimation to obtain the hyperprior feature map; The super-prior feature map is processed by a super-prior decoding network to obtain super-prior information; The mean and variance of the Gaussian distribution for each spatial group used to decode the second feature map are predicted by combining the aforementioned prior information and the spatial context model. The integer coefficient domain of the lattice vector quantizer is divided using a relaxed boundary. The second feature map is then arithmetically decoded based on the mean and variance of the Gaussian distribution. The second feature map is obtained by spatially grouping the data in the order of spatial grouping. The second feature map is inversely projected onto the feature domain to obtain the reconstructed first feature map; the feature domain refers to the real number domain in which the first feature map is located. The reconstructed first feature map is passed through a decoding network to obtain a reconstructed image.
9. The method of image decoding of vector quantization entropy modeling according to claim 8, characterized in that, The step of inversely projecting the second feature map onto the feature domain to obtain the reconstructed first feature map includes: the first projected feature vector of the second feature map row column of the second feature map generated by the lattice of the lattice vector quantizer inverse projected into the feature space ; vector after inverse projection stitching, constituting the first feature map of the reconstruction row column of feature vectors , obtaining the first feature map of the reconstruction ; wherein, , , , , and are the height, width and channel number of the second feature map respectively, is the dimension of the lattice.
10. A method of image compression with vector quantization entropy modeling, characterized by, include: Obtain the image to be compressed; The image encoding method according to any one of claims 1-7 is used to encode the image to be compressed to obtain a compressed image binary bitstream; The compressed image binary stream is decoded using the image decoding method according to any one of claims 8-9 to obtain a reconstructed image.
11. The method of image compression of claim 10, wherein, Obtain the prior binary code stream of the first feature map and the corresponding prior information, including: A super-prior coding network is used to obtain a super-prior feature map from the first feature map; Each feature in the super-prior feature map is rounded to an integer to obtain the quantized super-prior feature map. An independent nonparametric distribution estimate is established for each channel of the quantized prior feature map; Arithmetic coding of the super-prior second feature map is performed based on the non-parametric distribution estimation to obtain the super-prior feature binary code stream; The reconstructed hyperprior feature map is obtained by arithmetic decoding of the hyperprior binary code stream using the nonparametric distribution estimation. The reconstructed prior feature map is processed by a priori decoding network to obtain prior information; The parameters of the encoding network, decoding network, super-prior encoding network, super-prior decoding network, vector quantization entropy modeling, and spatial context model are obtained through rate-distortion optimization. ； Among them, distortion The image to be compressed and reconstructed image The mean square error, These are preset Lagrange coefficients used to balance distortion and bitrate; bitrate Calculated based on the Gaussian distribution of the second feature map and the nonparametric distribution of the prior feature map: ； in , , , , and These are the height, width, and number of channels of the second feature map, respectively. The dimension of the crystal lattice. and These are the encodings of the second feature map. Line number Liede eigenvectors The mean and variance of the Gaussian distribution. It is a priori feature map No. Distribution estimation of dimension, yes Uniform distribution on This refers to summing each dimension of the prior feature map.
12. An image compression system comprising an encoder and a decoder, characterized in that, The application further provides a computer readable storage medium storing a program, wherein the program is configured to cause a processor to execute any one of the following methods: - the image encoding method according to any one of claims 1-7, or - the image decoding method according to any one of claims 8-9, or 13. An image compression apparatus comprising a memory and a processor, the memory storing a program executable by the processor, characterized in that, - the image compression method according to any one of claims 10-11.