A tensor network-based entropy coding, entropy decoding method and image compression method

Abstract

The present invention provides an entropy coding and decoding method and an image compression method, system and device based on a tensor network, including: using Gray code to divide the discrete features of an image into multiple binary bit planes; using feature mapping to map the binary bit planes into corresponding Hilbert spaces to generate vectorized features of the bit planes; based on an entropy coding model of a tensor network, respectively probabilistically estimating and encoding the vectorized features of the bit planes to generate a bit plane feature binary code stream; merging all bit plane feature binary code streams to obtain a feature binary code stream. The use of a tensor network to model feature probability distribution can accurately calculate the joint probability of features without introducing any prior assumptions, further improving the flexibility and scalability of the compression structure compared to the design of the existing entropy model.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an entropy coding and decoding method and an image compression method, system and device based on a tensor network. Background Art

[0002] With the rapid development of deep learning, end-to-end image compression methods use deep neural networks to jointly optimize the modules in traditional image compression to achieve superior compression performance. Compared with traditional image compression methods, joint optimization requires additional constraints on the bit rate, and the accuracy of entropy modeling will greatly affect the bit rate. Therefore, the design of the entropy model module has become an important topic in end-to-end image compression.

[0003] After searching the literature on the prior art, it was found that J. Ballé et al. proposed a completely decomposable entropy model for feature encoding in end-to-end image compression in the paper "End-to-end Optimized Image Compression" published at the 2017 International Conference on Learning Representation. That is, the entropy model needs to assume a priori that the elements of the features are independent of each other, and the elements of the features obey a human-designed distribution. J. Ballé et al. proposed a hyper-prior entropy model in the paper "Variational Image Compression With a Scale Hyperprior" published at the 2018 International Conference on Learning Representation. Compared with the completely decomposed entropy model, it no longer assumes that the elements of the features are independent of each other, but encodes the correlation between the features as sidelobe information for transmission to satisfy conditional independence. The hyper-prior entropy model requires a priori assumption that the features obey a certain Gaussian scale mixture model, and the hyper-prior part assumes that the elements are independent of each other. Many subsequent end-to-end image compression works have been improved based on the hyper-prior entropy model. For example, Minnen et al. proposed combining the hyper-prior entropy model with the context-based autoregressive model in the paper "Joint Autoregressive and Hierarchical Priors for Learned Image Compression" published at the 2018 Conference and Workshop on Neural Information Processing Systems, further improving the rate-distortion performance of end-to-end image compression. However, since the probability space of the joint distribution of features is too large, it is very difficult to estimate it directly. Therefore, the design of existing entropy models has artificial assumptions such as the prior independence of Gaussian distribution and the causal conditional probability of the autoregressive model, which leads to inaccurate estimation of the joint probability of image features, thus having limitations. Summary of the invention

[0004] In view of the limitations of the prior art, the present invention provides an entropy encoding, entropy decoding and image compression method based on a tensor network.

[0005] A first aspect of the present invention provides an entropy coding method based on a tensor network, comprising:

[0006] The discrete features of the image are divided into L binary bit planes using Gray code, where L is a preset positive integer, and the discrete features of the image are pixel values ​​of the image or quantized transformation coefficients obtained after the image is transformed and quantized;

[0007] Mapping the binary bit plane into a corresponding Hilbert space using feature mapping to generate vectorized features of the bit plane;

[0008] Based on the entropy coding model of the tensor network, probability estimation and encoding are performed on the vectorized features of the bit plane in sequence to generate a bit plane feature binary code stream;

[0009] All bit plane feature binary code streams are merged to obtain a feature binary code stream.

[0010] Preferably, the adopting feature mapping to map all binary bit planes into corresponding Hilbert spaces to generate vectorized features of the bit planes includes:

[0011] Using local feature mapping, each element of the bit plane is mapped to the two-dimensional Hilbert space. The specific function form of the local feature mapping is where x i ∈{0,1}, i=1,2,…,N, is the i-th element of the bit plane;

[0012] Perform tensor product operations on all the two-dimensional Hilbert spaces in the bit plane to obtain the vectorized features of the bit plane

[0013] Preferably, the entropy coding model based on the tensor network includes:

[0014] L independent bit-plane entropy coding sub-models based on tensor networks;

[0015] Each of the bit plane entropy coding sub-models adopts a quadtree structure to perform joint probability modeling on the vectorized features of the bit plane.

[0016] Preferably, the entropy coding model based on the tensor network sequentially performs probability estimation and coding on the vectorized features of the bit plane to generate a bit plane feature binary code stream, including:

[0017] Probability estimation of the vectorized features of the bit plane is performed according to the tensor network-based bit plane entropy coding sub-model to obtain an estimated probability;

[0018] According to the vectorized features of the bit plane to be encoded, combined with the estimated probability, an implicit code table is established using an entropy coding model based on a tensor network to perform binary arithmetic coding;

[0019] The bit plane feature binary code stream is obtained.

[0020] Preferably, the performing probability estimation on the vectorized features of the bit plane to obtain the estimated probability includes:

[0021] The vectorized features of the bit plane are contracted and merged with the bit plane entropy coding sub-model, and the square of the obtained scalar is an unnormalized probability value. The contraction operation sums any pair of indexes with the same dimension in the tensor according to the Einstein summation convention. One contraction operation can obtain a new tensor with two orders lower than the original tensor;

[0022] The bit plane entropy coding sub-model is contracted and combined with its own conjugate transpose to obtain a normalized coefficient;

[0023] The unnormalized probability value and the normalization coefficient are used to calculate the true joint probability P(Φ(x)) of the vectorized features of the bit plane.

[0024] Preferably, the method of establishing an implicit code table using an entropy coding model based on a tensor network and performing binary arithmetic coding includes:

[0025] The vectorized marginal data points to be calculated are contracted with the indices of the corresponding positions of the bit plane entropy coding submodel to obtain a conditional probability tensor. The conditional probability tensor is contracted with its own conjugate transpose to remove the indices except the marginal data points to be calculated. The normalization coefficient can be used to obtain a series of marginal probabilities in the bit plane P(φ(x1)=φ(0)), P(φ(x1), φ(x2)=φ(0)),…, P(φ(x1), φ(x2),…, φ(x N )=φ(0)).

[0026] According to the vectorized features of the bit plane to be encoded, the series of marginal probabilities are added according to the actual values ​​of the features to be encoded to obtain the cumulative distribution probability, which is specifically in the form of:

[0027] F(Φ(x))=P(φ(x1)=φ(0))*l φ(1) (φ(x1))+P(φ(x1),φ(x2)=φ(0))*l φ(1) (φ(x2))+…+P(φ(x1),φ(x2),…,φ(x N )=φ(0))*l φ(1) (φ ( xN))

[0028] In the above formula, “=” indicates assignment. is an indicative function.

[0029] Using the cumulative distribution probability F(Φ(x)) and the joint probability P(Φ(x)), binary arithmetic coding can be performed on the vectorized features of the bit plane.

[0030] A second aspect of the present invention provides an entropy decoding method based on a tensor network, which is used to decode the code obtained by the above method, comprising:

[0031] Decompose the characteristic binary code stream to obtain the binary code stream of each bit plane;

[0032] Using the bit plane entropy coding sub-model and the bit plane binary code stream, through arithmetic decoding, the vectorized features of each bit plane are restored from the bit plane binary code stream;

[0033] Recovering each binary bit plane from the vectorized features of each bit plane by inverse operation of feature mapping;

[0034] Through Gray code inverse operation, discrete features of the image are restored from the binary bit planes.

[0035] Preferably, the method of using the bit plane entropy coding sub-model and the bit plane binary code stream to recover the vectorized features of each bit plane from the bit plane binary code stream through arithmetic decoding includes:

[0036] Calculate the marginal probability P(φ(x1)=φ(0)) by using the bit plane entropy coding sub-model, and decode φ(x1) according to the bit plane binary code stream;

[0037] Calculate the marginal probability P(φ(x1), φ(x2)=φ(0)), and decode φ(x2) according to the bit plane binary code stream;

[0038] According to the above method, complete the following steps in turn: φ(x1), φ(x2), …, φ(x N ) and restore the vectorized features of each plane.

[0039] A third aspect of the present invention provides an image compression method based on a tensor network, comprising:

[0040] The multi-channel features of the image to be compressed are obtained by using an encoding convolutional neural network;

[0041] quantizing each feature of the multi-channel features of the image into an integer to obtain a discrete feature of the image to be compressed;

[0042] Applying the entropy coding method based on a tensor network as described in any one of claims 1 to 6 to the discrete features of the image to be compressed to obtain a feature binary code stream;

[0043] Decompressing the characteristic binary code stream using the tensor network-based entropy decoding method described in claims 7-8 to obtain reconstructed discrete features;

[0044] The reconstructed discrete features are decoded through a convolutional neural network to obtain a reconstructed image.

[0045] A fourth aspect of the present invention provides an image compression system based on a tensor network, comprising:

[0046] The forward transformation module uses a coding convolutional neural network to obtain the multi-channel features of the image to be compressed;

[0047] A quantization module, quantizing each feature of the multi-channel features of the image into an integer to obtain a discrete feature of the image to be compressed;

[0048] An entropy coding module, which uses the entropy coding method based on the tensor network to obtain a feature binary code stream for the discrete features of the image to be compressed;

[0049] An entropy decoding module decompresses the feature binary code stream using the entropy decoding method based on the tensor network to obtain reconstructed discrete features;

[0050] The inverse transformation module decodes the reconstructed discrete features through a convolutional neural network to obtain a reconstructed image.

[0051] A fifth aspect of the present invention provides an image compression device based on a tensor network, which uses any of the following methods to perform image compression:

[0052] - the above mentioned tensor network based entropy coding method, or,

[0053] - the above mentioned tensor network based entropy decoding method, or,

[0054] -The above-mentioned tensor network-based image compression method.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The tensor network-based entropy coding, entropy decoding and image compression methods of the embodiments of the present invention utilize tensor networks to model the joint probability of image features. The joint probability of image features can be estimated without making any artificial assumptions such as the prior independence of image features, the parameter form obeyed by the joint probability distribution, and the causal conditional probability of autoregression. Compared with the design of existing entropy models, the flexibility and scalability of the compression structure are further improved.

[0057] The tensor network-based entropy coding, entropy decoding and image compression methods of the embodiments of the present invention can realize the calculation of accurate image feature joint probabilities through tensor contraction, and compared with the fully decomposable benchmark method, can further improve the rate-distortion performance while maintaining the same reconstructed image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0059] Figure 1 A flowchart of entropy coding based on a tensor network provided in one embodiment of the present invention;

[0060] Figure 2 A flowchart of a bit plane entropy coding sub-model based on a tensor network provided in a preferred embodiment of the present invention;

[0061] Figure 3 A flowchart of entropy decoding based on a tensor network provided in a preferred embodiment of the present invention;

[0062] Figure 4 A flowchart of a tensor network-based image compression method provided in a preferred embodiment of the present invention;

[0063] Figure 5 This is a flow chart of a training method for a tensor network-based image compression system provided in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0065] Example 1

[0066] Figure 1 Flow chart of an entropy coding method in one embodiment of the present invention. The entropy coding method based on a tensor network in this embodiment includes:

[0067] S410: Gray code conversion, using Gray code to divide the discrete features of the image into L binary bit planes, where L is a preset positive integer.

[0068] S420: Feature mapping: Mapping the binary bit plane into a corresponding Hilbert space using feature mapping to generate vectorized features of the bit plane;

[0069] S430: tensor network modeling probability distribution: based on the entropy coding model of the tensor network, probability estimation and coding are performed on the vectorized features of the bit plane in sequence to generate a bit plane feature binary code stream;

[0070] S440: Code stream synthesis: merging all the bit plane feature binary code streams to obtain a feature binary code stream.

[0071] In this embodiment, the joint probability of image features is modeled using a tensor network. The joint probability of image features can be estimated without making any artificial assumptions such as the prior independence of image features, the parameter form of the joint probability distribution, and the causal conditional probability of autoregression. Compared with the design of existing entropy models, the flexibility and scalability of the compression structure are further improved.

[0072] In a preferred embodiment of the present invention, the discrete features of the image in S410 are pixel values ​​of the image or quantized transformation coefficients obtained after the image is transformed and quantized.

[0073] In a preferred embodiment of the present invention, S420 is implemented, feature mapping: local feature mapping is used for the L binary bit planes generated in S410 in For the i-th element of the l-th bit plane, map each element of each binary bit plane into the two-dimensional Hilbert space; perform tensor product operations on all two-dimensional Hilbert spaces in each binary bit plane to obtain the vectorized features of L bit planes

[0074] In a preferred embodiment of the present invention, S430 is implemented to model the probability distribution of the tensor network: for the vectorized features of the L bit planes generated in S420, L independent bit plane entropy coding sub-models based on the tensor network are respectively established, and each bit plane entropy coding sub-model based on the tensor network adopts a quadtree structure to perform probability estimation and coding on the vectorized features of each bit plane to generate a bit plane feature binary code stream. Specifically, Figure 2 As shown, the tensor network-based bit plane entropy coding sub-model is used to perform probability estimation and coding on the vectorized features of each bit plane, including:

[0075] S431: Estimation of joint probability of bit planes: perform contraction operation on all corresponding indexes of the vectorized features of the L bit planes generated by S420 and the L bit plane entropy coding sub-models established by S430, and obtain L scalars. The squares of the L scalars are the unnormalized joint probability values ​​of the L bit planes. The contraction operation refers to summing any pair of indexes with the same dimension in the tensor according to the Einstein summation convention. One contraction operation can obtain a new tensor that is two orders lower than the original tensor; perform contraction operation on the L bit plane entropy coding sub-models established by S430 and their own conjugate transpose to obtain L normalization coefficients; using the unnormalized joint probability values ​​of the L bit planes and the L normalization coefficients, the true joint probability P of the vectorized features of the L bit planes can be accurately calculated. l (Φ(x l )), l = 1, 2, ... L;

[0076] S432: Bit plane marginal probability estimation: The vectorized marginal data points to be calculated are shrunk and merged with the indexes of the corresponding positions of the L bit plane entropy coding sub-models based on the tensor network established in S430 to obtain a conditional probability tensor. The conditional probability tensor is shrunk and merged with its own conjugate transpose to remove the indexes other than the marginal data points to be calculated. The L normalization coefficients generated in S431 can be used to obtain a series of marginal probabilities in the L bit planes.

[0077] S433: Bit plane cumulative probability estimation: Based on the vectorized features of the L bit planes generated in S420, a series of marginal probabilities in the L bit planes generated in S432 are added according to the actual values ​​of the features to be encoded to obtain the cumulative distribution probability F l (Φ(x l )), l = 1, 2, ... L, the specific form is:

[0078]

[0079] In the above formula, “=” indicates assignment. is the indicative function;

[0080] S434: Bit plane coding: Utilize the joint probability P of the vectorized features of the L bit planes generated in S431 l (Φ(x l )) and the cumulative distribution probability F generated by S433 l (Φ(x l )), where l = 1, 2, ... L, binary arithmetic coding is performed on the vectorized features of L bit planes to generate a bit plane feature binary code stream.

[0081] In a preferred embodiment of the present invention, S440 is implemented, code stream synthesis: all bit plane feature binary code streams generated in S434 are merged to obtain a feature binary code stream.

[0082] Example 2

[0083] See also Figure 3 Based on the same inventive concept, another embodiment of the present invention provides an entropy decoding method, including:

[0084] S510: code stream decomposition: decomposing the characteristic binary code stream obtained by the above-mentioned tensor network-based entropy coding method to obtain L bit plane binary code streams;

[0085] S520: Feature decoding: using the bit plane entropy coding sub-model and the L bit plane binary code streams, through arithmetic decoding, recovering the vectorized features of the L bit planes from the L bit plane binary code streams;

[0086] S530: feature mapping inverse operation: since the feature mapping in S420 is a one-to-one mapping, the feature mapping inverse operation is performed on the vectorized features of the L bit planes obtained in S520, so that the L binary bit planes can be uniquely restored;

[0087] S540: Gray code inverse operation: Gray code inverse operation is performed on the L binary bit planes obtained in S530 to restore the discrete features of the image.

[0088] The entropy decoding method of this embodiment is used to correspond to Figure 1 Similarly, by using tensor networks to model the joint probability distribution of discrete features of images, the joint probability of image features can be estimated and encoded without any prior assumptions such as the independence of discrete features of images, the parameter form of the joint probability distribution, and the causal conditional probability of autoregression, thereby improving the flexibility and scalability of the compression structure.

[0089] In a preferred embodiment of the present invention, S520 is implemented. Figure 2 The structure and parameters of the bit plane entropy coding sub-model based on the tensor network are completely consistent. Specifically, the L bit plane entropy coding sub-models based on the tensor network established in S430 are used to calculate the marginal probabilities respectively. And according to the binary code stream of L bit planes obtained in S510, decode each bit plane Calculate the marginal probability separately And according to the binary code stream of L bit planes obtained in S510, decode According to the above method, the L bit planes are respectively completed in sequence. Decoding of , where l = 1, 2, ... L, recovers the vectorized features of L bit planes;

[0090] Example 3

[0091] See also Figure 4 Based on the same inventive concept, in other embodiments of the present invention, a tensor network-based image compression method is provided, comprising:

[0092] S100: Acquire an image to be compressed;

[0093] S200: using a coding convolutional neural network to obtain multi-channel features of the image to be compressed;

[0094] S300: quantizing each feature in the multi-channel features of the image to be compressed into an integer to obtain a discrete feature of the image to be compressed;

[0095] S400: using a tensor network to compress discrete features of the image to obtain a feature binary code stream; wherein the above-mentioned entropy coding method based on the tensor network is used to compress the discrete features of the image; specifically, S410-S440 in Example 1 can be used to implement;

[0096] S500: Decompress the feature binary code stream using a tensor network to obtain reconstructed discrete features of the image; wherein the feature binary code stream is decompressed using the above-mentioned tensor network-based entropy decoding method; specifically, S510-S540 shown in Example 2 can be used to implement it.

[0097] S600: Using a decoding convolutional neural network on discrete features of the reconstructed image to obtain a reconstructed image.

[0098] Example 4

[0099] Based on the same inventive concept, in another embodiment of the present invention, a tensor network-based image compression system is provided, the system comprising:

[0100] Forward transformation module: uses the encoding convolutional neural network to obtain the multi-channel features of the image to be compressed;

[0101] Quantization module: quantizes each feature in the multi-channel features of the image into an integer to obtain the discrete features of the image to be compressed;

[0102] Entropy coding module: The entropy coding method based on tensor network is used to obtain the characteristic binary code stream for the discrete features of the compressed image;

[0103] Entropy decoding module: decompresses the feature binary code stream using an entropy decoding method based on a tensor network to obtain reconstructed discrete features;

[0104] Inverse transformation module: The reconstructed discrete features are decoded through a convolutional neural network to obtain a reconstructed image.

[0105] Example 5

[0106] Based on the same inventive concept, in this embodiment, the encoding convolutional neural network and the decoding convolutional neural network in the above embodiments 3 and 4 are simplified to an identical transformation, and the quantization operation is cancelled to realize a lossless image compression method and system for the image itself.

[0107] Example 6

[0108] Based on the same inventive concept, in this embodiment, a training method for an image compression model based on a tensor network is provided, and the process is as follows: Figure 5In this embodiment, the training set uses 20,000 images from the Flicker dataset. The trained image compression model can be used for actual encoding and decoding. Figure 5 As shown, including:

[0109] S1: Feature map extraction: The images in the Flicker dataset are cropped into 512×768 size and input into the encoding convolutional neural network to obtain the multi-channel features of the image to be compressed;

[0110] S2: Feature derivative quantization: For the multi-channel features of the image to be compressed generated by S1, the features are rounded off and quantized to generate discrete features of the image; since the gradient of the quantization operation is almost always 0, in order to allow the gradient of the quantization operation of the super-prior information feature to be back-propagated during training, the gradient of this quantization operation is set to be 1 during back-propagation;

[0111] S3: Image reconstruction: The discrete features of the image generated by S2 are decoded using a convolutional neural network to obtain a reconstructed image;

[0112] S4: Gray code conversion: Gray code conversion is used to divide the discrete features of the quantized image generated by S2 into L binary bit planes, where L is a preset positive integer;

[0113] S5: Feature Mapping: Local feature mapping is used for the L binary bit planes generated by S4 in For the i-th element of the l-th bit plane, map each element of each binary bit plane to the two-dimensional Hilbert space; perform tensor product operations on all two-dimensional Hilbert spaces in each binary bit plane to obtain the vectorized features of L bit planes

[0114] S6: Bit plane joint probability estimation: For the vectorized features of the L bit planes generated by S5, L independent bit plane entropy coding sub-models based on tensor networks are established respectively, and each bit plane entropy coding sub-model based on tensor networks adopts a quadtree structure; the vectorized features of the L bit planes generated by S5 and the established L bit plane entropy coding sub-models are respectively subjected to contraction operations on all corresponding indexes, and the obtained results are L scalars, and the squares of the L scalars are respectively the unnormalized joint probability values ​​of the L bit planes. The contraction operation refers to the summation of any pair of indexes with the same dimension in the tensor according to the Einstein summation convention. A contraction operation can obtain a new tensor that is two orders lower than the original tensor; the established L bit plane entropy coding sub-models are contracted and contracted with their own conjugate transpose to obtain L normalization coefficients; using the unnormalized joint probability values ​​of the L bit planes and the L normalization coefficients, the true joint probability P of the vectorized features of the L bit planes can be accurately calculated. l (Φ(x l )), l = 1, 2, ... L;

[0115] S7: Total bit rate estimation: The bit rate is calculated by the joint probability of the vectorized features of the L bit planes obtained in S6 and added to obtain the total bit rate estimation;

[0116] S8: Model training: Using the training data set, the back gradient propagation algorithm is used to learn the L tensor network models established in S5; the bit rate required for the vectorized feature encoding of the bit plane is estimated to be Among them, P l Φ(x l )) is the joint probability of the vectorized features of the l-th bit plane generated by S6, represents the training set, is the size of the training set; the loss function of the probability model based on the tensor network is defined as L = L BPP , where the encoding convolutional neural network and the decoding convolutional neural network use pre-trained models, fix their parameters, and do not perform gradient optimization;

[0117] Example 7

[0118] The tensor network model obtained through training is used in an image compression system and acts on the image encoding and decoding methods in the following embodiments.

[0119] Specifically, in an embodiment of an image encoding method, the Kodak test set image is compressed using the image encoder in the image compression system obtained through the above training to obtain a binary code stream of the compressed image. The specific implementation steps are as follows:

[0120] Step 1: Feature map extraction: Input the Kodak test set image and obtain the multi-channel features of the image to be compressed through the encoding convolutional neural network.

[0121] Step 2: Features can be quantized: For the multi-channel features of the image to be compressed generated in step 1, they are quantized by rounding to generate discrete features of the image.

[0122] Step 3: Gray code conversion: Gray code conversion is used to divide the discrete features of the image generated in step 2 into L binary bit planes, where L is a preset positive integer.

[0123] Step 4: Feature Mapping: Use local feature mapping on the L binary bit planes generated in step 3 in For the i-th element of the l-th bit plane, map each element of each binary bit plane to the two-dimensional Hilbert space; perform tensor product operations on all two-dimensional Hilbert spaces in each binary bit plane to obtain the vectorized features of L bit planes

[0124] Step 5: Estimation of the joint probability of bit planes: The vectorized features of the L bit planes generated in step 4 are shrunk and merged with the L trained bit plane entropy coding sub-models based on tensor networks respectively, and the result is L scalars, the squares of which are the unnormalized joint probability values ​​of the L bit planes; the trained L bit plane entropy coding sub-models are shrunk and merged with their own conjugate transpose to obtain L normalized coefficients; using the unnormalized joint probability values ​​of the L bit planes and the L normalized coefficients, the true joint probability P of the vectorized features of the L bit planes can be accurately calculated. l (Φ(x l )), l=1,2,…L.

[0125] Step 6: Bit plane marginal probability estimation: The vectorized marginal data points to be calculated are shrunk and merged with the indexes of the corresponding positions of the L trained tensor network-based bit plane entropy coding sub-models to obtain the conditional probability tensor. The conditional probability tensor is shrunk and merged with its own conjugate transpose to remove the indexes except the marginal data points to be calculated. The L normalized coefficients generated in step 5 can be used to obtain a series of marginal probabilities in the L bit planes.

[0126] Step 7: Based on the vectorized features of the L bit planes generated in step 4, add the series of marginal probabilities in the L bit planes generated in step 6 according to the actual value of the feature to be encoded to obtain the cumulative distribution probability F l (Φ(xl )), l = 1, 2, ... L, the specific form is:

[0127]

[0128] In the above formula, “=” indicates assignment. is an indicative function.

[0129] Step 8: Bit plane coding: Use the joint probability P of the vectorized features of the L bit planes generated in step 5 l (Φ(x l )) and the cumulative distribution probability F generated in step 7 l (Φ(x l )), where l = 1, 2, ... L, binary arithmetic coding is performed on the vectorized features of L bit planes to generate a bit plane feature binary code stream.

[0130] Step 9: Code stream synthesis: Merge all bit plane feature binary code streams generated in step 8 to obtain a feature binary code stream.

[0131] Example 8

[0132] In another embodiment of the image decoding method, the image decoder in the image compression system obtained by the above training is used to decompress the binary code stream generated by compressing the Kodak test set image to obtain a reconstructed image. The specific implementation steps are:

[0133] Step 1: Bitstream decomposition: Decompose the binary bitstream generated by compressing the Kodak test set image to obtain a binary bitstream of L bit planes.

[0134] Step 2: Feature decoding: Use the trained L bit plane entropy coding sub-models based on the tensor network to calculate the marginal probability And according to the binary code stream of L bit planes obtained in step 1, decode each bit plane Calculate the marginal probability separately And according to the binary code stream of L bit planes obtained in step 1, decode According to the above method, the L bit planes are completed in sequence. The decoding of , where l = 1, 2, ... L, recovers the vectorized features of L bit planes.

[0135] Step 3: Inverse feature mapping operation: Perform an inverse feature mapping operation on the vectorized features of the L bit planes obtained in step 2 to restore L binary bit planes.

[0136] Step 4: Gray code inverse operation: Perform Gray code inverse operation on the L binary bit planes obtained in step 3 to restore the discrete features of the image.

[0137] Step 5: Image reconstruction: Use a decoding convolutional neural network to decode the discrete features of the image obtained in step 4 to obtain a reconstructed image.

[0138] Example 9

[0139] Based on the same inventive concept, in this embodiment, a tensor network-based image compression device is also provided, which uses any of the following methods to perform image compression:

[0140] - the above mentioned tensor network based entropy coding method, or,

[0141] - the above mentioned tensor network based entropy decoding method, or,

[0142] -The above-mentioned tensor network-based image compression method.

[0143] Implementation effect:

[0144] In Example 4, the Flicker dataset is used to train the image compression model based on the tensor network. The virtual key dimension in the tensor network is set to 7, where the virtual key is the index to be summed when the tensor network performs a contraction operation. The number of bit planes L is set to 8. The encoding convolutional neural network and the decoding convolutional neural network use pre-trained models and fixed parameters. The Adam optimizer is used at a learning rate of 10 -4 After 1,000,000 steps of training, the model converged and training stopped.

[0145] In Example 4, 24 images in the Kodak test set are compressed using the image encoders of the 8 trained tensor network-based image compression models. When five pre-trained encoding convolutional neural networks and decoding convolutional neural networks with different rate distortions are used, the average bit rates of the 24 compressed images are 0.1149 bpp (bits per pixel), 0.1786 bpp, 0.2749 bpp, 0.4311 bpp, and 0.6229 bpp, respectively.

[0146] In Example 4, the binary code stream generated by compressing 24 images of the Kodak test set is decompressed using the image decoders of the 8 trained tensor network-based image compression models to obtain reconstructed images. When five pre-trained encoding convolutional neural networks and decoding convolutional neural networks with different rate distortions are used, the average peak signal-to-noise ratios (PSNRs) of the reconstructed 24 images are 26.9165 dB (decibels), 28.2326 dB, 29.6157 dB, 31.2678 dB, and 32.9450 dB, respectively.

[0147] In Example 5, the image compression model based on the tensor network is trained using 50,000 training set grayscale images in the MNIST data set, wherein the encoding convolutional neural network and the decoding convolutional neural network are simplified to the identity transformation, and the quantization operation is canceled. The virtual key dimensions in the tensor network are set to 1, 2, 3, 4, 5, 6 and 7, respectively, and the number of bit planes L is set to 8. The Adam optimizer is first used at a learning rate of 10 -4 Train for 1,000,000 steps, then reduce the learning rate to 10 -5 After 1500000 steps of training, the model converged and training stopped.

[0148] In Example 5, 10,000 test set grayscale images in the MNIST data set are compressed using image encoders of 8 trained tensor network-based image compression models. When the virtual key dimensions in the tensor network are 1, 2, 3, 4, 5, 6, and 7, the average bit rates of the 10,000 compressed images are 1.89 bpp (bits per pixel), 1.60 bpp, 1.44 bpp, 1.39 bpp, 1.48 bpp, 1.41 bpp, and 1.55 bpp, respectively, and the optimal bit rate of 1.39 bpp is achieved when the virtual key dimension is 4; when ordinary binary conversion is used instead of Gray code conversion, the optimal bit rate is 1.59 bpp.

[0149] In addition, compared with the benchmark method, the tensor network-based image compression method provided by the present invention can further improve the rate-distortion performance while reconstructing the same image quality without making any artificial assumptions such as a priori independence and autoregressive causal conditional probability on image features. Specifically, when the virtual key dimension is set to 7 and 4, Example 4 and Example 5 can save 9.8% and 6.1% of the bitstream overhead respectively.

[0150] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. An entropy coding method based on a tensor network, characterized in that: include: Gray code is used to divide the discrete features of the image into L binary bit planes, where L is a preset positive integer. Mapping the binary bit plane into a corresponding Hilbert space using feature mapping to generate vectorized features of the bit plane; Based on the entropy coding model of the tensor network, probability estimation and encoding are performed on the vectorized features of the bit plane in sequence to generate a bit plane feature binary code stream; All the bit plane feature binary code streams are combined to obtain a feature binary code stream.

2. The entropy coding method based on a tensor network according to claim 1, characterized in that: The feature mapping is used to map all binary bit planes into the corresponding Hilbert space to generate vectorized features of the bit planes, including: Using local feature mapping, each element of the binary bit plane is mapped to the two-dimensional Hilbert space. The function form of the local feature mapping is where x i ∈{0,1}, i=1,2,…,N, is the i-th element of the binary bit plane; Performing a tensor product operation on all the two-dimensional Hilbert spaces in the binary bit plane to obtain a vectorized feature of the bit plane 3. The entropy coding method based on tensor network according to claim 1, characterized in that: The entropy coding model based on the tensor network includes: L independent bit-plane entropy coding sub-models based on tensor networks; Each of the bit plane entropy coding sub-models adopts a quadtree structure to perform joint probability modeling on the vectorized features of the bit plane.

4. The entropy coding method based on tensor network according to claim 3, characterized in that: The entropy coding model based on the tensor network sequentially performs probability estimation and coding on the vectorized features of the bit plane to generate a bit plane feature binary code stream, including: Probability estimation of the vectorized features of the bit plane is performed according to the tensor network-based bit plane entropy coding sub-model to obtain an estimated probability; According to the vectorized features of the bit plane to be encoded, combined with the estimated probability, an implicit code table is established using an entropy coding model based on a tensor network to perform binary arithmetic coding; The bit plane feature binary code stream is obtained.

5. The entropy coding method based on tensor network according to claim 4, characterized in that: The performing probability estimation on the vectorized features of the bit plane to obtain the estimated probability includes: The vectorized feature of the bit plane is contracted with the bit plane entropy coding sub-model, and the square of the obtained scalar is an unnormalized probability value; Performing a contraction operation on the bit plane entropy coding sub-model and its own conjugate transpose to obtain a normalized coefficient; The unnormalized probability value and the normalization coefficient are used to calculate the true joint probability P(Φ(x)) of the vectorized features of the bit plane.

6. The entropy coding method based on tensor network according to claim 5, characterized in that: The method of establishing an implicit code table by using an entropy coding model based on a tensor network and performing binary arithmetic coding includes: The vectorized marginal data points to be calculated are contracted with the indices of the corresponding positions of the bit plane entropy coding submodel to obtain a conditional probability tensor, and the conditional probability tensor is contracted with its own conjugate transpose to remove the indices except the marginal data points to be calculated. The normalization coefficient can be used to obtain a series of marginal probabilities in the bit plane P(φ(x1)=φ(0)), P(φ(x1), φ(x2)=φ(0)),…, P(φ(x1), φ(x2),…, φ(x N )=φ(0)); According to the vectorized features of the bit plane to be encoded, the series of marginal probabilities are added according to the actual values ​​of the features to be encoded to obtain the cumulative distribution probability, which is specifically in the form of: F(Φ(x))=P(φ(x1)=φ(0))*l φ(1) (φ(x1))+P(φ(x1),φ(x2)=φ(0))*l φ(1) (φ(x2))+…+P(φ(x1),φ(x2),…,φ(x N )=φ(0))*l φ(1) (φ(x N )) In the formula, "=" indicates assignment. is the indicative function; Using the cumulative distribution probability F(Φ(x)) and the joint probability P(Φ(x)), binary arithmetic coding can be performed on the vectorized features of the bit plane.

7. An entropy decoding method based on a tensor network, characterized in that: include: Decomposing the characteristic binary code stream obtained by the tensor network-based entropy coding method according to any one of claims 1 to 6 to obtain a bit plane binary code stream; Using the bit plane entropy coding sub-model and the bit plane binary code stream, through arithmetic decoding, the vectorized features of each bit plane are restored from the bit plane binary code stream; Recovering each binary bit plane from the vectorized features of each bit plane by inverse operation of feature mapping; Through Gray code inverse operation, discrete features of the image are restored from the binary bit planes.

8. The entropy decoding method based on tensor network according to claim 7, characterized in that: The method of using the bit plane entropy coding sub-model and the bit plane binary code stream to recover the vectorized features of each bit plane from the bit plane binary code stream through arithmetic decoding includes: Calculate the marginal probability P(φ(x1)=φ(0)) by using the bit plane entropy coding sub-model, and decode φ(x1) according to the bit plane binary code stream; Calculate the marginal probability P(φ(x1), φ(x2)=φ(0)), and decode φ(x2) according to the bit plane binary code stream; Complete the following steps in sequence: φ(x1), φ(x2),…, φ(x N ) and restore the vectorized features of each plane.

9. A tensor network-based image compression method, characterized in that: include: The multi-channel features of the image to be compressed are obtained by using an encoding convolutional neural network; quantizing each feature of the multi-channel features of the image into an integer to obtain a discrete feature of the image to be compressed; Applying the entropy coding method based on a tensor network as described in any one of claims 1 to 6 to the discrete features of the image to be compressed to obtain a feature binary code stream; Decompressing the characteristic binary code stream using the tensor network-based entropy decoding method described in claims 7-8 to obtain reconstructed discrete features; The reconstructed discrete features are decoded through a convolutional neural network to obtain a reconstructed image.

10. An image compression system based on a tensor network, characterized in that: include: The forward transformation module uses a coding convolutional neural network to obtain the multi-channel features of the image to be compressed; A quantization module, quantizing each feature of the multi-channel features of the image into an integer to obtain a discrete feature of the image to be compressed; An entropy coding module, which uses the entropy coding method based on the tensor network to obtain a feature binary code stream for the discrete features of the image to be compressed; An entropy decoding module decompresses the feature binary code stream using the entropy decoding method based on the tensor network to obtain reconstructed discrete features; The inverse transformation module decodes the reconstructed discrete features through a convolutional neural network to obtain a reconstructed image.

11. An image compression device based on a tensor network, characterized in that: Use any of the following methods to compress the image: - the entropy coding method according to any one of claims 1 to 6, or, - the entropy decoding method according to any one of claims 7 to 8, or, - The image compression method according to claim 9.

Images