Integer implemented image encoding, decoding, compression method and system, terminal

Abstract

The present disclosure provides an integer implemented image encoding, decoding and compression method and system, terminal, wherein the method comprises: inputting a to-be-compressed image into a preset integerized encoding network to determine an integer feature map; inputting the integer feature map into a preset integerized hyper-prior encoding network to determine an integer hyper-prior feature map and a hyper-prior feature binary code stream by using non-parametric distribution estimation; inputting the integer hyper-prior feature map into a preset integerized hyper-prior decoding network to determine Gaussian distribution parameters of the integer feature map; encoding the integer feature map by using a preset entropy encoder to determine a feature map binary code stream; and merging the hyper-prior feature binary code stream and the feature map binary code stream to determine a compressed image binary code stream. By the present disclosure, the complexity of image encoding and image decoding is reduced without loss of rate-distortion performance, the running efficiency of image compression is improved, and resource consumption is saved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular, to an integer-implementation image encoding, decoding and compression method and a compression system and terminal. BACKGROUND

[0002] In recent years, deep learning technology has been widely applied in the field of image compression and image encoding and decoding. In 2017, Ballé et al. published an end-to-end optimizable image compression framework based on convolutional neural network in the paper "End-to-end Optimized Image Compression" in the International Conference on Learning Representations (ICLR) conference. The image compression is modeled as a rate-distortion optimization problem using a convolutional neural network, which can jointly optimize each module. On this framework, a series of end-to-end trainable image compression models are formed. These end-to-end learnable image compression models based on convolutional neural network show superior rate-distortion performance compared to traditional compression methods.

[0003] However, the full-precision neural network contains a large number of floating-point parameters, which leads to high complexity, high energy consumption, long time consumption and large storage space during network calculation, thereby limiting the practical deployment of end-to-end image compression models on hardware. In addition, the parallel calculation of floating-point numbers in the forward inference of floating-point number model has uncertainty, and the same input may lead to different calculation results, which is particularly obvious in cross-platform operation. This uncertainty will cause the full-precision end-to-end image compression model to predict different code stream distributions on different platforms, ultimately leading to decoding errors and image reconstruction failure. The full-integer network can avoid the above problems, reduce the calculation complexity and storage space of the network, and effectively eliminate the uncertainty of network operation. Although some works have studied the integerization of weight parameters, the full-integer quantization of the entire network has not been truly realized. SUMMARY

[0004] In view of the defects in the prior art, the purpose of the present disclosure is to provide an integer-implementation image encoding, decoding and compression method and system and terminal.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present disclosure, an integer-implementation image encoding method is provided, comprising:

[0006] inputting a to-be-compressed image into a preset integer-implementation encoding network to determine an integer feature map;

[0007] input the integer feature map into a preset integerized hyper-prior encoding network, determine an integer hyper-prior feature map and a hyper-prior feature binary code stream by using non-parametric distribution estimation;

[0008] input the integer hyper-prior feature map into a preset integerized hyper-prior decoding network, and determine the Gaussian distribution parameters of the integer feature map;

[0009] According to the Gaussian distribution parameters of the integer feature map, the preset entropy encoder is used for encoding processing of the integer feature map, and the feature Figure Two binary code stream is determined.

[0010] The hyper-prior feature binary code stream and the feature Figure Two binary code stream are merged to determine the encoded compressed image binary code stream.

[0011] Optionally, the method for determining the preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network comprises:

[0012] The outputs of the activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network are balanced in the channel dimension to determine that the output of each activation function has the same maximum value in each channel.

[0013] The network layers containing parameters in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior encoding network are integerized.

[0014] The integer weight parameters quantized to n w bit are mapped to integer weight parameters of n l bit, wherein the n l bit is less than the n w bit.

[0015] The balanced outputs of the activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior encoding network are integerized to determine the preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network.

[0016] Optionally, the output of the activation function in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network is equalized in the channel dimension, and it is determined that the output of the activation function has the same maximum value in each channel, including:

[0017] In the process of compressing the to-be-compressed image by using the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network, the maximum value of the output of each activation function in different channels is obtained;

[0018] Starting from the first network layer containing parameters, each output channel of the current convolutional layer parameter matrix is divided by the maximum value of the output of the activation function corresponding to the output channel, and each input channel of the next convolutional layer is multiplied by the maximum value of the output of the activation function corresponding to the input channel, so that it is determined that the output of the activation function has the same maximum value in each channel

[0019] Optionally, the integer weight parameter quantized to n w bits is mapped to an integer weight parameter of n l bits, including:

[0020] The integer weight parameter quantized to n w bits is mapped to an integer weight parameter of n l bits by using a nonlinear mapping function.

[0021] Optionally, the output of the equalized activation function in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior encoding network is integerized, and the preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network are determined, including:

[0022] The integer weight parameter of n l bits is restored to an integer weight parameter of n w bits;

[0023] The integer weight parameter of n w bits is subjected to matrix multiplication operation or convolution operation with the integer activation value quantized by the integerization operation of the output of the previous activation function, and an operation result is determined.

[0024] The operation result is summed with a bias term quantized to an integer, and an activation value is obtained through the activation function.

[0025] The activation value is multiplied by a preset scale factor, and the result of the multiplication is quantized to obtain an integer activation value, which is used for a next matrix multiplication or convolution operation.

[0026] According to a second aspect of the present disclosure, there is provided an integer- implemented image decoding method for decoding an image encoding obtained by the first aspect of the present disclosure, comprising:

[0027] obtaining an encoded compressed image binary code stream;

[0028] performing a decomposition process on the compressed image binary code stream to determine a hyper-prior feature binary code stream and a feature Figure Two binary code stream;

[0029] performing arithmetic decoding on the hyper-prior feature binary code stream using a non-parametric distribution estimation to determine an integer hyper-prior feature map;

[0030] inputting the integer hyper-prior feature map into a preset integer hyper-prior decoding network to determine Gaussian distribution parameters of an integer feature map;

[0031] based on the Gaussian distribution parameters of the integer feature map, performing decoding processing on the feature Figure Two binary code stream using a preset entropy decoder to determine the integer feature map;

[0032] inputting the integer feature map into a preset integer decoding network to determine a reconstructed image.

[0033] Optionally, the preset integer decoding network, the preset integer hyper-prior encoding network, and the preset integer hyper-prior decoding network are respectively obtained based on a preset full-precision floating-point encoding network, a preset full-precision floating-point hyper-prior encoding network, and a preset full-precision floating-point hyper-prior decoding network.

[0034] According to a third aspect of the present disclosure, there is provided an integer- implemented image compression method, comprising:

[0035] obtaining an image to be compressed;

[0036] encoding the image to be compressed based on any one of the integer- implemented image encoding methods provided by the first aspect of the present disclosure to obtain a compressed image binary code stream;

[0037] decoding the compressed image binary code stream based on any one of the integer- implemented image decoding methods provided by the second aspect of the present disclosure to obtain a reconstructed image.

[0038] According to a fourth aspect of the present disclosure, there is provided an integer- implemented image compression system, comprising:

[0039] an encoder configured to encode an image to be encoded by using any one of the integer implemented image encoding methods provided in the first aspect of the present disclosure, and obtain a compressed image binary code stream;

[0040] a decoder configured to decode the compressed image binary code stream by using any one of the integer implemented image decoding methods provided in the second aspect of the present disclosure, and obtain a reconstructed image.

[0041] According to a fifth aspect of the present disclosure, a terminal is provided, comprising a memory and a processor, the memory stores a program executed by the processor, and the processor can execute any one of the following methods when executing the program:

[0042] the integer implemented image encoding method provided in any one of the first aspect of the present disclosure, or

[0043] the integer implemented image decoding method provided in any one of the second aspect of the present disclosure, or

[0044] the integer implemented image compression method provided in the third aspect of the present disclosure.

[0045] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0046] Through the above technical solution, by using the preset integerized encoding network, the preset integerized hyper-prior encoding network, the preset integerized hyper-prior decoding network and the preset entropy encoder, the integer implemented image encoding is realized, the demand for the computing power and the storage resources of the hardware is reduced, thereby reducing the complexity of image encoding, improving the running efficiency of image compression, saving resource consumption, and facilitating deployment in mobile devices, embedded system terminals and resource energy consumption limited scenarios.

[0047] The embodiments of the present disclosure, by using the preset integerized decoding network, the preset integerized hyper-prior encoding network, the preset integerized hyper-prior decoding network and the preset entropy decoder, realize the integer implemented image decoding, improve the decoding stability, and avoid decoding inconsistency caused by different device floating point calculations.

[0048] The embodiments of the present disclosure, based on the integer implemented image encoding and the integer implemented image decoding, realize the integer implemented image compression method, which can speed up the encoding and decoding speed and reduce the encoding and decoding delay without significant rate distortion performance loss. BRIEF DESCRIPTION OF DRAWINGS

[0049] Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of the non-limiting embodiments, made with reference to the accompanying drawings:

[0050] Figure 1 FIG. 1 is a flowchart of an integer implemented image encoding method according to an example embodiment.

[0051] Figure 2 FIG. 2 is a flowchart of another integer implemented image encoding method according to an example embodiment.

[0052] Figure 3 FIG. 3 is a flowchart of a method for obtaining a preset integerized encoding network, a preset integerized hyper-prior encoding network, and a preset integerized hyper-prior decoding network according to an example embodiment.

[0053] Figure 4 FIG. 4 is a flowchart of a network layer containing parameters for integerization according to an example embodiment.

[0054] Figure 5 FIG. 5 is a flowchart of an integer implemented image decoding method according to an example embodiment.

[0055] Figure 6 FIG. 6 is a flowchart of another integer implemented image decoding method according to an example embodiment.

[0056] Figure 7 FIG. 7 is a flowchart of an integer implemented image compression method according to an example embodiment.

[0057] Figure 8 FIG. 8 is a block diagram of an integer implemented image compression system according to an example embodiment. DETAILED DESCRIPTION

[0058] The present disclosure will be described in detail below with reference to specific embodiments. The following embodiments will facilitate further understanding of the present disclosure for those skilled in the art, but do not limit the present disclosure in any form. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all belong to the protection scope of the present disclosure.

[0059] Figure 1 FIG. 1 is a flowchart of an integer implemented image encoding method according to an example embodiment. Figure 2 FIG. 2 is a flowchart of another integer implemented image encoding method according to an example embodiment.

[0060] As shown in FIGS. 1 and 2, the present disclosure provides an integer implemented image encoding method, which includes S11 to S15. Figure 1 Figure 2 As shown in FIGS. 1 and 2, the present disclosure provides an integer implemented image encoding method, which includes S11 to S15.

[0061] ​S11, input the to-be-compressed image into a preset integerized encoding network to determine an integer feature map.

[0062] S12, input the integer feature map into a preset integerized hyper-prior encoding network to determine an integer hyper-prior feature map and a hyper-prior feature binary code stream by using a non-parametric distribution estimation.

[0063] First, the integer feature map is input into the preset integerized hyper-prior encoding network to extract an integer hyper-prior feature map; second, a non-parametric distribution estimation is established in each channel of the integer hyper-prior feature map, and the integer hyper-prior feature map is arithmetically encoded by using the non-parametric distribution estimation to determine a hyper-prior feature binary code stream; finally, the hyper-prior feature binary code stream is arithmetically decoded by using the non-parametric distribution estimation to obtain a reconstructed integer hyper-prior feature map.

[0064] S13, input the integer hyper-prior feature map into a preset integerized hyper-prior decoding network to determine Gaussian distribution parameters of the integer feature map.

[0065] The integer hyper-prior feature map input into the preset integerized hyper-prior decoding network is the reconstructed integer hyper-prior feature map.

[0066] S14, encode the integer feature map by using a preset entropy encoder according to the Gaussian distribution parameters of the integer feature map to determine a feature Figure Two binary code stream.

[0067] S15, perform merging processing on the hyper-prior feature binary code stream and the feature Figure Two binary code stream to determine a compressed image binary code stream.

[0068] The preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network adopted by the present disclosure are respectively obtained by using a pre-trained floating-point encoding network, a pre-trained floating-point hyper-prior encoding, and a pre-trained floating-point hyper-prior decoding network.

[0069] By the above technical solution, the integer image encoding is realized by using the preset integerized encoding network, the preset integerized hyper-prior encoding network, the preset integerized hyper-prior decoding network, and the preset entropy encoder, the demand for the computing capability and the storage resource of the hardware is reduced, the complexity of the image encoding is reduced, the running efficiency of the image compression is improved, the resource consumption is saved, and the deployment in the mobile device, the embedded system terminal, and the scene with limited resource consumption is facilitated.

[0070] Figure 3is a preset integerized encoding network, a preset integerized hyper-prior encoding network, a preset integerized hyper-prior decoding network according to an exemplary embodiment.

[0071] As shown in Figure 3 in a possible embodiment, the method for determining a preset integerized encoding network, a preset integerized hyper-prior encoding network, a preset integerized hyper-prior decoding network comprises S21 to S24.

[0072] The preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network are obtained based on a preset full-precision floating-point encoding network, a preset full-precision floating-point hyper-prior encoding network, and a preset full-precision floating-point hyper-prior decoding network.

[0073] S21, the output of the activation function in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network is balanced in the channel dimension to determine that the output of each activation function has the same maximum value on each channel.

[0074] wherein the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network are respectively a pre-trained full-precision floating-point encoding network, a pre-trained full-precision floating-point hyper-prior encoding network, and a pre-trained full-precision floating-point hyper-prior decoding network, and the output of the activation function of the full-precision floating-point network is balanced in the channel dimension to ensure that the output ranges of the channels are consistent.

[0075] The activation function of the full-precision floating-point network trained end-to-end is a linear rectifier (ReLU) function.

[0076] S22, the network layer containing parameters in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior encoding network is integerized.

[0077] wherein the network layer containing parameters of the full-precision floating-point network is a convolutional layer.

[0078] The network layer containing parameters of the full-precision floating-point network is integerized to balance the calculation complexity and the compression effect, and the weight parameters are quantized to n w bit integers, wherein n wis an integer not greater than 16, and a bias term in the convolution of the convolution layer is quantized into an integer.

[0079] As an example, the weight parameter is quantized into an 8-bit integer, and the bias term in the convolution is quantized into a 32-bit integer.

[0080] S23, the quantized n w bit integer weight parameter is mapped to an n l bit integer weight parameter.

[0081] n l bit integer weight parameter is mapped to an n w bit integer weight parameter.

[0082] As an example, n w is 10, and n l is 8, that is, the quantized 10-bit integer weight parameter is mapped to an 8-bit integer weight parameter, so that a value in a weight parameter is stored in one byte, thereby reducing the storage requirement.

[0083] S24, the output of the equalized activation function in the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network is integerized to determine the preset integerized encoding network, the preset integerized hyper-prior encoding network, and the preset integerized hyper-prior decoding network.

[0084] In the formula, the activation value is quantized into an integer of a first number of bits, and the scaling factor is quantized into an integer of a second number of bits.

[0085] As an example, the activation value is quantized into an 8-bit integer, and the scaling factor is quantized into a 32-bit integer.

[0086] In a possible embodiment, S21 can include S31 to S32.

[0087] S31, in the process of compressing the to-be-compressed image by using the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network, the maximum value of each activation function output in different channels is obtained.

[0088] In the formula, the maximum value of each activation function output in different channels is represented as V={v i}, and i represents a channel index.

[0089] S32, from the first parameter-containing network layer, dividing each output channel of the current convolutional layer parameter matrix by the maximum output of the activation function corresponding to the output channel, and multiplying each input channel of the next convolutional layer by the maximum output of the activation function corresponding to the input channel, to determine that the output of the activation function has the same maximum value on each channel.

[0090] The adjusted current convolutional layer parameter matrix is represented as:

[0091] W a = W a V

[0092] wherein W' a represents the adjusted current convolutional layer parameter matrix, V represents the maximum output of the activation function corresponding to each output channel of the current convolutional layer parameter matrix, and W a represents the unadjusted current convolutional layer parameter matrix.

[0093] The adjusted next convolutional layer parameter matrix is represented as:

[0094] W' b = V·W b

[0095] wherein W' b represents the adjusted next convolutional layer parameter matrix, and W b represents the unadjusted next convolutional layer parameter matrix.

[0096] In a possible embodiment, S23 can include:

[0097] mapping the integer weight parameter quantized to n w bit to an integer weight parameter with n l bit by using a nonlinear mapping function, wherein the nonlinear mapping function is represented as:

[0098]

[0099] wherein w l represents the integer weight parameter with n l bit after mapping, n w represents the bit width of the integer weight parameter quantized to n w bit, w represents the integer weight parameter with n w bit before mapping, and n l represents the bit width of the integer weight parameter with n l bit after mapping.

[0100] In the present disclosure, both the weight parameter and the activation value are subjected to asymmetric uniform quantization.

[0101] In the present disclosure, specifically, the integer weight parameter quantized to 10 bits is mapped to an integer weight parameter of 8 bits, denoted as:

[0102]

[0103] Figure 4 is a flowchart of integerization of an activation function output according to an exemplary embodiment.

[0104] As shown in Figure 4 , in a possible embodiment, S24 can include S41 to S44.

[0105] S41, n l bit integer weight parameters are restored to n w bit integer weight parameters.

[0106] As an example, n l bit integer weight parameters, i.e., integer weight parameters saved in 8-bit data, are read and the 8-bit integer weight parameters are mapped back to 10-bit integer weight parameters.

[0107] S42, the n w bit integer weight parameters are subjected to matrix multiplication or convolution operation with the integer activation values quantized by the integerization operation of the output of the previous activation function, to determine the operation result.

[0108] As an example, the integer activation values quantized to 8 bits by the previous network layer with parameters are obtained, i.e., the integer activation values quantized to 8 bits; the 10-bit integer weight parameters and the 8-bit integer activation values are subjected to matrix multiplication or convolution operation, to obtain the operation result.

[0109] S43, the operation result is subjected to summation operation with the integer-quantized bias term, and the activation value is obtained by an activation function.

[0110] The integer-quantized bias term is an integer bias term quantized to 32 bits.

[0111] The result of the summation operation is applied to a linear rectifier function, to obtain the activation output, i.e., the activation value.

[0112] S44, the activation value is subjected to multiplication operation with a preset scale factor, and the result of the multiplication operation is quantized, to obtain the integer activation value.

[0113] The integer activation value is used for the next matrix multiplication or convolution operation.

[0114] In the above example, the activation value is multiplied by a 32-bit scale factor and quantized to obtain an 8-bit integer activation value, which is input to the integerization operation of the output of the next activation function.

[0115] According to steps S41 to S44, the output of each activation function is all quantized into an integer, and the preset quantized encoding network, the preset quantized hyper-prior encoding network, and the preset quantized hyper-prior decoding network are obtained.

[0116] Figure 5 is a flowchart of an integer-implemented image decoding method according to an exemplary embodiment. Figure 6 is a flowchart of another integer-implemented image decoding method according to an exemplary embodiment.

[0117] As shown in Figure 5 , Figure 6 , the present disclosure also provides an integer-implemented image decoding method, comprising S51 to S56.

[0118] S51, an encoded compressed image binary code stream is obtained.

[0119] S52, the compressed image binary code stream is decomposed to determine a hyper-prior feature binary code stream and a feature Figure Two binary code stream.

[0120] S53, the hyper-prior feature binary code stream is arithmetically decoded using a non-parametric distribution estimation to determine an integer hyper-prior feature map.

[0121] S54, the integer hyper-prior feature map is input into a preset quantized hyper-prior decoding network to determine Gaussian distribution parameters of an integer feature map.

[0122] S55, according to the Gaussian distribution parameters of the integer feature map, the feature Figure Two binary code stream is decoded using a preset entropy decoder to determine an integer feature map.

[0123] S56, the integer feature map is input into a preset quantized decoding network to determine a reconstructed image.

[0124] By using the preset quantized decoding network, the preset quantized hyper-prior encoding network, the preset quantized hyper-prior decoding network, and the preset entropy decoder, the integer-implemented image decoding is realized, the decoding stability is improved, and the decoding inconsistency caused by different device floating point calculations is avoided.

[0125] In a possible embodiment, the preset integer quantized decoding network, the preset integer quantized hyper-prior encoding network, and the preset integer quantized hyper-prior decoding network are obtained based on the preset full-precision floating-point encoding network, the preset full-precision floating-point hyper-prior encoding network, and the preset full-precision floating-point hyper-prior decoding network. For details, refer to steps S21 to S24, S31 to S32, and S41 to S44 of the present disclosure.

[0126] In a possible embodiment, the present disclosure further provides an integer implemented image compression method, comprising:

[0127] obtaining a to-be-compressed image;

[0128] According to the integer implemented image encoding method provided by the present disclosure, the to-be-compressed image is encoded to obtain a compressed image binary code stream.

[0129] According to the integer implemented image decoding method provided by the present disclosure, the to-be-compressed image is decoded to obtain a reconstructed image.

[0130] Figure 7 is a flowchart of an integer implemented image compression method according to an example embodiment.

[0131] As shown in Figure 7 In the present disclosure, an integer implemented image compression method comprises S101 to S112.

[0132] S101, obtaining a to-be-compressed image.

[0133] S102, inputting the to-be-compressed image into a preset integer quantized encoding network to determine an integer feature map.

[0134] S103, inputting the integer feature map into a preset integer quantized hyper-prior encoding network to determine an integer hyper-prior feature map and a hyper-prior feature binary code stream by using a non-parametric distribution estimation.

[0135] First, the integer feature map is input into the preset integer quantized hyper-prior encoding network to extract an integer hyper-prior feature map. Second, a non-parametric distribution estimation is established in each channel of the integer hyper-prior feature map, and the integer hyper-prior feature map is arithmetically encoded by using the non-parametric distribution estimation to determine a hyper-prior feature binary code stream. Finally, the hyper-prior feature binary code stream is arithmetically decoded by using the non-parametric distribution estimation to obtain a reconstructed integer hyper-prior feature map.

[0136] S104, input the integer super-prior feature map into the preset integer super-prior decoding network, and determine the Gaussian distribution parameters of the integer feature map.

[0137] The integer super-prior feature map input into the preset integer super-prior decoding network is the reconstructed integer super-prior feature map.

[0138] S105, according to the Gaussian distribution parameters of the integer feature map, using a preset entropy encoder to encode the integer feature map, and determine the feature Figure Two binary code stream.

[0139] S106, merge the super-prior feature binary code stream and the feature Figure Two binary code stream to determine the encoded compressed image binary code stream.

[0140] S107, obtain the encoded compressed image binary code stream.

[0141] S108, decompose the compressed image binary code stream to determine the super-prior feature binary code stream and the feature Figure Two binary code stream.

[0142] S109, using non-parametric distribution estimation to arithmetically decode the super-prior feature binary code stream to determine the integer super-prior feature map.

[0143] S110, input the integer super-prior feature map into the preset integer super-prior decoding network to determine the Gaussian distribution parameters of the integer feature map.

[0144] S111, according to the Gaussian distribution parameters of the integer feature map, using a preset entropy decoder to decode the feature Figure Two binary code stream to determine the integer feature map.

[0145] S112, input the integer feature map into the preset integer decoding network to determine the reconstructed image.

[0146] The integer-implemented image compression method of the integer-implemented image encoding and the integer-implemented image decoding can speed up the encoding and decoding speed and reduce the encoding and decoding delay without significant rate-distortion performance loss.

[0147] In one possible embodiment, an image compression network with 14 convolutional layers is trained end-to-end on the Flicker dataset, and the Adam optimizer is used to train 200,000 steps at a learning rate of 10 -4 -5 ​When the balance coefficient of the reconstruction error and the code rate in the loss function is set to 2048, the integer-implemented image compression method provided by the present disclosure can achieve a reconstruction error of 34.3487 dB and a code rate of 0.8016; when the balance coefficient of the reconstruction error and the code rate in the loss function is set to 512, the integer-implemented image compression method provided by the present disclosure can achieve a reconstruction error of 30.1784 dB and a code rate of 0.3708.

[0148] Figure 8 is a block diagram of an integer-implemented image compression system according to an exemplary embodiment.

[0149] As shown in Figure 8 in a possible embodiment, the present disclosure also provides an integer-implemented image compression system 1000, comprising:

[0150] An encoder 1100, the encoder is configured to encode a to-be-encoded image by using the integer-implemented image encoding method provided by the present disclosure, and obtain an encoded compressed image binary code stream;

[0151] A decoder 1200, the decoder is configured to decode the encoded compressed image binary code stream by using the integer-implemented image decoding method provided by the present disclosure, and obtain a reconstructed image.

[0152] Regarding the embodiments of the above system, the specific manner in which each device performs the operation has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0153] In a possible embodiment, the integer-implemented image compression system provided by the present disclosure is implemented on an embedded platform NVIDIA Jetson Xavier NX, and the code is written by using the TensorRT framework to implement integer calculation.

[0154] TensorRT has specially accelerated and optimized 8-bit integer convolution, in order to make full use of this advantage, the integer-implemented image compression network is quantized and integerized by using 8-bit. When the weight is 2048, the peak signal-to-noise ratio (PSNR) of the full integer network simulated by Pytorch is 33.82 dB, and the code rate is 0.80 bpp. The 8-bit full integer model deployed on the Xavier NX has the same calculation result as the full integer network simulated by Pytorch, and the rate-distortion performance of the deployed model does not change, and the forward inference time is reduced by about half compared with the floating-point model. The specific time is shown in Table 1:

[0155] Model Encoding side Decoding side Full precision model time (ms) 18 35 Full integer model time (ms) 10 12

[0156] Table 1

[0157] Table 1 is a comparison of forward inference time of full precision network and full integer network on Xavier NX. According to Table 1, the image compression system implemented by integers provided by the present disclosure can accelerate the coding speed and reduce the coding delay without causing obvious rate distortion performance.

[0158] Based on the same concept, in another embodiment of the present disclosure, a terminal is also provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor is configured to execute the integer implemented image encoding method, the integer implemented image decoding method, or the integer implemented image compression method when executing the program.

[0159] Optionally, the memory is configured to store the program. The memory can include volatile memory, such as random access memory (RAM), for example, static random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. The memory can also include non-volatile memory, such as flash memory. The memory is configured to store computer programs (such as application programs, functional modules, etc. for implementing the above method), computer instructions, etc. The computer programs, computer instructions, etc. described above can be stored in one or more memories. And the computer programs, computer instructions, data, etc. described above can be called by the processor.

[0160] The computer programs, computer instructions, etc. described above can be stored in one or more memories. And the computer programs, computer instructions, data, etc. described above can be called by the processor.

[0161] The processor is configured to execute the computer program stored in the memory to implement each step in the method described above. For details, please refer to the related description in the method embodiments.

[0162] The processor and the memory can be an independent structure or an integrated structure. When the processor and the memory are an independent structure, the memory and the processor can be coupled and connected through a bus.

[0163] The embodiment of the present disclosure further provides a non-transitory computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the integer implemented image coding method, the integer implemented image decoding method or the integer implemented image compression method in any of the above embodiments.

[0164] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system or a computer program product. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. In addition, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage and the like) containing computer-usable program code.

[0165] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device implemented in the flowcharts and / or block diagrams. Figure One one or more flows and / or blocks Figure One an apparatus that performs the functions specified in the flow(s) or block(s).

[0166] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure One one or more flows and / or blocks Figure One an apparatus that performs the functions specified in the flow(s) or block(s).

[0167] These computer program instructions can also be loaded into a computer or other programmable data processing device to produce a series of operation steps executed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the flowcharts and / or block diagrams. Figure One one or more flows and / or blocks Figure One an apparatus that performs the functions specified in the flow(s) or block(s).

[0168] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present disclosure.

[0169] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. An integer-implemented image coding method, characterized in that: include: Input the image to be compressed into a preset integer encoding network to determine an integer feature map; Inputting the integer feature map into a preset integerized super-prior coding network, and using non-parametric distribution estimation to determine the integer super-prior feature map and the super-prior feature binary code stream; Inputting the integer super-prior feature map into a preset integer super-prior decoding network to determine the Gaussian distribution parameters of the integer feature map; According to the Gaussian distribution parameters of the integer feature map, a preset entropy encoder is used to encode the integer feature map to determine a feature map binary code stream; Merging the super-a priori feature binary code stream and the feature map binary code stream to determine a coded compressed image binary code stream; The method for determining the preset integerized encoding network, the preset integerized super-a priori encoding network, and the preset integerized super-a priori decoding network includes: Equalizing the outputs of activation functions in a preset full-precision floating-point encoding network, a preset full-precision floating-point super-prior encoding network, and a preset full-precision floating-point super-prior decoding network in the channel dimension, and ensuring that the output of each activation function has the same maximum value in each channel; Integerizing the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the network layer containing parameters in the preset full-precision floating-point super-a priori encoding network; Quantize to n w The integer weight parameter of bits is mapped to n l An integer weight parameter of bits, the n l bits are smaller than the n w Bit; Integerizing the outputs of the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the equalized activation functions in the preset full-precision floating-point super-a priori encoding network, and determining the preset integerized encoding network, the preset integerized super-a priori encoding network, and the preset integerized super-a priori decoding network; Wherein, the quantization is n w The integer weight parameter of bits is mapped to n l Integer weight parameters for bits, including: A nonlinear mapping function is used to quantize the w The integer weight parameter of bits is mapped to n l The integer weight parameter of the bit, the nonlinear mapping function is expressed as: Among them, w l Indicates that the bit width after mapping is n l The integer weight parameter, n w Indicates quantization to n w The bit width of the integer weight parameter of bits, w means the bit width before mapping is n w The integer weight parameter, n l Indicates n after mapping l The bit width of the integer weight parameter in bits.

2. The method according to claim 1, characterized in that The step of balancing the outputs of the activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network in the channel dimension, and determining that the outputs of the activation functions have the same maximum value in each channel, includes: In a process of compressing a to-be-compressed image using the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network, obtaining a maximum value of each activation function output in different channels; Starting from the first network layer containing parameters, each output channel of the current convolutional layer parameter matrix is ​​divided by the maximum output of the activation function corresponding to the output channel, and each input channel of the next convolutional layer is multiplied by the maximum output of the activation function corresponding to the input channel to ensure that the output of the activation function has the same maximum value on each channel.

3. The method according to claim 1, characterized in that The step of integerizing the outputs of the equalized activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori encoding network, and determining the preset integerized encoding network, the preset integerized super-a priori encoding network, and the preset integerized super-a priori decoding network, comprises: The n l The integer weight parameter of the bit is restored to the n w integer weight parameter of bits; The n w Perform a matrix multiplication operation or a convolution operation on the integer weight parameter of the bit and the integer activation value quantized by the output integerization operation of the previous activation function to determine the operation result; Performing a sum operation on the operation result and the bias term quantized into an integer, and obtaining an activation value through the activation function; The activation value is multiplied by a preset scaling factor, and the result of the multiplication is quantized to obtain an integer activation value, which is used for the next matrix multiplication or convolution operation.

4. An integer-implemented image decoding method for decoding the image code obtained in claim 1, characterized in that: include: Get the encoded compressed image binary code stream; Decomposing the encoded compressed image binary code stream to determine a super-prior feature binary code stream and a feature map binary code stream; Performing arithmetic decoding on the super-a priori feature binary code stream using non-parametric distribution estimation to determine an integer super-a priori feature map; Inputting the integer super-prior feature map into a preset integer super-prior decoding network to determine Gaussian distribution parameters of the integer feature map; Decoding the feature map binary stream using a preset entropy decoder according to the Gaussian distribution parameters of the integer feature map to determine the integer feature map; The integer feature map is input into a preset integer decoding network to determine a reconstructed image.

5. The method according to claim 4, characterized in that The preset integerized decoding network, the preset integerized super-prior encoding network, and the preset integerized super-prior decoding network are respectively obtained based on the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior decoding network.

6. An integer-implemented image compression method, characterized in that: include: Get the image to be compressed; Encoding the image to be compressed using the image encoding method implemented by integers according to any one of claims 1 to 3 to obtain a compressed image binary code stream; The compressed image binary code stream is decoded based on the image decoding method implemented by integers according to any one of claims 4 to 5 to obtain a reconstructed image.

7. An integer-implemented image compression system, characterized in that include: An encoder, configured to encode an image to be encoded using the image encoding method implemented by integers according to any one of claims 1 to 3, to obtain a compressed image binary code stream; A decoder, wherein the decoder is configured to decode the compressed image binary code stream using the integer-implemented image decoding method according to any one of claims 4 to 5 to obtain a reconstructed image.

8. An image processing terminal based on integer implementation, characterized in that: The system comprises a memory and a processor, wherein the memory stores a program executed by the processor, and the processor can perform any of the following methods when executing the program: The image coding method implemented by integers according to any one of claims 1 to 3, or The image decoding method implemented by integers according to any one of claims 4 to 5, or The integer-implemented image compression method according to claim 6.

Images