Image coding, decoding and compressing method and system based on integer implementation, and terminal

Abstract

The invention provides an integer-implemented image coding, decoding and compression method and system, and a terminal, and the method comprises the steps: inputting a to-be-compressed image into a preset integer coding network, and determining an integer feature map; inputting the integer feature map into a preset integer super-prior coding network, and determining an integer super-prior feature map and a super-prior feature binary code stream by adopting non-parametric distribution estimation; inputting the integer super-prior feature map into a preset integer super-prior decoding network, and determining Gaussian distribution parameters of the integer feature map; a preset entropy encoder is adopted to encode the integer feature map, and a binary code stream of the feature map is determined; and merging the super-prior feature binary code stream and the feature map binary code stream, and determining a compressed image binary code stream. According to the image compression method and device, under the condition that the rate distortion performance is not lost, the complexity of image coding and image decoding is reduced, the operation efficiency of image compression is improved, and resource consumption is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image encoding, decoding, and compression method implemented by integers, as well as a compression system and a terminal. Background Art

[0002] In recent years, deep learning technology has been widely used in the fields of image compression and image encoding and decoding. In 2017, Ballé et al. published an end-to-end optimized image compression framework based on convolutional neural networks in the paper "End-to-end Optimized Image Compression" at the International Conference on Learning Representations (ICLR). The convolutional neural network was used to model image compression as a rate-distortion optimization problem, which can jointly optimize each module. Based on this framework, a series of end-to-end trainable image compression models were formed. These end-to-end learnable image compression models based on convolutional neural networks show better rate-distortion performance than traditional compression methods.

[0003] However, full-precision neural networks contain a large number of floating-point parameters, which leads to high complexity, high energy consumption, long time consumption, and large storage space required for network calculations, thus limiting the actual deployment of end-to-end image compression models on hardware. In addition, when floating-point models are used for forward inference, the parallel calculation of floating-point numbers is uncertain, and the same input may lead to different calculation results. This phenomenon is particularly evident when computing across platforms. This uncertainty will cause the full-precision end-to-end image compression model to predict different bitstream distributions on different platforms, ultimately leading to decoding errors and image reconstruction failures. The full-integer network can avoid the above problems, reduce the computational complexity and storage space of the network, and effectively eliminate the uncertainty of network operations. Although some work has studied the integerization of weight parameters, the full-integer quantization of the entire network has not yet been truly implemented. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide an image encoding, decoding, compression method, system and terminal implemented by integers.

[0005] To achieve the above objective, according to one aspect of the present disclosure, there is provided an integer-implemented image encoding method, comprising:

[0006] Inputting the image to be compressed into a preset integer encoding network to determine an integer feature map;

[0007] Inputting the integer feature map into a preset integerized super-prior coding network, and using non-parametric distribution estimation to determine the integer super-prior feature map and the super-prior feature binary code stream;

[0008] Inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine Gaussian distribution parameters of the integer feature map;

[0009] According to the Gaussian distribution parameters of the integer feature map, a preset entropy encoder is used to encode the integer feature map to determine the feature Figure 2 Binary code stream;

[0010] The super prior feature binary code stream and the feature Figure 2 The binary code streams are merged to determine the encoded compressed image binary code stream.

[0011] Optionally, the method for determining the preset integerized encoding network, the preset integerized super a priori encoding network, and the preset integerized super a priori decoding network includes:

[0012] The outputs of activation functions in a preset full-precision floating-point encoding network, a preset full-precision floating-point super-prior encoding network, and a preset full-precision floating-point super-prior decoding network are balanced in the channel dimension to ensure that the output of each activation function has the same maximum value in each channel;

[0013] Integerizing the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the network layer containing parameters in the preset full-precision floating-point super-a priori encoding network;

[0014] Quantize to n w The integer weight parameter of bits is mapped to n l bit integer weight parameter, the n l bits are less than the n w Bit;

[0015] The outputs of the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the equalized activation functions in the preset full-precision floating-point super-prior encoding network are integerized to determine the preset integerized encoding network, the preset integerized super-prior encoding network, and the preset integerized super-prior decoding network.

[0016] Optionally, balancing the outputs of activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network in a channel dimension, and determining that the outputs of the activation functions have the same maximum value in each channel, comprises:

[0017] In the process of compressing the image to be compressed by using the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network, obtaining the maximum value of each activation function output in different channels;

[0018] Starting from the first network layer containing parameters, each output channel of the current convolutional layer parameter matrix is ​​divided by the maximum value of the activation function output corresponding to the output channel, and each input channel of the next convolutional layer is multiplied by the maximum value of the activation function output corresponding to the input channel, and the output of the activation function is determined to have the same maximum value on each channel.

[0019] Optionally, the quantization is n w The integer weight parameter of bits is mapped to n l Integer weight parameters of bits, including:

[0020] A nonlinear mapping function is used to quantize the w The integer weight parameter of bits is mapped to n l An integer weight parameter of bits.

[0021] Optionally, the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the output of the equalized activation function in the preset full-precision floating-point super-a priori encoding network are integerized, and the preset integerized encoding network, the preset integerized super-a priori encoding network, and the preset integerized super-a priori decoding network are determined, including:

[0022] The n l The integer weight parameter of the bit is restored to the n w Integer weight parameter of bits;

[0023] The n w The integer weight parameter of the bit is subjected to a matrix multiplication operation or a convolution operation operation on the integer activation value quantized by the output integerization operation of the previous activation function to determine the operation result;

[0024] Performing a sum operation on the operation result and the bias term quantized as an integer, and obtaining an activation value through the activation function;

[0025] The activation value is multiplied by a preset scaling factor, and the result of the multiplication is quantized to obtain an integer activation value, which is used for the next matrix multiplication or convolution operation.

[0026] According to a second aspect of the present disclosure, an image decoding method implemented by integers is provided, which is used to decode the image code obtained in the first aspect of the present disclosure, comprising:

[0027] Get the encoded compressed image binary code stream;

[0028] Decomposing the compressed image binary code stream to determine the super-prior feature binary code stream and feature Figure 2 Binary code stream;

[0029] Performing arithmetic decoding on the super a priori feature binary code stream using non-parametric distribution estimation to determine an integer super a priori feature map;

[0030] Inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine Gaussian distribution parameters of the integer feature map;

[0031] According to the Gaussian distribution parameters of the integer feature map, a preset entropy decoder is used to decode the feature map. Figure 2 Decoding the binary code stream to determine the integer feature map;

[0032] The integer feature map is input into a preset integer decoding network to determine a reconstructed image.

[0033] Optionally, the preset integerized decoding network, the preset integerized super a priori encoding network, and the preset integerized super a priori decoding network are respectively obtained based on a preset full-precision floating-point encoding network, a preset full-precision floating-point super a priori encoding network, and a preset full-precision floating-point super a priori decoding network.

[0034] According to a third aspect of the present disclosure, there is provided an integer-implemented image compression method, comprising:

[0035] Obtain the image to be compressed;

[0036] Encode the image to be compressed based on any one of the integer-implemented image encoding methods provided in the first aspect of the present disclosure to obtain a compressed image binary code stream;

[0037] The compressed image binary code stream is decoded according to any one of the integer-implemented image decoding methods provided in the second aspect of the present disclosure to obtain a reconstructed image.

[0038] According to a fourth aspect of the present disclosure, there is provided an integer-implemented image compression system, comprising:

[0039] An encoder, the encoder being used to encode an image to be encoded by adopting any one of the image encoding methods implemented by integers provided in the first aspect of the present disclosure to obtain a compressed image binary code stream;

[0040] A decoder, wherein the decoder is used to decode the compressed image binary code stream by adopting any one of the integer-implemented image decoding methods provided in the second aspect of the present disclosure to obtain a reconstructed image.

[0041] According to a fifth aspect of the present disclosure, a terminal is provided, including a memory and a processor, wherein the memory stores a program executed by the processor, and the processor may perform any of the following methods when executing the program:

[0042] The image encoding method implemented by integers according to any one of the first aspects of the present disclosure, or,

[0043] The image decoding method implemented by integers according to any one of the second aspects of the present disclosure, or,

[0044] The third aspect of the present disclosure provides the image compression method implemented by integers.

[0045] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0046] Through the above technical scheme, by adopting a preset integerized coding network, a preset integerized hyper-prior coding network, a preset integerized hyper-prior decoding network and a preset entropy encoder, integer-implemented image coding is realized, reducing the demand for hardware computing power and storage resources, thereby reducing the complexity of image coding, improving the operating efficiency of image compression, saving resource consumption, and facilitating deployment in mobile devices, embedded system terminals and scenarios with limited resource and energy consumption.

[0047] The embodiments of the present disclosure implement integer-based image decoding by adopting a preset integerized decoding network, a preset integerized super-a priori coding network, a preset integerized super-a priori decoding network and a preset entropy decoder, thereby improving decoding stability and avoiding decoding inconsistencies caused by floating-point calculations of different devices.

[0048] The embodiments of the present disclosure provide an integer-implemented image compression method based on integer-implemented image encoding and integer-implemented image decoding, which can accelerate encoding and decoding speed and reduce encoding and decoding delay without causing significant rate-distortion performance loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0050] Figure 1 The figure is a flowchart of an image encoding method implemented by integers according to an exemplary embodiment.

[0051] Figure 2 The figure is a flowchart of another image encoding method implemented by integers according to an exemplary embodiment.

[0052] Figure 3 It is a flow chart of a method for obtaining a preset integerized encoding network, a preset integerized super a priori encoding network, and a preset integerized super a priori decoding network according to an exemplary embodiment.

[0053] Figure 4 The present invention is a flowchart showing integerization of a network layer containing parameters according to an exemplary embodiment.

[0054] Figure 5 The figure is a flowchart of an image decoding method implemented by integers according to an exemplary embodiment.

[0055] Figure 6 The figure is a flowchart of another image decoding method implemented by integers according to an exemplary embodiment.

[0056] Figure 7 The figure is a flowchart of an image compression method implemented by integers according to an exemplary embodiment.

[0057] Figure 8 The invention is a block diagram of an image compression system implemented by integers according to an exemplary embodiment. DETAILED DESCRIPTION

[0058] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.

[0059] Figure 1 The figure is a flowchart of an image encoding method implemented by integers according to an exemplary embodiment. Figure 2 The figure is a flowchart of another image encoding method implemented by integers according to an exemplary embodiment.

[0060] like Figure 1 , Figure 2 As shown, the present disclosure provides an integer-implemented image encoding method, including S11 to S15.

[0061] S11, inputting the image to be compressed into a preset integer encoding network to determine an integer feature map.

[0062] S12, inputting the integer feature map into a preset integerized super-prior coding network, and using non-parametric distribution estimation to determine the integer super-prior feature map and the super-prior feature binary code stream.

[0063] Among them, first, the integer feature map is input into a preset integerized super-prior coding network to extract the integer super-prior feature map; secondly, a non-parametric distribution estimate is established in each channel of the integer super-prior feature map, and the integer super-prior feature map is arithmetically encoded using the non-parametric distribution estimate to determine the super-prior feature binary code stream; finally, the super-prior feature binary code stream is arithmetically decoded using the non-parametric distribution estimate to obtain the reconstructed integer super-prior feature map.

[0064] S13, inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine the Gaussian distribution parameters of the integer feature map.

[0065] Among them, the integer super-prior feature map input into the preset integer super-prior decoding network is the reconstructed integer super-prior feature map.

[0066] S14, according to the Gaussian distribution parameters of the integer feature map, a preset entropy encoder is used to encode the integer feature map to determine the feature Figure 2 Binary code stream.

[0067] S15, super prior feature binary code stream and feature Figure 2 The binary code streams are merged to determine the encoded compressed image binary code stream.

[0068] The preset integerized encoding network, the preset integerized super-prior encoding network and the preset integerized super-prior decoding network adopted in the present invention are respectively obtained by using a pre-trained floating-point encoding network, a pre-trained floating-point super-prior encoding and a pre-trained floating-point super-prior decoding network.

[0069] Through the above technical scheme, by adopting a preset integerized coding network, a preset integerized hyper-prior coding network, a preset integerized hyper-prior decoding network and a preset entropy encoder, integer-implemented image coding is realized, reducing the demand for hardware computing power and storage resources, thereby reducing the complexity of image coding, improving the operating efficiency of image compression, saving resource consumption, and facilitating deployment in mobile devices, embedded system terminals and scenarios with limited resource and energy consumption.

[0070] Figure 3It is a flow chart of a method for obtaining a preset integerized encoding network, a preset integerized super a priori encoding network, and a preset integerized super a priori decoding network according to an exemplary embodiment.

[0071] like Figure 3 As shown, in a possible embodiment, a method for determining a preset integerized encoding network, a preset integerized super a priori encoding network, and a preset integerized super a priori decoding network includes S21 to S24.

[0072] Based on the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior decoding network, the preset integer encoding network, the preset integer super-prior encoding network, and the preset integer super-prior decoding network are obtained. The preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior decoding network can be used in the image compression process of the compressed image.

[0073] S21, balancing the outputs of activation functions in a preset full-precision floating-point encoding network, a preset full-precision floating-point hyper-prior encoding network, and a preset full-precision floating-point hyper-prior decoding network in the channel dimension, and ensuring that the output of each activation function has the same maximum value in each channel.

[0074] Among them, the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior decoding network are respectively the pre-trained full-precision floating-point encoding network, the pre-trained full-precision floating-point super-prior encoding network, and the pre-trained full-precision floating-point super-prior decoding network, which balance the output of the activation function of the full-precision floating-point network in the channel dimension to ensure that the output range of each channel is consistent.

[0075] The activation function of the full-precision floating-point network trained end-to-end is the rectified linear unit (ReLU) function.

[0076] S22, integerizing the network layers containing parameters in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior encoding network.

[0077] Among them, the network layer containing parameters of the full-precision floating-point network is the convolutional layer.

[0078] The network layers containing parameters of the full-precision floating-point network are integerized to balance the computational complexity and compression effect, and the weight parameters are quantized to n w An integer of bits, where n wis an integer not greater than 16, and the bias term in the convolution of the convolution layer is quantized to an integer.

[0079] As an example, the weight parameters are quantized into 8-bit integers, and the bias terms in the convolution are quantized into 32-bit integers.

[0080] S23, will be quantized to n w The integer weight parameter of bits is mapped to n l An integer weight parameter of bits.

[0081] n l Bits less than n w Bit.

[0082] As an example, n w is 10, n l That is, the integer weight parameter quantized to 10 bits is mapped to an integer weight parameter of 8 bits, so that one value in one weight parameter is stored in one byte length, thereby reducing storage requirements.

[0083] S24, integerize the outputs of the equalized activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the preset full-precision floating-point super-prior encoding network, and determine the preset integerized encoding network, the preset integerized super-prior encoding network, and the preset integerized super-prior decoding network.

[0084] The activation value is quantized into an integer of first numerical bits, and the proportionality factor is quantized into an integer of second numerical bits.

[0085] As an example, the activation value is quantized into an 8-bit integer, and the scaling factor is quantized into a 32-bit integer.

[0086] In a possible embodiment, S21 may include S31 to S32.

[0087] S3 1, in the process of compressing the image to be compressed using a preset full-precision floating-point encoding network, a preset full-precision floating-point super-prior encoding network, and a preset full-precision floating-point super-prior decoding network, obtain the maximum value of each activation function output in different channels.

[0088] Among them, the maximum value of each activation function output in different channels is expressed as V = {v i}, i represents the channel index.

[0089] S32, starting from the first network layer containing parameters, divide each output channel of the parameter matrix of the current convolution layer by the maximum value of the activation function output corresponding to the output channel, and multiply each input channel of the next convolution layer by the maximum value of the activation function output corresponding to the input channel, and determine that the output of the activation function has the same maximum value on each channel.

[0090] The adjusted parameter matrix of the current convolution layer is expressed as:

[0091] W a =W a / V

[0092] Among them, W′ a represents the adjusted current convolutional layer parameter matrix, V represents the maximum value of the activation function output corresponding to each output channel of the current convolutional layer parameter matrix, and W a Represents the unadjusted current convolutional layer parameter matrix.

[0093] The adjusted parameter matrix of the next convolutional layer is expressed as:

[0094] W′ b =V·W b

[0095] Among them, W′ b represents the adjusted parameter matrix of the next convolutional layer, W b Represents the unadjusted parameter matrix of the next convolutional layer.

[0096] In a possible embodiment, S23 may include:

[0097] A nonlinear mapping function is used to quantize n w The integer weight parameter of bits is mapped to n l The integer weight parameter of the bit, the nonlinear mapping function is expressed as:

[0098]

[0099] Among them, w l Indicates that the bit width after mapping is n l The integer weight parameter, n w Represents quantization as n w The bit width of the integer weight parameter of bits, w means the bit width before mapping is n w The integer weight parameter, n l Indicates the mapping after n l The bit width of the integer weight parameter in bits.

[0100] In the present disclosure, asymmetric uniform quantization is used for both weight parameters and activation values.

[0101] In the present disclosure, specifically, the integer weight parameter quantized to 10 bits is mapped to an integer weight parameter of 8 bits, which is expressed as:

[0102]

[0103] Figure 4 The present invention is a flowchart of integerizing an activation function output according to an exemplary embodiment.

[0104] like Figure 4 As shown, in a possible embodiment, S24 may include S41 to S44.

[0105] S41, n l The integer weight parameter of the bit is restored to n w An integer weight parameter of bits.

[0106] As an example, read n l The 8-bit integer weight parameter is mapped back to a 10-bit integer weight parameter, that is, an integer weight parameter stored in 8-bit data.

[0107] S42, n w The integer weight parameter of the bit is subjected to a matrix multiplication operation or a convolution operation operation on the integer activation value quantized by the output integerization operation of the previous activation function to determine the operation result.

[0108] As an example, obtain the quantized integer activation value of the previous network layer containing parameters, that is, quantize it to an 8-bit integer activation value; perform matrix multiplication or convolution operation on the 10-bit integer weight parameter and the 8-bit integer activation value to obtain the operation result.

[0109] S43, performing a sum operation on the operation result and the bias term quantized into an integer, and obtaining an activation value through an activation function.

[0110] The bias item quantized as an integer is a 32-bit integer bias item quantized as an integer.

[0111] Apply a linear rectification function to the result of the sum operation to obtain the activation output, i.e., the activation value.

[0112] S44, performing a multiplication operation on the activation value and a preset scaling factor, and quantizing the result of the multiplication operation to obtain an integer activation value.

[0113] Among them, the integer activation value is used for the next matrix multiplication or convolution operation.

[0114] Continuing with the above example, the activation value is multiplied by the 32-bit scaling factor and then quantized to obtain an 8-bit integer activation value, which is then input into the integerization operation of the output of the next activation function.

[0115] According to steps S41 to S44, the output of each activation function is all integerized to obtain a preset integerized encoding network, a preset integerized super a priori encoding network, and a preset integerized super a priori decoding network.

[0116] Figure 5 The figure is a flowchart of an image decoding method implemented by integers according to an exemplary embodiment. Figure 6 The figure is a flowchart of another image decoding method implemented by integers according to an exemplary embodiment.

[0117] like Figure 5 , Figure 6 As shown, the present disclosure also provides an image decoding method implemented by integers, including S51 to S56.

[0118] S51, obtaining a coded compressed image binary code stream.

[0119] S52, decomposing the compressed image binary code stream to determine the super-prior feature binary code stream and feature Figure 2 Binary code stream.

[0120] S53, using non-parametric distribution estimation to perform arithmetic decoding on the super-prior feature binary code stream to determine an integer super-prior feature map.

[0121] S54, inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine the Gaussian distribution parameters of the integer feature map.

[0122] S55, according to the Gaussian distribution parameters of the integer feature map, a preset entropy decoder is used to decode the feature Figure 2 The binary code stream is decoded to determine the integer feature map.

[0123] S56, inputting the integer feature map into a preset integer decoding network to determine the reconstructed image.

[0124] By adopting a preset integerized decoding network, a preset integerized super-prior encoding network, a preset integerized super-prior decoding network and a preset entropy decoder, integer-implemented image decoding is achieved, decoding stability is improved, and decoding inconsistency caused by floating-point calculations of different devices is avoided.

[0125] In a possible embodiment, a preset integerized decoding network, a preset integerized super a priori encoding network, and a preset integerized super a priori decoding network are obtained based on a preset full-precision floating-point encoding network, a preset full-precision floating-point super a priori encoding network, and a preset full-precision floating-point super a priori decoding network. Specifically, reference can be made to steps S21 to S24, S31 to S32, and S41 to S44 of the present disclosure to obtain the preset integerized decoding network, the preset integerized super a priori encoding network, and the preset integerized super a priori decoding network based on the preset full-precision floating-point encoding network, the preset full-precision floating-point super a priori encoding network, and the preset full-precision floating-point super a priori decoding network, which will not be repeated here.

[0126] In a possible embodiment, the present disclosure further provides an integer-implemented image compression method, comprising:

[0127] Obtain the image to be compressed;

[0128] Encode the image to be compressed according to the image encoding method implemented by integers provided by the present disclosure to obtain a compressed image binary code stream;

[0129] According to the image decoding method realized by integers provided by the present disclosure, a compressed image is decoded to obtain a reconstructed image.

[0130] Figure 7 The figure is a flowchart of an image compression method implemented by integers according to an exemplary embodiment.

[0131] like Figure 7 As shown, in the present disclosure, an integer-implemented image compression method includes S101 to S112.

[0132] S101, obtaining an image to be compressed.

[0133] S102, inputting the image to be compressed into a preset integer encoding network to determine an integer feature map.

[0134] S103, inputting the integer feature map into a preset integerized super-prior coding network, and using non-parametric distribution estimation to determine the integer super-prior feature map and the super-prior feature binary code stream.

[0135] Among them, first, the integer feature map is input into a preset integerized super-prior coding network to extract the integer super-prior feature map; secondly, a non-parametric distribution estimate is established in each channel of the integer super-prior feature map, and the integer super-prior feature map is arithmetically encoded using the non-parametric distribution estimate to determine the super-prior feature binary code stream; finally, the super-prior feature binary code stream is arithmetically decoded using the non-parametric distribution estimate to obtain the reconstructed integer super-prior feature map.

[0136] S104, inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine Gaussian distribution parameters of the integer feature map.

[0137] Among them, the integer super-prior feature map input into the preset integer super-prior decoding network is the reconstructed integer super-prior feature map.

[0138] S105, according to the Gaussian distribution parameters of the integer feature map, a preset entropy encoder is used to encode the integer feature map to determine the feature Figure 2 Binary code stream.

[0139] S106, super prior feature binary code stream and feature Figure 2 The binary code streams are merged to determine the encoded compressed image binary code stream.

[0140] S107, obtaining a coded compressed image binary code stream.

[0141] S108, decomposing the compressed image binary code stream to determine the super-prior feature binary code stream and feature Figure 2 Binary code stream.

[0142] S109, using non-parametric distribution estimation to perform arithmetic decoding on the super-prior feature binary code stream to determine an integer super-prior feature map.

[0143] S110, inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine the Gaussian distribution parameters of the integer feature map.

[0144] S111, according to the Gaussian distribution parameters of the integer feature map, a preset entropy decoder is used to decode the feature Figure 2 The binary code stream is decoded to determine the integer feature map.

[0145] S112, inputting the integer feature map into a preset integer decoding network to determine the reconstructed image.

[0146] The integer-implemented image compression method of integer-implemented image encoding and integer-implemented image decoding can speed up encoding and decoding speed and reduce encoding and decoding delay without causing obvious rate-distortion performance loss.

[0147] In one possible embodiment, an image compression network with 14 convolutional layers is trained end-to-end on the Flicker dataset, and an Adam optimizer is used at a learning rate of 10 -4 Train for 200,000 steps at a learning rate of 10 -5After 50,000 steps of training, when the balance coefficient between the reconstruction error and the bit rate in the loss function is set to 2048, the image compression method implemented by an integer provided by the present invention can achieve a reconstruction error of 34.3487 dB and a bit rate of 0.8016; when the balance coefficient between the reconstruction error and the bit rate in the loss function is set to 512, the image compression method implemented by an integer provided by the present invention can achieve a reconstruction error of 30.1784 dB and a bit rate of 0.3708.

[0148] Figure 8 The invention is a block diagram of an image compression system implemented by integers according to an exemplary embodiment.

[0149] like Figure 8 As shown, in a possible embodiment, the present disclosure further provides an integer-implemented image compression system 1000, comprising:

[0150] Encoder 1100, the encoder is used to encode the image to be encoded by using the image encoding method implemented by integers provided by the present disclosure, and obtain an encoded compressed image binary code stream;

[0151] Decoder 1200 is used to decode the encoded compressed image binary code stream using the integer-implemented image decoding method provided by the present disclosure to obtain a reconstructed image.

[0152] Regarding the embodiment of the above system, the specific manner in which each device performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0153] In a possible embodiment, an integer-implemented image compression system provided by the present disclosure is implemented on an embedded platform NVIDIA Jetson Xavier NX, and a TensorRT framework is used to write code to implement integer calculations.

[0154] TensorRT has specially accelerated and optimized 8-bit integer convolutions. In order to take full advantage of this, we have adopted 8-bit quantization and integerization for the convolution and activation of the network for image compression implemented by integers. When the weight is 2048, the peak signal-to-noise ratio (PSNR) of the fully integerized network simulated by Pytorch is 33.82dB, and the bit rate is 0.80bpp. The 8-bit full integer model deployed on Xavier NX has the same calculation results as the full integer network simulated by Pytorch. The rate-distortion performance of the deployed model remains unchanged, and the forward inference time is reduced by about half compared to the floating-point model. The specific time is shown in Table 1 below:

[0155] Model Encoding side Decoding end Full precision model time (ms) 18 35 Full integer model time (ms) 10 12

[0156] Table 1

[0157] Table 1 is a comparison of the forward inference time of a full-precision network and a full-integer network on XavierNX. It can be seen from Table 1 that an integer-implemented image compression system provided by the present disclosure can speed up the encoding and decoding speed and reduce the encoding and decoding delay without producing obvious rate-distortion performance.

[0158] Based on the same concept as above, in another embodiment of the present disclosure, a terminal is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor is used to execute an integer-implemented image encoding method, an integer-implemented image decoding method, or an integer-implemented image compression method when executing the program.

[0159] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0160] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0161] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0162] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0163] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of an integer-implemented image encoding method, an integer-implemented image decoding method, or an integer-implemented image compression method in any of the above embodiments are implemented.

[0164] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0168] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.

[0169] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. An integer-implemented image coding method, characterized in that: include: Inputting the image to be compressed into a preset integer encoding network to determine an integer feature map; Inputting the integer feature map into a preset integerized super-prior coding network, and using non-parametric distribution estimation to determine the integer super-prior feature map and the super-prior feature binary code stream; Inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine Gaussian distribution parameters of the integer feature map; According to the Gaussian distribution parameters of the integer feature map, a preset entropy encoder is used to encode the integer feature map to determine a feature map binary code stream; The super a priori feature binary code stream and the feature map binary code stream are combined to determine a coded compressed image binary code stream.

2. The method according to claim 1, characterized in that The method for determining the preset integerized encoding network, the preset integerized super a priori encoding network, and the preset integerized super a priori decoding network comprises: The outputs of activation functions in a preset full-precision floating-point encoding network, a preset full-precision floating-point super-prior encoding network, and a preset full-precision floating-point super-prior decoding network are balanced in the channel dimension to ensure that the output of each activation function has the same maximum value in each channel; Integerizing the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the network layer containing parameters in the preset full-precision floating-point super-a priori encoding network; Quantize to n w The integer weight parameter of bits is mapped to n l bit integer weight parameter, the n l bits are less than the n w Bit; The outputs of the preset full-precision floating-point encoding network, the preset full-precision floating-point super-prior encoding network, and the equalized activation functions in the preset full-precision floating-point super-prior encoding network are integerized to determine the preset integerized encoding network, the preset integerized super-prior encoding network, and the preset integerized super-prior decoding network.

3. The method according to claim 2, characterized in that The step of balancing the outputs of the activation functions in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network in the channel dimension, and determining that the outputs of the activation functions have the same maximum value in each channel, comprises: In the process of compressing the image to be compressed by using the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori decoding network, obtaining the maximum value of each activation function output in different channels; Starting from the first network layer containing parameters, each output channel of the parameter matrix of the current convolution layer is divided by the maximum output of the activation function corresponding to the output channel, and each input channel of the next convolution layer is multiplied by the maximum output of the activation function corresponding to the input channel to determine that the output of the activation function has the same maximum value on each channel.

4. The method according to claim 2, characterized in that: The quantization is n w The integer weight parameter of bits is mapped to n l Integer weight parameters of bits, including: A nonlinear mapping function is used to quantize the w The integer weight parameter of bits is mapped to n l The integer weight parameter of bit, the nonlinear mapping function is expressed as: Among them, w l Indicates that the bit width after mapping is n l The integer weight parameter, n w Represents quantization as n w The bit width of the integer weight parameter of bits, w means the bit width before mapping is n w The integer weight parameter, n l Indicates the mapping after n l The bit width of the integer weight parameter in bits.

5. The method according to claim 2, characterized in that: The step of integerizing the output of the equalized activation function in the preset full-precision floating-point encoding network, the preset full-precision floating-point super-a priori encoding network, and the preset full-precision floating-point super-a priori encoding network, and determining the preset integerized encoding network, the preset integerized super-a priori encoding network, and the preset integerized super-a priori decoding network comprises: The n l The integer weight parameter of the bit is restored to the n w integer weight parameter of bits; The n w The integer weight parameter of the bit is subjected to a matrix multiplication operation or a convolution operation operation on the integer activation value quantized by the output integerization operation of the previous activation function to determine the operation result; Performing a sum operation on the operation result and the bias term quantized into an integer, and obtaining an activation value through the activation function; The activation value is multiplied by a preset scaling factor, and the result of the multiplication is quantized to obtain an integer activation value, which is used for the next matrix multiplication or convolution operation.

6. An integer-implemented image decoding method for decoding the image code obtained in claim 1, characterized in that: include: Get the encoded compressed image binary code stream; Decomposing the encoded compressed image binary code stream to determine a super-a priori feature binary code stream and a feature map binary code stream; Performing arithmetic decoding on the super-a priori feature binary code stream using non-parametric distribution estimation to determine an integer super-a priori feature map; Inputting the integer super-prior feature map into a preset integerized super-prior decoding network to determine Gaussian distribution parameters of the integer feature map; According to the Gaussian distribution parameters of the integer feature map, a preset entropy decoder is used to decode the feature map binary code stream to determine the integer feature map; The integer feature map is input into a preset integer decoding network to determine a reconstructed image.

7. The method according to claim 6, characterized in that The preset integerized decoding network, the preset integerized super a priori encoding network, and the preset integerized super a priori decoding network are respectively obtained based on a preset full-precision floating-point encoding network, a preset full-precision floating-point super a priori encoding network, and a preset full-precision floating-point super a priori decoding network.

8. An integer-implemented image compression method, characterized in that: include: Obtain the image to be compressed; Encode the image to be compressed based on the image encoding method realized by integer according to any one of claims 1 to 5 to obtain a compressed image binary code stream; The compressed image binary code stream is decoded based on the image decoding method realized by integers according to any one of claims 6 to 7 to obtain a reconstructed image.

9. An integer-implemented image compression system, characterized in that: include: An encoder, the encoder being used to encode the image to be encoded by adopting the image encoding method realized by integers according to any one of claims 1 to 5, and obtain a compressed image binary code stream; A decoder, wherein the decoder is used to decode the compressed image binary code stream using the image decoding method implemented by integers according to any one of claims 6 to 7 to obtain a reconstructed image.

10. A terminal, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a program executed by the processor, and the processor can perform any of the following methods when executing the program: The image encoding method implemented by integers as claimed in any one of claims 1 to 5, or, The image decoding method implemented by integers as claimed in any one of claims 6 to 7, or, The integer-implemented image compression method of claim 8.

Images