Complex neural network-based multi-resolution cross-entropy encoding and decoding method, system and storage medium

Abstract

The application provides a multi-resolution cross-entropy coding and decoding method based on a complex neural network, a system and a storage medium, and the multi-resolution cross-entropy coding and decoding method comprises the following steps: S101, after an input image x passes through a multi-stage complex feature extraction unit, a complex value high-resolution latent representation y is obtained; S102, the complex value high-resolution latent representation y obtained in step 1 is discretized through a quantization unit Q to obtain step S103, the complex value latent y is gradually input into a multi-stage hyper encoder to obtain complex value latent representations y1 and y2 of different resolutions, and the latent representation output by the hyper encoder of the latter stage is a higher level feature abstraction of the former stage; and S104, the low-resolution complex value latent representation y2 obtained in step S103 is quantized to obtain a real part and an imaginary part, which are and respectively. The application has the beneficial effect that the robustness of an entropy model to adversarial samples is improved while the compression performance is ensured.

Description

Technical Field

[0001] This invention relates to the field of image compression technology, and in particular to a multi-resolution cross-entropy encoding and decoding method based on complex neural networks. Background Technology

[0002] Image compression refers to compressing an image into a binary bit stream to reduce the amount of data required to represent the image. Traditional compression techniques include JPEG, JPEG2000, etc. These traditional compression algorithms generally include three modules: transformation, quantization, and entropy coding. On the one hand, after decades of development, the performance of traditional algorithms is difficult to improve further. On the other hand, each module of traditional compression algorithms is tuned individually by humans, making it impossible to perform joint optimization between modules.

[0003] In recent years, with the rapid development of deep learning and artificial neural networks, many researchers have used neural networks to build image compression frameworks capable of end-to-end optimization. Currently, end-to-end image compression models based on Convolutional Neural Networks (CNNs) have surpassed traditional image compression methods in performance, becoming one of the current hot topics. However, most researchers blindly pursue compression performance while neglecting the stability and security of the model. For example, without affecting the reconstruction quality, they can add slight perturbations to the input that are not easily perceived by the human eye to generate adversarial examples, thereby greatly increasing the bit rate required to transmit the samples, consuming system resources, and achieving denial-of-service attacks. For an end-to-end image compression model, as shown in the figure above, both the transform module and the entropy coding module are composed of neural networks. Attack samples refer to images formed by adding noise imperceptible to the human eye to the original input image. The reconstructed input image and the reconstructed sample image refer to the original input image and the attack sample, respectively, after compression and reconstruction. Bit per rate (bpp) refers to the number of bits required to transmit the image; a smaller bpp indicates less resource consumption for image transmission. Inputting attack samples into the same neural network compression model significantly increases bit rate resources, easily leading to denial-of-service attacks, reducing the security of the image compression process, and limiting the development of neural network image compression models.

[0004] Existing work has designed an overall framework for image compression models based on complex neural networks. However, this framework mainly focuses on the transformation module in the image compression step. After obtaining complex features (also known as complex domain latent representations) using the complex numerical feature extraction unit, two branches are designed for the real and imaginary parts of the complex features. In the entropy coding process, the real and imaginary parts of the complex values ​​are still treated as two independent real values, without taking into account the unique advantages of complex values. Firstly, the model is vulnerable to denial-of-service attacks due to the poor robustness of the entropy coding module; secondly, the lack of removal between the two branches leads to poor compression performance. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides a multi-resolution cross-entropy encoding / decoding method, system, and storage medium based on complex neural networks.

[0006] This invention provides a multi-resolution cross-entropy encoding / decoding method based on complex neural networks, comprising the following steps:

[0007] Step S101: After the input image x passes through a multi-level complex feature extraction unit, a complex-valued high-resolution latent representation y is obtained;

[0008] Step S102: The complex-valued high-resolution latent representation y obtained in step S101 is discretized using the quantization unit Q to obtain... To achieve further compression;

[0009] Step S103: Input the complex numerical high-resolution latent representation y into the multi-level super encoder step by step to obtain complex numerical latent representations y1 and y2 with different resolutions. The latent representation y1 output by the first-level super encoder is a higher-level feature abstraction of y, while the latent representation y2 output by the second-level super encoder is a higher-level feature abstraction of y1.

[0010] Step S104: The low-resolution complex-valued latent representation y2 obtained in step S103 is quantized and further compressed to obtain... The corresponding real and imaginary parts are respectively and Similarly, quantizing the medium-resolution complex-valued latent representation y1 yields... The corresponding real and imaginary parts are respectively and

[0011] Step S105: Convert the real part of step S104... and the virtual part Each lossless encoding operation yields a binary bitstream, which is then subjected to lossless decoding to restore the original binary bitstream. and

[0012] Step S106: Restore the state described in step S105 and The real and imaginary parts of the complex value are input into the second-level super decoder, and the second-level super decoder reconstructs a medium-resolution latent representation. Prior information θ1 during encoding and decoding;

[0013] Step S107: Quantize the medium-resolution latent representation obtained in step S104. The input is fed into the first-level super decoder, which then processes it... The prior information θ2 required to reconstruct the high-resolution latent representation y of complex values;

[0014] Step S108: Put and Using the real and imaginary parts of the complex number respectively, we obtain the quantized high-resolution latent representation of the complex value.

[0015] As a further improvement of the present invention, step S106 includes:

[0016] Step S106-1: Put Model it as a Gaussian distribution, and set the real part θ1 to θ2. 1r As prior information for the Gaussian distribution, we obtain the Gaussian distribution q. 1r ;

[0017] Step S106-2: Using Gaussian distribution q 1r The real part of the quantized mid-resolution latent representation Lossless encoding is performed to obtain the binary code rate;

[0018] Step S106-3: Perform lossless decoding and recovery of the binary code stream from step S106-2. and put and θ 1i The real and imaginary parts of the complex number are respectively input into the third entropy parameter module EP3 to extract the encoding. Required prior information

[0019] Step S106-4: Utilizing prior information Bundle Modeled as a Gaussian distribution q 1i Through Gaussian distribution q 1i right Lossless encoding is performed to obtain a binary bitstream;

[0020] Step S106-5: Perform lossless decoding on the binary bitstream to recover the quantized intermediate resolution latent representation. imaginary part

[0021] As a further improvement of the present invention, in step S106-1, the obtained Gaussian distribution q 1r With the real part of the quantized mid-resolution latent representation The actual distribution p 1r The closer they are, the higher the real part of the encoded quantized medium-resolution latent representation. The smaller the bitrate required;

[0022] In step S106-2, the consumed code rate is:

[0023]

[0024] In step S106-4, the consumed code rate is:

[0025]

[0026] As a further improvement of the present invention, in step S107, the real part of the prior information θ2 is represented as θ 2r The imaginary part is represented by θ. 2i Step S107 further includes the following steps:

[0027] Step S107-1: Fuse the real part θ using the second entropy parameter module EP2 2r and the imaginary part θ 1r Get modeling Prior information And utilize prior information Bundle Modeled as a Gaussian distribution q r , Representing potential representations The real part;

[0028] Step S107-2: Using Gaussian distribution q r right Lossless encoding is performed to obtain a binary bitstream;

[0029] Step S107-3: Lossless decoding of the binary code rate to restore the complex value latent representation. real part

[0030] Step S107-4: Transform the latent representation real part The imaginary part θ of the prior information θ2 2iThe real and imaginary parts of the complex number are respectively input into the first entropy parameter module EP1 to obtain the modeling result. Prior information And utilize the extracted prior information Bundle Modeled as a Gaussian distribution q i , Representing potential representations The imaginary part;

[0031] Step S107-5: Utilize the Gaussian distribution q from step S107-4 i right Lossless encoding is performed to obtain a binary bitstream;

[0032] Step S107-6: Perform lossless decoding on the binary bitstream to restore the original data. imaginary part

[0033] As a further improvement of the present invention, in step S107-2, the code rate consumed is:

[0034]

[0035] In step S107-5, the consumed code rate is:

[0036]

[0037] As a further improvement of the present invention, in this multi-resolution cross-entropy encoding and decoding method, the super encoder adopts the form of complex convolution and activation function concatenation, the activation function adopts the ReLU function, and both the input and output are complex values.

[0038] As a further improvement of the present invention, in this multi-resolution cross-entropy encoding and decoding method, the first-stage super encoder performs a spatial downsampling of the latent representation by a factor of two, and the first-stage super decoder performs a spatial upsampling of the latent representation by a factor of two.

[0039] As a further improvement of the present invention, in this multi-resolution cross-entropy encoding and decoding method, the first entropy parameter module EP1 and the third entropy parameter module EP3 adopt the form of multi-level complex convolution concatenation, the second entropy parameter module EP2 adopts the form of real convolution concatenation, and the activation function adopts the ReLU function.

[0040] The present invention also discloses a multi-resolution cross-entropy encoding and decoding system based on a complex neural network, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the multi-resolution cross-entropy encoding and decoding steps described in the present invention when called by the processor.

[0041] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the multi-resolution cross-entropy encoding and decoding steps described in the present invention when invoked by a processor.

[0042] The beneficial effects of this invention are: 1. The method of this invention provides a robust complex numerical entropy encoding module, which no longer treats the real and imaginary parts of complex numbers as two independent parts during the entropy encoding process, but fully utilizes the spatial and channel correlation between the real and imaginary parts; 2. The method of this invention designs an entropy parameter module and a complex domain supercoder module, which improves the robustness of the entropy model against adversarial examples while ensuring compression performance. Attached Figure Description

[0043] Figure 1 This is the background diagram of the present invention;

[0044] Figure 2 This is a block diagram illustrating the principle of the multi-resolution cross-entropy encoding and decoding method of the present invention. Detailed Implementation

[0045] like Figure 1 As shown, this invention discloses a multi-resolution cross-entropy encoding / decoding method based on complex neural networks, comprising the following steps:

[0046] Step S101: After the input image x passes through a multi-level complex feature extraction unit, a complex-valued high-resolution latent representation y is obtained;

[0047] Step S102: The complex-valued high-resolution latent representation y obtained in step S101 is discretized using the quantization unit Q to obtain... To achieve further compression, we use... The quantized complex-valued high-resolution latent representation y is represented by the real and imaginary parts respectively. and express;

[0048] Step S103: In order to obtain the relevant information of the complex-valued high-resolution latent representation y in space and channels and improve the compression efficiency, the complex-valued high-resolution latent representation y is gradually input into the multi-level super encoder (corresponding to super encoder 1 and super encoder 2 in the figure. Super encoder 1 is also called the first-level super encoder, and super encoder 2 is also called the second-level super encoder). The latent representation y1 output by the first-level super encoder is a higher-level feature abstraction of y, while the latent representation y2 output by the second-level super encoder is a higher-level feature abstraction of y1.

[0049] Step S104: The low-resolution complex-valued latent representation y2 obtained in step S103 is quantized and further compressed to obtain... The corresponding real and imaginary parts are respectively and Similarly, quantizing the medium-resolution complex-valued latent representation y1 yields... The corresponding real and imaginary parts are respectively and

[0050] Step S105: Convert the real part of step S104... and the virtual part A lossless encoding (AE) operation is performed to obtain a binary bitstream, and then a lossless decoding (AD) operation is performed on the binary bitstream to restore it to its original form. and

[0051] Step S106: Restore the state described in step S105 and The real and imaginary parts of the complex value are input into the second-level superdecoder (superdecoder 2), and a medium-resolution latent representation is reconstructed through the second-level superdecoder (superdecoder 2). Prior information θ1 during encoding and decoding;

[0052] Step S106-1: Put Model it as a Gaussian distribution, and set the real part θ1 to θ2. 1r As prior information for the Gaussian distribution, the resulting Gaussian distribution q 1r With the real part of the quantized mid-resolution latent representation The actual distribution p 1r The closer they are, the higher the real part of the encoded quantized medium-resolution latent representation. The smaller the bitrate required;

[0053] Step S106-2: Using Gaussian distribution q 1r The real part of the quantized mid-resolution latent representation Lossless encoding yields a binary code rate, consuming a code rate of:

[0054]

[0055] Step S106-3: Lossless decoding and recovery of the binary bitstream and put and θ 1i The real and imaginary parts of the complex number are respectively input into the entropy parameter module EP3 to extract the encoding. Required prior information

[0056] Step S106-4: Similarly, using Bundle Modeled as a Gaussian distribution q 1iThrough Gaussian distribution q 1i right Lossless encoding is performed to obtain a binary bitstream, with a bitrate of:

[0057]

[0058] Step S106-5: Perform lossless decoding on the binary bitstream to recover the quantized intermediate resolution latent representation. imaginary part

[0059] Step S107: Quantize the medium-resolution latent representation obtained in step S104. The input is fed into the first-stage super decoder (super decoder 2), where the quantized medium-resolution latent representation is processed. The prior information θ2 (represented by θ2 for the real and imaginary parts) required to reconstruct the high-resolution latent representation y of complex values ​​is required. 2r and θ 2i ):

[0060] Step S107-1: Fuse the real part θ using the second entropy parameter module EP2 2r and the imaginary part θ 1r Get modeling (potential representation) Prior information of the real part And utilize prior information Bundle Gaussian distribution q r ;

[0061] Step S107-2: Using Gaussian distribution q r right Lossless encoding is performed to obtain a binary bitstream, with a bitrate of:

[0062]

[0063] Step S107-3: Lossless decoding of the binary code rate to restore the complex value latent representation. real part

[0064] Step S107-4: Transform the latent representation real part The imaginary part θ of the prior information θ2 2i The real and imaginary parts of the complex number are respectively input into the first entropy parameter module EP1 to obtain the modeling result. Prior information And using the extracted prior information to Modeled as a Gaussian distribution q i , Representing potential representations The imaginary part;

[0065] Step S107-5: Utilize the Gaussian distribution q from step S107-4 i right Lossless encoding is performed to obtain a binary bitstream, with a bitrate of:

[0066]

[0067] Step S107-6: Perform lossless decoding on the binary bitstream to restore the original data. imaginary part

[0068] Step S108: Put and Using the real and imaginary parts of the complex number respectively, we obtain the quantized high-resolution latent representation of the complex value. The above steps S101 to S108 complete one entropy coding process for an image.

[0069] The super encoder and super decoder employ a concatenated complex convolution and activation function, using the ReLU activation function. Both input and output are complex values. It's important to note that the first encoder (super encoder 1) spatially downsamples the latent representation by a factor of two, while the first super decoder (super decoder 1) spatially upsamples the latent representation by a factor of two. Using M... k (g) represents a complex convolution computation unit, then the computation steps of the supercoder / decoder can be represented as follows (k can be set according to actual needs):

[0070] g(y) = M k (M k-1 …M1(y)…) (5)

[0071] Entropy parameter module: combines real number calculation unit and complex number calculation unit.

[0072] The first entropy parameter module EP1 employs a multi-level complex convolution cascade approach, extracting the real and imaginary parts of the complex numbers, concatenating them along the channel dimension, and then fusing the real and imaginary parts using a residual block. Here, W represents the complex convolution operation, Cat represents the channel dimension concatenation operation, and RB represents the residual block. The specific calculation steps are as follows:

[0073]

[0074] The second entropy parameter module EP2: This module's input comes from real part information at different resolutions, so it uses a concatenated form of real convolutions. The activation function is the ReLU function, where U2 represents a real convolution with double upsampling, and V...k To represent ordinary real number convolution, the specific calculation steps are as follows:

[0075]

[0076] The third entropy parameter module EP3 is the same as the first entropy parameter module EP1. The specific calculation steps are as follows:

[0077]

[0078] Quantization: Quantization refers to the rounding operation, which can be uniform rounding or non-uniform rounding (vectorization).

[0079] Lossless encoding and decoding: Lossless encoding and lossless decoding modules refer to entropy coding technology.

[0080] Specific implementation methods can include Huffman coding, arithmetic coding, and inter-region coding. The technical innovation of this invention lies in:

[0081] 1. To address the latent representation characteristics of complex numbers, a dual-branch structure with real and imaginary parts is adopted, and cross-entropy coding is performed using the correlation between the two branches.

[0082] 2. The entropy model uses a multi-level encoder-decoder to extract features of different resolutions of the complex latent representation. Each level of encoder-decoder contains a complex numerical convolutional neural network computation unit and an activation unit.

[0083] 3. An entropy parameter module was designed, which can simultaneously integrate relevant information from different branches and relevant information at different resolutions to achieve entropy coding.

[0084] The beneficial effects of this invention are: 1. The method of this invention provides a robust complex numerical entropy encoding module, which no longer treats the real and imaginary parts of complex numbers as two independent parts during the entropy encoding process, but fully utilizes the spatial and channel correlation between the real and imaginary parts; 2. The method of this invention designs an entropy parameter module and a complex domain supercoder module, which improves the robustness of the entropy model against adversarial examples while ensuring compression performance.

[0085] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A multi-resolution cross-entropy encoding / decoding method based on complex neural networks, characterized in that, Includes the following steps: Step S101: After the input image x passes through a multi-level complex feature extraction unit, a complex-valued high-resolution latent representation y is obtained; Step S102: The complex-valued high-resolution latent representation y obtained in step S101 is discretized using the quantization unit Q to obtain... To achieve further compression; Step S103: Input the complex numerical high-resolution latent representation y into the multi-level super encoder step by step to obtain complex numerical latent representations y1 and y2 with different resolutions. The latent representation y1 output by the first-level super encoder is a higher-level feature abstraction of y, and the latent representation y2 output by the second-level super encoder is a higher-level feature abstraction of y1. Step S104: The low-resolution complex-valued latent representation y2 obtained in step S103 is quantized and further compressed to obtain... The corresponding real and imaginary parts are respectively and Similarly, quantizing the medium-resolution complex-valued latent representation y1 yields... The corresponding real and imaginary parts are respectively and Step S105: Convert the real part of step S104... and the virtual part Each lossless encoding operation yields a binary bitstream, which is then subjected to lossless decoding to restore the original binary bitstream. and Step S106: Restore the state described in step S105 and The real and imaginary parts of the complex value are input into the second-level superdecoder, and the second-level superdecoder reconstructs a medium-resolution latent representation. Prior information θ1 during encoding and decoding; Step S107: Quantize the medium-resolution latent representation obtained in step S104. The input is fed into the first-level super decoder, which then processes it... The prior information θ2 required to reconstruct the high-resolution latent representation y of complex values; Step S108: Put and Using the real and imaginary parts of the complex number respectively, we obtain the quantized high-resolution latent representation of the complex value.

2. The multi-resolution cross-entropy encoding / decoding method according to claim 1, characterized in that, Step S106 includes: Step S106-1: Put Model it as a Gaussian distribution, and set the real part θ1 to θ2. 1r As prior information for the Gaussian distribution, we obtain the Gaussian distribution q. 1r ; Step S106-2: Using Gaussian distribution q 1r The real part of the quantized mid-resolution latent representation Lossless encoding is performed to obtain the binary code rate; Step S106-3: Perform lossless decoding and recovery of the binary code stream from step S106-2. and put and θ 1i The real and imaginary parts of the complex number are respectively input into the third entropy parameter module EP3 to extract the encoding. Required prior information Step S106-4: Utilizing prior information Bundle Modeled as a Gaussian distribution q 1i Through Gaussian distribution q 1i right Lossless encoding is performed to obtain a binary bitstream; Step S106-5: Perform lossless decoding on the binary bitstream to recover the quantized intermediate resolution latent representation. imaginary part 3. The multi-resolution cross-entropy encoding / decoding method according to claim 2, characterized in that, In step S106-1, the Gaussian distribution q is obtained. 1r With the real part of the quantized mid-resolution latent representation The actual distribution p 1r The closer they are, the higher the real part of the encoded quantized medium-resolution latent representation. The smaller the bitrate required; In step S106-2, the consumed code rate is: In step S106-4, the consumed code rate is:

4. The multi-resolution cross-entropy encoding / decoding method according to claim 2, characterized in that, In step S107, the real part of the prior information θ2 is represented as θ 2r The imaginary part is represented by θ. 2i Step S107 further includes the following steps: Step S107-1: Fuse the real part θ using the second entropy parameter module EP2 2r and the imaginary part θ 1r Get modeling Prior information And utilize prior information Bundle Modeled as a Gaussian distribution q r , Representing potential representations The real part; Step S107-2: Using Gaussian distribution q r right Lossless encoding is performed to obtain a binary bitstream; Step S107-3: Lossless decoding of the binary code rate to restore the complex value latent representation. real part Step S107-4: Transform the latent representation real part The imaginary part θ of the prior information θ2 2i The real and imaginary parts of the complex number are respectively input into the first entropy parameter module EP1 to obtain the modeling result. Prior information And utilize the extracted prior information Bundle Modeled as a Gaussian distribution q i , Representing potential representations The imaginary part; Step S107-5: Utilize the Gaussian distribution q from step S107-4 i right Lossless encoding is performed to obtain a binary bitstream; Step S107-6: Perform lossless decoding on the binary bitstream to restore the original data. imaginary part 5. The multi-resolution cross-entropy encoding / decoding method according to claim 4, characterized in that, In step S107-2, the consumed code rate is: In step S107-5, the consumed code rate is:

6. The multi-resolution cross-entropy encoding / decoding method according to claim 1, characterized in that, In this multi-resolution cross-entropy encoding and decoding method, the super encoder uses a concatenation of complex convolution and activation function, with the ReLU function being used as the activation function, and both the input and output are complex values.

7. The multi-resolution cross-entropy encoding / decoding method according to claim 1, characterized in that, In this multi-resolution cross-entropy encoding and decoding method, the first-stage super encoder performs a spatial downsampling of the latent representation by a factor of two, and the first-stage super decoder performs a spatial upsampling of the latent representation by a factor of two.

8. The multi-resolution cross-entropy encoding / decoding method according to claim 4, characterized in that, In this multi-resolution cross-entropy encoding and decoding method, the first entropy parameter module EP1 and the third entropy parameter module EP3 adopt the form of multi-level complex convolution concatenation, the second entropy parameter module EP2 adopts the form of real convolution concatenation, and the activation function is the ReLU function.

9. A multi-resolution cross-entropy encoding / decoding system based on complex neural networks, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the multi-resolution cross-entropy encoding / decoding steps of any one of claims 1-8 when invoked by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of multi-resolution cross-entropy encoding / decoding as described in any one of claims 1-8.

Images