A method and system for determining an image compression model based on an asymmetric architecture

Abstract

This disclosure provides a method and system for determining an image compression model based on an asymmetric architecture. The method includes: acquiring a trained base neural network encoder / decoder; fixing the parameters of an analytical encoder, a super-prior encoder, a super-prior decoder, and a context model to determine a lightweight synthetic decoder; fixing the parameters of the analytical encoder and the lightweight synthetic decoder, and jointly determining a lightweight super-prior decoder, a lightweight context model, and a jointly trained super-prior encoder based on the super-prior decoder and the context model; and determining an asymmetric lightweight image compression model based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model. This disclosure simplifies the decoder network structure, reduces the computational complexity at the decoding end, improves decoding efficiency, and adapts to diverse low-end devices.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to a method and system for determining an image compression model based on an asymmetric architecture. Background Technology

[0002] With the increasing demand for image compression processing in Internet and multimedia applications, the development of efficient image compression methods has become increasingly important. Traditional image compression algorithms, such as JPEG, JPEG2000, HEVC, and VVC, typically employ an asymmetric computational architecture design, where the encoding end bears more computational complexity, while the decoding end is simplified to adapt to the computing power of low-end devices.

[0003] In recent years, deep learning-based codecs have made significant progress in both image and video processing. However, existing deep learning-based end-to-end image compression (LIC) methods mostly employ symmetric encoder-decoder architectures, resulting in roughly the same computational complexity for encoding and decoding. For example, image compression models such as Balle18, Minnen18, Cheng20, TinyLIC, and TCM maintain a symmetric structural design while improving compression performance. However, as the complexity of these models increases, the decoding time also lengthens, posing a challenge for practical deployment on resource-constrained low-end devices.

[0004] Although there are lightweight design schemes such as simplified model structure or knowledge distillation in existing research, there is still no asymmetric deterministic method for the field of image compression. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this disclosure is to provide a method and system for determining an image compression model based on an asymmetric architecture.

[0006] To achieve the above objectives, according to one aspect of this disclosure, a method for determining an image compression model based on an asymmetric architecture is provided, comprising:

[0007] Obtain a trained basic neural network codec, which includes an analysis encoder, a synthesis decoder, a super-prior encoder, a super-prior decoder, and a context model;

[0008] By fixing the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder, and the context model, a lightweight design process is performed on the synthesis decoder to determine a lightweight synthesis decoder.

[0009] By fixing the parameters of the analysis encoder and the lightweight synthesis decoder, and jointly designing the super-prior decoder and the context model with the super-prior encoder, the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder are determined.

[0010] An asymmetric lightweight image compression model is determined based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model.

[0011] Optionally, fixing the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder, and the context model, and performing lightweight design processing on the synthesizer to determine the lightweight synthesizer includes:

[0012] The parameters of the analytical encoder, the super-prior encoder, the super-prior decoder, and the context model are fixed, while the window-based multi-head attention in the Swin Transformer block of the synthetic decoder is preserved.

[0013] Remove the shift window configuration from the synthesizer;

[0014] The barrel channel structure in the synthesizer is designed as an inverted pyramid channel structure to determine the lightweight synthesizer.

[0015] Optionally, the method further includes:

[0016] The lightweight synthesizer is optimized and trained using a preset first loss function to determine the lightweight synthesizer. The preset first loss function is a distortion loss function.

[0017] Optionally, fixing the parameters of the analysis encoder and the lightweight super-prior decoder, and jointly designing a lightweight super-prior decoder and the context model using the super-prior encoder to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder includes:

[0018] Fix the parameters of the analysis encoder and the lightweight super-prior decoder, and remove the shift window configuration of the super-prior decoder and the context model;

[0019] The network that obtains the mean of the Gaussian distribution parameters and the network that obtains the variance in the super-prior decoder are merged to determine the lightweight super-prior decoder.

[0020] The lightweight context model is determined by reducing the number of channels in the self-attention module of the context model to a first preset threshold and reducing the number of layers in the residual network of the context model to a second preset threshold.

[0021] Optionally, the step of fixing the parameters of the analysis encoder and the lightweight super-prior decoder, and jointly designing the super-prior decoder and the context model using the super-prior encoder to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder further includes:

[0022] The super-prior encoder, the lightweight super-prior decoder, and the lightweight context model are jointly trained using a preset second loss function to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder. The preset second loss function includes a distortion component and a bit rate component.

[0023] Optionally, obtaining the trained basic neural network codec includes:

[0024] Obtain a baseline deep learning image codec, which includes an analytical encoder, a synthetic decoder, a super-prior encoder, a super-prior decoder, and a context model;

[0025] The training images are input into the analysis encoder to determine the latent representations of the training images;

[0026] The latent representation of the training image is quantized to determine the quantized latent representation of the training image;

[0027] The quantized latent representation of the training image is input into the synthetic decoder to determine the reconstructed image;

[0028] The basic neural network codec is optimized using a preset loss function to determine the trained basic neural network codec.

[0029] According to a second aspect of this disclosure, a system for determining an image compression model based on an asymmetric architecture is provided, comprising:

[0030] The acquisition module is used to acquire the trained basic neural network codec, which includes an analysis encoder, a synthesis decoder, a super-prior encoder, a super-prior decoder, and a context model.

[0031] The first lightweight design module is used to fix the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder and the context model, perform lightweight design processing on the synthesizer, and determine the lightweight synthesizer.

[0032] The second lightweight design module is used to fix the parameters of the analysis encoder and the lightweight super-prior decoder, and to perform lightweight design on the super-prior decoder and the context model in conjunction with the super-prior encoder, thereby determining the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder.

[0033] The merging module is used to determine an asymmetric lightweight image compression model based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model.

[0034] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method provided in the first aspect of this disclosure.

[0035] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0036] A memory on which computer programs are stored;

[0037] A processor for executing the computer program in the memory to implement the steps of the method provided in the first aspect of this disclosure.

[0038] Compared with the prior art, the embodiments disclosed herein have at least one of the following beneficial effects:

[0039] The above technical solution achieves an asymmetric architecture design for the image compression model by lightweighting the synthesis decoder at the decoding end and the super-prior decoder and context model of the decoding module on the super-prior path. This simplifies the decoder network structure, improves compression efficiency, and optimizes the balance between decoding complexity and compression performance. While maintaining high encoding efficiency, it can significantly reduce the computational complexity at the decoding end and improve decoding efficiency, thus adapting to diverse deployment needs of low-end devices. Attached Figure Description

[0040] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0041] Figure 1This is a flowchart illustrating a method for determining an image compression model based on an asymmetric architecture, according to an exemplary embodiment.

[0042] Figure 2 This is a schematic diagram illustrating the design structure of an image compression model based on an asymmetric architecture, according to an exemplary embodiment.

[0043] Figure 3 This is a schematic diagram illustrating a lightweight design process for a synthesizer decoder according to an exemplary embodiment.

[0044] Figure 4 This is a schematic diagram comparing the codec-PSNR index curves of various codecs according to an exemplary embodiment.

[0045] Figure 5 This is a schematic diagram illustrating a comparison of compression efficiency, decoding complexity, and number of decoding parameters for various codecs according to an exemplary embodiment.

[0046] Figure 6 This is a block diagram illustrating a determination system for an image compression model based on an asymmetric architecture, according to an exemplary embodiment. Detailed Implementation

[0047] The present disclosure will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all fall within the protection scope of the present disclosure.

[0048] Figure 1 This is a flowchart illustrating a method for determining an image compression model based on an asymmetric architecture, according to an exemplary embodiment. Figure 2 This is a schematic diagram illustrating the design structure of an image compression model based on an asymmetric architecture, according to an exemplary embodiment.

[0049] like Figure 1 , Figure 2 As shown, this disclosure provides a method for determining an image compression model based on an asymmetric architecture, including steps S11 to S14.

[0050] S11, obtain the trained base neural network encoder / decoder.

[0051] The basic neural network encoder-decoder includes the analysis encoder g a Synthesizer / decoder g s , Advanced prior encoder h a 、 Super-prior decoder h s and context model fc .

[0052] The basic neural network encoder and decoder are pre-trained.

[0053] S12, with fixed parameters for the analysis encoder, super-prior encoder, super-prior decoder, and context model, performs lightweight design processing on the synthesizer to determine the lightweight synthesizer.

[0054] Among them, the fixed analysis encoder g a , Advanced prior encoder h a 、 Super-prior decoder h s and context model f c The parameters, i.e., fixed except for the synthesizer / decoder g s All module parameters except those mentioned above.

[0055] The lightweight synthesizer is represented as

[0056] S13, with fixed parameters for the analysis encoder and lightweight synthesis decoder, performs lightweight design of the super-prior decoder and context model in conjunction with the super-prior encoder, and determines the lightweight super-prior decoder, lightweight context model, and jointly trained super-prior encoder.

[0057] Among them, the fixed analysis encoder g a Lightweight synthesizer / decoder That is, fix the super-prior encoder h a 、 Super-prior decoder h s and context model f c All module parameters except those mentioned above.

[0058] The lightweight hyperprior decoder is represented as The lightweight context model is represented as The jointly trained hyper-prior encoder represents h. a .

[0059] S14. Based on the analytical encoder, lightweight synthetic decoder, jointly trained super-prior encoder, lightweight super-prior decoder, and lightweight context model, determine the asymmetric lightweight image compression model.

[0060] Combined analysis of encoder g a Lightweight synthesizer / decoder The jointly trained hyper-prior encoder h a Lightweight advanced prior decoder and lightweight context models Obtain all parameters to get an asymmetric lightweight image compression model with an asymmetric encoder-decoder.

[0061] The above technical solution achieves an asymmetric architecture design for the image compression model by lightweighting the synthesis decoder at the decoding end and the super-prior decoder and context model of the decoding module on the super-prior path. This simplifies the decoder network structure, improves compression efficiency, and optimizes the balance between decoding complexity and compression performance. While maintaining high encoding efficiency, it can significantly reduce the computational complexity at the decoding end and improve decoding efficiency, thus adapting to diverse deployment needs of low-end devices.

[0062] In one possible embodiment, S11 may include S21 to S25.

[0063] S21, Obtain the baseline deep learning image codec, which also includes the analytic encoder, the synthesis decoder, the super-prior encoder, the super-prior decoder, and the context model.

[0064] The analyzer encoder is represented as g. a The synthesizer / decoder is represented as g. s The hyper-prior encoder is represented as h a The super-prior decoder is represented as h s And the context model is represented as f c The encoder is represented as g. a The synthesizer / decoder is represented as g. s The structures are mutually symmetrical, and the hyper-prior encoder is represented as h. a The super-prior decoder is represented as h s The structures are mutually symmetrical.

[0065] S22, input the training image into the analysis encoder to determine the latent representation of the training image.

[0066] The training image x is analyzed by the encoder g. a The mapping is to a compact latent representation y, which is the latent representation of the training image x.

[0067] S23, quantize the latent representation of the training image to determine the quantized latent representation of the training image.

[0068] The quantized latent representation of the training image is expressed as:

[0069] S24, input the quantized latent representation of the training image into the synthesis decoder to determine the reconstructed image.

[0070] Among them, the quantized latent representation of the training images Through the synthesizer g s Inverse mapping to reconstructed image

[0071] In this disclosure, the hyper-prior encoder h a Used to capture spatial dependencies and generate auxiliary information z, the super-prior decoder h s and context model f c Provides prior parameters for the entropy model.

[0072] S25, the basic neural network codec is optimized using a preset loss function to determine the trained basic neural network codec.

[0073] Based on reconstructed images Using a training image x and a preset loss function, the basic neural network codec is optimized through end-to-end learning and training. The preset loss function includes the bitrate and the reconstructed image. The distortion loss between the training image x and the training image x.

[0074] In one possible embodiment, the preset loss function is:

[0075] L=R+λD

[0076] Where L represents loss, R represents bit rate, D represents distortion, and λ represents Lagrange multiplier.

[0077] The parameters of the encoder end of the trained basic neural encoder-decoder, i.e. the parameters of the analysis encoder, remain unchanged in subsequent processing.

[0078] Figure 3 This is a schematic diagram illustrating a lightweight design process for a synthesizer decoder according to an exemplary embodiment.

[0079] like Figure 3 As shown, in one possible embodiment, S12 may include S31 to S33.

[0080] S31 fixes the parameters of the analytical encoder, super-prior encoder, super-prior decoder, and context model, while preserving the window-based multi-head attention in the Swing Transformer block of the synthetic decoder.

[0081] S32, Remove the shift window configuration in the synthesizer.

[0082] As an example, the Swing Transformer block in the synthesizer can expand the receptive field of the entire network. It can generate a compact latent representation by taking pixel correlations across the entire image. In fact, since local priors already exist in the encoding task, only the synthesizer g is retained.s The window-based multi-head attention in the Swing Transformer block removes the shift window configuration and restricts information collection to a fixed window size.

[0083] S33, by designing the barrel channel structure in the synthesizer as an inverted pyramid channel structure, determines a lightweight synthesizer.

[0084] As an example, design a synthesizer / decoder g s The inverted pyramid channel structure, in which the latent representation The number of channels is gradually reduced to a preset channel number threshold, the width is gradually increased to a preset width threshold, and the height is gradually increased to a preset height threshold to prevent a significant increase in the computational complexity of subsequent network layers.

[0085] Among them, in the synthesizer g s The process involves four adjustments to the number of channels, width, and height. Each adjustment includes an upsampling process, and the width and height of the potential representation are doubled in each adjustment. In the first adjustment, the number of channels is reduced from 256 to 128; in the second adjustment, it is reduced from 256 to 96; in the third adjustment, it is reduced from 256 to 64; and in the fourth adjustment, it is reduced from 64 to 3. The output of the fourth channel is 3, ensuring that the output image has RGB three channels.

[0086] Through the above steps S31 to S33, the synthesizer / decoder g s Modified to a lightweight network structure, employing a simple, lightweight synthesizer / decoder. Replace the original synthesizer / decoder g s .

[0087] In one possible embodiment, S12 may also include S33.

[0088] The lightweight synthesizer is optimized and trained using a pre-defined first loss function to determine the lightweight synthesizer.

[0089] The preset first loss function is the distortion loss function. Based on the reconstructed image... The distortion loss between the training image x and the lightweight synthetic decoder Optimize training and rapidly fine-tune to obtain a lightweight synthesis decoder.

[0090] In one possible embodiment, S13 may include S41 to S43.

[0091] S41 fixes the parameters of the analysis encoder and the lightweight super-prior decoder, and removes the shift window configuration for the super-prior decoder and the context model.

[0092] S42, merge the network that obtains the mean of the Gaussian distribution parameters and the network that obtains the variance in the super-prior decoder to determine the lightweight super-prior decoder.

[0093] Specifically, the network that obtains the mean of the Gaussian distribution parameters and the network that obtains the variance in the super-prior decoder are merged into one network.

[0094] S43, reduce the number of channels in the self-attention module of the context model to a first preset threshold, reduce the number of layers in the residual network of the context model to a second preset threshold, and determine the lightweight context model.

[0095] Among these improvements, the number of channels in the self-attention module of the context model was reduced from 128 to 64, and the residual network was reduced from three convolutional layers to one convolutional layer.

[0096] As an example, in the lightweight design of the super-prior path, the shift window configuration of the super-prior decoder and the context model is removed, the network for obtaining the mean of the Gaussian distribution parameters and the network for obtaining the variance in the super-prior decoder are merged, and the number of channels of the self-attention module and the number of layers of the residual network in the context network are further reduced, thus determining the lightweight super-prior decoder and the lightweight context model.

[0097] Through the above steps S41 to S43, the advanced prior decoder h s and context model f c Modified to a lightweight network structure, employing a simple and lightweight super-prior decoder. Replace the original hyper-prior decoder h s It adopts a lightweight context model. Replace the original context model f c .

[0098] In one possible embodiment, S13 may also include S44.

[0099] S44, using a pre-defined second loss function, jointly trains the super-prior encoder, the lightweight super-prior decoder, and the lightweight context model to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder.

[0100] The preset second loss function includes a distortion component and a bitrate component. Based on the reconstructed image... The distortion loss and bitrate between the training image x and the lightweight super-prior decoder, lightweight context model and super-prior encoder are jointly trained. The adaptive training of the lightweight super-prior decoder and super-prior encoder is achieved by fast fine-tuning, and the lightweight super-prior encoder, lightweight context model and jointly trained super-prior encoder are obtained.

[0101] In this phase, the hyper-prior encoder h is not fixed. a The parameters are obtained by using a hyper-prior encoder h. a With lightweight hyper-prior decoders Joint training is performed to ensure the lightweight super-prior decoder is optimized. Capable of adapting to the priori encoder h a The obtained prior knowledge.

[0102] This disclosure adopts a phased training strategy that gradually replaces complex decoders with simpler modules. It first replaces and trains the synthetic decoder, and then replaces and trains the super-prior decoder and context model included in the super-prior path. At the same time, it performs efficient design for the synthetic decoder, super-prior decoder and context model, optimizes the balance between computational complexity and compression performance, and finally obtains an asymmetric lightweight image compression model.

[0103] The training and inference method of the asymmetric lightweight image compression model disclosed herein is the same as the training method of the baseline deep learning image encoder described above, and will not be repeated here.

[0104] In one possible embodiment, the image compression results are evaluated using the image compression model based on an asymmetric architecture provided in this disclosure and the image compression results of a conventional codec.

[0105] The Kodak, Tecnick, and CLIC professional validation datasets were used as the test image dataset. The bitrate-PSNR curve was used as the evaluation metric, and the corresponding BD-rate savings were calculated. In the bitrate-PSNR curve, a higher PSNR is better; that is, the curve at the top indicates better reconstruction quality and better compression performance at the same bitrate. The BD-rate savings quantitatively indicate the percentage reduction in bitrate while maintaining the same reconstruction quality.

[0106] Figure 4 This is a schematic diagram comparing the codec-PSNR index curves of various codecs according to an exemplary embodiment.

[0107] like Figure 4As shown, the bitrate-PSNR performance of the image compression model based on asymmetric architecture provided in this disclosure and traditional codecs are compared. The traditional codecs include HM and VTM, and the learning-based image compression models include Balle18, Minnen18, Cheng20, Xie21, Qian22, and TinyLIC. The horizontal axis bpp (bits-per-pixel) represents the number of bits required per pixel, and the vertical axis represents the peak signal-to-noise ratio of the PSNR reconstructed image and the original image.

[0108] like Figure 4 As shown, the image compression model based on asymmetric architecture provided in this disclosure maintains similar compression performance to the traditional video encoder VTM on the Kodak, Tecnick, and CLIC professional validation datasets. Furthermore, on the anchor point BPG (HEVC intra), the image compression model based on asymmetric architecture provided in this disclosure achieves a BD-rate saving of 18.68%.

[0109] Figure 5 This is a schematic diagram illustrating a comparison of compression efficiency, decoding complexity, and number of decoding parameters for various codecs according to an exemplary embodiment.

[0110] like Figure 5 As shown, this disclosure also uses BD-rate-decoding computation-decoding parameter count to measure the balance between compression performance and decoding complexity. The horizontal axis represents the decoding end MACs (Multiply-Accumulate Operations), and the vertical axis represents the compression performance BD-rate (%). The closer the colored circle is to the upper left corner, the higher its compression performance and the lower its decoding complexity. The radius of the colored circle represents the number of decoding parameters. The longer the radius, the more decoding parameters there are.

[0111] like Figure 5 As shown, compared with other traditional codecs, the image compression model based on asymmetric architecture provided in this disclosure can achieve the lowest decoding complexity while maintaining high compression performance. Therefore, the image compression model based on asymmetric architecture determined by the method for determining the image compression model based on asymmetric architecture provided in this disclosure can achieve the best balance between compression performance and computational complexity.

[0112] Figure 6 This is a block diagram illustrating a determination system for an image compression model based on an asymmetric architecture, according to an exemplary embodiment.

[0113] Based on the same concept, this disclosure also provides a system 100 for determining an image compression model based on an asymmetric architecture, such as Figure 6 As shown, it includes: an acquisition module 110, a first lightweight design module 120, a second lightweight design module 130, and a combination module 140.

[0114] The acquisition module 110 is used to acquire the trained basic neural network codec, which includes an analysis encoder, a synthesis decoder, a super prior encoder, a super prior decoder, and a context model.

[0115] The first lightweight design module 120 is used to fix the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder and the context model, perform lightweight design processing on the synthesizer, and determine the lightweight synthesizer.

[0116] The second lightweight design module 130 is used to fix the parameters of the analysis encoder and the lightweight super-prior decoder, and to perform lightweight design on the super-prior decoder and the context model in conjunction with the super-prior encoder, thereby determining the lightweight super-prior decoder, the lightweight context model, and the super-prior encoder that has been jointly trained.

[0117] The combination module 140 is used to determine an asymmetric lightweight image compression model based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model.

[0118] The above technical solution achieves an asymmetric architecture design for the image compression model by lightweighting the synthesis decoder at the decoding end and the super-prior decoder and context model of the decoding module on the super-prior path. This simplifies the decoder network structure, improves compression efficiency, and optimizes the balance between decoding complexity and compression performance. While maintaining high encoding efficiency, it can significantly reduce the computational complexity at the decoding end and improve decoding efficiency, thus adapting to diverse deployment needs of low-end devices.

[0119] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0120] Based on the same concept described above, in another embodiment of this disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes a method for determining an image compression model based on an asymmetric architecture when executing the program.

[0121] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0122] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0123] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0124] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0125] In this embodiment of the disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of a method for determining an image compression model based on an asymmetric architecture in any of the above embodiments.

[0126] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this disclosure.

[0131] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for determining an image compression model based on an asymmetric architecture, characterized in that, include: Obtain a trained basic neural network codec, which includes an analysis encoder, a synthesis decoder, a super-prior encoder, a super-prior decoder, and a context model; By fixing the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder, and the context model, a lightweight design process is performed on the synthesis decoder to determine a lightweight synthesis decoder. By fixing the parameters of the analysis encoder and the lightweight synthesis decoder, and jointly designing the super-prior decoder and the context model with the super-prior encoder, the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder are determined. Based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model, an asymmetric lightweight image compression model is determined. The step of fixing the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder, and the context model, and performing lightweight design processing on the synthesizer to determine the lightweight synthesizer includes: The parameters of the analytical encoder, the super-prior encoder, the super-prior decoder, and the context model are fixed, while the window-based multi-head attention in the Swin Transformer block of the synthetic decoder is preserved. Remove the shift window configuration from the synthesizer; The barrel channel structure in the synthesizer is designed as an inverted pyramid channel structure to determine the lightweight synthesizer; The steps involve fixing the parameters of the analytical encoder and the lightweight synthetic decoder, and jointly designing a lightweight super-prior decoder and the context model using the super-prior encoder to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder, including: Fix the parameters of the analysis encoder and the lightweight synthesis decoder, and remove the shift window configuration of the super-prior decoder and the context model; The network that obtains the mean of the Gaussian distribution parameters and the network that obtains the variance in the super-prior decoder are merged to determine the lightweight super-prior decoder. The lightweight context model is determined by reducing the number of channels in the self-attention module of the context model to a first preset threshold and reducing the number of layers in the residual network of the context model to a second preset threshold.

2. The method according to claim 1, characterized in that, The method further includes: The lightweight synthesizer is optimized and trained using a preset first loss function to determine the lightweight synthesizer. The preset first loss function is a distortion loss function.

3. The method according to claim 1, characterized in that, The process of fixing the parameters of the analytical encoder and the lightweight synthetic decoder, and jointly designing a lightweight super-prior decoder and the context model with the super-prior encoder to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder, further includes: The super-prior encoder, the lightweight super-prior decoder, and the lightweight context model are jointly trained using a preset second loss function to determine the lightweight super-prior decoder, the lightweight context model, and the jointly trained super-prior encoder. The preset second loss function includes a distortion component and a bit rate component.

4. The method according to claim 1, characterized in that, The acquisition of the trained basic neural network codec includes: Obtain a baseline deep learning image codec, which includes an analytical encoder, a synthetic decoder, a super-prior encoder, a super-prior decoder, and a context model; The training images are input into the analysis encoder to determine the latent representations of the training images; The latent representation of the training image is quantized to determine the quantized latent representation of the training image; The quantized latent representation of the training image is input into the synthetic decoder to determine the reconstructed image; The basic neural network codec is optimized using a preset loss function to determine the trained basic neural network codec.

5. The method according to claim 4, characterized in that, The preset loss function is: ； Where L represents loss, R represents bit rate, and D represents distortion. It represents the Lagrange multiplier.

6. A system for determining an image compression model based on an asymmetric architecture, characterized in that, include: The acquisition module is used to acquire the trained basic neural network codec, which includes an analysis encoder, a synthesis decoder, a super-prior encoder, a super-prior decoder, and a context model. The first lightweight design module is used to fix the parameters of the analysis encoder, the super-prior encoder, the super-prior decoder and the context model, perform lightweight design processing on the synthesizer, and determine the lightweight synthesizer. The second lightweight design module is used to fix the parameters of the analysis encoder and the lightweight synthesis decoder, and to perform lightweight design on the super-prior decoder and the context model in conjunction with the super-prior encoder, thereby determining the lightweight super-prior decoder, the lightweight context model, and the super-prior encoder that has been jointly trained. The combination module is used to determine an asymmetric lightweight image compression model based on the analytical encoder, the lightweight synthetic decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder, and the lightweight context model. The first lightweight design module is used for: The parameters of the analytical encoder, the super-prior encoder, the super-prior decoder, and the context model are fixed, while the window-based multi-head attention in the Swin Transformer block of the synthetic decoder is preserved. Remove the shift window configuration from the synthesizer; The barrel channel structure in the synthesizer is designed as an inverted pyramid channel structure to determine the lightweight synthesizer; The second lightweight design module is used for: Fix the parameters of the analysis encoder and the lightweight synthesis decoder, and remove the shift window configuration of the super-prior decoder and the context model; The network that obtains the mean of the Gaussian distribution parameters and the network that obtains the variance in the super-prior decoder are merged to determine the lightweight super-prior decoder. The lightweight context model is determined by reducing the number of channels in the self-attention module of the context model to a first preset threshold and reducing the number of layers in the residual network of the context model to a second preset threshold.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-5.

8. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-5.

Images