Method and system for determining image compression model based on asymmetric architecture

Abstract

The invention provides a method and system for determining an image compression model based on an asymmetric architecture, and the method comprises the steps: obtaining a trained basic neural network codec; fixing parameters of the analysis encoder, the super-prior encoder, the super-prior decoder and the context model, and determining a lightweight synthesis decoder; fixing parameters of the analysis encoder and the lightweight synthesis decoder, and determining a lightweight super-prior decoder, a lightweight context model and a super-prior encoder subjected to joint training for the super-prior decoder and the context model in combination with the super-prior encoder; and determining an asymmetric lightweight image compression model according to the analysis encoder, the lightweight synthesis decoder, the super-prior encoder subjected to joint training, the lightweight super-prior decoder and the lightweight context model. According to the invention, the decoder network structure is simplified, the calculation complexity of the decoding end is reduced, the decoding efficiency is improved, and the decoder can adapt to diversified low-end equipment.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to a method and system for determining an image compression model based on an asymmetric architecture. Background Art

[0002] As the demand for image compression processing in Internet and multimedia applications continues to increase, the development of efficient image compression methods has become increasingly important. Traditional image compression algorithms, such as JPEG, JPEG2000, HEVC, and VVC, usually adopt an asymmetric computing architecture design, where the encoding end bears more computational complexity, while the decoding end is more simplified to adapt to the computing power of low-end devices.

[0003] In recent years, deep learning-based codecs have made significant progress in both the image and video fields. However, existing deep learning-based end-to-end image compression (LIC) methods mostly use a symmetric encoder-decoder architecture, resulting in roughly the same computational complexity for encoding and decoding. For example, image compression models such as Balle18, Minnen18, Cheng20, TinyLIC, and TCM improve compression performance while maintaining a symmetric structural design. However, as the complexity of these models increases, the decoding time also increases, which poses a challenge for actual deployment on low-end devices with limited resources.

[0004] Although there are lightweight design solutions such as simplifying model structure or knowledge distillation in existing research, there is no asymmetric determination method for the field of image compression. Summary of the invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for determining an image compression model based on an asymmetric architecture.

[0006] To achieve the above objective, according to one aspect of the present disclosure, a method for determining an image compression model based on an asymmetric architecture is provided, comprising:

[0007] Obtain a trained basic neural network codec, wherein the basic neural network codec includes an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model;

[0008] Fixing the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, performing lightweight design processing on the synthesis decoder, and determining a lightweight synthesis decoder;

[0009] Fixing the parameters of the analysis encoder and the lightweight synthesis decoder, jointly designing the super a priori decoder and the context model with the super a priori encoder, and determining a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder;

[0010] An asymmetric lightweight image compression model is determined according to the analysis encoder, the lightweight synthesis decoder, the jointly trained super a priori encoder, the lightweight super a priori decoder and the lightweight context model.

[0011] Optionally, fixing the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, performing lightweight design processing on the synthesis decoder, and determining a lightweight synthesis decoder includes:

[0012] Fixing the parameters of the analysis encoder, the super-a priori encoder, the super-a priori decoder, and the context model, retaining the window-based multi-head attention in the Swin Transformer block in the synthesis decoder;

[0013] removing a shift window configuration in the synthesis decoder;

[0014] The barrel channel structure in the synthesis decoder is designed as a reverse pyramid channel structure to determine the lightweight synthesis decoder.

[0015] Optionally, the method further comprises:

[0016] The lightweight synthesis decoder is optimized and trained by using a preset first loss function to determine the lightweight synthesis decoder, wherein the preset first loss function is a distortion loss function.

[0017] Optionally, the fixing of the analysis encoder and the parameters of the lightweight super a priori decoder, the lightweight design of the super a priori decoder and the context model in conjunction with the super a priori encoder, and the determination of the lightweight super a priori decoder, the lightweight context model and the jointly trained super a priori encoder include:

[0018] Fixing the parameters of the analysis encoder and the lightweight super-a priori decoder, and removing the shift window configuration of the super-a priori decoder and the context model;

[0019] The network for obtaining the mean of Gaussian distribution parameters and the network for obtaining the variance in the super a priori decoder are combined to determine the lightweight super a priori decoder;

[0020] The number of channels of the self-attention module in the context model is reduced to a first preset threshold, the number of layers of the residual network in the context model is reduced to a second preset threshold, and the lightweight context model is determined.

[0021] Optionally, the fixing of the parameters of the analysis encoder and the lightweight super a priori decoder, performing lightweight design on the super a priori decoder and the context model in conjunction with the super a priori encoder, and determining a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder further includes:

[0022] The super a priori encoder, the lightweight super a priori decoder and the lightweight context model are jointly trained using a preset second loss function to determine the lightweight super a priori decoder, the lightweight context model and the jointly trained super a priori encoder, and the preset second loss function includes a distortion part and a bit rate part.

[0023] Optionally, obtaining a trained basic neural network codec includes:

[0024] Obtaining a baseline deep learning image codec, the baseline deep learning image codec comprising an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model;

[0025] Inputting a training image into the analysis encoder and determining a latent representation of the training image;

[0026] quantizing the latent representation of the training image to determine the quantized latent representation of the training image;

[0027] Inputting the quantized latent representation of the training image into the synthesis decoder to determine a reconstructed image;

[0028] The basic neural network codec is optimized by using a preset loss function to determine the trained basic neural network codec.

[0029] According to a second aspect of the present disclosure, a system for determining an image compression model based on an asymmetric architecture is provided, comprising:

[0030] An acquisition module, used to acquire a trained basic neural network codec, wherein the basic neural network codec includes an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model;

[0031] A first lightweight design module is used to fix the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, perform lightweight design processing on the synthesis decoder, and determine a lightweight synthesis decoder;

[0032] A second lightweight design module is used to fix the parameters of the analysis encoder and the lightweight super a priori decoder, perform lightweight design on the super a priori decoder and the context model in conjunction with the super a priori encoder, and determine a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder;

[0033] A combination module is used to determine an asymmetric lightweight image compression model based on the analysis encoder, the lightweight synthesis decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder and the lightweight context model.

[0034] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method provided in the first aspect of the present disclosure are implemented.

[0035] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0036] a memory having a computer program stored thereon;

[0037] A processor is used to execute the computer program in the memory to implement the steps of the method provided in the first aspect of the present disclosure.

[0038] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0039] Through the above technical scheme, by lightweight design processing of the synthetic decoder at the decoding end and the hyper-prior decoder and context model of the decoding module on the hyper-prior path, the asymmetric architecture design of the image compression model is realized, the decoder network structure is simplified, the compression efficiency is improved, and the balance between decoding complexity and compression performance is optimized. While maintaining a high coding efficiency, the computational complexity of the decoding end can be significantly reduced, the decoding efficiency can be improved, and it can adapt to diverse low-end device deployment needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0041] Figure 1It is a flowchart of a method for determining an image compression model based on an asymmetric architecture according to an exemplary embodiment.

[0042] Figure 2 It is a schematic diagram of the design structure of an image compression model based on an asymmetric architecture according to an exemplary embodiment.

[0043] Figure 3 The present invention is a schematic diagram showing a process of lightweight design of a synthesis decoder according to an exemplary embodiment.

[0044] Figure 4 The figure is a schematic diagram showing a comparison of codec-PSNR indicator curves of a plurality of codecs according to an exemplary embodiment.

[0045] Figure 5 It is a schematic diagram showing a comparison of compression efficiency-decoding complexity-decoding parameter amount of various codecs according to an exemplary embodiment.

[0046] Figure 6 It is a block diagram of a system for determining an image compression model based on an asymmetric architecture according to an exemplary embodiment. DETAILED DESCRIPTION

[0047] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.

[0048] Figure 1 It is a flowchart of a method for determining an image compression model based on an asymmetric architecture according to an exemplary embodiment. Figure 2 It is a schematic diagram of the design structure of an image compression model based on an asymmetric architecture according to an exemplary embodiment.

[0049] like Figure 1 , Figure 2 As shown, the present disclosure provides a method for determining an image compression model based on an asymmetric architecture, including S11 to S14.

[0050] S11, obtain the trained basic neural network codec.

[0051] The basic neural network encoder-decoder includes the analysis encoder g a , synthesis decoder g s , super prior encoder h a , Super Prior Decoder h s And the context model fc .

[0052] Pre-trained base neural network encoder-decoder.

[0053] S12, fixing the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, performing lightweight design processing on the synthesis decoder, and determining a lightweight synthesis decoder.

[0054] Among them, the fixed analysis encoder g a , super prior encoder h a , Super Prior Decoder h s And the context model f c The parameters of the fixed synthesis decoder g s All module parameters except .

[0055] The lightweight synthetic decoder is represented as

[0056] S13, fix the parameters of the analysis encoder and the lightweight synthesis decoder, jointly use the super-prior encoder to lightweight design the super-prior decoder and the context model, and determine the lightweight super-prior decoder, the lightweight context model and the jointly trained super-prior encoder.

[0057] Among them, the fixed analysis encoder g a , lightweight synthesis decoder That is, the fixed super prior encoder h a , Super Prior Decoder h s And the context model f c All module parameters except .

[0058] The lightweight super-prior decoder is expressed as The lightweight context model is expressed as The jointly trained hyper-prior encoder representation h a .

[0059] S14, determining an asymmetric lightweight image compression model according to the analysis encoder, the lightweight synthesis decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder and the lightweight context model.

[0060] Combined analysis encoder g a , lightweight synthesis decoder The jointly trained hyper-prior encoder h a , lightweight super-prior decoder And lightweight context model All parameters of are used to obtain an asymmetric lightweight image compression model with an asymmetric encoder-decoder.

[0061] Through the above technical scheme, by lightweight design processing of the synthetic decoder at the decoding end and the hyper-prior decoder and context model of the decoding module on the hyper-prior path, the asymmetric architecture design of the image compression model is realized, the decoder network structure is simplified, the compression efficiency is improved, and the balance between decoding complexity and compression performance is optimized. While maintaining a high coding efficiency, the computational complexity of the decoding end can be significantly reduced, the decoding efficiency can be improved, and it can adapt to diverse low-end device deployment needs.

[0062] In a possible embodiment, S11 may include S21 to S25.

[0063] S21, obtain a baseline deep learning image codec, which also includes an analysis encoder, a synthesis decoder, a super prior encoder, a super prior decoder, and a context model.

[0064] Among them, the analysis encoder is represented by g a , the composite decoder is denoted as g s , the super prior encoder is denoted as h a , the super prior decoder is denoted as h s And the context model is represented as f c The analysis encoder is represented by g a , the synthesized decoder is denoted as g s The structures are symmetrical to each other, and the super prior encoder is represented by h a , the super prior decoder is denoted as h s The structures are symmetrical to each other.

[0065] S22, input the training image into the analysis encoder to determine the potential representation of the training image.

[0066] Among them, the training image x is passed through the analysis encoder g a is mapped to a compact latent representation y, which is the latent representation of the training image x.

[0067] S23, quantizing the latent representation of the training image to determine the quantized latent representation of the training image.

[0068] Among them, the quantized potential representation of the training image is expressed as

[0069] S24, inputting the quantized latent representation of the training image into a synthesis decoder to determine a reconstructed image.

[0070] Among them, the quantized potential representation of the training image By synthesizing the decoder g s Inverse mapping to reconstruct the image

[0071] In the present disclosure, the super prior encoder h a Used to capture spatial dependencies, generate auxiliary information z, and super prior decoder h s and context model f c Provide hyper-prior parameters for the entropy model.

[0072] S25, optimizing the basic neural network codec using a preset loss function to determine a trained basic neural network codec.

[0073] Based on the reconstructed image And the training image x and the preset loss function, optimize the basic neural network codec, and perform end-to-end learning and training on the basic neural network codec. Among them, the preset loss function includes the bit rate and the reconstructed image and the distortion loss between the training image x.

[0074] In a possible embodiment, the preset loss function is:

[0075] L=R+λD

[0076] Among them, L represents loss, R represents bit rate, D represents distortion, and λ represents Lagrange multiplier.

[0077] The parameters of the encoding side of the trained basic neural codec, i.e., the parameters of the analysis encoder, are kept unchanged during subsequent processing.

[0078] Figure 3 The present invention is a schematic diagram showing a process of lightweight design of a synthesis decoder according to an exemplary embodiment.

[0079] like Figure 3 As shown, in a possible embodiment, S12 may include S31 to S33.

[0080] S31, fix the parameters of the analysis encoder, super-prior encoder, super-prior decoder and context model, and retain the window-based multi-head attention in the Swin Transformer block in the synthesis decoder.

[0081] S32, remove the shift window configuration in the synthesis decoder.

[0082] As an example, the Swin Transformer block in the synthetic decoder can expand the receptive field of the entire network, which can use the pixel correlation in the entire image to generate a compact latent representation. In fact, there are local priors in the encoding task, so only the synthetic decoder g is retained.s The window-based multi-head attention in the Swin Transformer block of

[15] removes the shifted window configuration and restricts information collection to a fixed window size.

[0083] S33, designing the barrel channel structure in the synthesis decoder as a reverse pyramid channel structure, and determining a lightweight synthesis decoder.

[0084] As an example, design a synthetic decoder g s The reverse pyramid channel structure, where the potential representation The number of channels gradually decreases to a preset channel number threshold, the width gradually increases to a preset width threshold, and the height gradually increases to a preset height threshold to prevent the computational complexity of subsequent network layers from increasing significantly.

[0085] Among them, in the synthesis decoder g s It includes four adjustments of the number of channels, width, and height. Each adjustment process includes an upsampling process, and the width and height of the potential representation are doubled in each adjustment process. In the first adjustment process, the number of channels is reduced from 256 to 128; in the second adjustment process, the number of channels is reduced from 256 to 96; in the third adjustment process, the number of channels is reduced from 256 to 64; in the fourth adjustment process, the number of channels is reduced from 64 to 3, and the output of the fourth channel is 3, ensuring the output of the RGB three channels of the image.

[0086] Through the above steps S31 to S33, the synthesized decoder g s Modified to a lightweight network structure, using a simple lightweight synthetic decoder Replace the original synthesis decoder g s .

[0087] In a possible embodiment, S12 may further include S33.

[0088] The lightweight synthesis decoder is optimized and trained by using a preset first loss function to determine the lightweight synthesis decoder.

[0089] The preset first loss function is a distortion loss function. The distortion loss between the training image x and the lightweight synthetic decoder Optimize training and quickly fine-tune to obtain a lightweight synthetic decoder

[0090] In a possible embodiment, S13 may include S41 to S43.

[0091] S41, fix the parameters of the analysis encoder and lightweight super-prior decoder, remove the shift window configuration of the super-prior decoder and the context model.

[0092] S42, merging the network for obtaining the mean of Gaussian distribution parameters and the network for obtaining the variance in the super a priori decoder to determine a lightweight super a priori decoder.

[0093] Among them, the network for obtaining the mean of Gaussian distribution parameters and the network for obtaining the variance in the super prior decoder are merged into one network.

[0094] S43, reducing the number of channels of the self-attention module in the context model to a first preset threshold, reducing the number of layers of the residual network in the context model to a second preset threshold, and determining a lightweight context model.

[0095] Among them, the number of channels of the self-attention module in the context model is reduced from 128 to 64, and the residual network is reduced from three layers of convolution to one layer of convolution.

[0096] As an example, in the lightweight design of the super-prior path, the shift window configuration of the super-prior decoder and the context model is removed, the network for obtaining the mean of the Gaussian distribution parameters and the network for obtaining the variance in the super-prior decoder are merged, and the number of channels of the self-attention module in the context network and the number of layers of the residual network are further reduced to determine a lightweight super-prior decoder and a lightweight context model.

[0097] Through the above steps S41 to S43, the super-a priori decoder h s and context model f c Modified to a lightweight network structure, using a simple lightweight super-prior decoder Replace the original super prior decoder h s , using a lightweight context model Replace the original context model f c .

[0098] In a possible embodiment, S13 may further include S44.

[0099] S44, using a preset second loss function to jointly train the super-prior encoder, the lightweight super-prior decoder and the lightweight context model to determine the lightweight super-prior decoder, the lightweight context model and the jointly trained super-prior encoder.

[0100] The preset second loss function includes a distortion part and a bit rate part. The lightweight super-prior decoder, the lightweight context model and the super-prior encoder are jointly trained based on the distortion loss and bit rate between the training image x, and the adaptive training of the lightweight super-prior decoder and the super-prior encoder is realized through fast fine-tuning to obtain the lightweight super-prior encoder, the lightweight context model and the jointly trained super-prior encoder.

[0101] In this stage, the super prior encoder h is not fixed a The parameters of the super prior encoder h a With lightweight super-prior decoder Joint training is performed to ensure a lightweight super-prior decoder Able to adapt to the super prior encoder h a Obtained hyper-prior.

[0102] The present invention adopts a phased training strategy of gradually replacing complex decoders with simple modules. It first replaces and trains the synthetic decoder, and then replaces and trains the super-prior decoder and context model included in the decoder on the super-prior path. At the same time, efficient design is performed for the synthetic decoder, the super-prior decoder, and the context model to optimize the balance between computational complexity and compression performance, and finally an asymmetric lightweight image compression model is obtained.

[0103] The training and inference method of the asymmetric lightweight image compression model disclosed in the present invention is the same as the training method of the baseline deep learning image encoder described above, and will not be repeated here.

[0104] In a possible embodiment, the image compression model based on the asymmetric architecture provided by the present disclosure and the image compression results of a traditional codec are used for evaluation.

[0105] Kodak, Tecnick, and CLIC professional validation datasets are used as test image datasets, and the bitrate-PSNR index curve is used as the evaluation index, and the BD-rate savings are calculated accordingly. In the bitrate-PSNR index curve, the higher the PSNR index, the better, that is, the curve at the top can achieve better reconstruction quality at the same bitrate and have better compression performance. The BD-rate savings can quantitatively indicate the percentage of bitrate savings at the same reconstruction quality.

[0106] Figure 4 The figure is a schematic diagram showing a comparison of codec-PSNR indicator curves of a plurality of codecs according to an exemplary embodiment.

[0107] like Figure 4As shown, the bit rate-PSNR performance of the image compression model based on the asymmetric architecture provided by the present invention and the traditional codec are compared, wherein the traditional codecs include HM and VTM, and the learning-based image compression models include Balle18, Minnen18, Cheng20, Xie21, Qian22, and TinyLIC. The horizontal axis bpp (bits-per-pixel) represents the number of bits required for each pixel, and the vertical axis represents the peak signal-to-noise ratio of the PSNR reconstructed image to the original image.

[0108] like Figure 4 As shown, the image compression model based on the asymmetric architecture provided by the present invention maintains compression performance similar to that of the traditional video encoder VTM on the three data sets of Kodak, Tecnick, and CLIC professional validation, and, on the anchor point BPG (HEVC intra), the image compression model based on the asymmetric architecture provided by the present invention achieves 18.68% BD-rate savings.

[0109] Figure 5 It is a schematic diagram showing a comparison of compression efficiency-decoding complexity-decoding parameter amount of various codecs according to an exemplary embodiment.

[0110] like Figure 5 As shown, the present disclosure also uses BD-rate-decoding calculation amount-decoding parameter amount to measure the balance between compression performance and decoding complexity, wherein the horizontal axis represents the decoding end MACs (Multiply-Accumulate Operations) multiplication-accumulation operations represent the decoding end calculation complexity, and the vertical axis represents the compression performance BD-rate (%). The closer the colored circle is to the upper left corner, the higher its compression performance and the lower its decoding complexity. The radius of the colored circle represents the decoding parameter amount. The longer the radius, the more decoding parameters.

[0111] like Figure 5 As shown, compared with other traditional codecs, the image compression model based on asymmetric architecture provided by the present invention can achieve the lowest decoding complexity while maintaining high compression performance. Therefore, the image compression model based on asymmetric architecture determined by the method for determining an image compression model based on asymmetric architecture provided by the present invention can achieve the best balance between compression performance and computational complexity.

[0112] Figure 6 It is a block diagram of a system for determining an image compression model based on an asymmetric architecture according to an exemplary embodiment.

[0113] Based on the same concept, the present disclosure also provides a system 100 for determining an image compression model based on an asymmetric architecture, such as Figure 6 As shown, it includes: an acquisition module 110, a first lightweight design module 120, a second lightweight design module 130, and a combination module 140.

[0114] An acquisition module 110 is used to acquire a trained basic neural network codec, wherein the basic neural network codec includes an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model;

[0115] A first lightweight design module 120 is used to fix the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, perform lightweight design processing on the synthesis decoder, and determine a lightweight synthesis decoder;

[0116] A second lightweight design module 130 is used to fix the parameters of the analysis encoder and the lightweight super a priori decoder, perform lightweight design on the super a priori decoder and the context model in conjunction with the super a priori encoder, and determine a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder;

[0117] The combination module 140 is used to determine an asymmetric lightweight image compression model based on the analysis encoder, the lightweight synthesis decoder, the jointly trained super a priori encoder, the lightweight super a priori decoder and the lightweight context model.

[0118] Through the above technical scheme, by lightweight design processing of the synthetic decoder at the decoding end and the hyper-prior decoder and context model of the decoding module on the hyper-prior path, the asymmetric architecture design of the image compression model is realized, the decoder network structure is simplified, the compression efficiency is improved, and the balance between decoding complexity and compression performance is optimized. While maintaining a high coding efficiency, the computational complexity of the decoding end can be significantly reduced, the decoding efficiency can be improved, and it can adapt to diverse low-end device deployment needs.

[0119] Regarding the embodiment of the above system, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0120] Based on the same concept as above, in another embodiment of the present disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and a method for determining an image compression model based on an asymmetric architecture is used to execute when the processor executes the program.

[0121] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0122] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0123] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0124] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0125] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for determining an image compression model based on an asymmetric architecture in any of the above embodiments are implemented.

[0126] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0130] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.

[0131] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for determining an image compression model based on an asymmetric architecture, characterized in that: include: Obtain a trained basic neural network codec, wherein the basic neural network codec includes an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model; Fixing the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, performing lightweight design processing on the synthesis decoder, and determining a lightweight synthesis decoder; Fixing the parameters of the analysis encoder and the lightweight synthesis decoder, jointly designing the super a priori decoder and the context model with the super a priori encoder, and determining a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder; An asymmetric lightweight image compression model is determined according to the analysis encoder, the lightweight synthesis decoder, the jointly trained super a priori encoder, the lightweight super a priori decoder and the lightweight context model.

2. The method according to claim 1, characterized in that The fixing of the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, performing lightweight design processing on the synthesis decoder, and determining a lightweight synthesis decoder includes: Fixing the parameters of the analysis encoder, the super-a priori encoder, the super-a priori decoder, and the context model, retaining the window-based multi-head attention in the Swin Transformer block in the synthesis decoder; removing a shift window configuration in the synthesis decoder; The barrel channel structure in the synthesis decoder is designed as a reverse pyramid channel structure to determine the lightweight synthesis decoder.

3. The method according to claim 2, characterized in that The method further comprises: The lightweight synthesis decoder is optimized and trained by using a preset first loss function to determine the lightweight synthesis decoder, wherein the preset first loss function is a distortion loss function.

4. The method according to claim 1, characterized in that The fixing of the analysis encoder and the parameters of the lightweight super a priori decoder, the lightweight design of the super a priori decoder and the context model by combining the super a priori encoder, and the determination of the lightweight super a priori decoder, the lightweight context model and the jointly trained super a priori encoder include: Fixing the parameters of the analysis encoder and the lightweight super-a priori decoder, and removing the shift window configuration of the super-a priori decoder and the context model; The network for obtaining the mean of Gaussian distribution parameters and the network for obtaining the variance in the super a priori decoder are combined to determine the lightweight super a priori decoder; The number of channels of the self-attention module in the context model is reduced to a first preset threshold, the number of layers of the residual network in the context model is reduced to a second preset threshold, and the lightweight context model is determined.

5. The method according to claim 4, characterized in that The method of fixing the parameters of the analysis encoder and the lightweight super a priori decoder, performing lightweight design on the super a priori decoder and the context model in conjunction with the super a priori encoder, and determining the lightweight super a priori decoder, the lightweight context model, and the jointly trained super a priori encoder further includes: The super a priori encoder, the lightweight super a priori decoder and the lightweight context model are jointly trained using a preset second loss function to determine the lightweight super a priori decoder, the lightweight context model and the jointly trained super a priori encoder, and the preset second loss function includes a distortion part and a bit rate part.

6. The method according to claim 1, characterized in that The step of obtaining a trained basic neural network codec includes: Obtaining a baseline deep learning image codec, the baseline deep learning image codec comprising an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model; Inputting a training image into the analysis encoder and determining a latent representation of the training image; quantizing the latent representation of the training image to determine the quantized latent representation of the training image; Inputting the quantized latent representation of the training image into the synthesis decoder to determine a reconstructed image; The basic neural network codec is optimized by using a preset loss function to determine the trained basic neural network codec.

7. The method according to claim 6, characterized in that The preset loss function is: L=R+λD Among them, L represents loss, R represents bit rate, D represents distortion, and λ represents Lagrange multiplier.

8. A system for determining an image compression model based on an asymmetric architecture, characterized in that: include: An acquisition module, used to acquire a trained basic neural network codec, wherein the basic neural network codec includes an analysis encoder, a synthesis decoder, a super a priori encoder, a super a priori decoder, and a context model; A first lightweight design module is used to fix the parameters of the analysis encoder, the super a priori encoder, the super a priori decoder and the context model, perform lightweight design processing on the synthesis decoder, and determine a lightweight synthesis decoder; A second lightweight design module is used to fix the parameters of the analysis encoder and the lightweight super a priori decoder, perform lightweight design on the super a priori decoder and the context model in conjunction with the super a priori encoder, and determine a lightweight super a priori decoder, a lightweight context model, and a jointly trained super a priori encoder; A combination module is used to determine an asymmetric lightweight image compression model based on the analysis encoder, the lightweight synthesis decoder, the jointly trained super-prior encoder, the lightweight super-prior decoder and the lightweight context model.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images