A learning-based image compression method and system based on stage-wise modular distillation

Abstract

The application provides a learning-based image compression method and system based on stage module distillation, the method comprising: dividing the encoding network and the decoding network of a teacher model into n stages according to feature resolution change nodes, n being an integer greater than 1; based on the divided teacher model, generating a student model maintaining a stage corresponding relationship with the teacher model and performing pruning; freezing all parameters of the teacher model, distilling and training only the i-th stage of the student model, i being an integer greater than or equal to 1 and less than or equal to n, so that the distribution of the latent variable output by the i-th stage approximates the latent variable distribution of the corresponding stage of the teacher model; after completion, freezing the stage and moving to the next stage, until all stages are trained; using the trained student model to encode a to-be-compressed image to obtain a code stream and reconstruct an image. The application can realize synchronous reduction of parameter quantity, FLOPs (floating point operation times) and inference delay, and meet real-time application requirements.

Description

Technical Field

[0001] This application relates to the field of image data compression technology, and more specifically, to a learning-based image compression method and system based on staged module distillation. Background Technology

[0002] With the widespread success of neural networks in visual tasks, Learned Image Compression (LIC) has gradually replaced traditional codecs such as JPEG and HEVC / VVC, becoming a focus of attention in academia and industry. A typical LIC framework employs an end-to-end learnable "transform-quantization-entropy coding" process: the encoder maps the input image to a latent variable space, the quantizer discretizes the latent variables, and then performs bitrate estimation using an entropy model; the decoder then performs an inverse transform to complete the reconstruction. Compared to hand-designed Discrete Cosine Transform (DCT) or Variable Length Coding (VLC), LIC achieves higher peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) at the same bitrate by jointly optimizing the rate-distortion (R–D) objective.

[0003] However, existing LIC models also face three bottlenecks:

[0004] The models are large and computationally intensive: early end-to-end proposals based on CNNs (such as Ballé et al. 2018) already contained tens of millions of parameters; in order to improve the ability to model long-range dependencies, a large number of works in recent years have introduced self-attention / Transformer structures (such as TCM, FTIC, WeConvene), further expanding the model size to the level of nearly 100M parameters, with floating-point operations (FLOPs) often reaching hundreds of terabytes, and the decoding latency is difficult to meet the real-time requirements in high-resolution scenarios.

[0005] End-to-end distillation / pruning is unstable: Most existing pruning methods for compressed networks directly apply sparsity regularization to the convolutional channels or attention heads, and then distill the entire network into a lightweight student network. Due to the lack of fine alignment coupled with the encoding process, latent variable distribution drift often occurs, leading to a sharp increase in bit rate or image reconstruction distortion; overall distillation also brings problems of training instability and slow convergence.

[0006] Rate-distortion and complexity are difficult to balance: In pursuit of lower bitrates, researchers tend to improve RD performance by stacking deeper networks and increasing the complexity of context models, but this often comes at the cost of significantly sacrificing parameters and latency. Industrial deployment scenarios (mobile devices, edge devices) urgently need a lightweight solution that can maintain or even improve RD performance while significantly reducing the number of parameters, FLOPs, and even memory usage.

[0007] In-depth analysis reveals that the root cause of the aforementioned bottlenecks lies in insufficient exploration of the distillation framework. This is primarily due to significant structural differences between the student and teacher models, leading to signal mismatch during distillation. Furthermore, the hierarchical strategy does not strictly correspond to the encoding stage, making it difficult to ensure step-by-step alignment of latent variables. Other issues include the failure to fully utilize the gradients of the joint objectives of bit rate and distortion, and the lack of an end-to-end optimization loop for the compression task during the distillation process.

[0008] A search revealed a Chinese patent application (application number 202311620315.1) that discloses a self-distillation model compression method. This method constructs a teacher model requiring compression, dividing its basic structure into several residual blocks, and then constructs a student model based on these residual blocks. Knowledge distillation is used to enhance the resulting knowledge from each model, and attention mapping is used to enhance the process knowledge. Based on the enhanced results of effective and process knowledge, loss functions are calculated for both the teacher and student models. The student model whose classification performance is closest to the teacher model is extracted according to the loss function and used as the compressed model. This method constructs teacher-student models based on ResNet and optimizes them through knowledge distillation, but it suffers from high computational complexity. Knowledge enhancement and loss calculation involve a large amount of computation, are time-consuming, and hardware-dependent. Summary of the Invention

[0009] In view of the deficiencies in the prior art, the purpose of this application is to provide a learning-based image compression method and system based on stage module distillation.

[0010] A first aspect of this application provides a learning-based image compression method based on staged module distillation, comprising:

[0011] The encoding and decoding networks of the teacher model are divided into n stages according to the nodes where the feature resolution changes, where n is an integer greater than 1;

[0012] Based on the divided teacher model, a student model that maintains a stage correspondence with the teacher model is generated and pruned.

[0013] Freeze all parameters of the teacher model and all parameters of the student model except for the i-th stage. Distill and train only the i-th stage, 1≤i≤n, so that the distribution of the latent variables of the output of the i-th stage of the student model approximates the distribution of the latent variables of the corresponding stage of the teacher model. After completion, freeze the i-th stage of the student model and move to the i+1 stage until all stages are trained.

[0014] The trained student model is used to encode the image to be compressed, obtain the bitstream, and reconstruct the image.

[0015] Optionally, the step of dividing the encoding and decoding networks of the teacher model into n stages according to the feature resolution variation nodes, where n is an integer greater than 1, includes:

[0016] Traverse all layers of the encoding network and record all downsampling nodes n1 that cause a decrease in feature resolution;

[0017] Traverse all layers of the decoding network and record all upsampling nodes n2 that cause an increase in feature resolution;

[0018] At the downsampling node and the upsampling node, the teacher model is divided into n stages, where n = n1 + n2.

[0019] Optionally, the step of generating and pruning the student model based on the partitioned teacher model, which maintains a stage correspondence with the teacher model, includes:

[0020] Corresponding to the stages of the teacher model, the student model is divided into corresponding stages;

[0021] Pruning is performed at each stage by deleting channels, reducing the number of nonlinear transform blocks, and adjusting the feedforward spread factor.

[0022] Optionally, the step of making the distribution of the latent variables of the output in the i-th stage approximate the distribution of the latent variables of the teacher model in the corresponding stage includes:

[0023] We use RD Loss based on KL divergence as the stage distillation loss to constrain the distribution of student latent variables to converge to the distribution of teacher variables.

[0024] Optionally, it also includes: during the distillation training process at each stage, introducing an implicit supervision signal based on the joint goal of rate-distortion to adaptively adjust the latent variable energy compression and rate regularization.

[0025] Optionally, the step of encoding the image to be compressed using the trained student model to obtain a bitstream and reconstruct the image includes:

[0026] Obtain the image to be compressed as the input image;

[0027] The encoding network performs feature compression and transformation on the input image, extracts key information through the pruned lightweight structure, and outputs the quantized compressed representation.

[0028] The decoding network receives the compressed representation output by the encoding network, performs feature recovery and reconstruction through the corresponding lightweight inverse transform structure, and finally outputs the reconstructed image.

[0029] A second aspect of this application provides a learning-based image compression system based on staged module distillation, comprising:

[0030] Stage division module: The encoding and decoding networks of the teacher model are divided into n stages according to the feature resolution change nodes, where n is an integer greater than 1;

[0031] Pruning construction module: Based on the divided teacher model, generate the student model that maintains the stage correspondence with the teacher model and prune it;

[0032] Stage Distillation Module: Freezes all parameters of the teacher model and all parameters of the student model except for the i-th stage, and distills only the i-th stage for training, 1≤i≤n, so that the distribution of the latent variables of the output of the student model in the i-th stage approximates the distribution of the latent variables of the corresponding stage of the teacher model; Application Module: Encodes the image to be compressed using the trained student model, obtains the bitstream and reconstructs the image.

[0033] Optionally, it also includes an end-to-end supervision module that introduces an implicit supervision signal based on the joint goal of rate-distortion during each stage of distillation training to adaptively adjust the latent variable energy compression and rate regularization.

[0034] A third aspect of this application provides a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to perform any of the methods described in the previous application, or to run any of the systems described in the previous application.

[0035] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform any of the methods described in the previous application, or to run any of the systems described in the previous application.

[0036] The learning-based image compression method based on staged module distillation provided in this application precisely divides the distillation process into stages at feature resolution change nodes and stabilizes the process by progressively training and freezing each stage. This ultimately achieves a simultaneous reduction in parameter count, FLOPs (floating-point operations), and inference latency, meeting the requirements of real-time applications. Other technical effects brought about by the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0037] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0038] Figure 1 This is a flowchart illustrating a learning-based image compression method based on stage module distillation according to an exemplary embodiment;

[0039] Figure 2 This is a schematic diagram illustrating the overall performance of a learning-based image compression method based on staged module distillation according to an exemplary embodiment.

[0040] Figure 3 This is an exemplary embodiment illustrating a teacher-student model phase division and distillation execution framework;

[0041] Figure 4 This document presents a performance comparison between the learning-based image compression method based on staged module distillation of this application and existing state-of-the-art methods, according to an exemplary embodiment.

[0042] Figure 5 This is a demonstration of the distillation effect of a learning-based image compression method based on stage module distillation according to an exemplary embodiment;

[0043] Figure 6 This is a structural diagram of a learning-based image compression system based on stage module distillation, according to an exemplary embodiment. Detailed Implementation

[0044] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0045] In current lightweight solutions, the student and teacher models have significant structural differences, leading to signal mismatch during distillation. Furthermore, the hierarchical strategy and the encoding stage do not have a strict correspondence, making it difficult to ensure step-by-step alignment of latent variables. To address these issues, this application provides a learning-based image compression method based on stage-module distillation to resolve these problems.

[0046] Reference Figure 1 and Figure 2 As shown, a learning-based image compression method based on staged module distillation includes the following steps:

[0047] S100 divides the encoding and decoding networks of the teacher model into n stages according to the feature resolution change nodes, where n is an integer greater than 1;

[0048] S200, based on the teacher model after S100 partitioning, generates a student model that maintains the stage correspondence with the teacher model and performs pruning;

[0049] S300: Freeze all parameters of the teacher model, freeze all parameters of the student model except for the i-th stage, and distill only the i-th stage of the student model for training, where i is an integer greater than or equal to 1 and less than or equal to n, so that the distribution of the latent variables of the output of the i-th stage approximates the distribution of the latent variables of the corresponding stage of the teacher model; after completion, freeze the stage and move to the next stage until all stages are trained.

[0050] S400 uses the student model trained in S300 to encode the image to be compressed, obtain the bitstream, and reconstruct the image.

[0051] The embodiments described above in this application stabilize the distillation process by precisely dividing the stages at the feature resolution change nodes and stabilizing the process by training and freezing the model in stages. Ultimately, the obtained student model achieves a synchronous decrease in the number of parameters, FLOPs (floating-point operations), and inference latency, which can meet the requirements of real-time applications.

[0052] To simplify the knowledge transfer and learning process from the teacher model to the student model, the complex teacher model can be decomposed into multi-stage tasks. In some specific embodiments of this application, step S100, which divides the encoding and decoding networks of the teacher model into n stages according to the feature resolution variation nodes, where n is an integer greater than 1, can adopt the following steps:

[0053] S101, traverse all layers of the encoding network and record all downsampling nodes n1 that cause a decrease in feature resolution;

[0054] S102, traverse all layers of the decoding network and record all upsampling nodes n2 that cause the feature resolution to increase;

[0055] S103, at the downsampling node and the upsampling node, divide the teacher model into n stages, n = n1 + n2.

[0056] In the above embodiments of this application, each stage of the student model is subjected to knowledge imitation with the corresponding stage of the teacher model, thereby reducing the adaptation difficulty in the distillation process and effectively improving the overall performance of the model.

[0057] To ensure isomorphism between the student and teacher models and to make the model more lightweight, in one embodiment of this application, step S200, based on the partitioned teacher model, generates a student model that maintains a stage correspondence with the teacher model and performs pruning, can adopt the following steps:

[0058] S201 corresponds to the stages of the teacher model, and the student model is divided into corresponding stages.

[0059] For example, such as Figure 3 As shown, the teacher encoder and decoder are split at three upsampling / downsampling nodes, resulting in six stages T1–T6. The student model S1–S6 is also divided into six stages by preserving these node positions.

[0060] S202 prunes each stage by deleting channels, reducing the number of nonlinear transform blocks, and adjusting the feedforward spread factor.

[0061] Specifically, pruning is an important model compression and optimization technique in deep learning. Its core purpose is to reduce the number of parameters, computational load, and storage requirements of the model by removing "redundant" parameters, neurons, or structural components, thereby improving operational efficiency while ensuring that the model performance remains basically unchanged.

[0062] "Channel" refers to the dimension (or "depth") of the feature map in a neural network. Deleting channels means removing redundant dimensions (or the output dimension of the entire convolutional kernel / linear layer) from the feature map, thereby reducing the computational cost and parameter storage of the model during feature extraction.

[0063] "Nonlinear transform blocks" refer to repetitive structural units in a model that consist of nonlinear activation functions (such as ReLU and GELU) and linear transformations (such as convolution and fully connected layers). Examples include "attention blocks + feedforward blocks" in Transformers and "residual blocks" in ResNet. Reducing the number of blocks means decreasing the number of these repetitive units, thereby reducing the overall depth and computational cost of the model.

[0064] The "feedforward expansion factor" refers to the ratio of the intermediate dimension to the input dimension of a feedforward neural network (FFN) in models such as Transformer. Reducing the expansion factor means lowering this ratio, reducing the intermediate dimension of the feedforward layer, thereby reducing the computational cost and number of parameters in the feedforward layer.

[0065] For example, the student model in the above example is pruned, reducing the number of channels in the first stage of the encoder and decoder by 33% compared to the teacher model, and reducing the number of nonlinear transformation blocks in the second and third stages by 50% each. By reducing the number of channels and blocks, structural consistency is ensured and the capability gap is narrowed.

[0066] It should be noted that the reduction ratio of the number of channels and the adjustment ratio of the feedforward expansion factor are both empirical hyperparameters. In practical applications, these hyperparameters can be set to different values ​​at different stages of the model, and their optimal values ​​also differ for teacher models with different structures. Therefore, they need to be adjusted according to specific scenarios.

[0067] The embodiments described above in this application reduce the difficulty of single-step learning by breaking down knowledge levels into stages, allowing the student model to efficiently absorb key knowledge from the stages of the classroom model, thus avoiding ineffective distillation due to concentrated knowledge complexity. Through pruning, the structural skeletons of the student and teacher models are aligned, enabling the student model to possess the basic conditions to imitate the teacher model, avoiding a "lack of imitation ability" caused by excessive structural differences.

[0068] In some specific embodiments of this application, step S300 causes the latent variable distribution of the output of the i-th stage to approach the latent variable distribution of the corresponding stage of the teacher model, and its distillation loss consists of two parts: the latent variable distribution constrained by KL divergence and the reconstruction error constrained by MSE.

[0069] The embodiments described above in this application achieve "dual capture" of the teacher model's knowledge by simultaneously constraining the latent variable distribution (KL divergence) and the reconstruction error (MSE). This allows the student model to learn the probability distribution pattern (deep-seated patterns) of the intermediate features of the teacher model while ensuring the numerical consistency of the output results (surface manifestation), thus avoiding incomplete knowledge learning caused by a single constraint.

[0070] The embodiments described above in this application design a dedicated distillation loss for the i-th stage, which forces the student model to form an accurate match with the teacher model in the corresponding stage. This "point-to-point" constraint method avoids distortion or deviation in knowledge transfer across stages, making the imitation effect at each stage more reliable.

[0071] In the above embodiments of this application, after compression operations such as pruning, the number of parameters in the student model is reduced. Through this dual-constraint distillation method, the key features and output capabilities of the teacher model can be preserved to the maximum extent, maintaining or even improving model performance while making the model lightweight.

[0072] To reduce the complexity of the student model while maintaining the latent variable distribution and RD performance (Rate-Distortion performance, a metric for compression effectiveness), some specific implementations of this application introduce an end-to-end supervision approach. During each stage of distillation training, an implicit supervision signal based on the joint goal of rate-distortion is introduced to adaptively adjust latent variable energy compression and rate regularization.

[0073] Specifically, during the phase training process, global simultaneous computation

[0074] L RD =R+λD

[0075] LRD (Rate-Distortion Loss) refers to the bitrate estimated using an entropy model, D representing pixel MSE or MS-SSIM distortion, and λ set according to the target compression rate. This strategy enables the student model to adaptively perform energy compression and bitrate regularization without requiring explicit channel alignment.

[0076] The embodiments described above in this application achieve multiple technical gains by introducing end-to-end supervision and a joint bitrate-distortion objective: On the one hand, bitrate is estimated using an entropy model, and distortion is measured using pixel MSE or MS-SSIM, constructing a Rate-Distortion Loss (LRD). This allows the student model to adaptively balance energy compression and bitrate regularization during stage distillation training without requiring forced channel alignment, thus reducing the cost of model structure adaptation. On the other hand, based on the implicit supervision signal of this joint objective, the student model is driven to accurately maintain the latent variable distribution characteristics while reducing complexity, ensuring Rate-Distortion (RD) performance. This achieves a better synergy between model lightweighting and compression effect, providing a technical path that balances efficiency and accuracy for model deployment in resource-constrained scenarios.

[0077] In some specific embodiments of this application, S400, the student model trained in S300 is used to encode the image to be compressed, obtain the bitstream, and reconstruct the image. This can be achieved by the following steps:

[0078] S401, Obtain the image to be compressed as the input image;

[0079] S402, the coding network performs feature compression and transformation on the input image, extracts key information through the pruned lightweight structure, and outputs the quantized compressed representation;

[0080] S403, the decoding network receives the compressed representation output by the encoding network, performs feature recovery and reconstruction through the corresponding lightweight inverse transform structure, and finally outputs the reconstructed image.

[0081] The embodiments described above achieve efficient encoding and decoding through lightweight pruning, while retaining core compression capabilities by leveraging structural consistency with the teacher model. Ultimately, while reducing resource consumption and improving processing efficiency, they can still output high-quality reconstructed images, demonstrating strong practicality and broad deployment potential.

[0082] Figure 2 The following is a schematic diagram illustrating the overall performance of a learning-based image compression method based on staged module distillation according to an exemplary embodiment. (a) The figure is a radar chart. From the outermost circle to the center, the smaller the value, the better. As can be seen from the figure, the method of this application is significantly better than other methods in terms of parameter quantity, floating-point operations per second, decoding latency, throughput, BD-rate, etc. (b) The figure is a scatter plot. The horizontal axis (FLOPS(G)) represents the floating-point operations of model inference (unit: 10^9 times). The larger the value, the higher the computational complexity. The vertical axis (BD-Rate over VTM-21.0(%)) represents the comparison of BD-rate based on VTM 21.0 (Video Coding Standard Reference Model). The smaller the value, the higher the compression rate (better performance) relative to VTM-21.0.

[0083] Figure (b) is a scatter plot (compression ratio vs. computational cost).

[0084] The horizontal axis (FLOPS(G)): the number of floating-point operations in model inference (unit: 10^9 times). The larger the value, the higher the computational complexity (the more computing power is required).

[0085] The vertical axis (BD-Rate over VTM-21.0 (%)): BD-rate comparison based on VTM-21.0 (Video Coding Standard Reference Model). The smaller the value, the higher the compression ratio (better performance) relative to VTM-21.0. Each point represents a model, and color / shape distinguishes the model type (e.g., S2CFormer (Teacher) is the teacher model, KDIC wKD and KDIC w / o KD are its derived student models).

[0086] The red arrows show the advantages of "KDIC w KD" compared to other models, indicating that it achieves a breakthrough of "lighter weight and higher compression quality" while "reducing computational load (smaller FLOPS) and lower BD-rate (higher compression ratio)".

[0087] Figure 4 The performance comparison of the learning-based image compression method based on stage module distillation of this application with existing advanced methods, as shown in an exemplary embodiment, reveals that the novel model (such as KDIC) has higher compression efficiency: better reconstruction quality at the same bitrate, or lower bitrate at the same quality, verifying the value of lightweight techniques such as pruning and distillation.

[0088] Figure 5 The distillation effect of the learning-based image compression method based on stage module distillation, as illustrated in an exemplary embodiment, shows that the student model (KDIC) outperforms / matches the teacher model (SSC-Hybrid): This demonstrates that through model compression techniques (such as pruning and knowledge distillation), the student model can still inherit or even surpass the compression performance of the teacher model while "reducing complexity".

[0089] Based on the same technical concept, some specific embodiments of this application provide a learning-based image compression system 100 based on stage module distillation, comprising:

[0090] Stage division module 110: Divide the encoding network and decoding network of the teacher model into n stages according to the feature resolution change nodes, where n is an integer greater than 1;

[0091] Pruning Module 120: Based on the partitioned teacher model, generate a student model that maintains the stage correspondence with the teacher model and perform pruning;

[0092] Stage Distillation Module 130: Freeze all parameters of the teacher model, freeze all parameters of the student model except for the i-th stage, and distill only the i-th stage of the student model for training, where i is an integer greater than or equal to 1 and less than or equal to n, so that the distribution of the latent variables of the output of the i-th stage approximates the distribution of the latent variables of the corresponding stage of the teacher model; after completion, freeze the stage and move to the next stage until all stages are trained.

[0093] Application Module 140: Encodes the image to be compressed using the trained student model, obtains the bitstream, and reconstructs the image.

[0094] The specific implementation techniques of each module / unit in the above examples of this application can be referred to the steps of the learning-based image compression method based on stage module distillation in the above embodiments, and will not be repeated here.

[0095] Based on the same technical concept, some specific embodiments of this application provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any of the learning-based image compression methods based on stage module distillation, or to run any of the learning-based image compression systems based on stage module distillation.

[0096] Based on the same technical concept, some specific embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the learning-based image compression methods based on stage module distillation, or to run any of the learning-based image compression systems based on stage module distillation.

[0097] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0098] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0099] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0100] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0101] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A learning-based image compression method based on staged module distillation, characterized in that, include: The encoding and decoding networks of the teacher model are divided into n stages according to the nodes where the feature resolution changes, where n is an integer greater than 1; Based on the divided teacher model, a student model that maintains a stage correspondence with the teacher model is generated and pruned. Freeze all parameters of the teacher model and all parameters of the student model except for the i-th stage. Distill and train only the i-th stage, 1≤i≤n, so that the distribution of the latent variables of the output of the i-th stage of the student model approximates the distribution of the latent variables of the corresponding stage of the teacher model. After completion, freeze the i-th stage of the student model and move to the i+1 stage until all stages are trained. The trained student model is used to encode the image to be compressed, obtain the bitstream, and reconstruct the image. The method of dividing the encoding and decoding networks of the teacher model into n stages according to the feature resolution change nodes, where n is an integer greater than 1, includes: Traverse all layers of the encoding network and record all downsampling nodes n1 that cause a decrease in feature resolution; Traverse all layers of the decoding network and record all upsampling nodes n2 that cause an increase in feature resolution; At the downsampling node and the upsampling node, the teacher model is divided into n stages, where n = n1 + n2; The method of making the distribution of the latent variables of the output in stage i approximate the distribution of the latent variables of the teacher model in the corresponding stage includes: We use the RD Loss based on KL divergence as the stage distillation loss to constrain the distribution of student latent variables to converge to the distribution of teacher variables. It also includes: introducing an implicit supervision signal based on the joint goal of rate-distortion during the distillation training process at each stage, so as to adaptively adjust the latent variable energy compression and rate regularization.

2. The learning-based image compression method based on stage module distillation according to claim 1, characterized in that, The process of generating and pruning the student model based on the partitioned teacher model, while maintaining a stage correspondence with the teacher model, includes: Corresponding to the stages of the teacher model, the student model is divided into corresponding stages; Pruning is performed at each stage by deleting channels, reducing the number of nonlinear transform blocks, and adjusting the feedforward spread factor.

3. The learning-based image compression method based on stage module distillation according to claim 1, characterized in that, The process of encoding the image to be compressed using the trained student model to obtain a bitstream and reconstruct the image includes: Obtain the image to be compressed as the input image; The encoding network performs feature compression and transformation on the input image, extracts key information through the pruned lightweight structure, and outputs the quantized compressed representation. The decoding network receives the compressed representation output by the encoding network, performs feature recovery and reconstruction through the corresponding lightweight inverse transform structure, and finally outputs the reconstructed image.

4. A learning-based image compression system based on staged module distillation, used to implement the method of claim 1, characterized in that, include: Stage division module: The encoding and decoding networks of the teacher model are divided into n stages according to the feature resolution change nodes, where n is an integer greater than 1; Pruning construction module: Based on the divided teacher model, generate a student model that maintains a stage correspondence with the teacher model and prune it; Stage Distillation Module: Freezes all parameters of the teacher model and all parameters of the student model except for the i-th stage, and distills only the i-th stage for training, 1≤i≤n, so that the distribution of the latent variables of the output of the student model in the i-th stage approximates the distribution of the latent variables of the corresponding stage of the teacher model; Application Module: Encodes the image to be compressed using the trained student model, obtains the bitstream and reconstructs the image.

5. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it performs the method according to any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the method according to any one of claims 1-3.

Images