Space-channel perception based image compression domain adaptive method and system
By introducing a dual-branch spatial-channel adapter and a spatial-channel interactive low-rank adaptive module into the LIC model, the problem of poor generalization of the LIC model in different image content domains is solved, achieving efficient domain adaptation, improving compression performance and reducing computational overhead.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
- Filing Date
- 2026-01-12
- Publication Date
- 2026-05-12
AI Technical Summary
Existing deep learning-based learnable image compression (LIC) models have poor generalization ability when facing different image content domains and cannot flexibly adapt to new scenarios, resulting in a decline in the compression performance of out-of-domain data. Furthermore, existing domain adaptation methods have high computational overhead or complex parameter updates.
We adopt a spatial-channel-aware image compression domain adaptive method. By constructing a dual-branch spatial-channel adapter and a spatial-channel interaction low-rank adaptive module, we integrate them into a deep learning learnable image compression model. We use a small number of samples for fine-tuning to independently capture spatial layout and channel correlation, and jointly model spatial and channel context.
It achieves efficient domain adaptation in specific domains, improves the generalization of the LIC model, matches or surpasses the performance of traditional VVC codecs, and reduces the number of parameters and computational overhead.
Smart Images
Figure CN121486579B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and specifically to an image compression domain adaptive method and system based on spatial-channel awareness. Background Technology
[0002] Image compression is a fundamental technology in computer vision. Traditional compression methods such as JPEG, JPEG2000, BPG, and the latest VVC have accumulated mature technical systems through long-term application, but their fixed transformation and prediction mechanisms have gradually revealed limitations when facing complex and ever-changing image scenes. In recent years, with the development of deep learning, many deep learning-based learnable image compression (LIC) methods have achieved better performance than these traditional image compression methods. However, the performance of existing LIC models is heavily dependent on training data, and they cannot flexibly adapt to new scenes and expand their functionality, resulting in poor generalization in practical applications.
[0003] Specifically, LIC models are typically trained on natural image datasets, but real-world applications often involve data such as comics, screen content, and pixel images. These contents have significant domain differences, leading to a severe decrease in compression performance for out-of-domain data, sometimes even lagging behind traditional codecs. This limitation underscores the necessity of developing efficient domain adaptation techniques to improve the generalization of pre-trained LIC models across different image content domains.
[0004] Specifically, LIC models are typically trained on natural image datasets, but real-world applications often involve data such as comics, screen content, and pixel images. These contents have significant domain differences, leading to a severe decrease in compression performance for out-of-domain data, sometimes even lagging behind traditional codecs. This limitation underscores the necessity of developing efficient domain adaptation techniques to improve the generalization of pre-trained LIC models across different image content domains.
[0005] While the IA (Integrated Image) method can improve the compression performance of individual images, its computational and time overhead makes it difficult to deploy in practical applications. In practical applications, the DA (Data Adaptation) method is a more ideal solution. However, previous DA methods require updating and storing a large number of parameters. State-of-the-art few-sample DA methods only consider the channel dimension; in fact, spatial information is also crucial in domain adaptation. Since distribution offsets typically involve changes in cross-domain layout, texture, and local patterns, the lack of spatial modeling leads to inadequate representation and poor compression performance. To address these issues, we design a domain adaptation scheme based on spatial-channel-aware fine-tuning to further improve the generalization of LIC (Library Adaptation). Summary of the Invention
[0006] To address the aforementioned issues, this invention provides a spatial-channel-aware image compression domain adaptive method and system, aiming to improve the generalization of deep learning-based learnable image compression (LIC) methods.
[0007] According to a first aspect of the present disclosure, a spatial-channel-aware image compression domain adaptive method is provided, the method comprising the following steps:
[0008] Deep learning-based learnable image compression models:
[0009] Construct a dual-branch spatial-channel adapter for the transformation network to independently capture spatial layout and channel correlation;
[0010] Construct a low-rank adaptive module for space-channel interactions in entropy models to jointly model spatial and channel contexts;
[0011] The dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module are integrated into a deep learning learnable image compression model, and fine-tuned using a small number of samples, thus achieving domain adaptation for a specific domain.
[0012] In some embodiments, the dual-branch space-channel adapter specifically includes:
[0013] For input features The output aggregates adaptive features of spatial and channel operations. The spatial operation is depthwise separable convolution, and the channel operation includes: dimensionality reduction through a 1x1 convolution, passing the output through the LeakyReLU activation function, and then passing it through a 1x1 convolution to restore the dimension to be consistent with the dimension of the channel branch input.
[0014] The dual-branch space-channel adapter is initialized using identity mapping, including: the space branch is initialized to pass-through output, while the channel branch output is initialized to zero.
[0015] In some embodiments, the specific expression of the space-channel interaction low-rank adaptive module is:
[0016] ,
[0017] Indicates input features, Let a be the original pre-trained weights. For updating the convolution weights, , For projection along the channel dimension, For spatial operations that can be applied independently in low-rank dimensions, this is used to achieve fine-grained local filtering and enrich spatial information, where... Gaussian initialization is used. Set to zero. Uniform initialization using Xavier is employed.
[0018] In some embodiments, integrating the dual-branch spatial-channel adapter and the spatial-channel interaction low-rank adaptive module into a deep learning learnable image compression model specifically includes:
[0019] Analysis of the variation in serially inserting a dual-branch spatial-channel adapter into a deep learning-based learnable image compression model. With synthetic changes Following the nonlinear transformation in the model, a space-channel interactive low-rank adaptive module is added to the supersynthetic transformation network of the entropy model. In the convolutional layer.
[0020] In some embodiments, fine-tuning is performed using a small number of samples, employing a two-stage fine-tuning training strategy, which only fine-tunes the total parameters of the dual-branch spatial-channel adapter and the spatial-channel interaction low-rank adaptive module. ,in,
[0021] The specific expression for the first stage of fine-tuning is as follows: ,in Represents the rate function and distortion function The trade-off coefficient between them The potential representation of quantization, This represents auxiliary information for quantification. Indicates the input sample. Indicates the output;
[0022] The second stage of fine-tuning freezes the parameters of each encoder-side component and the entropy model in the learnable image compression model, and synthesizes the variations. It is divided into four stacks, and only the last stack at the decoder end is used to fine-tune the distortion loss.
[0023] In some embodiments, the domain adaptive object includes one or more of the following: pixel domain, screen content domain, and crater domain.
[0024] In some embodiments, the deep learning-based learnable image compression model is selected from any one of Cheng2020, ELIC, TCM, and MLICpp.
[0025] In some embodiments, only adapter parameters are stored during the fine-tuning process, and the fine-tuning is completed in one step, allowing for the analysis and changes of the image compression model to be learned. The dual-branch space-channel adapter parameters are stored in the encoder, while the remaining adapter parameters are transmitted to the decoder.
[0026] According to a second aspect of the present disclosure, a spatial-channel-aware image compression domain adaptive system is provided, the system comprising:
[0027] DBSCA building blocks are used for deep learning-based learnable image compression models to build a two-branch spatial-channel adapter for transform networks to independently capture spatial layout and channel correlation.
[0028] The SCI-LoRA building block is used for deep learning-based learnable image compression models, and to build a low-rank adaptive module for space-channel interaction in entropy models to jointly model spatial and channel contexts.
[0029] The module integration and fine-tuning module is used to integrate the dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module into a deep learning learnable image compression model, and to fine-tune it using a small number of samples, thereby achieving domain adaptation for a specific domain.
[0030] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described spatial-channel-aware image compression domain adaptive method.
[0031] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein computer instructions are stored on the storage medium, and when executed by a processor, the instructions implement the steps of the above-described spatial-channel-aware image compression domain adaptive method.
[0032] This disclosure provides a spatial-channel-aware image compression domain adaptation method, system, and storage medium for learnable image compression, achieving efficient domain adaptation for specific domains. Specifically, by introducing a dual-branch spatial-channel adapter into the transform network and incorporating spatial-channel interaction LoRA into the entropy model, it effectively adapts to cross-domain spatial and channel information offsets, thus addressing the domain gap problem. This invention not only continuously improves pre-trained LIC models with less than 1.5% additional parameters to match or surpass the performance of traditional VVC codecs across all target domains, but also achieves results comparable to fully fine-tuned schemes. These results make this invention a practical and scalable solution for LIC domain adaptation.
[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0034] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0035] Figure 1 This is a schematic diagram of the image compression domain adaptive method based on spatial-channel awareness in an embodiment of the present invention;
[0036] Figure 2 This is a schematic diagram illustrating the framework principle of the image compression domain adaptive method based on spatial-channel perception in an embodiment of the present invention;
[0037] Figure 3 This is a structural diagram of the image compression domain adaptive system based on spatial-channel awareness in an embodiment of the present invention;
[0038] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation
[0039] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the drawings, not the entire structure.
[0040] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.
[0041] A spatial-channel-aware image compression domain adaptive method, system, and storage medium are provided, with the following embodiments:
[0042] like Figure 1 As shown, the specific implementation steps of the image compression domain adaptive method based on spatial-channel awareness are as follows:
[0043] S1. A learnable image compression model based on deep learning is constructed to build a dual-branch spatial-channel adapter for the transformation network to independently capture spatial layout and channel correlation.
[0044] S2. A deep learning-based learnable image compression model is constructed to build a low-rank adaptive module for space-channel interaction in the entropy model, so as to jointly model the spatial and channel contexts.
[0045] S3. Integrate the dual-branch spatial-channel adapter and the spatial-channel interaction low-rank adaptive module into the deep learning learnable image compression model, and fine-tune it using a small number of samples to achieve domain adaptation for a specific domain.
[0046] Specifically, the parameter-efficient transfer learning method tailored for LIC emphasizes the importance of spatial information and implements a spatial channel-aware mechanism to effectively capture and adjust the spatial and channel feature statistics of a specific domain. Two core components specifically tailored for the unified LIC framework are proposed: a dual-branch spatial-channel adapter (DBSCA) for the transform network in LIC to independently capture spatial layout and channel correlations, and a spatial-channel interaction low-rank adaptive module (SCI-LoRA) for the entropy model in LIC to jointly model spatial and channel contexts. Integrating these two modules into a pre-trained LIC model and fine-tuning with a small number of samples achieves efficient domain adaptation for a specific domain.
[0047] The Dual Branch Space-Channel Adapter (DBSCA) specifically includes:
[0048] For input features The output aggregates adaptive features of spatial and channel operations. The spatial operation is depthwise separable convolution, and the channel operation includes: dimensionality reduction through a 1x1 convolution, passing the output through the LeakyReLU activation function, and then passing it through a 1x1 convolution to restore the dimension to be consistent with the dimension of the channel branch input.
[0049] The dual-branch space-channel adapter is initialized using identity mapping, including: the space branch is initialized to pass-through output, while the channel branch output is initialized to zero.
[0050] Specifically, to enhance the spatial adaptability of the transform network, DBSCA separates the modeling of spatial structure and channel correlation, with the two branches complementarily optimized during domain adaptation.
[0051] The spatial and channel branch output features are represented as follows: and ,a The input features are defined as follows. Specifically, the formulas for the spatial and channel branches are as follows, and their outputs are aggregated to produce the final adaptive features:
[0052] ,
[0053] ,
[0054] ,
[0055] in , and . Represents spatial operations. and Represents channel operation. This represents the activation function.
[0056] In the spatial branching design, depthwise separable convolution is chosen as the spatial operation, and the convolution kernel is... .set up k represents the kernel size, C represents the number of channels, the number of input channels is equal to the number of output channels (both are equal to C), and group is the number of groups in the convolution. This structure can independently filter each channel, preserve spatial continuity and local features, and avoid interference from global information, providing a strong balance between modeling ability and computational efficiency.
[0057] In the channel branch design, a lightweight bottleneck layer is employed. First, a 1x1 convolution is used for dimensionality reduction. Then, the output is activated by the LeakyReLU function, and finally, another 1x1 convolution restores the dimension to match the branch input dimension. This branch captures channel statistics and reallocates them to adapt to the target domain representation, significantly improving nonlinear representation capabilities for better cross-domain generalization.
[0058] To facilitate stable training, DBSCA is initialized using an identity mapping: the spatial branch is initialized to the pass-through output, while the channel branch output is initialized to zero, in order to avoid destroying pre-trained features.
[0059] The specific expression for the spatial-channel interaction low-rank adaptive module SCI-LoRA is:
[0060] ,
[0061] Indicates input features, For the original pre-trained weights 'a', For updating the convolution weights, , For projection along the channel dimension, For spatial operations that can be applied independently in low-rank dimensions, this is used to achieve fine-grained local filtering and enrich spatial information, where... Gaussian initialization is used. Set to zero. Uniform initialization using Xavier is employed.
[0062] Specifically, in the design of the Spatial-Channel Interaction Low-Rank Adaptive Module (SCI-LoRA), the original LoRA lacks spatial awareness, and for entropy modeling in LIC, the ability to extract and utilize spatial correlations is crucial. Domain offset often affects local texture or structural statistics, and these factors directly affect entropy estimation. Therefore, based on the above issues, the module design aims to make entropy estimation more accurate. The specific steps are as follows:
[0063] The original LoRA introduces a low-rank decomposition for the frozen weights, updating the pre-trained weights. . Two low-rank decomposition matrices , , , , These represent the number of input channels and the number of output channels, respectively. r represents the low-rank dimension. The specific update process is as follows:
[0064] ,
[0065] While highly efficient, the original LoRA only considers the problem from the channel dimension and lacks spatial modeling. To overcome this limitation, a direct solution is to decompose the updated convolutional weights. , where the matrix , . It performs convolution operations in a low-dimensional (rxr) space. This operation has rich spatial-channel interactions and strong adaptability, but its... The cost of the parameters makes it impractical for lightweight fine-tuning;
[0066] To reduce the parameter burden while maintaining spatial modeling capabilities, the inventors designed SCI-LoRA: inserting lightweight depth separability between low-rank channel projections. Convolutional kernels introduce the ability to model local spatial correlations at a minimal parameter cost, thus enabling stable estimation of latent variable distributions even when cross-domain texture or structural statistical shifts occur. The overall SCI-LoRA formula is as follows:
[0067] ,
[0068] ,
[0069] in and It is a projection along the channel dimension. It is a spatial operation applied independently in the low-rank dimension, enabling fine-grained local filtering and enriching spatial information. This represents the convolution operation;
[0070] To ensure stable optimization, Gaussian initialization is used. Set to zero. Using Xavier uniform initialization ensures This ensures training stability.
[0071] The fine-tuning module is integrated with the pre-trained LIC model. Specifically, the dual-branch spatial-channel adapter and the spatial-channel interaction low-rank adaptive module are integrated into a deep learning learnable image compression model, such as... Figure 2 As shown, it includes:
[0072] Analysis of the variation in serially inserting a dual-branch spatial-channel adapter into a deep learning-based learnable image compression model. With synthetic changes Following the nonlinear transformation in the model, a space-channel interactive low-rank adaptive module is added to the supersynthetic transformation network of the entropy model. In the convolutional layer.
[0073] Fine-tuning is performed using a small number of samples. Specifically, all parameters in the pre-trained model are frozen, and a two-stage fine-tuning training strategy is adopted to fine-tune only the total parameters of the dual-branch spatial-channel adapter (DBSCA) and the spatial-channel interaction low-rank adaptive module (SCI-LoRA). ,in,
[0074] The first phase of fine-tuning ensures that only... The parameters are jointly optimized, and the specific expression is: ,in Represents the rate function and distortion function The trade-off coefficient between them The potential representation of quantization, This represents auxiliary information for quantification. Indicates the input sample. Indicates the output;
[0075] The second stage of fine-tuning freezes the parameters of each encoder-side component and the entropy model in the learnable image compression model, and synthesizes the variations. Divided into four stacks, only the last stack at the decoder end is used to fine-tune the distortion loss (represented as...). The specific expression is: .
[0076] In some embodiments, the pixel domain, screen content domain, and crater domain are selected as the domain adaptation target domains. In specific implementation, a fixed test set containing 100 images is selected for each domain.
[0077] In some embodiments, the deep learning-based learnable image compression model is selected from any one of Cheng2020, ELIC, TCM, and MLICpp. Specifically, when selecting the pre-trained model, a domain adaptation method is tested on several state-of-the-art LIC models, including but not limited to Cheng2020, ELIC, TCM, and MLICpp.
[0078] In some embodiments, the training details are configured such that 25 target domain image samples are used for training at each fine-tuning stage. These images are divided into 20 training images and 5 validation images. The image cropping size is set to 256x256, and a batch size of 4 is used during training. The bottleneck of DBSCA is the middle dimension. Set to 64, and The convolutional kernel size is set to 3, and the low-rank dimension r=10. The first phase of training consists of 750 epochs, with a learning rate set... Dynamic decay will occur. The second stage will use... The program was trained for another 750 epochs using a fixed learning rate. Throughout the fine-tuning process, all backbone parameters were frozen, with only the DBSCA and SCI-LoRA parameters being updated.
[0079] During fine-tuning, only adapter parameters are stored, and fine-tuning is completed in one go, allowing for the analysis and change of the image compression model. The dual-branch spatial-channel adapter parameters are stored in the encoder, while the remaining adapter parameters are transmitted to the decoder. In this way, the entire set of adapters corresponds to a specific domain, making it plug-and-play and allowing for flexible handling of image compression for specific target domains.
[0080] The above steps complete the domain adaptation process of a pre-trained LIC model in the target domain.
[0081] Another embodiment illustrates a spatial-channel-aware image compression domain adaptive system, such as... Figure 3 As shown, the system 300 includes:
[0082] DBSCA building block 310 is used for deep learning-based learnable image compression models to build a two-branch spatial-channel adapter for transform networks to independently capture spatial layout and channel correlation.
[0083] SCI-LoRA building block 320 is used for deep learning-based learnable image compression models to build a low-rank adaptive module for space-channel interaction for entropy models to jointly model spatial and channel contexts.
[0084] The module integration and fine-tuning module 330 is used to integrate the dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module into a deep learning learnable image compression model, and to fine-tune it using a small number of samples, thereby achieving domain adaptation for a specific domain.
[0085] In addition to the modules described above, the system may also include other components; however, since these components are not relevant to the embodiments of this disclosure, their illustrations and descriptions are omitted here.
[0086] Other specific working processes of the spatial-channel-aware image compression domain adaptive system are described in the above-described embodiment of the spatial-channel-aware image compression domain adaptive method, and will not be repeated here.
[0087] Another embodiment illustrating that the system of the present invention can also be achieved by means of... Figure 4 The architecture of the computing device shown is used to implement this. Figure 4 The architecture of the computing device is shown. For example... Figure 4 As shown, the computer system 410 includes a system bus 430, one or more CPUs 440, input / output 420, and memory 450. Memory 450 can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU, including the spatial-channel-aware image compression domain adaptive method of the embodiment. Figure 4 The architecture shown is merely exemplary and should be adjusted according to actual needs when implementing different devices. Figure 4 One or more components in the system. The memory 450, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the spatial-channel-aware image compression domain adaptation method in this embodiment of the invention (e.g., the DBSCA construction module 310, SCI-LoRA construction module 320, and module integration and fine-tuning module 330 in the spatial-channel-aware image compression domain adaptation system 300). One or more CPUs 440 execute various functional applications and data processing of the system of the present invention by running the software programs, instructions, and modules stored in the memory 450, that is, implementing the above-described spatial-channel-aware image compression domain adaptation method, which includes the following steps:
[0088] Deep learning-based learnable image compression models:
[0089] Construct a dual-branch spatial-channel adapter for the transformation network to independently capture spatial layout and channel correlation;
[0090] Construct a low-rank adaptive module for space-channel interactions in entropy models to jointly model spatial and channel contexts;
[0091] The dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module are integrated into a deep learning learnable image compression model, and fine-tuned using a small number of samples, thus achieving domain adaptation for a specific domain.
[0092] Of course, the processor of the server provided in the embodiments of the present invention is not limited to performing the method operations described above, but can also perform related operations in the image compression domain adaptive method based on spatial-channel awareness provided in any embodiment of the present invention.
[0093] The memory 450 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on terminal usage. Furthermore, the memory 450 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 450 may further include memory remotely configured relative to one or more CPUs 440, these remote memories being connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0094] Input / output 420 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Input / output 420 may also include a display device such as a display screen.
[0095] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the space-channel-aware image compression domain adaptive method described in the above embodiments. The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0096] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device.
[0097] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0098] Furthermore, other specific operational processes of a non-transitory computer-readable storage medium are described in the above-described embodiment of the image compression domain adaptation method based on spatial-channel awareness, and will not be repeated here.
[0099] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.
[0100] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. An image compression domain adaptive method based on spatial-channel awareness, characterized in that, The method includes: Deep learning-based learnable image compression models: Construct a dual-branch spatial-channel adapter for the transformation network to independently capture spatial layout and channel correlation; Construct a low-rank adaptive module for space-channel interactions in entropy models to jointly model spatial and channel contexts; The dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module are integrated into a deep learning learnable image compression model, and fine-tuned using a small number of samples, thus achieving domain adaptation for a specific domain. The dual-branch space-channel adapter specifically includes: For input features The output aggregates adaptive features of spatial and channel operations. The spatial operation is depthwise separable convolution, and the channel operation includes: dimensionality reduction through a 1x1 convolution, passing the output through the LeakyReLU activation function, and then passing it through a 1x1 convolution to restore the dimension to be consistent with the dimension of the channel branch input. The dual-branch space-channel adapter is initialized using identity mapping, including: the space branch is initialized to a pass-through output, while the channel branch output is initialized to zero; Integrating the dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module into a deep learning-based learnable image compression model specifically includes: Analysis of the variation in serially inserting a dual-branch spatial-channel adapter into a deep learning-based learnable image compression model. With synthetic changes Following the nonlinear transformation in the model, a space-channel interactive low-rank adaptive module is added to the supersynthetic transformation network of the entropy model. In the convolutional layer.
2. The image compression domain adaptive method based on spatial-channel awareness according to claim 1, characterized in that, The specific expression for the space-channel interaction low-rank adaptive module is as follows: ， Indicates input features, For the original pre-trained weights, For updating the convolution weights, , For projection along the channel dimension, For spatial operations that can be applied independently in low-rank dimensions, this is used to achieve fine-grained local filtering and enrich spatial information, where... Gaussian initialization is used. Set to zero. Uniform initialization using Xavier is employed.
3. The image compression domain adaptive method based on spatial-channel awareness according to claim 1, characterized in that, Fine-tuning is performed using a small number of samples, employing a two-stage fine-tuning training strategy that only fine-tunes the total parameters of the bi-branch spatial-channel adapter and the spatial-channel interaction low-rank adaptive module. ,in, The specific expression for the first stage of fine-tuning is as follows: ,in Represents the rate function and distortion function The trade-off coefficient between them The potential representation of quantization, This represents auxiliary information for quantification. Indicates the input sample. Indicates the output; The second stage of fine-tuning freezes the parameters of each encoder-side component and the entropy model in the learnable image compression model, and synthesizes the variations. It is divided into four stacks, and only the last stack at the decoder end is used to fine-tune the distortion loss.
4. The image compression domain adaptive method based on spatial-channel awareness according to claim 1, characterized in that, The deep learning-based learnable image compression model is selected from any one of Cheng2020, ELIC, TCM, and MLICpp.
5. The image compression domain adaptive method based on spatial-channel awareness according to claim 3, characterized in that, During fine-tuning, only adapter parameters are stored, and fine-tuning is completed in one go, allowing for the analysis and change of the image compression model. The dual-branch space-channel adapter parameters are stored in the encoder, while the remaining adapter parameters are transmitted to the decoder.
6. An image compression domain adaptive system based on spatial-channel awareness, characterized in that, The system includes: DBSCA building blocks are used for deep learning-based learnable image compression models to build a two-branch spatial-channel adapter for transform networks to independently capture spatial layout and channel correlation. The SCI-LoRA building block is used for deep learning-based learnable image compression models, and to build a low-rank adaptive module for space-channel interaction in entropy models to jointly model spatial and channel contexts. The module integration and fine-tuning module is used to integrate the dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module into the deep learning learnable image compression model, and to fine-tune it using a small number of samples, thereby achieving domain adaptation for a specific domain. The dual-branch space-channel adapter specifically includes: For input features The output aggregates adaptive features of spatial and channel operations. The spatial operation is depthwise separable convolution, and the channel operation includes: dimensionality reduction through a 1x1 convolution, passing the output through the LeakyReLU activation function, and then passing it through a 1x1 convolution to restore the dimension to be consistent with the dimension of the channel branch input. The dual-branch space-channel adapter is initialized using identity mapping, including: the space branch is initialized to a pass-through output, while the channel branch output is initialized to zero; Integrating the dual-branch spatial-channel adapter and the spatial-channel interactive low-rank adaptive module into a deep learning-based learnable image compression model specifically includes: Analysis of the variation in serially inserting a dual-branch spatial-channel adapter into a deep learning-based learnable image compression model. With synthetic changes Following the nonlinear transformation in the model, a space-channel interactive low-rank adaptive module is added to the supersynthetic transformation network of the entropy model. In the convolutional layer.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the spatial-channel-aware image compression domain adaptive method as described in any one of claims 1 to 5.
8. A non-transitory computer-readable storage medium, wherein computer instructions are stored on the storage medium, characterized in that, When the instructions are executed by the processor, they implement the steps of the spatial-channel-aware image compression domain adaptive method as described in any one of claims 1 to 5.