Layered multi-context space adaptive image compression method

Abstract

The invention relates to the technical field of image compression, in particular to a hierarchical multi-context space adaptive image compression method. The method comprises the steps of obtaining a training set and a test set; constructing a spatial adaptive feature modulation network, wherein the spatial adaptive feature modulation network comprises an encoder, a decoder, a super-prior encoder and a super-prior decoder; constructing a multi-context joint entropy estimation model, wherein the multi-context joint entropy estimation model comprises a local context model, a channel context model, an in-layer anchor point context model and a cross-layer global context model; obtaining an image compression network in combination with the spatial adaptive feature modulation network and the multi-context joint entropy estimation model; respectively training and testing an image compression network by using the training set and the test set to obtain a trained image compression network; and inputting a to-be-processed image into the trained image compression network to obtain a reconstructed image. According to the invention, the image compression effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image compression, and in particular to a layered multi-context space adaptive image compression method. Background Art

[0002] With the explosive growth of digital image and video data, image compression technology has become a key technology for solving the storage and transmission problems of massive data. While traditional image compression techniques can reduce data redundancy to a certain extent, they rely on hand-crafted transform kernels and prediction models, making it difficult to fully exploit the complex features of images. This leads to a significant bottleneck between compression efficiency and reconstruction quality.

[0003] Image compression methods based on deep learning have gradually become a research hotspot due to their powerful nonlinear feature extraction capabilities and end-to-end optimization characteristics. Existing image compression methods based on convolutional neural networks (CNNs) struggle to simultaneously capture both long-range global dependencies and fine-grained local texture information in an image due to the fixed receptive field of the convolution kernel. This makes it difficult to simultaneously capture both global semantic associations and detailed texture features in an image. Furthermore, the computational complexity of the window-based attention mechanism of the Transformer results in inefficient processing of high-resolution images. Existing entropy models typically use autoregressive entropy models and context models. The encoding and decoding time of the autoregressive entropy model increases exponentially with resolution, while the context model relies only on spatial information at a single scale, ignoring inter-channel redundancy and global spatial correlation, resulting in limited entropy estimation accuracy. Existing image compression methods often sacrifice computational efficiency in pursuit of high compression rates, resulting in a loss of global information during image compression. Summary of the Invention

[0004] In order to solve the problem that the existing methods cause the loss of global information of the image during the image compression process, the purpose of the present invention is to provide a layered multi-context spatial adaptive image compression method. The technical solution adopted is as follows:

[0005] The present invention provides a layered multi-context spatially adaptive image compression method, which comprises the following steps:

[0006] Get the training set and test set;

[0007] Construct a spatially adaptive feature modulation network, including an encoder, a decoder, a super-prior encoder, and a super-prior decoder;

[0008] Construct a multi-context joint entropy estimation model, including a local context model, a channel context model, an intra-layer anchor context model, and a cross-layer global context model;

[0009] Combining the spatial adaptive feature modulation network and the multi-context joint entropy estimation model to obtain an image compression network;

[0010] Using the training set and the test set to train and test the image compression network respectively, to obtain a trained image compression network;

[0011] The image to be processed is input into the trained image compression network to obtain the reconstructed image.

[0012] Preferably, after obtaining the training set and the test set, the method further includes: cropping the images in the training set.

[0013] Preferably, the constructing of a spatially adaptive feature modulation network includes:

[0014] Constructing an encoder and a super-prior encoder consisting of a stack of residual downsampling modules and spatial adaptive feature modulation modules; wherein the encoder is composed of a cascade of four residual downsampling modules and four spatial adaptive feature modulation modules, and the super-prior encoder is composed of a cascade of two residual downsampling modules and two spatial adaptive feature modulation modules, and the residual downsampling module uses GDN as the activation function;

[0015] A decoder and a super-prior decoder are constructed with a structure symmetrical to the encoder along the horizontal axis.

[0016] Preferably, the spatial adaptive feature modulation module adopts a multi-scale dynamic fusion architecture. The front section is combined with a multi-branch spatial modulation layer after layer normalization, and uses a parallel void convolution group to extract multi-receptive field features and retain the original features with residuals; the back section is a layer normalization-connected channel mixing enhancement module, which uses point-by-point convolution to compress the channel dimension and generate dynamic gating weights; the front and back cascades form an adaptive enhancement path to obtain enhanced features.

[0017] Preferably, the local context model uses checkerboard convolution to separate the anchor and non-anchor domains, and uses the Mamba state space model to capture long-range dependencies in the anchor domain, which is fused with the non-anchor feature residuals; the channel context model groups channels asymmetrically by frequency, uses double-layer attention for the low-frequency group, and single-layer attention for the high-frequency group; cross-group features are fused through dynamic gating; the intra-layer anchor context model performs 5×5 masked convolution on the anchor features to generate spatial priors, and the non-anchor features calculate the association weights through cross-position linear attention; the cross-layer global context model deconvolves the deep features with the shallow features, generates a global scale factor through lightweight attention, and optimizes the local entropy parameters.

[0018] The present invention has at least the following beneficial effects:

[0019] The present invention constructs a spatial adaptive feature modulation network, which extracts multi-scale features in parallel through multi-branch void convolution groups, combines with a dynamic weight fusion mechanism, adaptively aggregates local details and global semantic information, and introduces a channel mixing enhancement module to form a "space-channel" two-dimensional adaptive modeling, which significantly improves the feature expression capability while maintaining linear computational complexity; this embodiment constructs a multi-context joint entropy estimation model, dynamically captures long-range dependencies, joint layer anchor-non-anchor interaction and cross-layer feature fusion, and combines channel grouping attention to optimize the efficiency of information transmission between channels, supports fully parallel encoding and decoding, ensures bit rate savings while improving speed, and achieves better image compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of a layered multi-context spatially adaptive image compression method provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of an image compression network provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of the structure of ResDownSample of the image compression network provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of the ResUpSample of the image compression network provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure of the MSC Block of the image compression network provided by an embodiment of the present invention;

[0026] Figure 6 This is the data flow diagram of the multi-context joint entropy estimation model;

[0027] Figure 7 is the original image in the Kodak dataset;

[0028] Figure 8 This is the result image obtained by compressing the original image using the image compression network based on wavelet adaptive convolutional neural network;

[0029] Figure 9This is the result image obtained by compressing the original image using the multi-reference entropy model to learn the image compression network;

[0030] Figure 10 This is a graph showing the test results of the image compression network provided by an embodiment of the present invention on the original images in the Kodak dataset. DETAILED DESCRIPTION

[0031] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a layered multi-context spatial adaptive image compression method proposed in accordance with the present invention in combination with the accompanying drawings and preferred embodiments.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] The following describes in detail a specific scheme of a layered multi-context spatial adaptive image compression method provided by the present invention with reference to the accompanying drawings.

[0034] A hierarchical multi-context spatially adaptive image compression method embodiment:

[0035] This embodiment proposes a hierarchical multi-context space adaptive image compression method, such as Figure 1 As shown, a layered multi-context spatial adaptive image compression method of this embodiment includes the following steps:

[0036] Step S1: Obtain training set and test set.

[0037] First, we obtained a training set consisting of 58,106 images. Each image in the training set was randomly cropped to a uniform size of 256×256. This ensures consistent image resolution, prominent features, and conserves video memory. Following the principles of reproducible research, we selected the frequently cited Kodak, CLIC Professional Valid, and Tecnick datasets as the test set for the model. This test set is independent of the training process.

[0038] It should be noted that the images used subsequently are all cropped images.

[0039] Step S2: constructing a spatial adaptive feature modulation network, including an encoder, a decoder, a super-prior encoder and a super-prior decoder.

[0040] First, an encoder and a super priori encoder are constructed, which are composed of a stack of residual downsampling modules and attention modules. The encoder is composed of a cascade of 4 residual downsampling modules and 4 spatial adaptive feature modulation modules. The super priori encoder is composed of a cascade of 2 residual downsampling modules and 2 spatial adaptive feature modulation modules. Each residual downsampling block consists of a 3×3 convolution and a 1×1 convolution in parallel. The 3×3 convolution branch adds an activation function GDN after the convolution. Figure 3 As shown in the figure, this figure is a schematic diagram of the structure of ResDownSample.

[0041] like Figure 5 As shown in the figure, this is a structural diagram of the Multi-Scale Convolution Block (MSC Block). The spatial adaptive feature modulation module adopts a hierarchical multi-path architecture, which divides the feature map into four groups along the channel dimension, and respectively processes downsampling reconstruction, channel compression, local detail enhancement, and cross-scale upsampling. Branch one uses two 3×3 convolutions with a step size of 2 to gradually downsample to 1 / 4 of the original resolution, and then restores it to half resolution and reduces the channel to half through PixelShuffle; Branch two uses 3×3 convolution with a step size of 2 to downsample and then compresses the channel to one-quarter through Gaussian normalization (GDN); Branch three retains the original resolution and uses two 3×3 convolutions with a step size of 1 combined with GDN and IGDN to enhance local details; Branch four uses a 3×3 convolution with a step size of 1 and PixelShuffle to achieve double resolution upsampling and halve the channel. The outputs of the four branches are fused and connected to the input residual to achieve spatial adaptive feature modulation.

[0042] Then, a decoder and a super-prior decoder with a symmetrical structure along the horizontal axis with the encoder are constructed, such as Figure 4 As shown in the figure, this is a schematic diagram of the structure of ResUpSample. The decoder and super-prior decoder use PixelShuffle as the basic unit, combine 3×3 convolution and inverse activation function IGDN to construct an upsampling residual module, and construct a decoder and super-prior decoder with a symmetrical structure along the horizontal axis with the encoder.

[0043] The spatially adaptive feature modulation network is combined with the super-prior variational autoencoder framework, including the codec, super-prior codec, and quantization and arithmetic codec parts, to form a super-prior variational autoencoder framework with spatially adaptive feature modulation network.

[0044] Step S3: construct a multi-context joint entropy estimation model, including a local context model, a channel context model, an intra-layer anchor context model, and a cross-layer global context model.

[0045] Furthermore, this embodiment constructs a multi-context joint entropy estimation model, which mainly includes four context models and a channel autoregressive model. The four context models are: local context model, channel context model, intra-layer anchor context model and cross-layer global context model. A hierarchical context modeling framework is adopted. Four types of context features are extracted synchronously when the current slice is decoded. The extracted local spatial features, channel features and super-prior parameters are generated into a Gaussian distribution mean and standard deviation through a channel autoregressive network to drive the arithmetic decoder to reconstruct the anchor information in the potential representation. The anchor information is fed into the local context model and the intra-layer anchor context model to update the local and intra-layer anchor information; the extracted cross-layer global context features and channel features and the local and intra-layer anchor information are input into the channel autoregressive network to update the probability parameters for generating non-anchor information and guiding subsequent slice decoding to form an iteration. The data flow of the multi-context joint entropy estimation model is as follows: Figure 6 shown.

[0046] Among them, the channel autoregressive model consists of three 1×1 convolutions and two ReLU functions alternately cascaded;

[0047] The channel context module consists of three 3×3 convolutions and two ReLU functions alternately cascaded. It uses a non-uniform channel grouping approach, with the first encoded channel group being allocated more bits and subsequent channel groups being allocated fewer bits. Spatial context modeling is performed in each channel slice, and masked convolution is used to constrain the current channel to depend only on the decoded channel, generating inter-channel dependency weights.

[0048] The local context model uses a three-stage progressive operation. In the first stage, the input features are divided into a sequence of local windows using 5×5 overlapping windows. Rotational position encoding (RoPE) is injected into the window dimension to embed the absolute position information into the feature vector. In the second stage, 3×3 depthwise separable convolution is used to perform neighborhood feature enhancement on each window to extract local details. Finally, a dynamic gating mechanism of the state-space model is introduced. The SiLU activation function is used to generate adaptive weight-assisted linear attention for dynamic feature capture. The extracted results retain the original features through residual links to obtain the final local spatial context features.

[0049] The intra-layer anchor context model constructs a dual-path feature acquisition channel. First, the anchor and non-anchor regions are decomposed through a 3×3 checkerboard convolution. Multi-scale pooling features are fused through bilinear upsampling to obtain a continuous feature map. On this basis, group attention calculation and transposed convolution are performed to obtain enhanced features for local-global joint modeling. Finally, weighted original anchor and enhanced features are obtained through dynamic gating.

[0050] The cross-layer global spatial context model adopts a memory-enhanced architecture and employs multi-scale dilated convolution groups to extract local details, medium-range dependencies, and long-range context features in parallel. A dynamic sampling network with variable convolution corrects cross-layer feature space offsets. A dual-path gated attention mechanism is constructed based on multi-scale features to generate spatial weights and channel weights, which are then fused to obtain cross-layer joint features.

[0051] The local context module, channel context module, intra-layer anchor context module, cross-layer global context module and channel autoregressive model are combined with entropy encoding and decoding to establish a multi-context joint entropy estimation model.

[0052] Step S4: combining the spatial adaptive feature modulation network and the multi-context joint entropy estimation model to obtain an image compression network.

[0053] The spatially adaptive feature modulation network is combined with the super-prior variational autoencoder framework, including the codec, super-prior codec, as well as the quantization and arithmetic codec parts; the multi-context joint entropy estimation model is combined with the super-prior variational autoencoder, and end-to-end coupling is achieved through joint rate-distortion optimization to form a hierarchical spatially adaptive image compression network based on the Mamba-driven entropy model.

[0054] So far, by sampling the above method, this embodiment obtains the image compression network, such as Figure 2 As shown in FIG, this figure is a structural diagram of the image compression network provided by this embodiment.

[0055] Step S5: Use the training set and the test set to train and test the image compression network respectively to obtain a trained image compression network.

[0056] After building the image compression network, the constructed image compression network needs to be trained and tested.

[0057] Specifically, the training set is grouped into a fixed batch size B, the first group of images is fed into the image compression network, and the convolutional layer weight matrix and bias vector are initialized through forward propagation, where the input tensor specification is B×256×256×3; with the purpose of minimizing the loss function, the Adam optimizer is used to update the gradient direction, and MSE and MS-SSIM are used as optimization directions to train end-to-end image compression network models with different compression rates.

[0058] During the image compression network training process, As the rate-distortion loss function in the image compression network training process; where, is the loss function, θ e ,θ d ,θ he and θ hdare the trainable parameters of the encoder, decoder, super encoder and super decoder, represents the fully decomposed differentiable entropy model, and for and The bit rate, For the original image x and the reconstructed image , λ represents the penalty coefficient, and the compression rate is controlled by adjusting λ.

[0059] The reconstruction error and the bitrate loss of the verification image are calculated by optimizing the rate-distortion objective function. The gradient backpropagation mechanism of the Adam optimizer is used to iteratively adjust the network weights with the goal of minimizing the loss function to generate an updated image compression network. The parameter update process is executed cyclically until the iterative optimization process of all training samples in the feedforward network is completed.

[0060] The test set is input into the trained image compression network to test the image compression network. The output of the image compression network is the reconstructed image, that is, the compressed image.

[0061] In this embodiment, the image compression network can perform quality assessment based on PSNR (peak signal-to-noise ratio), MS-SSIM (multi-scale structural similarity) and compression ratio;

[0062] PSNR can be obtained from the MSE value, and the calculation formula of MSE is:

[0063]

[0064] Where MSE represents mean square error, I1 and I2 are two single-channel images with m rows and n columns, i represents the pixel in the i-th row of the image, and j represents the pixel in the j-th column of the image;

[0065] The PSNR calculation formula is:

[0066]

[0067] Among them, MAX represents the maximum color intensity value of the image pixel. For example, the maximum pixel value of an image with a bit width of 8 bits is 2 8 -1=255;

[0068] The MS-SSIM calculation formula is:

[0069]

[0070] Among them, MS-SSIM represents multi-scale structural similarity, μ x 、 are x and The mean ofx 、 are x and The variance of c1 and c2 are small constants used to stabilize the coefficients. S represents the original image scale level, m represents the starting scale level, and α S and β S is used to adjust the weights of different components.

[0071] The higher the PSNR and MS-SSIM values, the better the image quality and the smaller the distortion.

[0072] At this point, this embodiment has obtained a trained image compression network. The image compression network provided in this embodiment and other algorithms were tested on the images in the Kodak dataset and the results were compared with the original images. Figure 7 、 Figure 8 、 Figure 9 、 Figure 10 As shown, Figure 7 is the original image in the Kodak dataset, Figure 8 This is the result of compressing the original image using an image compression network based on a wavelet adaptive convolutional neural network. The peak signal-to-noise ratio of the image is 30.2619 and the bit value is 0.0697. Figure 9 This is the result of compressing the original image using the multi-reference entropy model to learn the image compression network. The peak signal-to-noise ratio of the image is 30.0982 and the bit value is 0.0651. Figure 10 This is a graph showing the test results of the image compression network provided in this embodiment on images in the Kodak dataset. The peak signal-to-noise ratio of this graph is 32.4872 and the number of bits is 0.0645.

[0073] Step S6: input the image to be processed into the trained image compression network to obtain a reconstructed image.

[0074] After obtaining the trained image compression network, this embodiment directly inputs the image to be processed into the trained image compression network when the image to be processed is subsequently compressed. The image compression network outputs a reconstructed image, which is the image after the image to be processed has been compressed.

[0075] Thus, the image compression network provided by this embodiment can ensure the compression quality of the image while compressing the image.

[0076] This embodiment constructs a spatial adaptive feature modulation network, which extracts multi-scale features in parallel through multi-branch void convolution groups, combines with a dynamic weight fusion mechanism, adaptively aggregates local details and global semantic information, and introduces a channel mixing enhancement module to form a "space-channel" two-dimensional adaptive modeling, which significantly improves the feature expression capability while maintaining linear computational complexity; this embodiment constructs a multi-context joint entropy estimation model, dynamically captures long-range dependencies, anchor-non-anchor interactions within the joint layer, and cross-layer feature fusion, and combines channel grouping attention to optimize the efficiency of information transmission between channels, supports fully parallel encoding and decoding, ensures bit rate savings while improving speed, and achieves better image compression performance.

[0077] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hierarchical multi-context spatially adaptive image compression method, characterized in that: The method comprises the following steps: Get the training set and test set; Construct a spatially adaptive feature modulation network, including an encoder, a decoder, a super-prior encoder, and a super-prior decoder; Construct a multi-context joint entropy estimation model, including a local context model, a channel context model, an intra-layer anchor context model, and a cross-layer global context model; Combining the spatial adaptive feature modulation network and the multi-context joint entropy estimation model to obtain an image compression network; Using the training set and the test set to train and test the image compression network respectively, to obtain a trained image compression network; The image to be processed is input into the trained image compression network to obtain the reconstructed image.

2. The hierarchical multi-context spatially adaptive image compression method according to claim 1, characterized in that: After obtaining the training set and the test set, the process also includes: cropping the images in the training set.

3. The hierarchical multi-context spatially adaptive image compression method according to claim 1, characterized in that: The constructing of the spatial adaptive feature modulation network includes: Constructing an encoder and a super-prior encoder consisting of a stack of residual downsampling modules and spatial adaptive feature modulation modules; wherein the encoder is composed of a cascade of four residual downsampling modules and four spatial adaptive feature modulation modules, and the super-prior encoder is composed of a cascade of two residual downsampling modules and two spatial adaptive feature modulation modules, and the residual downsampling module uses GDN as the activation function; A decoder and a super-prior decoder are constructed with a structure symmetrical to the encoder along the horizontal axis.

4. The hierarchical multi-context spatially adaptive image compression method according to claim 3, characterized in that: The spatial adaptive feature modulation module adopts a multi-scale dynamic fusion architecture. The front end is normalized and then combined with a multi-branch spatial modulation layer. It uses parallel dilated convolution groups to extract multi-receptive field features and retain the original features. The latter part is a layer normalization and channel mixing enhancement module, which uses point-by-point convolution to compress the channel dimension and generate dynamic gating weights; The front and back cascades form an adaptive enhancement pathway to obtain enhanced features.

5. The hierarchical multi-context spatially adaptive image compression method according to claim 1, characterized in that: The local context model uses checkerboard convolution to separate the anchor and non-anchor domains, and uses the Mamba state space model to capture long-range dependencies in the anchor domain, which is then fused with the non-anchor feature residuals. The channel context model groups channels asymmetrically by frequency, using double-layer attention for the low-frequency group and single-layer attention for the high-frequency group. Fusion of cross-group features via dynamic gating; The intra-layer anchor context model performs 5×5 masked convolution on the anchor features to generate spatial priors, and the non-anchor features calculate the association weights through cross-position linear attention; the cross-layer global context model deconvolves deep features with shallow features, generates a global scale factor through lightweight attention, and optimizes the local entropy parameters.

Images