A novel image compression method and system based on the RWKV model

Abstract

This invention relates to the field of image compression and discloses a novel image compression method and system based on the RWKV model. The method includes: a local-global bidirectional RWKV module, which captures neighborhood details and global semantics respectively through the collaborative fusion of local and global WKV; a multi-scale context shifting module, which integrates depthwise separable convolutions with different dilation rates and 1×1 convolutions to achieve flexible multi-scale receptive field expansion; an efficient channel attention module, which adaptively learns channel importance based on one-dimensional convolution to suppress redundant features; and an RWKV space-channel autoregressive context entropy model, which jointly models the conditional dependencies of space and channel dimensions to improve the accuracy of probability distribution prediction. This invention maintains the linear computational complexity of RWKV while enhancing local texture preservation and global structure modeling capabilities, effectively improving rate-distortion performance. It is suitable for end-to-end image compression in resource-constrained scenarios such as streaming media and cloud storage.

Description

Technical Field

[0001] This invention relates to the field of image compression, and specifically to a novel image compression method and system based on the RWKV model. Background Technology

[0002] In recent years, deep learning-based image compression methods have made significant progress, demonstrating the potential to surpass traditional hand-designed standards in terms of coding efficiency. To address the shortcomings of traditional models that only scan causal sequences in a single direction, the latest RWKV-based image compression techniques propose Bi-RWKV with bidirectional scanning to effectively model non-causal correlations between adjacent features. Furthermore, to better adapt to visual tasks, an Omni-Shift module with multi-directional label fusion has been proposed to accommodate two-dimensional latent representations. However, this model's context modeling primarily performs linear recursive modeling along the sequence dimension, without explicitly distinguishing between local neighborhood context and global semantic context. The modeling of local and global dependencies is coupled within the same mechanism, limiting its ability to model local details. While Omni-Shift achieves feature fusion through multi-directional convolution, its fixed-scale convolutional kernels limit the flexibility of the receptive field, making it difficult to simultaneously cover local details and broad contextual information. Summary of the Invention

[0003] To address the problems of existing image compression technologies, such as the high coupling between local and global dependency modeling, complex network structure, and difficulty in engineering implementation in RWKV-based image compression models, as well as the relatively small receptive field, insufficient modeling of inter-channel dependencies, redundant feature representation, and impact on encoding and decoding efficiency, this invention provides a novel image compression method and system based on the RWKV model. The technical solution is as follows:

[0004] A novel image compression method based on the RWKV model includes the following steps:

[0005] Step 1: Given the original image Analysis and transformation Map it to a latent representation ;

[0006] Step 2: Represent the latent representation Input super encoder Obtain super potential ;

[0007] Step 3: Using quantization operators Obtaining Quantitative Super Potential ;

[0008] Step 4: Quantifying hyperpotentials based on deep learning Perform entropy encoding;

[0009] Step 5: Use a super decoder at the decoding end. To obtain the initial mean and variance;

[0010] Step 6: Quantify the latent representation Furthermore, the spatial-channel context entropy model is used to predict the distribution parameters, thereby obtaining the enhanced quantized latent representation. ;

[0011] Step 7: Through the decoder Enhanced quantization latent representation Reconstructed into a compressed image ;

[0012] The method , , , Enhancements are made through the local-global RWKV module.

[0013] Furthermore, the local-global RWKV module is based on the RWKV model. In Vision-RWKV, the WKV operation is modified to a bidirectional form. The attention level of each token is calculated as follows:

[0014] ,

[0015] in, Represents a bidirectional WKV sequence of key vectors. Sum value vector Given the input, calculate the first... One Token; Indicates the total number of tokens; For traversal index; and These represent the current token's space decay and reward, respectively; and express and The One characteristic.

[0016] First, the local-global RWKV module consists of a spatial blending module and a channel blending module. K and V are divided into 4×4 blocks and fed into BiWKV to focus local features. The results of local enhancement are then input together with K into the next BiWKV to focus global features.

[0017] Secondly, the spatial mixing module performs layer normalization on the input features; then performs multi-scale context shifting; and processes the input features through three parallel linear projection layers to obtain the receiving vector, key vector, and value vector; the local-global RWKV module obtains the local and global attention results; finally, the local and global attention results are multiplied by the gated receiving vector to modulate the receiving probability.

[0018] Then, the channel mixing module performs layer normalization on the input features; then performs multi-scale context shifting; and then processes the input features through a linear projection layer to obtain the receiving vector, key vector, and value vector. By multiplying the value vector with the gated receiving vector, the receiving probability of the control value vector is obtained as an output. Finally, the attention to the channels is enhanced through an efficient channel attention module.

[0019] Then, the specific implementation formula for the multi-scale context shift is as follows:

[0020] ,

[0021] in, and They represent the first indivual Convolutional layers and The weight matrix of a depthwise separable convolutional layer Indicates an expansion rate The convolution operation.

[0022] The formula for the high-efficiency channel attention module is as follows:

[0023] ,

[0024] ,

[0025] in, For channel attention weights, For the Sigmoid function, For 1D convolution, For global average pooling, This is the enhanced output.

[0026] Finally, the space-channel context entropy model is specifically as follows: In the spatial dimension, a chessboard method is used to divide symbols into anchor groups and non-anchor groups, the anchor groups are encoded, and then the context derived from them is used to encode the non-anchor groups; in the channel dimension, the channel is divided into K blocks:

[0027] ,

[0028] in, This represents the previously decoded channel block. For the i-th part of the k-th block, a checkerboard mask convolution is used to apply the spatial context model. To capture spatial context, and then use an RWKV-based network. Decoding blocks to access channel-level context Perform modeling;

[0029] Spatial context and channel context and prior context Cascading, in a parameter aggregation network, positionally fuses the combined context and predicts Gaussian distribution parameters. Using the predicted parameters, the latent representation Perform the following encoding:

[0030] ,

[0031] Decoding the latent representation Used for calculation or The context, iterating until the entire Encoded.

[0032] A novel image compression system based on the RWKV model, used in any of the methods described above, includes the following modules:

[0033] The multi-scale context displacement module integrates longitudinally separable convolutions with different dilation rates and 1×1 convolutions to model multi-scale information.

[0034] The local-global RWKV module consists of a spatial mixing module and a channel mixing module, which are used for feature interaction in the spatial dimension and feature fusion in the channel dimension, and structurally integrate local and global contexts.

[0035] The efficient channel attention module calculates the importance of each channel to adjust the channel weights, and performs feature filtering and enhancement at the channel level.

[0036] The RWKV spatial-channel autoregressive context entropy model predicts Gaussian distribution parameters and performs entropy encoding by leveraging the redundancy between spatial and channel data.

[0037] Beneficial effects

[0038] This invention proposes an LG-RWKV attention mechanism that jointly captures local and global contextual information relationships, enhancing local and global dependencies and thus effectively modeling the correlation between adjacent features. Furthermore, it proposes an MVC-Shift module to enhance multi-scale receptive field coverage by introducing additional convolutional layers to extract inter-channel feature information and fuse it with spatial information, resulting in more effective feature representations. In addition, it proposes an ECA channel enhancement module to strengthen cross-channel feature interaction and semantic saliency modeling by adaptively learning inter-channel dependencies while suppressing channel redundancy. This scheme overcomes the performance bottleneck of traditional image coding. Under the same bitrate conditions, the measured PSNR is significantly better than the current mainstream standards, achieving high compression ratio and high-fidelity reconstruction. It also constructs a novel end-to-end image compression framework, establishing a technical benchmark for the large-scale application of next-generation image compression technology in commercial scenarios such as streaming media and cloud storage. Attached Figure Description

[0039] Figure 1 This is a general block diagram of a novel image compression method based on the RWKV model;

[0040] Figure 2 This is a diagram of the LG-RWKV module architecture in an embodiment of the present invention;

[0041] Figure 3 This is a diagram of the space-channel hybrid module architecture in the LG-RWKV module;

[0042] Figure 4 This is a comparison diagram of the receptive fields of the MVC-Shift module in an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the high-efficiency channel attention module in an embodiment of the present invention;

[0044] Figure 6 This is a flowchart of the RWKV space-channel autoregressive context entropy model in an embodiment of the present invention. Detailed Implementation

[0045] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0046] like Figure 1 As shown, the novel image compression method based on the RWKV model of this invention has the following overall process: Given the original image Analysis and transformation Map it to a latent representation Furthermore, a hyper-prior encoder is used. To obtain super potential The formula is as follows:

[0047] ,

[0048] ,

[0049] Quantified superpotential period Using learned factorization priors for bit rate Entropy encoding is performed. At the decoding end, a super-prior decoder is first used. The initial mean and variance are obtained using the following formula:

[0050] ,

[0051] Then, the quantization operator Will Discretize into And assume It follows a conditional Gaussian distribution: Its distribution parameters are predicted by the Spatial-Channel Context (SCCTX) entropy model, and the latent representation is the bit rate. Depend on Calculation. Next, the decoder... Used to quantify potential Obtain compressed image :

[0052] ,

[0053] Based on the above process, the LG-RWKV module is used , , , To strengthen the backbone. Finally, optimize the following training loss function:

[0054] ,

[0055] in, It is the Lagrange multiplier used for the tradeoff between control rate and distortion.

[0056] The RWKV model architecture is defined by four fundamental elements inherent in the temporal and channel mixing modules: R - the receiver vector, which acts as the receiver of past information; W - the weights, representing the positional weight decay vectors, which are trainable parameters in the model; K - the key vector, which acts similarly to K in traditional attention mechanisms; and V - the value vector, which operates similarly to V in traditional attention processes. These core components interact multiplicatively at each time step.

[0057] To enable RWKV to adapt to image data input lacking causal relationships, in Vision-RWKV, the WKV operation is modified to a bidirectional form, namely Bi-WKV. The attention value of the t-th token is calculated as follows:

[0058] ,

[0059] in, Represents a bidirectional WKV sequence of key vectors. Sum value vector Given the input, calculate the first... One Token; The total number of tokens is equal to ; For traversal index; and These are two learnable vectors of dimension d, representing the spatial decay and reward of the current token, respectively. and express and The One characteristic.

[0060] like Figure 2 As shown, in this embodiment, K and V are first divided into 4×4 blocks and fed into Bi-WKV to focus local features. The results of local enhancement are then input together with K into the next Bi-WKV to focus on global features. LG-RWKV is designed based on the original Bi-RWKV and consists of a spatial mixing module and a channel mixing module, which respectively realize feature interaction in the spatial dimension and feature fusion in the channel dimension. In addition, two residual links are used to enhance features, and the enhanced features can be obtained by the following equation:

[0061] ,

[0062] ,

[0063] in This represents the input features of LGRB. Indicates the output of spatial mixing. This represents the output of LGRB. Operations representing spatial mixing, This indicates the operation of mixing channels.

[0064] like Figure 3 As shown, the spatial mixing module and the channel mixing module are specifically implemented as follows:

[0065] Spatial Hybrid Module: Designed to construct long-range dependencies between tokens across d-dimensional space. For sequences flattened into one dimension... The input features, where This represents the total number of tokens. The module initially processes this feature through layer normalization (LN) followed by a multi-scale context shift (MVC-Shift) operation, as follows:

[0066] ,

[0067] In this process, LN is used to stabilize the training process, and MVC-Shift is used to capture local context and expand the context scope of individual tokens. Subsequently, three parallel linear projection layers are used to... The processing is performed to obtain the R, K, and V matrices. The calculation logic is as follows:

[0068] ,

[0069] in, , and This represents three linear projection layers. Subsequently, local and global attention results are obtained using K and V. :

[0070] ,

[0071] Finally, the gated accept vector ∈RT×C, the attention result is modulated by element-wise multiplication. Receive probability:

[0072] .

[0073] Channel fusion module: Designed to perform feature fusion along the channel dimension. For a given input features The channel blending process treats it as spatial blending through the LN layer and the MVC shift layer:

[0074] ,

[0075] Gaining acceptance ,key Sum By and Multiplication is controlled Output of the receiving probability :

[0076]

[0077] Finally, before outputting, the ECA module is added to the channel combination to enhance channel focus and produce the final output. :

[0078] .

[0079] Token shifting is a key technique for enhancing the receptive field of RWKV: In Vision-RWKV, a Q-shift operation involving offsets in four directions is used to expand the field of view of the tokens within the image. This operation extracts the C / 4 value from the tokens above, below, to the left, and to the right of the current token and concatenates them into a new token representing past information. This operation allows the model to incorporate information from neighboring tokens. In Restore-RWKV, the Omni-Shift method is used, which simultaneously trains 1*1, 3*3, and 5*5 convolutional kernels and mixes their final weights to produce the final shifted output.

[0080] This method embodiment uses the MVC-Shift module. To capture multi-scale features, this module effectively models multi-scale information by integrating longitudinally separable convolutions with different dilation rates and 1×1 convolutions, thereby obtaining a larger receptive field while reducing computational cost during training. Figure 4 As shown, compared to the omnidirectional separable convolution design, MVC-Shift introduces an additional 1×1 convolutional layer to extract inter-channel feature information. It fuses this information with the spatial information obtained through convolution operations, thereby producing a more effective feature representation. The specific formula for this module is as follows:

[0081] ,

[0082] in, and They represent the first indivual Convolutional layers and The weight matrix of a depthwise separable convolutional layer Indicates an expansion rate The convolution operation.

[0083] The original channel blending module uses multiple linear layers to blend channel information, but it fails to consider the different importance of different channels. This invention integrates an efficient channel attention (ECA) mechanism into the traditional channel combination to enhance the model's attention to different channels. The ECA mechanism adjusts the channel weights by calculating the importance of each channel, enabling the model to more effectively regulate its attention to different channels. The implementation process of the ECA module is as follows: Figure 5 As shown, the formula is as follows:

[0084] ,

[0085] ,

[0086] in, For channel attention weights, For the Sigmoid function, For 1D convolution, For global average pooling, This is the enhanced output.

[0087] Latent representation in the transformed domain Redundancy along the spatial and channel axes is preserved. Utilizing this redundancy, the entropy model of this invention associates currently decoded symbols with previously decoded symbols to further reduce the bit rate. This embodiment proposes a RWKV spatial-channel autoregressive context entropy model (RWKV-SCCTX) to more effectively model the conditional distribution of latent variables, such as... Figure 6 As shown.

[0088] In the spatial dimension, this invention uses a chessboard approach to divide symbols into two groups: anchor points and non-anchor points. The anchor point group is first encoded, and then the non-anchor point group is encoded using the context derived from it, thereby capturing spatial relevance. For the channel dimension, channels are divided into K blocks to construct the channel environment:

[0089] ,

[0090] in, The channels representing previously decoded blocks tend to carry most of the fundamental information because the initial blocks are referenced more frequently by subsequent blocks. Allocating fewer channels to the initial blocks helps to establish a more accurate conditional distribution. Therefore, for a latent representation with M (M=320) channels... We divided it into 5 blocks. The channel allocations are {16, 16, 32, 64, M−128}.

[0091] like Figure 6 As shown, for the i-th part of the k-th block, we use chessboard mask convolution to apply the spatial context model. To capture the spatial context. If the component is used as an anchor group, the non-anchor group context is used. Then, an RWKV-based network is used. To use decoded blocks to access the channel-level context Modeling is performed.

[0092] Spatial context and channel context and prior context Cascading. In the parameter aggregation network, this combined context is fused positionally to predict Gaussian distribution parameters. A channel mixing module without full-shift layers is used to preserve a 1×1 receptive field for causal decoding. The predicted entropy parameter is used to analyze the latent representation. Perform the following encoding:

[0093] ,

[0094] Decoded code It was subsequently used for calculation. or In the context of the problem, proceed iteratively until the entire process continues. Encoded.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A novel image compression method based on the RWKV model, characterized in that, Includes the following steps: Step 1: Given the original image Analysis and transformation Map it to a latent representation ; Step 2: Represent the latent representation Input super encoder Obtain super potential ; Step 3: Using quantization operators Obtaining Quantitative Super Potential ; Step 4: Quantifying hyperpotentials based on deep learning Perform entropy encoding; Step 5: Use a super decoder at the decoding end. To obtain the initial mean and variance; Step 6: Quantify the latent representation Furthermore, the spatial-channel context entropy model is used to predict the distribution parameters, thereby obtaining the enhanced quantized latent representation. ; Step 7: Through the decoder Enhanced quantization latent representation Reconstructed into a compressed image ; The method , , , Enhancements are made through the local-global RWKV module.

2. The novel image compression method based on the RWKV model as described in claim 1, characterized in that: The local-global RWKV module is based on the RWKV model. In Vision-RWKV, the WKV operation is modified to a bidirectional form. The attention level of each token is calculated as follows: ， in, Represents a bidirectional WKV sequence of key vectors. Sum value vector Given the input, calculate the first... One Token; Indicates the total number of tokens; For traversal index; and These represent the current token's space decay and reward, respectively; and express and The One characteristic.

3. The novel image compression method based on the RWKV model as described in claim 2, characterized in that: The local-global RWKV module consists of a spatial blending module and a channel blending module. K and V are divided into 4×4 blocks and fed into BiWKV to focus local features. The results of local enhancement are then input together with K into the next BiWKV to focus on global features.

4. The novel image compression method based on the RWKV model as described in claim 3, characterized in that: The spatial mixing module performs layer normalization on the input features; then performs multi-scale context shifting; and processes the input features through three parallel linear projection layers to obtain the receiving vector, key vector, and value vector. The local-global RWKV module then obtains the local and global attention results. Finally, the local and global attention results are multiplied by the gated receiving vector to modulate the receiving probability.

5. The novel image compression method based on the RWKV model as described in claim 3, characterized in that: The channel mixing module performs layer normalization on the input features; then performs multi-scale context shifting; processes the input features through a linear projection layer to obtain the receiving vector, key vector, and value vector; by multiplying the value vector with the gated receiving vector, the receiving probability of the control value vector is obtained as an output; finally, the channel attention is enhanced through an efficient channel attention module.

6. The novel image compression method based on the RWKV model as described in claim 4 or 5, characterized in that: The specific implementation formula for the multi-scale context shift is as follows: ， in, and They represent the first indivual Convolutional layers and The weight matrix of a depthwise separable convolutional layer Indicates an expansion rate The convolution operation.

7. The novel image compression method based on the RWKV model as described in claim 5, characterized in that: The formula for the high-efficiency channel attention module is as follows: ， ， in, For channel attention weights, For the Sigmoid function, For 1D convolution, For global average pooling, This is the enhanced output.

8. The novel image compression method based on the RWKV model as described in claim 1, characterized in that: The space-channel context entropy model is specifically as follows: In the spatial dimension, a chessboard method is used to divide symbols into anchor groups and non-anchor groups. The anchor groups are encoded, and then the context derived from them is used to encode the non-anchor groups; In the channel dimension, the channel is divided into K blocks: ， in, This represents the previously decoded channel block. For the i-th part of the k-th block, a checkerboard mask convolution is used to apply the spatial context model. To capture spatial context, and then use an RWKV-based network. Decoding blocks to access channel-level context Perform modeling; Spatial context and channel context and prior context Cascading, in a parameter aggregation network, positionally fuses the combined context and predicts Gaussian distribution parameters. Using the predicted parameters, the latent representation Perform the following encoding: ， Decoding the latent representation Used for calculation or The context, iterating until the entire Encoded.

9. A novel image compression system based on the RWKV model, characterized in that, The method as described in any one of claims 1 to 8 includes the following modules: The multi-scale context displacement module integrates longitudinally separable convolutions with different dilation rates and 1×1 convolutions to model multi-scale information. The local-global RWKV module consists of a spatial mixing module and a channel mixing module, which are used for feature interaction in the spatial dimension and feature fusion in the channel dimension, and structurally integrate local and global contexts. The efficient channel attention module calculates the importance of each channel to adjust the channel weights, and performs feature filtering and enhancement at the channel level. The RWKV spatial-channel autoregressive context entropy model predicts Gaussian distribution parameters and performs entropy encoding by leveraging the redundancy between spatial and channel data.

Images