Deep learning image compression method and system based on wavelet domain double branch

Abstract

The application discloses a deep learning image compression method and system based on a wavelet domain double branch, encodes and decodes wavelet domain features of high frequency and low frequency double branches through a deep neural network, maps original image coding to compact latent features, carries out hyper-prior coding, quantization and entropy coding to generate a code stream, obtains global context information through hyper-prior decoding, divides channels of the compact latent features, uses the global context information to model a space and channel-based context for each channel block to predict a channel block mean value and a standard deviation subject to a Gaussian distribution, uses the channel block mean value and the standard deviation to guide quantization and entropy coding of the channel block, generates a code stream after image compression, maps the decoded latent features back to a reconstructed image, constructs a rate-distortion loss based on a control code rate of the decoded latent features, a control code rate of the hyper-prior decoding, and distortion of the original image and the reconstructed image, and trains the deep neural network for image compression.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image compression, and particularly relates to a deep learning image compression method and system based on a wavelet domain double-branch. BACKGROUND

[0002] With the popularity of Internet multimedia applications and the rapid development of artificial intelligence technology, image data presents the characteristics of large scale and high real-time requirement in the process of storage, transmission and processing. In order to meet the limitation of bandwidth and storage resources, an efficient image compression method becomes one of the key technologies.

[0003] Traditional image compression methods (such as JPEG, JPEG2000 and HEVC) mainly rely on artificially designed transformation and prediction mechanism, which can improve the compression performance to a certain extent, but in the low code rate scene, it is easy to produce obvious block effect, texture blur and detail loss, and it is difficult to balance the compression efficiency and reconstruction quality at the same time.

[0004] In recent years, end-to-end image compression methods based on deep learning have gradually become a research hotspot. Convolutional neural network (CNN) can efficiently extract local features, but due to the limited receptive field, it is difficult to fully model long-distance dependence, which limits the global consistency of the reconstructed image. This shows that it is difficult to achieve an ideal balance between compression rate and image quality by relying only on convolution structure, and it is urgent to combine a stronger global modeling method.

[0005] In the prior art, the Transformer structure has made remarkable achievements in natural language processing and computer vision fields due to its global modeling capability, and some studies have introduced it into the image compression task. Although the Transformer has advantages in global dependence modeling, it has high computational complexity, and it is insufficient to capture local features such as high-frequency details, and it still has deficiencies in multi-scale feature modeling and high-low frequency information separation.

[0006] Although the use of complex modules such as global attention can improve performance, it will bring high computational and memory overhead, making it difficult to apply to actual compression scenarios of high-resolution images, which restricts the practicality and deployment prospect of the method.

[0007] Natural images contain anisotropic multi-scale frequency components, and existing compression models based on convolutional neural networks or standard Transformers lack mechanisms for explicitly modeling and separating directional frequency components (such as horizontal and vertical directions), resulting in redundancy in the latent feature representation, which limits the further improvement of compression efficiency.

[0008] On the other hand, as a typical multi-scale time-frequency analysis tool, wavelet transform can decompose the image into low-frequency and high-frequency components, providing a good multi-scale representation for image compression. However, traditional wavelet-based compression methods lack the ability to adaptively model complex image content, and perform poorly in low bit rate and high fidelity scenarios.

[0009] In addition, as a core part of image compression, the performance of entropy coding directly determines the bit rate and reconstruction quality. Existing entropy models mostly rely on channel modeling or context modeling, and the probability distribution estimation of potential features in complex scenes is not accurate enough, which can easily cause bit rate redundancy and limit compression efficiency.

[0010] Therefore, how to organically combine the multi-scale analysis capability of wavelet transform with the global modeling advantage of Transformer, and introduce an efficient context-adaptive entropy model to improve the rate-distortion performance and reconstruction quality of image compression, has become a technical problem to be solved in the field. SUMMARY

[0011] To solve the problems of the prior art, improve the detail recovery capability of compressed images, reduce the computational complexity, improve the clarity, and improve the compression rate, the technical scheme adopted by the present application is as follows:

[0012] The deep learning image compression method based on wavelet domain double branch encodes, quantizes, entropy encodes and decodes the original image through a deep neural network;

[0013] The encoding and decoding process combines the high-frequency and low-frequency double-branch wavelet domain features to construct a nonlinear transformation network for multi-domain feature mapping;

[0014] The encoding process maps the original image to a highly compact latent feature, enhancing the global modeling and detail preservation capability of analysis and synthesis transformation;

[0015] The compact latent feature is hyper-prior encoded, and the obtained hyper-prior feature is quantized and entropy encoded to generate a code stream. The global context information is obtained by hyper-prior decoding, and the compact latent feature is divided into channels. The global context information is used for spatial and channel-based context modeling for each channel block to predict the mean and standard deviation of the channel block following the Gaussian distribution, which is used to guide the quantization and entropy coding of the channel block to generate the code stream after image compression.

[0016] The decoding process is based on the decoded code stream to map back to the reconstructed image through the decoded latent feature;

[0017] The deep neural network is trained based on the control bit rate of the latent features after decoding and the control bit rate of the super-prior decoding, as well as the distortion construction rate and distortion loss of the original image and the reconstructed image. The trained deep neural network is then used for image compression.

[0018] Furthermore, during the encoding process, the original image is processed through a nonlinear analysis transform network composed of alternatingly stacked wavelet Transformer blocks and convolutional modules to extract features. At the decoding end, the latent features to be decoded are mapped back to the reconstructed image through a nonlinear synthesis transform network composed of alternatingly stacked wavelet Transformer blocks and transposed convolutional modules. The wavelet Transformer block includes a layer normalization unit, a wavelet transform-based self-attention module, a batch normalization unit, and a wavelet multilayer perceptron. The input features are sequentially processed through the layer normalization unit, the wavelet transform-based self-attention module, and the batch normalization unit before being added to the original input features. The resulting attention features are sequentially processed through the layer normalization unit and the wavelet multilayer perceptron before being added to the original attention features to obtain the output features of the wavelet Transformer block. Through the above cascaded structure, the wavelet Transformer block can adaptively balance the modeling of global context and local details in the frequency domain, effectively improving feature representation capabilities and providing richer and less redundant feature representations for subsequent compression and reconstruction tasks.

[0019] Furthermore, the wavelet transform-based self-attention module is a wavelet transform-based bi-branch self-attention module. It performs multi-scale and multi-directional decomposition on the acquired features in the frequency domain and uses the self-attention mechanism to enhance low-frequency structural information and high-frequency detail information respectively. The wavelet transform-based bi-branch self-attention module has stronger adaptability and flexibility in high-frequency and low-frequency feature modeling, thereby alleviating the problem of excessive accumulation of low-frequency components and insufficient high-frequency details.

[0020] Furthermore, the wavelet transform-based dual-branch self-attention module performs wavelet transform on the acquired features, decomposing low-frequency and high-frequency components into low-frequency and high-frequency branches respectively. The low-frequency branch introduces a low-frequency multi-head self-attention mechanism, calculating low-frequency attention features using the dimensions of the low-frequency query vector, low-frequency key vector, and low-frequency key vector. Similarly, the high-frequency branch introduces a high-frequency multi-head self-attention mechanism, calculating high-frequency attention features using the dimensions of the high-frequency query vector, high-frequency key vector, and high-frequency key vector. The low-frequency and high-frequency attention features are then concatenated dimensionally to obtain the fused features, which serve as the output of the wavelet transform-based dual-branch self-attention module. This allows the wavelet Transformer block to adaptively enhance low-frequency and high-frequency features, effectively compensating for detailed features while ensuring global information modeling, thereby improving the feature representation capability and reconstruction quality of the compressed network.

[0021] Furthermore, the compact latent features are mapped through a super-prior encoder via a super-analysis transform network to obtain super-prior features. These super-prior features are then quantized and entropy-encoded to generate a bitstream. The corresponding super-prior features are recovered through a super-prior decoder. The recovered super-prior features are then mapped through a super-synthetic transform network to obtain global context information, which is used to assist in channel probability modeling. The super-analysis transform network and the super-synthetic transform network have symmetrical structures. The super-analysis transform network includes multiple alternately stacked downsampling layers and convolutional layers, and the super-synthetic transform network includes multiple alternately stacked upsampling layers and convolutional layers. Activation functions are used to introduce nonlinearity between layers.

[0022] Furthermore, to further improve the accuracy of channel probability modeling, this invention introduces a channel context model based on wavelet domain Transformer. The decoded channel blocks and the global context information are modeled through multi-stage wavelet Transformer blocks. Each stage of the wavelet Transformer block includes a dual-branch self-attention module based on wavelet transform. Low-frequency and high-frequency components are modeled using low-frequency and high-frequency multi-head self-attention mechanisms respectively, to remove redundant local and global information. The resulting context features are then enhanced through a windowed channel attention module using window partitioning and channel weighting mechanisms. Inter-channel correlation is used to accurately predict the mean and standard deviation of the current channel block to be encoded, which guides the quantization and entropy coding of the channel block, thereby significantly improving compression performance. The super-prior features can provide global mean and standard deviation prior information, thus providing effective auxiliary constraints for the probability distribution modeling of the channel block. By modeling the global prior of latent features, the accuracy of the channel entropy model in measuring the mean and standard deviation is effectively improved. Through the synergy of the super-prior path and the context modeling path, the global statistical dependence and local channel context dependence of latent features are jointly modeled, which significantly improves the accuracy of probability estimation and establishes an accurate probability distribution model for the quantized latent features to minimize the coding bit rate.

[0023] Furthermore, the window channel attention module utilizes window partitioning to model attention in local regions and, combined with a channel attention mechanism, effectively captures high-frequency detail features in complex regions, enhancing feature representation capabilities during compression. The window channel attention module includes a window attention block, a residual block, a convolutional layer, and a depthwise separable multilayer perceptron. The window attention block divides the contextual features into multiple non-overlapping windows. For any element of a feature within a window, the window's output feature is obtained by a weighted sum of all elements within the window, thus achieving fine-grained dependency modeling within a local range. This significantly reduces computational complexity while maintaining the global receptive field. The window output feature passes through a residual block to enhance the nonlinear representation capability of deep features. After effectively mitigating the gradient vanishing problem in deep networks, convolutional layers compress and reorganize the channel dimensions to further eliminate redundant information between channels. Then, a depth-separable multilayer perceptron is used to jointly model the spatial and channel dimensions in a depth-separable manner, improving the model's feature selection capability. The obtained channel attention results are then fused with the context features through element-wise multiplication and residual addition to complete the modeling and feature enhancement of the channel context, which is used to predict the mean and standard deviation of the channel block to be encoded. The window channel attention module not only effectively highlights the feature expression of complex regions (such as texture details and edges), but also reduces redundant bit allocation in low-contrast regions, thereby improving the rate-distortion performance of the overall image compression.

[0024] Furthermore, the window attention block employs a standard query-key-value attention mechanism and computes self-attention within a local window to efficiently model spatial-channel context dependencies. It obtains the attention weight of an element by performing a query linear transformation on a specific element within the window, and obtains the attention weights of other elements by performing a key linear transformation on the keys of other elements within the window. A Gaussian embedding function is constructed based on these attention weights. The similarity between elements is calculated using the dot product of the query and the key. Then, the similarity is summed using the Gaussian embedding function constructed based on a specific element and all other elements within the window to generate a normalization factor. Finally, the other elements... After mapping the prime values ​​to a value vector, multiply it by the output of the corresponding Gaussian embedding function. Summing the products of all other elements within the window, the ratio of the sum of the products to the normalization factor is used as the weighted summation of the attention output features. The scale and distribution of the attention output features are adjusted by a learnable projection weight matrix, and then added to a certain element to obtain the output of the window attention block. Through the above design, the window attention block can effectively capture long-range dependencies in local regions, enhance the model's ability to model the detailed structure of the image, and avoid the high computational overhead of the global self-attention mechanism, making it suitable for high-resolution image compression tasks.

[0025] Furthermore, the rate-distortion loss employs a rate-distortion loss function, obtaining the conditional probability distribution of the decoded latent representation under the given recovered super-prior features, and the prior probability distribution of the recovered super-prior features themselves. The expected value of each probability distribution is calculated using a base-2 exponent to obtain the control code rate of the decoded latent features and the control code rate of the super-prior decoding. The Lagrange multiplier, used as a parameter to balance the control code rate and distortion, is multiplied by the distortion of the original image and the reconstructed image. The product is then added to the control code rate of the decoded latent features and the control code rate of the super-prior decoding to obtain the rate-distortion loss function.

[0026] The deep learning image compression system based on wavelet domain dual-branch includes a master encoder, a quantizer, an entropy encoder, and a master decoder. According to the deep learning image compression method based on wavelet domain dual-branch, a deep neural network is constructed and trained to encode, quantize, entropy encode, and decode the original image, and then the image is compressed based on the trained deep neural network.

[0027] The advantages and beneficial effects of this invention are as follows:

[0028] This invention achieves a balance between low-frequency and high-frequency modeling based on a wavelet domain dual-branch Transformer, avoiding the loss of high-frequency information caused by traditional downsampling and enhancing the ability to recover details. The window-based attention mechanism of this invention effectively reduces the complexity of attention calculation and improves the fineness of bit allocation, making the reconstruction of texture and edge regions clearer. The channel entropy model of this invention improves the accuracy of probability estimation, reduces redundant bits, and improves the compression ratio by jointly using channel dependency and super-prior modeling. Attached Figure Description

[0029] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0030] Figure 2 This is a diagram of a deep learning image compression framework based on a wavelet domain dual-branch Transformer in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of the wavelet Transformer block in an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of the structure of the dual-branch self-attention module (DWAM) of wavelet transform in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of the channel-based entropy model structure in an embodiment of the present invention.

[0034] Figure 6 This is a structural diagram of the advanced prior codec in an embodiment of the present invention.

[0035] Figure 7 This is a schematic diagram of the channel context model structure based on wavelet domain Transformer in an embodiment of the present invention.

[0036] Figure 8 This is a schematic diagram of the window channel attention module (WBCAM) in an embodiment of the present invention.

[0037] Figure 9 This is a schematic diagram of the window attention block in an embodiment of the present invention. Detailed Implementation

[0038] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0039] Current deep learning-based image compression methods generally suffer from problems such as limited receptive fields of convolutional neural networks, lack of explicit modeling and separation of multi-scale frequency components of images, and inaccurate estimation of latent feature distributions.

[0040] like Figure 1 As shown, this invention proposes a deep learning-based image compression method based on a two-branch wavelet domain, comprising the following steps:

[0041] Step 1: Constructing an end-to-end image compression system based on deep learning; encoding, quantizing, entropy encoding, and decoding the original image using a deep neural network. During the encoding process, features are extracted from the original image to map it to a highly compact latent representation. The latent representation is then divided into channels, and each channel is modeled as a Gaussian distribution with mean and standard deviation based on joint predictions of spatial and channel context models. After quantization and entropy encoding of the channel blocks guided by the mean and standard deviation, a compressed bitstream is generated. The bitstream is then decoded, and a loss is constructed between the decoded and reconstructed image and the original image to train the deep neural network.

[0042] End-to-end image compression overcomes the limitations of traditional step-by-step optimization by jointly optimizing the master encoder, quantizer, entropy model, and master decoder through deep neural networks.

[0043] Step 2: For image encoding and decoding, a nonlinear transform network is constructed by combining wavelet domain features to perform multi-domain feature mapping; such as... Figure 2 As shown, the main encoder and main decoder of this invention combine wavelet domain feature decomposition to optimize the input representation, and use a two-branch Transformer structure to construct a nonlinear transformation network, where H and W represent the height and width of the feature map.

[0044] At the encoding end, the original image First, a nonlinear analysis and transformation network is used. This network extracts features by stacking three wavelet Transformer blocks (containing layer normalization, a two-branch wavelet attention module, and a wavelet multilayer perceptron) with a convolutional module, mapping the image to a highly compact latent representation. This enhances the global modeling and detail preservation capabilities of analysis and synthetic transformations. The above process can be formally expressed as:

[0045]

[0046] At the decoding end, the latent representation via a nonlinear synthetic transformation network The network maps back to the reconstructed image by stacking three wavelet Transformer blocks (containing layer normalization, a two-branch wavelet attention module, and a wavelet multilayer perceptron) and transposed convolutional modules. The above process can be formally expressed as:

[0047]

[0048] To more effectively enhance high-frequency and low-frequency features after multiple downsampling, this invention proposes a Wavelet Transformer Block (WTB). This module employs a dual-branch Wavelet Attention Mechanism (DWAM) based on wavelet transform, which has stronger adaptability and flexibility in high-frequency and low-frequency feature modeling, thereby alleviating the problems of excessive accumulation of low-frequency components and insufficient high-frequency details.

[0049] Specifically, such as Figure 3 As shown, given the input feature map The calculation process for WTB is as follows:

[0050]

[0051]

[0052]

[0053] Here, LN stands for Layer Normalization, BN stands for Batch Normalization, and MLP stands for Multi-Layer Perceptron. Through this cascaded structure, the WTB module can adaptively balance the modeling of global context and local details in the frequency domain, effectively improving feature representation capabilities and providing richer and less redundant feature representations for subsequent compression and reconstruction tasks.

[0054] The Dual-branch Wavelet Attention Mechanism (DWAM) is a core component of the Wavelet Transformer block (WTB). It decomposes input features in the frequency domain at multiple scales and directions, and utilizes a self-attention mechanism to enhance low-frequency structural information and high-frequency detail information respectively. For example... Figure 4 As shown, DWAM is calculated as follows:

[0055] First, the input features Perform wavelet transform Decompose it into low-frequency components With high frequency components :

[0056]

[0057] Among them, the low-frequency branch is mainly responsible for modeling the global structure of the image, while the high-frequency branch is used to preserve details such as texture and edges, thereby enhancing detail reconstruction while ensuring structural consistency.

[0058] In the low-frequency branch, a low-frequency multi-head self-attention (L-MSA) mechanism is introduced, whose attention weights are determined by the low-frequency query vector. With key vector The calculation yielded:

[0059]

[0060] In the high-frequency branch, a high-frequency multi-head self-attention (H-MSA) mechanism is introduced, whose attention weights are determined by the high-frequency query vector. With key vector The calculation yielded:

[0061]

[0062] in, The dimension of the key vector. Parameters The proportion of heads allocated to frequency domain attention is set in this invention. =0.9, to enhance the processing of low-frequency components.

[0063] Ultimately, WTB will output the low-frequency branch. With the output of the high-frequency branch By concatenating along the feature dimensions, we obtain the fused feature representation:

[0064]

[0065] Through the above design, WTB can adaptively enhance low-frequency and high-frequency features, effectively compensating for detailed features while ensuring global information modeling, thereby improving the feature representation ability and reconstruction quality of the compressed network.

[0066] Step 3: Entropy coding adopts a channel-based entropy model to establish an accurate probability distribution model for the quantized latent representation in order to minimize the coding bit rate. This model is macroscopically composed of a super-prior path and a context modeling path. By jointly modeling the global statistical dependence and local channel context dependence of latent features, it significantly improves the accuracy of probability estimation.

[0067] like Figure 5 As shown, in order to make full use of the statistical information of latent features, this invention proposes a hyperprior encoder-decoder for the channel entropy model. By modeling the latent features globally in advance, the accuracy of the channel entropy model in measuring the mean and standard deviation is effectively improved.

[0068] Specifically, at the super-prior encoder, the input latent features are first... via hyperanalysis transform network Mapping yields the super-prior features. :

[0069]

[0070] in, For hyperanalysis transformation networks, such as Figure 6 As shown, its structure consists of multiple convolutional layers (Conv3×3) and downsampling layers (TConv3×3 ↓ 2) stacked alternately, with the ReLU activation function used between layers to introduce nonlinearity.

[0071] Subsequently, the prior features Quantization and entropy coding are performed to generate the bitstream, and the corresponding prior is recovered at the prior decoder. .

[0072] At the advanced prior decoder, a super-synthetic transform network is employed. Decoded super-prior Upsampling and mapping are performed to obtain features used for auxiliary channel probability modeling. :

[0073]

[0074] in, Structurally and It is basically symmetrical and consists of multiple upsampling layers (TConv3×3 ↑ 2) and convolutional layers (Conv3×3) stacked alternately, and also uses the ReLU activation function.

[0075] Through this process, the prior features can provide global mean and standard deviation prior information, thus providing effective auxiliary constraints for modeling the probability distribution of channel blocks.

[0076] To further improve the accuracy of channel probability modeling, this invention introduces a channel context model based on wavelet domain Transformer. (Wavelet Transformer-based Channel ContextModel), such as Figure 7 As shown, this model accurately predicts the current group of channels to be encoded by modeling the decoded channel blocks and their context. mean With standard deviation parameter .

[0077] Specifically, the channel block to be decoded is denoted as Decoded channel group To compare it with the priori Global context information obtained from decoding The inputs are fed into a four-stage wavelet transformer block (WTB) to perform low-frequency and high-frequency feature modeling, respectively.

[0078] Each WTB stage includes a wavelet-based dual-branch self-attention mechanism (DWAM), which models low-frequency and high-frequency components using low-frequency multi-head self-attention (L-MSA) and high-frequency multi-head self-attention (H-MSA) respectively to remove local and global redundant information. The number of channels for the four WTBs is set to... and between adjacent WTBs Convolutions adjust the number of channels to maintain feature matching.

[0079] Subsequently, the contextual features output by WTB are input into the Windowed Channel Attention Module (WBCAM), which further enhances the correlation between channels using window partitioning and channel weighting mechanisms. Finally, based on the prior features and contextual modeling results, channel blocks are processed. mean with standard deviation Conduct joint forecasting:

[0080]

[0081] in, This represents the context modeling function implemented through the combination of WTB and WBCAM. The predicted... , This will be used to guide the quantization and entropy coding of channel blocks, thereby significantly improving compression performance.

[0082] Furthermore, the Window Channel Attention Module (WBCAM) such as Figure 8 As shown, it consists of a Window-based Attention Block, a Residual Block, a Convolutional Layer (Conv 1×1), and a Depth Separable Multilayer Perceptron (DWMLP). This module aims to model attention in local regions by using window partitioning and combining it with a channel attention mechanism to effectively capture high-frequency detail features in complex regions and improve feature representation capabilities during compression.

[0083] Specifically, input features First, the window is divided and attention-weighted through a window attention block. Features within each window undergo self-attention computation, thus achieving fine-grained dependency modeling within a local scope. The output of this process is... It can be represented as:

[0084]

[0085] Next, output features The feature passes through three cascaded residual blocks to enhance the nonlinear representation of deep features and effectively alleviate the gradient vanishing problem in deep networks.

[0086] Subsequently, after Convolutional layers compress and reorganize the channel dimension, further eliminating redundant information between channels. This is then input into a depthwise separable multilayer perceptron, which performs joint modeling in both spatial and channel dimensions through depthwise separability, enhancing the model's feature selection capability.

[0087] Finally, the channel attention results output by WBCAM are fused with the original input through element-wise multiplication and residual addition to complete the channel context modeling and feature enhancement. This module can not only effectively highlight the feature representation of complex regions (such as texture details and edges), but also reduce redundant bit allocation in low-contrast regions, thereby improving the rate-distortion performance of overall image compression.

[0088] Furthermore, the Window Attention Block (WBCAM) is a core component of the Window-Channel Attention Module. It employs a standard query-key-value attention mechanism and computes self-attention within local windows to efficiently model spatial-channel context dependencies. This module first divides the input feature map into multiple non-overlapping local windows, and independently computes attention weights within each window, thereby significantly reducing computational complexity while maintaining the global receptive field.

[0089] The core calculation process is as follows: Given input features First, divide it into A non-overlapping window, denoted as . For the first Any element within a window Its output From all elements within the window The weighted summation is obtained, and the calculation formula is as follows:

[0090]

[0091]

[0092]

[0093]

[0094] in, and They represent the first The first window The and the first One element, and Through Convolution implements a linear transformation of the query and key, which is used to generate attention weights. It is a linear mapping function. pass Convolution maps input features to a value vector. This is a Gaussian embedding function that calculates the similarity between elements by the dot product of the query and the key. This is a normalization factor used for stable numerical calculations.

[0095] likeFigure 9 As shown, input features Simultaneously through three independent Convolutional layer ( , and The system generates queries, keys, and values. Then, it calculates the similarity between the queries and keys within a window and obtains the attention output through a weighted summation. Subsequently, the window attention output is fused with the original input features using a residual connection mechanism to obtain the module's final output:

[0096]

[0097] in, For learnable projective weight matrix (through (Convolutional implementation) is used to adjust the scale and distribution of attention output features.

[0098] Through the above design, the window attention block can effectively capture long-range dependencies in local regions, enhance the model's ability to model the detailed structure of images, and avoid the high computational overhead of the global self-attention mechanism, making it suitable for high-resolution image compression tasks.

[0099] To optimize the entire model, this invention employs a Lagrange rate-distortion optimization framework, constructing a rate-distortion loss function for end-to-end training:

[0100]

[0101] Where λ is a Lagrange multiplier, used to control the code rate ( ) and distortion ( The key parameter for weighing between ) and They represent the quantized latent representations respectively. With prior The estimated bit rate, Used to measure the original input image With reconstructed images The distortion between components is typically measured using mean squared error (MSE) or multi-scale structural similarity (MS-SSIM) as a distortion metric. This represents the mathematical expectation, which is approximated during training by averaging the batch of samples. Indicates that given a priori Under what conditions is the latent representation The conditional probability distribution, Indicates prior knowledge Its own prior probability distribution.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based image compression method based on a wavelet domain bi-branch architecture, characterized by: The original image is encoded, quantized, entropy encoded, and decoded using a deep neural network. The encoding and decoding process combines wavelet domain features from both high-frequency and low-frequency branches to construct a nonlinear transform network for multi-domain feature mapping. The encoding process maps the original image to compact latent features; The compact latent features are encoded using a priori methods, and then the obtained priori features are quantized and entropy encoded to generate a bitstream. Global context information is obtained through priori decoding, and the compact latent features are divided into channels. The global context information is used to perform spatial and channel-based context modeling for each channel block to predict the mean and standard deviation of the channel block that follow a Gaussian distribution. This is used to guide the quantization and entropy encoding of the channel block and generate the image compressed bitstream. During the encoding process, the original image is processed through a nonlinear analysis transform network composed of alternatingly stacked wavelet blocks and convolutional modules to extract features. At the decoding end, the latent features to be decoded are mapped back to the reconstructed image through a nonlinear synthesis transform network composed of alternatingly stacked wavelet blocks and transposed convolutional modules. The wavelet blocks include self-attention modules based on wavelet transform. The self-attention modules based on wavelet transform are bi-branch self-attention modules based on wavelet transform. Each stage of the wavelet block contains a bi-branch self-attention module based on wavelet transform. Through low-frequency multi-head self-attention mechanism and high-frequency multi-head self-attention mechanism, low-frequency components and high-frequency components are modeled respectively. The obtained context features are processed through a window channel attention module. The window partitioning and channel weighting mechanism are used to enhance the correlation between channels to predict the mean and standard deviation of the current channel block to be encoded, which is used to guide the quantization and entropy encoding of the channel block. The decoding process is based on the compressed bitstream, and the decoded latent features are mapped back to the reconstructed image. The deep neural network is trained based on the control bit rate of the latent features after decoding and the control bit rate of the super-prior decoding, as well as the distortion construction rate and distortion loss of the original image and the reconstructed image. The trained deep neural network is then used for image compression.

2. The deep learning image compression method based on wavelet domain bi-branch as described in claim 1, characterized in that: The wavelet block further includes a layer normalization unit, a batch normalization unit, and a wavelet multilayer perceptron. The input features are sequentially passed through the layer normalization unit, the wavelet transform-based self-attention module, and the batch normalization unit before being added to the original input features. The resulting attention features are sequentially passed through the layer normalization unit and the wavelet multilayer perceptron before being added to the original attention features to obtain the output features of the wavelet block.

3. The deep learning image compression method based on wavelet domain bi-branch as described in claim 2, characterized in that: The wavelet transform-based dual-branch self-attention module decomposes the acquired features in the frequency domain in multiple scales and directions, and uses the self-attention mechanism to enhance low-frequency structural information and high-frequency detail information respectively.

4. The deep learning image compression method based on wavelet domain bi-branch as described in claim 3, characterized in that: The wavelet transform-based dual-branch self-attention module performs wavelet transform on the acquired features, and the decomposed low-frequency and high-frequency components are respectively fed into the low-frequency branch and the high-frequency branch. The low-frequency branch introduces a low-frequency multi-head self-attention mechanism, which calculates low-frequency attention features by using the low-frequency query vector, the low-frequency key vector, and the dimension of the low-frequency key vector. The high-frequency branch introduces a high-frequency multi-head self-attention mechanism, which calculates high-frequency attention features by using the high-frequency query vector, the high-frequency key vector, and the dimension of the high-frequency key vector. The low-frequency attention features and the high-frequency attention features are then concatenated according to their dimensions to obtain the fused features, which are used as the output of the wavelet transform-based dual-branch self-attention module.

5. The deep learning image compression method based on wavelet domain bi-branch as described in claim 1, characterized in that: The compact latent features are mapped through a super-analysis transform network by a super-prior encoder to obtain super-prior features. The super-prior features are then quantized and entropy encoded to generate a bitstream. The corresponding super-prior features are recovered by a super-prior decoder. The recovered super-prior features are then mapped through a super-synthesis transform network to obtain global context information, which is used to assist in channel probability modeling.

6. The deep learning image compression method based on wavelet domain bi-branch as described in claim 5, characterized in that: A wavelet-domain-based channel context model is introduced to model the decoded channel blocks and the global context information through multi-stage wavelet blocks.

7. The deep learning image compression method based on wavelet domain bi-branch as described in claim 6, characterized in that: The window channel attention module uses window partitioning to model attention in local regions and combines it with channel attention mechanisms to capture high-frequency detail features in complex regions. The window channel attention module includes a window attention block, a residual block, a convolutional layer, and a depth-separable multilayer perceptron. The window attention block divides the context features into multiple non-overlapping windows. For any element of the feature within a window, the output feature of the window is obtained by weighted summation of all elements within the window. After the window output feature passes through the residual block, the channel dimension is compressed and recombined by the convolutional layer. Then, the depth-separable multilayer perceptron performs joint modeling in the spatial and channel dimensions in a depth-separable manner. The obtained channel attention result is then fused with the context features by element-wise multiplication and residual addition to predict the mean and standard deviation of the channel block to be encoded.

8. The deep learning image compression method based on wavelet domain bi-branch as described in claim 7, characterized in that: The window attention block employs a query-key-value attention mechanism and calculates self-attention within a local window. It obtains the attention weight of an element by performing a query linear transformation on a specific element within the window, and obtains the attention weights of other elements by performing a key linear transformation on the keys of other elements within the window. A Gaussian embedding function is constructed based on these attention weights. The similarity between elements is calculated by the dot product of the query and the key. Then, the similarity of the Gaussian embedding function constructed based on the specific element and all other elements within the window is summed to generate a normalization factor. Other elements are then mapped to value vectors and multiplied by the output of the corresponding Gaussian embedding function. The sum of these products is then calculated based on the sum of all other elements within the window. The ratio of the sum of these products to the normalization factor is used as the weighted summation of the attention output feature. The scale and distribution of the attention output feature are adjusted using a learnable projection weight matrix, and then added to the specific element to obtain the output of the window attention block.

9. The deep learning image compression method based on wavelet domain bi-branch as described in claim 1, characterized in that: The rate-distortion loss employs a rate-distortion loss function, which obtains the conditional probability distribution of the decoded latent representation under the given recovered super-prior features, as well as the prior probability distribution of the recovered super-prior features themselves. The expected value of each probability distribution is calculated using a base-2 exponent to obtain the control code rate of the decoded latent features and the control code rate of the super-prior decoding. The Lagrange multiplier, used as a parameter to balance the control code rate and distortion, is multiplied by the distortion of the original and reconstructed images. This product is then added to the control code rate of the decoded latent features and the control code rate of the super-prior decoding to obtain the rate-distortion loss function.

10. A deep learning image compression system based on a wavelet domain dual-branch architecture, comprising a master encoder, a quantizer, an entropy encoder, and a master decoder, characterized in that: According to any one of claims 1 to 9, a deep learning image compression method based on wavelet domain bi-branch is constructed and trained to encode, quantize, entropy encode and decode the original image, and then the image is compressed based on the trained deep neural network.

Images