Deep learning image compression method and system based on wavelet domain double branches

Abstract

The invention discloses a deep learning image compression method and system based on wavelet domain double branches, and the method comprises the steps: carrying out the coding and decoding through a deep neural network in combination with the wavelet domain features of high-frequency and low-frequency double branches, carrying out the coding and mapping of an original image to a compact potential feature, carrying out the super-prior coding, quantization and entropy coding, so as to generate a code stream, and carrying out the compression of a deep learning image. Global context information is obtained through hyper-prior decoding, channel division is performed on the compact potential features, and context modeling based on space and channels is performed on each channel block by using the global context information so as to predict a mean value and a standard deviation of the channel blocks obeying Gaussian distribution, and the mean value and the standard deviation are used for guiding quantization and entropy coding of the channel blocks; and generating a code stream after image compression, mapping the potential features obtained by decoding back to the reconstructed image, and constructing rate distortion loss based on the control code rate of the potential features after decoding, the control code rate of super-prior decoding and the distortion of the original image and the reconstructed image so as to train a deep neural network for image compression.

Description

Technical Field

[0001] This invention belongs to the field of image compression, specifically relating to a deep learning-based image compression method and system based on a wavelet domain bi-branch. Background Technology

[0002] With the widespread adoption of internet multimedia applications and the rapid development of artificial intelligence technology, image data is characterized by its massive scale and high real-time requirements during storage, transmission, and processing. To meet the limitations of bandwidth and storage resources, efficient image compression methods have become a key technology.

[0003] Traditional image compression methods (such as JPEG, JPEG2000 and HEVC) mainly rely on manually designed transformation and prediction mechanisms. Although they improve compression performance to some extent, they often produce obvious block artifacts, texture blurring and loss of detail in low bit rate scenarios, making it difficult to balance compression efficiency and reconstruction quality at the same time.

[0004] In recent years, end-to-end image compression methods based on deep learning have gradually become a research hotspot. Convolutional neural networks (CNNs) can efficiently extract local features, but due to their limited receptive field, they struggle to fully model long-range dependencies, thus limiting the global consistency of reconstructed images. This indicates that relying solely on convolutional structures is insufficient to achieve an ideal balance between compression ratio and image quality, necessitating the combination of more robust global modeling methods.

[0005] In existing technologies, the Transformer architecture has achieved significant results in natural language processing and computer vision due to its global modeling capabilities, and some studies have also introduced it into image compression tasks. Although the Transformer has advantages in global dependency modeling, it has high computational complexity, is insufficient in capturing local features such as high-frequency details, and still has shortcomings in multi-scale feature modeling and separation of high- and low-frequency information.

[0006] While employing complex modules such as global attention can improve performance, it also incurs high computational and memory overhead, making it difficult to apply to real-world compression scenarios for high-resolution images and limiting the practicality and deployment prospects of the method.

[0007] Natural images contain anisotropic multi-scale frequency components. However, existing compression models based on convolutional neural networks or standard Transformers lack mechanisms for explicitly modeling and separating directional frequency components (such as horizontal and vertical directions), resulting in redundancy in latent feature representations and limiting further improvements in compression efficiency.

[0008] On the other hand, wavelet transform, as a typical multi-scale time-frequency analysis tool, can decompose images into low-frequency and high-frequency components, providing a good multi-scale representation for image compression. However, traditional wavelet-based compression methods lack the ability to adaptively model complex image content and have limited performance in low bitrate and high-fidelity scenarios.

[0009] Furthermore, as a core component of image compression, entropy coding directly determines the bit rate and reconstruction quality. Most existing entropy models rely on channel modeling or context modeling, which is insufficiently accurate in estimating the probability distribution of latent features in complex scenes, easily leading to bit rate redundancy and limiting compression efficiency.

[0010] Therefore, how to organically combine the multi-scale analysis capabilities of wavelet transform with the global modeling advantages of Transformer, and introduce an efficient context-adaptive entropy model to improve the rate-distortion performance and reconstruction quality of image compression, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0011] To address the shortcomings of existing technologies and achieve the goals of improving the detail recovery capability of compressed images, reducing computational complexity, enhancing clarity, and increasing compression ratio, this invention adopts the following technical solution:

[0012] A deep learning-based image compression method based on a two-branch wavelet domain uses a deep neural network to encode, quantize, entropy encode, and decode the original image.

[0013] The encoding and decoding process combines wavelet domain features from both high-frequency and low-frequency branches to construct a nonlinear transform network for multi-domain feature mapping.

[0014] The encoding process maps the original image to a highly compact latent feature, enhancing the global modeling and detail preservation capabilities of analysis and synthetic transformation;

[0015] The compact latent features are encoded using a priori methods, and then the obtained priori features are quantized and entropy encoded to generate a bitstream. Global context information is obtained through priori decoding, and the compact latent features are divided into channels. The global context information is used to perform spatial and channel-based context modeling for each channel block to predict the mean and standard deviation of the channel block that follow a Gaussian distribution. This is used to guide the quantization and entropy encoding of the channel block and generate the image compressed bitstream.

[0016] The decoding process is based on the compressed bitstream, and the decoded latent features are mapped back to the reconstructed image.

[0017] The deep neural network is trained based on the control bit rate of the latent features after decoding and the control bit rate of the super-prior decoding, as well as the distortion construction rate and distortion loss of the original image and the reconstructed image. The trained deep neural network is then used for image compression.

[0018] Furthermore, during the encoding process, the original image is processed through a nonlinear analysis transform network composed of alternatingly stacked wavelet Transformer blocks and convolutional modules to extract features. At the decoding end, the latent features to be decoded are mapped back to the reconstructed image through a nonlinear synthesis transform network composed of alternatingly stacked wavelet Transformer blocks and transposed convolutional modules. The wavelet Transformer block includes a layer normalization unit, a wavelet transform-based self-attention module, a batch normalization unit, and a wavelet multilayer perceptron. The input features are sequentially processed through the layer normalization unit, the wavelet transform-based self-attention module, and the batch normalization unit before being added to the original input features. The resulting attention features are sequentially processed through the layer normalization unit and the wavelet multilayer perceptron before being added to the original attention features to obtain the output features of the wavelet Transformer block. Through the above cascaded structure, the wavelet Transformer block can adaptively balance the modeling of global context and local details in the frequency domain, effectively improving feature representation capabilities and providing richer and less redundant feature representations for subsequent compression and reconstruction tasks.

[0019] Furthermore, the wavelet transform-based self-attention module is a wavelet transform-based bi-branch self-attention module. It performs multi-scale and multi-directional decomposition on the acquired features in the frequency domain and uses the self-attention mechanism to enhance low-frequency structural information and high-frequency detail information respectively. The wavelet transform-based bi-branch self-attention module has stronger adaptability and flexibility in high-frequency and low-frequency feature modeling, thereby alleviating the problem of excessive accumulation of low-frequency components and insufficient high-frequency details.

[0020] Furthermore, the wavelet transform-based dual-branch self-attention module performs wavelet transform on the acquired features, decomposing low-frequency and high-frequency components into low-frequency and high-frequency branches respectively. The low-frequency branch introduces a low-frequency multi-head self-attention mechanism, calculating low-frequency attention features using the dimensions of the low-frequency query vector, low-frequency key vector, and low-frequency key vector. Similarly, the high-frequency branch introduces a high-frequency multi-head self-attention mechanism, calculating high-frequency attention features using the dimensions of the high-frequency query vector, high-frequency key vector, and high-frequency key vector. The low-frequency and high-frequency attention features are then concatenated dimensionally to obtain the fused features, which serve as the output of the wavelet transform-based dual-branch self-attention module. This allows the wavelet Transformer block to adaptively enhance low-frequency and high-frequency features, effectively compensating for detailed features while ensuring global information modeling, thereby improving the feature representation capability and reconstruction quality of the compressed network.

[0021] Furthermore, the compact latent features are mapped through a super-prior encoder via a super-analysis transform network to obtain super-prior features. These super-prior features are then quantized and entropy-encoded to generate a bitstream. The corresponding super-prior features are recovered through a super-prior decoder. The recovered super-prior features are then mapped through a super-synthetic transform network to obtain global context information, which is used to assist in channel probability modeling. The super-analysis transform network and the super-synthetic transform network have symmetrical structures. The super-analysis transform network includes multiple alternately stacked downsampling layers and convolutional layers, and the super-synthetic transform network includes multiple alternately stacked upsampling layers and convolutional layers. Activation functions are used to introduce nonlinearity between layers.

[0022] Furthermore, to further improve the accuracy of channel probability modeling, this invention introduces a channel context model based on wavelet domain Transformer. The decoded channel blocks and the global context information are modeled through multi-stage wavelet Transformer blocks. Each stage of the wavelet Transformer block includes a dual-branch self-attention module based on wavelet transform. Low-frequency and high-frequency components are modeled using low-frequency and high-frequency multi-head self-attention mechanisms respectively, to remove redundant local and global information. The resulting context features are then enhanced through a windowed channel attention module using window partitioning and channel weighting mechanisms. Inter-channel correlation is used to accurately predict the mean and standard deviation of the current channel block to be encoded, which guides the quantization and entropy coding of the channel block, thereby significantly improving compression performance. The super-prior features can provide global mean and standard deviation prior information, thus providing effective auxiliary constraints for the probability distribution modeling of the channel block. By modeling the global prior of latent features, the accuracy of the channel entropy model in measuring the mean and standard deviation is effectively improved. Through the synergy of the super-prior path and the context modeling path, the global statistical dependence and local channel context dependence of latent features are jointly modeled, which significantly improves the accuracy of probability estimation and establishes an accurate probability distribution model for the quantized latent features to minimize the coding bit rate.

[0023] Furthermore, the window channel attention module utilizes window partitioning to model attention in local regions and, combined with a channel attention mechanism, effectively captures high-frequency detail features in complex regions, enhancing feature representation capabilities during compression. The window channel attention module includes a window attention block, a residual block, a convolutional layer, and a depthwise separable multilayer perceptron. The window attention block divides the contextual features into multiple non-overlapping windows. For any element of a feature within a window, the window's output feature is obtained by a weighted sum of all elements within the window, thus achieving fine-grained dependency modeling within a local range. This significantly reduces computational complexity while maintaining the global receptive field. The window output feature passes through a residual block to enhance the nonlinear representation capability of deep features. After effectively mitigating the gradient vanishing problem in deep networks, convolutional layers compress and reorganize the channel dimensions to further eliminate redundant information between channels. Then, a depth-separable multilayer perceptron is used to jointly model the spatial and channel dimensions in a depth-separable manner, improving the model's feature selection capability. The obtained channel attention results are then fused with the context features through element-wise multiplication and residual addition to complete the modeling and feature enhancement of the channel context, which is used to predict the mean and standard deviation of the channel block to be encoded. The window channel attention module not only effectively highlights the feature expression of complex regions (such as texture details and edges), but also reduces redundant bit allocation in low-contrast regions, thereby improving the rate-distortion performance of the overall image compression.

[0024] Furthermore, the window attention block employs a standard query-key-value attention mechanism and computes self-attention within a local window to efficiently model spatial-channel context dependencies. It obtains the attention weight of an element by performing a query linear transformation on a specific element within the window, and obtains the attention weights of other elements by performing a key linear transformation on the keys of other elements within the window. A Gaussian embedding function is constructed based on these attention weights. The similarity between elements is calculated using the dot product of the query and the key. Then, the similarity is summed using the Gaussian embedding function constructed based on a specific element and all other elements within the window to generate a normalization factor. Finally, the other elements... After mapping the prime values ​​to a value vector, multiply it by the output of the corresponding Gaussian embedding function. Summing the products of all other elements within the window, the ratio of the sum of the products to the normalization factor is used as the weighted summation of the attention output features. The scale and distribution of the attention output features are adjusted by a learnable projection weight matrix, and then added to a certain element to obtain the output of the window attention block. Through the above design, the window attention block can effectively capture long-range dependencies in local regions, enhance the model's ability to model the detailed structure of the image, and avoid the high computational overhead of the global self-attention mechanism, making it suitable for high-resolution image compression tasks.

[0025] Furthermore, the rate-distortion loss employs a rate-distortion loss function, obtaining the conditional probability distribution of the decoded latent representation under the given recovered super-prior features, and the prior probability distribution of the recovered super-prior features themselves. The expected value of each probability distribution is calculated using a base-2 exponent to obtain the control code rate of the decoded latent features and the control code rate of the super-prior decoding. The Lagrange multiplier, used as a parameter to balance the control code rate and distortion, is multiplied by the distortion of the original image and the reconstructed image. The product is then added to the control code rate of the decoded latent features and the control code rate of the super-prior decoding to obtain the rate-distortion loss function.

[0026] The deep learning image compression system based on wavelet domain dual-branch includes a master encoder, a quantizer, an entropy encoder, and a master decoder. According to the deep learning image compression method based on wavelet domain dual-branch, a deep neural network is constructed and trained to encode, quantize, entropy encode, and decode the original image, and then the image is compressed based on the trained deep neural network.

[0027] The advantages and beneficial effects of this invention are as follows:

[0028] This invention achieves a balance between low-frequency and high-frequency modeling based on a wavelet domain dual-branch Transformer, avoiding the loss of high-frequency information caused by traditional downsampling and enhancing the ability to recover details. The window-based attention mechanism of this invention effectively reduces the complexity of attention calculation and improves the fineness of bit allocation, making the reconstruction of texture and edge regions clearer. The channel entropy model of this invention improves the accuracy of probability estimation, reduces redundant bits, and improves the compression ratio by jointly using channel dependency and super-prior modeling. Attached Figure Description

[0029] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0030] Figure 2 This is a diagram of a deep learning image compression framework based on a wavelet domain dual-branch Transformer in an embodiment of the present invention.

[0031] Figure 3 This is a schematic diagram of the wavelet Transformer block in an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram of the structure of the dual-branch self-attention module (DWAM) of wavelet transform in an embodiment of the present invention.

[0033] Figure 5 This is a schematic diagram of the channel-based entropy model structure in an embodiment of the present invention.

[0034] Figure 6 This is a structural diagram of the advanced prior codec in an embodiment of the present invention.

[0035] Figure 7 This is a schematic diagram of the channel context model structure based on wavelet domain Transformer in an embodiment of the present invention.

[0036] Figure 8 This is a schematic diagram of the window channel attention module (WBCAM) in an embodiment of the present invention.

[0037] Figure 9 This is a schematic diagram of the window attention block in an embodiment of the present invention. Detailed Implementation

[0038] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0039] Current deep learning-based image compression methods generally suffer from problems such as limited receptive fields of convolutional neural networks, lack of explicit modeling and separation of multi-scale frequency components of images, and inaccurate estimation of latent feature distributions.

[0040] like Figure 1 As shown, this invention proposes a deep learning-based image compression method based on a two-branch wavelet domain, comprising the following steps:

[0041] Step 1: Constructing an end-to-end image compression system based on deep learning; encoding, quantizing, entropy encoding, and decoding the original image using a deep neural network. During the encoding process, features are extracted from the original image to map it to a highly compact latent representation. The latent representation is then divided into channels, and each channel is modeled as a Gaussian distribution with mean and standard deviation based on joint predictions of spatial and channel context models. After quantization and entropy encoding of the channel blocks guided by the mean and standard deviation, a compressed bitstream is generated. The bitstream is then decoded, and a loss is constructed between the decoded and reconstructed image and the original image to train the deep neural network.

[0042] End-to-end image compression overcomes the limitations of traditional step-by-step optimization by jointly optimizing the master encoder, quantizer, entropy model, and master decoder through deep neural networks.

[0043] Step 2: For image encoding and decoding, a nonlinear transform network is constructed by combining wavelet domain features to perform multi-domain feature mapping; such as... Figure 2 As shown, the main encoder and main decoder of this invention combine wavelet domain feature decomposition to optimize the input representation, and use a two-branch Transformer structure to construct a nonlinear transformation network, where H and W represent the height and width of the feature map.

[0044] At the encoding end, the original image First, a nonlinear analysis and transformation network is used. This network extracts features by stacking three wavelet Transformer blocks (containing layer normalization, a two-branch wavelet attention module, and a wavelet multilayer perceptron) with a convolutional module, mapping the image to a highly compact latent representation. This enhances the global modeling and detail preservation capabilities of analysis and synthetic transformations. The above process can be formally expressed as:

[0045]

[0046] At the decoding end, the latent representation via a nonlinear synthetic transformation network The network maps back to the reconstructed image by stacking three wavelet Transformer blocks (containing layer normalization, a two-branch wavelet attention module, and a wavelet multilayer perceptron) and transposed convolutional modules. The above process can be formally expressed as:

[0047]

[0048] To more effectively enhance high-frequency and low-frequency features after multiple downsampling, this invention proposes a Wavelet Transformer Block (WTB). This module employs a dual-branch Wavelet Attention Mechanism (DWAM) based on wavelet transform, which has stronger adaptability and flexibility in high-frequency and low-frequency feature modeling, thereby alleviating the problems of excessive accumulation of low-frequency components and insufficient high-frequency details.

[0049] Specifically, such as Figure 3 As shown, given the input feature map The calculation process for WTB is as follows:

[0050]

[0051]

[0052]

[0053] Here, LN stands for Layer Normalization, BN stands for Batch Normalization, and MLP stands for Multi-Layer Perceptron. Through this cascaded structure, the WTB module can adaptively balance the modeling of global context and local details in the frequency domain, effectively improving feature representation capabilities and providing richer and less redundant feature representations for subsequent compression and reconstruction tasks.

[0054] The Dual-branch Wavelet Attention Mechanism (DWAM) is a core component of the Wavelet Transformer block (WTB). It decomposes input features in the frequency domain at multiple scales and directions, and utilizes a self-attention mechanism to enhance low-frequency structural information and high-frequency detail information respectively. For example... Figure 4 As shown, DWAM is calculated as follows:

[0055] First, the input features Perform wavelet transform Decompose it into low-frequency components With high frequency components :

[0056]

[0057] Among them, the low-frequency branch is mainly responsible for modeling the global structure of the image, while the high-frequency branch is used to preserve details such as texture and edges, thereby enhancing detail reconstruction while ensuring structural consistency.

[0058] In the low-frequency branch, a low-frequency multi-head self-attention (L-MSA) mechanism is introduced, whose attention weights are determined by the low-frequency query vector. With key vector The calculation yielded:

[0059]

[0060] In the high-frequency branch, a high-frequency multi-head self-attention (H-MSA) mechanism is introduced, whose attention weights are determined by the high-frequency query vector. With key vector The calculation yielded:

[0061]

[0062] in, The dimension of the key vector. Parameters The proportion of heads allocated to frequency domain attention is set in this invention. =0.9, to enhance the processing of low-frequency components.

[0063] Ultimately, WTB will output the low-frequency branch. With the output of the high-frequency branch By concatenating along the feature dimensions, we obtain the fused feature representation:

[0064]

[0065] Through the above design, WTB can adaptively enhance low-frequency and high-frequency features, effectively compensating for detailed features while ensuring global information modeling, thereby improving the feature representation ability and reconstruction quality of the compressed network.

[0066] Step 3: Entropy coding adopts a channel-based entropy model to establish an accurate probability distribution model for the quantized latent representation in order to minimize the coding bit rate. This model is macroscopically composed of a super-prior path and a context modeling path. By jointly modeling the global statistical dependence and local channel context dependence of latent features, it significantly improves the accuracy of probability estimation.

[0067] like Figure 5 As shown, in order to make full use of the statistical information of latent features, this invention proposes a hyperprior encoder-decoder for the channel entropy model. By modeling the latent features globally in advance, the accuracy of the channel entropy model in measuring the mean and standard deviation is effectively improved.

[0068] Specifically, at the super-prior encoder, the input latent features are first... via hyperanalysis transform network Mapping yields the super-prior features. :

[0069]

[0070] in, For hyperanalysis transform networks, such as Figure 6 As shown, its structure consists of multiple convolutional layers (Conv3×3) and downsampling layers (TConv3×3 ↓ 2) stacked alternately, with ReLU activation function used between layers to introduce nonlinearity.

[0071] Subsequently, the prior features Quantization and entropy coding are performed to generate the bitstream and the corresponding prior is recovered at the prior decoder. .

[0072] At the advanced prior decoder, a super-synthetic transform network is employed. Decoded super-prior Upsampling and mapping are performed to obtain features used for auxiliary channel probability modeling. :

[0073]

[0074] in, Structurally and It is basically symmetrical and consists of multiple upsampling layers (TConv3×3 ↑ 2) and convolutional layers (Conv3×3) stacked alternately, and also uses the ReLU activation function.

[0075] Through this process, the prior features can provide global mean and standard deviation prior information, thus providing effective auxiliary constraints for modeling the probability distribution of channel blocks.

[0076] To further improve the accuracy of channel probability modeling, this invention introduces a channel context model based on wavelet domain Transformer. (Wavelet Transformer-based Channel ContextModel), such as Figure 7 As shown, this model accurately predicts the current group of channels to be encoded by modeling the decoded channel blocks and their context. mean With standard deviation parameter .

[0077] Specifically, the channel block to be decoded is denoted as Decoded channel group To compare it with the priori Global context information obtained from decoding The inputs are fed into a four-stage wavelet transformer block (WTB) to perform low-frequency and high-frequency feature modeling, respectively.

[0078] Each WTB stage includes a wavelet-based dual-branch self-attention mechanism (DWAM), which models low-frequency and high-frequency components using low-frequency multi-head self-attention (L-MSA) and high-frequency multi-head self-attention (H-MSA) respectively to remove local and global redundant information. The number of channels for the four WTBs is set to... and between adjacent WTBs Convolutions adjust the number of channels to maintain feature matching.

[0079] Subsequently, the contextual features output by WTB are input into the Windowed Channel Attention Module (WBCAM), which further enhances the correlation between channels using window partitioning and channel weighting mechanisms. Finally, based on the prior features and contextual modeling results, channel blocks are processed. mean with standard deviation Conduct joint forecasting:

[0080]

[0081] in, This represents the context modeling function implemented through the combination of WTB and WBCAM. The predicted... , This will be used to guide the quantization and entropy coding of channel blocks, thereby significantly improving compression performance.

[0082] Furthermore, the Window Channel Attention Module (WBCAM) such as Figure 8 As shown, it consists of a Window-based Attention Block, a Residual Block, a Convolutional Layer (Conv 1×1), and a Depth Separable Multilayer Perceptron (DWMLP). This module aims to model attention in local regions by using window partitioning and combining it with a channel attention mechanism to effectively capture high-frequency detail features in complex regions and improve feature representation capabilities during compression.

[0083] Specifically, input features First, the window is divided and attention-weighted through a window attention block. Features within each window undergo self-attention computation, thus achieving fine-grained dependency modeling within a local scope. The output of this process is... It can be represented as:

[0084]

[0085] Next, output features The feature passes through three cascaded residual blocks to enhance the nonlinear representation of deep features and effectively alleviate the gradient vanishing problem in deep networks.

[0086] Subsequently, after Convolutional layers compress and reorganize the channel dimension, further eliminating redundant information between channels. This is then input into a depthwise separable multilayer perceptron, which performs joint modeling in both spatial and channel dimensions through depthwise separability, enhancing the model's feature selection capability.

[0087] Finally, the channel attention results output by WBCAM are fused with the original input through element-wise multiplication and residual addition to complete the channel context modeling and feature enhancement. This module can not only effectively highlight the feature representation of complex regions (such as texture details and edges), but also reduce redundant bit allocation in low-contrast regions, thereby improving the rate-distortion performance of overall image compression.

[0088] Furthermore, the Window Attention Block (WBCAM) is a core component of the Window-Channel Attention Module. It employs a standard query-key-value attention mechanism and computes self-attention within local windows to efficiently model spatial-channel context dependencies. This module first divides the input feature map into multiple non-overlapping local windows, and independently computes attention weights within each window, thereby significantly reducing computational complexity while maintaining the global receptive field.

[0089] The core calculation process is as follows: Given input features First, divide it into A non-overlapping window, denoted as . For the first Any element within a window Its output From all elements within the window The weighted summation is obtained, and the calculation formula is as follows:

[0090]

[0091]

[0092]

[0093]

[0094] in, and They represent the first The first window The and the first One element, and Through Convolution implements a linear transformation of the query and key, which is used to generate attention weights. It is a linear mapping function. pass Convolution maps input features to a value vector. This is a Gaussian embedding function that calculates the similarity between elements by the dot product of the query and the key. This is a normalization factor used for stable numerical calculations.

[0095] like Figure 9 As shown, input features Simultaneously through three independent Convolutional layer ( , and The system generates queries, keys, and values. Then, it calculates the similarity between the queries and keys within a window and obtains the attention output through a weighted summation. Subsequently, the window attention output is fused with the original input features using a residual connection mechanism to obtain the module's final output:

[0096]

[0097] in, For learnable projective weight matrix (through (Convolutional implementation) is used to adjust the scale and distribution of attention output features.

[0098] Through the above design, the window attention block can effectively capture long-range dependencies in local regions, enhance the model's ability to model the detailed structure of images, and avoid the high computational overhead of the global self-attention mechanism, making it suitable for high-resolution image compression tasks.

[0099] To optimize the entire model, this invention employs a Lagrange rate-distortion optimization framework, constructing a rate-distortion loss function for end-to-end training:

[0100]

[0101] Where λ is a Lagrange multiplier, used to control the code rate ( ) and distortion ( The key parameter for weighing between ) and They represent the quantized latent representations respectively. With prior The estimated bit rate, Used to measure the original input image With reconstructed images The distortion between components is typically measured using mean squared error (MSE) or multi-scale structural similarity (MS-SSIM) as a distortion metric. This represents the mathematical expectation, which is approximated during training by averaging the batch of samples. Indicates that given a priori Under what conditions is the latent representation The conditional probability distribution, Indicates prior knowledge Its own prior probability distribution.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning image compression method based on wavelet domain double branch, characterized in that: Encoding, quantization, entropy coding, decoding of the original image by a deep neural network; The encoding and decoding process combines high-frequency and low-frequency dual-branch wavelet domain features to construct a nonlinear transformation network for multi-domain feature mapping. The encoding process maps the original image to compact latent features. The compact latent features are hyper-prior encoded, and the obtained hyper-prior features are quantized and entropy coded to generate a code stream. The decoding process is based on the decoded latent features and the reconstructed image. The rate-distortion loss is constructed based on the control code rate of the decoded latent features and the hyper-prior decoding control code rate, as well as the distortion of the original image and the reconstructed image, to train the deep neural network.

2. The wavelet domain dual-branch based deep learning image compression method of claim 1, wherein: In the encoding process, the original image is extracted by a nonlinear analysis transformation network composed of multiple wavelet blocks and convolution modules stacked alternately; in the decoding end, the latent features to be decoded are mapped back to the reconstructed image by a nonlinear synthesis transformation network composed of multiple wavelet blocks and transpose convolution modules stacked alternately; the wavelet block includes a layer normalization unit, a wavelet transform-based self-attention module, a batch normalization unit, and a wavelet multi-layer perceptron; the input features are sequentially passed through the layer normalization unit, the wavelet transform-based self-attention module, and the batch normalization unit, and then added to the original input features; the obtained attention features are sequentially passed through the layer normalization unit and the wavelet multi-layer perceptron, and then added to the original attention features; and the output features of the wavelet block are obtained.

3. The wavelet domain dual-branch based deep learning image compression method of claim 2, wherein: The wavelet transform-based self-attention module is a wavelet transform-based dual-branch self-attention module that decomposes the obtained features in the frequency domain in multiple scales and directions, and respectively enhances the low-frequency structural information and the high-frequency detail information using the self-attention mechanism.

4. The wavelet domain dual-branch based deep learning image compression method of claim 3, wherein: The wavelet transform-based dual-branch self-attention module performs wavelet transform on the obtained features, and the decomposed low-frequency components and high-frequency components enter the low-frequency branch and the high-frequency branch, respectively; in the low-frequency branch, a low-frequency multi-head self-attention mechanism is introduced, and the low-frequency attention features are calculated by the low-frequency query vector, the low-frequency key vector, and the dimension of the low-frequency key vector; in the high-frequency branch, a high-frequency multi-head self-attention mechanism is introduced, and the high-frequency attention features are calculated by the high-frequency query vector, the high-frequency key vector, and the dimension of the high-frequency key vector; then, the low-frequency attention features and the high-frequency attention features are concatenated by dimension to obtain the fused features as the output of the wavelet transform-based dual-branch self-attention module.

5. The wavelet domain dual-branch based deep learning image compression method of claim 1, wherein: The compact latent feature is mapped through a hyper-prior encoder to obtain a hyper-prior feature, and the hyper-prior feature is quantized and entropy encoded to generate a bitstream, and the corresponding hyper-prior feature is recovered through a hyper-prior decoder, and the recovered hyper-prior feature is mapped through a hyper-synthesis transform network to obtain global context information, which is used to assist channel probability modeling.

6. The wavelet domain dual-branch based deep learning image compression method of claim 5, wherein: A wavelet domain-based channel context model is introduced, and the decoded channel block and the global context information are modeled through multiple stages of wavelet blocks. Each stage of the wavelet block includes a double-branch self-attention module based on wavelet transform, which models low-frequency components and high-frequency components through low-frequency multi-head self-attention mechanism and high-frequency multi-head self-attention mechanism, respectively. The obtained context features are input into a window channel attention module to enhance the correlation between channels by using window division and channel weighting mechanism, so as to predict the mean and standard deviation of the current channel block to be encoded, which are used to guide the quantization and entropy encoding of the channel block.

7. The wavelet domain dual-branch based deep learning image compression method of claim 6, wherein: The window channel attention module models the attention of the local region by using window division, and captures the high-frequency detail features of the complex region by combining the channel attention mechanism. The window channel attention module includes a window attention block, a residual block, a convolution layer, and a depth separable multi-layer perceptron. The window attention block divides the context features into multiple non-overlapping windows. For any element in the window, the output feature of the window is obtained by weighted summation of all elements in the window. After the residual block, the convolution layer is used to compress and reorganize the channel dimension. Then, the depth separable multi-layer perceptron is used to jointly model the spatial dimension and the channel dimension in a depth separable manner. The obtained channel attention result is fused with the context features by element-wise multiplication and residual addition, which is used to predict the mean and standard deviation of the channel block to be encoded.

8. The wavelet domain dual-branch based deep learning image compression method of claim 7, wherein: The window attention block adopts a query-key-value attention mechanism and calculates self-attention in the local window. The attention weight of a certain element is obtained by linear transformation of the query. The attention weights of other elements are obtained by linear transformation of the keys. Based on the attention weights, a Gaussian embedding function is constructed. The similarity between elements is calculated by the dot product of the query and the key. Then, the similarity sum of the Gaussian embedding function constructed based on a certain element and all other elements in the window is calculated to generate a normalization factor. After the other elements are mapped to value vectors, they are multiplied by the output of the corresponding Gaussian embedding function. The sum of the products of all other elements is divided by the normalization factor to obtain the attention output feature. The scale and distribution of the attention output feature are adjusted by a learnable projection weight matrix, and then added to the certain element to obtain the output of the window attention block.

9. The wavelet domain dual branch based deep learning image compression method of claim 1, wherein: The rate-distortion loss adopts a rate-distortion loss function, respectively obtains a conditional probability distribution of the decoded latent representation under the condition of given recovered hyper-prior features and a prior probability distribution of the recovered hyper-prior features itself, respectively calculates a mathematical expectation of an exponential with a base of 2 of the probability distribution, and obtains a control code rate of the decoded latent feature and a control code rate of the hyper-prior decoding; a Lagrange multiplier is taken as a parameter for balancing the control code rate and the distortion degree, and is multiplied by the distortion degree of the original image and the reconstructed image, and then the product is added to the control code rate of the decoded latent feature and the control code rate of the hyper-prior decoding, to obtain the rate-distortion loss function.

10. A deep learning image compression system based on wavelet domain dual branch, comprising a main encoder, a quantizer, an entropy encoder, a main decoder, characterized in that: The wavelet domain double-branch-based deep learning image compression method according to any one of claims 1 to 9 constructs and trains a deep neural network to encode, quantize, entropy encode and decode an original image, and then compresses the image based on the trained deep neural network.

Images