Screen content image compression method and system based on frequency perception feature extraction and entropy model construction

Abstract

The invention relates to a screen content image compression method and system based on frequency perception feature extraction and entropy model construction. Comprising the following steps: analyzing transformation frequency division: inputting an original screen content image to obtain multi-frequency potential features; frequency perception entropy model and quantization: inputting multi-frequency potential features to obtain quantized multi-frequency potential features, and performing entropy coding and entropy decoding; performing hyper-prior analysis conversion frequency division: inputting the multi-frequency potential features to obtain hyper-prior multi-frequency potential features; quantization and entropy coding: inputting the hyper-priori multi-frequency potential features to obtain quantized hyper-priori multi-frequency potential features, and performing entropy coding and entropy decoding; super-prior synthesis transformation frequency division: quantized super-prior multi-frequency potential features obtained after entropy decoding are input; and synthesizing, transforming and dividing frequency: inputting the quantized multi-frequency potential features obtained after entropy decoding to obtain a final reconstructed screen content image. The method provided by the invention has better performance on a public data set.

Description

Technical Field

[0001] The present invention relates to a method and system for compressing screen content images (SCI), and in particular to a technology combining frequency perception feature processing and entropy model construction during the image compression process, belonging to the fields of image processing, compression coding and artificial intelligence. Background Art

[0002] With the development of cloud computing and remote collaboration platforms, the compression of screen content images (SCI) has become increasingly important in practical applications. While traditional image compression methods (such as JPEG, H.265 / HEVC, and H.266 / VVC) perform well on natural scene images (NSI), they fail to fully account for the unique characteristics of SCI, such as sharp edges, regular patterns, and repetitive structures. These characteristics make traditional coding methods less effective for SCI compression. In recent years, international standards organizations have actively explored extensions to screen content coding (SCC), including H.265 / HEVC-SCC and H.266 / HEVC-SCC, to address these challenges. However, these extensions typically consist of separate coding modules, and traditional methods are unable to effectively jointly optimize multiple modules, thus limiting the rate-distortion performance of the coding. To address this, researchers have recently proposed learning-based image compression (LIC) methods. These methods leverage the powerful nonlinear capabilities of deep neural networks to replace the individual modules in traditional coding schemes, effectively improving compression performance through joint optimization. However, existing learning-based LIC methods still face two key issues:

[0003] (1) Insufficient frequency-aware processing: Existing methods fail to fully consider the different distribution characteristics of low-frequency and high-frequency features and their targeted processing requirements.

[0004] (2) The entropy model of mixed frequency components is not accurate enough: the existing entropy model methods fail to design optimized probability models for low-frequency and high-frequency features respectively. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention proposes a novel screen content image compression method, which effectively solves the problems of insufficient frequency perception processing and inaccurate frequency component entropy model modeling in existing methods through frequency-aware feature refinement extraction and entropy model construction. The proposed frequency-aware refinement network uses a Transformer-based model to reduce the redundancy of low-frequency features and a reversible neural network to retain key high-frequency details. In addition, the frequency-aware entropy model provides accurate conditional probability estimates for high-frequency and low-frequency features, and uses a parallel context model to improve computational efficiency. Experimental results demonstrate the effectiveness of the proposed scheme and show its potential for efficient SCI compression.

[0006] The present invention also proposes a screen content image compression system based on frequency-aware feature extraction and entropy model construction.

[0007] Explanation of terms: 1. LIC (Learned image compression): An image compression algorithm based on deep learning.

[0008] 2. NSI (Natural Scene Image): Natural scene image, an image captured by a camera.

[0009] 3. SCI (Screen Content Image): Screen content image, an image generated by computer rendering.

[0010] 4. GoCB (Generalized Octave Convolution Block): generalized octave convolution block.

[0011] 5. FARN (Frequency-Aware Refinement Network): Frequency-Aware Refinement Network.

[0012] 6. FAEM (Frequency-Aware Entropy Model): Frequency-Aware Entropy Model.

[0013] 7. STB (Swin Transformer Block): A transformer-based feature extraction network.

[0014] 8. INN (Invertible Neural Network): Reversible neural network.

[0015] 9. IToRB (Improved Two-stage Octave Residual block): Improved two-stage octave convolution residual block.

[0016] 10. Q (Quantization): Quantization module, implemented using the round function.

[0017] 11. AE / AD (Arithmetic Encoding and Arithmetic Decoding): Arithmetic encoding / arithmetic decoding, which uses entropy coding to write quantized features into the bitstream at the encoding end and decode them at the decoding end.

[0018] 12. 、 : Entropy parameter network, which is used to learn the distribution parameters of the entropy model corresponding to high-frequency potential features and low-frequency potential features respectively.

[0019] 13. bpp (bits per pixel): The number of bits per pixel, used to indicate the encoding bit rate.

[0020] 14. PSNR (Peak signal-to-noise ratio): Peak signal-to-noise ratio, used to indicate the encoding quality of the image.

[0021] 15. The SCID dataset is mainly used for quality assessment. It contains 40 original images of screen content with a resolution of 1280×720. The content covers text, graphics, symbols, patterns, and some natural images. The sources are web pages, PDF files, slides, comics, and digital magazines.

[0022] 16. SIQAD dataset, mainly used for quality assessment, contains 20 original images of screen content with a resolution of approximately 600×900. The content covers text, graphics, patterns and some natural images, and the sources are web pages, slides, PDF files and digital magazines.

[0023] 17. OMR-Net: A screen content image compression model based on two-stage octave convolution and multi-scale residual network.

[0024] 18. RealNVP: A learnable reversible transformation network.

[0025] 19. MobileNet: A lightweight feature extraction network.

[0026] 20. IMB (Improved Mobile block): An improved MobileNet module.

[0027] 21. GMM (Gaussian mixture model): Gaussian mixture model.

[0028] 22. LMM (Laplacian mixture model): Laplace mixture model.

[0029] The technical solution of the present invention is: A screen content image compression method based on frequency-aware feature extraction and entropy model construction, comprising: Input the original screen content image into the image compression model to achieve screen content image compression, specifically including: Analysis, transformation and frequency division: Input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; Frequency-aware entropy model and quantization: Input multi-frequency latent features, quantize them to obtain quantized multi-frequency latent features, and use the frequency-aware entropy model to perform entropy encoding and entropy decoding on the quantized multi-frequency latent features; Super-prior analysis transformation and frequency division: Input multi-frequency potential features, and obtain super-prior multi-frequency potential features through super-prior analysis transformation and frequency division; Quantization and entropy coding: Input super-prior multi-frequency potential features, undergo quantization and entropy coding; obtain the quantized super-prior multi-frequency potential features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency potential features; entropy coding and entropy decoding refer to: using arithmetic encoder AD and arithmetic decoder AE to achieve entropy coding and entropy decoding; Super-prior synthesis transform frequency division: The quantized super-prior multi-frequency potential features obtained after input entropy decoding are subjected to super-prior synthesis transform frequency division to obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; Synthetic transform frequency division: The quantized multi-frequency potential features obtained after input entropy decoding are subjected to synthetic transform frequency division to obtain the final reconstructed screen content image.

[0030] According to the preferred embodiment of the present invention, the image compression model includes an analysis transformation frequency division module , synthesis transformation and frequency division module , super-prior analysis transformation frequency division module , super-prior synthesis transformation frequency division module , quantization module Q, frequency-aware entropy model FEAM, arithmetic encoder AE and arithmetic decoder AD; Analysis, transformation and frequency division module , enter the original screen content image , after analyzing and transforming the frequency division, we can get multi-frequency potential features, including high-frequency potential space features and low-frequency latent space features , analysis, transformation and frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features. In the frequency-aware entropy model FAEM and the quantization module Q, high-frequency latent space features are input and low-frequency latent space features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features , combined with the context model, the decoded information is used to guide the current decoding information, and the frequency-aware entropy model FAEM is used to and Perform entropy encoding AE and entropy decoding AD operations; Super priori analysis transform frequency division module In the middle, input high-frequency latent space features and low-frequency latent space features , after super prior analysis transformation frequency division module , get the super prior high frequency potential features and super prior low-frequency latent features , super prior analysis transform frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN includes an invertible neural network INN and a Transformer-based Swin Transformer module (STB). The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Quantization and entropy coding: input super-prior high-frequency potential features and super prior low-frequency latent features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features ,right and Perform entropy encoding AE and entropy decoding AD operations; Super priori synthesis transform frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , through the super prior synthesis transform frequency division module , get the frequency perception entropy model distribution parameters of multi-frequency potential features, super prior synthesis transformation frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Synthesis transformation and frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , after the synthesis transformation frequency division module, the final reconstructed screen content image is obtained , synthesis transformation frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features.

[0031] According to the preferred embodiment of the present invention, the encoding and decoding process is as follows: During the encoding process, for the input original screen content image First, after analyzing and transforming the frequency division module Transformed into high-frequency latent space features and low-frequency latent space features , using super prior analysis to transform the frequency division module Generate super-prior high-frequency latent features and super prior low-frequency latent features , and The quantized high-frequency latent space features are obtained through the quantization operation of the quantization module Q. and quantized low-frequency latent space features , and Write the bitstream through entropy coding; During the decoding process, and Frequency conversion module through super prior synthesis Get about and The super prior information is sent to the frequency perception entropy model FAEM to obtain the distribution parameters of different frequency entropy models to guide and Performing arithmetic encoding and decoding operations in sequence through an arithmetic encoder AE and an arithmetic decoder AD; Characteristics after arithmetic decoding operation and It will be further sent to the synthesis transformation frequency division module for operation Decode it in the decoder, and finally get the reconstructed screen content image .

[0032] According to the preferred embodiment of the present invention, the reversible neural network INN includes a reversible Convolution, feature separation, affine coupling layer and feature fusion; The given i Input features of a reversible neural network , where the width is W, the height is H, and the number of channels is D, first apply reversible convolution IC , and along the channel dimension the input features are indexed at Divided into two features and : ; ; in, refers to the dth layer in the channel dimension D, ; Two characteristics and Then as the input of the affine coupling layer, the affine coupling layer transformation expression is: ; ; in, represents the exponential function, Element-wise multiplication, 、 and is an arbitrary transformation function; 、 Respectively and Features output after the affine coupling layer; The affine coupling layer includes an improved MobileNet module IMB (IMB, Improved Mobile block), which integrates a multi-branch convolutional structure; the input features of IMB first pass through three layers of branches, the first layer of branches includes two Convolution, a Depthwise separable convolution DWconv and two ReLU6 activation functions, the second layer branch includes a Convolution, the third layer branch includes a skip connection, then, after the outputs of the three layers are spliced ​​together, finally, a ReLU6 activation function is passed to obtain the output features of IMB; The corresponding inverse transform is shown below: ; .

[0033] According to the preferred embodiment of the present invention, in the STB, for the input feature First, after a layer normalization operation, the window-based multi-head self-attention mechanism and the jump connection splicing, the output features are then subjected to a layer normalization operation, the multi-layer perceptron network and the jump connection splicing, the output features are then subjected to a layer normalization operation, the sliding window-based multi-head self-attention mechanism and the jump connection splicing, the output features are then subjected to a layer normalization operation, the multi-layer perceptron network and the jump connection splicing, and the final output features are obtained. .

[0034] According to the preferred embodiment of the present invention, the frequency-aware entropy model adopts two independent entropy parameter networks and , respectively estimate the distribution probability parameters of high-frequency and low-frequency features; specifically, the potential features are parameterized using discrete mixed distributions, where the entropy parameter network and Each includes three Convolutional layer, respectively learns the probability distribution parameters of different frequency features; for low-frequency latent space features The discrete Gaussian mixture model GMM is used to model the entropy model for high-frequency latent space features. Apply discrete Laplace mixture model LMM to model entropy model; ; ; in, and are the weight coefficients of Gaussian mixture distribution and Laplace mixture distribution, , , 、 The number of discrete Gaussian mixture models and discrete Laplace mixture model distributions, and Representing the i Gaussian distribution and j the mean and scale parameter of the Laplace distribution; represents the conditional probability distribution of low-frequency latent features, represents the conditional probability distribution of high-frequency latent features, represents a Gaussian distribution, represents the Laplace distribution.

[0035] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the screen content image compression method based on frequency perception-based feature extraction and entropy model construction.

[0036] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the screen content image compression method based on frequency-aware feature extraction and entropy model construction.

[0037] A screen content image compression system based on frequency-aware feature extraction and entropy model construction includes: The analysis, transformation and frequency division module is configured to: input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; The frequency-aware entropy model and quantization module is configured to: input multi-frequency latent features, obtain quantized multi-frequency latent features through quantization, and perform entropy encoding and entropy decoding on the quantized multi-frequency latent features using the frequency-aware entropy model; The super-prior analysis transformation and frequency division module is configured to: input the multi-frequency potential feature, perform super-prior analysis transformation and frequency division, and obtain the super-prior multi-frequency potential feature; The quantization and entropy coding module is configured to: input the super-prior multi-frequency latent features, perform quantization and entropy coding, obtain the quantized super-prior multi-frequency latent features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency latent features; entropy coding and entropy decoding refer to: using an arithmetic encoder AD and an arithmetic decoder AE to implement entropy coding and entropy decoding; The super-prior synthesis transformation frequency division module is configured to: input the quantized super-prior multi-frequency potential features obtained after entropy decoding, perform super-prior synthesis transformation frequency division on them, and obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; The synthesis transformation frequency division module is configured to: input the quantized multi-frequency potential features obtained after entropy decoding, and obtain the final reconstructed screen content image through synthesis transformation frequency division.

[0038] The beneficial effects of the present invention are: 1. This paper constructs a frequency-aware refinement network (FARN). Taking into account the different roles of high-frequency and low-frequency components in image reconstruction, the network adopts a Transformer-based model to reduce the redundancy of low-frequency features and an invertible neural network (INN) model to retain key high-frequency details, thereby enhancing the compact expression of low-frequency and high-frequency features.

[0039] 2. This paper proposes a frequency-aware entropy model (FAEM), which takes into account the distribution characteristics of different frequency features and constructs different entropy models based on frequency perception for different frequency features, thereby achieving more flexible and accurate conditional probability estimation.

[0040] 3. Experimental results verify that the performance of the method proposed in this invention on public datasets is better than the existing deep learning-based methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural block diagram of the image compression model of the present invention.

[0042] Figure 2 It is a detailed structural diagram of the image compression model of the present invention.

[0043] Figure 3 This is a diagram of the reversible neural network structure used in the present invention.

[0044] Figure 4This is a comparison chart of the rate-distortion performance of the present invention and different coding schemes on the SCID dataset.

[0045] Figure 5 This is a comparison chart of the rate-distortion performance of the present invention and different coding schemes on the SIQAD dataset. DETAILED DESCRIPTION

[0046] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0047] Example 1 A screen content image compression method based on frequency-aware feature extraction and entropy model construction, comprising: Input the original screen content image into the image compression model to achieve screen content image compression, specifically including: Analysis, transformation and frequency division: Input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; Frequency-aware entropy model and quantization: Input multi-frequency latent features, quantize them to obtain quantized multi-frequency latent features, and use the frequency-aware entropy model to perform entropy encoding and entropy decoding on the quantized multi-frequency latent features; Super-prior analysis transformation and frequency division: Input multi-frequency potential features, and obtain super-prior multi-frequency potential features through super-prior analysis transformation and frequency division; Quantization and entropy coding: Input super-prior multi-frequency potential features, undergo quantization and entropy coding; obtain the quantized super-prior multi-frequency potential features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency potential features; entropy coding and entropy decoding refer to: using arithmetic encoder AD and arithmetic decoder AE to achieve entropy coding and entropy decoding; Super-prior synthesis transform frequency division: The quantized super-prior multi-frequency potential features obtained after input entropy decoding are subjected to super-prior synthesis transform frequency division to obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; Synthetic transform frequency division: The quantized multi-frequency potential features obtained after input entropy decoding are subjected to synthetic transform frequency division to obtain the final reconstructed screen content image.

[0048] Example 2 The screen content image compression method based on frequency-aware feature extraction and entropy model construction described in Example 1 is different in that: like Figure 1 、 Figure 2 As shown in (a), the image compression model includes an analysis transform frequency division module , synthesis transformation and frequency division module , super-prior analysis transformation frequency division module , super-prior synthesis transformation frequency division module , quantization module Q, frequency-aware entropy model FEAM, arithmetic encoder AE and arithmetic decoder AD; Analysis, transformation and frequency division module , enter the original screen content image , after analyzing and transforming the frequency division, we can get multi-frequency potential features, including high-frequency potential space features and low-frequency latent space features , analysis, transformation and frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features. In the frequency-aware entropy model FAEM and the quantization module Q, high-frequency latent space features are input and low-frequency latent space features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features , combined with the context model, the decoded information is used to guide the current decoding information, and the frequency-aware entropy model FAEM is used to and Perform entropy encoding AE and entropy decoding AD operations; Super priori analysis transform frequency division module In the middle, input high-frequency latent space features and low-frequency latent space features , after super prior analysis transformation frequency division module , get the super prior high frequency potential features and super prior low-frequency latent features , super prior analysis transform frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN includes an invertible neural network INN and a Transformer-based Swin Transformer module (STB). The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Quantization and entropy coding: input super-prior high-frequency potential features and super prior low-frequency latent features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features ,right and Perform entropy encoding AE and entropy decoding AD operations; Super priori synthesis transform frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , through the super prior synthesis transform frequency division module , get the frequency perception entropy model distribution parameters of multi-frequency potential features, super prior synthesis transformation frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Synthesis transformation and frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , after the synthesis transformation frequency division module, the final reconstructed screen content image is obtained , synthesis transformation frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features.

[0049] The encoding and decoding process is as follows: During the encoding process, for the input original screen content image First, after analyzing and transforming the frequency division module Transformed into high-frequency latent space features and low-frequency latent space features , after quantization operation, the quantized high-frequency latent space features are generated and quantized low-frequency latent space features , and The code stream is written through entropy coding and passed to the decoding network for decoding. In order to further remove the correlation in the latent feature space of different frequencies and establish a more accurate entropy model, the super prior network is first used to generate the latent features. and , using super prior analysis to transform the frequency division module Generate super-prior high-frequency latent features and super prior low-frequency latent features , and The quantized high-frequency latent space features are obtained through the quantization operation of the quantization module Q. and quantized low-frequency latent space features , and The bitstream is written using entropy coding and then passed to the super-prior decoding network for decoding. Secondly, the context model is combined to better predict the probability model of the latent representation. A frequency-fractionated perceptual entropy model (FEAM) is constructed. The FEAM network generates the distribution parameters of the entropy model for different frequency features to more accurately fit the distribution of different frequency features.

[0050] During the decoding process, and Frequency conversion module through super prior synthesis Get about and The super prior information is sent to the frequency perception entropy model FAEM to obtain the distribution parameters of different frequency entropy models to guide and Performing arithmetic encoding and decoding operations in sequence through an arithmetic encoder AE and an arithmetic decoder AD; Characteristics after arithmetic decoding operation and It will be further sent to the synthesis transformation frequency division module for operation Decode it in the decoder, and finally get the reconstructed screen content image .

[0051] The network architecture of the present invention is based on OMR-Net, but the OMR-Net solution has shortcomings in processing frequency perception and constructing an inaccurate entropy model for mixed frequency components. Therefore, the present invention has made improvements, and the specific innovations of the solution are as follows:

[0052] A frequency-aware refinement network (FARN) is constructed, which adopts a Transformer-based model to reduce the redundancy of low-frequency features and an invertible neural network (INN) model to retain key high-frequency details, while considering the different roles of high- and low-frequency components in image reconstruction.

[0053] A frequency-aware entropy model (FAEM) is proposed, which involves constructing different entropy models for high-frequency and low-frequency features, thereby achieving more flexible and accurate conditional probability estimation.

[0054] The purpose of a transform network is to extract compact latent features. However, most existing methods fail to consider the heterogeneity of frequency features during feature extraction. Taking into account the spatial redundancy of frequency features and the need for downsampling, OMR-Net employs an improved two-stage octave residual block (IToRB) during encoding to extract and downsample high- and low-frequency features. However, this does not explicitly address the heterogeneity in processing requirements for different frequency components. To overcome this limitation, the Frequency-Aware Refinement Network (FARN) was proposed based on IToRB. FARN consists of an invertible neural network (INN) and a Transformer-based Swin Transformer (STB) module. Specifically, the INN is used to refine high-frequency features, while the STB is used to refine low-frequency features. This enhances the compactness of both high- and low-frequency features.

[0055] High-frequency features represent important edge and texture details in the image and play a key role in the quality of image reconstruction. To avoid the loss of high-frequency information in the feature extraction process, this paper designs a reversible neural network based on RealNVP to achieve bijective transformation of high-frequency features from input to output; the network structure is as follows: Figure 3 As shown, the reversible neural network INN includes reversible Convolution, feature separation, affine coupling layer and feature fusion;

[0056] The given i Input features of a reversible neural network , where the width is W, the height is H, and the number of channels is D, first apply reversible convolution IC , and along the channel dimension the input features are indexed at Divided into two features and : ; ; in, refers to the dth layer in the channel dimension D, ; Two characteristics and Then as the input of the affine coupling layer, the affine coupling layer transformation expression is: ; ; in, represents the exponential function, Element-wise multiplication, 、 and is an arbitrary transformation function; 、 Respectively and Features output after the affine coupling layer; In the affine coupling layer, it is an arbitrary transformation function 、 and An improved MobileNet module (IMB) is designed. The affine coupling layer includes the improved MobileNet module IMB, which integrates a multi-branch convolutional structure; at the same time, it maintains a lightweight design, such as Figure 3 As shown. The input features of IMB first pass through three layers of branches. The first layer of branches includes two Convolution, a Depthwise separable convolution DWconv and two ReLU6 activation functions, the second layer branch includes a Convolution, the third layer branch includes a skip connection, then, after the outputs of the three layers are spliced ​​together, finally, a ReLU6 activation function is passed to obtain the output features of IMB;

[0057] The corresponding inverse transform is shown below: ; .

[0058] In STB, visual transformers have shown great potential in visual tasks such as image classification, semantic segmentation, and image compression. Low-frequency features capture slowly changing areas in an image and usually contain most of the image energy. CNN models perform well in extracting local features, but generally lack the ability to capture global contextual information. To compensate for this shortcoming, the present invention adopts the Swin Transformer module (STB), which uses the self-attention mechanism to model long-range dependencies on low-frequency features, such as Figure 2 As shown in (b).

[0059] The specific architecture of the Swin Transformer module (STB) is as follows Figure 2 As shown in (b), for the input feature First, it undergoes a layer normalization operation (LN), a window-based multi-head self-attention mechanism (W-MSA) and a jump connection. The output features then undergo a layer normalization operation (LN), a multi-layer perceptron network (MLP) and a jump connection. The output features then undergo a layer normalization operation (LN), a sliding window-based multi-head self-attention mechanism (SW-MSA) and a jump connection. The output features then undergo a layer normalization operation (LN), a multi-layer perceptron network (MLP) and a jump connection. The final output features are obtained. .

[0060] The present invention extracts latent features at different frequencies and performs refined processing on each of them. In the analysis and transformation frequency division modules and the synthesis and transformation frequency division modules, the multi-frequency latent features are divided into high-frequency and low-frequency components for refined processing. However, multi-frequency latent features are not limited to two frequency components and can be divided into two, three, or even more frequency components, with refined feature processing performed on each frequency.

[0061] Existing entropy models approximate the true marginal distribution of latent features by estimating the distribution. The accuracy of this approximation directly affects the optimization of network parameters and, in turn, the rate-distortion (RD) performance of the network. We found that the use of a single conditional Gaussian distribution in OMR-Net is not sufficient to fully capture the latent features, especially the significantly different marginal distribution characteristics between high-frequency and low-frequency components. To address this problem, this solution proposes a frequency-aware entropy model (FAEM), which uses two independent entropy parameter networks. and , respectively estimate the distribution probability parameters of high-frequency and low-frequency features; specifically, the potential features are parameterized using discrete mixed distributions, where the entropy parameter network and Each includes three The convolution layer learns the probability distribution parameters of different frequency features respectively; considering the extreme high and low frequency characteristics of the screen content image (SCI), the low frequency latent space features are The discrete Gaussian mixture model GMM is used to model the entropy model for high-frequency latent space features. Apply discrete Laplace mixture model LMM to model entropy model;

[0062] ; ; in, and are the weight coefficients of Gaussian mixture distribution and Laplace mixture distribution, , , 、 The number of discrete Gaussian mixture models and discrete Laplace mixture model distributions, and Representing the i Gaussian distribution and j the mean and scale parameter of the Laplace distribution; represents the conditional probability distribution of low-frequency latent features, represents the conditional probability distribution of high-frequency latent features, represents a Gaussian distribution, represents the Laplace distribution.

[0063] This solution extracts latent features at different frequencies by modeling them using frequency-aware entropy models based on the characteristics of each frequency feature. Specifically, the discrete Gaussian mixture model (GMM) and discrete Laplace mixture model (LMM) are applied to low-frequency and high-frequency features, but these distribution models are not limited to these two. Other possible models include generalized Gaussian models, generalized Gaussian mixture models, logistic mixture models, Cauchy mixture models, and gamma mixture models.

[0064] In this embodiment, 2000 screen content images are used as a training set, covering a variety of content types, including text, charts, graphics, web pages, games, animated films, and mixed images of natural scenes and screen content. During the training process, the images in the training set are randomly divided into image blocks of 256×256 size, the batchsize of the network is set to 8, and the Adam optimizer is used for training 400 epochs. The initial learning rate is 1e-4, and the learning rate is reduced to 1e-5 after the 300th epoch. The optimization index of the model uses the mean square error (MSE), the parameters of the rate distortion optimization are set to {0.0018, 0.0035, 0.0067, 0.013, 0.025, 0.0483}, and the parameters of the Gaussian mixture distribution and the Laplace mixture distribution are set to In terms of network channel settings, the codec has 192 convolutional channels at low bit rates and 320 channels at high bit rates. The performance evaluation of the model is tested using the screen content dataset SCID.

[0065] The experiment tested the rate-distortion (RD) performance comparison between the method of the present invention and other deep learning-based image coding methods. Figure 4 and Figure 5The coding performance of different methods was compared on the SCID and SIQAD datasets. The horizontal axis represents bits per pixel (bpp), which indicates the coding rate, and the vertical axis represents the peak signal-to-noise ratio (PSNR), which indicates the coding quality of the image. Under the same PSNR conditions, the lower the bpp, the better; under the same bpp conditions, the higher the PSNR, the better. It can be seen that compared with other deep learning-based methods, under the same PSNR, the objective evaluation indicator of the decoded image, the coding rate of the present invention is lower, indicating that the method proposed in this invention has better coding performance, that is, better image quality can be achieved at a lower bit rate.

[0066] Example 3 A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the screen content image compression method based on frequency perception feature extraction and entropy model construction described in Example 1 or 2 are implemented.

[0067] Example 4 A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the screen content image compression method based on frequency-aware feature extraction and entropy model construction described in Example 1 or 2.

[0068] Example 5 A screen content image compression system based on frequency-aware feature extraction and entropy model construction includes: The analysis, transformation and frequency division module is configured to: input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; The frequency-aware entropy model and quantization module is configured to: input multi-frequency latent features, obtain quantized multi-frequency latent features through quantization, and perform entropy encoding and entropy decoding on the quantized multi-frequency latent features using the frequency-aware entropy model; The super-prior analysis transformation and frequency division module is configured to: input the multi-frequency potential feature, perform super-prior analysis transformation and frequency division, and obtain the super-prior multi-frequency potential feature; The quantization and entropy coding module is configured to: input the super-prior multi-frequency latent features, perform quantization and entropy coding, obtain the quantized super-prior multi-frequency latent features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency latent features; entropy coding and entropy decoding refer to: using an arithmetic encoder AD and an arithmetic decoder AE to implement entropy coding and entropy decoding; The super-prior synthesis transformation frequency division module is configured to: input the quantized super-prior multi-frequency potential features obtained after entropy decoding, perform super-prior synthesis transformation frequency division on them, and obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; The synthesis transformation frequency division module is configured to: input the quantized multi-frequency potential features obtained after entropy decoding, and obtain the final reconstructed screen content image through synthesis transformation frequency division.

Claims

1. A screen content image compression method based on frequency-aware feature extraction and entropy model construction, characterized in that: include: Input the original screen content image into the image compression model to achieve screen content image compression, specifically including: Analysis, transformation and frequency division: Input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; Frequency-aware entropy model and quantization: Input multi-frequency latent features, quantize them to obtain quantized multi-frequency latent features, and use the frequency-aware entropy model to perform entropy encoding and entropy decoding on the quantized multi-frequency latent features; Super-prior analysis transformation and frequency division: Input multi-frequency potential features, and obtain super-prior multi-frequency potential features through super-prior analysis transformation and frequency division; Quantization and entropy coding: Input super-prior multi-frequency potential features, undergo quantization and entropy coding; obtain the quantized super-prior multi-frequency potential features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency potential features; entropy coding and entropy decoding refer to: using arithmetic encoder AD and arithmetic decoder AE to achieve entropy coding and entropy decoding; Super-prior synthesis transform frequency division: The quantized super-prior multi-frequency potential features obtained after input entropy decoding are subjected to super-prior synthesis transform frequency division to obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; Synthetic transform frequency division: The quantized multi-frequency potential features obtained after input entropy decoding are subjected to synthetic transform frequency division to obtain the final reconstructed screen content image.

2. The screen content image compression method based on frequency-aware feature extraction and entropy model construction according to claim 1, characterized in that: Image compression model includes analysis transform frequency division module , synthesis transformation and frequency division module , super-prior analysis transformation frequency division module , super-prior synthesis transformation frequency division module , quantization module Q, frequency-aware entropy model FEAM, arithmetic encoder AE and arithmetic decoder AD; Analysis, transformation and frequency division module , enter the original screen content image , after analyzing and transforming the frequency division, we can get multi-frequency potential features, including high-frequency potential space features and low-frequency latent space features , analysis, transformation and frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features. In the frequency-aware entropy model FAEM and the quantization module Q, high-frequency latent space features are input and low-frequency latent space features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features , combined with the context model, the decoded information is used to guide the current decoding information, and the frequency-aware entropy model FAEM is used to and Perform entropy encoding AE and entropy decoding AD operations; Super priori analysis transform frequency division module In the middle, input high-frequency latent space features and low-frequency latent space features , after super prior analysis transformation frequency division module , get the super prior high frequency potential features and super prior low-frequency latent features , super prior analysis transform frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN includes an invertible neural network INN and a Transformer-based Swin Transformer module (STB). The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Quantization and entropy coding: input super-prior high-frequency potential features and super prior low-frequency latent features , after quantization operation, the quantized high-frequency latent space features are obtained and quantized low-frequency latent space features ,right and Perform entropy encoding AE and entropy decoding AD operations; Super priori synthesis transform frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , through the super prior synthesis transform frequency division module , get the frequency perception entropy model distribution parameters of multi-frequency potential features, super prior synthesis transformation frequency division module It includes ItoRB and frequency-aware refinement network FARN. The frequency-aware refinement network FARN uses STB to refine the super-prior low-frequency potential features and uses INN to refine the super-prior high-frequency potential features. Synthesis transformation and frequency division module : quantized high-frequency latent space features obtained by input entropy decoding and quantized low-frequency latent space features , after the synthesis transformation frequency division module, the final reconstructed screen content image is obtained , synthesis transformation frequency division module It includes GoCB, ItoRB and frequency-aware refinement network FARN. FARN uses STB to refine low-frequency potential features and uses INN to refine high-frequency potential features.

3. The screen content image compression method based on frequency perception feature extraction and entropy model construction according to claim 1, characterized in that: The encoding and decoding process is as follows: During the encoding process, for the input original screen content image First, after analyzing and transforming the frequency division module Transformed into high-frequency latent space features and low-frequency latent space features , using super prior analysis to transform the frequency division module Generate super-prior high-frequency latent features and super prior low-frequency latent features , and The quantized high-frequency latent space features are obtained through the quantization operation of the quantization module Q. and quantized low-frequency latent space features , and Write the bitstream through entropy coding; During the decoding process, and Frequency conversion module through super prior synthesis Get about and The super prior information is sent to the frequency perception entropy model FAEM to obtain the distribution parameters of different frequency entropy models to guide and Performing arithmetic encoding and decoding operations in sequence through an arithmetic encoder AE and an arithmetic decoder AD; Characteristics after arithmetic decoding operation and It will be further sent to the synthesis transformation frequency division module for operation Decode it in the decoder, and finally get the reconstructed screen content image .

4. The screen content image compression method based on frequency-aware feature extraction and entropy model construction according to claim 2, characterized in that: Reversible neural network INN includes reversible Convolution, feature separation, affine coupling layer and feature fusion; The given i Input features of a reversible neural network , where the width is W, the height is H, and the number of channels is D, first apply reversible convolution IC , and along the channel dimension the input features are indexed at Divided into two features and : ； ； in, refers to the dth layer in the channel dimension D, ; Two characteristics and Then as the input of the affine coupling layer, the affine coupling layer transformation expression is: ； ； in, represents the exponential function, Element-wise multiplication, 、 and is an arbitrary transformation function; 、 Respectively and Features output after the affine coupling layer; The affine coupling layer includes an improved MobileNet module IMB, which integrates a multi-branch convolutional structure. The input features of IMB first pass through three layers of branches. The first layer of branches includes two Convolution, a Depthwise separable convolution DWconv and two ReLU6 activation functions, the second layer branch includes a Convolution, the third layer branch includes a skip connection, then, after the outputs of the three layers are spliced ​​together, finally, a ReLU6 activation function is passed to obtain the output features of IMB; The corresponding inverse transform is shown below: ； 。 5. The screen content image compression method based on frequency perception feature extraction and entropy model construction according to claim 2, characterized in that: In STB, for input features First, after a layer normalization operation, the window-based multi-head self-attention mechanism and the jump connection splicing, the output features are then subjected to a layer normalization operation, the multi-layer perceptron network and the jump connection splicing, the output features are then subjected to a layer normalization operation, the sliding window-based multi-head self-attention mechanism and the jump connection splicing, the output features are then subjected to a layer normalization operation, the multi-layer perceptron network and the jump connection splicing, and the final output features are obtained. .

6. A screen content image compression method based on frequency-aware feature extraction and entropy model construction according to any one of claims 1 to 5, characterized in that: The frequency-aware entropy model uses two independent entropy parameter networks and , respectively estimate the distribution probability parameters of high-frequency and low-frequency features; specifically, the potential features are parameterized using discrete mixed distributions, where the entropy parameter network and Each includes three Convolutional layer, respectively learns the probability distribution parameters of different frequency features; for low-frequency latent space features The discrete Gaussian mixture model GMM is used to model the entropy model for high-frequency latent space features. Apply discrete Laplace mixture model LMM to model entropy model; ； ； in, and are the weight coefficients of Gaussian mixture distribution and Laplace mixture distribution, , , 、 The number of discrete Gaussian mixture models and discrete Laplace mixture model distributions, and Representing the i Gaussian distribution and j the mean and scale parameter of the Laplace distribution; represents the conditional probability distribution of low-frequency latent features, represents the conditional probability distribution of high-frequency latent features, represents a Gaussian distribution, represents the Laplace distribution.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the screen content image compression method based on frequency perception feature extraction and entropy model construction described in any one of claims 1-6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the screen content image compression method based on frequency perception feature extraction and entropy model construction according to any one of claims 1 to 6 are implemented.

9. A screen content image compression system based on frequency-aware feature extraction and entropy model construction, characterized in that: include: The analysis, transformation and frequency division module is configured to: input the original screen content image, and obtain multi-frequency potential features through analysis, transformation and frequency division; The frequency-aware entropy model and quantization module is configured to: input multi-frequency latent features, obtain quantized multi-frequency latent features through quantization, and perform entropy encoding and entropy decoding on the quantized multi-frequency latent features using the frequency-aware entropy model; The super-prior analysis transformation and frequency division module is configured to: input the multi-frequency potential feature, perform super-prior analysis transformation and frequency division, and obtain the super-prior multi-frequency potential feature; The quantization and entropy coding module is configured to: input the super-prior multi-frequency latent features, perform quantization and entropy coding, obtain the quantized super-prior multi-frequency latent features, and perform entropy coding and entropy decoding on the quantized super-prior multi-frequency latent features; entropy coding and entropy decoding refer to: using an arithmetic encoder AD and an arithmetic decoder AE to implement entropy coding and entropy decoding; The super-prior synthesis transformation frequency division module is configured to: input the quantized super-prior multi-frequency potential features obtained after entropy decoding, perform super-prior synthesis transformation frequency division on them, and obtain the frequency perception entropy model distribution parameters of the multi-frequency potential features; The synthesis transformation frequency division module is configured to: input the quantized multi-frequency potential features obtained after entropy decoding, and obtain the final reconstructed screen content image through synthesis transformation frequency division.

Images