Remote sensing image compression method based on multi-level domain similarity guidance

Abstract

The invention relates to a remote sensing image compression method, in particular to a remote sensing image compression method based on multilevel domain similarity guidance. The problem that the quality of a reconstructed image obtained by an existing method is low is solved. The method comprises the following steps: constructing an MDSNet; comprising an encoder, a decoder and a probability model. The encoder comprises a first two-dimensional convolution layer, a first GDN, a second two-dimensional convolution layer, a second GDN, a third two-dimensional convolution layer, a third GDN, a GIE-AM, a fourth two-dimensional convolution layer, a fourth GDN, a fifth two-dimensional convolution layer, a sixth two-dimensional convolution layer, a seventh two-dimensional convolution layer, an eighth two-dimensional convolution layer and an MDEM. The decoder comprises a first IGDN, a fourteenth two-dimensional convolution layer, a GIE-AM, a second IGDN, a fifteenth two-dimensional convolution layer, a third IGDN, a sixteenth two-dimensional convolution layer, a fourth IGDN, a seventeenth two-dimensional convolution layer, an eighteenth two-dimensional convolution layer, a nineteenth two-dimensional convolution layer and an MDEM. And compressing the to-be-measured remote sensing image to obtain a compressed to-be-measured remote sensing image. The method is applied to the field of remote sensing image compression.

Description

Technical Field

[0001] The invention relates to a remote sensing image compression method based on multi-level domain similarity guidance. Background Art

[0002] Remote sensing images are digital representations of information on the earth's surface. [1][2] ([1] Fang H, Guo S, Wang X, et al. Automatic urban scene-level binary change detection based on a novel sample selection approach and advanced triplet neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-18. [2] Zhao Y, Chen P, Chen Z, et al. A Triple-Stream Network With Cross-Stage Feature Fusion for High-Resolution Image Change Detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-17.) It has many ground features that are difficult to reflect in ordinary images, including surface cover type, terrain, landform, temperature, etc.; therefore, it is widely used in many fields such as environmental monitoring, meteorology, geological science, etc. [3-6]([3]TangW,He F,BashirAK,et al.Aremote sensing image rotation objectdetection approach for real-time environmental monitoring[J].SustainableEnergy Technologies and Assessments,2023,57:103270.[4]Gama PHT,Oliveira HN,Marcato J,et al.Weakly supervised few-shot segmentation via meta-learning[J].IEEE Transactions on Multimedia, 2022.[5]Han W, Zhang capacity and sustainable development under remote sensing interpretation combined with integrated model[J].Ecotoxicology and Environmental Safety, 2023, 257: 114956.). With the rapid development of imaging technology, the spatial resolution and spectral resolution of remote sensing images are constantly improving, and the amount of data is growing exponentially. In addition, due to the influence of factors such as imaging perspective, shooting distance, and atmospheric refraction, remote sensing images have the characteristics of high information entropy, rich texture information, mixed high-frequency and low-frequency features, and weak correlation. These problems make it difficult for traditional image compression methods to effectively compress remote sensing images. [7-9]([7]Xing Z, Yang S, Zan X, et al. Flood vulnerability assessment of urban buildings based on integrating high-resolution remote sensing and street view images[J]. Sustainable Cities and Society, 2023, 92: 104467. [8]Xia GS, Hu J, Hu F, et al. AID: A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981. [9]Lu X, Wang B, Zheng X, et al. Exploring models and data for remote sensing image caption generation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(4): 2183-2195.). Therefore, efficient remote sensing image compression networks are urgently needed to be designed and developed.

[0003] At present, common traditional image compression methods have achieved some results. For example, Báscones et al. proposed a method that combines principal component analysis and JPEG2000

[10] (

[10] JPEG2000official software OpenJPEG, https: / / jpeg.org / jpeg2000 / software.html) combined with the method of compressing hyperspectral image data

[11] (

[11] Báscones D, González C, Mozos D. Hyperspectral image compression using vector quantization, PCA and JPEG2000 [J]. Remote sensing, 2018, 10 (6): 907.). In addition, BPG has better performance

[12] (

[12] F.Bellard.Bpg image format.[Online].Available:http: / / bellard.org / bpg / ) and WebP

[13] (

[13] Maldonado M, WebP Ja new web oriented image format.Universitat Oberta de Catalunya,2010[J].) was also born in the field of image compression. Among them, JPEG

[14] (

[14] Wallace G K.The JPEG still picture compression standard[J].Communications ofthe ACM,1991,34(4):30-44.) and JPEG2000 consists of three parts, including image transformation, quantization and entropy coding. In general, first, the image is compressed and dequantized. Secondly, the valuable information is retained through quantization. Finally, the decorrelation coefficients are compressed using entropy coding. Traditional image compression methods can be divided into algorithms based on vector quantization.

[15] (

[15] Wang Z, Nasrabadi NM, Huang T S. Spatial–spectral classification of hyperspectral images using discriminative dictionary designed by learning vector quantization [J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 52 (8): 4808-4822.), Algorithms based on predictive coding

[16] (

[16] Hu Y, Yang W, Ma Z, et al. Learning end-to-end lossy image compression: A benchmark [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44 (8): 4194-4211.) and transform coding-based algorithms

[17] (

[17] Aulí-Llinàs F, Marcellin MW, Serra-Sagrista J, et al. Lossy-to-lossless 3D image coding through prior coefficient lookuptables [J]. Information Sciences, 2013, 239: 266-282.) A multispectral image compression algorithm that converts the input vector into a codeword index that matches the input vector in the codebook for data transmission and storage

[18] (

[18] QianS E.Hyperspectral data compression using a fast vector quantization algorithm[J].IEEE Transactions on Geoscience and Remote Sensing,2004,42(8):1791-1798.) 3D-MBLP uses a prediction-based technique to first eliminate the spatial redundant information of the image, then predict the frequency band, and finally encode the prediction residual through an entropy decoder.

[19] (

[19] Pizzolante R, Carpentieri B. Multiband and lossless compression of hyperspectral images [J]. Algorithms, 2016, 9 (1): 16.) 3D-SPIHT is a transformation method for three-dimensional image compression, which applies 3D wavelet transform in spatial and spectral domains.

[20] (

[20] Thornton L, Soraghan J, Kutil R, et al. Unequally protected SPIHT video codec for low bit rate transmission over highly error-prone mobile channels [J]. Signal Processing: Image Communication, 2002, 17 (4): 327-335.) However, there are some problems in using traditional image compression methods for remote sensing image compression, such as high computational complexity and lack of targeted processing of remote sensing image characteristics such as high information entropy, rich texture, mixed high-frequency and low-frequency features, and weak correlation. These limit the improvement of remote sensing image compression performance.

[0004] In order to seek breakthroughs, relevant researchers have focused on the hot neural network technology in recent years. The classic deep learning-based image compression framework mainly includes autoencoders (AE)

[21]

[22] (

[21] La Grassa R,Re C,Cremonese G,et al.Hyperspectral data compression using fullyconvolutional autoencoder[J].Remote Sensing,2022,14(10):2472.

[22] Liu J,YuanF,Xue C,et al.An Efficient and Robust Underwater Image Compression SchemeBased on Autoencoder[J].IEEE Journal of Oceanic Engineering, 2023.) and Variational Autoencoder (VAE)

[23]

[24] (

[23] Alves de OliveiraV,Chabert M,Oberlin T,et al.Reduced-complexity end-to-end variational autoencoder for onboard satellite image compression[J].Remote Sensing,2021,13(3):447.

[24] Xu Q,Xiang Y,Di Z,et al.Synthetic aperture radar image compression based on avariational autoencoder[J].IEEE Geoscience and Remote Sensing Letters,2021,19:1-5.). The main reason is that the image compression task mainly includes two symmetrical data processing processes, namely compression and reconstruction. There is a similar and reversible relationship between these two tasks, which means that the network with a symmetric encoding and decoding structure will have better performance. Among them, the VAE-based framework has a more powerful image reconstruction capability. This is because VAE has a continuous mapping space that AE does not have, and can reconstruct images with smooth transitions. In recent years, many excellent VAE-based baseline networks have been proposed.

[25]

[26]

[27]

[28]

[29]

[30]

[31] (

[25] BalléJ,Laparra V,Simoncelli E P.End-to-end optimization ofnonlinear transform codes for perceptual quality[C] / / 2016Picture Coding Symposium(PCS).IEEE,2016:1-5.

[26] Minnen D,BalléJ,TodericiG D.Joint autoregressive and hierarchical priors for learned imagecompression[J].Advances in neural information processing systems,2018,31.

[27] BalléJ,Minnen D,Singh S,et al.Variational image compression with a scalehyperprior[J].arXiv preprint arXiv:1802.01436,2018.

[28] Cheng Z,Sun H,TakeuchiM,et al.Learned image compression with discretized gaussian mixturelikelihoods and attention modules[C] / / Proceedings of the IEEE / CVF conferenceon computer vision and pattern recognition.2020:7939-7948.

[29] Guo Z,Zhang Z,Feng R,et al.Causal contextual prediction for learned image compression[J].IEEE Transactions on Circuits and Systems for Video Technology,2021,32(4):2329-2341.

[30] Chen T,Liu H,Ma Z,et al.End-to-end learnt image compression vianon-local attention optimization and improved context modeling[J].IEEE Transactions on Image Processing, 2021, 30: 3179-3191.

[31] Cao M, Dai W, Li S, et al. End-to-end optimized image compression with deep Gaussian process regression [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022.), achieving rate-distortion performance that is superior to traditional image compression methods. The VAE-based image compression network consists of three parts, including an encoder, entropy coding, and a decoder. This type of compression framework usually uses a neural network to perform preliminary compression on the data block, then maps the pixel data to a quantized representation, and finally uses traditional coding methods to compress the data into the smallest form of existence (bitstream). In addition, some compression models introduce entropy models into the framework, such as Laplace models, single-core Gaussian models, mixed Gaussian models, hierarchical entropy models, etc., to introduce prior information and achieve more accurate modeling.

[26]

[28]

[32]

[33] (

[26] Minnen D,BalléJ,Toderici G D.Joint autoregressive and hierarchicalpriors for learned image compression[J].Advances in neural informationprocessing systems,2018,31.

[27] BalléJ,Minnen D,Singh S,et al.Variationalimage compression with a scale hyperprior[J].arXiv preprint arXiv:1802.01436,2018.

[28] Cheng Z,Sun H,Takeuchi M,et al.Learned image compression withdiscretized gaussian mixture likelihoods and attention modules[C] / / Proceedings of the IEEE / CVF conference on computer vision and patternrecognition.2020:7939-7948.

[29] Guo Z,Zhang Z,Feng R,et al.Causal contextualprediction for learned image compression[J].IEEE Transactions on Circuits andSystems for Video Technology,2021,32(4):2329-2341.

[30] Chen T,Liu H,Ma Z,etal.End-to-end learnt image compression via non-local attention optimizationand improved context modeling[J].IEEE Transactions on Image Processing,2021,30:3179-3191.

[31] Cao M,Dai W,Li S,et al.End-to-end optimized imagecompression with deep Gaussian process regression[J].IEEE Transactions onCircuits and Systems for Video Technology,2022.

[32] Liu D,Sun X,Wu F,etal.Edge-oriented uniformintra prediction[J].IEEE Transactions on ImageProcessing,2008,17(10):1827-1836.

[33] Kong F,Cao T,Li Y,etal.Multi-scalespatial-spectral attention network for multispectral image compression basedon variational autoencoder[J].Signal Processing,2022,198:108589.). As a result, many scholars have adopted compression models based on deep learning to achieve efficient remote sensing image compression.

[34]

[35]

[36]

[37]

[38] (

[34] GuoY,Chong Y,Ding Y,et al.Learned hyperspectral compression using a student's Thyperprior[J].Remote Sensing,2021,13(21):4390.

[35] Xiang S,Liang Q.Remotesensing image compression with long-range convolution and improved non-localattention model[J].Signal Processing,2023,209:109005.

[36] Zhao S,Yang S,Gu J,et al.Symmetrical lattice generative adversarial network for remote sensingimages compression[J].ISPRS Journal of Photogrammetry and Remote Sensing,2021,176:169-181.

[37] Zhang L,Hu X,Pan T,et al.Global Priors with Anchored-stripe Attention and MultiScale Convolution for Remote Sensing ImagesCompression[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023.

[38] Han P, Zhao B, Li X. Edge-Guided Remote Sensing Image Compression [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023.). It is worth mentioning that the common deep learning techniques used for remote sensing image compression mainly include three categories: CNN-based image compression methods, Transformer-based image compression methods, and GAN-based image compression methods.

[39] In (

[39] Tang Z, Wang H, Yi X, et al. Joint graph attention and asymmetric convolutional neural network for deep image compression [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 33 (1): 421-433.), Tang et al. proposed an end-to-end image compression method by integrating graph attention and asymmetric convolutional neural network (ACNN). This method effectively overcomes the problem that traditional convolutional neural networks pay too much attention to local features of the image, promotes information interaction, and fully considers the interdependence and position information between channels. In the literature

[40] In (

[40] Li B, Liang J, Han J. Variable-rate deep image compression with vision transformers [J]. IEEE Access, 2022, 10: 50323-50334.), Li et al. proposed a learning image compression network based on Vision Transformer (ViT). This method divides the input image into blocks and adopts different types of Transformer blocks in the encoder and decoder to achieve efficient image compression. At low bit rates, it achieves good rate-distortion performance. In the literature

[38] , Han et al. proposed a new end-to-end framework, namely edge-guided adversarial network, which aims to preserve sharp edges and texture information at the same time. It optimizes the reconstructed image structure through edge fidelity constraints and guided network to solve the local smoothness problem in existing methods. The above methods achieve good rate-distortion performance, but they all enhance the quality of the reconstructed image from a single factor (graph structure, long-distance context information, edge information), without comprehensively and balancedly improving the quality of the reconstructed image.

[0005] The essence of the remote sensing image compression framework is to improve the similarity between the original image and the reconstructed image. The similarity between the original image and the reconstructed image depends on multiple factors, such as the reconstruction quality of high-frequency information, the recovery of long-distance context information, the impact of noise, the strength of the correlation between different features, etc. How to comprehensively take various factors into consideration and improve the image compression method to improve the quality of remote sensing image reconstruction has become a severe challenge. Summary of the invention

[0006] The purpose of the present invention is to solve the problem of low quality of reconstructed images obtained by existing compression methods, and to propose a remote sensing image compression method based on multi-level domain similarity guidance.

[0007] The specific process of the remote sensing image compression method based on multi-level domain similarity guidance is as follows:

[0008] Step 1: Construct a remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance;

[0009] The remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance includes: encoder, decoder, probability model Probability Model;

[0010] The encoder includes a first 7×7 two-dimensional convolution layer, a first GDN, a second 3×3 two-dimensional convolution layer, a second GDN, a third 3×3 two-dimensional convolution layer, a third GDN, a GIE-AM, a fourth 3×3 two-dimensional convolution layer, a fourth GDN, a fifth 3×3 two-dimensional convolution layer, a sixth 3×3 two-dimensional convolution layer, a seventh 3×3 two-dimensional convolution layer, an eighth 3×3 two-dimensional convolution layer, and an encoding multi-level domain similarity enhancement module MDEM;

[0011] The encoding multi-level domain similarity enhancement module MDEM includes an encoding multi-level feature interaction module MFIM and an encoding multi-channel information extraction module MIEM;

[0012] The encoding multi-level feature interaction module MFIM includes a ninth 1×1 two-dimensional convolutional layer, a tenth 1×1 two-dimensional convolutional layer, and an eleventh 1×1 two-dimensional convolutional layer;

[0013] The encoding multi-channel information extraction module MIEM includes a first 3×3 maximum pooling, a first 3×3 average pooling, and a Softmax activation function layer;

[0014] The GIE-AM includes a first CEB, a second CEB, a third CEB, a fourth CEB, a Sigmod activation function, and a multi-head attention layer MHSA;

[0015] Each of the first CEB, the second CEB, the third CEB, and the fourth CEB includes a twelfth 3×3 two-dimensional convolutional layer and a thirteenth 3×3 two-dimensional convolutional layer in sequence;

[0016] The decoder includes a first IGDN, a fourteenth 3×3 two-dimensional convolutional layer, a GIE-AM, a second IGDN, a fifteenth 3×3 two-dimensional convolutional layer, a third IGDN, a sixteenth 3×3 two-dimensional convolutional layer, a fourth IGDN, a seventeenth 3×3 two-dimensional convolutional layer, an eighteenth 3×3 two-dimensional convolutional layer, a nineteenth 3×3 two-dimensional convolutional layer, and a decoding multi-level domain similarity enhancement module MDEM;

[0017] The decoding multi-level domain similarity enhancement module MDEM includes a decoding multi-channel information extraction module MIEM and a decoding multi-level feature interaction module MFIM

[0018] The encoded multi-channel information extraction module MIEM includes a 20th 1×1 two-dimensional convolutional layer, a 21st 1×1 two-dimensional convolutional layer, and a 22nd 1×1 two-dimensional convolutional layer;

[0019] The decoding multi-level feature interaction module MFIM includes a second 3×3 maximum pooling, a second 3×3 average pooling, and a Softmax activation function layer;

[0020] The GDN represents the generalized split normalization function; the IGDN represents the inverse operation of the GDN;

[0021] Step 2: Compress the remote sensing image to be tested based on the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement to obtain the compressed remote sensing image to be tested.

[0022] The beneficial effects of the present invention are:

[0023] Remote sensing image compression networks are designed to enhance the similarity between input images and reconstructed images. Remote sensing images contain rich channel features and global visual features, however, the current network does not efficiently combine the two. In addition, the present invention proposes a concept of multi-level domain similarity and fully explores how to improve the quality of remote sensing image reconstruction by enhancing this pre-condition multi-level domain similarity. Therefore, the present invention proposes a remote sensing image compression network guided by multi-level domain similarity enhancement (Multi-level domain similarity enhancement-guided remote sensing image compression network, MDSNet). First, an efficient compression baseline network (BaselineA) for remote sensing images is designed, and image compression at low complexity is achieved by reasonably selecting the convolution kernel size and allocating the number of channels. Secondly, a multi-level domain similarity enhancement module (Multi-level domain similarity enhancement module, MDEM) is constructed, which improves the quality of reconstructed images by enhancing the pre-condition multi-level domain similarity. Thirdly, a global information enhanced attention module (Global information enhanced attention module, GIE-AM) is constructed to extract and enhance channel features and global features. Finally, the entire network performs efficient remote sensing image compression and reconstruction under the guidance of the overall loss reconstructed by the proposed multi-level domain similarity enhancement loss (MDEM-Loss) method. Experimental results show that compared with some advanced compression models, the proposed MDSNet method has achieved better compression performance in multiple compression evaluation indicators.

[0024] The present invention proposes a multi-level domain similarity enhancement-guided remote sensing image compression network (MDSNet), which mainly enhances the overall similarity between the original remote sensing image and the reconstructed image (the highest level domain) from the perspective of enhancing the previous multi-level domain similarity, so as to comprehensively improve the quality of remote sensing reconstructed images. MDSNet is mainly analyzed and optimized from three aspects. Aspect one, the essence of the remote sensing image compression network is to enhance the similarity between the input image and the reconstructed image. The present invention enhances the similarity of the highest level domain from the perspective of enhancing the previous multi-level domain similarity. (In the present invention, we regard the compression process of the encoder and the reconstruction process of the decoder as multi-level compression and multi-level reconstruction, respectively, and regard the multi-level image in the compression process and the multi-level image in the reconstruction process as two different domains. The present invention calls the similarity between the compression domain and the reconstruction domain at multiple levels as multi-level domain similarity). First, the present invention designs a multi-level feature interaction module (MFIM) and a multi-channel information extraction module (MIEM), and uses the two to form a multi-level domain similarity enhancement module (MDEM), so as to realize the interaction of multi-channel feature extraction and multi-level features. Secondly, the present invention proposes a multi-level domain similarity enhancement loss (MDEM-Loss) to reconstruct the objective function of the network, so as to guide the compression model to perform high-quality remote sensing image compression and reconstruction. Aspect two, remote sensing images contain rich global information and channel features. If such information is ignored, the model may lose the understanding of important contexts such as geomorphic features, and the current mainstream methods do not realize the efficient extraction and combination of these two types of information. In the present invention, a channel extrusion block (CEB) is first designed to efficiently enhance the channel information. In addition, the present invention adopts the corresponding multi-head self-attention (MHSA) to efficiently extract global information.Finally, based on these two modules, a global information enhanced attention module (GIE-AM) is constructed to enhance the extraction and combination of channel features and global features. Aspect three, CNN has a powerful feature extraction capability.

[41]

[42] (

[41] Ma H, Liu D, Xiong R, et al. iWave: CNN-based wavelet-like transform for image compression [J]. IEEE Transactions on Multimedia, 2019, 22 (7): 1667-1679.

[42] Fu C, Du B, Zhang L. SAR Image Compression Based on Multi-Resblock and Global Context [J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 1-5.), researchers control the range of the receptive field by adjusting the size of the convolution kernel, thereby adjusting the network's emphasis on extracting high- and low-frequency information. However, high-quality global information extraction means the selection of larger convolution kernels, which will lead to a sharp increase in the number of parameters. In addition, common networks do not effectively analyze and process the data distribution of images, that is, there is a lack of research on the potential relationship between the number of channels and image size during compression and reconstruction. Therefore, the present invention designs an efficient remote sensing image compression baseline network (BaselineA), which takes advantage of the characteristics of remote sensing images such as high information entropy, rich texture information, mixed high-frequency and low-frequency features, and weak correlation, and achieves high-quality image compression and reconstruction under the premise of low complexity through reasonable convolution kernel size selection and reallocation of channel numbers.

[0025] In summary, the present invention constructs a low-complexity, high-performance MDSNet based on the proposed MFIM, MIEM, MDEM, MDEM-Loss, CEB, MHSA, GIE-AM and BaselineA.

[43] (

[43] (https: / / resources.maxar.com / product-samples / analysis-ready-data-san-francisco-california)NWPU-RESISC45

[44] (

[44] Cheng G, Han J, Lu

[45] A large number of experiments have been conducted on remote sensing image datasets such as (

[45] Y.Yang and S.Newsam, "Bag-of-visual-words and spatial extensions for land-use classification," in ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM GIS), 2010.) Experimental results show that compared with the traditional image compression method JPEG2000

[10] , BPG

[12] and WebP

[13] , and image compression methods based on deep learning Minnen ta al.

[26] 、Minnen et al.(mean)

[26] 、Balle et al.(hyperprior)

[27] , Balle et al.(factorized-relu)

[27] 、Tong2023

[46] (

[46] Tong K, Wu Y, Li Y, et al. QVRF: A Quantization-error-aware Variable Rate Framework for Learned Image Compression [J]. arXiv preprint arXiv: 2303.05744, 2023.), the MDSNet proposed in this paper performs better in terms of evaluation indicators such as Peak Signal-to-Noise Ratio (PSNR) and Multiscale Structural Similarity Index Metric (MS-SSIM). In addition, this paper also verifies the effectiveness of MFIM, MIEM, MDEM, MDEM-Loss, CEB, MHSA, GIE-AM and BaselineA through a variety of ablation experiments.

[0026] The main contributions of the present invention are as follows:

[0027] 1) The present invention proposes a multi-level domain similarity enhancement module (MDEM), which includes two core components, MFIM and MIEM, which can realize the interaction of multi-channel feature extraction and multi-level features; in addition, the present invention proposes a multi-level domain similarity enhancement loss (MDEM-Loss) for reconstructing the objective function of the entire network. Finally, the similarity between the highest-level domains is enhanced by enhancing the similarity between the front-end multi-level domains.

[0028] 2) The present invention designs a global information enhanced attention module (GIE-AM), whose core components mainly include a channel extrusion block (CEB) and multi-head self-attention (MHSA), which are used to efficiently enhance the extraction and effective fusion of channel features and global features.

[0029] 3) This paper designs an efficient compression baseline network (BaselineA) for remote sensing images. The network takes into account the characteristics of remote sensing images, such as high information entropy, rich texture, mixed high-frequency and low-frequency features, and weak correlation. By cleverly selecting the convolution kernel size and redistributing the number of channels, it achieves excellent compression performance at low complexity.

[0030] 4) This paper effectively embeds MFIM, MIEM, MDEM, MDEM-Loss, CEB, MHSA, GIE-AM and Factorized Entropy Model into Baseline A, and constructs a low-complexity, high-performance multi-level domain similarity enhancement-guided remote sensing image compression network (MDSNet). Through a large number of experiments on San Francisco, NWPU-RESISC45 and UC-Merced, the superior compression performance of MDSNet for remote sensing images is fully demonstrated. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The overall structure diagram of MDSNet is proposed for the present invention; Figure 2 The structural diagram of MDEM proposed in the present invention; Figure 3 The schematic diagram of the structure of GIE-AM proposed in the present invention; Figure 4 A BaselineA schematic diagram is proposed for the present invention;

[0032] Figure 5 This is a sample image of San Francisco, including (a) houses, (b) coastline, (c) roads, (d) basketball courts, (e) tennis courts, (f) ports, (g) parking lots, (h) forests, (i) farmland, and (j) lakes.

[0033] Figure 6 These are NWPU-RESISC45 sample images: (a) airport, (b) basketball court, (c) beach, (d) bridge, (e) desert, (f) church, (g) cloud, (h) forest, (i) port, and (j) island.

[0034] Figure 7 The following are sample images of UC-Merced: (a) farmland, (b) airplane, (c) baseball field, (d) beach, (e) building, (f) forest, (g) highway, (h) golf course, (i) port, and (j) overpass.

[0035] Figure 8 Rate-distortion performance curves of different methods on San Francisco: (a) PSNR curve, (b) MS-SSIM curve;

[0036] Fig. 9 Rate-distortion performance curves of different methods on NWPU-RESISC45: (a) PSNR curve, (b) MS-SSIM curve;

[0037] Fig.10 Rate-distortion performance curves of different methods on UC-Merced: (a) PSNR curve, (b) MS-SSIM curve;

[0038] Fig.11 The difference loss diagrams of each level on the San Francisco dataset: (a) first-level difference loss, (b) second-level difference loss, and (c) third-level difference loss.

[0039] Fig.12 Loss diagrams of each level of domains on the dataset NWPU-RESISC45: (a) first-level difference loss, (b) second-level difference loss, (c) third-level difference loss;

[0040] Fig.13Visual comparison of the reconstructed images obtained by different methods on the San Francisco dataset, (a) original image, (b) Minnen et al. (bpp: 0.297; PSNR: 31.12; MS-SSIM: 9.48), (c) Minnen et al. (mean) (bpp: 0.298; PSNR: 30.93; MS-SSIM: 9.44), (d) Balle et al. (hyperprior) (bpp: 0.294 PSNR: 30.93 MS-SSIM:

[0041] 9.52), (e) Balle et al. (factorized-relu) (bpp: 0.295; PSNR: 29.41; MS-SSIM: 8.86), (f) Tong2023 (bpp: 0.303; PSNR: 30.92; MS_SSIM: 9.40), (g) JPEG2000 (bpp: 0.309; PSNR: 22.73; MS-S SIM:2.46), (h) Webp (bpp: 0.413; PSNR: 24.48; MS-SSIM: 4.39), (i) BPG (bpp: 0.307; PSNR: 24.65; MS-SSIM: 3.97), (j) MDSNet (bpp: 0.301; PSNR: 31.36; MS-SSIM: 9.86);

[0042] Fig.14Visual comparison of the reconstructed images obtained by different methods on the dataset NWPU-RESISC45, (a) original image, (b) Minnen et al. (bpp: 0.260; PSNR: 33.26; MS-SSIM: 7.73), (c) Minnen et al. (mean) (bpp: 0.257; PSNR: 33.20; MS-SSIM: 7.67), (d) Balle et al. (hyperprior) (bpp: 0.263; PSNR: 33.30; MS-SSIM: 7.64), (e) Balle et al. (factorized-relu) (bpp: 0.261; PSNR: 31.87; MS-SSIM: 7.49), (f) To ng2023 (bpp: 0.263; PSNR: 33.57; MS-SSIM: 7.76), (g) JPEG2000 (bpp: 0.257; PSNR: 22.40; MS-SSI M:1.92), (h) Webp (bpp: 0.249; PSNR: 22.69; MS-SSIM: 2.13), (i) BPG (bpp: 0.259; PSNR: 24.26; MS-SSIM: 3.72), (j) MDSNet (bpp: 0.262; PSNR: 33.92; MS-SSIM: 8.02);

[0043] Fig.15 This is the OA value diagram of the reconstructed images obtained by different compression methods in remote sensing scene classification (the dataset used is NWPU-RESISC45). DETAILED DESCRIPTION

[0044] Specific implementation method 1: The specific process of the remote sensing image compression method based on multi-level domain similarity guidance in this implementation method is as follows:

[0045] Step 1: Construct a multi-level domain similarity enhancement-guided remote sensing image compression network MDSNet (Multi-level domain similarity enhancement-guided remote sensing image compression network);

[0046] The remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance includes: encoder, decoder, probability model Probability Model;

[0047] The encoder includes a first 7×7 two-dimensional convolutional layer (Conv2D, 7×7, 3, N / 4, 2, ↓), a first GDN, a second 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 4, N / 2, 2, ↓), a second GDN, a third 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4, 2, ↓), a third GDN, GIE-AM, a fourth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N, 2, ↓), and a fourth GD N, the fifth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2, 2, ↑), the sixth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, N / 4, 2, ↑), the seventh 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 4, N / 2, 2, ↓), the eighth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4, 2, ↓), and the encoding multi-level domain similarity enhancement module MDEM (Multi-level domainsimilarity enhancement module);

[0048] The encoding multi-level domain similarity enhancement module MDEM includes an encoding multi-level feature interaction module MFIM (Multi-level feature interaction module) and an encoding multi-channel information extraction module MIEM (Multi-channel information extraction module);

[0049] The encoding multi-level feature interaction module MFIM includes a ninth 1×1 two-dimensional convolutional layer, a tenth 1×1 two-dimensional convolutional layer, and an eleventh 1×1 two-dimensional convolutional layer;

[0050] The encoding multi-channel information extraction module MIEM includes a first 3×3 maximum pooling, a first 3×3 average pooling, and a Softmax activation function layer;

[0051] The GIE-AM includes a first CEB, a second CEB, a third CEB, a fourth CEB, a Sigmod activation function, and a multi-head attention layer MHSA;

[0052] Each of the first CEB, the second CEB, the third CEB, and the fourth CEB includes a twelfth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2) and a thirteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4) in sequence;

[0053] Conv2D 1×1, N / 2, N / 4, 2, ↓ represents the parameter settings of the convolution layer, where 1×1 represents the kernel size of the convolution layer, N / 2 represents the number of input channels, N / 4 represents the number of output channels, 2 represents the stride, ↓ represents downsampling, ↑ represents upsampling, and the parameter settings of other convolution layers are the same as above.

[0054] The decoder includes a first IGDN, a fourteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N, 3N / 4, 2, ↑), a GIE-AM, a second IGDN, a fifteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2, 2, ↑), a third IGDN, a sixteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, N / 4, 2, ↑), a fourth IGDN, a seventeenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 4, 3, 2, ↑), an eighteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2, 2, ↑), a nineteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, N / 4, 2, ↑), and a decoding multi-level domain similarity enhancement module MDEM (Multi-level domainsimilarity enhancement module);

[0055] The decoding multi-level domain similarity enhancement module MDEM includes a decoding multi-channel information extraction module MIEM (Multi-channel information extraction module) and a decoding multi-level feature interaction module MFIM (Multi-level feature interaction module)

[0056] The encoded multi-channel information extraction module MIEM includes a 20th 1×1 two-dimensional convolutional layer, a 21st 1×1 two-dimensional convolutional layer, and a 22nd 1×1 two-dimensional convolutional layer;

[0057] The decoding multi-level feature interaction module MFIM includes a second 3×3 maximum pooling, a second 3×3 average pooling, and a Softmax activation function layer;

[0058] The GDN represents the generalized split normalization function; the IGDN represents the inverse operation of the GDN;

[0059] Step 2: Compress the remote sensing image to be tested based on the multi-level domain similarity enhancement-guided remote sensing image compression network MDSNet (Multi-level domain similarity enhancement-guided remote sensing image compression network) to obtain the compressed remote sensing image to be tested.

[0060] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the specific processing process of the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement in step 1 is:

[0061] Step 1. The remote sensing image x is sequentially input into the first 7×7 two-dimensional convolutional layer (Conv2D, 7×7, 3, N / 4, 2, ↓) and the first GDN, and the first GDN outputs the feature InputB;

[0062] The first GDN output feature InputB is input into the encoding multi-channel information extraction module MIEM, and the encoding multi-channel information extraction module MIEM outputs feature D;

[0063] The first GDN outputs feature InputB, which is sequentially input into the second 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 4, N / 2, 2, ↓), and the second GDN outputs feature A.

[0064] The second GDN output feature B is sequentially input into the third 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4, 2, ↓), the third GDN, and the third GDN outputs feature C;

[0065] The third GDN output feature C is sequentially input into the fifth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2, 2, ↑) and the sixth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, N / 4, 2, ↑). The sixth 3×3 two-dimensional convolutional layer outputs feature InputA.

[0066] The sixth 3×3 two-dimensional convolutional layer outputs the feature InputA input encoding multi-level feature interaction module MFIM;

[0067] The output feature D of the coded multi-channel information extraction module MIEM is input into the coded multi-level feature interaction module MFIM;

[0068] Encode the output feature E of the multi-level feature interaction module MFIM;

[0069] The output feature D of the encoding multi-channel information extraction module MIEM is added element by element with the output feature E of the encoding multi-level feature interaction module MFIM to obtain the feature OutputA;

[0070] The output feature E of the encoding multi-level feature interaction module MFIM is sequentially input into the seventh 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 4, N / 2, 2, ↓) and the eighth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4, 2, ↓), and the eighth 3×3 two-dimensional convolutional layer outputs feature F;

[0071] The eighth 3×3 two-dimensional convolutional layer output feature F and the third GDN output feature C are element-wise added to obtain feature G;

[0072] Feature G is input into GIE-AM, and GIE-AM outputs feature H;

[0073] The feature H is sequentially input into the fourth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N, 2, ↓) and the fourth GDN, and the fourth GDN outputs the feature y;

[0074] Feature y is input into the probability model Probability Model, and the probability model Probability Model outputs arithmetic encoding AE and arithmetic decoding AD respectively;

[0075] The features y are sequentially input into the Q quantizer, and the Q quantizer outputs the features feature Input AE arithmetic coding and AD arithmetic decoding in sequence, and AD arithmetic decoding outputs features

[0076] AD arithmetic decoding output characteristics Input the first IGDN and the fourteenth 3×3 two-dimensional convolutional layer in sequence, and the fourteenth 3×3 two-dimensional convolutional layer outputs feature J;

[0077] The feature J input is sequentially input into the eighteenth 3×3 two-dimensional convolutional layer and the nineteenth 3×3 two-dimensional convolutional layer, and the nineteenth 3×3 two-dimensional convolutional layer outputs the feature InputD;

[0078] Feature J is input into GIE-AM, and GIE-AM outputs feature K;

[0079] Feature K is sequentially input into the second IGDN and the fifteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2, 2, ↑), and the fifteenth 3×3 two-dimensional convolutional layer outputs feature L;

[0080] The feature L is sequentially input into the third IGDN and the sixteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, N / 4, 2, ↑), and the sixteenth 3×3 two-dimensional convolutional layer outputs the feature InputC;

[0081] The feature InputC is input into the decoding multi-channel information extraction module MIEM, and the decoding multi-channel information extraction module MIEM outputs the feature N;

[0082] Feature N input decoding multi-level feature interaction module MFIM;

[0083] Feature InputD input decoding multi-level feature interaction module MFIM;

[0084] Decode the multi-level feature interaction module MFIM output feature O;

[0085] Decode the multi-level feature interaction module MFIM output feature O and the sixteenth 3×3 two-dimensional convolutional layer output feature InputC element by element to obtain feature P;

[0086] Feature P is input into the fourth IGDN and the seventeenth 3×3 two-dimensional convolutional layer in sequence, and the seventeenth 3×3 two-dimensional convolutional layer outputs feature

[0087] The feature O output by the decoding multi-level feature interaction module MFIM and the feature N output by the decoding multi-channel information extraction module MIEM are added element by element to obtain the feature OutputB.

[0088] The other steps and parameters are the same as those in the first embodiment.

[0089] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the first GDN output feature InputB is input into the encoding multi-channel information extraction module MIEM, and the encoding multi-channel information extraction module MIEM outputs feature D; the specific processing process is:

[0090] The feature InputB is respectively input into the ninth 1×1 two-dimensional convolutional layer, the tenth 1×1 two-dimensional convolutional layer, and the eleventh 1×1 two-dimensional convolutional layer; the output features of the ninth 1×1 two-dimensional convolutional layer, the tenth 1×1 two-dimensional convolutional layer, and the eleventh 1×1 two-dimensional convolutional layer are added element by element to obtain feature D.

[0091] The other steps and parameters are the same as those in the first or second embodiment.

[0092] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that the sixth 3×3 two-dimensional convolutional layer output feature InputA inputs the encoded multi-level feature interaction module MFIM;

[0093] The output feature D of the coded multi-channel information extraction module MIEM is input into the coded multi-level feature interaction module MFIM;

[0094] Encode the output feature E of the multi-level feature interaction module MFIM;

[0095] The specific processing process is as follows:

[0096] The sixth 3×3 2D convolutional layer outputs the feature InputA which is input into the first 3×3 maximum pooling;

[0097] The output feature D of the encoding multi-channel information extraction module MIEM is input into the first 3×3 average pooling;

[0098] Add the first 3×3 maximum pooling and the first 3×3 average pooling output features element by element to obtain feature ζ;

[0099] Feature ζ is input into the Softmax activation function layer, and the output feature of the Softmax activation function layer is Hadamard-producted with feature ζ to obtain feature E.

[0100] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0101] Specific implementation method 5: This implementation method is different from the specific implementation methods 1 to 4 in that the feature G is input into GIE-AM, and GIE-AM outputs feature H; the specific processing process is:

[0102] The feature G is input into the first CEB, the second CEB, and the Sigmod activation function in sequence, and the Sigmod activation function outputs the feature x 2 ;

[0103] Feature G is input into the third CEB and the fourth CEB in sequence, and the fourth CEB outputs feature x 1 ;

[0104] The feature x 1 and feature x 2 Perform Hadamard product to get the characteristic x 3 ;

[0105] Features 3 Add element-by-element with feature G to get feature x CBE ;

[0106] Features CBE Input the multi-head attention layer MHSA, and the multi-head attention layer MHSA outputs the feature H;

[0107] Each of the first CEB, the second CEB, the third CEB, and the fourth CEB sequentially includes a twelfth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2) and a thirteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4).

[0108] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0109] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that the feature InputC is input into the decoding multi-channel information extraction module MIEM, and the decoding multi-channel information extraction module MIEM outputs the feature N; the specific processing process is:

[0110] The feature InputC is input into the 20th 1×1 2D convolutional layer, the 21st 1×1 2D convolutional layer, and the 22nd 1×1 2D convolutional layer respectively;

[0111] The output features of the twentieth 1×1 two-dimensional convolutional layer, the output features of the twenty-first 1×1 two-dimensional convolutional layer, and the output features of the twenty-second 1×1 two-dimensional convolutional layer are added element by element to obtain feature N.

[0112] The other steps and parameters are the same as those in Specific Implementations 1 to 5.

[0113] Specific embodiment 7: This embodiment differs from any one of specific embodiments 1 to 6 in that the feature N is input into a decoding multi-level feature interaction module MFIM;

[0114] Feature InputD input decoding multi-level feature interaction module MFIM;

[0115] Decode the multi-level feature interaction module MFIM output feature O;

[0116] The specific processing process is as follows:

[0117] Feature N inputs the second 3×3 maximum pooling; feature InputD inputs the second 3×3 average pooling;

[0118] Add the output features of the second 3×3 maximum pooling and the second 3×3 average pooling element by element to obtain feature ξ;

[0119] The feature ξ is input into the Softmax activation function layer, and the output feature of the Softmax activation function layer is Hadamard-producted with the feature ζ to obtain the feature O.

[0120] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0121] Specific implementation eight: This implementation is different from any one of specific implementations one to seven in that the feature J is input into GIE-AM, and GIE-AM outputs feature K; the specific processing process is as follows:

[0122] Feature J is input into the first CEB, the second CEB, and the Sigmod activation function in sequence, and the Sigmod activation function outputs feature x′ 2 ;

[0123] Feature J is input into the third CEB and the fourth CEB in sequence, and the fourth CEB outputs feature x1 ′;

[0124] The feature x 1 ′ and feature x′ 2 Perform Hadamard product to get the characteristic x 3 ′;

[0125] Features 3 ′ is added element by element with feature J to obtain feature x′ CBE ;

[0126] Feature x′ CBE Input the multi-head attention layer MHSA, and the multi-head attention layer MHSA outputs feature K;

[0127] Each of the first CEB, the second CEB, the third CEB, and the fourth CEB sequentially includes a twelfth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, 3N / 4, N / 2) and a thirteenth 3×3 two-dimensional convolutional layer (Conv2D, 3×3, N / 2, 3N / 4).

[0128] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0129] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that in step 2, the remote sensing image to be measured is compressed by a multi-level domain similarity enhancement-guided remote sensing image compression network MDSNet (Multi-level domain similarity enhancement-guided remote sensing image compression network) to obtain a compressed remote sensing image to be measured; the specific process is:

[0130] Step 21: Obtain a trained remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance;

[0131] The loss function of the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement is set as Loss Total , obtain the trained remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance;

[0132] The training process is:

[0133] argminLoss Total =R+λ(D+ψLossδ)

[0134] Where ψ represents the weight coefficient of multi-level domain loss, Lossδ represents MDEM-Loss; D represents the distortion between the original image and the reconstructed image, and different bit rates can be controlled by adjusting the penalty coefficient λ; λ represents adjusting the penalty coefficient to control different bit rates, and the bit rate is determined by the potential representation information and side information Together, R represents the entropy rate, which is the cross entropy between the underlying marginal distribution and the learned entropy model.

[0135] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0136] Specific implementation method 10: This implementation method is different from the specific implementation methods 1 to 9 in that the working process of Lossδ is expressed as follows:

[0137] MDEM-Loss=L MSE (I MFIM(Encoder) ,I MFIM(Decoder) )

[0138] Where, L MSE Represents the loss measured by MSE. In the present invention, when describing the overall loss, for convenience, Lossδ is used to represent MDEM-Loss;

[0139] I MFIM(Encoder) Represents the feature OutputA; I MFIM(Decoder) Represents feature OutputB.

[0140] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0141] A. Overall framework of the proposed MDSNet

[0142] The proposed MDSNet approaches the task of remote sensing image compression from a new perspective of enhancing the similarity between the previous multi-level domains to improve the similarity between the highest level domains. It mainly achieves high-quality remote sensing image compression through the efficient frameworks of BaselineA, MDEM and GIE-AM. MDEM contains three submodules, namely MFIM, MIEM and MDEM-Loss. GIE-AM contains two submodules, namely CEB and MHSA; in addition, the present invention constructs the overall loss function Loss total , in order to coordinate the compression and reconstruction work between the basic network and the proposed module, so as to achieve higher compression performance. MDSNet uses VAE as the basic framework. Its overall structure is shown in the figure Figure 1As shown in the figure, MDSNet mainly includes the following parts: encoder, decoder, multi-level domain similarity enhancement loss, ProbabilityModel and traditional encoding and decoding part. In the image compression part, the encoder maps the remote sensing image data block into a preliminarily compressed potential representation feature by combining MFIM, MIEM, GIE-AM and two-dimensional convolutional layers of different scales and different numbers of channels. Then, statistical redundancy is further removed through quantization, arithmetic coding and ProbabilityModel to obtain the bit stream after data processing of this model. In the image decompression part, the model combines the mean μ in ProbabilityModel i and scale σ i 2 Parameters, through arithmetic decoding, GIE-AM, MFIM, MIEM and two-dimensional convolutional layers of different scales and different numbers of channels, the bit stream is processed to reconstruct the image with high quality. All the above work is to build the loss function Loss total Under the guidance of Figure 1 In, L e and L d They are the main encoder and main decoder, respectively, used to learn the potential discrete representation features of remote sensing images. ProbabilityModel mainly includes a hyper-prior network (hyper-encoder (HyperEncoder), hyper-decoder (HyperDecoder)), Q quantizer, AE arithmetic coding and AD arithmetic decoding. Between AE and AD is the minimum form (bit stream) of data in this model. In the present invention, the hyper-prior network is used to learn the probability model (i.e., entropy model) on which entropy coding depends, and is also used to generate the parameters of the entropy model (i.e., the mean parameter μ i and the scale parameter σ i 2), the entropy model is modeled as a conditional Gaussian. The modules on the left and right represent MDEM, mainly including MFIM, MIEM, MDEM-Loss and various convolutional layers. MDEM-Loss is composed of the loss between the multi-level feature maps obtained by the MDEM in the compression part and the multi-level feature maps obtained by the MDEM in the reconstruction part. The loss used here is MeanAquaredError (MSE). Lossδ is used to represent MDEM-Loss in this paper. The Multi-levellosses part in the middle represents: when in the same level of domain, the difference between the feature maps of each level of compression domain and the feature maps of each level of reconstruction domain. In the experiment, MSE is used to measure the difference between such feature maps. The smaller the value, the more similar the compression domain and the reconstruction domain at this level are. Among them, Lossα represents the difference between the first-level domains, which is called the first-level difference loss in this paper. Similarly, Lossβ represents the second-level difference loss, and Lossγ is called the third-level difference loss. These three-level losses are collectively referred to as the pre-domain loss. The difference between the original image and the reconstructed image is called the zeroth-level difference loss or the highest-level difference loss. MSE can be expressed as:

[0143]

[0144] Where m represents the number of pixels. represents the reconstructed image and X represents the original image.

[0145] GIE-AM stands for Global Information Enhanced Attention Module, which mainly includes CEB and MHSA. N stands for the number of channels, ↓ stands for downsampling, ↑ stands for upsampling, and RELU stands for linear rectification function. GDN stands for Generalized Divided Normalization Function, and IGDN stands for the inverse operation of GDN. They are nonlinear activation functions and are more suitable for normalization of image data than other normalization functions.

[0146] Specifically, according to the characteristics of remote sensing images such as complex background, rich information, and weak correlation between features, the present invention studies a low-complexity and efficient remote sensing image baseline network (BaselineA), MDEM based on multi-level domain similarity enhancement, and GIE-AM based on enhanced channel features and global features. Compared with the common remote sensing image compression model, BaselineA adopts a smaller convolution kernel size as a whole to greatly reduce the number of parameters. At the same time, in the design of the main encoder, the present invention adopts a larger convolution kernel to compensate for the loss of important long-distance context information caused by the use of small-size convolution kernels in the overall framework. In addition, the present invention reallocates the number of channels of the convolution layer in the overall framework so as to extract the most appropriate features at the appropriate compression level. MDEM is a multi-level domain feature map extraction module used to cooperate with MDEM-Loss to enhance the similarity between the front multi-level domains. It mainly includes two core components: MIEM for multi-channel feature extraction and MFIM for multi-level feature interaction. Among them, MIEM is in the first-level domain, with a large feature map and more spatial information. Therefore, MIEM uses multiple point convolution layers to extract and superimpose spatial information to achieve the purpose of enhancing spatial information, while MFIM uses AvgPool2d and MaxPool2d to pool the third-level domain feature map obtained after compressing channel features and the first-level domain feature map obtained by MIEM, respectively, and then fuses and strengthens the pooled feature maps through attention mechanism and residual operation, and finally obtains multi-level domain feature maps that can be used to feed back to the network and calculate MDEM-Loss. In addition, GIE-AM is mainly used to efficiently enhance the extraction of channel features and global features. Its core components include CEB for strengthening channel features and MHSA for enhancing the ability to extract global context information. Among them, CEB combines attention mechanism and residual operation to reduce and increase the dimension of high-dimensional feature maps to strengthen the channel information in remote sensing images and obtain feature maps for input into MHSA. MHSA adopts a self-attention mechanism that can efficiently model long-distance information, and adopts a multi-head form to speed up the training speed of the network and the fusion of information in different subspaces. Overall, MDSNet efficiently explores the potential relationship between remote sensing image compression and reconstruction through BaselineA, MDEM and GIE-AM, thereby achieving high efficiency and high quality of compression and reconstruction.

[0147] B. MDEM

[0148] At present, common remote sensing image compression methods are improved from one or several angles, such as strengthening the extraction of high-frequency information, improving the quality of global visual features, removing complex background noise of remote sensing images, enhancing the correlation between features, reducing spatial and spectral redundant information, etc. However, these methods are not able to compress and reconstruct various features of remote sensing images at the same time very efficiently. Therefore, the present invention starts from the perspective of improving the overall similarity between the original remote sensing image and the reconstructed image, and achieves the purpose of enhancing the highest-level domain similarity by improving the previous multi-level domain similarity. To this end, the present invention constructs a method for enhancing multi-level domain similarity;

[0149] Figure 2 Schematic diagram of the structure of MDEM, where InputA represents the three-level reconstruction domain feature map, InputB represents the first-level reconstruction domain feature map, InputC represents the first-level compression domain feature map, InputD represents the three-level compression domain feature, OutputA represents the multi-level domain feature map of the reconstruction domain, OutputB represents the multi-level domain feature map of the compression domain, MDEM ofEncoder represents the MDEM of the main encoder part, MDEM ofDecoder represents the MDEM of the main decoder part, and MDEM-Loss represents the loss between the multi-level domain feature map of the compression part and the long feature map of the reconstruction part. This loss is measured by MSE. MDEM mainly includes three core components, namely MIEM for extracting multi-channel information, MFIM for multi-level feature interaction, and MDEM-Loss for guiding the network to improve the multi-level domain similarity. The working process of MIEM can be expressed as:

[0150] I MIEM =W CONV1×1 * Input+W CONV1×1 * Input+W CONV1×1 *Input(2)

[0151] In the formula, Input represents the input image data block, * represents convolution, W represents the weight parameter of point convolution, I MIEM Represents the output of MIEM.

[0152] The working process of MFIM in the main encoder part can be expressed as:

[0153]

[0154] In the formula, I MIEM(Encoder) represents the output of the main encoder part MIEM, W represents the weight parameters of each convolutional layer, Softmax layer and Pooling layer, InputD represents the three-level compressed domain features, represents element-by-element addition, ⊙ represents the Hadamard product, I MFIM(Encoder)Representing the multi-level domain feature maps of the main encoder part.

[0155] The working process of the MFIM of the main decoder part is similar to that of the MFIM of the main encoder part. The difference is that the two input positions of the MFIM are swapped. The working process can be expressed as:

[0156]

[0157] In the formula, InputA represents the three-level reconstruction domain feature map, I MFIM(Decoder) Multi-level domain feature maps representing the main decoder part.

[0158] The working process of MDEM-Loss can be expressed as:

[0159] MDEM-Loss=L MSE (I MFIM(Encoder) ,I MFIM(Decoder) )(5)

[0160] Where, L MSE Represents the loss measured by MSE. In the present invention, when describing the overall loss, for convenience, Lossδ is used to represent MDEM-Loss.

[0161] The overall structure of MDEM is as follows Figure 2As shown. It mainly includes the following parts: MIEM, MFIM and MDEM-Loss. Among them, the main body of the MIEM part uses three point convolutions to extract and enhance multi-channel information. The reason why MIEM uses multi-channels instead of single channels to extract spatial information is that the input of MIEM is the feature map on the first-level domain, which has a large size, which leads to more spatial information to be extracted. Therefore, the low amount of information in a single channel will lead to the loss of some spatial information. The reason for using element-by-element addition to process multi-channel information is that the similarity between the first-level domain feature maps has the greatest impact on the similarity between the original remote sensing image and the reconstructed image. When the first-level domain feature information processed by MIEM and the third-level domain feature information are interacted later, it is necessary to give priority to ensuring that the first-level domain feature map has a high proportion in the output of MFIM (multi-level domain feature map). Therefore, the element-by-element addition method is used to further enhance the spatial information on the first-level domain feature map. Among them, the core function of MFIM is to efficiently model the potential relationship between the first-level domain feature information and the third-level domain feature information processed by MIEM, and realize the effective fusion of the two, so as to obtain a multi-level domain feature map with the first-level domain feature as the main feature and the second-level domain feature and the third-level domain feature as the auxiliary feature. First, the first-level domain feature information and the third-level domain feature information enhanced by MIEM are extracted by AvgPooling2d and MaxPooling2d for average information and maximum information, and then the information on different levels of domains is fused by element-by-element addition. After that, the corresponding attention coefficient is obtained by Softmax, and the fused information on different levels of domains is multiplied by the attention coefficient, so as to increase the distinguishability between different features. Finally, in order to ensure the proportion of the first-level domain feature in the multi-level domain feature, the present invention introduces a branch of the first-level domain feature information enhanced by MIEM at the end of MFIM. Finally, MDEM calculates the loss between the multi-level domain feature map of the main encoder part and the multi-level domain feature map of the main decoder part by using MSE, i.e. MDEM-Loss (Lossδ). In addition, in order to control the influence of MDEM-Loss on the final network performance, the present invention applies a weight coefficient (similarity weight ψ) to Lossδ and introduces it into the overall loss, which will be described in detail in the rate-distortion optimization section.

[0162] C:GIE-AM

[0163] There is a wealth of global information and channel information in remote sensing images. However, the existing remote sensing image methods either design a separate channel attention to enhance the channel information, or design a corresponding global attention module to enhance the global visual features. They do not efficiently explore and fuse the potential relationship between the channel information and the global context information. Therefore, the present invention designs a GIE-AM that efficiently enhances and fuses the channel information and the global context information, and its core components include Attention process of CEB and MHSA. Among them, Attention process of CEB is essentially a channel attention; remote sensing images contain a large amount of channel information. Therefore, in neural networks, remote sensing image data is usually mapped from low dimensions to high dimensions to explore the potential relationship between channels. However, the number of bands of remote sensing images themselves is not as many as hundreds of bands as hyperspectral images. If it is mapped to too high a dimension, the channel information features will be attenuated and faded, which leads to the contradiction that the remote sensing image data needs high-dimensional mapping, but cannot be efficiently mapped. Therefore, the present invention designs CEB, which first compresses the channel features and then maps them to high-dimensional operations. The working process of Attentionprocess ofCEB can be expressed as:

[0164] Output=Input+W CEB (W CEB (Input)+W Sigmoid (W CEB (W CEB (Input))(6)

[0165] In the formula, Input represents the input remote sensing image data block, W CEB Represents the weight parameter of CEB, W Sigmoid Represents the parameters of the Sigmoid layer.

[0166] Another core component is MHSA: In recent years, Vision Transformer (ViT) has been widely used in the field of computer vision. Its excellent ability to capture long-distance contextual information enables the model introduced with ViT to show excellent performance at all levels of tasks. This powerful context feature capture capability mainly comes from its core module (self-attention machine). Multi-head self-attention (MHSA) is based on self-attention and introduces a multi-head mechanism to achieve faster training speed and information fusion in different subspaces. The structural diagram of MHSA is shown below. Figure 3 As shown in Figure 2, MHSA obtains global semantic features through modeling;

[0167] Figure 3Schematic diagram of the structure of GIE-AM, the left half of which is the Attention process of CEB, and the right half is MHSA. Conv2D 3×33N / 4N / 2 represents the parameter setting of convolution, where 3×3 represents the convolution kernel size, 3N / 4 represents the number of input channels, and N / 2 represents the number of output channels. The parameter settings of other convolutions are the same as above.

[0168] It includes linear mapping layer, matrix multiplication, Softmax layer, Dropout layer, Hadamard product, and convolution layer. Its core lies in the three parts of Query (Q), Key (K) and Value (V). It maps remote sensing image data into different projection spaces, and then multiplies the data in different projection spaces through tensor multiplication to enhance the global visual features. The working process of MHSA can be expressed as:

[0169]

[0170] In the formula, Q stands for Query, K stands for Key, V stands for Value, and W Conv3×3 Represents the weight parameter of the convolution, W Linear represents the convolution parameters of the linear layer, W Dropout represents the weight parameter of the random dropout layer, W Softmax Represents the weight parameter of the Softmax layer, Represents matrix multiplication.

[0171] The process of GIE-AM feature extraction is shown in Algorithm 1.

[0172]

[0173]

[0174] It is worth mentioning that GIE-AM does not simply extract rich channel features and global visual features from remote sensing images, but effectively integrates the two. It first performs multiple dimensionality upscaling and dimensionality reduction operations on channel features through the Attention process of CEB, so as to achieve the purpose of compressing channel features and obtaining sufficient channel capacity, and combines the attention mechanism to obtain high-quality channel feature information. Secondly, the information obtained by the Attention process of CEB is mapped into different subspaces through the multi-head effect in MHSA, and the remote sensing image data containing rich channel features is further explored. The global contextual relationship is further explored, and the long-distance information and low-frequency features are efficiently enhanced through the self-attention mechanism. Finally, through the organic combination of Attention process of CEB and MHSA, a remote sensing image feature map that efficiently integrates channel features and global visual features is obtained.

[0175] D:BaselineA:

[0176] At present, the learning-based remote sensing image compression method mainly has the following common problems: First, in the compression process and reconstruction process at different levels, the number of channels is not reasonably distributed, resulting in the inability to fully utilize the parameters of the neural network. Second, the size distribution of the selected convolution kernel is unreasonable, resulting in too many parameters under the same receptive field and slow network training. In response to the above problems, the present invention designs BaselineA, which can obtain the same receptive field and better rate-distortion performance under smaller parameters. BaselineA is Figure 4 As shown. In order to make the transition between pixels of the reconstructed image smoother, BaselineA selects the common VAE as the basic framework. Since the common components have been described in the part of the overall framework of the proposed MDSNet, they will not be repeated here. The features contained in the remote sensing image can be summarized into two categories, one is spatial information and the other is channel information. For problem one, in the common remote sensing image compression framework, the number of channels for all downsampling and upsampling is set to the same number. However, as the downsampling proceeds, the size of the feature map is getting smaller and smaller, which means that the spatial information available for learning is getting smaller and smaller. At this time, the network should pay more attention to the channel information of the image. Therefore, the present invention redistributes the number of downsampled channels in the main encoder, that is, as the downsampling proceeds, the present invention gradually increases the number of channels, so as to achieve the purpose of paying more attention to the extraction of spatial information when the spatial size of the feature map is larger, and paying more attention to the extraction of channel information when the spatial size of the feature map is smaller. The decoder is also improved in the same way. For problem 2, the main framework abandons the commonly used convolution kernel size of 5×5 and selects a smaller convolution kernel of 3×3. In order to make up for the lost receptive field of the small convolution kernel, the present invention selects a convolution of size 7×7 for compression during the first downsampling of the main encoder. The calculation formula of the receptive field is as follows:

[0177] F(i)=(F(i+1)-1)×S+K(8)

[0178] In the formula, F(i) represents the receptive field size of the i-th layer, S represents the step size, and K represents the convolution kernel size.

[0179] The calculation formula of the convolution kernel parameter is as follows:

[0180] P = h × w × c × c (9)

[0181] In the formula, P represents the number of parameters, h represents the width of the convolution, w represents the height of the convolution, and c represents the number of channels.

[0182] With the help of the above two formulas, we can calculate the total number of parameters and receptive field size consumed by convolution in the main encoder and main decoder. Through calculation, for a common VAE framework (convolution kernel size 5×5, channel set to 3C / 4), the receptive field size of the main encoder is 61 and the number of parameters is 112.5C 2 The receptive field size of BaselineA (main encoder) is 63 and the number of parameters is 36.25C. 2 By comparison, it is found that BaselineA has almost the same receptive field, but the number of parameters is only one-third of that of the common VAE framework, which has a huge advantage in parameter quantity.

[0183] E:Rate-Distortion Optimization

[0184] The training goal of the compression framework is to achieve a harmonious balance between compression rate and distortion. Therefore, a common rate-distortion optimization strategy is introduced into the compression framework to guide the efficient training of the model, which can be expressed as:

[0185] argminLoss Totel =R+λD(10)

[0186] Where R represents the entropy rate, i.e., the cross entropy between the potential marginal distribution and the learned entropy model. D represents the distortion between the original image and the reconstructed image, and different bit rates can be controlled by adjusting the penalty coefficient λ.

[0187]

[0188] In the formula, the bit rate is represented by the potential information and side information Together form.

[0189]

[0190] In the formula, is a learnable entropy model, It is a super encoder.

[0191] In order to further improve the efficiency and quality of image compression, the present invention proposes a new rate-distortion optimization strategy, namely, formula (14), where Loss Totel The MDEM-Loss (Lossδ) loss is introduced to improve the similarity of the front multi-level domains. Its essence is the MSE loss between the multi-level domain feature maps of the main encoder part and the multi-level domain feature maps of the main decoder part. It increases the similarity between the highest-level domains by enhancing the similarity of the front multi-level domains, and optimizes the training of the overall network from a new and efficient perspective. This rate-distortion optimization strategy can be expressed as:

[0192] argminLoss Totel =R+λ(D+ψLossδ)(14)

[0193] Where ψ represents the weight coefficient of multi-level domain loss, and Lossδ represents MDEM-Loss.

[0194] Experimental results and analysis

[0195] In San Francisco

[43] 、NWPU-RESISC45

[44] and UC-Merced

[45] The proposed MDSNet method is compared with some excellent compression methods, including traditional codecs and learned compression models, to verify the superiority of the proposed method. Traditional image compression methods include JPEG2000

[10] , BPG

[12] and WebP

[13] . Compression models for learning, including Minnen ta al.

[26] 、Minnen et al.(mean)

[26] 、Balle et al.(hyperprior)

[27] , Balle et al.(factorized-relu)

[27] Tong2023

[46] The experimental results show that the proposed MDSNet method has the best performance in terms of evaluation indicators such as PSNR and MS-SSIM. In addition, this experiment also evaluates the quality of reconstructed images of different methods from the perspective of remote sensing image classification to further verify the superiority of the MDSNet method.

[0196] A. Dataset

[0197] 1) Dataset San Francisco: San Francisco is a remote sensing image dataset from

[43] . San Francisco is a remote sensing image with a resolution of 17408×17408. It contains various types of ground features such as buildings, coasts, roads, ports, lakes, etc. This paper crops San Francisco with a resolution of 256×256 pixels and selects 3000 valid images to form a dataset; it is divided into training set, validation set and test set in a ratio of 8:1:1. Figure 5 Some samples of this dataset are shown.

[0198] 2) Dataset NWPU-RESISC45: NWPU-RESISC45 is provided by Northwestern Polytechnical University (NWPU), China. The dataset contains 45 different remote sensing image scene categories, each category contains 700 images, and the resolution of each image is 256×256 pixels. The dataset contains a variety of geographical environments and scenes, covering images of different categories such as airports, deserts, churches, forests, ports, etc. 140 images in each category were selected to form a dataset containing 6,300 remote sensing images, which were then divided into training set, validation set and test set in a ratio of 8:1:1. Figure 6 Show some samples of this dataset.

[0199] 3) The UC-Merced dataset is a remote sensing image dataset provided by the University of California, Merced. The UC-Merced dataset includes 21 different land object categories, each category consists of 100 images, a total of 2100 images, and each image has a resolution of 256×256 pixels. The images cover different land object types such as farmland, airports, forests, ports, islands, etc. The UC-Merced dataset is divided into training set, validation set and test set in a ratio of 8:1:1. Figure 7 Show some samples of this dataset.

[0200] B. Evaluation Indicators

[0201] In terms of image quality assessment, the present invention uses two common indicators, peak signal-to-noise ratio and multi-scale structural similarity index, to comprehensively evaluate the quality of reconstructed images. In the remote sensing image scene classification part, the present invention uses overall accuracy and confusion matrix to measure the classification performance.

[0202] 1) Peak signal-to-noise ratio: PSNR compares the reconstructed image with the original image from the perspective of mean square error. The larger the PSNR value, the higher the fidelity of the reconstructed image. The peak signal-to-noise ratio can be expressed as

[0203]

[0204] Among them, MSE represents the mean square error between the original image and the reconstructed image, and max 2 (X i ) represents the square of the maximum pixel in the i-th band, and C represents the number of bands.

[0205] 2) Multi-scale structural similarity index measurement: MS-SSIM is a multi-scale structural similarity index that measures the difference between the original image and the reconstructed image by merging image details at different resolutions. Its value range is 0 to 1, and the higher the value, the higher the similarity, that is, the higher the quality of the reconstructed image. The formula of MS-SSIM can be expressed as:

[0206]

[0207] Among them, M represents different resolutions, μ X , Represent the means of the original image and the reconstructed image, σ X , represents the standard deviation between the original image and the reconstructed image, represents the covariance between the original image and the reconstructed image, α m , m Indicates the relative importance between two items, N 1 、N 2 is a constant term used to prevent division by zero.

[0208] In order to more clearly compare the difference in MS-SSIM values, it is converted into decibel values, which can be expressed as:

[0209] MS-SSIM=-10log 10 (1-D MS-SSIM )(17)

[0210] 3) Classification index: This paper selects two widely used classification evaluation indexes to measure the quality of reconstructed images, including overall accuracy (OA) and confusion matrix (CM). The OA value is obtained by dividing the number of correctly classified images by the total number of test images, which reflects the overall performance of a classification model. CM reflects the detailed classification error and confusion degree between different scene categories. Each row in CM represents the real category, and each column represents the predicted category.

[0211] C. Experimental Setup

[0212] In the present invention, the proposed MDSNet method is implemented by the PyTorch framework. The optimizer selected is the Adam optimizer. There are two optimizers in this network, one is the optimizer of the codec, and the other is the optimizer of the supercodec. For the main optimizer in the codec, the present invention sets the initialization learning rate at 10-4. During the network training process, when the learning rate decays to 10-6, the best training model of MDSNet will be stored. For the supercodec, the initial learning rate of the auxiliary optimizer is set to 10-3. The batch size is set to 8 during training. In this experiment, the neural network models were all trained on NVIDIA GeForce RTX 3090, and the traditional encoding and decoding parts were all performed on the same CPU (i9-9900K CPU@3.60GHz). For the fairness of the experiment, all experiments of the present invention were carried out in the above environment. The present invention selected eight comparison methods, three traditional image compression methods based on codecs, including JPEG200

[10] , WebP

[12] and BPG

[13] , and five deep learning-based image compression methods, including Minnentaal.

[26] 、Minnen et al.(mean)

[26] 、Balle et al.(hyperprior)

[27] , Balle et al.(factorized-relu)

[27] Tong2023

[46] . Among them, the ffmpeg version of JPEG2000 is 6.0, and the version of BPG is 0.9.8. The values ​​of the penalty coefficient λ commonly used in the present invention are [0.660, 0.508, 0.211, 0.072, 0.033, 0.013, 0.007]. The weight coefficient ψ of the multi-level domain loss in MDEM-Loss is set to 0.1. In GIE-AM, the number of MHSA heads is set to 4. In the overall framework, the number of channels N is set to 256. In the remote sensing scene classification part, the benchmark model used for testing is EMTCAL (Efficient Multiscale Transformer and Cross-Level Attention Learning)

[47] (

[47] Tang X, Li M, MaJ, et al. EMTCAL: Efficient multiscale transformer and cross-level attention learning for remote sensing scene classification [J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-15.), the training dataset used is NWPU-RESISC45, and the training-test ratio is 10%-90%. The images used for compression and the images used for remote sensing scene classification training are not crossed, and the reconstructed images are only used for testing the classification performance, not for training the classification network.

[0213] D. Rate-distortion performance curve

[0214] In this experiment, all models are evaluated for rate-distortion performance by PSNR and MS-SSIM. The present invention selects eight comparison methods, including three traditional image compression methods based on codec and five image compression methods based on deep learning. Figure 8 , Fig. 9 , Fig.10 The PSNR and MS-SSIM rate-distortion performance curves of all models obtained by experiments on the datasets San Francisco, NWPU-RESISC45, and UC-Merced are shown. By comparison, it is found that in general, the PSNR and MS-SSIM performance of deep learning methods are better than those of traditional image compression methods based on codecs. For traditional image compression methods based on codecs, it can be found that the rate-distortion performance of BPG is better than that of WebP and JPEG2000 in most cases. The main reason is that BPG can encode different color channels separately through multi-channel encoding technology, thereby strengthening the fine control of various detail features and helping to reconstruct high-assurance images. For image compression methods based on deep learning, Balle et al. (factorized-relu) has poor rate-distortion performance, mainly because it only uses simple convolutional layers, and this simple structure has weak feature extraction capabilities; Balle et al. (factorized-relu) can improve feature extraction capabilities to a certain extent by stacking the number of convolutional layers, but this will greatly increase the number of parameters and inference time.

[0215] Tong2023 achieved the highest PSNR and MS-SSIM rate-distortion performance on the datasets NWPU-RESISC45 and UC-Merced, except for the proposed MDSNet method. The main reason is that it uses a good attention mechanism and a more reasonable residual convolution kernel module. However, Tong2023 achieved poor PSNR and MS-SSIM rate-distortion performance on the dataset SanFrancisco, which proves that this compression model does not have good robustness. Minnen ta al., Minnen al. (mean), and Balle et al. (hyperprior), three image compression models based on deep learning, achieved mediocre rate-distortion performance. This is mainly because: although they have good convolution modules, they lack a strong attention mechanism and a more efficient and reasonable Loss Totel The MDSNet method proposed in this paper achieves the highest PSNR and MS-SSIM rate-distortion performance on the three datasets at the same time, and this performance advantage will be further expanded at high bpp. This superior rate-distortion performance not only strongly proves the robustness of the MDSNet method, but also strongly proves the effectiveness and rationality of MDEM, GIE-AM and BaselineA in the MDSNet method.

[0216] Analysis and verification of MDEM action principle

[0217] In order to verify the effectiveness of the proposed MDEM, the present invention conducts relevant experiments to verify it. Here, the baseline network selected is the proposed BaselineA, and the difference calculation function is MSE. The dark red curve is BaselineA without MDEM, which is recorded as the original difference loss. The blue curve is BaselineA with MDEM added, which is recorded as the optimized difference loss. Among them, Fig.11 and Fig.12 These are the difference loss curves on the dataset San Francisco and the dataset NWPU-RESISC45. The design principle of MDEM is to improve the similarity between the highest-level domains by optimizing the differences between the front multi-level domains. The difference loss experiment here strongly confirms this view. By comparison, it is found that on the dataset San Francisco, whether it is the first-level difference loss, the second-level difference loss, or the third-level difference loss, the loss is significantly reduced after adding MDEM, which means that the difference between the multi-level image features in the compressed domain and the multi-level image features in the reconstructed domain is significantly improved, that is, the similarity between the compressed domain features and the reconstructed domain features is greatly improved. The difference loss experiment on the dataset NWPU-RESISC45 also confirms the effectiveness of the MDEM principle, and also indirectly illustrates the high robustness of MDEM.

[0218] F. Comparative experiment on reconstructed image visualization

[0219] To further verify the effectiveness of the proposed MDSNet method, the present invention conducts a visual comparison of the reconstructed images of different methods. Fig.13 and Fig.14 They are visualization images on the San Francisco dataset and the NWPU-RESISC45 dataset. Fig.13 This is a visual comparison experiment chart when the bit rate is controlled at 0.3bpp. Fig.14 This is a visual comparison experiment chart when the bit rate is controlled at around 0.25bpp. Fig.13 For example, from left to right are the original image, eight comparison methods, and the reconstructed image of MDSNet. Fig.13 This is a city building map, with the roof area in the upper left corner and the square building area in the middle area magnified. Compared with the traditional image compression method based on codec, the rate-distortion performance of BPG is significantly higher than that of JPEG2000 and WebP, and the edge texture of the object in the roof area in the upper left corner is clearer, and the square building area in the middle retains more stripe information, while the JPEG2000 and WebP reconstruction areas have lost some detail features and appear blurred. The main reason for this phenomenon is that BPG uses multi-channel encoding technology, which improves the ability to reconstruct detail features. For the image compression method based on deep learning, the reconstructed image presents a relatively good visualization effect, but there is still a certain gap with the proposed MDSNet method. Fig.13 In the local method area of ​​the image reconstructed by Minnen et al. (mean), Balle et al., Balle et al. (factorized-relu), Tong2023 and other methods, there are some artifacts and noise, resulting in incomplete reconstruction of the texture information of the image. Among them, observing the reconstructed images of Minnen et al. (mean), Balle et al., Balle et al. (factorized-relu) and Tong2023, it can be found that the distinction between pixels at the edge of the image is too low and the transition between pixels is too rough, which leads to color flattening and distortion in the reconstructed image. Finally, Minnen et al., which has a relatively good visual effect among the comparison methods, is compared with the proposed MDSNet. The quality of the reconstructed image of MDSNet has more detailed features; while the transition between pixels of the reconstructed image of Minnen et al. is too smooth, which leads to the loss of some contrasting detailed features. Fig.14In the above experiments, MDSNet also achieved the best visualization effect. From the perspective of visualization, the above experiments fully demonstrate the effectiveness of the proposed MDSNet in capturing channel features and global information, as well as the accuracy of guiding image compression and reconstruction work.

[0220] H. Remote Sensing Image Scene Classification

[0221] This paper verifies the effectiveness of the MDSNet method from a new perspective by using the reconstructed image for remote sensing scene classification. The selected dataset is NWPU-RESISC45. The image compression methods compared include Minnen taal., Minnen et al. (mean), Balle et al. (hyperprior), Balle et al. (factorized-relu), Tong2023, and the proposed MDSNet method. The remote sensing scene classification benchmark model is EMTCA L

[47] ,The training-test ratio is 10%-90%. Among them, the reconstructed images are not used for the training of remote sensing scene classification, but only participate in the classification performance test. To ensure the fairness of the experiment, the reconstructed images of different reconstruction methods are obtained at a bit rate of 0.81bpp. Fig.15 The overall accuracy (OA) obtained by reconstructing images with different methods for remote sensing image scene classification. Among them, in terms of overall accuracy (OA), the proposed MDSNet achieved the highest OA value and achieved the best classification performance, exceeding 0.55% compared with Minnen ta al.. Compared with Minnen et al. (mean), it exceeded 0.55%. Compared with Balle et al. (hyperprior), it exceeded 0.73%. Compared with Balle et al. (factorized-relu), it exceeded 0.73%. Compared with Tong2023, it exceeded 0.37%.

[0222] I Complexity Analysis

[0223] In order to compare the computational complexity and resource consumption of different compression methods, the present invention tests these methods on the same device and in the same environment. The test parameters mainly include Parameter, FLOPs, GPU Memory, Compression time and Inference time. Table 1 shows the computational complexity of the existing methods and the proposed MDSNet method when the input image format is 3×256×256. By comparison, it is found that the number of parameters of MDSNet is second only to Balle et al. (factorized-relu). It is worth mentioning that although the number of parameters of Balle et al. (factorized-relu) is less than that of the proposed MDSNet, at the same bit rate, its PSNR and MS-SSIM rate-distortion performance are much lower than our method. By comparing FLOPs, we can find that MDSNet has achieved the smallest FLOPs parameter amount, which is only 51.41%, 51.90%, 52.47%, 53.56%, and 20.68% of the FLOPs parameter amount of Minnen taal., Minnen et al. (mean), Balle et al. (hyperprior), Balle et al. (factorized-relu), and Tong2023 methods, respectively. This fully demonstrates the superiority of the proposed MDSNet method. Compared with GPU Memory, MDSNet is in the middle of all methods. The reason is that some networks in the designed MDEM need to be calculated in parallel, which leads to the consumption of a certain amount of GPU resources. Comparing Compression time and Inference time, it can be found that the encoding and decoding speed of MDSNet is much faster than that of Minnentaal. and Tong2023 methods, and the encoding and decoding time is only one-tenth of them, which is slightly inferior to Minnen et al. (mean), Balle et al. (hyperprior) and Balle et al. (factorized-relu) methods. However, under the same bit rate, the PSNR and MS-SSIM performance of MDSNet are much better than these methods. These experiments strongly prove that MDSNet can achieve very excellent rate-distortion performance under the premise of low complexity, and the rate-distortion performance of this method is better than the comparison method in the present invention.

[0224] Table 1 Complexity experiments of different compression methods

[0225]

[0226] The present invention proposes a remote sensing image compression network (MDSNet) guided by multi-level domain similarity enhancement. First, the present invention constructs BaselineA by reasonably allocating the number of channels in the network and redesigning the convolution kernel size, which can achieve a larger receptive field and better rate-distortion performance under the premise of low parameters. Secondly, the present invention constructs MDEM by designing MIEM for extracting multi-channel information, MFIM for multi-level feature interaction, and MDEM-Loss for guiding the network to improve multi-level domain similarity, which can improve the quality of the final reconstructed image by enhancing the multi-level domain similarity of the front. Thirdly, the present invention constructs GIE-AM by designing CEB for enhancing channel features and MHSA for improving the global visual feature capture capability, which can achieve efficient enhancement and fusion of channel features and global visual features. Finally, all modules and networks of the present invention are reconstructed using Lossδ. Total Under the guidance of , efficient training is carried out, achieving the best rate-distortion performance among the comparison methods and obtaining the reconstructed image with the best classification performance. However, at present, MDEM only fuses and improves the similarity of features on three levels of domains. In subsequent work, we will explore the possibility of interaction and similarity improvement of features on more levels of domains. In addition, the complexity of the proposed GIE-AM is moderate. In subsequent work, we will explore how to further reduce the number of parameters under the same rate-distortion performance. In addition, we will further perform more detailed hierarchical processing on the compression and reconstruction process of remote sensing images, reduce the information gap between the potential representation features and specific tasks, and thus further improve the compression performance of remote sensing images.

[0227] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A remote sensing image compression method based on multi-level domain similarity guidance, characterized by: The specific process of the method is: Step 1: Construct a remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance; The remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance includes: encoder, decoder, probability model Probability Model; The encoder includes a first 7×7 two-dimensional convolution layer, a first GDN, a second 3×3 two-dimensional convolution layer, a second GDN, a third 3×3 two-dimensional convolution layer, a third GDN, a GIE-AM, a fourth 3×3 two-dimensional convolution layer, a fourth GDN, a fifth 3×3 two-dimensional convolution layer, a sixth 3×3 two-dimensional convolution layer, a seventh 3×3 two-dimensional convolution layer, an eighth 3×3 two-dimensional convolution layer, and an encoding multi-level domain similarity enhancement module MDEM; The encoding multi-level domain similarity enhancement module MDEM includes an encoding multi-level feature interaction module MFIM and an encoding multi-channel information extraction module MIEM; The encoding multi-level feature interaction module MFIM includes a ninth 1×1 two-dimensional convolutional layer, a tenth 1×1 two-dimensional convolutional layer, and an eleventh 1×1 two-dimensional convolutional layer; The encoding multi-channel information extraction module MIEM includes a first 3×3 maximum pooling, a first 3×3 average pooling, and a Softmax activation function layer; The GIE-AM includes a first CEB, a second CEB, a third CEB, a fourth CEB, a Sigmod activation function, and a multi-head attention layer MHSA; Each of the first CEB, the second CEB, the third CEB, and the fourth CEB includes a twelfth 3×3 two-dimensional convolutional layer and a thirteenth 3×3 two-dimensional convolutional layer in sequence; The decoder includes a first IGDN, a fourteenth 3×3 two-dimensional convolutional layer, a GIE-AM, a second IGDN, a fifteenth 3×3 two-dimensional convolutional layer, a third IGDN, a sixteenth 3×3 two-dimensional convolutional layer, a fourth IGDN, a seventeenth 3×3 two-dimensional convolutional layer, an eighteenth 3×3 two-dimensional convolutional layer, a nineteenth 3×3 two-dimensional convolutional layer, and a decoding multi-level domain similarity enhancement module MDEM; The decoding multi-level domain similarity enhancement module MDEM includes a decoding multi-channel information extraction module MIEM and a decoding multi-level feature interaction module MFIM; The encoded multi-channel information extraction module MIEM includes a 20th 1×1 two-dimensional convolutional layer, a 21st 1×1 two-dimensional convolutional layer, and a 22nd 1×1 two-dimensional convolutional layer; The decoding multi-level feature interaction module MFIM includes a second 3×3 maximum pooling, a second 3×3 average pooling, and a Softmax activation function layer; The GDN stands for Generalized Divisive Normalization Function; The IGDN represents the inverse operation of GDN; Step 2: Compress the remote sensing image to be tested based on the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement to obtain the compressed remote sensing image to be tested.

2. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 1, characterized in that: The specific processing process of the remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance in step 1 is: Step 1: The remote sensing image x is sequentially input into the first 7×7 two-dimensional convolutional layer and the first GDN, and the first GDN outputs the feature InputB; The first GDN output feature InputB is input into the encoding multi-channel information extraction module MIEM, and the encoding multi-channel information extraction module MIEM outputs feature D; The first GDN outputs feature InputB which is sequentially input into the second 3×3 two-dimensional convolutional layer and the second GDN, and the second GDN outputs feature A. The second GDN output feature B is sequentially input into the third 3×3 two-dimensional convolutional layer and the third GDN, and the third GDN outputs feature C; The third GDN output feature C is sequentially input into the fifth 3×3 two-dimensional convolutional layer and the sixth 3×3 two-dimensional convolutional layer, and the sixth 3×3 two-dimensional convolutional layer outputs feature InputA; The sixth 3×3 two-dimensional convolutional layer outputs the feature InputA input encoding multi-level feature interaction module MFIM; The output feature D of the coded multi-channel information extraction module MIEM is input into the coded multi-level feature interaction module MFIM; Encode the output feature E of the multi-level feature interaction module MFIM; The output feature D of the encoding multi-channel information extraction module MIEM is added element by element with the output feature E of the encoding multi-level feature interaction module MFIM to obtain the feature OutputA; The output feature E of the encoding multi-level feature interaction module MFIM is sequentially input into the seventh 3×3 two-dimensional convolutional layer and the eighth 3×3 two-dimensional convolutional layer, and the eighth 3×3 two-dimensional convolutional layer outputs feature F; The eighth 3×3 two-dimensional convolutional layer output feature F and the third GDN output feature C are element-wise added to obtain feature G; Feature G is input into GIE-AM, and GIE-AM outputs feature H; The feature H is sequentially input into the fourth 3×3 two-dimensional convolutional layer and the fourth GDN, and the fourth GDN outputs the feature y; Feature y is input into the probability model Probability Model, and the probability model ProbabilityModel outputs arithmetic encoding AE and arithmetic decoding AD respectively; The features y are sequentially input into the Q quantizer, and the Q quantizer outputs the features feature Input AE arithmetic coding and AD arithmetic decoding in sequence, and AD arithmetic decoding outputs features AD arithmetic decoding output characteristics Input the first IGDN and the fourteenth 3×3 two-dimensional convolutional layer in sequence, and the fourteenth 3×3 two-dimensional convolutional layer outputs feature J; The feature J input is sequentially input into the eighteenth 3×3 two-dimensional convolutional layer and the nineteenth 3×3 two-dimensional convolutional layer, and the nineteenth 3×3 two-dimensional convolutional layer outputs the feature InputD; Feature J is input into GIE-AM, and GIE-AM outputs feature K; Feature K is sequentially input into the second IGDN and the fifteenth 3×3 two-dimensional convolutional layer, and the fifteenth 3×3 two-dimensional convolutional layer outputs feature L; The feature L is sequentially input into the third IGDN and the sixteenth 3×3 two-dimensional convolutional layer, and the sixteenth 3×3 two-dimensional convolutional layer outputs the feature InputC; The feature InputC is input into the decoding multi-channel information extraction module MIEM, and the decoding multi-channel information extraction module MIEM outputs the feature N; Feature N input decoding multi-level feature interaction module MFIM; Feature InputD input decoding multi-level feature interaction module MFIM; Decode the multi-level feature interaction module MFIM output feature O; Decode the multi-level feature interaction module MFIM output feature O and the sixteenth 3×3 two-dimensional convolutional layer output feature InputC element by element to obtain feature P; Feature P is input into the fourth IGDN and the seventeenth 3×3 two-dimensional convolutional layer in sequence, and the seventeenth 3×3 two-dimensional convolutional layer outputs feature The feature O output by the decoding multi-level feature interaction module MFIM and the feature N output by the decoding multi-channel information extraction module MIEM are added element by element to obtain the feature OutputB.

3. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 2 is characterized in that: The first GDN output feature InputB is input into the encoding multi-channel information extraction module MIEM, and the encoding multi-channel information extraction module MIEM outputs feature D; the specific processing process is: The feature InputB is respectively input into the ninth 1×1 two-dimensional convolutional layer, the tenth 1×1 two-dimensional convolutional layer, and the eleventh 1×1 two-dimensional convolutional layer; the output features of the ninth 1×1 two-dimensional convolutional layer, the tenth 1×1 two-dimensional convolutional layer, and the eleventh 1×1 two-dimensional convolutional layer are added element by element to obtain feature D.

4. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 3 is characterized in that: The sixth 3×3 two-dimensional convolutional layer outputs the feature InputA input encoding multi-level feature interaction module MFIM; The output feature D of the coded multi-channel information extraction module MIEM is input into the coded multi-level feature interaction module MFIM; Encode the output feature E of the multi-level feature interaction module MFIM; The specific processing process is as follows: The sixth 3×3 2D convolutional layer outputs the feature InputA which is input into the first 3×3 maximum pooling; The output feature D of the encoding multi-channel information extraction module MIEM is input into the first 3×3 average pooling; Add the first 3×3 maximum pooling and the first 3×3 average pooling output features element by element to obtain feature ζ; Feature ζ is input into the Softmax activation function layer, and the output feature of the Softmax activation function layer is Hadamard-producted with feature ζ to obtain feature E.

5. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 4 is characterized in that: The feature G is input into GIE-AM, and GIE-AM outputs feature H; The specific processing process is as follows: The feature G is sequentially input into the first CEB, the second CEB, and the Sigmod activation function, and the Sigmod activation function outputs the feature x2; Feature G is input into the third CEB and the fourth CEB in sequence, and the fourth CEB outputs feature x1; Perform Hadamard product of feature x1 and feature x2 to obtain feature x3; Feature x3 is added to feature G element by element to get feature x CBE ; Features CBE Input the multi-head attention layer MHSA, and the multi-head attention layer MHSA outputs the feature H; Each of the first CEB, the second CEB, the third CEB, and the fourth CEB includes a twelfth 3×3 two-dimensional convolutional layer and a thirteenth 3×3 two-dimensional convolutional layer in sequence.

6. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 5, characterized in that: The feature InputC is input into the decoding multi-channel information extraction module MIEM, and the decoding multi-channel information extraction module MIEM outputs feature N; The specific processing process is as follows: The feature InputC is input into the 20th 1×1 2D convolutional layer, the 21st 1×1 2D convolutional layer, and the 22nd 1×1 2D convolutional layer respectively; The output features of the twentieth 1×1 two-dimensional convolutional layer, the output features of the twenty-first 1×1 two-dimensional convolutional layer, and the output features of the twenty-second 1×1 two-dimensional convolutional layer are added element by element to obtain feature N.

7. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 6, characterized in that: The feature N is input into a decoding multi-level feature interaction module MFIM; Feature InputD input decoding multi-level feature interaction module MFIM; Decode the multi-level feature interaction module MFIM output feature O; The specific processing process is as follows: Feature N is input into the second 3×3 maximum pooling; Feature InputD inputs the second 3×3 average pooling; Add the output features of the second 3×3 maximum pooling and the second 3×3 average pooling element by element to obtain feature ξ; The feature ξ is input into the Softmax activation function layer, and the output feature of the Softmax activation function layer is Hadamard-producted with the feature ζ to obtain the feature O.

8. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 7, characterized in that: The feature J is input into GIE-AM, and GIE-AM outputs feature K; The specific processing process is as follows: Feature J is input into the first CEB, the second CEB, and the Sigmod activation function in sequence, and the Sigmod activation function outputs feature x′2; Feature J is input into the third CEB and the fourth CEB in sequence, and the fourth CEB outputs feature x1′; Perform Hadamard product of feature x1′ and feature x′2 to obtain feature x3′; Feature x3′ is added to feature J element by element to get feature x′ CBE ; Feature x′ CBE Input the multi-head attention layer MHSA, and the multi-head attention layer MHSA outputs feature K; Each of the first CEB, the second CEB, the third CEB, and the fourth CEB includes a twelfth 3×3 two-dimensional convolutional layer and a thirteenth 3×3 two-dimensional convolutional layer in sequence.

9. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 8, characterized in that: In the step 2, the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement compresses the remote sensing image to be tested to obtain a compressed remote sensing image to be tested; The specific process is: Step 21: Obtain a trained remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance; The loss function of the remote sensing image compression network MDSNet guided by multi-level domain similarity enhancement is set as Loss Total , obtain the trained remote sensing image compression network MDSNet based on multi-level domain similarity enhancement guidance; The training process is: argminLoss Total =R+λ(D+ψLossδ) Where ψ represents the weight coefficient of multi-level domain loss, Lossδ represents MDEM-Loss; D represents the distortion between the original image and the reconstructed image; λ represents the adjustment penalty coefficient; and R represents the entropy rate.

10. The remote sensing image compression method based on multi-level domain similarity guidance according to claim 9, characterized in that: The working process of Lossδ is expressed as: MDEM-Loss=L MSE (I MFIM(Encoder) ,I MFIM(Decoder) ) Where, L MSE represents the loss measured using MSE; I MFIM(Encoder) Represents feature OutputA; I MFIM(Decoder) Represents feature OutputB.

Images