A task-oriented remote sensing image compression method and system

Abstract

The present invention provides a task-oriented remote sensing image compression method and system, which constructs a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset; trains a target detection model, including parameter training of a first-order target detection model on the target detection dataset; trains a semantic segmentation model, including parameter training of the semantic segmentation model on the semantic segmentation dataset; sets and trains an image compression model, constructs an importance feature map, and constructs a representation feature map extracted by the image compression model based on target regions of interest extracted by the target detection model or the semantic segmentation model; and fuses the importance feature maps to obtain a new representation feature map, thereby achieving remote sensing image compression. The present invention not only effectively improves the compression ratio of traditional image compression methods, but also has little impact on the detection and segmentation results of remote sensing images before and after compression.

Description

Technical Field

[0001] The present invention belongs to the field of satellite remote sensing image processing, and particularly relates to a semantic perception compression method for remote sensing images. A new compression model scheme is proposed to achieve adaptive bit rate allocation of remote sensing images for different tasks and high-rate compression tasks. Background Art

[0002] In recent years, with the advancement of deep learning, learning-based compression models have been continuously proposed to achieve high-rate image compression. Image compression technology plays a crucial role in the transmission of remote sensing images from satellites to the ground. While on orbit, it can reduce data transmission volume by removing redundant information from remote sensing images and encoding them into binary bitstream files, alleviating data transmission pressure under limited bandwidth. After receiving the bitstream files on the ground, decoding them on the ground can regenerate the remote sensing image.

[0003] With the rapid development of high-resolution remote sensing satellites, the resolution of remote sensing images is increasing, necessitating increased onboard data storage capacity and satellite-to-ground transmission bandwidth. For example, panchromatic images collected by the Gaofen-6 satellite have a resolution of 2 meters, 48,312 × 43,760 pixels, and require approximately 32,259 Mb of storage space. At a transmission rate of 2 × 450 Mbit / s, a single panchromatic image would take at least 36 seconds to transmit to the ground. Typically, the satellite transit window is only about ten minutes, making it difficult to transmit large quantities of remote sensing images to the ground. However, in emergencies such as natural disasters like earthquakes, mudslides, and floods, large quantities of images of affected areas must be transmitted to ground processing centers, but current transmission rates make it difficult to guarantee complete data transmission. Therefore, image compression technology plays a key role in reducing image storage and data transmission time.

[0004] Currently, the most commonly used on-orbit remote sensing image compression methods include JPEG and JPEG2000. JPEG is primarily based on the discrete cosine transform (DCT), while JPEG2000 employs the discrete wavelet transform (DWT) for entropy coding, respectively converting the image pixel domain into the frequency domain. However, to ensure image quality, on-orbit image compression ratios are typically set between 4 and 8 times. Low compression ratios make it difficult to guarantee real-time data transmission. Regarding image compression, convolutional neural networks (CNNs) offer superior compression performance compared to traditional image compression methods, and numerous deep learning-based compression models have been proposed. However, utilizing image compression to address bandwidth-constrained data transmission bottlenecks remains a pressing issue. Remote sensing image data exhibits multi-platform and multi-modal characteristics, with the Gaofen series of satellites encompassing panchromatic, multispectral, hyperspectral, radar, and infrared satellites. Existing on-orbit compression methods treat images indiscriminately and fail to consider image information of interest and context, resulting in significant additional computational overhead. Unfortunately, most existing methods fail to consider the practical issues of data transmission under limited bandwidth. With current compression methods, if the compression ratio is too high (e.g., exceeding 60x), severe image distortion can result. Therefore, in environments with limited computing power and large data volumes, traditional compression methods are no longer suitable for compressing on-orbit remote sensing images. No relevant papers in domestic or international journals have proposed methods for compressing high-magnification remote sensing images in on-orbit environments. Currently, there are no relevant solutions or authorized patents in China. Summary of the Invention

[0005] Aiming at the current high-rate intelligent perception compression problem of remote sensing images in constrained environments on satellites, the present invention provides a compression model scheme based on regions of interest to achieve adaptive bit rate allocation and high-rate compression tasks for remote sensing images of different tasks.

[0006] The technical solution provided by the present invention is a task-oriented remote sensing image compression method, comprising the following steps:

[0007] Step 1: Construct a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset. The target detection dataset is used for model training, verification, and testing of target detection tasks, and the semantic segmentation dataset is used for model training, verification, and testing of semantic segmentation tasks.

[0008] Step 2: Target detection model training, including parameter training of the first-order target detection model on the target detection dataset;

[0009] Step 3: semantic segmentation model training, including semantic segmentation model parameter training on the semantic segmentation dataset;

[0010] Step 4, setting and training an image compression model, which is implemented based on a convolutional neural network and includes four parts. The first part is the image encoding module Encoder, which is used to extract image representation feature information; the second part is the decoding module Decoder, which is used to decode the entropy-encoded code stream file into an image; the third part is the prior encoding module Hpyer-Encoder, which is used to re-encode the representation information output by the Encoder; the fourth part is the prior decoding module Hpyer-Decoder, which helps the Encoder and Decoder to better realize the encoding and decoding of the representation information, and learns the Gaussian distribution of the representation information through Hpyer-Encoder and Hpyer-Decoder to optimize the entropy model encoding;

[0011] Step 5: constructing an importance feature map, wherein the importance feature map is the result of reconstructing the representation feature map extracted by the image compression model based on the target region of interest extracted by the target detection model or the semantic segmentation model;

[0012] In step 6, the importance feature map obtained in step 5 is fused to obtain a new representation feature map to achieve remote sensing image compression.

[0013] Moreover, when constructing the target detection dataset and semantic segmentation dataset in step 1, the target detection dataset uses two data sources, the Gaofen-6 remote sensing satellite and the Gaofen multi-mode remote sensing satellite, and the data in the semantic segmentation dataset uses Gaofen-2 satellite images.

[0014] Moreover, when training the target detection model in step 2, the first-order target detection model adopts the YOLOV5 model.

[0015] Moreover, when training the semantic segmentation model in step 3, the semantic segmentation model adopts the PSPNet model.

[0016] Moreover, in the image compression model,

[0017] The Encoder module includes several Conv layers and GDN layers. The input of the Encoder module is the original image x, and the output is recorded as representation y. Then, the quantization function Q and entropy arithmetic coding AE are used to generate a bitstream file. The entropy arithmetic decoding DE arithmetic decodes the bitstream file to generate the corresponding result and is recorded as

[0018] The Hpyer-Encoder module includes several Conv layers and ReLU layers. The input of the Hpyer-Encoder module is y, and the output is denoted as z. Then, the quantization function Q and arithmetic entropy coding AE are used in sequence to generate a bitstream file. The arithmetic entropy decoding DE arithmetically decodes the bitstream file to generate the corresponding result and denotes it as

[0019] The Hpyer-Decoder module includes several TConv layers and ReLU layers. The input of the Hpyer-Decoder is The decoded output is the predicted standard deviation It is used to learn the standard deviation of the elements in the representation y that follow a Gaussian distribution Then, according to the feature distribution of the representation y, codeword estimation is performed on the arithmetic entropy coding AE and arithmetic entropy decoding DE parts;

[0020] The Decoder module includes several TConv layers and IGDN layers. The input of the Decoder module is The output is the compressed image

[0021] Moreover, in step 5, the construction of the importance feature map is achieved as follows,

[0022] 1) Use the information extraction network to generate a mask for the region of interest;

[0023] 2) Downsample the mask of the region of interest obtained in 1) to get a new mask;

[0024] 3) Replace all elements with a value of 0 in the mask after downsampling in 2) with a preset parameter α, where 0 < α ≤ 1;

[0025] 4) Construct an importance feature map M(m k,i,j ), where m k,i,j represents the element in the i-th row and j-th column of the k-th feature map, with an initial value of 0 for all. k represents the feature map number, c represents the number of feature maps, and 0 ≤ k ≤ c - 1;

[0026] Perform the following feature selection process,

[0027] When k < c × α, the k-th feature map retains all feature information,

[0028] When k ≥ c × α, the k-th feature map only retains the feature information of the region of interest, and the feature information of the background region is removed. By selecting the representation information, a new representation feature map M is obtained. The present invention calls M the importance feature map.

[0029] Moreover, it is used for high倍率 image compression in the on-orbit environment on the satellite.

[0030] It should be noted that the "高倍率" in the original text seems to be an incorrect expression. I translated it as "high magnification" according to the context, but it may need to be further confirmed according to the specific technical content.On the other hand, the present invention also provides a task-oriented remote sensing image compression system for implementing the task-oriented remote sensing image compression method described above.

[0031] Furthermore, the following modules are included,

[0032] The first module is used to construct a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset. The target detection dataset is used for model training, verification, and testing of target detection tasks, and the semantic segmentation dataset is used for model training, verification, and testing of semantic segmentation tasks.

[0033] The second module is used for target detection model training, including parameter training of the first-order target detection model on the target detection dataset;

[0034] The third module is used for semantic segmentation model training, including semantic segmentation model parameter training on the semantic segmentation dataset;

[0035] The fourth module is used to set up and train the image compression model. The image compression model is implemented based on a convolutional neural network and includes four parts. The first part is the image encoding module Encoder, which is used to extract image representation feature information; the second part is the decoding module Decoder, which is used to decode the entropy-encoded code stream file into an image; the third part is the prior encoding module Hpyer-Encoder, which is used to re-encode the representation information output by the Encoder; the fourth part is the prior decoding module Hpyer-Decoder, which helps the Encoder and Decoder to better realize the encoding and decoding of the representation information, and learns the Gaussian distribution of the representation information through Hpyer-Encoder and Hpyer-Decoder to optimize the entropy model encoding;

[0036] The fifth module is used to construct an important feature map, which is the result of reconstructing the representation feature map extracted by the image compression model based on the target region of interest extracted by the target detection model or the semantic segmentation model;

[0037] The sixth module is used to obtain a new representation feature map based on the importance feature map obtained in the fifth module to achieve remote sensing image compression.

[0038] Alternatively, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the task-oriented remote sensing image compression method as described above.

[0039] This paper provides an intelligent perceptual compression scheme for on-orbit environments based on different missions. It addresses the issues of data storage onboard and data transmission delays caused by limited bandwidth between the satellite and the ground. This scheme is simple, effective, highly accurate, and easy to implement. It also proposes an automatic bitrate allocation method for generating importance feature maps, which can be applied to implement region-of-interest (ROI)-based perceptual compression for different targets / regions of interest.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] (1) The automatic bitrate allocation scheme for generating importance feature maps proposed in the present invention can extract the region of interest and perform adaptive bitrate allocation according to the region according to different task requirements (e.g., detection of objects of interest, semantic segmentation of typical scenes).

[0042] (2) It is of great significance for data transmission under limited bandwidth. It can adaptively allocate compression bit rate according to ROI based on different task requirements, realize automatic bit rate allocation tasks, and effectively reduce the storage space and data transmission time of remote sensing images.

[0043] (3) It is highly practical and versatile. It can compress target information, scene information, and change information with high fidelity to reduce the amount of data. The detailed information of the target, scene, and change area of ​​interest will be well preserved. At the same time, the compression threshold can be set manually to achieve intelligent perception compression.

[0044] The solution of the present invention is simple and convenient to implement and has strong practicality. It solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of a task-oriented remote sensing image compression framework according to an embodiment of the present invention.

[0046] Figure 2 Schematic diagram of the structure of a remote sensing image compression model based on encoding and decoding according to an embodiment of the present invention.

[0047] Figure 3 A visual diagram of feature selection according to an embodiment of the present invention.

[0048] Figure 4 2 is a diagram showing target detection results before and after compression according to an embodiment of the present invention.

[0049] Figure 5 This is a diagram showing the semantic segmentation results before and after compression according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0051] The task-oriented remote sensing image compression method provided by the present invention can not only effectively improve the compression ratio of traditional image compression methods, but also has almost no impact on the detection and segmentation results of remote sensing images before and after compression.

[0052] The technical solution of the present invention aims to solve the technical problems that need to be solved and provides a task-oriented remote sensing image intelligent perception compression framework design. Figure 1 , by using the information extraction network (including target detection, semantic segmentation model) to extract the mask of the target / region of interest, and using the mask and the representation features output by the compression model encoder Figure 1 Together, they construct an importance feature map, which is then used for quantization and entropy coding, serving as the decoder's bitstream file to reconstruct the image. The technology of the present invention can be used for experiments using the Python programming language, or for engineering applications using the C / C++ programming language.

[0053] The embodiment of the present invention provides a task-oriented remote sensing image compression method, see Figure 1 , including the following steps:

[0054] Step 1: Construct a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset;

[0055] The object detection dataset is used for model training, verification and testing of object detection tasks, and the semantic segmentation dataset is used for model training, verification and testing of semantic segmentation tasks;

[0056] This embodiment constructs a target detection dataset and a semantic segmentation dataset for training and testing target detection and semantic segmentation models, respectively. Both models are used to train the compression model. The target detection dataset uses two data sources: my country's Gaofen-6 remote sensing satellite and Gaofen multi-mode remote sensing satellite. The acquired image data has a spatial resolution of 0.5m to 2m panchromatic images and a size of 1000×1000. The embodiment of the present invention pre-labels the target detection dataset using the open-source LabelMe software, with a total of 6,000 images labeled. The software allows users to easily label targets of interest, including three typical targets: aircraft, oil tanks, and ships. 3,000 images are used as a training set, 1,000 images as a validation set, and 2,000 images as a test set. This embodiment of the present invention constructs a semantic segmentation dataset, all data from Gaofen-2 satellite images, with a spatial resolution of 2m and an image size of 1000×1000. The regions of interest are buildings, and all building areas in the images are pre-labeled using LabelMe software, with building areas represented by 1 and non-building areas represented by 0. A total of 2000 images were annotated, of which 1000 images were used as training sets, 500 images were used as validation sets, and 500 images were used as test sets.

[0057] Step 2: Target detection model training;

[0058] In step 2 of the embodiment, the target detection model is trained, including parameter training of a first-order target detection model on the target detection dataset. The first-order target detection model adopts the open source YOLOV5 model.

[0059] During implementation, the target detection model can be selected as needed. This embodiment preferably uses the open-source target detection model YOLOV5, which currently has high detection accuracy and high speed, to complete the target detection task. YOLOV5 is trained on the training set of the target detection dataset constructed in step 1, and the model accuracy is verified on the validation set. After each training cycle, the model is verified, and the loss function value and detection accuracy on the validation set are recorded. If the loss function value changes by less than a small value for 10 consecutive cycles, it indicates that the model has converged, and model training is stopped. The model weight parameters at the best detection accuracy are saved and used as parameters for model testing on the test set. Target model training requires an Ubuntu 18.04LTS system and a software environment of PyTorch 1.8 and Python 3.7 or higher. The primary computing platform is an NVIDIA RTX2080Ti graphics card, with a CPU memory of at least 8GB and a solid-state drive of at least 1TB. During model training, a total of 100 cycles are trained, each cycle including all training set images. During model training, 16 images are loaded into the model for batch training at a time. The initial learning rate is set to 0.01, and the learning rate is reduced every 30 cycles, each time to 1 / 10 of the original value.

[0060] Step 3: semantic segmentation model training;

[0061] In step 3, the semantic segmentation model is trained, including the semantic segmentation model parameter training on the semantic segmentation dataset.

[0062] During specific implementation, the semantic segmentation model can be selected as needed. The semantic segmentation model preferably used in this embodiment is the open source PSPNet model. This model uses a spatial pyramid pooling module, which can effectively capture semantic features at different scales and plays an important role in the segmentation of remote sensing images. Model training is performed on the training set of the semantic segmentation dataset constructed in step 1. The method for training the target detection model described in step 2 is consistent with that of training on the training set, verifying the model accuracy and recording the change of the loss function on the validation set. When the value of the loss function changes less than a very small value within 10 consecutive cycles, the model is considered to have converged, and model training is stopped. The parameter weights at the best accuracy are saved as parameters for testing on the test set. The hardware configuration and software environment required for semantic segmentation are the same as those in step 2. When performing model training, a total of 300 cycles are trained, and each cycle includes all training set images. During model training, 16 images are loaded into the model for batch training at a time. The initial learning rate is set to 0.001, and the learning rate is reduced every 50 cycles, each time to 1 / 10 of the original.

[0063] Step 4: Set up and train the image compression model.

[0064] The specific implementation of the image compression model is as follows:

[0065] The embodiment of the present invention preferably adopts a relatively stable image compression model based on convolutional neural network. Figure 2 The model consists of four parts. The first part is the image encoding module (Encoder), which is used to extract image representation feature information; the second part is the decoding module (Decoder), which is used to decode the entropy-encoded bitstream file into an image; the third part is the prior encoding module (Hpyer-Encoder), which is used to re-encode the representation information output by the Encoder; the fourth part is the prior decoding module (Hpyer-Decoder), which can help the Encoder and Decoder better realize the encoding and decoding of the representation information. Through the Hyper-Encoder and Hyper-Decoder, the Gaussian distribution of the representation information can be learned to optimize the entropy model encoding.

[0066] like Figure 2 As shown, the specific steps of the image compression model provided by the embodiment of the present invention are as follows:

[0067] 1) The Encoder module includes several convolution layers (Convolution layer, Conv) and Generalized Divisive Normalization layers (GDN).

[0068] like Figure 2 The encoder modules preferably used in the embodiment are the first Conv, the first GDN, the second Conv, the second GDN, the third Conv, the third GDN, and the fourth Conv, and a set of feature maps will be output after each operation. Figure 2 The first parameter following the Conv layer is the number of channels in the input feature map, the second parameter is the size of the convolution kernel k, and the third parameter is the stride s of the convolution layer. For example, if the number of channels is 3, N, or M, k = 5 or 3, and s = 2. The parameters of other modules are similar. Arrows pointing right (→) in the figure indicate that the feature map output from this module will be downsampled.

[0069] The input of the encoder module is the original image x, and the output is recorded as the representation y. Then, the quantization function (Q) and entropy arithmetic coding (AE) are used to generate the bitstream file. The entropy arithmetic decoding (DE) arithmetic decoding of the bitstream file generates the corresponding result and is recorded as in Figure 2The square between AE and DE represents the code stream file.

[0070] 2) The Hyper-Encoder module includes several Conv layers and Rectified Linear Unit (ReLU) layers. Figure 2 The preferred embodiment adopts the Hyper-Encoder module, which is the first Conv, the first ReLU, the second Conv, the second ReLU, and the third Conv. After each operation, a set of feature maps will be output. The input of the Hyper-Encoder module is y, and the output is recorded as z. Then, the quantization function Q and entropy arithmetic coding AE are used in turn to generate a bitstream file. DE arithmetic decodes the bitstream file to generate the corresponding result and record it as

[0071] 3) The Hyper-Decoder module includes several deconvolution layers (TConv) and ReLU layers. Figure 2 As shown, the preferred Hyper-Decoder modules used in the embodiment are the first TConv, the first ReLU, the second TConv, the second ReLU, the third TConv, and the third ReLU. After each operation, a set of feature maps will be output. The input of Hyper-Decoder is The decoded output of HPYER-Decoder is the predicted standard deviation Used to learn the standard deviation of the Gaussian distribution of the elements in y Then, according to the feature distribution of y, codeword estimation is performed on the two modules AE and ED in entropy coding.

[0072] 4) The Decoder module includes several deconvolution (Transposed Convolution, TConv) and Inverse Generalized Divisive Normalization (IGDN). Figure 2 As shown in FIG, the decoder modules preferably used in the embodiment are the first TConv, the first IGDN, the second TConv, the second IGDN, the third TConv, the third IGDN, and the fourth TConv. After each operation, a set of feature maps will be output. In the figure, the arrow pointing to the right (←) indicates that the feature map output from the module will be upsampled. The input of the decoder module is Output is compressed image

[0073] In specific implementation, the image compression model can be trained once on the target detection dataset and the semantic segmentation dataset respectively to support solving the image compression tasks based on target detection and the image compression tasks based on semantic segmentation.

[0074] Step 5, construction of the importance feature map. The importance feature map is constructed by reconstructing the representation feature map extracted by the image compression model in step 4 based on the target / region of interest generated by the target detection model or the semantic segmentation model. The reconstructed feature map has different focus information for the region of interest and the region of non-interest, so it is called the importance feature map.

[0075] In specific implementation, based on the task requirements, the representation feature map extracted by the corresponding trained image compression model is reconstructed based on the target of interest generated by the target detection model or semantic segmentation model. The specific implementation method of constructing the importance feature map in the embodiment is as follows:

[0076] 1) Generate ROI region mask G(g) using information extraction network i,j );

[0077] In the embodiment, an information extraction network is used, including a target detection network or a semantic segmentation network, to extract the mask G (g i,j ): And g i,j ∈(0,1), if the element value is 0, it represents the background area, and if it is 1, it represents the area of ​​interest, where G is G(g i,j ), g i,j is the element value of the i-th row and j-th column of the mask image, H is the height of the mask, W is the width of the mask, Indicates the spatial dimension and size of the image.

[0078] 2) The ROI region mask is downsampled to obtain a new mask G′(g i,j );

[0079] In the embodiment, the mask G (g i,j ): If the downsampling is 32 times, then W = 32 × w, H = 32 × h, where H and W represent the height and width of the input mask respectively, g i,j represents the element value of the i-th row and j-th column of the mask image, h and w represent the mask G′(g i,j )'s height and width, Represents the spatial dimension and size of the image. Refer to formula (1) for downsampling:

[0080] G′(g i,j )=GMP(G(g i,j )) (1)

[0081] Among them, GMP() represents global max pooling, and the output feature map after downsampling

[0082] 3) Replace all elements with value 0 in the downsampled mask G′(g i,j ) with the parameter α to obtain the processing result

[0083] In the embodiment, α is used to control the minimum value of the elements in G′(g i,j ), as shown in formula (2):

[0084]

[0085] Moreover, 0 < α ≤ 1, then the elements in this feature map only include α and 1. Specifically in implementation, the value of α can be preset according to experience.

[0086] 4) Construct an importance feature map M(m k,i,j ):

[0087] In the embodiment, the initial value of the importance feature map M is all 0, where m k,i,j represents the element at the i-th row and j-th column of the k-th feature map, c represents the number of feature maps, and 0 ≤ k ≤ c - 1. The value of α can represent the selection of the number of feature maps for the background region. If the value of α is smaller, it means that the number of selected feature maps for the background region is less. Let the representation feature map be Y(y k.i,j ), and each feature map contains the representation information of the background region and the region of interest. Among them, y k.i,j represents the element at the i-th row and j-th column of the representation feature map. Since the number of feature maps of the representation feature Y is c, it means that there are selections for the feature maps of the background region. Then, the first feature maps of the representation Y are all retained, and the remaining feature maps only retain the features of the region of interest.

[0088] To more clearly represent the feature selection process of the compression framework in this embodiment, refer to Figure 3 , which shows the feature selection process of the background features and the region of interest. Among them, k represents the serial number of the feature map therein.

[0089] When k < c × α, the k-th feature map retains all feature information, that is, the first feature maps retain all feature information.

[0090] When k ≥ c × α, the k-th feature map only retains the feature information of the region of interest, that is, the last The feature map information only retains the feature information of the region of interest, and the feature information of the background area will be removed.

[0091] Before this The feature map saves all the features, and for the remaining The feature map only retains the features of the region of interest. This method is used to select the representation information of the background and the region of interest, and obtain a new representation feature map M(m k,i,j ). This embodiment refers to M(m k,i,j ) is the importance feature map.

[0092] Step 6: Fusion to obtain a new representation feature map

[0093] Step 6 of the embodiment is to fuse the original characterization feature map Y with the pixel multiplication operation to obtain a new characterization feature map. Refer to formula (3).

[0094]

[0095] Among them, ⊙ represents the dot multiplication operation. The data is input into the quantization function Q for quantization and arithmetic encoding into a bitstream file, and then input into the encoder module through arithmetic decoding to regenerate the image. In this embodiment, a feature selection process is performed on the z output of the Hyper-Encoder module to make the Hyper-Encoder and Hyper-Decoder pay more attention to the region of interest.

[0096] Through the above steps 5 and 6, the feature selection of different backgrounds and regions of interest can be effectively realized, and the adaptive bit rate allocation problem can be realized in conjunction with the quantization function. The bit rate allocation of the background area can be controlled according to the set α value. If the α value is lower, the allocated bit rate is lower.

[0097] In order to facilitate understanding of the technical effects of the present invention, the application comparison between the present invention and the traditional method is provided. Figure 4 and Figure 5 . Figure 4 The comparison of the target detection task before and after compression is shown. It can be seen that at a lower compression rate (bits per pixel, bpp), the method proposed in this invention has almost no effect on the detection results of remote sensing images before and after compression. Figure 5 The comparison of the semantic segmentation task before and after compression is shown. It can also be seen that the method proposed in the present invention has almost no effect on the segmentation results of the remote sensing image before and after compression.

[0098] By comparing the detection results before and after compression of the above-mentioned target detection task and semantic segmentation task, the task-oriented remote sensing image compression method provided by the present invention can meet the high-magnification image compression requirements in the on-orbit environment, and in the compressed image, the accuracy of target detection and semantic segmentation is little affected.

[0099] In specific implementation, the method proposed in the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. System devices that implement the method, such as computer-readable storage media that store the corresponding computer program of the technical solution of the present invention and computer equipment that runs the corresponding computer program, should also be within the scope of protection of the present invention.

[0100] In some possible embodiments, a task-oriented remote sensing image compression system is provided, comprising the following modules:

[0101] The first module is used to construct a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset. The target detection dataset is used for model training, verification, and testing of target detection tasks, and the semantic segmentation dataset is used for model training, verification, and testing of semantic segmentation tasks.

[0102] The second module is used for target detection model training, including parameter training of the first-order target detection model on the target detection dataset;

[0103] The third module is used for semantic segmentation model training, including semantic segmentation model parameter training on the semantic segmentation dataset;

[0104] The fourth module is used to set the image compression model. The image compression model is implemented based on a convolutional neural network and includes four parts. The first part is the image encoding module Encoder, which is used to extract image representation feature information; the second part is the decoding module Decoder, which is used to decode the entropy-encoded code stream file into an image; the third part is the prior encoding module Hpyer-Encoder, which is used to re-encode the representation information output by the Encoder; the fourth part is the prior decoding module Hpyer-Decoder, which helps the Encoder and Decoder to better realize the encoding and decoding of the representation information, and learns the Gaussian distribution of the representation information through Hpyer-Encoder and Hpyer-Decoder to optimize the entropy model encoding;

[0105] The fifth module is used to construct an importance feature map, which is the result of reconstructing the representation feature map extracted by the image compression model according to the target area of ​​interest;

[0106] The sixth module is used to obtain a new representation feature map based on the importance feature map obtained in the fifth module to achieve remote sensing image compression.

[0107] In some possible embodiments, a task-oriented remote sensing image compression system is provided, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a task-oriented remote sensing image compression method as described above.

[0108] In some possible embodiments, a task-oriented remote sensing image compression system is provided, including a readable storage medium having a computer program stored thereon. When the computer program is executed, the task-oriented remote sensing image compression method as described above is implemented.

[0109] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A task-oriented remote sensing image compression method, characterized in that: It includes the following steps: Step 1: Construction of high-resolution remote sensing image datasets, including object detection datasets and semantic segmentation datasets. The object detection datasets are used for model training, validation, and testing of object detection tasks. The pre-annotated objects of interest include airplanes, storage tanks, and ships. The semantic segmentation datasets are used for model training, validation, and testing of semantic segmentation tasks. The pre-annotated objects of interest include buildings; Step 2: Training of object detection models, including parameter training of first-order object detection models on object detection datasets; Step 3: Training of semantic segmentation models, including parameter training of semantic segmentation models on semantic segmentation datasets; Step 4: Setting up and training an image compression model. The image compression model is implemented based on a convolutional neural network and includes four parts. The first part is the encoding module Encoder of the image, which is used to extract the characteristic information of the image representation. The second part is the decoding module Decoder, which is used to decode the entropy-encoded bitstream file into an image. The third part is the prior encoding module Hyper-Encoder, which is used to re-encode the representation information output by the Encoder. The fourth part is the prior decoding module Hyper-Decoder, which helps the Encoder and Decoder better implement the encoding and decoding of the representation information. By learning the Gaussian distribution of the representation information through the Hyper-Encoder and Hyper-Decoder, the entropy model encoding is optimized. The image compression model is trained once on both the object detection dataset and the semantic segmentation dataset to support solving image compression tasks based on object detection and image compression tasks based on semantic segmentation; Step 5: Construction of importance feature maps. The importance feature maps are the results of reconstructing the representation feature maps extracted by the image compression model based on the regions of interest of the objects detected by the object detection model or the semantic segmentation model according to the task requirements; The implementation method is as follows: 1) Use an information extraction network to generate a mask for the region of interest; 2) Downsample the mask of the region of interest obtained in 1) to get a new mask; 3) Replace all elements with a value of 0 in the downsampled mask in 2) with a preset parameter α, where 0 < α ≤ 1; 4) Construct an important feature map M(m k,i,j ), where m k,i,j The element in the i-th row and j-th column of the k-th feature map is initially all 0, k represents the feature map number, c represents the number of feature maps, and 0≤k≤c-1; Perform the following feature selection process: When k < c × α, the kth feature map retains all feature information; When k ≥ c × α, the kth feature map only retains the feature information of the region of interest, and the feature information of the background region is removed; Select the representation information of the background and the region of interest according to the above method to obtain a new representation feature map M, which is called the importance feature map; Step 6: Fuse the importance feature maps obtained in Step 5 to obtain a new representation feature map, and realize remote sensing image compression.

2. The task-oriented remote sensing image compression method according to claim 1, characterized in that: When constructing the object detection dataset and the semantic segmentation dataset in Step 1, the object detection dataset uses two data sources, namely the GF-6 remote sensing satellite and the GF multi-mode remote sensing satellite. The data in the semantic segmentation dataset uses GF-2 satellite images.

3. The task-oriented remote sensing image compression method according to claim 1, characterized in that: When training the target detection model in step 2, the first-order target detection model adopts the YOLOV5 model.

4. The task-oriented remote sensing image compression method according to claim 1, characterized in that: When training the semantic segmentation model in step 3, the semantic segmentation model adopts the PSPNet model.

5. The task-oriented remote sensing image compression method according to claim 1, 2, 3 or 4, characterized in that: In the image compression model, The Encoder module includes several Conv layers and GDN layers. The input of the Encoder module is the original image x, and the output is recorded as representation y. Then, the quantization function Q and entropy arithmetic coding AE are used to generate a bitstream file. The entropy arithmetic decoding DE arithmetic decodes the bitstream file to generate the corresponding result and is recorded as The Hyper-Encoder module includes several Conv layers and ReLU layers. The input of the Hyper-Encoder module is y, and the output is z. Then, the quantization function Q and entropy arithmetic coding AE are used in turn to generate a bitstream file. The entropy arithmetic decoding DE arithmetic decodes the bitstream file to generate the corresponding result and is recorded as The Hyper-Decoder module includes several TConv layers and ReLU layers. The input of Hyper-Decoder is The decoded output is the predicted standard deviation Used to learn the standard deviation of the Gaussian distribution of the elements in y Then, according to the distribution of the features representing y, the codeword estimation is performed on the entropy arithmetic encoding AE and entropy arithmetic decoding DE parts; The Decoder module includes several TConv layers and IGDN layers. The input of the Decoder module is Output is compressed image 6. The task-oriented remote sensing image compression method according to claim 1, 2, 3 or 4, characterized in that: Used for high-magnification image compression in on-orbit environments.

7. A task-oriented remote sensing image compression system, characterized by: Used to implement a task-oriented remote sensing image compression method as described in any one of claims 1-6.

8. The task-oriented remote sensing image compression system according to claim 7, characterized in that: Includes the following modules, The first module is used to construct a high-resolution remote sensing image dataset, including a target detection dataset and a semantic segmentation dataset. The target detection dataset is used for model training, verification, and testing of target detection tasks, and the semantic segmentation dataset is used for model training, verification, and testing of semantic segmentation tasks. The second module is used for target detection model training, including parameter training of the first-order target detection model on the target detection dataset; The third module is used for semantic segmentation model training, including semantic segmentation model parameter training on the semantic segmentation dataset; The fourth module is used to set up and train the image compression model. The image compression model is implemented based on a convolutional neural network and includes four parts. The first part is the image encoding module Encoder, which is used to extract image representation feature information; the second part is the decoding module Decoder, which is used to decode the entropy-encoded bitstream file into an image; the third part is the prior encoding module Hyper-Encoder, which is used to re-encode the representation information output by the Encoder; the fourth part is the prior decoding module Hyper-Decoder, which helps the Encoder and Decoder better implement encoding and decoding of the representation information. The Gaussian distribution of the representation information is learned by the Hyper-Encoder and Hyper-Decoder to optimize the entropy model encoding; The fifth module is used to construct an important feature map, which is the result of reconstructing the representation feature map extracted by the image compression model based on the target region of interest extracted by the target detection model or the semantic segmentation model; The sixth module is used to obtain a new representation feature map based on the importance feature map obtained in the fifth module to achieve remote sensing image compression.

9. The task-oriented remote sensing image compression system according to claim 7, characterized in that: The system comprises a processor and a memory, wherein the memory is used for storing program instructions, and the processor is used for calling the stored instructions in the memory to execute the task-oriented remote sensing image compression method.

Images