A remote sensing image hybrid compression method based on data block complexity classification

Abstract

The application relates to a remote sensing image hybrid compression method based on data block complexity classification, which comprises the following steps: dividing an original remote sensing image into multiple image blocks; analyzing the image complexity of each image block, and determining whether the image block is a smooth image block or a complex image block according to the image complexity; directly increasing the quantization step of the smooth image block to realize high-compression-ratio compression; constructing a compression model based on a deep neural network to compress the high-frequency component of the complex image block; adopting a JPEG2000 compression model to compress the low-frequency component of the complex image block; and then performing entropy coding on each component. The application can improve the image compression efficiency under the condition of guaranteeing the image quality, and realizes the best remote sensing image compression performance.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing, and more specifically, to a method for hybrid compression of remote sensing images based on data block complexity classification. Background Technology

[0002] The data acquisition capabilities of modern high-resolution remote sensing satellites (such as Gaofen-5 and Gaofen-7) are constantly improving, and the amount of data acquired by satellites is growing exponentially. The average daily data acquisition volume of a single satellite has jumped from the GB level to the TB and PB level. Against this backdrop, efficient and reliable on-orbit image compression technology has become an indispensable key link in space information processing and transmission systems. However, the on-board environment greatly limits the power consumption and computing resources required for compression algorithms, resulting in a limitation on the compression algorithms that can be deployed. Currently, the mainstream compression scheme on satellites is still based on the JPEG2000 algorithm. However, due to the huge differences in texture structure, statistical characteristics, and information density of remote sensing data, in order to ensure the reconstruction quality of images in extremely complex scenes (such as high-detail urban areas and dense forest cover), the compression system has to adopt a relatively conservative strategy, limiting the overall compression ratio to about 4 times. The downlink of a single high-resolution image can even take up to 8 minutes.

[0003] The JPEG2000 compression method is the most widely used image coding method. Its basic idea is to transform the image described in the spatial domain to the wavelet domain, then quantize the wavelet coefficients using an appropriate method, and finally entropy-encode the quantization results to remove correlations and reduce the amount of data stored and transmitted. For remote sensing images, the terrain and landforms vary greatly. To ensure the reconstruction quality of images in complex scenes, the compression factor needs to be strictly controlled, making it difficult to improve compression efficiency. The main factor contributing to the loss of image reconstruction quality in complex scenes is the quantization loss of high-frequency components during the coding process. Summary of the Invention

[0004] To overcome at least one of the shortcomings of the prior art, this application provides a remote sensing image hybrid compression method based on data block complexity classification.

[0005] Firstly, a remote sensing image hybrid compression method based on data block complexity classification is provided, including: The original remote sensing image is divided into multiple image blocks; Analyze the image complexity of each image patch and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity; For smooth image blocks, the quantization step size is increased, and the JPEG2000 compression model is used to perform high compression ratio image compression to obtain the compressed bitstream of the smooth image blocks; For complex image patches, a second wavelet decomposition is performed to obtain low-frequency and high-frequency components. The low-frequency components are quantized using the JPEG2000 compression model, and the quantization results are entropy encoded to obtain the coded bitstream of the low-frequency components. The high-frequency components are feature extracted using a neural network coding module, and the extracted features are entropy encoded to obtain the coded bitstream of the high-frequency components. The coded bitstreams of the low-frequency and high-frequency components are merged to obtain the compressed bitstream of the complex image patch.

[0006] In one embodiment, the image complexity of each image patch is analyzed, and the image patch is determined to be either a smooth image patch or a complex image patch based on the image complexity, including: For any image patch A, construct the corresponding image patch A Mosaic B and Mosaic C; The pixels in mosaic B and the pixels in image block A satisfy the following relationship: ,

[0007] in, Here are the pixel coordinates, and n is a natural number. for Pixels in mosaic B, For pixels in image block A; The pixels in mosaic C and the pixels in image block A satisfy the following relationship: ,

[0008] in, for Pixels in mosaic C; Calculate the image complexity of image patch A :

[0009] in, This indicates calculating the average value. Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic B Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic C; If the image complexity of image block A is... Less than the threshold If the condition is met, then image block A is a complex image block; otherwise, image block A is a smooth image block.

[0010] In one embodiment, the neural network encoding module is used to sequentially perform generalized division normalization, convolution operation, ReLU activation function operation, convolution operation, and ReLU activation function operation on the high-frequency components of the input.

[0011] In one embodiment, the training process of the neural network encoding module includes: Obtain the training dataset, where the samples are remote sensing images; Obtain the low-frequency and high-frequency components corresponding to the sample; The low-frequency components are quantized using the JPEG2000 compression model to obtain the quantization result; the quantization result is then decoded to obtain the recovered low-frequency image patch. The high-frequency components are extracted using a neural network encoding module, and the extracted features are decoded using a neural network decoding module to obtain the recovered high-frequency image patches. Calculate the low-frequency loss based on the recovered low-frequency image patches and low-frequency components; calculate the high-frequency loss based on the recovered high-frequency image patches and high-frequency components; multiply the low-frequency loss and high-frequency loss by their respective weights and sum them up to obtain the final loss. The neural network encoding module is trained based on the final loss, resulting in the trained neural network encoding module.

[0012] Secondly, a remote sensing image hybrid compression device based on data block complexity classification is provided, comprising: The image segmentation module is used to divide the original remote sensing image into multiple image blocks; The image complexity determination module is used to analyze the image complexity of each image patch and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity. The first compression module is used to increase the quantization step size for smooth image blocks and perform high compression ratio image compression using the JPEG2000 compression model to obtain the compressed bitstream of the smooth image blocks. The second compression module performs secondary wavelet decomposition on complex image patches to obtain low-frequency and high-frequency components. The low-frequency components are quantized using the JPEG2000 compression model, and the quantization results are entropy-coded to obtain the coded bitstream of the low-frequency components. The high-frequency components are feature-extracted using a neural network coding module, and the extracted features are entropy-coded to obtain the coded bitstream of the high-frequency components. The coded bitstreams of the low-frequency and high-frequency components are then merged to obtain the compressed bitstream of the complex image patch.

[0013] Compared with existing technologies, this application has the following advantages: By predicting image complexity, this application classifies image blocks to be compressed into smooth image blocks and complex image blocks. For smooth image blocks, its quantization step size is directly increased to achieve high compression ratios. For complex image blocks, a deep neural network-based compression model is constructed to compress its high-frequency components, and the JPEG2000 compression model is used to compress its low-frequency components. Then, entropy encoding is performed on each component. By training this end-to-end image compression model and determining the parameters of the deep neural network model, this application can improve image compression efficiency while ensuring image quality, achieving optimal remote sensing image compression performance. Attached Figure Description

[0014] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings: Figure 1 A flowchart of a remote sensing image hybrid compression method based on data block complexity classification is shown; Figure 2 A schematic diagram illustrating the principle of an efficient compression method for complex image patches is shown. Figure 3 A schematic diagram of the training process of the neural network encoding module is shown. Detailed Implementation

[0015] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0016] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0017] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0018] This application provides a remote sensing image hybrid compression method based on data block complexity classification. Figure 1 A flowchart of a remote sensing image hybrid compression method based on data block complexity classification is shown. See [link to flowchart]. Figure 1The method mainly includes the following steps: Step S1: Divide the original remote sensing image into multiple image patches. Here, the size of each image patch is... m can be 128, 256, 512 or 1024.

[0019] Step S2: Analyze the image complexity of each image patch and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity.

[0020] Here, a method is proposed. Mosaic and The mosaic method is a method for calculating data complexity. For any image patch A, it constructs the corresponding image patch A. Mosaic B and Mosaic C; The pixels in mosaic B and the pixels in image block A satisfy the following relationship: ,

[0021] in, Here are the pixel coordinates, and n is a natural number. for Pixels in mosaic B, For pixels in image block A; The pixels in mosaic C and the pixels in image block A satisfy the following relationship: ,

[0022] in, for Pixels in mosaic C; Calculate the image complexity of image patch A :

[0023] in, This indicates calculating the average value. Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic B Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic C; If the image complexity of image block A is... Less than the threshold If the threshold is met, then image patch A is a complex image patch; otherwise, image patch A is a smooth image patch. These are empirical values ​​from the experimental process.

[0024] Step S3: For smooth image blocks, increase the quantization step size and use the JPEG2000 compression model to perform high compression ratio image compression to obtain the compressed bitstream of the smooth image blocks.

[0025] In the traditional JPEG2000 compression model, the quantization step size can be 4 when compressing images. In this embodiment, for smooth image blocks, the quantization step size is increased, for example, to 8, which can still meet the relevant requirements of image compression.

[0026] Step S4: For the complex image patch, perform secondary wavelet decomposition to obtain low-frequency and high-frequency components; quantize the low-frequency components using the JPEG2000 compression model and entropy encode the quantization results to obtain the coded bitstream of the low-frequency components; extract features from the high-frequency components using a neural network coding module and entropy encode the extracted features to obtain the coded bitstream of the high-frequency components; merge the coded bitstreams of the low-frequency and high-frequency components to obtain the compressed bitstream of the complex image patch.

[0027] Figure 2 A schematic diagram illustrating the principle of an efficient compression method for complex image patches is shown. (See attached diagram) Figure 2 The neural network encoding module is used to sequentially perform generalized division normalization (GDN function), convolution operation, ReLU activation function operation, convolution operation, and ReLU activation function operation on the high-frequency components of the input.

[0028] In this embodiment, by predicting the image complexity, the image blocks to be compressed are classified into smooth image blocks and complex image blocks. For smooth image blocks, the quantization step size is directly increased to achieve high compression ratio compression. For complex image blocks, a compression model based on a deep neural network is constructed to compress its high-frequency components, and the JPEG2000 compression model is used to compress its low-frequency components. Then, entropy coding is performed on each component. This can improve image compression efficiency while ensuring image quality, and achieve the best remote sensing image compression performance.

[0029] Furthermore, in practical applications, the neural network encoding module is the trained module. Figure 3 The diagram illustrates the training process of a neural network encoding module, which includes: First, obtain the training dataset, which consists of remote sensing images; here, the training dataset can be the DOTA dataset.

[0030] Obtain the low-frequency and high-frequency components corresponding to the sample; here, the low-frequency and high-frequency components corresponding to the sample are obtained according to the process in steps S1-S4 of the aforementioned embodiment, which will not be repeated here.

[0031] Then, the low-frequency components are quantized using the JPEG2000 compression model to obtain the quantization result; the quantization result is decoded to obtain the recovered low-frequency image patch; here, the decoding of the quantization result is mainly based on the inverse wavelet transform (IDWT).

[0032] Then, the high-frequency components are used for feature extraction using a neural network encoding module, and the extracted features are then decoded using a neural network decoding module to obtain the recovered high-frequency image blocks. Here, the neural network decoding module has the same structure as the neural network encoding module and can perform generalized division normalization (GDN function), convolution operation, ReLU activation function operation, and so on.

[0033] Then, based on the recovered low-frequency image patches and low-frequency components, the low-frequency loss is calculated; based on the recovered high-frequency image patches and high-frequency components, the high-frequency loss is calculated; the low-frequency loss and high-frequency loss are multiplied by their corresponding weights and summed to obtain the final loss; the final loss can be calculated using the following formula. :

[0034] in, The low-frequency loss is obtained by calculating the mean square error between the recovered low-frequency image patch and the low-frequency component; The high-frequency loss is obtained by calculating the mean square error between the recovered high-frequency image patch and the high-frequency component; and The weights for low-frequency loss and high-frequency loss are respectively set to 1, and can both be set to 1.

[0035] The neural network encoding module is trained based on the final loss, resulting in the trained neural network encoding module.

[0036] To further verify the effectiveness of the remote sensing image hybrid compression method based on data block complexity classification proposed in this application, the following experimental analysis was conducted.

[0037] Simulation experiments were conducted using the DOTA dataset, a large dataset specifically designed for aerial image target detection, containing high-resolution remote sensing images and multi-class target annotations. 20,000 randomly selected images were used for training and testing respectively. Since the JPEG2000 algorithm is an on-orbit algorithm, the method in this application was compared with the JPEG2000 algorithm.

[0038] The method described in this application can significantly improve the compression quality of images at the same compression ratio. It can achieve high compression ratios while ensuring image quality, effectively reducing the amount of data stored and transmitted in the data transmission system. At the same time, the method reuses the quantization module in the JPEG2000 module, which can reduce the decrease in throughput caused by neural network data processing to a certain extent, thereby improving compression efficiency with only a small increase in computing resources. The full encoding of wavelet high-frequency coefficients of complex image blocks improves compression efficiency while ensuring image quality.

[0039] Employing the same inventive concept as the remote sensing image hybrid compression method based on data block complexity classification, this embodiment also provides a corresponding remote sensing image hybrid compression apparatus based on data block complexity classification, comprising: The image segmentation module is used to divide the original remote sensing image into multiple image blocks; The image complexity determination module is used to analyze the image complexity of each image patch and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity. The first compression module is used to increase the quantization step size for smooth image blocks and perform high compression ratio image compression using the JPEG2000 compression model to obtain the compressed bitstream of the smooth image blocks. The second compression module performs secondary wavelet decomposition on complex image patches to obtain low-frequency and high-frequency components. The low-frequency components are quantized using the JPEG2000 compression model, and the quantization results are entropy-coded to obtain the coded bitstream of the low-frequency components. The high-frequency components are feature-extracted using a neural network coding module, and the extracted features are entropy-coded to obtain the coded bitstream of the high-frequency components. The coded bitstreams of the low-frequency and high-frequency components are then merged to obtain the compressed bitstream of the complex image patch.

[0040] The remote sensing image hybrid compression device based on data block complexity classification in this embodiment has the same inventive concept as the remote sensing image hybrid compression method based on data block complexity classification described above. Therefore, the specific implementation of this device can be found in the embodiment section of the remote sensing image hybrid compression method based on data block complexity classification described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0041] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A remote sensing image hybrid compression method based on data block complexity classification, characterized in that, include: The original remote sensing image is divided into multiple image blocks; Analyze the image complexity of each image patch, and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity; For smooth image blocks, the quantization step size is increased, and the JPEG2000 compression model is used to perform high compression ratio image compression to obtain the compressed bitstream of the smooth image blocks; For complex image patches, a second wavelet decomposition is performed to obtain low-frequency and high-frequency components; The low-frequency components are quantized using the JPEG2000 compression model, and the quantization results are entropy encoded to obtain the coded bitstream of the low-frequency components. The high-frequency components are feature extracted using a neural network coding module, and the extracted features are entropy encoded to obtain the coded bitstream of the high-frequency components. The coded bitstreams of the low-frequency components and the coded bitstreams of the high-frequency components are merged to obtain the compressed bitstream of the complex image patch. Analyze the image complexity of each image patch, and determine whether the image patch is a smooth image patch or a complex image patch based on the image complexity, including: For any image patch A, construct the corresponding image patch A Mosaic B and Mosaic C; the aforementioned The pixels in mosaic B and the pixels in image block A satisfy the following relationship: ， in, Here are the pixel coordinates, and n is a natural number. for Pixels in mosaic B, For pixels in image block A; The The pixels in mosaic C and the pixels in image block A satisfy the following relationship: ， in, for Pixels in mosaic C; Calculate the image complexity of image patch A : in, This indicates calculating the average value. Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic B Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic C; If the image complexity of image block A is... Less than the threshold If the condition is met, then image block A is a complex image block; otherwise, image block A is a smooth image block.

2. The method as described in claim 1, characterized in that, The neural network encoding module is used to sequentially perform generalized division normalization, convolution operation, ReLU activation function operation, convolution operation, and ReLU activation function operation on the high-frequency components of the input.

3. The method as described in claim 1, characterized in that, The training process of the neural network encoding module includes: Obtain a training dataset, wherein the samples in the training dataset are remote sensing images; Obtain the low-frequency and high-frequency components corresponding to the sample; The low-frequency components are quantized using the JPEG2000 compression model to obtain quantization results; the quantization results are then decoded to obtain the recovered low-frequency image blocks. The high-frequency components are used to extract features using a neural network encoding module, and the extracted features are then decoded using a neural network decoding module to obtain the recovered high-frequency image blocks. Based on the recovered low-frequency image patch and the low-frequency component, calculate the low-frequency loss; based on the recovered high-frequency image patch and the high-frequency component, calculate the high-frequency loss; multiply the low-frequency loss and the high-frequency loss by their respective weights and sum them up to obtain the final loss. The neural network encoding module is trained based on the final loss to obtain the trained neural network encoding module.

4. A remote sensing image hybrid compression device based on data block complexity classification, characterized in that, include: The image segmentation module is used to divide the original remote sensing image into multiple image blocks; The image complexity determination module is used to analyze the image complexity of each image block and determine whether the image block is a smooth image block or a complex image block based on the image complexity. The first compression module is used to increase the quantization step size for smooth image blocks and perform high compression ratio image compression using the JPEG2000 compression model to obtain the compressed bitstream of the smooth image blocks. The second compression module is used to perform secondary wavelet decomposition on complex image blocks to obtain low-frequency and high-frequency components. The low-frequency components are quantized using the JPEG2000 compression model, and the quantization results are entropy encoded to obtain the coded bitstream of the low-frequency components. The high-frequency components are feature extracted using a neural network coding module, and the extracted features are entropy encoded to obtain the coded bitstream of the high-frequency components. The coded bitstreams of the low-frequency components and the coded bitstreams of the high-frequency components are merged to obtain the compressed bitstream of the complex image patch. The image complexity determination module is also used for: For any image patch A, construct the corresponding image patch A Mosaic B and Mosaic C; the aforementioned The pixels in mosaic B and the pixels in image block A satisfy the following relationship: ， in, Here are the pixel coordinates, and n is a natural number. for Pixels in mosaic B, For pixels in image block A; The The pixels in mosaic C and the pixels in image block A satisfy the following relationship: ， in, for Pixels in mosaic C; Calculate the image complexity of image patch A : in, This indicates calculating the average value. Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic B Indicates the computation of image block A and Peak signal-to-noise ratio of Mosaic C; If the image complexity of image block A is... Less than the threshold If the condition is met, then image block A is a complex image block; otherwise, image block A is a smooth image block.

Images