High-power-ratio micro-loss satellite image data lightweight method

Abstract

The invention designs a high-magnification micro-loss satellite image data lightweight method, which comprises the following steps of: firstly, acquiring a region of interest by using an information extraction model according to different task requirements, and extracting the region of interest and a background region; in the aspect of an interested selection strategy, an image is divided into a key area, an associated area and an auxiliary area, lossless / micro-lossy compression is adopted for the key area and the associated area, lossy compression is adopted for the auxiliary area, and then the compressed image is reconstructed. Compared with the prior art, the method has the advantages that high-magnification light weight of the satellite image data is realized, and under the condition that the light magnification is as high as 20 times, the volume of the satellite image data is greatly reduced, and the transmission, visualization and re-editing efficiency of the satellite image data is greatly improved.

Description

Technical Field

[0001] The invention relates to the field of machine learning, and in particular to a lightweight method for high-magnification and minimally damaged satellite image data. Background Art

[0002] With the development of satellite remote sensing technology, the spatial resolution of satellite images has been continuously improved. The sub-satellite spatial resolution of civilian, commercial and military satellite images has been improved to 0.5 meters. As a result, the amount of satellite image data has increased dramatically, which has brought many challenges in data storage and transmission, and put forward higher requirements for the real-time and accuracy of data. The various types of space-based intelligence and the various formats of intelligence products will cause complex processing application platforms.

[0003] Existing technologies generally use transform coding image compression methods (such as discrete cosine, wavelet transform, etc.) to perform lossy compression on the entire image. Most existing image compression methods use established image features for transformation, and the optimization processes for encoders and decoders are separate from each other, and the compression results obtained are not the best. Remote sensing images are characterized by large widths and small targets. Usually, the target accounts for a much smaller proportion than the background area. These non-interesting areas often increase additional transmission capacity, which is not conducive to image transmission. With the development of deep learning, the processing tasks of computer vision are gradually transitioning from shallow learning technology to deep learning.

[0004] The most direct and effective way to solve this problem is to use big data and machine learning technology to ensure high-quality and accurate data while significantly reducing the amount of satellite image data and effectively reducing the transmission bandwidth without affecting intelligence, combat and other applications and retaining the spatial attributes of satellite images. To this end, the present invention focuses on studying a method for lightweighting satellite image data with high-multiplier and low-loss. Summary of the invention

[0005] In view of the above problems, the present invention discloses a method for lightweighting high-magnification and low-loss satellite image data, which can significantly reduce the amount of satellite image data without reducing the image interpretation capability and improving the transmission rate requirements, thereby effectively reducing the transmission bandwidth.

[0006] The technical solution of the present invention is: a method for lightweighting high-magnification and low-loss satellite image data:

[0007] Step 1: Select the original remote sensing image to be processed and use the information extraction model to extract the region of interest and the background area;

[0008] Step 2: Differentially compress the ROI and the background area; use lossless / near-lossless compression for the ROI and lossy compression for the background area, thereby achieving local image differential compression based on deep learning;

[0009] Step 3: Reconstruct the compressed image;

[0010] Step 4: Evaluate the reconstructed image; evaluate the reconstructed image from two aspects: image distortion after compression and geometric accuracy.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] The present invention designs a high-ratio and slightly lossy satellite image data lightweight method; adopts a region of interest image compression network method to realize the high-ratio and lightweight technology of satellite image data. When the lightweight ratio is as high as 20 times, the volume of satellite image data is greatly reduced, and the transmission, visualization and re-editing efficiency of satellite image data are greatly improved. At the same time, it can achieve the goal of not reducing the intelligence capability of satellite images, not destroying the spatial reference (coordinate system and projection) of satellite image data, and not changing the format of satellite image data, thereby providing effective technical support for the expansion of space-based reconnaissance intelligence support capabilities from strategic to campaign, especially tactical levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart for extracting a region of interest according to the present invention.

[0014] Figure 2 Schematic diagram of the image compression network structure based on the multi-scale region of interest attention module of the present invention.

[0015] Figure 3 It is a technical flow chart of the present invention. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described below in conjunction with the examples. If no specific techniques or conditions are specified in the examples, the techniques or conditions described in the literature in the art or in accordance with the product instructions are used.

[0017] The data used for compression in the embodiment is a high-resolution satellite image of Osan Air Force Base. The present invention designs a lightweight method for high-ratio lossy satellite image data. Under the premise of not reducing the image interpretation ability and improving the transmission rate requirements, a differentiated image compression model based on the region of interest is proposed through deep learning technology, that is, lossless compression of the region of interest and lossy compression of the background area. The model can perform lightweight processing on high-resolution satellite image data, achieve high-ratio compression of remote sensing images (up to 20 times), reduce the network burden of data transmission, keep the file format and spatial reference (coordinate system, projection) unchanged before and after processing, and the image before and after lightweight does not affect the visual interpretation requirements.

[0018] The specific steps are as follows:

[0019] Step 1: Select the original remote sensing image to be processed and use the information extraction model to extract the region of interest and the background area;

[0020] The area of ​​interest includes key areas and related areas; the background area is the auxiliary area; among them, the key areas include: military targets such as airports, bunkers, armored clusters, barracks, oil depots, and positions; the related areas include roads, residential areas, etc.; the auxiliary areas are usually large areas of vegetation, waters, etc.

[0021] Step 1.1: Establish an information extraction model based on deep learning;

[0022] The information extraction model is a FasterR-CNN network model in deep learning;

[0023] like Figure 1 As shown, the FasterR-CNN model has 2 output layers;

[0024] ① Used to predict the classification probability of each proposed area and distinguish between targets and non-targets;

[0025] ②Optimize the offset of each suggested area coordinate to obtain a more accurate target location.

[0026] Step 1.2: Use a region proposal network (such as RPN) to extract the proposed region;

[0027] Step 1.2.1: The initial image is processed through a multi-level convolution module (convolution layer + activation layer + pooling layer) to obtain a feature map;

[0028] Step: 1.2.2: The region proposal network generates candidate regions and classifies them (foreground or background) based on the feature map obtained in step 1.2.1. The process structure is as follows Figure 2 As shown;

[0029] from Figure 2 It can be seen that RPN has 2 branches and 2 parallel fully connected layers:

[0030] ①Window regression layer: This branch predicts the suggested area on the original image based on each position of the feature map;

[0032] ② Classification layer, which is used to predict the probability that the proposed area belongs to the foreground and background (i.e., the area of ​​interest and the background area) to achieve classification of each proposed area.

[0033] Step 2: Differential compression of the region of interest (ROI) and the background area;

[0034] By compressing the area of ​​interest at different bit rates to achieve high-rate compression tasks to meet the needs of lightweight load, the detailed information of the area of ​​interest is guaranteed, while the semantic information around the area of ​​interest is retained, ensuring the reliability and accuracy of image interpretation.

[0035] Specifically, lossless / near-lossless compression is used for the region of interest, and lossy compression is used for the background area, thereby achieving local image differential compression based on deep learning;

[0036] Step 2.1: Construct an image compression network model based on a multi-scale region of interest attention module to perform lossless / near-lossless compression on the region of interest;

[0037] Step 2.1.1: The structure of the attention module based on the region of interest is as follows Figure 3 As shown, the spatial attention mechanism is applied to learn the importance normalized weight of the image spatial position information, and the weight is multiplied by the unprocessed original image to be compressed to obtain the image information after the spatial position information is enhanced;

[0038] Step 2.1.2: Fuse the output of the multi-scale ROI attention module with the original image to achieve image enhancement and supplement spatial position information;

[0039] Step 2.1.3: Use a multi-scale ROI attention module to enhance the ROI in the original image at different scales to achieve better ROI reconstruction performance;

[0040] Step 2.2: Use traditional wavelet algorithm to perform lossy compression on the background area.

[0041] Step 3: Reconstruct the compressed image;

[0042] The multi-scale feature information of each layer obtained in step 2.1.3 is summarized, and the information aggregation decoding subnet is used to realize the decoding and reconstruction of the compressed image. The decoding network upsamples the main super-prior potential feature representation and smaller-scale high-order features to half the resolution of the original image. After feature fusion, the network uses a residual block and superimposed convolutional layers to map the features back to the dimension of the original image, and finally realizes the decoding and reconstruction of the image.

[0043] Step 4: Evaluate the reconstructed image;

[0044] The reconstructed image is evaluated from two aspects: image distortion after compression and geometric accuracy. For image distortion after compression, peak signal-to-noise ratio (PSNR), mean square error (MSE) and structural similarity (SSIM) are used as evaluation indicators; for geometric accuracy error, mean error is used for statistics.

[0045] Step 4.1: Measure the quality of image compression by peak signal-to-noise ratio (PSNR) and mean square error (MSE);

[0046] The quality of image compression is measured by comparing the pixel error between the reconstructed image and the original image. The specific calculation formulas for peak signal-to-noise ratio (PSNR) and mean square error (MSE) are as follows:

[0047] (1)

[0049]

[0050] Among them, x is the original image, y is the compressed and decompressed image, x(i,j) is the pixel value of the original image, y(i,j) is the pixel value of the reconstructed image, and M and N are the width and height of the image.

[0051] Step 4.2: Use structural similarity (SSIM) to determine whether the structural information of the image has changed;

[0052] Compare the similarity between the reconstructed image and the original image to determine whether the structural information of the image has changed; the specific calculation formula of structural similarity (SSIM) is as follows:

[0053]

[0054] In the formula, x is the original image, y is the compressed and decompressed image, μ x is the average value of the original image pixels, μ y is the average value of the pixels of the reconstructed image, is the variance of the original image pixels, is the variance of the pixels of the reconstructed image, σ xy is the covariance of the pixels of the original image and the reconstructed image. 1 =(k 1 L) 2 , c 2 =(k 2 L) 2 is a constant used to maintain stability, L is the dynamic range of pixel values, and k 1、 k 2 is a constant, k 1 =0.01, k 2 =0.03.

[0055] Step 4.3: Use mean square error to calculate geometric accuracy;

[0056] Using Sift feature point extraction, Ransac residual point elimination algorithm and image geo-filtering error points, the geometric accuracy is statistically calculated by the mean error, and the change (displacement) of the feature point position on the reconstructed image relative to the original image is compared to measure the ability of the compression algorithm to maintain the geometric accuracy of the original image.

[0057] The calculation formula of mean error is:

[0058]

[0059] Among them, Δ is the mean error, a i , b i a' is the horizontal and vertical coordinates of the matching point before image compression; i , b' i are the horizontal and vertical coordinates of the image after compression; n is the number of matching points.

[0060] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention without departing from the principles and intent of the present invention.

Claims

1. A method for lightweighting high-magnification and slightly damaged satellite image data, characterized in that: Step 1: Select the original remote sensing image to be processed and use the information extraction model to extract the region of interest and the background area; Step 2: Differentially compress the ROI and the background area; use lossless / near-lossless compression for the ROI and lossy compression for the background area, thereby achieving local image differential compression based on deep learning; Step 3: Reconstruct the compressed image; Step 4: Evaluate the reconstructed image; evaluate the reconstructed image from two aspects: image distortion after compression and geometric accuracy.

2. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 1, characterized in that: In step 1, the area of ​​interest includes key areas and associated areas; the background area is an auxiliary area; among them, the key areas include: airports, bunkers, armored clusters, barracks, oil depots, and positions; the associated areas include roads and residential areas; the auxiliary areas are usually large areas of vegetation and waters.

3. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 2, characterized in that: Step 1 includes: Step 1.1: Establish an information extraction model based on deep learning; the information extraction model is the FasterR-CNN network model in deep learning; the FasterR-CNN model has two output layers; they are used to predict the classification probability of each suggested area and distinguish between targets and non-targets; optimize the offset of the coordinates of each suggested area to obtain the target position; Step 1.2: Use the region proposal network to extract the proposed region.

4. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 3 is characterized in that: In step 1.2, the specific method of extracting the recommended region using the region proposal network is: Step 1.2.1: The initial image is processed through multiple layers of convolution modules to obtain a feature map; Step: 1.2.2: The region proposal network generates candidate regions and classifies them according to the feature map obtained in step 1.2.1 to distinguish between foreground and background; The region proposal network has 2 branches and 2 parallel fully connected layers: ① The window regression layer predicts the suggested area on the original image based on each position of the feature map; ② Classification layer, which is used to predict the probability that the proposed area belongs to the foreground and background, and to classify each proposed area.

5. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 1, characterized in that: In step 2, the specific steps of differentially compressing the region of interest and the background region are as follows: Step 2.1: Construct an image compression network model based on a multi-scale region of interest attention module to perform lossless / near-lossless compression on the region of interest; Step 2.2: Use traditional wavelet algorithm to perform lossy compression on the background area.

6. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 5, characterized in that: The specific method for lossless / near lossless compression of the region of interest is: Step 2.1.1: Based on the attention module structure of the region of interest, apply the spatial attention mechanism to learn the importance normalized weight of the image spatial position information, multiply the weight by the unprocessed original image to be compressed, and obtain the image information after the spatial position information is enhanced; Step 2.1.2: Fuse the output of the multi-scale ROI attention module with the original image to achieve image enhancement and supplement spatial position information; Step 2.1.3: Use a multi-scale ROI attention module to enhance the ROI in the original image at different scales to achieve ROI reconstruction performance.

7. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 1, characterized in that: In step 3, the multi-scale feature information of each layer is summarized, and the information aggregation decoding subnet is used to realize the decoding and reconstruction of the compressed image. The decoding network upsamples the main super-prior latent feature representation and smaller-scale high-order features to half the resolution of the original image. After feature fusion, the network uses a residual block and superimposed convolutional layers to map the features back to the dimension of the original image, and finally realizes the decoding and reconstruction of the image.

8. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 1, characterized in that: In step 4, the peak signal-to-noise ratio, mean square error and structural similarity are used as evaluation indicators for the distortion of the compressed image; the mean error is used to calculate the geometric accuracy error.

9. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 8, characterized in that: The specific calculation formula of structural similarity (SSIM) is as follows: In the formula, SSIM is the structural similarity, μ x is the mean value of x, μ y is the mean value of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y, and c1 is a constant used to maintain stability.

10. The method for lightweighting high-magnification and minimally-damaged satellite image data according to claim 8, characterized in that: The calculation formula of mean error is: Among them, Δ is the mean error, a i , b i a' is the horizontal and vertical coordinates of the matching point before image compression; i , b' i are the horizontal and vertical coordinates of the image after compression; n is the number of matching points.

Images