High-resolution remote sensing image compression method, equipment and product based on cross-image prior

Abstract

The present invention discloses a high-resolution remote sensing image compression method, device and product based on cross-image prior, which first splits the original image into multiple sub-images, and then encodes and decodes each sub-image one by one, so that when encoding subsequent sub-images, all the information of the previous sub-images can be fully utilized. This prior information is called cross-image prior information. When encoding the first image, there is no previously encoded sub-image, so only the super-prior extracted from the original image is used to construct the entropy model, and when encoding the last sub-image, the cross-image prior information of all previously encoded sub-images can be used to construct the entropy model. When using cross-image prior, subsequent sub-images can use all the information of the previous sub-images, and also implicitly use local and non-local context information, so that the local and non-local redundancy existing in high-resolution remote sensing images can be better mined, thereby improving the final compression performance.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing technology, and relates to a remote sensing image compression method, equipment and product, and specifically to a high-resolution remote sensing image compression method, equipment and product based on cross-image prior. Background Art

[0002] High-resolution remote sensing images have high spatial resolution, providing clear edge details for Earth observation. These details provide rich, detailed information for subsequent image processing tasks such as object recognition. Because high-resolution remote sensing images typically have large spatial extents, a single high-resolution image typically occupies a significant amount of space, placing a significant burden on their storage and compression. Therefore, compressing high-resolution remote sensing images is crucial for reducing storage space and conserving bandwidth.

[0003] The current mainstream image compression algorithms are mainly designed for ordinary natural images. However, high-resolution remote sensing images are significantly different from ordinary natural images. Using image compression algorithms based on natural images to compress high-resolution remote sensing images will result in poor compression performance. Some researchers have also proposed some compression algorithms for high-resolution remote sensing images. However, the current lossless compression of high-resolution remote sensing images has the following limitations:

[0004] (1) Some studies have shown that context extraction based on autoregression can better mine local context information in images and achieve better performance. However, the use of autoregression will cause the compression and encoding and decoding processes to be performed pixel by pixel, which seriously hinders the encoding and decoding speed.

[0005] (2) High-resolution remote sensing images have large spatial sizes, and a large number of similar objects are widely present in local and non-local areas. However, existing algorithms fail to fully exploit these local and non-local spatial redundancies, resulting in insufficient compression performance.

[0006] (3) The sizes of objects and backgrounds in high-resolution remote sensing images are quite different from those in ordinary natural images. Image compression algorithms designed for natural images cannot perform probabilistic modeling on high-resolution remote sensing images well, thus hindering the final compression performance.

[0007] Therefore, there is an urgent need for an efficient (avoiding the use of autoregressive) lossless compression method for high-resolution remote sensing images that can fully exploit local and non-local spatial redundancy. Summary of the Invention

[0008] In order to provide a high-resolution remote sensing image compression solution that avoids the use of autoregressive models while being able to efficiently mine local and non-local features, the present invention provides a high-resolution remote sensing image compression method, device, and product based on cross-image priors.

[0009] In a first aspect, the present invention provides a high-resolution remote sensing image compression method based on cross-image priors, comprising the following steps:

[0010] Step 1: Split the original remote sensing image into several sub-images, and the pixels at the same position in the split sub-images belong to adjacent positions on the original remote sensing image;

[0011] Step 2: Use the analysis network to extract the latent representation information from the sub-image. After quantization, use the factorized entropy model to build an entropy model for each pixel of the latent representation information. Then calculate the probability of each pixel. Using this entropy model and the probability value, use the entropy coding algorithm to encode the latent representation information to obtain the bit stream bit_0;

[0012] Step 3: Use the synthesis network to obtain hyperprior information from the potential representation information as the global hyperprior information for all subsequent sub-image compression;

[0013] Step 4: Use the entropy parameter prediction network to build the entropy model of each sub-image, and then use the entropy coding algorithm to encode;

[0014] The entropy parameter pre-network includes a sequentially connected convolutional layer, a residual module, a convolutional layer, a residual module and a convolutional layer; the residual module is composed of a sequentially connected Norm+ReLU layer, a conv1 convolutional layer, a Norm+ReLU layer, a conv3 convolutional layer, a NormReLU layer and a conv1 convolutional layer;

[0015] When encoding sub-image 1, the input of the entropy parameter prediction network is only the hyperprior information. When encoding subsequent sub-images, the previously encoded sub-image information, i.e., the cross-image prior information, is used to input the hyperprior information and the cross-image prior information into the CT module to obtain the prior information of the subsequent sub-images. The information is then input into the entropy parameter prediction network to predict the entropy model parameters to obtain the entropy model. The obtained entropy model is used to obtain the bitstream bit_1, ..., bit_N using the entropy coding algorithm, where N is the number of sub-images.

[0016] Step 5: Combine all the code streams together to obtain the final lossless image compression result.

[0017] Preferably, in step 2, the analysis network is composed of four conv 5 convolutional layers connected sequentially.

[0018] Preferably, in step 4, the entropy model is to construct a Gaussian mixture model for each pixel of the quantized depth potential representation. The entropy model of each pixel has k*3 parameters that need to be predicted, where k represents the number of Gaussian models in the Gaussian mixture model; then the probability value p of each point is calculated, and the probability value is used to count the entropy value of the deep potential representation; where The parameter w of the Gaussian mixture model i , μ i , δ i The prediction is made through the entropy parameter prediction network Parameter, [w i ,μ i ,δ i ]=Parameter(Hyperprior).

[0019] Preferably, in step 3, the synthetic network is composed of four deconv 5 deconvolution layers connected sequentially.

[0020] As an advantage, in step 3, the global super prior information Y=G a (X), where X represents the original image data, Y, Respectively represent the potential representation information before and after quantization, U|Q represents the quantization process; G a , G s They represent the backbone analysis network and the synthesis network respectively; the backbone analysis network is composed of four sequentially connected convolutional layers; the synthesis network is composed of four sequentially connected deconvolutional layers.

[0021] Preferably, in step 4, the CT module passes the cross-image prior information through a convolutional layer and then concatenates it with the hyperprior information, and inputs it into the MixCNN and Transformer hybrid modules to respectively mine local and non-local information; then the outputs of the two parts are concatenated as the prior knowledge for the final entropy model parameter prediction;

[0022] The MixCNN consists of four convolutional layers with kernel sizes of 1, 3, 5, and 7 and a stride of 1 connected in parallel;

[0023] The Transformer consists of a Norm layer, an Attention layer, a Norm layer, and a Channel MLP connected in sequence. The input and the output of the Attention layer are added pixel by pixel before being input into the Norm layer. Meanwhile, the input and the output of the Attention layer are added pixel by pixel before being added pixel by pixel with the Channel MLP and output.

[0024] Preferably, in step 4, for the first encoded sub-image X_1, the global hyper-prior information hyperprior extracted from the original image is used as the prior information, and its entropy value is: H(X_1)=H(X_1|Hyperprior); when subsequent sub-images are encoded, the entropy value is H(X_{k+1})=H(X_{k+1}|X_1,X_2,...,X_k,Hyperprior); wherein X_1,X_2,...,X_k,X_{k+1} represent k+1 sub-images.

[0025] Preferably, when performing network training, that is, training the analysis transformation network, the synthesis transformation network, the CT module and the entropy parameter prediction network, the loss function is the sum of the entropy values ​​of all coefficients that need to be stored. During training, the quantization process uses noise-added simulation quantization, while the testing process uses integer quantization.

[0026] In a second aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the high-resolution remote sensing image compression method based on cross-image prior when executing the program.

[0027] In a third aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the high-resolution remote sensing image compression method based on cross-image prior.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] (1) The present invention splits the original image into multiple sub-images and performs encoding and decoding on a sub-image-by-sub-image basis. When encoding subsequent sub-images, the information of all previously encoded sub-images can be used as a cross-image prior to obtain a more accurate entropy prediction model, thereby improving the final compression performance.

[0030] (2) When the present invention uses cross-image priors, the information of the previously encoded sub-image already contains local and non-local redundant information. By mining this local and non-local redundant information through a neural network, the performance of the compression network can be improved.

[0031] (3) The entropy encoding and decoding employed in the present invention do not use any autoregressive context, that is, when performing entropy prediction on the entropy model of the current pixel, they do not rely on any pixels before the current pixel. Therefore, when encoding and decoding a single sub-image, the processes can be parallelized, thereby improving the final encoding and decoding speed. This is of great value and significance for real-time compression, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The technical solution of the present invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without making any creative efforts.

[0033] Figure 1 Schematic diagram of the method in an embodiment of the present invention.

[0034] Figure 2 This is a diagram of the entropy parameter prediction network structure in an embodiment of the present invention;

[0035] Figure 3 This is a network structure diagram of the CT module in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0037] Taking into account the large spatial scale of high-resolution remote sensing images and the high spatial local and non-local redundancy. Taking into account that large-scale high-resolution images will result in poor performance if local context priors based on autoregression are not used, and the use of priors based on autoregression will result in pixel-by-pixel decoding of prior information, which seriously affects the decoding speed. The present invention provides a high-resolution remote sensing image compression method based on cross-image priors, which splits the original image into multiple sub-images and encodes and decodes them sub-image by sub-image. When encoding subsequent sub-images, the information of the previously encoded sub-image is used as a cross-image prior for constructing an entropy model for the subsequent sub-image compression. While avoiding the use of autoregressive context, the local and non-local redundant information in the image is fully exploited.

[0038] At the same time, global super-prior information is introduced. When encoding and decoding the first sub-image, no previously encoded sub-images are available, and therefore no cross-image priors are available. Yinri's global super-prior information solves the problem of lacking a prior when modeling the first sub-image. Furthermore, this global super-prior information can be used for all subsequently encoded sub-images. By fully utilizing the available information in the image, the proposed algorithm constructs a more accurate entropy model, thereby improving the final compression effect.

[0039] In addition, when the entropy model is being constructed, a more accurate entropy model can obtain a lower entropy value during encoding and decoding, thereby achieving a higher compression ratio and a lower bit rate. However, the model representation capability of a conventional single Gaussian mixture model is limited, and the learned entropy model may differ from the actual distribution of the image data, thereby limiting the final image compression performance to a certain extent. Therefore, the present invention utilizes a Gaussian mixture model to perform entropy modeling.

[0040] Please see Figure 1 The present invention provides a high-resolution remote sensing image compression method based on cross-image prior, comprising the following steps:

[0041] Step 1: Split the original remote sensing image into several sub-images, and the pixels at the same position in the split sub-images belong to adjacent positions on the original remote sensing image;

[0042] Step 2: Use the analysis network to extract the latent representation information from the sub-image. After quantization, use the factorized entropy model to build an entropy model for each pixel of the latent representation information. Then calculate the probability of each pixel. Using this entropy model and the probability value, use an entropy coding algorithm such as arithmetic coding to encode the latent representation information to obtain the bit stream bit_0.

[0043] Please see Figure 1 In one embodiment, the analysis network is composed of four sequentially connected conv 5 convolutional layers. In the figure, conv / deconv 5 2 represents a convolution / deconvolution layer with a convolution kernel of 5 and a stride of 2.

[0044] Step 3: Use the synthesis network to obtain hyperprior information from the potential representation information as the global hyperprior information for all subsequent sub-image compression;

[0045] Please see Figure 1 ,In one embodiment, the synthetic network ,is composed of four deconv 5 deconvolution layers connected sequentially.

[0046] In one embodiment, the global super prior information Y=G a (X), where X represents the original image data, Y, Respectively represent the potential representation information before and after quantization, U|Q represents the quantization process; G a , G s They represent the backbone analysis network and the synthesis network respectively; the backbone analysis network is composed of four sequentially connected convolutional layers; the synthesis network is composed of four sequentially connected deconvolutional layers.

[0047] Step 4: Use the entropy parameter prediction network to build the entropy model of each sub-image, and then use the entropy coding algorithm to encode;

[0048] Please see Figure 2 In one embodiment, the entropy parameter pre-network includes a sequentially connected convolutional layer, a residual module, a convolutional layer, a residual module, and a convolutional layer; the residual module is composed of a sequentially connected Norm+ReLU layer, a conv1 convolutional layer, a Norm+ReLU layer, a conv3 convolutional layer, a NormReLU layer, and a conv1 convolutional layer;

[0049] In one embodiment, the entropy model is to construct a Gaussian mixture model for each pixel of the quantized potential representation. The entropy model of each pixel has k*3 parameters to be predicted, where k represents the number of single Gaussian models in the Gaussian mixture model. Then the probability value p of each point is calculated, and the probability value is used to calculate the entropy value of the potential representation. The parameter w of the Gaussian mixture model i , μ i , δ i The prediction is made through the entropy parameter prediction network Parameter, [w i ,μ i ,δ i ]=Parameter(Hyperprior).

[0050] When encoding sub-image 1, the input of the entropy parameter prediction network is only the hyperprior information. When encoding subsequent sub-images, the previously encoded sub-image information, i.e., the cross-image prior information, is used to input the hyperprior information and the cross-image prior information into the CT module to obtain the prior information of the subsequent sub-images. The information is then input into the entropy parameter prediction network to predict the entropy model parameters to obtain the entropy model. The obtained entropy model is used to obtain the bitstream bit_1, ..., bit_N using the entropy coding algorithm, where N is the number of sub-images.

[0051] Please see Figure 3 In one embodiment, the CT module passes the cross-image prior information through a convolutional layer and then concatenates it with the hyperprior information. The concatenation is then fed into the MixCNN and Transformer hybrid modules to mine local and non-local information respectively. The outputs of the two parts are then concatenated to serve as the prior knowledge for the final entropy model parameter prediction.

[0052] The MixCNN consists of four convolutional layers with kernel sizes of 1, 3, 5, and 7 and a stride of 1 connected in parallel;

[0053] The Transformer consists of a sequentially connected Norm layer, an Attention layer, a Norm layer, and a Channel MLP. The input and the output of the Attention layer are added pixel by pixel before being fed into the Norm layer. Simultaneously, the input and the output of the Attention layer are added pixel by pixel before being added to the Channel MLP and output. In the figure, conv k 1 indicates a convolutional layer with a convolution kernel of k and a stride of 1.

[0054] In one embodiment, in step 4, for the first encoded sub-image X_1, the global hyper-prior information hyperprior extracted from the original image is used as the prior information, and its entropy value is: H(X_1)=H(X_1|Hyperprior); when subsequent sub-images are encoded, the entropy value is H(X_{k+1})=H(X_{k+1}|X_1,X_2,...,X_k,Hyperprior); where X_1,X_2,...,X_k,X_{k+1} represent k+1 sub-images.

[0055] Step 5: Combine all the code streams together to obtain the final lossless image compression result.

[0056] In the network proposed in this embodiment, the information that needs to be stored includes the potential representation and all sub-image information. Therefore, during network training, the loss function is the number of bits required to store these components. According to Shannon's theorem, the lower limit of the bit rate for lossless compression is the entropy value. Many current entropy coding algorithms, such as arithmetic coding and ANS coding, can achieve coding efficiencies close to the entropy value. Therefore, the loss function of the proposed algorithm can be expressed as follows:

[0057]

[0058] In the above loss function, H(·) represents the calculation of entropy. If we know the probability model of the pixel, then the probability value table of this point can be known. Assuming it is p, then the entropy value at this time can be expressed as:

[0059] H=-p*log p;

[0060] When constructing the entropy model, assuming that the model is a Gaussian mixture model, then the probability P should obey the Gaussian mixture distribution, that is:

[0061]

[0062] Among them, w i Represents the i-th Gaussian model N(μ i ,δ i ) weight, μ i ,δi Represents the mean and variance of the Gaussian model. i The sum of is 1.

[0063] Through end-to-end training, the network parameters can be obtained. It is worth mentioning that when storing the potential representation data, a quantization process is required. The quantization process is non-differentiable, which will hinder the gradient return during the network training process, making the network difficult to train. Therefore, during network training, this embodiment uses noise to simulate quantization. However, during actual compression, gradient return is not required, so this embodiment directly uses the rounding quantization method.

[0064] In one embodiment, adding noise to simulate quantization refers to adding Gaussian noise with a mean of 0 and a variance of 1 to the potential representation, while rounding quantization refers to rounding each pixel value of the potential representation to an integer.

[0065] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the high-resolution remote sensing image compression method based on cross-image prior is implemented.

[0066] This embodiment also provides a computer program product, including a computer program, which implements the high-resolution remote sensing image compression method based on cross-image prior when executed by a processor.

[0067] The present invention first splits the original image into multiple sub-images, where pixels at corresponding positions in the sub-images are adjacent pixels in the original image. Super-prior information is then extracted from all sub-images (or the original image). This involves using an analysis network to obtain a latent representation, storing it using a factorized model, and then using a synthetic transformation to obtain the super-prior information. Sub-image encoding is then performed one by one. When encoding sub-image 1, the entropy parameter prediction network only inputs the super-prior information. When encoding subsequent sub-images, information from previously encoded sub-images, namely cross-image prior information, can be utilized. The super-prior and cross-image prior are then input into the CT module to obtain a cross-image prior, which is then input into the entropy prediction network. When obtaining the entropy model parameters for each sub-image, such as the parameters of the final Gaussian mixture model, the cumulative probability density of each pixel in the image can be calculated using this parameter. Once this parameter is obtained, entropy encoding and decoding can be performed on each pixel individually. Since the entropy model construction for each pixel is independent throughout the entire process, the present invention has great potential for parallelization.

[0068] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0069] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A high-resolution remote sensing image compression method based on cross-image priors, characterized in that: The following steps are involved: Step 1: Split the original remote sensing image into several sub-images, and the pixels at the same position in the split sub-images belong to adjacent positions on the original remote sensing image; Step 2: Use the analysis network to extract the latent representation information from the sub-image. After quantization, use the factorized entropy model to build an entropy model for each pixel of the latent representation information. Then calculate the probability of each pixel. Using this entropy model and the probability value, use the entropy coding algorithm to encode the latent representation information to obtain the bit stream bit_0; Step 3: Use the synthesis network to obtain hyperprior information from the potential representation information as the global hyperprior information for all subsequent sub-image compression; Step 4: Use the entropy parameter prediction network to build the entropy model of each sub-image, and then use the entropy coding algorithm to encode; The entropy parameter prediction network includes a sequentially connected convolutional layer, a residual module, a convolutional layer, a residual module and a convolutional layer; the residual module is composed of a sequentially connected Norm+ReLU layer, a conv1 convolutional layer, a Norm+ReLU layer, a conv3 convolutional layer, a NormReLU layer and a conv1 convolutional layer; When encoding sub-image 1, the input of the entropy parameter prediction network is only the hyperprior information. When encoding subsequent sub-images, the previously encoded sub-image information, i.e., the cross-image prior information, is used to input the hyperprior information and the cross-image prior information into the CT module to obtain the prior information of the subsequent sub-images. The information is then input into the entropy parameter prediction network to predict the entropy model parameters to obtain the entropy model. The obtained entropy model is used to obtain the bitstream bit_1, ..., bit_N using the entropy coding algorithm, where N is the number of sub-images. Step 5: Combine all the code streams together to obtain the final lossless image compression result.

2. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 2, the analysis network consists of four sequentially connected convolutional layers with a kernel of 5 and a stride of 2.

3. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 4, the entropy model is to construct a Gaussian mixture model for each pixel of the quantized potential representation The entropy model of each pixel has k*3 parameters to be predicted, where k represents the number of single Gaussian models in the Gaussian mixture model. Then the probability value p of each point is calculated, and the probability value is used to calculate the entropy value of the potential representation. The parameter w of the Gaussian mixture model i , μ i , δ i The prediction is made through the entropy parameter prediction network Parameter, [w i ,μ i ,δ i ]=Parameter()。 4. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 3, the synthetic network consists of four sequentially connected deconv 5 deconvolution layers with a convolution kernel of 5 and a stride of 2.

5. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 3, the global super prior information Y=G a (), where X represents the original image data, Y, Respectively represent the potential representation information before and after quantization, U|Q represents the quantization process; G a , G s They represent the backbone analysis network and the synthesis network respectively; the backbone analysis network is composed of four sequentially connected convolutional layers; the synthesis network is composed of four sequentially connected deconvolutional layers.

6. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 4, the CT module passes the cross-image prior information through a convolutional layer and then concatenates it with the hyperprior information. The information is then input into the MixCNN and Transformer hybrid modules to mine local and non-local information respectively. The outputs of the two parts are then concatenated as the prior knowledge for the final entropy model parameter prediction. The MixCNN consists of four convolutional layers with kernel sizes of 1, 3, 5, and 7 and a stride of 1 connected in parallel; The Transformer consists of a Norm layer, an Attention layer, a Norm layer, and a Channel MLP connected in sequence. The input and the output of the Attention layer are added pixel by pixel before being input into the Norm layer. Meanwhile, the input and the output of the Attention layer are added pixel by pixel before being added pixel by pixel with the Channel MLP and output.

7. The high-resolution remote sensing image compression method based on cross-image prior according to claim 1, characterized in that: In step 4, for the first encoded sub-image X_1, the global hyper-prior information hyperprior extracted from the original image is used as the prior information, and its entropy value is: H(X_1) = H(X_1|Hyperprior); when subsequent sub-images are encoded, the entropy value is H(X_{k+1}) = H(X_{k+1}|X_1,X_2,...,X_k,Hyperprior); where X_1,X_2,...,X_k,X_{k+1} represent k+1 sub-images.

8. The high-resolution remote sensing image compression method based on cross-image prior according to any one of claims 1 to 7, characterized in that: When training the network, that is, training the analysis network, synthesis network, CT module and entropy parameter prediction network, the loss function is the sum of the entropy values ​​of all coefficients that need to be stored. The quantization process during training uses noise simulation quantization, while the testing process uses integer quantization; noise simulation quantization refers to adding Gaussian noise with a mean of 0 and a variance of 1 to the potential representation, while integer quantization rounds each pixel value of the potential representation to an integer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the high-resolution remote sensing image compression method based on cross-image prior as described in any one of claims 1 to 8 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the high-resolution remote sensing image compression method based on cross-image prior as claimed in any one of claims 1 to 8 is implemented.

Images