Vector diagram assistance-based remote sensing image ultra-low bit rate compression method and device

Abstract

The invention relates to the technical field of satellite remote sensing images, in particular to a remote sensing image ultra-low code rate compression method and device based on vector diagram assistance, and the method comprises the steps: obtaining a remote sensing image set and a vector diagram of each remote sensing image in the remote sensing image set, so as to construct a training set and a test set; performing optimization training on a pre-constructed remote sensing image ultra-low bit rate compression network based on vector diagram assistance by using the training set, so as to obtain a trained remote sensing image ultra-low bit rate compression network based on vector diagram assistance, the vector diagram assistance-based remote sensing image ultra-low code rate compression network comprises a hidden space compression module and a conditional diffusion model; and inputting the test set into the trained vector diagram assistance-based remote sensing image extremely low bit rate compression network so as to reconstruct a compressed remote sensing image. Therefore, the problem that an image with high perceptual quality and accurate semantic information cannot be reconstructed at an extremely low bit rate by a traditional codec and a method based on deep learning is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of satellite remote sensing images, and in particular to a method and device for extremely low bit rate compression of remote sensing images based on vector graphics assistance. Background Art

[0002] With the rapid development of remote sensing technology, the amount of high-resolution image data acquired has increased dramatically, which poses a huge challenge to the storage and transmission of images. In practical application scenarios, people urgently need a technology that can compress remote sensing images at extremely low bit rates. Application examples of this technology include: efficient long-term storage and analysis in systems with limited storage space in cloud infrastructure; and faster data transmission in emergency communication networks that may have bandwidth limitations. The main challenge in developing such compression technology is how to maintain the basic characteristics of remote sensing images while greatly reducing the bit rate, ensuring that the decompressed images can still be used accurately and reliably in a variety of remote sensing applications.

[0003] In the past few decades, the mainstream technologies in the field of image compression are mostly based on frequency domain analysis, such as JPEG, JPEG2000, BPG and VTM standards. These technologies generally adopt block-based transform coding architectures and rely on manually designed features and heuristic rules to eliminate redundant information in images. However, when compressed to extremely low bit rates, these traditional methods will lead to severe loss of image information, thereby weakening the practicality of remote sensing images in the interpretation and analysis process.

[0004] In recent years, deep learning-based methods, represented by variational autoencoders (VAE) and generative adversarial networks (GANs), have shown better performance than traditional methods in the field of image compression. These methods first downsample the input image at the encoding end to generate a compact feature representation that contains most of the key information about the original image. Subsequently, the feature representation is quantized and entropy encoded and transmitted to the decoding end. During the decoding process, the image is reconstructed through an upsampling process symmetrical to the encoding end. This learning-based image compression framework adopts a data-driven end-to-end training strategy, which can adaptively learn the optimal weight configuration based on the specific content of the image, and has shown excellent potential in many image compression tasks.

[0005] However, the methods mentioned above only show acceptable performance at normal bit rates. At extremely low bit rates, i.e., less than 0.1 bits per pixel (bpp), these methods often produce severe blurring and erroneous textures, which is unacceptable for remote sensing images with practical use value. Therefore, it is necessary to explore a method that is less dependent on bit rate and can achieve efficient compression of remote sensing images even in low-end bit rate scenarios. Summary of the invention

[0006] The present invention provides a method and device for extremely low bit rate compression of remote sensing images based on vector graphics assistance, so as to solve the problem that traditional codecs and methods based on deep learning cannot reconstruct images with high perceptual quality and accurate semantic information at extremely low bit rates.

[0007] The first aspect of the present invention provides a method for ultra-low bit rate compression of remote sensing images based on vector diagram assistance, comprising the following steps: obtaining a remote sensing image set and a vector diagram of each remote sensing image in the remote sensing image set to construct a training set and a test set; using the training set to optimize and train a pre-constructed ultra-low bit rate compression network for remote sensing images based on vector diagram assistance to obtain a trained ultra-low bit rate compression network for remote sensing images based on vector diagram assistance, wherein the ultra-low bit rate compression network for remote sensing images based on vector diagram assistance includes a latent space compression module and a conditional diffusion model; and inputting the test set into the trained ultra-low bit rate compression network for remote sensing images based on vector diagram assistance to reconstruct a compressed remote sensing image.

[0008] Optionally, the acquiring of a remote sensing image set and a vector map corresponding to each remote sensing image to construct a training set includes:

[0009] A remote sensing image dataset of a target area is obtained to select the remote sensing image set from the remote sensing image dataset; a vector map of a corresponding area is obtained according to the geographical location of each remote sensing image to construct a vector map set; and the training set and the test set are generated according to the remote sensing image set and the vector map set.

[0010] Optionally, the latent space compression module includes a main transformation network and an entropy model, wherein the main transformation network includes an encoder and a decoder, and the entropy model includes a super-prior encoder, a super-prior decoder and a channel context model; the conditional diffusion model includes a pre-trained U-Net network, a pre-trained encoder, a pre-trained decoder, a semantic encoder and a semantic adaptation module.

[0011] Optionally, the optimizing training of a pre-constructed vector graphics-assisted remote sensing image ultra-low bit rate compression network using the training set to obtain a trained vector graphics-assisted remote sensing image ultra-low bit rate compression network includes:

[0012] The remote sensing image set in the training set is mapped using a pre-trained encoder to obtain a feature representation set; the latent space compression module is iteratively compressed using the feature representation set until a target rate-distortion loss function converges to obtain a compressed feature representation set; a feature representation set with noise is obtained based on the remote sensing image set in the training set; the conditional diffusion model is iteratively optimized using the vector map in the training set, the compressed feature representation set and the feature representation set with noise until the target diffusion model loss function converges to obtain the trained vector map-assisted remote sensing image ultra-low bit rate compression network.

[0013] Optionally, the solution of the rate-distortion loss function is:

[0014] L 1st =L rate +λL 1d

[0015]

[0016] Among them, L 1st is the rate-distortion loss function of the first stage, L rate is the bit rate loss, L ld is the distortion of each feature representation in the feature representation set, λ is the Laplace multiplier used to control the bit rate and distortion, and p z is the probability distribution of each feature representation in the feature representation set, and They are and The probability distribution of .

[0017] Optionally, the solution of the conditional diffusion model loss function is:

[0018]

[0019] Among them, L 2nd is the conditional diffusion model loss function of the second stage, ∈ is the randomly sampled Gaussian noise, ∈ θ is the noise predicted by the pre-trained U-Net network, c is the condition of the conditional diffusion model, and t is the time step of the conditional diffusion model.

[0020] The second aspect of the present invention provides an extremely low bit rate compression device for remote sensing images based on vector diagram assistance, including: a construction module, used to obtain a remote sensing image set and a vector diagram of each remote sensing image in the remote sensing image set to construct a training set and a test set; a training module, used to use the training set to optimize and train a pre-constructed extremely low bit rate compression network for remote sensing images based on vector diagram assistance to obtain a trained extremely low bit rate compression network for remote sensing images based on vector diagram assistance, wherein the extremely low bit rate compression network for remote sensing images based on vector diagram assistance includes a latent space compression module and a conditional diffusion model; a compression module, used to input the test set into the trained extremely low bit rate compression network for remote sensing images based on vector diagram assistance to reconstruct a compressed remote sensing image.

[0021] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vector graphics-assisted remote sensing image ultra-low bit rate compression method as described in the above embodiment.

[0022] A fourth aspect of the present invention provides a computer program product, which, when executed by a processor, implements the above-mentioned vector diagram-assisted remote sensing image ultra-low bit rate compression method.

[0023] A fifth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned vector diagram-assisted remote sensing image ultra-low bit rate compression method.

[0024] The embodiment of the present invention proposes a method and device for ultra-low bit rate compression of remote sensing images assisted by vector diagrams. The remote sensing image and the vector diagram of the corresponding area are input into the constructed ultra-low bit rate compression network of remote sensing images assisted by vector diagrams. The vector diagram is used to provide additional structural information in the latent space compression module to further reduce the bit rate. The conditional diffusion model is used as a semantic guide to assist in generating a remote sensing image with correct semantic information. That is, the additional structural information of the vector diagram is used to compress the remote sensing image to an ultra-low bit rate bit stream at the encoding end, and the vector diagram is used to provide semantics to reconstruct the remote sensing image from the bit stream at the decoding end, so that a high-fidelity compressed remote sensing image can be efficiently reconstructed at an ultra-low bit rate.

[0025] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 A flowchart of a method for extremely low bit rate compression of remote sensing images based on vector graphics assistance provided by an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a specific implementation of a vector diagram-assisted remote sensing image ultra-low bit rate compression method provided by an embodiment of the present invention;

[0029] Figure 3 A two-stage implementation flow chart provided by an embodiment of the present invention;

[0030] Figure 4 A deep learning network flow chart of a latent space compression module provided in an embodiment of the present invention;

[0031] Figure 5 A network structure diagram of SPADE-Block, ResBlock, SPADE-ResBlock, and Basic-Block provided in an embodiment of the present invention;

[0032] Figure 6 A network structure diagram of a semantic encoder and a semantic adaptation module SAM provided in an embodiment of the present invention;

[0033] Figure 7 A block diagram of a remote sensing image ultra-low bit rate compression device based on vector diagram assistance provided by an embodiment of the present invention;

[0034] Figure 8 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0036] The following describes a method and device for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to an embodiment of the present invention with reference to the accompanying drawings.

[0037] Figure 1 The present invention provides a flowchart of a method for extremely low bit rate compression of remote sensing images based on vector diagram assistance provided by an embodiment of the present invention.

[0038] like Figure 1 As shown, the extremely low bit rate compression method of remote sensing images based on vector diagram assistance includes the following steps:

[0039] In step S101, a remote sensing image set and a vector map of each remote sensing image in the remote sensing image set are obtained to construct a training set and a test set.

[0040] In some embodiments, obtaining a remote sensing image set and a vector map corresponding to each remote sensing image to construct a training set includes:

[0041] Acquire a remote sensing image dataset of a target area to select a remote sensing image set in the remote sensing image dataset;

[0042] According to the geographical location of each remote sensing image, a vector map of the corresponding area is obtained to construct a vector map set;

[0043] Generate training sets and test sets based on remote sensing image sets and vector graphics sets.

[0044] In the actual implementation process, in order to train and test the vector-assisted remote sensing image ultra-low bit rate compression network pre-constructed by the present invention, the embodiment of the present invention collects pairs of remote sensing images from different regions, and performs preprocessing operations such as normalization and cropping on them. The vector map of the corresponding area is obtained according to the geographical location of the remote sensing image, and is cropped to the same size as the remote sensing image. The preprocessed remote sensing image set and vector map set are used as training set and test set, and are divided into training set and test set according to a ratio of 9:1.

[0045] It should be noted that all remote sensing images are from World Imagery, and all vector images are obtained from Open Street Map. The resolution of each image is 256×256 pixels, and the spatial resolution is 1.07 meters / pixel.

[0046] In step S102, the pre-constructed vector graphics-assisted remote sensing image ultra-low bit rate compression network is optimized and trained using the training set to obtain a trained vector graphics-assisted remote sensing image ultra-low bit rate compression network, wherein the vector graphics-assisted remote sensing image ultra-low bit rate compression network includes a latent space compression module (Latent Compression Module, LCM) and a conditional diffusion model.

[0047] In some embodiments, the latent space compression module includes a main transformation network and an entropy model, wherein the main transformation network LCM includes an encoder and a decoder, the entropy model includes a super prior encoder, a super prior decoder and a channel context model; the conditional diffusion model includes a pre-trained U-Net network, a pre-trained encoder, a pre-trained decoder, a semantic encoder and a semantic adaptation module (Semantic Adapter Module, SAM).

[0048] It should be noted that if Figure 3As shown, the vector graphics-assisted remote sensing image ultra-low bit rate compression network pre-constructed in an embodiment of the present invention includes a latent space compression module LCM and a conditional diffusion model.

[0049] like Figure 4 As shown in the figure, the main transformation network in the latent space compression module LCM consists of an encoder g a and decoder g s It is used to compress and decompress the feature representation from remote sensing images. The specific process is as follows:

[0050] y=g a (z,m)

[0051]

[0052]

[0053] Among them, z is the feature representation from the remote sensing image, m is the vector graph tensor, and y and is a compact latent variable and a quantized latent variable, Q is quantized, Feature representation reconstructed for the decoding end.

[0054] like Figure 5 As shown, the encoder g a It is composed of a stack of downsampling convolutional layers and residual blocks ResBlock. Decoder g s The pixel shuffle method is used to upsample the features and perform the same operation as the encoder g a Symmetric inverse transformation process. ResBlock consists of two Basic-Blocks and residual connection. Basic-Block consists of convolution layer, LayerNorm layer and LeakyRelu activation function.

[0055] like Figure 5 As shown, the semantic information of the vector graph is passed through the encoder g a and decoder g s The SPADE-Block in is introduced into the compression of feature representation. The SPADE-Block receives the input features and the semantic features generated by the vector map. The input features are concatenated with the semantic features after passing through the BatchNorm layer and input to the Basic-Block and convolution layer to generate the scale factor γ and bias β. γ and β are multiplied and added to the input feature tensor pixel by pixel respectively. The whole process can be expressed by the following formula:

[0056] f bn =BatchNorm(f in )

[0057] γ=Convγ (BasicBlk([f bn , f sem ])

[0058] β=Conv β (BasicBlk([f bn , f sem ]))

[0059]

[0060] Among them, f in 、f bn 、f sem and f out They are input features, normalized features, semantic features, and features output by SPADE-Block. BasicBlk represents Basic-Block, Conv γ and Conv β are the convolutional layers used to obtain γ and β, respectively.

[0061] Furthermore, if Figure 4 As shown, the entropy model consists of a super prior encoder h a , Super Prior Decoder h s , and the channel context model, which is used to estimate the quantized latent variables The probability of is used to assist entropy coding (decoding). During coding (decoding), the quantized latent variables Divide into K slices and encode (decode) them in sequence. When encoding (decoding) the current slice, use the encoded (decoded) slices as context conditions to assist in predicting the probability of the current slice through the autoregressive method. The entire entropy model can be expressed as the following formula:

[0062] h=h a (y)

[0063]

[0064]

[0065] Among them, h is the side information, y is the compact latent variable, Q is the quantization, is the quantized side information, g cm is the channel context model, i is the slice number of the current encoding (decoding), μ i , σ i are the mean and standard deviation of the i-th slice predicted by the network, is Gaussian distribution, U is uniform distribution, Predicted for the network probability.

[0066] With encoder g a and decoder g s Similarly, the super prior encoder h a The super prior decoder h is composed of a down-sampling convolutional layer and a residual block ResBlock stack. s Execution with the hyper-prior encoder h a Symmetric inverse transformation process.

[0067] like Figure 3 As shown, the conditional diffusion model includes a pre-trained U-Net network, a pre-trained encoder, a pre-trained decoder, a semantic encoder and a semantic adaptation module, wherein the conditional diffusion model uses the pre-trained U-Net network as the noise estimation network of the conditional diffusion model, the pre-trained U-Net network is connected to the latent space compression module LCM, the semantic adaptation module is connected to the pre-trained U-Net network, and the semantic encoder is connected to the semantic adaptation module.

[0068] Furthermore, the conditional diffusion model also includes two conditions: implicit guidance and explicit guidance. Implicit guidance refers to the compressed feature representation. Since it carries most of the information of the original image, using it as an implicit guide can help the diffusion model generate an image with the same content as the original image; in order to adapt to the number of channels after splicing, the embodiment of the present invention expands the number of channels of the first convolution kernel of the diffusion model from 4 to 8, and initializes the expanded number of channels to zero at the beginning of training.

[0069] like Figure 6 As shown in Figure 2, explicit guidance refers to the multi-scale features generated by the vector map after it passes through the semantic encoder and the semantic adaptation module SAM. The vector map is first downsampled in the semantic encoder to a size similar to the noisy feature representation z. t The same size is concatenated as the input of the semantic adaptive module SAM, and multi-scale features are generated in the semantic adaptive module SAM. These features have the same size as the intermediate features of the encoding end of U-Net, and the semantic information of the vector graph is introduced by directly adding the two. The specific process is as follows:

[0070] f ms =SAM(SE(m),z t )

[0071]

[0072] Among them, m is the vector graph, f ms and f enc They are the multi-scale input features output by the semantic adaptation module SAM and the intermediate features of the U-Net encoding end, and SE is the semantic encoder.

[0073] The semantic encoder is composed of three downsampling convolutional layers and two GELU layers stacked together to map the input vector map into a feature representation z with size and noise. t Same semantic features; semantic features and noisy features represent z t The concatenated input is sent to the semantic adaptation module SAM, and the first scale feature is generated after the first ResBlock. Then, each downsampling convolution layer and two ResBlocks generate a scale feature A total of 4 features of different scales are generated These features are added to the intermediate features from the encoding side of the U-Net as explicit guidance for the conditional diffusion model.

[0074] like Figure 2 As shown in the figure, in the actual implementation process, the training set is used to perform two-stage training on the pre-built vector-assisted remote sensing image ultra-low bitrate compression network. The first training stage is to iteratively compress the latent space compression module LCM under the Pytorch framework, and the second training stage is to iteratively optimize the conditional diffusion model under the Pytorch framework, as follows:

[0075] The process of iterative compression of the latent space compression module LCM under the Pytorch framework is:

[0076] The optimization parameters of the first training phase include the encoder g of the latent space compression module LCM a , decoder g s , a priori encoder h a , Super Prior Decoder h s , and channel context models.

[0077] The image tensor x(B,3,H,W) (i.e., the remote sensing image set in the training set) is mapped to the feature representation z0(B,c z ,h z ,w z ), where B is the batch size, H is the image height, W is the image width, (c i ,h i ,w i ,i∈{z,y,h}) respectively represent the number of channels, height and width of the corresponding tensor;

[0078] The feature representation z0(B,c z ,h z ,w z ) is input into the latent space compression module LCM and passes through the encoder g a and the super prior encoder h a Get the compact latent variable y(B,cy ,h y ,w y ) and the corresponding side information h(B,c h ,h h ,w h );

[0079] Compact latent variables y(B,c y ,h y ,w y ) and the corresponding side information h(V,c h ,h h ,w h ) is quantized and converted into a binary code stream through entropy coding and transmitted to the decoding end g s ;

[0080] Decoding end s The bitstream is converted into the corresponding tensor through entropy decoding, and the quantized side information After the super prior decoder h s Obtain the probability distribution parameters of the hidden variables to assist entropy coding, and the quantized hidden variables Through the decoder g s Get the compressed feature representation;

[0081] The rate distortion loss function is selected as the loss function of this training stage. The AdamW optimizer and gradient descent method are used to iterate the training until convergence with the goal of minimizing the loss function, and the compressed feature representation set is obtained. The learning rate of the AdamW optimizer in this training stage is 5×10 -5 , Batch-size is 16, and 250K iterations are performed until the model converges. In the first 10K iterations, a linear learning rate warm-up strategy is used to gradually increase the learning rate from 0 to the set value.

[0082] The process of iterative optimization of the conditional diffusion model under the Pytorch framework is:

[0083] The optimization parameters of the second training phase include the LCM decoder g s , Super Prior Decoder h s , linear layers of the diffusion model, semantic encoder and semantic adaptation module SAM;

[0084] The image tensor x is compressed through the process described in the first training stage and compared with the noisy feature representation z t Splicing as input to the conditional diffusion model;

[0085] The vector graph tensor (i.e., the vector graph set in the training set) is converted to the noisy feature representation z after passing through the semantic encoder. tSplicing, as the input of the semantic adaptation module SAM, outputs multi-scale feature maps;

[0086] The multi-scale feature map output by the semantic adaptation module SAM has the same dimension as the intermediate feature map at the encoding end of the conditional diffusion model. These feature maps are added together to introduce the structural information of the vector graph.

[0087] The conditional diffusion model loss function is selected as the loss function of this training stage. The AdamW optimizer and gradient descent method are used to iterate the training until convergence with the goal of minimizing the loss function. The trained vector-assisted remote sensing image ultra-low bitrate compression network is obtained. The learning rate of the AdamW optimizer in this training stage is 5×10 -5 , Batch-size is 48, and 150K iterations are performed until the model converges. In the first 10K iterations, a linear learning rate warm-up strategy is used to gradually increase the learning rate from 0 to the set value.

[0088] Among them, the solution of the rate-distortion loss function is:

[0089] L 1st =L rate +λL 1d

[0090]

[0091] Among them, L 1st is the rate-distortion loss function of the first stage, L rate is the bit rate loss, L ld is the distortion of each feature representation in the feature representation set, λ is the Laplace multiplier used to control the bit rate and distortion, and p z is the probability distribution of each feature representation in the feature representation set, and They are and Taking into account the requirements of different bitrate points, the embodiment of the present invention sets λ for weighing the bitrate and distortion. The larger λ is, the smaller the distortion is and the higher the bitrate is; the smaller λ is, the greater the distortion is and the lower the bitrate is.

[0092] The solution of the conditional diffusion model loss function is:

[0093]

[0094] Among them, L 2nd is the conditional diffusion model loss function of the second stage, ∈ is the randomly sampled Gaussian noise, ∈ θ is the noise predicted by the pre-trained U-Net network, c is the condition of the conditional diffusion model, and t is the time step of the conditional diffusion model, which is sampled from [0, T].

[0095] In step S103, the test set is input into the trained vector-assisted remote sensing image ultra-low bit rate compression network to reconstruct the compressed remote sensing image.

[0096] like Figure 2 As shown in the figure, in the actual execution process, the test set is input into the trained vector-assisted remote sensing image ultra-low bit rate compression network for inference testing. Specifically, the entire encoding and decoding process can be explained from the two parts of the encoding end and the decoding end.

[0097] The specific steps on the encoding side are as follows:

[0098] (1) Input the original image x of the test set into the pre-trained encoder ε to obtain the feature representation z0;

[0099] (2) Input the feature representation z0 into the latent space compression module LCM and pass through the encoder g a Get the compact latent variable y;

[0100] (3) Input the compact latent variable y into the super prior encoder h a In the process, we get the side information h;

[0101] (4) Quantified side information Input to the super prior decoder h s In the above example, the super prior decoder h s The output and quantized latent variables The concatenation is sent into the channel context model to obtain the quantized latent variables The probability of the quantized latent variable Entropy encoding into code stream;

[0102] (5) Quantified side information Entropy coded into a bit stream as side information and quantized hidden variables Passed to the decoding end together.

[0103] The specific steps of the decoding end are as follows:

[0104] (1) Entropy decoding of quantized side information from the bitstream

[0105] (2) Quantified side information and decoded The splicing is used as the input of the channel context model to obtain the probability of the current slice until the complete

[0106] (3) Input to decode g sIn the compressed feature representation

[0107] (4) Feature Representation after Compression With the noisy feature representation z t The splicing is used as the input of the conditional diffusion model, and the reverse diffusion process is performed 50 times to obtain the denoised feature representation

[0108] (5) Feature representation after denoising Input to the pre-trained decoder Get reconstructed compressed remote sensing image

[0109] In summary, according to the vector diagram-assisted ultra-low bit rate compression method for remote sensing images proposed in an embodiment of the present invention, the remote sensing image and the vector diagram of the corresponding area are input into the constructed vector diagram-assisted ultra-low bit rate compression network for remote sensing images, and the vector diagram is used to provide additional structural information in the latent space compression module to further reduce the bit rate, and the conditional diffusion model is used as a semantic guide to assist in generating a remote sensing image with correct semantic information, that is, the additional structural information of the vector diagram is used to compress the remote sensing image to an extremely low bit rate bit stream at the encoding end, and the vector diagram is used to provide semantics to reconstruct the remote sensing image from the bit stream at the decoding end, so that a high-fidelity compressed remote sensing image can be reconstructed efficiently at an extremely low bit rate.

[0110] Next, a remote sensing image ultra-low bit rate compression device based on vector diagram assistance proposed in an embodiment of the present invention will be described with reference to the accompanying drawings.

[0111] Figure 7 It is a block diagram of a remote sensing image ultra-low bit rate compression device based on vector diagram assistance according to an embodiment of the present invention.

[0112] like Figure 7 As shown, the remote sensing image ultra-low bit rate compression device 70 based on vector graphics assistance includes: a construction module 701, a training module 702 and a compression module 703.

[0113] Among them, the construction module 701 is used to obtain the remote sensing image set and the vector map of each remote sensing image in the remote sensing image set to construct a training set and a test set. The training module 702 is used to optimize the training of the pre-constructed vector map-assisted remote sensing image ultra-low bit rate compression network using the training set to obtain a trained vector map-assisted remote sensing image ultra-low bit rate compression network, wherein the vector map-assisted remote sensing image ultra-low bit rate compression network includes a latent space compression module and a conditional diffusion model. The compression module 703 is used to input the test set into the trained vector map-assisted remote sensing image ultra-low bit rate compression network to reconstruct the compressed remote sensing image.

[0114] It should be noted that the above explanation of the embodiment of the method for ultra-low bit rate compression of remote sensing images assisted by vector diagrams is also applicable to the ultra-low bit rate compression device for remote sensing images assisted by vector diagrams of this embodiment, and will not be repeated here.

[0115] According to the device for extremely low bit rate compression of remote sensing images assisted by vector diagrams proposed in an embodiment of the present invention, the remote sensing image and the vector diagram of the corresponding area are input into the constructed extremely low bit rate compression network for remote sensing images assisted by vector diagrams, and the vector diagram is used to provide additional structural information in the latent space compression module to further reduce the bit rate, and the conditional diffusion model is used as a semantic guide to assist in generating a remote sensing image with correct semantic information, that is, the additional structural information of the vector diagram is used to compress the remote sensing image to an extremely low bit rate bit stream at the encoding end, and the vector diagram is used to provide semantics to reconstruct the remote sensing image from the bit stream at the decoding end, so that a high-fidelity compressed remote sensing image can be efficiently reconstructed at an extremely low bit rate.

[0116] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0117] A memory 801 , a processor 802 , and a computer program stored in the memory 801 and executable on the processor 802 .

[0118] When the processor 802 executes the program, the vector diagram-assisted remote sensing image ultra-low bit rate compression method provided in the above embodiment is implemented.

[0119] Furthermore, the electronic device further comprises:

[0120] The communication interface 803 is used for communication between the memory 801 and the processor 802 .

[0121] The memory 801 is used to store computer programs that can be executed on the processor 802 .

[0122] The memory 801 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0123] If the memory 801, the processor 802 and the communication interface 803 are implemented independently, the communication interface 803, the memory 801 and the processor 802 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0124] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can communicate with each other through an internal interface.

[0125] The processor 802 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0126] An embodiment of the present invention further provides a computer program product, which, when executed by a processor, implements the above-mentioned vector diagram-assisted remote sensing image ultra-low bit rate compression method.

[0127] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned vector diagram-assisted remote sensing image ultra-low bit rate compression method.

[0128] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0129] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0130] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0132] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0133] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0135] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A very low bit rate compression method for remote sensing images based on vector graphics, characterized in that: The following steps are involved: Acquire a remote sensing image set and a vector map of each remote sensing image in the remote sensing image set to construct a training set and a test set; The training set is used to optimize and train the pre-constructed vector graphics-assisted remote sensing image extremely low bit rate compression network to obtain a trained vector graphics-assisted remote sensing image extremely low bit rate compression network, wherein the vector graphics-assisted remote sensing image extremely low bit rate compression network includes a latent space compression module and a conditional diffusion model; The test set is input into the trained vector-image-assisted remote sensing image ultra-low bit rate compression network to reconstruct the compressed remote sensing image.

2. The method for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to claim 1 is characterized in that: The step of obtaining a remote sensing image set and a vector map corresponding to each remote sensing image to construct a training set includes: Acquire a remote sensing image dataset of a target area to select the remote sensing image set from the remote sensing image dataset; According to the geographical location of each remote sensing image, a vector map of the corresponding area is obtained to construct a vector map set; The training set and the test set are generated according to the remote sensing image set and the vector image set.

3. The method for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to claim 1 is characterized in that: The latent space compression module includes a main transformation network and an entropy model, wherein the main transformation network includes an encoder and a decoder, and the entropy model includes a super prior encoder, a super prior decoder and a channel context model; The conditional diffusion model includes a pre-trained U-Net network, a pre-trained encoder, a pre-trained decoder, a semantic encoder and a semantic adaptation module.

4. The method for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to claim 3 is characterized in that: The method of optimizing and training the pre-constructed vector-assisted remote sensing image extremely low bit rate compression network using the training set to obtain a trained vector-assisted remote sensing image extremely low bit rate compression network includes: Mapping the remote sensing image set in the training set using a pre-trained encoder to obtain a feature representation set; Iteratively compressing the latent space compression module using the feature representation set until a target rate-distortion loss function converges to obtain a compressed feature representation set; Acquire a feature representation set with noise according to the remote sensing image set in the training set; The conditional diffusion model is iteratively optimized using the vector graph in the training set, the compressed feature representation set and the feature representation set with noise until the target diffusion model loss function converges, thereby obtaining the trained vector graph-assisted remote sensing image ultra-low bit rate compression network.

5. The method for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to claim 4 is characterized in that: The solution of the rate-distortion loss function is: THE 1st =L rate +λL 1d Among them, L 1st is the rate-distortion loss function of the first stage, L rate is the bit rate loss, L ld is the distortion of each feature representation in the feature representation set, λ is the Laplace multiplier used to control the bit rate and distortion, and p z is the probability distribution of each feature representation in the feature representation set, and They are and The probability distribution of .

6. The method for extremely low bit rate compression of remote sensing images based on vector diagram assistance according to claim 4 is characterized in that: The solution formula of the conditional diffusion model loss function is: Among them, L 2nd is the conditional diffusion model loss function of the second stage, ∈ is the randomly sampled Gaussian noise, ∈ θ is the noise predicted by the pre-trained U-Net network, c is the condition of the conditional diffusion model, and t is the time step of the conditional diffusion model.

7. A remote sensing image ultra-low bit rate compression device based on vector graphics, characterized in that: include: A construction module is used to obtain a remote sensing image set and a vector map of each remote sensing image in the remote sensing image set to construct a training set and a test set; A training module is used to optimize and train a pre-constructed vector graphics-assisted remote sensing image extremely low bit rate compression network using the training set to obtain a trained vector graphics-assisted remote sensing image extremely low bit rate compression network, wherein the vector graphics-assisted remote sensing image extremely low bit rate compression network includes a latent space compression module and a conditional diffusion model; The compression module is used to input the test set into the trained vector map-assisted remote sensing image ultra-low bit rate compression network to reconstruct the compressed remote sensing image.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vector graphics-assisted remote sensing image ultra-low bit rate compression method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that When the computer program / instruction is executed by a processor, the vector diagram-assisted remote sensing image ultra-low bit rate compression method as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the vector diagram-assisted remote sensing image ultra-low bit rate compression method as described in any one of claims 1 to 6.

Images