Remote sensing image on-orbit compression method based on adversarial neural network

Abstract

The invention discloses a remote sensing image on-orbit compression method based on an adversarial neural network, and the method comprises the steps: building an image compression recovery model on the ground based on an adversarial neural network model HIFIC; the image compression recovery model comprises an encoder E, a probability model P and a decoder G; deploying an encoder E and a probability model P on a satellite, and deploying a decoder G on the ground; uploading an original remote sensing image to a satellite, and processing the original remote sensing image by an encoder E and a probability model P to obtain an intermediate file; and downloading the intermediate file to the ground, processing the intermediate file by a decoder G, and recovering the intermediate file into an original remote sensing image. According to the method, on-orbit high-magnification compression of the remote sensing image can be realized, high-fidelity recovery can be realized after the compressed intermediate file is downloaded to the ground, the problems of serious detail distortion, high algorithm complexity and the like existing in a traditional image compression method are solved, the communication pressure of image downloading is greatly relieved, and the communication efficiency is improved. And the accuracy and reliability of ground image processing are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of payload intelligent data processing, and in particular relates to an on-orbit compression method for remote sensing images based on an adversarial neural network. Background Art

[0002] With the rapid development of earth observation remote sensing satellites, the resolution, width and data volume of remote sensing images have increased sharply. The contradiction between the growing amount of remote sensing image data and the limited storage space and transmission bandwidth has become increasingly prominent. Therefore, it is necessary to study remote sensing image compression methods with large compression ratios and high recovery quality to alleviate the contradiction. Traditional image compression methods used to compress natural images are not completely applicable to remote sensing images. Compared with traditional images, remote sensing images have the following characteristics: complex local detail textures, low correlation, and a wide variety of ground features. The use of traditional image compression methods often has problems such as severe detail distortion and high algorithm complexity. Summary of the invention

[0003] The technology of the present invention solves the problem: overcomes the shortcomings of the prior art, provides an on-orbit compression method for remote sensing images based on an adversarial neural network, based on an adversarial neural network architecture, adopts ground training and on-orbit reasoning to achieve high-rate compression of remote sensing images on-orbit, and the compressed intermediate file can be transmitted to the ground to restore the image with high fidelity, thereby solving the problems of severe detail distortion and high algorithm complexity in traditional image compression methods, greatly alleviating the communication pressure of image transmission, and improving the accuracy and reliability of ground image processing.

[0004] In order to solve the above technical problems, the present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network, comprising:

[0005] Based on the adversarial neural network model HIFIC, an image compression recovery model is constructed on the ground; wherein the image compression recovery model includes: an encoder E, a probability model P and a decoder G;

[0006] The encoder E and probability model P are deployed on the satellite, and the decoder G is deployed on the ground;

[0007] The original remote sensing image is uploaded to the satellite and processed by the encoder E and probability model P deployed on the satellite to compress the original remote sensing image into an intermediate file;

[0008] The intermediate file is transmitted to the ground and processed by the decoder G deployed on the ground to restore the intermediate file to the original remote sensing image.

[0009] In the above-mentioned on-orbit compression method of remote sensing images based on adversarial neural network, the image compression recovery model is constructed in the following way:

[0010] Determine the network structure of the adversarial neural network model HIFIC;

[0011] Design the loss function according to the determined network structure of the adversarial neural network model HIFIC;

[0012] Construct a training sample set and a validation sample set;

[0013] Based on the training sample set, the loss function is continuously minimized and the model parameters are optimized to obtain a series of trained adversarial neural network models HIFIC to be selected;

[0014] Based on the verification sample set, the model with the best verification result is selected from the adversarial neural network model HIFIC to be selected as the final image compression recovery model.

[0015] In the above-mentioned on-orbit compression method of remote sensing images based on adversarial neural network, the network structure of the adversarial neural network model HIFIC includes: encoder E', probability model P', decoder G' and discriminator D';

[0016] Encoder E' is used to extract features of the input real remote sensing image to obtain image features of the real remote sensing image;

[0017] The probability model P' is used to learn the probability distribution of real remote sensing images based on image features;

[0018] Decoder G', used to generate a pseudo remote sensing image based on the image features obtained by encoder E' and the probability distribution obtained by probability model P';

[0019] The discriminator D' is used to judge the input remote sensing image and determine whether the input remote sensing image is a real remote sensing image sample or a pseudo remote sensing image generated by the decoder G'.

[0020] In the above-mentioned on-orbit compression method of remote sensing images based on adversarial neural network, the loss function is designed in the following way:

[0021] For the loss function, the input is the real image x;

[0022] The encoder E' generates the feature representation y of the real image x, y = E(x), E(x) represents the function of encoding the image, mapping the image to the feature representation y, which is composed of multiple convolutional layers connected in series;

[0023] The probability model P' is introduced to calculate the probability distribution of y, which is used to guide the decoder G' to generate images. P'(y) represents the probability distribution function of y.

[0024] The decoder G' restores the feature representation y to a pseudo image x', x' = G(y), where G(y) represents a function that restores y to a pseudo image under the guidance of the probability model P';

[0025] The loss function satisfies: Minimize d(x,x') and r(y) at the same time, use λ to control the trade-off, and get the final loss function L (E,G) :

[0026] L (E,G) =E x～PX [λr(y)+d(x,x')]

[0027] Among them, E x～PX represents the expected function; d(x,x') represents the distortion of the real image x and the pseudo image x', which is calculated using the mean square error; r(y) represents the storage size required for y, which is calculated using information entropy, r(y) = -log(P'(y)).

[0028] In the above-mentioned on-orbit compression method of remote sensing images based on adversarial neural network, the training sample set and the verification sample set are constructed in the following way:

[0029] Obtain an initial sample data set; wherein the initial sample data set includes: public data and user-collected data; the public data includes: natural images and remote sensing images; the user-collected data is remote sensing images;

[0030] All images in the initial sample data set are divided into a training sample set and a validation sample set in a ratio of 8:2; the training sample set is used to optimize the model parameters, and the validation sample set is used to select the optimal model.

[0031] The present invention has the following advantages:

[0032] (1) The present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network. Compared with traditional image compression methods, the method has the advantages of high compression ratio and higher quality after image restoration, and is very suitable for on-orbit compression of remote sensing images in the aerospace field. The high compression ratio greatly reduces the amount of data transmitted from the satellite to the ground after the image is compressed, further alleviating communication pressure and improving real-time performance. The higher quality after image restoration ensures that the detailed information of the image is fully retained after it is transmitted to the ground, thereby improving the accuracy and reliability of ground image processing.

[0033] (2) The present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network, which can deploy the two processes of image compression and image restoration separately; the encoder and probability model deployed on-orbit only need to complete the compression of the original image, and the generated compressed file is transmitted to the ground and then restored to the original image by the ground decoder; this method of completely decoupling the compression process and the restoration process is naturally suitable for on-orbit deployment and is very suitable for the aerospace field.

[0034] (3) The present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network, which is suitable for on-orbit compression of remote sensing images, especially for on-orbit compression of large-scale remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of an implementation principle of an on-orbit compression method for remote sensing images based on an adversarial neural network in an embodiment of the present invention;

[0036] Figure 2 It is a schematic diagram of the network structure of an adversarial neural network model HIFIC in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments disclosed in the present invention will be further described in detail below with reference to the accompanying drawings.

[0038] The present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network, which is generally divided into two stages: in the first stage, the training of an image compression and recovery model is completed on the ground; in the second stage, the on-orbit deployment and reasoning of the image compression and recovery model are completed on the satellite and on the ground respectively.

[0039] like Figure 1 As shown, in this embodiment, the on-orbit compression method of remote sensing images based on adversarial neural network includes:

[0040] Step 1: Based on the adversarial neural network model HIFIC, an image compression recovery model is constructed on the ground. The image compression recovery model includes: encoder E, probability model P and decoder G 。

[0041] Step 2: deploy the encoder E and probability model P on the satellite, and deploy the decoder G on the ground.

[0042] Step 3: Upload the original remote sensing image to the satellite, and compress the original remote sensing image into an intermediate file after being processed by the encoder E and probability model P deployed on the satellite.

[0043] Step 4: The intermediate file is transmitted to the ground and processed by the decoder G deployed on the ground to restore the intermediate file to the original remote sensing image.

[0044] In this embodiment, the image compression and recovery model can be constructed in the following manner: determine the network structure of the adversarial neural network model HIFIC; design a loss function according to the determined network structure of the adversarial neural network model HIFIC; construct a training sample set and a verification sample set; based on the training sample set, continuously minimize the loss function, optimize the model parameters, and obtain a series of trained adversarial neural network models HIFIC to be selected; based on the verification sample set, select the model with the best verification result from the adversarial neural network model HIFIC to be selected as the final image compression and recovery model.

[0045] Preferably, Figure 2 As shown in the figure, the network structure of the adversarial neural network model HIFIC includes: an encoder E', a probability model P', a decoder G' and a discriminator D'. The encoder E' is used to extract features of the input real remote sensing image to obtain the image features of the real remote sensing image; the probability model P' is used to learn the probability distribution of the real remote sensing image based on the image features; the decoder G' is used to generate a pseudo remote sensing image according to the image features obtained by the encoder E' and the probability distribution obtained by the probability model P'; the discriminator D' is used to judge the input remote sensing image to determine whether the input remote sensing image is a real remote sensing image sample or a pseudo remote sensing image generated by the decoder G'.

[0046] Preferably, the loss function is designed as follows:

[0047] For the loss function, the input is the real image x.

[0048] The encoder E' generates the feature representation y of the real image x, y = E(x), E(x) represents the function for encoding the image, mapping the image to the feature representation y, which is composed of multiple convolutional layers in series.

[0049] The probability model P' is introduced to calculate the probability distribution of y, which is used to guide the decoder G' to generate images. P'(y) represents the probability distribution function of y.

[0050] The decoder G' restores the feature representation y to the pseudo image x', x'=G(y), and G(y) represents the function that restores y to the pseudo image under the guidance of the probability model P'.

[0051] The loss function satisfies: Minimize d(x,x') and r(y) at the same time, use λ to control the trade-off, and get the final loss function L (E,G) :

[0052] L (E,G) =E x～PX [λr(y)+d(x,x')]

[0053] Among them, E x～PXrepresents the expected function; d(x,x') represents the distortion of the real image x and the pseudo image x', which is calculated using the mean square error; r(y) represents the storage size required for y, which is calculated using information entropy, r(y) = -log(P'(y)).

[0054] Preferably, the training sample set and the validation sample set are constructed in the following manner: an initial sample data set is obtained; wherein the initial sample data set includes: public data and user-collected data; the public data includes: natural images and remote sensing images; the user-collected data is remote sensing images. Further, all images in the initial sample data set are divided into a training sample set and a validation sample set in a ratio of 8:2; wherein the training sample set is used to optimize model parameters, and the validation sample set is used to select the optimal model.

[0055] In summary, the present invention discloses an on-orbit compression method for remote sensing images based on an adversarial neural network. It adopts a method of combining ground training with on-orbit reasoning to achieve high-ratio compression of remote sensing images on-orbit. After the compressed file is transmitted to the ground, the image can be restored with high fidelity, thereby solving the problems of severe detail distortion and high algorithm complexity in traditional image compression methods; it greatly alleviates the communication pressure of image transmission and improves the accuracy and reliability of ground image processing.

[0056] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

[0057] The contents not described in detail in the specification of the present invention belong to the common knowledge of the professionals in this field.

Claims

1. An on-orbit compression method for remote sensing images based on adversarial neural networks, characterized in that: include: Based on the adversarial neural network model HIFIC, an image compression recovery model is constructed on the ground; wherein the image compression recovery model includes: an encoder E, a probability model P and a decoder G; The encoder E and probability model P are deployed on the satellite, and the decoder G is deployed on the ground; The original remote sensing image is uploaded to the satellite and processed by the encoder E and probability model P deployed on the satellite to compress the original remote sensing image into an intermediate file; The intermediate file is transmitted to the ground and processed by the decoder G deployed on the ground to restore the intermediate file to the original remote sensing image.

2. The on-orbit compression method for remote sensing images based on adversarial neural networks according to claim 1, characterized in that: The image compression recovery model is constructed as follows: Determine the network structure of the adversarial neural network model HIFIC; Design the loss function according to the determined network structure of the adversarial neural network model HIFIC; Construct a training sample set and a validation sample set; Based on the training sample set, the loss function is continuously minimized and the model parameters are optimized to obtain a series of trained adversarial neural network models HIFIC to be selected; Based on the verification sample set, the model with the best verification result is selected from the adversarial neural network model HIFIC to be selected as the final image compression recovery model.

3. The on-orbit compression method for remote sensing images based on adversarial neural network according to claim 2 is characterized in that: The network structure of the adversarial neural network model HIFIC includes: encoder E', probability model P', decoder G' and discriminator D'; Encoder E' is used to extract features of the input real remote sensing image to obtain image features of the real remote sensing image; The probability model P' is used to learn the probability distribution of real remote sensing images based on image features; Decoder G', used to generate a pseudo remote sensing image based on the image features obtained by encoder E' and the probability distribution obtained by probability model P'; The discriminator D' is used to judge the input remote sensing image and determine whether the input remote sensing image is a real remote sensing image sample or a pseudo remote sensing image generated by the decoder G'.

4. The on-orbit compression method for remote sensing images based on adversarial neural network according to claim 3 is characterized in that: The loss function is designed as follows: For the loss function, the input is the real image x; The encoder E' generates the feature representation y of the real image x, y = E(x), E(x) represents the function of encoding the image, mapping the image to the feature representation y, which is composed of multiple convolutional layers connected in series; The probability model P' is introduced to calculate the probability distribution of y, which is used to guide the decoder G' to generate images. P'(y) represents the probability distribution function of y. The decoder G' restores the feature representation y to a pseudo image x', x' = G(y), where G(y) represents a function that restores y to a pseudo image under the guidance of the probability model P'; The loss function satisfies: Minimize d(x,x') and r(y) at the same time, use λ to control the trade-off, and get the final loss function L (E,G) : L (E,G) =E x～PX [λr(y)+d(x,x')] Among them, E x～PX represents the expected function; d(x,x') represents the distortion of the real image x and the pseudo image x', which is calculated using the mean square error; r(y) represents the storage size required for y, which is calculated using information entropy, r(y) = -log(P'(y)).

5. The on-orbit compression method for remote sensing images based on adversarial neural network according to claim 2, characterized in that: The training sample set and validation sample set are constructed in the following way: Obtain an initial sample data set; wherein the initial sample data set includes: public data and user-collected data; the public data includes: natural images and remote sensing images; the user-collected data is remote sensing images; All images in the initial sample data set are divided into a training sample set and a validation sample set in a ratio of 8:2; the training sample set is used to optimize the model parameters, and the validation sample set is used to select the optimal model.

Images