Satellite image data restoration method and system using generative adversarial network

Abstract

The invention discloses a satellite image data restoration method and system using a generative adversarial network, and the method comprises the steps: S1, constructing a satellite image data restoration model containing the generative adversarial network, enabling a defect image generation module to generate a defect image sample which is partially missing or / and partially distorted and is associated with a real image sample, the feature extraction module is used for extracting feature maps of the real image sample and the associated defect image sample; s2, a data restoration module carries out all and local association learning training on the feature maps of the real image samples and the associated defect image samples, and the data restoration module carries out restoration training by taking the defect image samples as input and taking the corresponding real image samples as targets; and S3, collecting and slicing current satellite image data, inputting the current satellite image data into the learned and trained satellite image data restoration model, performing image data restoration, and recording the restoration probability. According to the satellite image data restoration model, multi-level features can be obtained, and the image restoration quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of satellite image data restoration, and in particular to a satellite image data restoration method and system using a generative adversarial network. Background Art

[0002] Satellite image data is an important data collection method that can obtain various information about the earth. It has many unique advantages, such as being not restricted by national boundaries, being able to cover the earth's surface extensively, being able to conduct regular observations, and obtaining objective and reliable data. Satellite image data (including remote sensing data) can also be integrated and analyzed with other geographic information data, providing a basis and support for interdisciplinary research. Therefore, satellite image data has become an indispensable support method in the fields of global environmental monitoring, environmental resource management, and ecological management. Satellites can observe the entire surface of the earth in as little as one day and provide rich satellite image data. They also collect bands from visible light to thermal infrared through MODIS sensors, which can provide characteristic information for the atmosphere, land surface, and oceans, providing a large amount of data for global users. It plays an important role in vegetation index, marine ecosystem research, soil erosion, and agricultural management, and is the basic scientific data for regional ecological environment analysis.

[0003] In the process of earth observation, rainfall, changes in ground and soil moisture, and melting of ice and snow will cause uncertainty in observation. Clouds, atmosphere, aerosols, etc. have become important factors affecting data quality. The results show that on average, 35% of the global land surface is covered by clouds, and the coverage rate in some areas is more than 50% (such as the high-altitude Qinghai-Tibet Plateau). These factors cause clouds to absorb a large amount of signals, making the observation affected by clouds. Under the conditions of thick clouds and rainy weather, the radiation under the clouds obtained is not real, resulting in the loss of satellite image data (such as MODIS reflectivity data) products, which has affected the application of satellite image data (such as MODIS reflectivity data products). The lack of data may reduce the accuracy of meteorological and climate models and affect the understanding and prediction of weather and climate phenomena; secondly, the missing data will affect environmental monitoring, such as vegetation growth, water temperature and chlorophyll concentration, which may lead to incomplete monitoring and assessment of environmental changes; in addition, satellite image data (including MODIS reflectivity data) plays an important role in natural disaster monitoring and response, such as fire monitoring, flood monitoring and vegetation recovery after earthquakes. Missing data may reduce the ability to monitor and assess disasters and their impacts in a timely manner, and have an adverse impact on emergency response and post-disaster recovery. Satellite image data (including MODIS reflectivity data) are also used in agricultural management, including crop growth monitoring, soil moisture assessment, and drought monitoring; missing data may affect agricultural decision-making and management, and farmers and agricultural management agencies may not be able to accurately assess crop health and water resource utilization. Therefore, satellite image data or information obtained by satellites are inevitably contaminated. This contaminated data will greatly cause deviations in the research of scientific researchers, resulting in results that do not conform to the actual situation. Therefore, how to repair the images of missing or distorted areas becomes very important, and it is also a technical difficulty that needs to be solved urgently. Summary of the invention

[0004] The purpose of the present invention is to provide a satellite image data restoration method using a generative adversarial network. The satellite image data restoration model includes a generative adversarial network. The defect image generation module can generate defect image samples that are partially missing or / and partially distorted and associated with real image samples. The generation network unit and the discrimination network unit can perform adversarial deep learning training of defect and real image features. The feature extraction module adopts multi-level encoding and decoding, has an excellent perception field of view, and at the same time, jump connections are added between the same-level encoding and decoding, so that multi-level features can be obtained, thereby improving the image restoration quality.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A satellite image data restoration method using a generative adversarial network, the method comprising:

[0007] S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples. The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates partially missing or / and partially distorted defect image samples associated with the real image samples. The feature extraction module extracts feature maps of the real image samples and the associated defect image samples.

[0008] S2, the discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs all and local association learning training on the feature graphs of the real image samples and the associated defective image samples and performs image data repair on the defective image samples. The data repair module takes the defective image samples as input and the corresponding real image samples as targets for repair training; the probability judgment module is used to judge and record the repair probability, and the repair probability represents the probability that the repaired image data is the real image data;

[0009] S3. Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability.

[0010] In order to better implement the present invention, in step S1, the defect image generation module includes the following method:

[0011] A11. Select some areas or some data in the real image samples to remove or fill in the erroneous data;

[0012] A12. Select some areas in the real image samples to perform image blurring processing.

[0013] Preferably, in step S1, the feature extraction module of the generative adversarial network includes four encoding units and three decoding units, the first to fourth encoding units of the four encoding units sequentially perform feature extraction of four scales on the input image, and the third to first decoding units of the three decoding units sequentially perform scale restoration decoding processing, wherein the third decoding unit decodes the third encoding unit with a jump connection, the second decoding unit decodes the second encoding unit with a jump connection, and the first decoding unit decodes the first encoding unit with a jump connection; the first decoding unit outputs a feature map after processing by a full convolutional layer and a global jump connection.

[0014] Preferably, the encoding unit includes a convolution layer, a pooling layer, an activation layer and a normalization layer, and the pooling layer adopts a multi-scale downsampling module. The multi-scale downsampling module method is as follows:

[0015] B11. The scale of the input image is 2N× 2N, and the number of channels is C. The multi-scale downsampling module uses the Space_to_depth algorithm to traverse the input image from the upper left corner to obtain N×N 2×2 matrix units. Then, the pixels at the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are used as the four channel data of the output feature, and the channel reconstruction size of the output feature map is N×N×4C.

[0016] B12. Use several convolutional layers with serial structures, and first perform convolution on the four channels of the feature map N×N×4C in step B11 respectively. Then perform point-by-point convolution on the features corresponding to each channel and obtain linear combinations and concatenations of the feature vectors.

[0017] Preferably, the discriminant network unit is constructed with a 30×30 two-dimensional matrix, and the data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data, while the probability judgment module judges and records the repair probability based on the two-dimensional matrix.

[0018] Preferably, when the discriminant network unit is trained, the probability judgment module processes the restored image data and the real image data through convolution, pooling, and normalization respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment grid by grid.

[0019] Preferably, the loss function L in the generative adversarial network is G The expression is as follows:

[0020] L G =λ1×L a +λ2×L L1 , where L a Indicates the probability of the discriminative network unit correctly judging the data, L L1 represents the L1 loss between the restored image data and the real image data; λ1 and λ2 are L a , L L1 The coefficient of .

[0021] Preferably, each of the decoding units outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

[0022] Preferably, the defective image sample source also includes defective satellite images corresponding to real image samples.

[0023] A satellite image data restoration system using a generative adversarial network comprises a satellite image data restoration model, a data acquisition module and a sample data set containing real image samples. The satellite image data restoration model includes a generative adversarial network, the generative adversarial network includes a generative network unit and a discriminative network unit, the generative network unit includes a defect image generation module and a feature extraction module, the defect image generation module is used to generate a defect image sample that is partially missing or / and partially distorted and associated with the real image sample, and the feature extraction module is used to respectively extract feature maps of the real image sample and the associated defect image sample; the discriminative network unit comprises a data restoration module and a probability judgment module, the data restoration module is used to perform full and partial association learning and training on the feature maps of the real image sample and the associated defect image sample and perform image data restoration on the defect image sample, the data restoration module is used to perform restoration training with the defect image sample as input and the corresponding real image sample as target; the probability judgment module is used to judge and record the restoration probability, and the restoration probability indicates the probability that the restored image data is the real image data; the data acquisition module is used to acquire the current satellite image data for slicing processing and sequentially encode and input the satellite image data restoration model after learning and training, the satellite image data restoration model performs image data restoration on the current satellite image data and records the restoration probability.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] (1) The satellite image data restoration model of the present invention includes a generative adversarial network. The defect image generation module can generate defect image samples that are partially missing or / and partially distorted and associated with real image samples. The generation network unit and the discrimination network unit can perform adversarial deep learning training of defect and real image features. In addition, the feature extraction module adopts multi-level encoding and decoding, and has an excellent perception field of view. At the same time, jump connections are added between the same-level encoding and decoding, so that multi-level features can be obtained, thereby improving the image restoration quality.

[0026] (2) The encoding unit of the present invention adopts a multi-scale downsampling module, using N×N 2×2 matrix units; then the pixels at the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are respectively used as the four channel data of the output feature, and after channel-wise convolution, channel reconstruction, point-by-point convolution, and linear combination and splicing of feature vectors, the efficiency and quality of multi-scale feature downsampling are improved.

[0027] (3) The present invention uses a sample data set to train a satellite image data restoration model, and through multi-level feature learning training, improves the image restoration quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1A method flow chart of the satellite image data repair method of the present invention;

[0029] Figure 2 Schematic diagram of the principle of the feature extraction module in the embodiment;

[0030] Figure 3 Schematic diagram of the principle structure of a decoding unit of a feature extraction module in an embodiment;

[0031] Figure 4 A schematic diagram of comparing and judging the restored image with the real image by the probability judgment module in the embodiment;

[0032] Figure 5 Schematic diagram of the principle of the multi-scale downsampling module in the embodiment;

[0033] Figure 6 A schematic diagram of dividing four channels according to the matrix unit in the embodiment;

[0034] Figure 7 It is a schematic diagram of the principle of processing four channels separately and linearly combining the features in the embodiment;

[0035] Figure 8 To give a schematic diagram of the principle of channel processing;

[0036] Fig. 9 Two defect image samples are provided as an example of an embodiment, including a schematic image diagram of a defect area;

[0037] Fig.10 For example, Fig. 9 Schematic diagram of the image after defect area repair processing of the defect image sample. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with embodiments:

[0039] Example

[0040] like Figure 1 As shown, a satellite image data restoration method using a generative adversarial network comprises:

[0041] S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples; preferably, the sample data set includes defective image samples corresponding to the real image samples, that is, when preparing the sample data set, set a time threshold, collect real images without defects (including defects such as missing or distortion due to occlusion) within the time threshold, and collect defective image samples corresponding to the real images in position within the time threshold at the same time. For example, if the time threshold is set to one month, several real image samples in the first, middle and last ten days of a month can be collected, and several defective image samples in the first, middle and last ten days of the same month can also be collected. The real image samples and defective image samples within the same time threshold are grouped into corresponding groups (the real image samples and the defective image samples are generally corresponding single-band image maps) to facilitate subsequent restoration training.

[0042] The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates partially missing or / and partially distorted defect image samples associated with real image samples (subsequent defect image samples are grouped correspondingly with real image samples). In some embodiments, in step S1, the defect image generation module includes the following method:

[0043] A11. Select some areas or some data in the real image samples to remove or fill in the erroneous data;

[0044] A12. Select some areas in the real image samples to perform image blurring processing.

[0045] The defect image generation module may use method A11 or method A12 or other methods to generate defect image samples associated with real image samples.

[0046] In addition to the defect image samples generated by the defect image generation module, the defect image sample source of the present invention also includes defective satellite images corresponding to the real image samples. The real image samples and defect image samples that correspond to each other (the position and time thresholds are both corresponding) are grouped together to facilitate the subsequent feature extraction and repair learning training of the corresponding real image samples and defect image samples (taking the real image samples as the target, repairing the defect image samples and performing learning training accordingly).

[0047] The feature extraction module extracts feature maps of real image samples and associated defect image samples. The feature extraction module extracts features (including features such as edge and texture information) from the corresponding real image samples and defect image samples. Figure 2As shown, the feature extraction module of the generative adversarial network includes four encoding units and three decoding units. The first to fourth encoding units of the four encoding units sequentially extract features of four scales from the input image (the encoding process realizes the amplification of the number of channels while reducing the size of the feature map, so that the network can learn features of different scales. The number of channels of the first encoding unit is 64, and the size of the feature map output by each encoding unit is halved). The third to first decoding units of the three decoding units sequentially perform scale restoration (continuously amplifying the size of the feature map while reducing the number of feature map channels, and gradually returning the encoded information to the feature map at the image size). The third decoding unit decodes the third encoding unit with a jump connection, the second decoding unit decodes the second encoding unit with a jump connection, and the first decoding unit decodes the first encoding unit with a jump connection. The first decoding unit is processed by a full convolution layer (including a 3×3 convolution layer, outputting a non-uniform residual image) and a global jump connection (since the pooling operation in the encoding process will lose part of the image information, the global jump connection can effectively solve this problem) and outputs a feature map. Figure 3 As shown in the figure (W and H represent the width and height of the image respectively), each decoding unit outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

[0048] Preferably, the encoding unit includes a convolution layer (the preferred convolution layer in this embodiment uses a 3×3 convolution layer for feature extraction), a pooling layer, an activation layer (the preferred activation layer in this embodiment uses a LeayRelu activation function) and a normalization layer. The convolution layer and the pooling layer use a multi-scale downsampling module. The multi-scale downsampling module method is as follows:

[0049] B11, the input image scale is 2N×2N, the number of channels is C, such as Figure 5 As shown, the multi-scale downsampling module uses the Space_to_depth algorithm to start from the upper left corner of the input image and traverse the input image to obtain N×N 2×2 matrix units; then the pixels in the upper left, upper right, lower left, and lower right corners of the matrix unit are used as the four channel data of the output feature, and the channel reconstruction size of the output feature map is N×N×4C. When the input image scale is 2N×2N, the algorithm starts from the upper left corner of the image and traverses the image to obtain N×N 22 matrix units; then the pixels in the upper left, upper right, lower left, and lower right corners of the matrix unit are used as the first, second, third, and fourth channel data of the output feature, respectively. At this time, the size of the output feature map becomes N×N×4. When the number of input feature map channels is C, the size of the output feature map is N×N×4C. In a single image, a certain pixel often has a strong correlation with the surrounding pixels, that is, the image pixels themselves have a certain spatial similarity. Using the Space_to_depth algorithm, while reducing the two-dimensional size of the image and expanding the number of channels, the structural characteristics of the original image are less affected. As shown Figure 6 As shown in the figure, after the original image generates four downsampled images, it still has certain structural similarities; therefore, using the Space_to_depth algorithm, efficient downsampling operations can be achieved without adding additional weight parameters and calculations.

[0050] B12, using several convolutional layers in series (see Figure 8 , perform feature extraction of 1×1 and multiple 3×3 convolutional layers on the feature map, and then concatenate the extracted multi-scale features and pass them into subsequent operations), and convolve the four channels of the feature map N×N×4C in step B11 separately, such as Figure 7 As shown, the features corresponding to each channel are then convolved point by point and the linear combination and concatenation of the feature vectors are obtained. The features corresponding to each channel are convolved point by point and the linear combination of the feature vectors is obtained as the channel reconstruction stage. The purpose of the channel reconstruction stage is to compress and adjust the number of channels of the downsampled feature map. The block_size of the Space_to_depth algorithm is fixed to 2, that is, when the input feature map is MMC1, the output feature map is fixed to M / 2M / 24C1. Since the Space_to_depth algorithm will cause certain information redundancy in the split channels, if compression is not performed, the number of channels of the feature map will increase exponentially with the increase in the number of downsampling times. In addition, the feature map will also be expanded in channels during the multi-scale feature extraction stage. Therefore, when designing the generative network, it is necessary to pre-set the number of channels of the output feature map of each module, and use the convolution layer to reconstruct the channels of the downsampled feature map.

[0051] The multi-scale feature fusion of the present invention can obtain the extracted features of different receptive fields. Its basic structure is composed of multiple convolution layers in parallel. The more types of convolution kernels, the richer the extracted features; finally, all the output features of the convolution layers are superimposed, and multi-layer convolution is used to perform feature fusion to reduce redundant information. Since multi-scale feature fusion will lead to an increase in training parameters and complexity, the complexity can be effectively reduced by compressing the downsampled feature map in the channel reconstruction stage. However, due to the complexity of the parallel structure, a series structure is used for multi-scale feature extraction; the series structure effectively reduces the amount of calculation and the amount of parameters while ensuring network performance; first, for large-size convolution kernels, such as 55 convolution kernels, it can be split into two 33 convolutions connected in series, which reduces the amount of calculation of large-size convolution kernels on the one hand, while ensuring that the range of the receptive field remains unchanged; and the use of two 33 convolutions effectively increases the learnability of the network and appropriately increases the generalization ability of the network.

[0052] S2. The discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs all and partial association learning training on the feature maps of the real image samples and the associated defective image samples and repairs the image data of the defective image samples. The data repair module takes the defective image samples as input and the corresponding real image samples as the target for repair training (using the image groups grouped into groups corresponding to the real image samples and the defective image samples for repair training, and repairing the partially missing or / and partially distorted areas to obtain the trained satellite image data repair model); the probability judgment module is used to judge and record the repair probability, and the repair probability indicates the probability that the repaired image data is the real image data. Use the prepared images to perform adversarial training on the generative adversarial network, and after the network converges, the network model finally used for the distorted area correction task is obtained.

[0053] In some embodiments, Figure 4 As shown, the discriminant network unit is constructed with a 30×30 two-dimensional matrix (that is, after obtaining a 900×1 patch, its dimension is still 900×1, and its input patch is encoded from left to right and from top to bottom, so 900 data can be refilled into the 30×30×1 feature according to this rule. Based on this, the output can be restored to a 30×30×1 feature map through a fully connected layer and a reshaping layer. The self-attention mechanism is mainly used for feature extraction, so richer and higher-level features can be extracted, which is very important for image texture discrimination. Moreover, since it is only used for image texture discrimination and does not need to understand the content or semantics of the image, the feature capability requirements are not high, and more requirements are placed on its generalization ability. Therefore, fewer Transformer encoders can be used to meet the design requirements, reduce computational complexity and memory consumption, and effectively improve its generalization ability; the present invention can also ensure the convergence speed of the discriminant network unit by introducing residual connections), the data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data, and the probability judgment module judges and records the repair probability based on the two-dimensional matrix. When the discriminative network unit is trained, the probability judgment module processes the restored image data and the real image data through convolution, pooling, and normalization respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment on a grid-by-grid basis.

[0054] S3: Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability. Fig. 9 As shown, the image data of two areas of the current satellite image data are intercepted, and after being repaired by the satellite image data repair model of the present invention, the following is obtained: Fig.10 As shown, in Fig. 9There are extra missing images in two regions of the defective image sample; after repair, Fig.10 As shown, the image data of the two regions have been repaired at the missing parts.

[0055] In this embodiment, the loss function L is generated in the adversarial network. G The expression is as follows:

[0056] L G =λ1×L a +λ2×L L1 , where L a Indicates the probability of the discriminative network unit correctly judging the data, L L1 represents the L1 loss between the restored image data and the real image data; λ1 and λ2 are L a , L L1 The coefficient of .

[0057] In this embodiment, the loss function L is generated in the adversarial network. G (Expressed as L G (G, D)) can also be expressed as follows:

[0058]

[0059] Where D(G(z|y)|y) represents the probability that the discriminant network unit judges the defective data Z as a real image after repairing it under the repair data y. It means to select defect data Z from the defect data set formed by defect data to find the mathematical expectation, It means to find the mathematical expectation of log(1-D(G(z|y)|y)); E x，y，z represents the mathematical expectation of the repair variable x, defect data Z, and repair data y. The repair data y is the data after the defect data is repaired, and the repair variable x is the modification amount of the defect data repair process. E x，y，z [||yG(x, z)||1 represents the mathematical expectation of ||yG(x, z)||1, and yG(x, z) represents the residual between the repaired data y and the image generated under the repair variable x and defect data Z.

[0060] Loss function L of probability judgment module in generative adversarial network D The expression of (G, D) is as follows:

[0061]

[0062] Represents the real image dataset P data In (x), the restoration variable x is selected to calculate the mathematical expectation, and D(x|y) represents the probability that the restoration variable x and the restoration data y are judged as images.

[0063] A satellite image data restoration system using a generative adversarial network comprises a satellite image data restoration model, a data acquisition module and a sample data set containing real image samples. The satellite image data restoration model includes a generative adversarial network, the generative adversarial network includes a generative network unit and a discriminative network unit, the generative network unit includes a defect image generation module and a feature extraction module, the defect image generation module is used to generate a defect image sample that is partially missing or / and partially distorted and associated with the real image sample, and the feature extraction module is used to respectively extract feature maps of the real image sample and the associated defect image sample; the discriminative network unit comprises a data restoration module and a probability judgment module, the data restoration module is used to perform full and partial association learning and training on the feature maps of the real image sample and the associated defect image sample and perform image data restoration on the defect image sample, the data restoration module is used to perform restoration training with the defect image sample as input and the corresponding real image sample as target; the probability judgment module is used to judge and record the restoration probability, and the restoration probability indicates the probability that the restored image data is the real image data; the data acquisition module is used to acquire the current satellite image data for slicing processing and sequentially encode and input the satellite image data restoration model after learning and training, the satellite image data restoration model performs image data restoration on the current satellite image data and records the restoration probability.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A satellite image data restoration method using a generative adversarial network, characterized in that: The methods include: S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples. The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates partially missing or / and partially distorted defect image samples associated with the real image samples. The feature extraction module extracts feature maps of the real image samples and the associated defect image samples. S2, the discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs all and local association learning training on the feature graphs of the real image samples and the associated defective image samples and performs image data repair on the defective image samples. The data repair module takes the defective image samples as input and the corresponding real image samples as targets for repair training; the probability judgment module is used to judge and record the repair probability, and the repair probability represents the probability that the repaired image data is the real image data; S3. Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability.

2. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: In step S1, the defect image generation module includes the following method: A11. Select some areas or some data in the real image samples to remove or fill in the erroneous data; A12. Select some areas in the real image samples to perform image blurring processing.

3. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: In step S1, the feature extraction module of the generative adversarial network includes four encoding units and three decoding units. The first to fourth encoding units of the four encoding units sequentially extract features of four scales on the input image, and the third to first decoding units of the three decoding units sequentially perform scale restoration decoding processing, wherein the third decoding unit decodes the third encoding unit with a jump connection, the second decoding unit decodes the second encoding unit with a jump connection, and the first decoding unit decodes the first encoding unit with a jump connection; the first decoding unit outputs a feature map after processing by a full convolutional layer and a global jump connection.

4. The satellite image data restoration method using a generative adversarial network according to claim 3, characterized in that: The encoding unit includes a convolution layer, a pooling layer, an activation layer and a normalization layer. The pooling layer adopts a multi-scale downsampling module. The multi-scale downsampling module method is as follows: B11. The scale of the input image is 2N× 2N, and the number of channels is C. The multi-scale downsampling module uses the Space_to_depth algorithm to traverse the input image from the upper left corner to obtain N×N 2×2 matrix units. Then, the pixels at the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are used as the four channel data of the output feature, and the channel reconstruction size of the output feature map is N×N×4C. B12. Use several convolutional layers with serial structures, and first perform convolution on the four channels of the feature map N×N×4C in step B11 respectively. Then perform point-by-point convolution on the features corresponding to each channel and obtain linear combinations and concatenations of the feature vectors.

5. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: The discriminant network unit is constructed with a 30×30 two-dimensional matrix. The data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data. At the same time, the probability judgment module judges and records the repair probability based on the two-dimensional matrix.

6. The satellite image data restoration method using a generative adversarial network according to claim 5, characterized in that: When the discriminative network unit is trained, the probability judgment module processes the restored image data and the real image data through convolution, pooling, and normalization respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment on a grid-by-grid basis.

7. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: Loss function L in generative adversarial networks G The expression is as follows: L G =λ1×L a +λ2×L L1 , where L a Indicates the probability of the discriminative network unit correctly judging the data, L L1 represents the L1 loss between the restored image data and the real image data; λ1 and λ2 are L a , L L1 The coefficient of .

8. The satellite image data restoration method using a generative adversarial network according to claim 3, characterized in that: Each decoding unit outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

9. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: The defective image sample source also includes defective satellite images corresponding to real image samples.

10. A satellite image data restoration system using a generative adversarial network, characterized in that: The invention comprises a satellite image data repair model, a data acquisition module and a sample data set containing real image samples. The satellite image data repair model comprises a generative adversarial network, the generative adversarial network comprises a generative network unit and a discriminative network unit, the generative network unit comprises a defect image generation module and a feature extraction module, the defect image generation module is used to generate defect image samples which are partially missing or / and partially distorted and are associated with real image samples, and the feature extraction module is used to extract feature maps of real image samples and associated defect image samples respectively; the discriminative network unit comprises a data repair module and a probability judgment module, the data repair module is used to perform full and partial association learning and training on feature maps of real image samples and associated defect image samples and perform image data repair on defect image samples, the data repair module is used to perform repair training with defect image samples as input and corresponding real image samples as targets; the probability judgment module is used to judge and record the repair probability, and the repair probability indicates the probability that the repaired image data is the real image data; the data acquisition module is used to collect the current satellite image data for slice processing and sequentially encode and input the learned and trained satellite image data repair model, the satellite image data repair model performs image data repair on the current satellite image data and records the repair probability.

Images