Satellite image data repairing method and system using generative adversarial network

Abstract

The application discloses a satellite image data repairing method and system using a generative adversarial network, and the method comprises the following steps: S1, a satellite image data repairing model containing a generative adversarial network is constructed, a defect image sample associated with a real image sample is generated by a defect image generation module, and a feature map of the real image sample and the associated defect image sample is extracted by a feature extraction module; S2, a data repairing module is used for performing all and local correlation learning training on the feature map of the real image sample and the associated defect image sample, and the data repairing module is used for performing repairing training by taking the defect image sample as input and the corresponding real image sample as target; S3, current satellite image data is collected and sliced, and then input into the satellite image data repairing model after learning training, and image data is repaired and the repairing probability is recorded. The satellite image data repairing model can obtain multi-level features, and the image repairing quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of satellite image data restoration, and in particular to a satellite image data restoration method and system using a generative adversarial network. Background Art

[0002] Satellite imagery is a crucial data collection method, capable of capturing a wide range of Earth information. It offers numerous unique advantages, including its unrestricted national boundaries, extensive coverage of the Earth's surface, regular observations, and objective and reliable data. Satellite imagery data (including remote sensing data) can also be integrated and analyzed with other geographic information, providing a foundation and support for interdisciplinary research. Therefore, satellite imagery has become an indispensable tool for global environmental monitoring, environmental resource management, and ecological management. Satellites can observe the entire Earth's surface in as little as one day, providing a wealth of satellite imagery data. MODIS sensors also capture wavelengths from visible light to thermal infrared, providing characteristic information on the atmosphere, land surface, and oceans. This provides a wealth of data to users worldwide, playing a vital role in vegetation indices, marine ecosystem research, soil erosion, and agricultural management, and serving as fundamental scientific data for regional ecological and environmental analysis.

[0003] During Earth observation, rainfall, changes in ground and soil moisture, and snow and ice melt all contribute to observational uncertainties. Clouds, the atmosphere, and aerosols are significant factors affecting data quality. Research indicates that an average of 35% of the global land surface is covered by clouds, with coverage exceeding 50% in some areas (such as the high-altitude Qinghai-Tibet Plateau). These factors cause clouds to absorb significant signal, impacting observations. Under thick cloud and rainy conditions, the resulting subcloud radiation is inaccurate, leading to missing data in satellite imagery products (such as MODIS reflectivity), which has already impacted their application. Data loss can reduce the accuracy of meteorological and climate models, hindering our understanding and prediction of weather and climate phenomena. Furthermore, missing data can affect environmental monitoring, such as vegetation growth, water temperature, and chlorophyll concentration, potentially leading to incomplete monitoring and assessment of environmental change. Furthermore, satellite imagery (including MODIS reflectivity) plays a vital role in natural disaster monitoring and response, such as fire monitoring, flood monitoring, and post-earthquake vegetation recovery. Missing data may reduce the ability to monitor and assess disasters and their impacts in a timely manner, and have an adverse impact on emergency response and post-disaster recovery. Satellite image data (including MODIS reflectivity data) is also used in agricultural management, including crop growth monitoring, soil moisture assessment, and drought monitoring; missing data may affect agricultural decision-making and management, and farmers and agricultural management agencies may not be able to accurately assess crop health and water resource utilization. Therefore, satellite image data or information obtained by satellites is inevitably contaminated. This contaminated data will greatly cause deviations in the research of scientific researchers, resulting in results that do not conform to the actual situation. Therefore, how to repair the images of missing or distorted areas becomes very important and is also a technical difficulty that urgently needs to be solved. Summary of the Invention

[0004] The purpose of the present invention is to provide a satellite image data restoration method using a generative adversarial network. The satellite image data restoration model includes a generative adversarial network. The defect image generation module can generate defect image samples that are partially missing or / and partially distorted and associated with real image samples. The generation network unit and the discrimination network unit can perform adversarial deep learning training on defect and real image features. In addition, the feature extraction module adopts multi-level encoding and decoding, has an excellent perception field of view, and adds jump connections between the same-level encoding and decoding to obtain multi-level features, thereby improving the quality of image restoration.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A satellite image data restoration method using a generative adversarial network, the method comprising:

[0007] S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples. The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates partially missing and / or partially distorted defect image samples associated with the real image samples. The feature extraction module extracts feature maps of the real image samples and the associated defect image samples.

[0008] S2, the discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs full and local correlation learning training on the feature maps of the real image samples and the associated defect image samples and performs image data repair on the defect image samples. The data repair module uses the defect image samples as input and the corresponding real image samples as targets for repair training; the probability judgment module is used to judge and record the repair probability, which represents the probability that the repaired image data is the real image data;

[0009] S3. Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability.

[0010] In order to better implement the present invention, in step S1, the defect image generation module includes the following method:

[0011] A11. Select some areas or some data in the real image sample to remove or fill in the erroneous data;

[0012] A12. Select some areas in the real image sample to perform image blurring processing.

[0013] Preferably, in step S1, the feature extraction module of the generative adversarial network includes four encoding units and three decoding units, the first to fourth encoding units of the four encoding units sequentially perform feature extraction of four scales on the input image, and the third to first decoding units of the three decoding units sequentially perform scale restoration decoding processing, wherein the third decoding unit decodes the third encoding unit with a jump connection, the second decoding unit decodes the second encoding unit with a jump connection, and the first decoding unit decodes the first encoding unit with a jump connection; the first decoding unit outputs a feature map after processing by a full convolutional layer and a global jump connection.

[0014] Preferably, the encoding unit includes a convolution layer, a pooling layer, an activation layer and a normalization layer. The pooling layer adopts a multi-scale downsampling module. The multi-scale downsampling module method is as follows:

[0015] B11. The input image scale is 2N×2N and the number of channels is C. The multi-scale downsampling module uses the Space_to_depth algorithm to traverse the input image from the upper left corner to obtain N×N 2×2 matrix units. The pixels in the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are used as the four channel data of the output feature. The channel reconstruction size of the output feature map is N×N×4C.

[0016] B12. Use several convolutional layers with a serial structure, and first perform convolution on the four channels of the feature map N×N×4C in step B11. Then, perform point-by-point convolution on the features corresponding to each channel and obtain a linear combination and concatenation of the feature vectors.

[0017] Preferably, the discriminant network unit is constructed with a 30×30 two-dimensional matrix, and the data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data, while the probability judgment module judges and records the repair probability based on the two-dimensional matrix.

[0018] Preferably, when the discriminant network unit is trained, the probability judgment module performs convolution, pooling, and normalization on the restored image data and the real image data, respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment grid by grid.

[0019] Preferably, the loss function L in the generative adversarial network is G The expression is as follows:

[0020] L G =λ1×L a +λ2×L L1 , where L a Indicates the probability of the discriminant network unit correctly judging the data, L L1 represents the L1 loss between the restored image data and the real image data; λ1 and λ2 are L a , L L1 The coefficient of .

[0021] Preferably, each of the decoding units outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

[0022] Preferably, the defective image sample source further includes defective satellite images corresponding to real image samples.

[0023] A satellite image data restoration system utilizing a generative adversarial network comprises a satellite image data restoration model, a data acquisition module, and a sample data set comprising real image samples. The satellite image data restoration model internally includes a generative adversarial network, which comprises a generative network unit and a discriminative network unit. The generative network unit comprises a defect image generation module and a feature extraction module. The defect image generation module is configured to generate defect image samples that are partially missing or / and partially distorted and are associated with the real image samples. The feature extraction module is configured to respectively extract feature maps of the real image samples and the associated defect image samples. The discriminative network unit comprises a data restoration module and a probability judgment module. The data restoration module is configured to perform full and partial association learning and training on the feature maps of the real image samples and the associated defect image samples and to perform image data restoration on the defect image samples. The data restoration module is configured to perform restoration training using the defect image samples as input and the corresponding real image samples as targets. The probability judgment module is configured to judge and record a restoration probability, which represents the probability that the restored image data is real image data. The data acquisition module is configured to acquire current satellite image data for slice processing and sequentially encode and input the slices into a trained satellite image data restoration model. The satellite image data restoration model performs image data restoration on the current satellite image data and records the restoration probability.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] (1) The satellite image data restoration model of the present invention includes a generative adversarial network. The defect image generation module can generate defect image samples that are partially missing or / and partially distorted and associated with real image samples. The generation network unit and the discrimination network unit can perform adversarial deep learning training on defect and real image features. In addition, the feature extraction module adopts multi-level encoding and decoding, and has an excellent perception field of view. At the same time, jump connections are added between the same-level encoding and decoding, which can obtain multi-level features and improve the quality of image restoration.

[0026] (2) The encoding unit of the present invention adopts a multi-scale downsampling module and utilizes N×N 2×2 matrix units; then the pixels at the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are respectively used as the four channel data of the output feature, and through channel-wise convolution, channel reconstruction, point-by-point convolution, linear combination and splicing of feature vectors, the efficiency and quality of multi-scale feature downsampling are improved.

[0027] (3) The present invention uses a sample data set to train a satellite image data restoration model, and through multi-level feature learning training, improves the image restoration quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1This is a flow chart of the satellite image data repair method of the present invention;

[0029] Figure 2 Schematic diagram of the principle of the feature extraction module in the embodiment;

[0030] Figure 3 Schematic diagram of the principle structure of the decoding unit of the feature extraction module in the embodiment;

[0031] Figure 4 Schematic diagram of the probability judgment module in the embodiment performing comparison and judgment between the restored image and the real image;

[0032] Figure 5 Schematic diagram of the principle of the multi-scale downsampling module in the embodiment;

[0033] Figure 6 A schematic diagram of dividing four channels according to the matrix unit in the embodiment;

[0034] Figure 7 Schematic diagram of the principle of processing four channels separately and linearly combining features in the embodiment;

[0035] Figure 8 To give a schematic diagram of the principle of channel processing;

[0036] Figure 9 Two defect image samples are provided as an example of an embodiment, including a schematic image diagram of a defect area;

[0037] Figure 10 For example, Figure 9 Schematic diagram of the image after defect area repair processing of the defect image sample. DETAILED DESCRIPTION

[0038] Below in conjunction with embodiment, the present invention is described in further detail:

[0039] Example

[0040] like Figure 1 As shown, a satellite image data restoration method using a generative adversarial network includes:

[0041] S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples; preferably, the sample data set includes defective image samples corresponding to the real image samples, that is, when preparing the sample data set, set a time threshold, collect real images without defects (including defects such as missing or distorted due to occlusion) within the time threshold, and at the same time collect defective image samples corresponding to the real images in position within the time threshold. For example, if the time threshold is set to one month, several real image samples in the early, middle and late weeks of a month can be collected, and several defective image samples in the early, middle and late weeks of the same month can also be collected. The real image samples and defective image samples within the same time threshold are grouped into corresponding groups (the real image samples and the defective image samples are generally corresponding single-band image maps) to facilitate subsequent restoration training.

[0042] The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates defect image samples that are partially missing or / and partially distorted and are associated with real image samples (subsequent defect image samples are grouped correspondingly with real image samples). In some embodiments, in step S1, the defect image generation module includes the following method:

[0043] A11. Select some areas or some data in the real image sample to remove or fill in the erroneous data;

[0044] A12. Select some areas in the real image sample to perform image blurring processing.

[0045] The defect image generation module may use method A11 or method A12 or other methods to generate defect image samples associated with real image samples.

[0046] In addition to the defect image samples generated by the defect image generation module, the present invention also includes defective satellite images corresponding to real image samples. Corresponding real image samples (with corresponding locations and time thresholds) are grouped together to facilitate subsequent feature extraction and restoration training (using real image samples as the target and restoration training for corresponding defect image samples).

[0047] The feature extraction module extracts feature maps of real image samples and associated defect image samples. The feature extraction module extracts features (including edge, texture information, etc.) from the corresponding real image samples and defect image samples. In some embodiments, Figure 2As shown, the feature extraction module of the generative adversarial network includes four encoding units and three decoding units. The first to fourth encoding units of the four encoding units sequentially extract features of four scales on the input image (the encoding process realizes the amplification of the number of channels while reducing the size of the feature map, so that the network learns features of different scales. The number of channels of the first encoding unit is 64, and the size of the feature map output is halved after each encoding unit). The third to first decoding units of the three decoding units sequentially perform scale restoration (continuously amplifying the size of the feature map while reducing the number of feature map channels, and gradually returning the encoded information to the feature map at the image size). The third decoding unit decodes the jump connection of the third encoding unit, the second decoding unit decodes the jump connection of the second encoding unit, and the first decoding unit decodes the jump connection of the first encoding unit; the first decoding unit is processed by the full convolution layer (including a 3×3 convolution layer, outputting a non-uniform residual image) and the global jump connection (since the pooling operation in the encoding process will lose part of the image information, the global jump connection can effectively solve this problem) to output the feature map. As shown Figure 3 As shown in the figure (W and H represent the width and height of the image respectively), each decoding unit outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

[0048] Preferably, the encoding unit includes a convolution layer (the preferred convolution layer in this embodiment uses a 3×3 convolution layer for feature extraction), a pooling layer, an activation layer (the preferred activation layer in this embodiment uses a LeayRelu activation function) and a normalization layer. The convolution layer and the pooling layer use a multi-scale downsampling module. The multi-scale downsampling module method is as follows:

[0049] B11. The input image size is 2N×2N and the number of channels is C, such as Figure 5 As shown, the multi-scale downsampling module uses the Space_to_depth algorithm to start from the upper left corner of the input image and traverse the input image to obtain N×N 2×2 matrix units; then the pixels in the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are used as the four channel data of the output feature, and the channel reconstruction size of the output feature map is N×N×4C. When the input image scale is 2N×2N, the algorithm starts from the upper left corner of the image and traverses the image to obtain N×N 22 matrix units; then the pixels in the upper left corner, upper right corner, lower left corner, and lower right corner of the matrix unit are used as the first, second, third, and fourth channel data of the output feature, and the size of the output feature map becomes N×N×4. When the number of input feature map channels is C, the size of the output feature map is N×N×4C. In a single image, a certain pixel and the surrounding pixels often have a strong correlation, that is, the image pixels themselves have a certain spatial similarity. Using the Space_to_depth algorithm, while reducing the two-dimensional size of the image and expanding the number of channels, the structural characteristics of the original image are less affected. As shown Figure 6 As shown in the figure, after the original image generates four downsampled images, it still has certain structural similarities; therefore, using the Space_to_depth algorithm, efficient downsampling operations can be achieved without adding additional weight parameters and calculation amount.

[0050] B12, using several convolutional layers in series (see Figure 8 , perform feature extraction of 1×1 and multiple 3×3 convolution layers on the feature map, and then concatenate the extracted multi-scale features and pass them into subsequent operations), and perform convolution on the four channels of the feature map N×N×4C in step B11, as shown in Figure 7 As shown, the features corresponding to each channel are then convolved point-by-point to obtain a linear combination and concatenation of the feature vectors. This point-by-point convolution of the features corresponding to each channel and the resulting linear combination of the feature vectors constitutes the channel reconstruction stage, the purpose of which is to compress and adjust the number of channels in the downsampled feature map. The block_size of the Space_to_depth algorithm is fixed at 2, meaning that when the input feature map is MMC1, the output feature map is fixed at M / 2M / 24C1. Because the Space_to_depth algorithm causes a certain amount of information redundancy in the split channels, without compression, the number of channels in the feature map will increase exponentially with increasing downsampling times. Furthermore, during the multi-scale feature extraction stage, feature maps are also expanded. Therefore, when designing the generative network, it is necessary to pre-set the number of channels in the output feature map of each module and use convolutional layers to reconstruct the channels of the downsampled feature map.

[0051] The multi-scale feature fusion of the present invention can obtain extracted features of different receptive fields. Its basic structure is composed of multiple convolution layers in parallel. The more types of convolution kernels, the richer the extracted features. Finally, the output features of all convolution layers are superimposed, and multi-layer convolution is used to perform feature fusion to reduce redundant information. Since multi-scale feature fusion will lead to an increase in training parameters and complexity, the complexity can be effectively reduced by compressing the down-sampled feature map in the channel reconstruction stage. However, due to the complexity of the parallel structure, a series structure is used for multi-scale feature extraction. The series structure effectively reduces the amount of calculation and parameters while ensuring network performance. First, for large-size convolution kernels, such as 55 convolution kernels, they can be split into two 33 convolutions connected in series. On the one hand, the amount of calculation of the large-size convolution kernel is reduced, while ensuring that the range of the receptive field remains unchanged. And the use of two 33 convolutions effectively increases the learnability of the network and appropriately increases the generalization ability of the network.

[0052] S2. The discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs full and partial correlation learning training on the feature maps of real image samples and associated defective image samples, and performs image data repair on the defective image samples. The data repair module uses the defective image samples as input and the corresponding real image samples as targets for repair training (using image groups grouped by real image samples and defective image samples for repair training, repairing partially missing and / or partially distorted areas to obtain a trained satellite image data repair model). The probability judgment module is used to determine and record the repair probability, which represents the probability that the repaired image data is the real image data. The prepared images are used to train the generative adversarial network. After the network converges, the final network model for the distorted area correction task is obtained.

[0053] In some embodiments, as Figure 4 As shown, the discriminant network unit is constructed with a 30×30 two-dimensional matrix (that is, after obtaining a 900×1 patch, its dimension is still 900×1, and its input patch is encoded from left to right and from top to bottom, so the 900 data can be refilled into the 30×30×1 feature according to this rule. Based on this, the output can be restored to a 30×30×1 feature map through a fully connected layer and a reshaping layer. The self-attention mechanism is mainly used for feature extraction, so richer and higher-level features can be extracted, which is very important for image texture discrimination. Moreover, since it is only used for image texture discrimination and does not require understanding the content or semantics of the image, its feature capability is not required, and more requirements are placed on its generalization capability. Therefore, fewer Transformer encoders can be used to meet the design requirements, reduce computational complexity and memory consumption, and effectively improve its generalization capability; the present invention can also ensure the convergence speed of the discriminant network unit by introducing residual connections), the data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data, and the probability judgment module judges and records the repair probability based on the two-dimensional matrix. During the training of the discriminative network unit, the probability judgment module performs convolution, pooling, and normalization on the restored image data and the real image data, respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment on each grid.

[0054] S3. Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability. Figure 9 As shown, the image data of two areas of the current satellite image data are intercepted and repaired by the satellite image data repair model of the present invention to obtain the following Figure 10 As shown, in Figure 9There are extra missing images in two areas of the defective image sample; after repair, Figure 10 As shown, the image data of the two regions have been repaired at the missing parts.

[0055] In this embodiment, the loss function L is generated in the adversarial network. G The expression is as follows:

[0056] L G =λ1×L a +λ2×L L1 , where L a Indicates the probability of the discriminant network unit correctly judging the data, L L1 represents the L1 loss between the restored image data and the real image data; λ1 and λ2 are L a , L L1 The coefficient of .

[0057] In this embodiment, the loss function L is generated in the adversarial network. G (Expressed as L G (G, D)) can also be expressed as follows:

[0058]

[0059] Where D(G(z|y)|y) represents the probability that the discriminant network unit judges the defective data Z as a real image after repairing it under the repair data y. It means to select defect data Z from the defect data set formed by defect data to find the mathematical expectation, Indicates the mathematical expectation of log(1-D(G(z|y)|y)); E x，y，z It represents the mathematical expectation of repair variable x, defect data Z, and repair data y. Repair data y is the data after the defect data is repaired, and repair variable x is the modification amount of the defect data repair process. E x，y，z [||yG(x, z)||1 represents the mathematical expectation of ||yG(x, z)||1, and yG(x, z) represents the residual between the repaired data y and the image generated under the repair variable x and defect data Z.

[0060] Loss function L of the probability judgment module in the generative adversarial network D The expression of (G, D) is as follows:

[0061]

[0062] Represents the real image dataset P data In (x), the restoration variable x is selected to calculate the mathematical expectation, and D(x|y) represents the probability that the restoration variable x and the restoration data y are judged to be images.

[0063] A satellite image data restoration system utilizing a generative adversarial network comprises a satellite image data restoration model, a data acquisition module, and a sample data set comprising real image samples. The satellite image data restoration model internally includes a generative adversarial network, which comprises a generative network unit and a discriminative network unit. The generative network unit comprises a defect image generation module and a feature extraction module. The defect image generation module is configured to generate defect image samples that are partially missing or / and partially distorted and are associated with the real image samples. The feature extraction module is configured to respectively extract feature maps of the real image samples and the associated defect image samples. The discriminative network unit comprises a data restoration module and a probability judgment module. The data restoration module is configured to perform full and partial association learning and training on the feature maps of the real image samples and the associated defect image samples and to perform image data restoration on the defect image samples. The data restoration module is configured to perform restoration training using the defect image samples as input and the corresponding real image samples as targets. The probability judgment module is configured to judge and record a restoration probability, which represents the probability that the restored image data is real image data. The data acquisition module is configured to acquire current satellite image data for slice processing and sequentially encode and input the slices into a trained satellite image data restoration model. The satellite image data restoration model performs image data restoration on the current satellite image data and records the restoration probability.

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A satellite image data restoration method using a generative adversarial network, characterized by: The methods include: S1. Construct a satellite image data restoration model including a generative adversarial network, and construct a sample data set including real image samples. The generative adversarial network includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module generates defect image samples that are partially missing or / and partially distorted and associated with the real image samples. The feature extraction module extracts feature maps of the real image samples and the associated defect image samples. The defect image generation module includes the following methods: A11. Select some areas or some data in the real image sample to remove or fill in the erroneous data; A12. Select some areas in the real image sample to perform image blurring. The feature extraction module of the generative adversarial network includes four encoding units and three decoding units. The first to fourth encoding units of the four encoding units sequentially extract features of the input image at four scales. The third to first decoding units of the three decoding units sequentially perform scale restoration decoding processing, wherein the third decoding unit decodes the third encoding unit with a jump connection, the second decoding unit decodes the second encoding unit with a jump connection, and the first decoding unit decodes the first encoding unit with a jump connection; the first decoding unit outputs a feature map after processing through a full convolutional layer and a global jump connection; S2, the discriminant network unit includes a data repair module and a probability judgment module. The data repair module performs full and local correlation learning training on the feature maps of the real image samples and the associated defect image samples and performs image data repair on the defect image samples. The data repair module uses the defect image samples as input and the corresponding real image samples as targets for repair training; the probability judgment module is used to judge and record the repair probability, which represents the probability that the repaired image data is the real image data; S3. Collect the current satellite image data, slice it, and input it into the trained satellite image data repair model. The satellite image data repair model repairs the current satellite image data and records the repair probability.

2. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: The encoding unit includes a convolution layer, a pooling layer, an activation layer, and a normalization layer. The pooling layer adopts a multi-scale downsampling module. The multi-scale downsampling module method is as follows: B11. The input image has a scale of 2N×2N and a number of channels of C. The multi-scale downsampling module uses the Space_to_depth algorithm to traverse the input image from the upper left corner to obtain N×N 2×2 matrix units. The pixels at the upper left, upper right, lower left, and lower right corners of the matrix units are used as the four channel data of the output feature, and the channel reconstruction size of the output feature map is N×N×4C. B12. Use several convolutional layers with a serial structure, and first perform convolution on the four channels of the feature map N×N×4C in step B11. Then, perform point-by-point convolution on the features corresponding to each channel and obtain a linear combination and concatenation of the feature vectors.

3. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: The discriminant network unit is constructed with a 30×30 two-dimensional matrix. The data repair module of the discriminant network unit uses the two-dimensional matrix to repair the image data. At the same time, the probability judgment module judges and records the repair probability based on the two-dimensional matrix.

4. The satellite image data restoration method using a generative adversarial network according to claim 3, characterized in that: During the training of the discriminative network unit, the probability judgment module performs convolution, pooling, and normalization on the restored image data and the real image data, respectively, and fuses and encodes them into a 30×30 two-dimensional matrix grid, and then performs restoration probability judgment on a grid-by-grid basis.

5. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: Loss function in generative adversarial networks The expression is as follows: ,in It represents the probability that the discriminant network unit correctly judges the data, Represents the L1 loss between the restored image data and the real image data; 、 They are 、 The coefficient of .

6. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: Each decoding unit outputs a feature map after skip connection, feature fusion, and deconvolution layer decoding processing.

7. The satellite image data restoration method using a generative adversarial network according to claim 1, characterized in that: The defective image sample source also includes defective satellite images corresponding to real image samples.

8. A satellite image data restoration system using a generative adversarial network for implementing the satellite image data restoration method according to any one of claims 1 to 7, characterized in that: The invention comprises a satellite image data repair model, a data acquisition module and a sample data set containing real image samples. The satellite image data repair model internally includes a generative adversarial network, which includes a generative network unit and a discriminative network unit. The generative network unit includes a defect image generation module and a feature extraction module. The defect image generation module is used to generate defect image samples that are partially missing or / and partially distorted and associated with the real image samples. The feature extraction module is used to extract feature maps of the real image samples and the associated defect image samples respectively. The discriminative network unit includes a data repair module and a probability judgment module. The data repair module is used to perform full and partial association learning and training on the feature maps of the real image samples and the associated defect image samples and perform image data repair on the defect image samples. The data repair module is used to perform repair training with the defect image samples as input and the corresponding real image samples as targets. The probability judgment module is used to judge and record the repair probability, which represents the probability that the repaired image data is the real image data. The data acquisition module is used to collect the current satellite image data for slice processing and sequentially encode and input the learned and trained satellite image data repair model. The satellite image data repair model performs image data repair on the current satellite image data and records the repair probability.

Images