Satellite cloud image prediction method based on WGAN-GP network and optical flow method

Abstract

This paper proposes a satellite cloud image prediction method based on a dual discriminator WGAN-GP and an optical flow method. First, the satellite cloud image dataset is histogram equalized. Then, a dual discriminator WGAN-GP network is constructed. The satellite cloud image sequence is input into the generator. The optical flow method is used to obtain the optical flow sequence of the satellite cloud image. The encoder-decoder module is then used to generate the predicted optical flow. The historical satellite cloud image and the predicted optical flow are fused to generate the satellite cloud image prediction. The generated satellite cloud image and the real satellite cloud image are input into the discriminator for training. The spatial discriminator and the temporal discriminator respectively evaluate the generated samples and the real samples. Through multiple adversarial training, the dual discriminator WGAN-GP network reaches a Nash equilibrium. Finally, the historical satellite cloud image data is input into the generator of the trained dual discriminator WGAN-GP network to obtain the satellite cloud image prediction result. This method can effectively reduce the interference of ground background information on the satellite cloud image prediction task and improve the accuracy and efficiency of satellite cloud image prediction.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a satellite cloud image prediction method based on a dual discriminator WGAN-GP and an optical flow method. Background Art

[0002] Solar energy, with its abundant reserves and widespread distribution, is one of the most promising renewable energy sources. Compared to traditional power generation methods like thermal and hydropower, solar energy is subject to fluctuations and randomness, influenced by diurnal variations, seasonal changes, and changes in the Earth's atmosphere. Consequently, photovoltaic power output exhibits instability. Among the many meteorological factors influencing photovoltaic power generation, predicting cloud motion over short periods of time with high spatial resolution is a key challenge in photovoltaic power and volatility forecasting.

[0003] Satellite cloud images play an important role in weather forecasting. Traditional satellite cloud image prediction methods include those based on numerical weather forecasting and those based on recurrent neural networks. Numerical weather forecasting methods require a large amount of meteorological data, consume a lot of resources, and have a high computational load, making them difficult to meet the needs of short-term satellite cloud image prediction. Recurrent neural network methods have deeper network layers, making it difficult for gradients to propagate deep into the layers. As the prediction time increases, the generated prediction results become increasingly blurred. Furthermore, when processing satellite cloud image data, the above methods are susceptible to interference from ground background information, resulting in inaccurate prediction results. The present invention utilizes optical flow to extract motion information from satellite cloud images and uses a dual-discriminator WGAN-GP network for prediction model training. The optical flow method can calculate the motion information of pixels in two adjacent frames of a continuous motion image sequence, namely the direction and distance each pixel moves. As an unsupervised learning method, the WGAN-GP network can learn the potential features of real samples and generate pseudo samples that are very similar to them through adversarial training between the generator and the discriminator. By training the WGAN-GP network to generate predicted optical flow, the spatial discriminator and temporal discriminator are used to evaluate the spatial similarity and temporal similarity between the generated samples and the real samples, respectively. Through adversarial training, the dual-discriminator WGAN-GP network reaches Nash equilibrium. Finally, the predicted optical flow is fused with historical satellite cloud images to obtain the predicted satellite cloud image. This effectively avoids the interference of ground background information on the prediction results and makes the generated predicted image have a clarity similar to the original image. Summary of the Invention

[0004] Purpose of the invention: To address the above-mentioned issues, this application provides a satellite cloud image prediction method. This method uses optical flow and a dual-discriminator WGAN-GP network to effectively eliminate the interference of ground background on the prediction task, better extract the spatiotemporal characteristics of satellite cloud images, and realize the prediction of future cloud movement.

[0005] Technical solution: A satellite cloud image prediction method based on the dual discriminator WGAN-GP and the optical flow method includes the following steps:

[0006] Step S1: Obtain historical satellite cloud image sequence data;

[0007] Step S2: performing histogram equalization preprocessing on the satellite cloud image;

[0008] Step S3: For the pre-processed images, take a sequence of t satellite cloud images as historical data, and the t+1th satellite cloud image as the prediction target;

[0009] Step S4: Construct a dual-discriminator WGAN-GP network. Each training step uses a sequence of t consecutive satellite cloud images as input to the generator. An optical flow map is calculated for every two adjacent satellite cloud images, resulting in a total of t-1 optical flow maps. The optical flow sequence is passed through the encoder-decoder structure to output the predicted optical flow. The predicted optical flow is fused with the satellite cloud image at time t to generate the predicted satellite cloud image at time t+1.

[0010] Step S5: Compare the generated satellite cloud image with the real satellite cloud Figure 1 The samples are input into the discriminator for training. The spatial discriminator and the temporal discriminator respectively evaluate the spatial similarity and temporal similarity between the generated samples and the real samples. Through multiple iterative adversarial training, the dual discriminator WGAN-GP network reaches Nash equilibrium.

[0011] Step S6: Input the historical satellite cloud image data into the trained dual discriminator WGAN-GP network generator to obtain the satellite cloud image prediction image, add the generated satellite cloud image prediction image to the sequence, and iteratively execute the generator algorithm to obtain the t+1 to 2t satellite cloud image prediction images.

[0012] The formula for performing histogram equalization preprocessing on the image in step S2 is:

[0013]

[0014]

[0015] Where round() means rounding the result, cdf(i) means the cumulative distribution function of the pixel with value i, p x (i) represents the probability of occurrence of a pixel with value i, L is the grayscale level, v is the original image pixel value, and h(v) is the pixel value after histogram equalization operation.

[0016] The step S4 comprises the following steps:

[0017] Step S41: For a sequence of t consecutive satellite cloud images inputted into the generator, an optical flow map is obtained by calculating each two adjacent satellite cloud images, and a total of t-1 optical flow maps are obtained;

[0018] Step S42: Construct a generator in the dual-discriminator WGAN-GP network. The structure of the generator network is as follows: input layer → optical flow extraction layer → 3D convolution layer 1 → 3D convolution layer 2 → 3D convolution layer 3 → 3D convolution layer 4 → upsampling layer 1 → upsampling layer 2 → upsampling layer 3 → upsampling layer 4 → fusion layer → output layer. 3D convolution layer 1, 3D convolution layer 2, 3D convolution layer 3, and 3D convolution layer 4 belong to the encoder part, and upsampling layer 1, upsampling layer 2, upsampling layer 3, and upsampling layer 4 belong to the decoder part.

[0019] Step S43: construct the discriminator in the dual-discriminator WGAN-GP network. The discriminator network includes a spatial discriminator and a temporal discriminator. The structure of the spatial discriminator network is as follows: input layer → 2D convolutional layer 1 → 2D convolutional layer 2 → 2D convolutional layer 3 → 2D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output.

[0020] Step S44: construct the discriminator in the dual-discriminator WGAN-GP network, where the structure of the temporal discriminator network is as follows: input layer → 3D convolutional layer 1 → 3D convolutional layer 2 → 3D convolutional layer 3 → 3D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output;

[0021] Step S45: Input the t-1 optical flow map into the generator, perform encoding processing through the 3D convolution layer, and then perform decoding processing through the upsampling layer to generate the prediction of the t-th optical flow map;

[0022] Step S46: Use the tth optical flow map to remap the tth satellite cloud image to generate a prediction of the t+1th satellite cloud image. The formula for the remapping operation is:

[0023] p t+1 (x,y)=p t (xu,yv)

[0024] where p t is the pixel value at the satellite cloud point (x, y) at time t, u and v are the information in the optical flow map, representing the motion components of the pixel in the x and y directions respectively.

[0025] In step S42, the upsampling layer first uses bilinear interpolation to double the size of the feature map, and then uses convolution to halve the number of channels; three skip connections are added after the three-dimensional convolution layer 1, the three-dimensional convolution layer 2, and the three-dimensional convolution layer 3, which are connected to the upsampling layer 1, the upsampling layer 2, and the upsampling layer 3 respectively. The feature map processed by the three-dimensional convolution in the encoder is spliced ​​with the feature map after upsampling in the decoder, and then passed to the next layer as the input feature map, so that the decoder can obtain the information learned by the shallow encoder.

[0026] In step S42, the convolution kernel size of the three-dimensional convolution layer 1 is 3×3×3, the convolution kernel size of the three-dimensional convolution layers 2, 3, and 4 is 2×3×3, the step size of the three-dimensional convolution layer 4 is (1, 1, 1), and the step size of the three-dimensional convolution layers 1, 2, and 3 is (1, 2, 2), and the padding is (0, 1, 1).

[0027] In step S43, the convolution kernel size of the convolution layer is 3×3, the stride is 2, and the padding is 1.

[0028] In step S44, the convolution kernel size of the three-dimensional convolution layer 1 to the three-dimensional convolution layer 3 of the temporal discriminator network is 3×3×3, the convolution kernel size of the three-dimensional convolution layer 4 is 2×3×3, the step size of each layer is (1, 2, 2), and the padding is (0, 1, 1).

[0029] The step S5 includes the following steps:

[0030] Step S51: Fix the generator parameters and train the discriminator. The loss function of the dual discriminator WGAN-GP network discriminator is:

[0031]

[0032]

[0033]

[0034] L D =L S +L T +L GP

[0035] Among them, L S is the spatial discriminator loss, L T is the temporal discriminator loss, L GP is the gradient penalty term, L D is the final loss of the discriminator, m is the batch size of training data, G() is the output of the generator, i.e., the prediction of the satellite cloud image, x f Represents the historical satellite cloud image sequence, x r is the target satellite cloud image, s rrepresents the real satellite cloud image sequence, i.e. the historical satellite cloud image plus the target satellite cloud image, s f Represents the predicted satellite cloud image sequence, x p The samples obtained by linear interpolation sampling between the generated cloud image and the real cloud image, s p The samples are obtained by linear interpolation sampling between the generated cloud sequence and the real cloud sequence;

[0036] Step S52: Update the discriminator parameters using the Adam optimization algorithm;

[0037] Step S53: Fix the discriminator parameters and train the generator. The loss function of the dual discriminator WGAN-GP network generator is:

[0038]

[0039] Among them, L G is the final loss function of the generator, m is the batch size of training data, x f Represents historical satellite cloud images, s f Represents the predicted satellite cloud image sequence;

[0040] Step S54: Use the Adam optimization algorithm to update the generator parameters.

[0041] The step S52 uses the following formula

[0042] m dω =β1m dω +(1-β1)dω

[0043] v dω =β2v dω +(1-β2)(dω) 2

[0044]

[0045]

[0046]

[0047] Among them, ω is the discriminator parameter, m dω is the first-order moment estimate of the parameter gradient, v dω is the second-order moment estimation of the parameter gradient, β1 and β2 are the decay rates, and For m dω and v dω is the deviation correction, α is the learning rate, and ε is a very small constant.

[0048] Beneficial effects: The present invention uses optical flow to extract satellite cloud image motion information, effectively eliminating the interference of ground background information on the satellite cloud image prediction task. Skip connections are added to the generator part of the dual-discriminator WGAN-GP network, enabling the prediction model to obtain richer information from multi-scale feature maps. The discriminator part uses two discriminators, spatial and temporal, and the trained prediction model can effectively predict complex cloud motion processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is the flowchart of satellite cloud image prediction based on dual discriminator WGAN-GP;

[0050] Figure 2 Schematic diagram of the overall structure of the dual-discriminator WGAN-GP network;

[0051] Figure 3 Schematic diagram of the dual-discriminator WGAN-GP network generator;

[0052] Figure 4 Schematic diagram of the dual-discriminator WGAN-GP network spatial discriminator;

[0053] Figure 5 Schematic diagram of the dual-discriminator WGAN-GP network time discriminator;

[0054] Figure 6 This is a graph showing the convergence of the algorithm of this embodiment;

[0055] Figure 7 Comparison between the real satellite cloud image sequence and the generated predicted satellite cloud image sequence. DETAILED DESCRIPTION

[0056] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0057] Figure 1 is a flow chart of an embodiment of the present invention, Figure 2 This is the structure diagram of the dual discriminator WGAN-GP network generator. Figure 3 The dual discriminator WGAN-GP network discriminator structure diagram includes the following steps:

[0058] Step S1: Obtain historical satellite cloud image sequence data;

[0059] Step S2: Preprocess the source data, adjust the size of the satellite cloud image to 128×128 pixels, and perform histogram equalization on the satellite cloud image. The formula for histogram equalization of the image is:

[0060]

[0061]

[0062] Where round() means rounding the result, cdf(i) means the cumulative distribution function of the pixel with value i, p x (i) represents the probability of occurrence of pixel with value i, L is the grayscale level, v is the original image pixel value, and h(v) is the pixel value after histogram equalization operation;

[0063] Step S3: For the pre-processed images, take a sequence of t satellite cloud images as historical data, and the t+1th satellite cloud image as the prediction target;

[0064] Step S4: Construct a dual-discriminator WGAN-GP network. Each training step uses a sequence of t consecutive satellite cloud images as input to the generator. An optical flow map is calculated for every two adjacent satellite cloud images, resulting in a total of t-1 optical flow maps. The optical flow sequence is passed through the encoder-decoder structure to output the predicted optical flow. The predicted optical flow is fused with the satellite cloud image at time t to generate the predicted satellite cloud image at time t+1.

[0065] Step S41: For a sequence of t consecutive satellite cloud images input to the generator, an optical flow map is obtained by calculating each of two adjacent satellite cloud images, and a total of t-1 optical flow maps are obtained. The formula for calculating the optical flow map of two adjacent satellite cloud images is:

[0066] I x u+I y v+I t =0

[0067] Where u and v represent the motion components of the pixel in the x and y directions respectively, I x and I y Represent the gradient of image grayscale in the x and y directions respectively, I t Represents the gradient of the image grayscale in the time dimension t. In order to obtain u and v, a window of size 5×5 is selected, and the 25 pixel values ​​in the window are used to construct the equation system and solve it using the least squares method:

[0068]

[0069] Where m is the number of pixels in the window, are the gradients of the image grayscale of pixel i in the x and y directions, is the gradient in the time dimension t;

[0070] Step S42, construct the generator in the dual discriminator WGAN-GP, Figure 2This is the generator architecture of the dual-discriminator WGAN-GP network. The generator network structure is as follows: input layer → optical flow extraction layer → 3D convolution layer 1 → 3D convolution layer 2 → 3D convolution layer 3 → 3D convolution layer 4 → upsampling layer 1 → upsampling layer 2 → upsampling layer 3 → upsampling layer 4 → fusion layer → output. The convolution kernel size of 3D convolution layer 1 is 3×3×3, and the convolution kernel sizes of 3D convolution layers 2, 3, and 4 are 2×3×3. The stride of 3D convolution layer 4 is (1,1,1), and the stride of 3D convolution layers 1, 2, and 3 is (1,2,2). Padding is (0,1,1). The upsampling layer first doubles the size of the feature map using bilinear interpolation, and then uses convolution to halve the number of channels. 3D Convolution Layers 1, 2, 3, and 4 belong to the encoder, while Upsampling Layers 1, 2, 3, and 4 belong to the decoder. Three skip connections are added after 3D Convolution Layers 1, 2, and 3, connecting to Upsampling Layers 1, 2, and 3, respectively. These connections concatenate the feature maps processed by 3D convolution in the encoder with the upsampled feature maps in the decoder, and then pass them to the next layer as input feature maps. This allows the decoder to obtain information learned by the shallow encoder layer, minimizing information loss during feature map propagation.

[0071] Step S43: Construct the discriminator in the dual-discriminator WGAN-GP. The discriminator network is divided into a spatial discriminator and a temporal discriminator. The structure of the spatial discriminator network is as follows: input layer → 2D convolutional layer 1 → 2D convolutional layer 2 → 2D convolutional layer 3 → 2D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output. The convolution kernel size of each convolution layer is 3×3, the stride is 2, and the padding is 1.

[0072] Step S44: Construct the discriminator in the dual-discriminator WGAN-GP. The temporal discriminator network has the following structure: input layer → 3D convolutional layer 1 → 3D convolutional layer 2 → 3D convolutional layer 3 → 3D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output. The convolution kernel size of 3D convolutional layers 1 to 3 is 3×3×3, and the convolution kernel size of 3D convolutional layer 4 is 2×3×3. The stride of each layer is (1, 2, 2), and the padding is (0, 1, 1).

[0073] In step S45, the t-1 optical flow map is encoded by three-dimensional convolution, and then decoded by the upsampling layer to generate the prediction of the t-th optical flow map. The LeakyReLU function is used for nonlinear activation processing on each layer of the convolutional neural network. The formula of the LeakyReLU function is:

[0074]

[0075] Among them, x represents the data after convolution operation, and α is a very small non-negative constant.

[0076] Step S45: Use the tth optical flow map to remap the tth satellite cloud image to generate a prediction of the t+1th satellite cloud image. The formula for the remapping operation is:

[0077] p t+1 (x, y) = p t (xu,yv)

[0078] where p t is the pixel value at the satellite cloud point (x, y) at time t, u and v are the information in the optical flow map, representing the motion components of the pixel in the x and y directions respectively.

[0079] Step S5: Compare the generated satellite cloud image with the real satellite cloud Figure 1 The samples are input into the discriminator for training. The spatial discriminator and the temporal discriminator respectively evaluate the spatial similarity and temporal similarity between the generated samples and the real samples. Through multiple iterative adversarial training, the dual discriminator WGAN-GP network reaches Nash equilibrium.

[0080] Step S51: Fix the generator parameters and train the discriminator. The loss function of the dual discriminator WGAN-GP network discriminator is:

[0081]

[0082]

[0083]

[0084] L D =L S +L T +L GP

[0085] Among them, L S is the spatial discriminator loss, L T is the temporal discriminator loss, L GP is the gradient penalty term, L D is the final loss of the discriminator, m is the batch size of training data, G() is the output of the generator, i.e., the prediction of the satellite cloud image, x f Represents the historical satellite cloud image sequence, x r is the target satellite cloud image, s r represents the real satellite cloud image sequence, i.e. the historical satellite cloud image plus the target satellite cloud image, s f Represents the predicted satellite cloud image sequence, x pThe samples obtained by linear interpolation sampling between the generated cloud image and the real cloud image, s p The samples are obtained by linear interpolation sampling between the generated cloud sequence and the real cloud sequence;

[0086] Step S52: Use the Adam optimization algorithm to update the discriminator parameters, set the learning rate to 0.0002, the exponential decay rate of the first-order moment estimate to 0.9, and the exponential decay rate of the second-order moment estimate to 0.999:

[0087] m dω =β1m dω +(1-β1)dω

[0088] v dω =β2v dω +(1-β2)(dω) 2

[0089]

[0090]

[0091]

[0092] Among them, ω is the discriminator parameter, m dω is the first-order moment estimate of the parameter gradient, v dω is the second-order moment estimation of the parameter gradient, β1 and β2 are the decay rates, and For m dω and v dω Bias correction, α is the learning rate, and ε is a very small constant;

[0093] Step S53: Fix the discriminator parameters and train the generator. The loss function of the dual discriminator WGAN-GP network generator is:

[0094]

[0095] Among them, L G is the final loss function of the generator, m is the batch size of training data, x f Represents historical satellite cloud images, s f Represents the predicted satellite cloud image sequence;

[0096] Step S54: Use the Adam optimization algorithm to update the generator parameters, set the learning rate to 0.0002, the exponential decay rate of the first-order moment estimate to 0.9, and the exponential decay rate of the second-order moment estimate to 0.999.

[0097] Step S6: Input the historical satellite cloud image data into the trained dual discriminator WGAN-GP network generator to obtain the satellite cloud image prediction image, add the generated satellite cloud image prediction image to the sequence, and iteratively execute the generator algorithm to obtain the t+1 to 2t satellite cloud image prediction images.

[0098] Figure 4 This is the convergence of the prediction model of the improved dual discriminator WGAN-GP network combined with the optical flow method used in the present invention. As the number of training times increases, the distribution of generated samples is getting closer to the distribution of real samples, and the generator loss and discriminator loss are approaching 0. It can be seen from the figure that the model tends to converge after about 200 training times.

[0099] Figure 6 The paper presents a comparison between the predicted satellite cloud images generated by this method and those generated by several traditional methods. The cloud images generated by these traditional methods gradually become blurred as the prediction time increases. This method uses the optical flow method and the dual-discriminator WGAN-GP network to generate predicted images. With the help of optical flow, the predicted results can be extrapolated based on the original image, resulting in a predicted image with a clarity similar to that of the original image.

[0100]

[0101] Table 1

[0102] Table 1 is a quantitative analysis of the clarity of the prediction results generated by this method and several traditional prediction methods, showing the average results when predicting 4 frames and 8 frames, respectively. This paper uses the grayscale variance (SMD) function to evaluate image clarity. The larger the value, the higher the clarity.

[0103]

[0104]

[0105] Table 2

[0106] Table 2 is a quantitative analysis of the accuracy of the prediction results generated by this method and several traditional prediction methods, showing the average results when predicting 4 frames and 8 frames, respectively. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are used to evaluate the accuracy of the prediction results. The larger the PSNR and SSIM values, the closer the prediction results are to the true results.

[0107] The above description is only a specific implementation method of the present invention and is not intended to limit the scope of the present invention. Any modification or equivalent replacement made based on the technical essence of the present invention should be included in the scope of protection required by the present invention.

Claims

1. A satellite cloud image prediction method based on WGAN-GP network and optical flow method, characterized by: The following steps are involved: Step S1: Obtain historical satellite cloud image sequence data; Step S2: performing histogram equalization preprocessing on the satellite cloud image; Step S3: For the pre-processed images, take the t-th satellite cloud image sequence as historical data and the t+1th satellite cloud image as the prediction target; Step S4: Construct a dual-discriminator WGAN-GP network. Each training step uses a sequence of t consecutive satellite cloud images as input to the generator. An optical flow map is calculated for every two adjacent satellite cloud images, resulting in a total of t-1 optical flow maps. The optical flow sequence is passed through the encoder-decoder structure to output the predicted optical flow. The predicted optical flow is fused with the satellite cloud image at time t to generate the predicted satellite cloud image at time t+1. Step S5: The generated satellite cloud image and the real satellite cloud image are input into the discriminator for training. The spatial discriminator and the temporal discriminator respectively evaluate the spatial similarity and temporal similarity of the generated samples and the real samples. Through multiple iterative adversarial training, the dual discriminator WGAN-GP network reaches Nash equilibrium. Step S6: Input the historical satellite cloud image data into the trained dual discriminator WGAN-GP network generator to obtain the satellite cloud image prediction image, add the generated satellite cloud image prediction image to the sequence, and iteratively execute the generator algorithm to obtain the t+1 to 2t satellite cloud image prediction images.

2. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 1, characterized in that: The formula for performing histogram equalization preprocessing on the image in step S2 is: Where round() means rounding the result, cdf(i) means the cumulative distribution function of the pixel with value i, p x (i) represents the probability of occurrence of a pixel with value i, L is the grayscale level, v is the original image pixel value, and h(v) is the pixel value after histogram equalization operation.

3. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 1, characterized in that: The step S4 comprises the following steps: Step S41: For a sequence of t consecutive satellite cloud images input to the generator, an optical flow map is obtained by calculating each two adjacent satellite cloud images, and a total of t-1 optical flow maps are obtained; Step S42: Construct a generator in the dual-discriminator WGAN-GP network. The structure of the generator network is as follows: input layer → optical flow extraction layer → 3D convolution layer 1 → 3D convolution layer 2 → 3D convolution layer 3 → 3D convolution layer 4 → upsampling layer 1 → upsampling layer 2 → upsampling layer 3 → upsampling layer 4 → fusion layer → output layer. 3D convolution layer 1, 3D convolution layer 2, 3D convolution layer 3, and 3D convolution layer 4 belong to the encoder part, and upsampling layer 1, upsampling layer 2, upsampling layer 3, and upsampling layer 4 belong to the decoder part. Step S43: construct the discriminator in the dual-discriminator WGAN-GP network. The discriminator network includes a spatial discriminator and a temporal discriminator. The structure of the spatial discriminator network is as follows: input layer → 2D convolutional layer 1 → 2D convolutional layer 2 → 2D convolutional layer 3 → 2D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output. Step S44: construct the discriminator in the dual-discriminator WGAN-GP network, where the structure of the temporal discriminator network is as follows: input layer → 3D convolutional layer 1 → 3D convolutional layer 2 → 3D convolutional layer 3 → 3D convolutional layer 4 → fully connected layer 1 → fully connected layer 2 → fully connected layer 3 → output; Step S45: Input the t-1 optical flow map into the generator, encode it through the 3D convolution layer, and then decode it through the upsampling layer to generate the prediction of the t-th optical flow map; Step S46: Use the tth optical flow map to remap the tth satellite cloud image to generate a prediction of the t+1th satellite cloud image. The formula for the remapping operation is: p t+1 (x,y)=p t (x-u,y-v) where p t is the pixel value at the satellite cloud point (x, y) at time t, u and v are the information in the optical flow map, representing the motion components of the pixel in the x and y directions respectively.

4. The satellite cloud image prediction method based on the WGAN-GP network and the optical flow method according to claim 3, characterized in that: In step S42, the upsampling layer first uses bilinear interpolation to double the size of the feature map, and then uses convolution to halve the number of channels; three skip connections are added after the three-dimensional convolution layer 1, the three-dimensional convolution layer 2, and the three-dimensional convolution layer 3, which are connected to the upsampling layer 1, the upsampling layer 2, and the upsampling layer 3 respectively. The feature map processed by the three-dimensional convolution in the encoder is spliced ​​with the feature map after upsampling in the decoder, and then passed to the next layer as the input feature map, so that the decoder can obtain the information learned by the shallow encoder.

5. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 3, characterized in that: In step S42, the convolution kernel size of the three-dimensional convolution layer 1 is 3×3×3, the convolution kernel size of the three-dimensional convolution layers 2, 3, and 4 is 2×3×3, the step size of the three-dimensional convolution layer 4 is (1, 1, 1), and the step size of the three-dimensional convolution layers 1, 2, and 3 is (1, 2, 2), and the padding is (0, 1, 1).

6. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 3, characterized in that: In step S43, the convolution kernel size of the convolution layer is 3×3, the stride is 2, and the padding is 1.

7. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 3, characterized in that: In step S44, the convolution kernel size of the three-dimensional convolution layer 1 to the three-dimensional convolution layer 3 of the temporal discriminator network is 3×3×3, the convolution kernel size of the three-dimensional convolution layer 4 is 2×3×3, the step size of each layer is (1, 2, 2), and the padding is (0, 1, 1).

8. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 1, characterized in that: The step S5 includes the following steps: Step S51: Fix the generator parameters and train the discriminator. The loss function of the dual discriminator WGAN-GP network discriminator is: L D =L S +L T +L GP Among them, L S is the spatial discriminator loss, L T is the temporal discriminator loss, L GP is the gradient penalty term, L D is the final loss of the discriminator, m is the batch size of training data, G() is the output of the generator, i.e., the prediction of the satellite cloud image, x f Represents the historical satellite cloud image sequence, x r is the target satellite cloud image, s r represents the real satellite cloud image sequence, i.e. the historical satellite cloud image plus the target satellite cloud image, s f Represents the predicted satellite cloud image sequence, x p The samples obtained by linear interpolation sampling between the generated cloud image and the real cloud image, s p The samples are obtained by linear interpolation sampling between the generated cloud sequence and the real cloud sequence; Step S52: Update the discriminator parameters using the Adam optimization algorithm; Step S53: Fix the discriminator parameters and train the generator. The loss function of the dual discriminator WGAN-GP network generator is: Among them, L G is the final loss function of the generator, m is the batch size of training data, x f Represents historical satellite cloud images, s f Represents the predicted satellite cloud image sequence; Step S54: Use the Adam optimization algorithm to update the generator parameters.

9. The satellite cloud image prediction method based on WGAN-GP network and optical flow method according to claim 8, characterized in that: The step S52 uses the following formula m dω =β1m dω +(1-β1)dω v dω =β2v dω +(1-β2)(dω) 2 Among them, ω is the discriminator parameter, m dω is the first-order moment estimate of the parameter gradient, v dω is the second-order moment estimation of the parameter gradient, β1 and β2 are the decay rates, and For m dω and v dω The deviation correction is α, the learning rate, and ε is a very small constant.

Images