Satellite cloud picture nowcasting method based on convolutional gated circulation attention network CGAFNet

Abstract

The invention discloses a satellite cloud picture nowcasting method based on a convolutional gated loop attention network CGAFNet, and the method comprises the steps: constructing a convolutional gated loop attention network CGAFNet model, extracting cloud picture features at all moments through an attention feature extraction module, and replacing a ReLU activation function with a Leaky ReLU activation function; extracting time sequence characteristics of the cloud picture through a ConvGRU (Convolutional Gated Recirculating Unit) network; and main and auxiliary loss PaSLoss is provided as a loss function of the model, a coding-decoding structure is constructed, and the space-time information of the satellite cloud picture is better extracted. Results show that the CGAFNet has a root mean square error of 10.00 K, structural similarity of 0.74 and a peak signal-to-noise ratio of 31.43 in a cloud picture extrapolation task, the model can accurately predict the growth and elimination evolution process of the cloud cluster, and is superior to other networks in various indexes, and it is proved that the method can obtain more accurate prediction precision.

Description

Technical Field

[0001] The present invention belongs to the field of satellite cloud image prediction, and in particular to a satellite cloud image nowcasting method based on a convolutional gated recurrent attention network (CGAFNet). Background Art

[0002] Meteorological satellite cloud images are the most intuitive presentation of weather conditions. The changes in cloud clusters on the cloud images reflect and affect the occurrence, development, evolution and extinction of weather systems to a certain extent. Studying the evolution of cloud clusters (especially severe convective cloud clusters) is an effective means of monitoring and forecasting severe convective weather. Therefore, using satellite cloud images to achieve effective extrapolation forecasts has important research significance.

[0003] Traditional satellite cloud image prediction methods usually use linear extrapolation methods such as cross-correlation method, time mutation method, variational echo tracking method, as well as statistical forecasting methods such as maximum correlation coefficient and model output statistical method. With the iterative update of the method, the research on optical flow method in nowcasting has gradually increased, but its ability to predict changes in the formation and development of cloud clusters is insufficient. The optical flow method regards the movement between images as linear motion, so it has a weak ability to capture nonlinear motion. However, the movement of clouds has nonlinear changes such as deformation and rotation, so the prediction effect of traditional methods is inaccurate. Under the influence of the rapid development of deep learning, the meteorological field has also incorporated relevant methods to assist or achieve accurate weather forecasts. Although the problem of insufficient forecasting ability of traditional methods has been solved to a certain extent, cloud image prediction still has the problem that the clarity and granularity of the image gradually decrease due to the increasing extrapolation time. At the same time, a lot of important forecasting information will be lost in long time intervals, and it is easy to ignore the process from the birth to the extinction of clouds in a long time interval. It is known that strong convective clouds have the characteristics of suddenness and short life history (short duration, generally a few minutes to more than ten hours). The time interval is too long to accurately monitor the disastrous weather. In addition, the model has poor forecasting ability for cloud image details, and it is difficult to accurately predict the development or attenuation characteristics of clouds (especially small-scale clouds). The existing satellite cloud image prediction methods have problems such as difficulty in capturing the development of small-scale clouds, unclear cloud image details, and gradually blurred prediction results, resulting in unsatisfactory final forecasting results. Therefore, it is still facing great challenges to achieve accurate cloud image extrapolation to provide reference for nowcasting. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a satellite cloud image nowcasting method based on the convolutional gated recurrent attention network CGAFNet. The convolutional attention module (Convolutional BlockAttentionModule, CBAM) is introduced to enhance the network's feature extraction capability for satellite cloud images and depict the detailed features of the cloud images. The present invention proposes primary and secondary loss (PaSLoss) as the loss function of the model to guide the model to learn more valuable features, thereby improving the extrapolation ambiguity problem. The present invention is aimed at satellite cloud images with a time step of 15 minutes, and the forecast time is 1h15min, which solves the problem that the time step is too long and the clouds that disappear quickly cannot be captured. By improving the cloud image extrapolation model, the prediction accuracy of the model is improved.

[0005] Technical solution: A satellite cloud image nowcasting method based on a convolutional gated recurrent attention network CGAFNet of the present invention comprises the following steps:

[0006] Step 1: Extract the satellite cloud image sequence of the target area at the previous t moments;

[0007] Step 2: Build a convolutional gated recurrent attention network CGAFNet model and input the satellite cloud image sequence into the CGAFNet model;

[0008] Step 3: Extract the cloud features at each moment through the attention feature extraction module, and use the LeakyReLU activation function instead of the ReLU activation function;

[0009] Step 4: Use the convolutional gated recurrent unit network ConvGRU network to extract the temporal features of the cloud image;

[0010] Step 5: Use the main and auxiliary loss functions PaSLoss as the loss function of the CGAFNet model, set the evaluation index, and optimize the model;

[0011] Step 6: Through the encoder and decoder structure of the model, the cloud map sequence at the next t moments is output to achieve sequence prediction of the satellite cloud map of the target area.

[0012] Furthermore, in step 1, the satellite cloud image sequence of the target area at the previous t moments is a continuous time sequence X1, X2, X3, ..., X t , where X∈R C×H×W They represent the real image and extrapolated image of the satellite cloud image respectively, C is the number of channels, H is the height, and W is the width.

[0013] Furthermore, step 2 is specifically as follows: the encoder of the model includes an attention feature extraction module and a convolutional gated recurrent unit network ConvGRU network. The model inputs the satellite cloud image sequence of the previous t moments, first extracts the cloud image features of each moment through the attention feature extraction module, and then extracts the temporal features of the cloud image through the ConvGRU network. The decoder of the model improves the continuity between pixels through step-by-step decoding.

[0014] Furthermore, step 3 is specifically as follows: the attention feature extraction module introduces a convolutional attention module to focus on the key information of the image on the basis of using 3×3 convolution to extract features. The convolutional attention module includes a channel attention module and a spatial attention module, and then outputs through a LeakyReLU activation function. The LeakyReLU activation function is shown in the following formula:

[0015]

[0016] Among them, x is the input value, α is a very small positive constant, usually α=0.01.

[0017] Furthermore, step 4 is specifically as follows: the convolutional gated recurrent unit network ConvGRU network combines the neural network structure of the convolutional neural network and the recurrent neural network to capture time and space information.

[0018] Furthermore, in step 5: the use of the main and secondary loss functions PaSLoss as the loss function of the CGAFNet model is specifically to select the standard mean square error as the main loss function and the additional structural similarity as the supplementary loss function to guide the model to learn more useful features, wherein the standard mean square error formula is as follows:

[0019]

[0020] Among them, y i Represents the output value of the network, x i Represents the input value of the network;

[0021] S(p,r) is the structural similarity SSIM between the predicted image and the real image. The difference from the mean square error is that S(p,r) is more sensitive to local structural changes and details. Its definition is:

[0022]

[0023] Among them, p and r are the predicted value and the true value, μ p and μ r is the average of p and r, σ p and σ r is the variance of x and y, σ pris the covariance of p and r, c1 and c2 are constants, and to avoid the instability problem caused by 0, the loss function of structural similarity is defined as L s , optimize the network with the goal of minimizing the objective function PaSLoss:

[0024] PaSLoss=L2+βL s =L2+β(1-S(p,r))

[0025] Where L2 represents the standard mean square error loss, and β is the control of L in the training process s Hyperparameters that affect loss.

[0026] Furthermore, in step 5, the evaluation index is set specifically as follows: the root mean square error RMSE, structural similarity SSIM and peak signal-to-noise ratio PSNR of each frame of the extrapolated image are selected as indicators; the calculation formula of RMSE is shown in the following formula:

[0027]

[0028] The calculation formula of PSNR is as follows:

[0029]

[0030] Among them, MAX I It refers to the maximum value of the pixels in the image. The maximum value is 255 when the bit number is 8. MSE stands for mean square error, which is used to measure the difference between the model prediction value and the true value. The calculation formula is as follows:

[0031]

[0032] Among them, y i Represents the output value of the network, x i Represents the input value of the network.

[0033] Furthermore, step 6 is as follows: the decoder improves the continuity between pixels by decoding in steps. First, the high-level features of the encoder are used as the input of ConvGRU, and then the low-resolution feature map is upsampled to the high-resolution feature map through deconvolution operation; secondly, the output features of the second layer ConvGRU of the encoder are fused to perform the same operation to gradually restore the original image size; then the intermediate features of the first layer ConvGRU of the encoder are fused to strengthen the model's learning of features at all levels; finally, the image is restored to an output result of 5×256×256×1 through two convolutional layers and activation layers, i.e., the satellite cloud image at the last t consecutive moments.

[0034] Furthermore, the first t moments are the first five moments, and the last t moments are the last five moments.

[0035] The present invention also discloses a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0036] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0037] The present invention introduces a convolutional attention module to enhance the network's ability to extract features from satellite cloud images, and depicts the detailed features of the cloud images. The present invention proposes primary and secondary losses as the loss function of the model to guide the model to learn more valuable features, thereby improving the extrapolation ambiguity problem. By improving the cloud image extrapolation model, the prediction accuracy of the model is improved.

[0038] The results show that the root mean square error of CGAFNet in the cloud image extrapolation task is 10.00K, the structural similarity is 0.74, and the peak signal-to-noise ratio is 31.43. The model can accurately predict the formation and evolution of clouds, and is superior to other networks in all indicators, proving that this method can obtain more accurate prediction accuracy.

[0039] The model proposed in this paper performs well at different extrapolation times, with high structural similarity and peak signal-to-noise ratio. At different extrapolation times, the structural similarity of the CGAFNet prediction results is above 0.65, and the peak signal-to-noise ratio is greater than 30dB, which shows that the model performs well at each moment of extrapolation and has good generalization ability and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the overall network structure diagram of CGAFNet of the present invention;

[0041] Figure 2 Schematic diagram of the attention feature extraction module;

[0042] Figure 3 It is a schematic diagram of the ConvGRU structure;

[0043] Figure 4 This is a comparison chart of the extrapolation effects of different models at different time lengths;

[0044] Figure 5 This is a comparison chart of different extrapolation schemes for the cloud cluster inception case from 13:45 to 16:00 on September 1, 2018;

[0045] Figure 6 This is a comparison chart of different extrapolation schemes for the cloud cluster merging case from 05:15 to 07:30 on July 8, 2018;

[0046] Figure 7 This is a comparison chart of different extrapolation schemes for the cloud splitting case from 13:30 to 15:45 on July 19, 2018. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0048] Satellite cloud image prediction is different from traditional time series tasks. While paying attention to the changes in time information, it is also necessary to pay attention to the changes in spatial characteristics. Therefore, satellite cloud image sequence prediction can be regarded as a spatiotemporal sequence prediction task. Its essence is to give a continuous time series X1,X2,X3,...,X t As input to the model, predict the future time series X t+1 ,X t+2 ,X t+3 ,...,X t+n , where X∈R C×H×W They represent the real image and extrapolated image of the satellite cloud image respectively, C is the number of channels, H is the height, and W is the width.

[0049] In order to achieve accurate cloud map extrapolation tasks, this paper proposes a convolutional gated recurrent attention fusion network (CGAFNet). The overall network structure of the model is as follows: Figure 1 As shown in the figure. First, the satellite cloud images of the first five moments are used as the input of the model, and the input is convolved to extract the feature information in the image. Considering that some information will be lost when extracting features through convolution, the channel attention module and the spatial attention module are introduced into the network to improve the network's feature extraction and detail capture capabilities; then, the LeakyReLU activation function is used instead of the ReLU (Rectified Linear Unit) activation function to alleviate the gradient explosion problem and make the model more flexible; then the ConvGRU network is used to extract the temporal features of the cloud image, and finally, through the encoder and decoder structure, the cloud image of the last five moments is output to achieve sequence prediction of the satellite cloud image.

[0050] The encoder consists of an attention feature extraction module and a spatiotemporal information extraction network. The model inputs the satellite cloud image sequence of the first five moments. The cloud image features of each moment are first extracted through the attention feature extraction module, and then the temporal features of the cloud image are extracted through the ConvGRU network. The three-layer network is constructed with the two as basic units to enhance and enrich the model's learning of feature semantic information.

[0051] Attention feature extraction module:

[0052] It is common that the network cannot capture the changes in the cloud map features of a small area or the image tends to be blurred over time. The attention feature extraction module of the present invention introduces a convolutional attention module to focus on the key information of the image based on the use of 3×3 convolution to extract features, and then outputs it through the LeakyReLU activation function. The schematic diagram is shown in Figure 2 As shown in the figure. The convolutional attention module consists of a channel attention module and a spatial attention module, which processes the input feature layer twice to achieve feature refinement. The channel attention module aims to analyze what feature information is more important in the image, reflect the importance of the channel, and enable the model to obtain the attention weights of different channels; the spatial attention module aims to focus on which position area information in the image is more important, capture the spatial correlation between pixel positions, and enable the model to adaptively learn the attention weights of different areas.

[0053] Spatiotemporal Information Extraction Network:

[0054] The spatiotemporal feature extraction of the present invention adopts the ConvGRU network. ConvGRU (Convolutional Gated Recurrent Unit Network) is a neural network structure that combines convolutional neural network and recurrent neural network. It has unique advantages in spatiotemporal data modeling and prediction. ConvGRU can effectively capture time and space information. In the field of meteorology, radar and satellite image data sets all represent information in two dimensions of time and space. Therefore, ConvGRU is used as the spatiotemporal feature extraction network. The structural diagram is shown in FIG. Figure 3 shown.

[0055] The decoder improves the continuity between pixels through step-by-step decoding. First, the high-level features of the encoder are used as the input of ConvGRU, and then the low-resolution feature map is upsampled to the high-resolution feature map through deconvolution operation. Secondly, the output features of the second layer ConvGRU of the encoder are fused and the same operation is performed to gradually restore the original image size. Then, the intermediate features of the first layer ConvGRU of the encoder are fused to strengthen the model's learning of features at all levels. Finally, the image is restored to an output result of 5×256×256×1 through two convolutional layers and activation layers, which is the satellite cloud image of the last five consecutive moments.

[0056] The loss function plays an important role in the training of deep learning networks. It affects the generalization ability of neural network models. The present invention adopts the form of a joint loss function to improve the defects of a single loss function. Since the standard mean square error (MSE) imposes severe penalties on larger errors and is relatively tolerant of smaller errors, it leads to excessive attention to large differences during training, thereby losing some detailed information. Therefore, the present invention proposes primary and secondary losses (PaSLoss) as the loss function of this model, selects the standard mean square error as the main loss function, and adds structural similarity as a supplementary loss function to guide the model to learn more useful features. The standard mean square error formula is as follows:

[0057]

[0058] S(p,r) is the structural similarity SSIM between the predicted image and the real image. The difference from the mean square error is that S(p,r) is more sensitive to local structural changes and details. Its definition is:

[0059]

[0060] Among them, p and r are the predicted value and the true value, μ p and μ r is the average of p and r, σ p and σ r is the variance of x and y, σ pr is the covariance of p and r, c1 and c2 are constants, and to avoid the instability problem caused by 0, the loss function of structural similarity is defined as L s , optimize the network with the goal of minimizing the objective function PaSLoss:

[0061] PaSLoss=L2+βL s =L2+β(1-S(p,r))

[0062] Where L2 represents the standard mean square error loss, and β is the control of L in the training process s The hyperparameter affected by the loss is set to β=0.25 according to the optimal experimental effect.

[0063] Example

[0064] The experimental data of the present invention uses FY-4A meteorological satellite data and AGRI-L1 level 12 channel (10.8μm) infrared brightness temperature data, which is helpful for the analysis of events such as heavy precipitation. The data set contains satellite cloud images from April to September 2018. The research area ranges from 23.0° to 33.2° north latitude and 105.6° to 115.8° east longitude, that is, the central provinces around Hunan with Hunan as the center. The spatial resolution is 0.04°×0.04°, and the time resolution is unified to 15min. The data preprocessing operations include data extraction, projection conversion, and radiation calibration. The resolution of the cloud map is 256×256. In order to reduce the amount of calculation, the brightness temperature value is normalized to 0-1 using the normalization method, and the missing time points in the data are statistically eliminated to form a total of 3890 groups of cloud map sequences with equal time resolution. Each group of cloud maps uses the images of the first five frames of input to predict the images of the last five frames. The extrapolation time is 1h15min. They are randomly and independently sampled in a ratio of 6:2:2 as training set, validation set, and test set, respectively.

[0065] Experimental environment and parameter settings: The hardware environment of the experiment uses Linux Ubuntu 20.04 system, CUDA 12.0 and NVIDIA GeForce A5000 graphics card. The software environment uses the Pytorch deep learning framework, the initial learning rate of the model is set to 0.001, the batch size is set to 4, Adam is used as the optimizer for training, and the maximum number of training iterations is set to 100. The experiment uses the automatic attenuation method of learning rate. During the training process, if the loss of the validation set still does not decrease after 5 rounds of training, the learning rate value is reduced to half of the original value, which helps the model obtain the optimal solution.

[0066] Evaluation indicators: In order to comprehensively evaluate the quality of the extrapolated image and the effectiveness of the model, the root mean square error (RMSE), structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) of each frame of the extrapolated image are selected as indicators:

[0067]

[0068] Among them, MAX I It refers to the maximum value of the pixels in the image, which is 255 when the bit number is 8.

[0069] Result analysis: In order to prove the effectiveness of the present invention, the model proposed in the present invention is compared with the generative adversarial network (GAN-CLSTM), the convolutional long short-term memory model (ConvLSTM) and the optical flow method (Optical Flow) in an independent test set, and the prediction results are evaluated using evaluation indicators. As shown in Table 1, the three evaluation indicators of the model of the present invention (CGAFNet) are better than those of other models, and all indicators have been improved, indicating that the satellite cloud images predicted by CGAFNet are closer to the true value, more accurately predict the movement trajectory and shape changes of the cloud clusters, and have the best prediction performance.

[0070] Table 1 Comparison of test results of different models

[0071]

[0072] Depend on Figure 4 It can be seen that as the extrapolation time increases, the extrapolation effect shows a downward trend. Among them, the model proposed in the present invention performs best at different extrapolation times, with the highest structural similarity and peak signal-to-noise ratio. At different extrapolation times, the structural similarity of the CGAFNet prediction results is above 0.65, and the peak signal-to-noise ratio is greater than 30dB, which shows that the model performs well at every moment of extrapolation.

[0073] In order to better verify the effect of the CGAFNet network, the present invention takes into account the possible temporal logical relationship between the sampled data, and conducts supplementary experiments on the basis of the above-mentioned independent sampling experiments. The supplementary experiment uses the 12-channel infrared brightness temperature data of the FY-4A meteorological satellite from April to September 2019, and adopts the same data preprocessing method to form a total of 3422 groups of cloud map sequences with equal temporal resolution, which are used as test sets for comparative experimental evaluation of the CGAFNet model. As can be seen from Table 2, the CGAFNet network proposed in the present invention shows significant advantages in cloud map forecasting. Its highest structural similarity and peak signal-to-noise ratio indicate that CGAFNet can not only effectively capture the detailed features of the cloud map, but also maintain the overall structural integrity of the image. The lowest root mean square error further confirms the advantages of the model in prediction accuracy. The above results show that CGAFNet has good generalization ability and practicality in cloud map extrapolation tasks.

[0074] Table 2 Comparison of test results of different models in supplementary experiments

[0075]

[0076]

[0077] The cloud image extrapolation model proposed in this invention can accurately predict the shape and development trend of clouds, effectively capture the changes of small-area clouds, have a good grasp of the overall details of the cloud image, and accurately predict the infrared cloud top brightness temperature value. The following three cases are analyzed, representing the typical cloud formation, cloud merger and cloud splitting processes, and there will be different degrees of enhancement or weakening of cloud brightness temperature intensity in the cloud image.

[0078] 1. Case analysis of the initial birth of cloud clusters

[0079] The onset of convection is a sign of the beginning of severe convective weather activity, and the infrared cloud top brightness temperature and its change rate are key indicators for the prediction of the onset of convection. Figure 5It can be seen that the cloud cluster at the junction of southwestern Hubei Province and Hunan Province has a continuous decrease in cloud top brightness temperature from inception to development, and CGAFNet can accurately predict its development morphology, trend and intensity. The GAN-CLSTM model can predict the changes in cloud clusters, but the prediction of cloud cluster morphology is inaccurate. The ConvLSTM model can also predict the changes in cloud clusters, but the prediction of the brightness temperature intensity of the cloud top of the incipient cloud cluster is inaccurate. However, the optical flow method has almost no ability to capture and predict the incipient small-area cold clouds. This model effectively captures the change process, which is of great reference significance for the study of the incipient convection, and is helpful in assisting forecasters to identify signs of the incipient convection. In addition, there is a process in which the cold cloud cluster at the junction of Guizhou Province and Hunan Province gradually develops and becomes larger in morphology, but its cloud top temperature gradually decreases. CGAFNet can well predict this change trend and has a good grasp of the overall morphology and detailed characteristics of the cloud cluster. The GAN-CLSTM and ConvLSTM models can also accurately predict this change trend, but the prediction of the detailed characteristics of the cloud cluster is not ideal. The optical flow method is relatively accurate in predicting the cloud map at the first moment, but as time goes by, its errors continue to accumulate, and the prediction errors for the subsequent cloud maps are large. The optical flow method only relies on calculating the optical flow of image changes, but the simple linear motion vector field generated by this may cause the change effect of each frame image to converge, or even cause image distortion and deformation.

[0080] 2. Case analysis of cloud group merger

[0081] The merger of cloud clusters, especially convective clouds, can promote the development of cloud bodies, produce more precipitation, and even cause severe convective weather. Figure 6It can be seen that in the southeastern part of Guizhou Province, two tiny clouds began to appear at 5:30, and the cloud top temperature was below zero degrees Celsius. After half an hour, the cloud in the west cooled down to below -65℃ at a faster rate, while the cloud in the east only cooled down and developed slightly. This process was used as the input of CGAFNet, and the results were: the cold cloud in the west continued to cool down and grow larger, and the cloud in the east also continued to cool down to below -65℃, and the area of ​​the cloud continued to increase. The two isolated cloud clusters merged at 7:15, and the model prediction results were highly consistent with the actual observation results. The model can predict the development process of cloud cluster merger, which is of great significance for the forecast and warning of severe convective weather. In the early stage of development, the two cloud clusters only occupied 2-3 pixels in the image, and CGAFNet can still effectively capture this change process and give correct prediction results, proving that the model has good processing and extraction capabilities for detail features. GAN-CLSTM and ConvLSTM can also predict the cloud merging process, but there is a large degree of deviation in the prediction of brightness temperature. In addition, from the overall point of view, there is a large area of ​​cold cloud system in Chongqing, and the local brightness temperature of cold clouds is slowly increasing (that is, the blue area gradually decreases), and the CGAFNet prediction result is accurate. However, the GAN-CLSTM and ConvLSTM models have serious ambiguity problems in the prediction of cold cloud system in Chongqing, and the detailed features of the cloud system are almost smoothed. The optical flow method performs the worst in this prediction, and fails to accurately grasp the movement trend, morphology and brightness temperature changes of the cloud clusters. The optical flow method uses the correlation between adjacent frames to calculate the optical flow vector field. If the motion difference between adjacent frames of the input cloud map is small, the overall change of the prediction result is not obvious.

[0082] 3. Case analysis of cloud splitting

[0083] Cloud splitting often indicates that severe convective weather is in the process of weakening, which often corresponds to the end of severe weather such as heavy precipitation and strong winds, making it a sign of turning weather. However, in actual events, severe weather may still occur after cloud splitting. Therefore, correctly predicting the decay process of cloud splitting can help improve the accuracy of forecasting future weather conditions. Figure 7It can be seen that a large area of ​​cold clouds is developing at the junction of Hunan Province and Jiangxi Province. A small area of ​​cold cloud clusters appears in the lower left corner of the cold cloud, and the temperature continues to drop below -65℃, showing a trend of cloud cluster splitting. CGAFNet can predict this trend well, but there is a certain lag. The cloud top brightness temperature of the small cloud cluster has dropped below -60℃ at 15:00, while the predicted value shows a significant drop in temperature in this area at 15:15. The ConvLSTM model does not accurately predict the development trend of cold cloud clusters at the junction of Hunan and Jiangxi. During the predicted time period, the cloud clusters first split and then merge, and first cool down and then warm up, which is inconsistent with the actual observation results. GAN-CLSTM and optical flow methods fail to capture the trend of cloud cluster splitting, but simply show the movement trend of cloud clusters, and have almost no forecasting ability for the development of small-scale cloud clusters. In addition, from the overall analysis of the cold cloud system at the junction of Guizhou Province and Hunan Province, the cloud top temperature continued to rise within the next two hours from 13:30, and there was a tendency to drift slowly to the eastern region. The model proposed in the present invention can accurately predict the dissipation process and movement trend of the cloud system, while the GAN-CLSTM and ConvLSTM models have poor prediction effects on it. In the third frame of the prediction (i.e. within half an hour), there is an obvious blurring effect, the detailed features of the entire cloud system are very unclear, and there are obvious deviations in the position, shape and intensity of the cold cloud. The prediction results of the optical flow method are quite different from the actual true value. The prediction results show that the overall cloud system drifts eastward and moves faster. Since the optical flow method only relies on the extrapolation of the motion vector field, it is easy to cause deformation of the cloud cluster.

[0084] In summary, the CGAFNet model proposed in the present invention has a good prediction effect on the inception, development and dissipation process of clouds, and can effectively predict the development process of cloud merging and cloud splitting. It is generally better than the other models mentioned above in the task of cloud map extrapolation. This model improves the fuzzy problems that often occur in nowcasting, enhances the ability to extract detailed features, improves the temporal resolution, and has a certain prediction ability for the birth and dissipation evolution process of small-scale clouds.

[0085] In order to realize the cloud image extrapolation task, this paper deeply analyzes the existing problems and proposes a convolutional gated recurrent attention fusion network (CGAFNet), which better extracts the spatiotemporal characteristics of satellite cloud images and realizes accurate prediction of satellite cloud images. Through experimental comparative analysis, the following conclusions are drawn:

[0086] (1) In order to solve the problem that the extrapolated cloud image time interval is too long, which makes it impossible to effectively predict the cloud clusters that appear and disappear in a short time, the present invention constructs a cloud image sequence dataset with a forecast time resolution of 15 minutes, and uses five frames of cloud images as input to predict the next five frames of cloud images, thus realizing the cloud image prediction for the next 1h15min. This model has the lowest root mean square error, the highest structural similarity and peak signal-to-noise ratio in independent tests, and can effectively predict the cloud clusters that appear and disappear rapidly, showing a high forecasting advantage.

[0087] (2) Blurring often occurs during the cloud map extrapolation process, especially as the extrapolation time increases, this problem becomes more obvious. To address this problem, the present invention uses the PaSLoss function to optimize the original loss, better maintains the overall structural information of the image, and guides the model to learn more valuable features. From the case analysis proposed by the present invention, it can be seen that the model effectively improves the blurring problem of the cloud map during extrapolation.

[0088] (3) The study of the formation and evolution of clouds has always been a difficult point in forecasting, especially for small-scale clouds. Models often find it difficult to accurately depict the detailed features of cloud images. To this end, this paper introduces an attention mechanism to enhance the network's ability to extract detailed features of images in order to mine richer spatiotemporal information. Experimental results show that the model has a good grasp of the detailed features of clouds and has a good forecasting ability for the development of small-scale clouds.

[0089] (4) Compared with other algorithms (including traditional algorithms), the extrapolation model of the present invention has a more ideal prediction effect on the inception, development and attenuation trend of clouds, as well as the intensity, location and morphology of clouds. The CGAFNet network is superior to other algorithms and can better extract the changing characteristics of infrared cloud images.

Claims

1. A satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet, characterized in that: The steps include: Step 1: Extract the satellite cloud image sequence of the target area at the previous t moments; Step 2: Build a convolutional gated recurrent attention network CGAFNet model and input the satellite cloud image sequence into the CGAFNet model; Step 3: Extract the cloud features at each moment through the attention feature extraction module, and use the LeakyReLU activation function instead of the ReLU activation function; Step 4: Use the convolutional gated recurrent unit network ConvGRU network to extract the temporal features of the cloud image; Step 5: Use the main and auxiliary loss functions PaSLoss as the loss function of the CGAFNet model, set the evaluation index, and optimize the model; Step 6: Through the encoder and decoder structure of the model, the cloud map sequence at the next t moments is output to achieve sequence prediction of the satellite cloud map of the target area.

2. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: In step 1, the satellite cloud image sequence of the target area at the previous t moments is a continuous time series X1, X2, X3, ..., X t , where X∈R C×H×W They represent the real image and extrapolated image of the satellite cloud image respectively, C is the number of channels, H is the height, and W is the width.

3. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: Step 2 is as follows: the encoder of the model includes an attention feature extraction module and a convolutional gated recurrent unit network ConvGRU network. The model inputs the satellite cloud image sequence of the previous t moments, first extracts the cloud image features of each moment through the attention feature extraction module, and then extracts the temporal features of the cloud image through the ConvGRU network. The decoder of the model improves the continuity between pixels through step-by-step decoding.

4. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: Step 3 is specifically as follows: the attention feature extraction module introduces a convolutional attention module to focus on the key information of the image based on the use of 3×3 convolution to extract features. The convolutional attention module includes a channel attention module and a spatial attention module, and then outputs through the LeakyReLU activation function. The LeakyReLU activation function is shown in the following formula: Among them, x is the input value, α is a very small positive constant, and α=0.01 is taken.

5. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: Step 4 is specifically as follows: the convolutional gated recurrent unit network ConvGRU network combines the neural network structure of the convolutional neural network and the recurrent neural network to capture time and space information.

6. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: In step 5: the main and secondary loss functions PaSLoss are used as the loss functions of the CGAFNet model. Specifically, the standard mean square error is selected as the main loss function, and the additional structural similarity is used as the supplementary loss function to guide the model to learn more useful features. The standard mean square error formula is as follows: Among them, y i Represents the output value of the network, x i Represents the input value of the network; S(p,r) is the structural similarity SSIM between the predicted image and the real image. The difference from the mean square error is that S(p,r) is more sensitive to local structural changes and details. Its definition is: Among them, p and r are the predicted value and the true value, μ p and μ r is the average of p and r, σ p and σ r is the variance of x and y, σ pr is the covariance of p and r, c1 and c2 are constants, and to avoid the instability problem caused by 0, the loss function of structural similarity is defined as L s , optimize the network with the goal of minimizing the objective function PaSLoss: PaSLoss=L2+βL s =L2+β(1-S(p,r)) Where L2 represents the standard mean square error loss, and β is the control of L in the training process s Hyperparameters that affect loss.

7. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: In step 5, the evaluation index is specifically set as follows: the root mean square error RMSE, structural similarity SSIM and peak signal-to-noise ratio PSNR of each frame of the extrapolated image are selected as indicators; the calculation formula of RMSE is shown in the following formula: The calculation formula of PSNR is as follows: Among them, MAX I It refers to the maximum value of the pixels in the image. The maximum value is 255 when the bit number is 8. MSE stands for mean square error, which is used to measure the difference between the model prediction value and the true value. The calculation formula is as follows: Among them, y i Represents the output value of the network, x i Represents the input value of the network.

8. According to claim 1, a satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet is characterized in that: Step 6 is as follows: the decoder improves the continuity between pixels by decoding in steps. First, the high-level features of the encoder are used as the input of ConvGRU, and then the low-resolution feature map is upsampled to the high-resolution feature map through deconvolution operation. Secondly, the output features of the second layer ConvGRU of the encoder are fused to perform the same operation to gradually restore the original image size. Then, the intermediate features of the first layer ConvGRU of the encoder are fused to strengthen the model's learning of features at all levels. Finally, the image is restored to an output result of 5×256×256×1 through two convolutional layers and activation layers, which is the satellite cloud image of the last t consecutive moments.

9. The satellite cloud image nowcasting method based on convolutional gated recurrent attention network CGAFNet according to claim 1, characterized in that: The first t moments are the first five moments, and the last t moments are the last five moments.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

Images