3D Cumulus Very Short-Term Spatiotemporal Prediction Model Structure, System and Model Training Method

Abstract

The present invention provides a three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure, system and model training method. Taking the three-dimensional cumulus sequence as the research object, a three-dimensional spatio-temporal prediction network based on 3dCLSTM is constructed based on deep learning, realizing the ultra-short-term spatio-temporal prediction of three-dimensional cumulus. On this basis, a cumulus instantaneous motion calculation unit WindGRU considering wind speed and direction is designed and embedded between the 3dCLSTM network layers, realizing the consideration of the influence of wind speed and direction on the prediction result in the ultra-short-term spatio-temporal prediction of cumulus, and strengthening the spatio-temporal motion modeling ability of the main network. The present invention can combine the three-dimensional cumulus data set and the wind vector data set to predict the motion and deformation of cumulus affected by wind speed and direction in the ultra-short-term time.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional spatio-temporal prediction, and particularly to a three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure, system and model training method. Background Art

[0002] As an important clean energy, solar energy is gradually becoming one of the important directions for renewable energy development. However, photovoltaic power generation has strong randomness and volatility. Especially in a clear sky environment, the appearance of low-altitude cumulus clouds will instantaneously block the sun and form a shadow area with obvious edges on the ground surface, resulting in a reduction in solar irradiance in this area, and the photovoltaic power generation level can drop to 30% of that in a cloudless clear sky. Therefore, predicting the deformation and movement trend of cumulus clouds is the premise and basis for realizing ultra-short-term prediction of solar irradiance, and is of great significance for evaluating photovoltaic power fluctuations and ensuring the safe and stable operation of the power grid.

[0003] In order to achieve ultra-short-term prediction of cumulus clouds, the ground-based cloud images with high spatio-temporal resolution are usually taken as the research object, and the methods based on spatio-temporal statistics or artificial intelligence are used to predict the movement of clouds. The traditional spatio-temporal statistics-based method detects and extracts characteristic cloud pixels from several consecutive ground-based cloud images, determines the movement trajectory by calculating the movement speed and direction of the cloud cluster, and then predicts the position of the cloud at a future moment. This kind of method realizes the movement detection of cloud pixels in the ground-based cloud images, but the prediction result does not describe the deformation and generation / elimination characteristics of the clouds. In recent years, image prediction methods based on artificial intelligence such as Convolutional Neural Networks (CNN), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM) have been introduced into the ultra-short-term prediction research of cloud cluster sequences, and the prediction accuracy has been improved to a certain extent. However, these methods lack the ability to process high-dimensional data and can only obtain limited spatio-temporal information, especially the cumulus cloud density information that is difficult to express in images, which directly affects the intensity of solar irradiance reaching the ground surface.

[0004] In addition, the movement and deformation of low-altitude clouds represented by cumulus clouds will be affected by meteorological factors such as wind speed and wind direction. Existing research has tried to adopt the method of combined modeling prediction using ground-based cloud images and numerical weather prediction models. However, the numerical weather prediction model has higher accuracy for medium and ultra-short-term prediction after 4 hours and later, and has limited optimization for the results of ultra-short-term prediction. Currently, the spatio-temporal prediction method for cumulus clouds mainly obtains a movement trend information applicable to all moments in this time series by learning the change characteristics of each time series in the training set, and transmits it in the network to predict the state of cumulus clouds at a future moment. However, the wind speed and wind direction may strengthen or change this trend, thereby affecting the accuracy of the prediction result.

[0005] Therefore, if the spatio-temporal prediction based on ground cloud images is extended to the spatio-temporal prediction based on a three-dimensional cumulus model, it can effectively capture the motion and deformation information of three-dimensional cumulus in multiple consecutive time series, and then explore the spatio-temporal four-dimensional characteristics with three-dimensional space and time dimensions. At the same time, introducing wind speed and direction information into the prediction model and driving the model to learn the cumulus motion characteristics driven by wind can strengthen or correct the motion trend to improve the accuracy of spatio-temporal prediction. Summary of the Invention

[0006] To overcome the deficiencies of the prior art, the present invention aims to propose a three-dimensional cumulus ultra-short-term spatio-temporal prediction method based on 3dCLSTM+WindGRU. Applying the method of the present invention to the time series of three-dimensional voxel cumulus can achieve the ultra-short-term spatio-temporal prediction of three-dimensional cumulus considering wind speed and direction. The technical solution of the present invention includes the following processes.

[0007] The structure of the three-dimensional cumulus ultra-short-term spatio-temporal prediction model designed by the present invention is specifically as follows:

[0008] The 3dCLSTM spatio-temporal prediction main network is a multi-layer recursive network structure based on 3d-ConvLSTM as the basic module. The entire network contains n groups of 6-layer 3D ConvLSTM vertical structures (n≥4), and realizes the state transfer in time and space along the arrow direction. Among them, n represents the n future moments predicted by the model. If n = 6, it means predicting the three-dimensional cumulus states at future moments t+1 to t+6 from the current moment t; if n = 10, it means predicting the three-dimensional cumulus states at future moments t+1 to t+10 from the current moment t. As Figure 1 shown, there are 6 layers in the vertical direction, namely Layer1, Layer2, Layer3, Layer4, Layer5, and Layer6, constituting a group of 3d-ConvLSTM; at this time, n = 6, that is, there are 6 groups of 3d-ConvLSTM arranged in the horizontal direction, indicating predicting the three-dimensional cumulus states at future moments t+1 to t+6 from the current moment t. The entire model forms a recursive spatio-temporal prediction network framework through the arrows in the horizontal and vertical directions.

[0009] The 3d-ConvLSTM unit is a convolutional unit based on ConvLSTM that can input and output high-order tensors. This unit accepts the four-dimensional tensor X t at the current moment t as the input, and the outputs are the cell state and the hidden state where, here B represents the batch size, C represents the number of input channels, C outIndicates the number of output channels. D, H, and W represent the depth, height, and width of the three-dimensional space respectively, all of which are 64.

[0010] In constructing each group of 3d-ConvLSTM, downsampling with max pooling is introduced between Layer1 and Layer2, and between Layer2 and Layer3 for dimensionality reduction. Upsampling with trilinear interpolation is introduced between Layer4 and Layer5, and between Layer5 and Layer6 to restore to the original data size. The true value at the current time t is input through the 3d-ConvLSTM of Layer1, and features are extracted layer by layer along Layer1 to Layer6 through convolution. Finally, the predicted value at the next time t+1 is output from the 3d-ConvLSTM unit of Layer6, and this value is passed into Layer1 at time t+1 as the new input of the 3d-ConvLSTM unit of this layer. Among them, X t Represents the true value input at time t, Represents the predicted value at the next time predicted from time t. At time points from t to t+(n / 2), the input data is randomly assigned true value X t Or predicted value The long-term and short-term memories are passed along the arrow direction. The horizontal arrows represent the transfer of neuron states to the next time in the time dimension And hidden states The vertical arrows represent that in the spatial dimension, the memory state output by the lower layer (l) Is passed to the higher layer (l+1), and the memory state output by the 3d-ConvLSTM unit of the top layer (l = 6) Will be passed to the next time as the input of the bottom layer (l = 1), thus enhancing the dependence on the top layer information of the previous time and forming a recursive spatio-temporal memory stream.

[0011] Furthermore, construct the WindGRU unit. The unit structure diagram is as Figure 2 Shown, and the derivation expression is as follows:

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018] Among them, Represents a motion filter, which is the combination of voxel instantaneous motion momentum and trend momentum ; is the voxel instantaneous motion momentum, which is jointly determined by the wind vector and the voxel motion vector ; is the trend momentum, which is continuously updated according to the trend momentum before time t. Realize the registration with the original hidden layer through the Warp operation in the three-dimensional grid, and output the gate g t which is jointly determined by and , and output

[0019] Embed WindGRU between the layers of the 3dCLSTM network, that is, embed WindGRU units between the Layer1 and Layer2, Layer2 and Layer3, Layer3 and Layer4, Layer4 and Layer5, Layer5 and Layer6 at each moment respectively to form a 3dCLSTM+WindGRU network. The framework structure of the l-th layer of the 3dCLSTM+WindGRU network at time t+1 is as Figure 3 shown, and the derivation expression is as follows:

[0020]

[0021]

[0022]

[0023] where l∈(1,2,…,6), the tensors represent the convective cloud instantaneous motion state and convective cloud motion trend momentum corrected by the wind vector respectively, and are transmitted through the WindGRU unit; are the hidden state and storage state at the previous moment respectively; o t represents the output gate of the 3d-ConvLSTM basic module.

[0024] Construct a three-dimensional convective cloud time series dataset based on the ground-based cloud map. According to the time points and spatial ranges of the three-dimensional convective cloud time series dataset, establish a matching wind vector dataset;

[0025] Step a1, obtain the ground-based cloud map, and downsample the ground-based cloud map to obtain an image with a resolution of 64×64.

[0026] Step a2: Perform distortion correction, strong light spot removal, and image correction operations on the downsampled image to extract the cumulus cloud area.

[0027] Step a3: Based on the geographical location and imaging time of the ground-based imaging device, combine the image pixel values of the cumulus cloud area to calculate the cloud base height and cloud cluster thickness as three-dimensional cumulus cloud modeling parameters for voxel three-dimensional cumulus cloud modeling. Each voxel is assigned a cloud particle density as voxel attribute information. The spatial range of the three-dimensional cumulus cloud is 64×64×64.

[0028] Step a4: Organize the three-dimensional voxel cumulus cloud data into a three-dimensional voxel cumulus cloud time series dataset to form a 6D tensor (S, L, C, D, H, W) as the input of the spatio-temporal prediction model. Among them, S represents the sequence length, L represents the number of sequences, C represents the number of channels, and D, H, W represent the depth, height, and width of the three-dimensional space respectively. The sequence length represented by S is consistent with the sequence length of the 3dCLSTM+WindGRU model in the horizontal direction, that is, S = n, where n is the number of future moments predicted by this model.

[0029] Step a5: Obtain the time range T and spatial range R of the cumulus cloud dataset, expand the spatial range, denoted as R*, and extract the wind speed and wind direction information matching the spatio-temporal range T and R* from the derived motion winds (DMW) product.

[0030] Step a6: Obtain the wind speed and wind direction distribution in the entire R* range through Kriging spatial interpolation.

[0031] Step a7: Extract the wind speed and wind direction values within the region R from the calculation results in the R* range, denoted as Wind=(WindSp, WindDir), where WindSp represents the wind speed and WindDir represents the wind direction in the horizontal direction. Make a wind vector dataset matching the cumulus cloud dataset, and the dataset is organized into a 4D tensor (S, L, WindSp_S, WindDir_S) for storage. Among them, S represents the sequence length, L represents the number of sequences, WindSp_S represents the sequence length of the wind speed value, and WindDir_S represents the sequence length of the wind direction value. The sequence length represented by S is consistent with the sequence length of the 3dCLSTM+WindGRU model in the horizontal direction, that is, S = n, where n is the number of future moments predicted by this model.

[0032] After completing the above operations, further perform model training and data testing:

[0033] Step c1: Divide the three-dimensional cumulus dataset and the wind vector dataset into a training set and a test set according to a ratio of 7:3 respectively. Input the three-dimensional cumulus training set into the 3dCLSTM in the form of a 6D tensor, and input the wind vector training set into the WindGRU unit in the form of a 4D tensor to prepare for network training.

[0034] Step c2: Adopt the planned sampling mode for training, and the training process stops after 300 epochs. All experiments are carried out using 4 GPUs on Nvidia Tesla V100 16GB to obtain a trained model.

[0035] Step c3: Input the three-dimensional cumulus test set into the 3dCLSTM in the form of a 6D tensor, and input the wind vector test set into the WindGRU unit in the form of a 4D tensor to obtain the results of the three-dimensional cumulus very short-term spatio-temporal prediction.

[0036] Furthermore, in step a5, the spatial range R is defined as the rectangular area composed of A(lon min ,lat min ), B(lon max ,lat min ), C(lon max ,lat max ), D(lon min ,lat max ). Among them, lon min represents the minimum longitude, lon max represents the maximum longitude, lat min represents the minimum latitude, and lat max represents the maximum latitude. The expanded spatial range R * is defined as composed of A * (lon min -r,lat min -r), B * (lon max +r,lat min -r), C * (lon max +r,lat max +r), D * (lon min -r,lat max +r), where r represents the expanded longitude and latitude degrees.

[0037] The advantages of the present invention are as follows: A three-dimensional cumulus ultra-short-term spatio-temporal prediction model based on 3dCLTSM+WindGRU is constructed. This model is a four-dimensional spatio-temporal prediction network based on deep learning, which can consider the three-dimensional spatial characteristics of cumulus in spatio-temporal prediction and further consider the influence of wind speed and direction on the movement trend of three-dimensional cumulus, making the prediction results more conform to the movement laws in the objective world, thereby improving the accuracy of three-dimensional cumulus ultra-short-term spatio-temporal prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the 3dCLSTM framework diagram of the embodiment of the present invention.

[0039] Figure 2 It is the structural diagram of the WindGRU unit of the embodiment of the present invention.

[0040] Figure 3 It is the architecture diagram of 3dCLSTM+WindGRU of the embodiment of the present invention.

[0041] Figure 4 It is the flowchart of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] To make the objectives, technical solutions and advantages of the present invention clearer, the following will further describe the real-time implementation manner of the present invention in detail with reference to the drawings.

[0043] The embodiment adopts the method of the present invention, constructs a three-dimensional voxel cumulus dataset and wind vector data using OpenCV and Python, and constructs an ultra-short-term spatio-temporal prediction framework based on pytorch to realize the ultra-short-term spatio-temporal prediction of three-dimensional cumulus.

[0044] A three-dimensional cumulus ultra-short-term spatio-temporal prediction method based on 3dCLSTM+WindGRU provided by the embodiment of the present invention includes the following processes:

[0045] Initialize the OpenCV and Python environments, and establish a three-dimensional cumulus time series dataset based on the ground-based cloud images. According to the time points and spatial ranges of the three-dimensional cumulus time series dataset, establish a matching wind vector dataset.

[0046] Step a1, obtain the ground-based cloud images, and downsample the ground-based cloud images to obtain images with a resolution of 64×64. Perform distortion correction, strong light point removal, and image correction operations on the downsampled images to extract the cumulus regions. According to the geographical location and imaging time of the ground-based imaging device, combine the image pixel values of the cumulus regions to calculate the cloud base height and cloud mass thickness as three-dimensional cumulus modeling parameters for voxel three-dimensional cumulus modeling, and assign cloud particle density to each voxel as voxel attribute information. Compose it into a three-dimensional voxel cumulus time series dataset according to the time sequence, and store the dataset as a 6D tensor.

[0047] Step a2: Obtain the time range T and spatial range R of the cumulus dataset, expand the spatial range, denoted as R*, and extract the wind speed and direction information that matches the spatio-temporal range T and R* from the derived motion winds (DMW) product. Furthermore, obtain the wind speed and direction distribution over the entire R* range through Kriging spatial interpolation, and extract the wind speed and direction values within the range R. Create a wind vector dataset that matches the cumulus dataset, and store the dataset as a 4D tensor.

[0048] Step a3: Construct 3d-ConvLSTM cells based on ConvLSTM, stack 3d-ConvLSTM as the basic module to extract features layer by layer, form a recursive network through arrows in the horizontal and vertical directions, and introduce max-pooling downsampling between layers for dimensionality reduction, and then restore to the original data size through trilinear interpolation upsampling, thus constructing a three-dimensional cumulus ultra-short-term spatio-temporal prediction network architecture 3dCLSTM.

[0049] Step a4: Construct WindGRU cells, and embed the WindGRU cells between the layers of the 3dCLSTM network to form a 3dCLSTM+WindGRU network.

[0050] Step a5: Input the three-dimensional cumulus data training set and the matching wind vector training set into the constructed three-dimensional cumulus ultra-short-term spatio-temporal prediction network based on 3dCLSTM+WindGRU for training to obtain a trained model.

[0051] Step a6: Input the test set into the trained model for testing to obtain the results of three-dimensional cumulus ultra-short-term spatio-temporal prediction.

[0052] In specific implementation, it can be realized in a computer software manner to automatically run the process, providing the above three-dimensional cumulus ultra-short-term spatio-temporal prediction method based on 3dCLSTM+WindGRU.

[0053] For the convenience of reference in implementation, the specific process design of the embodiment is provided as follows. See Figure 4 :

[0054] Step 1: Obtain ground-based cloud images and downsample them to a resolution of 64×64. Then, perform distortion correction, strong light point removal, and image correction operations to extract the cumulus cloud region. Based on the geographical location and imaging time of the ground-based imaging device, and combining the image pixel values of the cumulus cloud region, calculate the cloud base height and cloud mass thickness as three-dimensional cumulus cloud modeling parameters for voxel three-dimensional cumulus cloud modeling. Each voxel is assigned a cloud particle density as voxel attribute information. Organize the three-dimensional voxel cumulus cloud data into a three-dimensional voxel cumulus cloud time series dataset, and the dataset is stored as a 6D tensor (S, L, C, D, H, W). Among them, S represents the sequence length, L represents the number of sequences, C represents the number of channels, and D, H, W represent the depth, height, and width of the three-dimensional space respectively. The sequence length S represented is the same as the sequence length of the 3dCLSTM+WindGRU model in the horizontal direction, that is, S = n, where n is the number of future moments predicted by the model.

[0055] The specific implementation in the embodiment is as follows:

[0056] ① Obtain 3000 ground-based cloud images. The imaging time interval between every two ground-based cloud images is 10 minutes. Divide the 3000 ground-based cloud images into 500 groups, with each group containing 6 consecutive time series images. Downsample all the images to obtain images with a resolution of 64×64.

[0057] ② Perform distortion correction, strong light point removal, and image correction operations to extract the cumulus cloud region. Based on the geographical location and imaging time of the ground-based imaging device, and combining the image pixel values of the cumulus cloud region, calculate the cloud base height and cloud mass thickness as three-dimensional cumulus cloud modeling parameters for voxel three-dimensional cumulus cloud modeling with a spatial range of 64×64×64. Each voxel is assigned a cloud particle density as voxel attribute information.

[0058] ③ According to the sequence length of 6, the number of sequences of 500, the number of channels of 1, and the depth, height, and width of the three-dimensional space all being 64, organize the three-dimensional cumulus cloud dataset into a 6D tensor (6, 500, 1, 64, 64, 64) to store the data.

[0059] During specific implementation, the corresponding size of the tensor can be defined according to the actual situation of the dataset size to store the three-dimensional cumulus cloud dataset. The sequence length S is the same as the length of the subsequent constructed 3dCLSTM+WindGRU model in the horizontal direction.

[0060] Step 2: Obtain the time range \(T\) and spatial range \(R\) of the cumulus cloud dataset. Expand the spatial range, denoted as \(R^*\). Extract the wind speed and direction information that matches the spatio-temporal range \(T\) and \(R^*\) from the derived motion winds (DMW) product. Obtain the wind speed and direction distribution across the entire \(R^*\) range through Kriging spatial interpolation. Extract the wind speed and direction values within the region \(R\) from the calculation results within the \(R^*\) range, create a wind vector dataset that matches the cumulus cloud dataset, and store it using a 4D tensor \((S, L, WindSp_S, WindDir_S)\). Here, \(S\) represents the sequence length, \(L\) represents the number of sequences, \(WindSp_S\) represents the sequence length of the wind speed values, and \(WindDir_S\) represents the sequence length of the wind direction values.

[0061] The specific implementation in the embodiment is as follows:

[0062] The time range \(T\) of the cumulus cloud dataset is from 2019 to 2021, and the spatial range is from \(39.6417^{\circ}N\) to \(39.8384^{\circ}N\), \(105.0522^{\circ}W\) to \(105.3079^{\circ}W\). Expand the spatial range to \(38^{\circ}N\) to \(40^{\circ}N\), \(104^{\circ}W\) to \(106^{\circ}W\), denoted as \(R^*\). Extract the wind speed and direction information that matches the spatio-temporal range \(T\) and \(R^*\) from the derived motion winds (DMW) product. Obtain the wind speed and direction distribution across the entire \(R^*\) range through Kriging spatial interpolation. Extract the wind speed and direction values within the region \(R\) from the calculation results within the \(R^*\) range, create a wind vector dataset that matches the cumulus cloud dataset. According to the cumulus cloud dataset with a sequence length of 6 and a number of sequences of 500, form a 4D tensor \((6, 500, 6, 6)\) to store the wind vector.

[0063] During specific implementation, the wind speed and direction information corresponding to the corresponding time and range can be calculated according to the actual situation and stored as a 4D tensor of the corresponding size. The sequence length \(S\) is consistent with the length in the horizontal direction of the subsequent constructed 3dCLSTM + WindGRU model.

[0064] Step 3: In Pytorch, construct 3d-ConvLSTM cells based on ConvLSTM, stack 3d-ConvLSTM as the basic module to extract features layer by layer, and form a recursive spatio-temporal prediction network framework through arrows in the horizontal and vertical directions. Introduce downsampling with max pooling between layers for dimensionality reduction, and then restore to the original data size through upsampling with trilinear interpolation, thus constructing a three-dimensional cumulus cloud ultra-short-term spatio-temporal prediction network architecture 3dCLSTM.

[0065] The specific implementation in the embodiment is as follows:

[0066] ① First, construct a 3d-ConvLSTM cell based on ConvLSTM. This cell takes as input a four-dimensional tensor X at the current time step t and outputs the cell state C and the hidden state h at the current time step t. t Here, B represents the batch size, C represents the number of input channels, C' represents the number of output channels, and D, H, and W respectively represent the depth, height, and width of the three-dimensional space, all of which are 64. t and the hidden state where, Here B represents the batch size (batch size), C represents the number of input channels, C out represents the number of output channels, D, H, W respectively represent the depth, height, width of the three-dimensional space, all of which are 64.

[0067] ② Stack 3d-ConvLSTM as the basic module to extract features layer by layer. As shown in the figure, there are 6 layers in the vertical direction, namely Layer1, Layer2, Layer3, Layer4, Layer5, and Layer6, forming a group of 3d-ConvLSTM; there are 6 groups of 3d-ConvLSTM arranged in the horizontal direction, and a recursive spatio-temporal prediction network framework is formed by arrows in the horizontal and vertical directions. Figure 1 As shown, there are a total of 6 layers in the vertical direction, namely Layer1, Layer2, Layer3, Layer4, Layer5, and Layer6, which form a group of 3d-ConvLSTM; there are 6 groups of 3d-ConvLSTM arranged in the horizontal direction, and a recursive spatio-temporal prediction network framework is formed by arrows in the horizontal and vertical directions.

[0068] ③ Introduce max-pooling downsampling between Layer1 and Layer2, and between Layer2 and Layer3 for dimensionality reduction, and introduce trilinear interpolation upsampling between Layer4 and Layer5, and between Layer5 and Layer6 to restore to the original data size, thus forming a three-dimensional cumulus very short-term spatio-temporal prediction network architecture 3dCLSTM.

[0069] Specifically in implementation, the number of 3dCLSTM groups in the horizontal direction can be determined by sequences of different lengths according to the actual situation, forming a three-dimensional cumulus very short-term spatio-temporal prediction network architecture 3dCLSTM that conforms to the input of sequence information. In this embodiment, 6 groups of 3d-ConvLSTM in the horizontal direction are used as a relatively optimal solution for elaboration, and the technology of other combinations of numbers is similar to that of 6 groups of 3d-ConvLSTM.

[0070] Step 4, construct a WindGRU cell and embed it between the layers of the 3dCLSTM network to form a 3dCLSTM + WindGRU network.

[0071] The specific implementation in the embodiment is as follows:

[0072] ① Construct a WindGRU cell. The cell structure diagram is as shown, and the derivation expression is as follows: Figure 2 as shown, and the derivation expression is as follows:

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] Among them, represents a motion filter, which is a combination of the voxel instantaneous motion momentum and the trend momentum ; is the voxel instantaneous motion momentum, which is jointly determined by the wind vector and the voxel motion vector ; is the trend momentum, which is continuously updated according to the trend momentum before time t. Realize the registration with the original hidden layer through the Warp operation in the three-dimensional grid, and output the gate g t is jointly determined by and , and output

[0080] ② Embed WindGRU between the layers of the 3dCLSTM network, that is, embed WindGRU units between the Layer1 and Layer2, Layer2 and Layer3, Layer3 and Layer4, Layer4 and Layer5, Layer5 and Layer6 at each moment respectively to form a 3dCLSTM+WindGRU network. The framework structure of the l-th layer of the 3dCLSTM+WindGRU network at time t+1 is as Figure 3 shown, and the derivation expression is as follows:

[0081]

[0082]

[0083]

[0084] Among them, l∈(1,2,…,6), and the tensors respectively represent the instantaneous motion state of the cumulus corrected by the wind vector and the cumulus motion trend momentum, and are transmitted through the WindGRU unit; are the hidden state and storage state of the previous moment respectively; o t represents the output gate of the 3d-ConvLSTM basic module.

[0085] In specific implementation, the WindGRU unit can be embedded into the 3dCLSTM framework accordingly according to the actual situation.

[0086] Step 5: Input the training set into the constructed three-dimensional cumulus ultra-short-term spatio-temporal prediction network based on 3dCLSTM+WindGRU for training to obtain a trained model.

[0087] The specific implementation in the embodiment is as follows:

[0088] ① Divide the three-dimensional cumulus dataset and the wind vector dataset into a training set and a test set according to a ratio of 7:3 respectively. Input the three-dimensional cumulus training set into 3dCLSTM in the form of a 6D tensor, and input the wind vector training set into the WindGRU unit in the form of a 4D tensor to prepare for network training.

[0089] ② Hyperparameter settings of the network: The batch size is 4, the kernel size is 3, and the hidden size is 130.

[0090] ③ Adopt the mode of scheduled sampling for training, and the training process stops after 300 epochs. All experiments are carried out on 4 GPUs on Nvidia Tesla V100 16GB to obtain a trained model.

[0091] In specific implementation, the hyperparameters of the model can be set and the model can be trained according to the actual situation.

[0092] Step 6: Input the test set into the trained model for testing to obtain the results of three-dimensional cumulus ultra-short-term spatio-temporal prediction.

[0093] The specific implementation in the embodiment is as follows:

[0094] Input the three-dimensional cumulus test set into 3dCLSTM in the form of a 6D tensor, and input the wind vector test set into the WindGRU unit in the form of a 4D tensor to obtain the results of three-dimensional cumulus ultra-short-term spatio-temporal prediction.

[0095] In specific implementation, spatio-temporal prediction results of different sequence lengths can be obtained from three-dimensional cumulus ultra-short-term spatio-temporal prediction models with different sequence lengths according to the actual situation.

[0096] As can be seen from the above specific implementation, the three-dimensional cumulus ultra-short-term spatio-temporal prediction method based on 3dCLSTM+WindGRU can realize the learning and prediction of the motion and deformation trend characteristics of three-dimensional cumulus. Although the model is relatively complex, by constructing a WindGRU unit in the model to analyze the influence of wind speed and direction on the motion of cumulus, the spatio-temporal motion modeling ability of the prediction network is strengthened, thereby improving the accuracy of the ultra-short-term spatio-temporal prediction results of three-dimensional cumulus.

[0097] The specific embodiments described in this article are only illustrative of the spirit of the present invention. Those skilled in the art to which the present invention pertains can make various modifications or supplements to the described specific embodiments or use similar methods for substitution, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure, characterized in that: The spatiotemporal prediction network framework includes n 6-layer structures, that is, there are 6 layers in the vertical direction, forming a group of 3d-ConvLSTM units, and there are n groups of 3d-ConvLSTMs in parallel in the horizontal direction to form a 3dCLSTM network, n ≥ 4. The 3d-ConvLSTM unit is a convolution unit based on ConvLSTM that can input and output high-order tensors; the unit accepts the four-dimensional tensor X at the current time t t As input, the output is the cell state at the current time t and hidden state in, Here B represents the batch size, C represents the number of input channels, and C out Indicates the number of output channels, D, H, and W represent the depth, height, and width of the three-dimensional space respectively; In each group of 3D-ConvLSTM, downsampling with max pooling is introduced between layers for dimensionality reduction, and then upsampling by trilinear interpolation is used to restore the data to the original size; in the vertical direction, the ground truth X at the current time t is input through the 3D-ConvLSTM of the first layer in the first group. t Feature extraction is performed by convolutional layers from the first layer to the sixth layer, and finally the predicted value at the next time t+1 is output from the 3D-ConvLSTM unit of the sixth layer. This value is then passed into the first layer at time t+1. During the time points from t to t+2, the input data is the randomly assigned ground truth X t or the predicted value. Between the 3D-ConvLSTM units in the horizontal direction, the neuron state at the previous time is passed between the corresponding layers. and the hidden state. A WindGRU unit is constructed and embedded between two adjacent 3d-ConvLSTM units in the vertical direction of a 3dCLSTM network to form a 3dCLSTM+WindGRU network structure. The derivation expression of the WindGRU unit is as follows: Among them, represents a motion filter, which is the combination of voxel instantaneous motion momentum and trend momentum ; is the voxel instantaneous motion momentum, which is jointly determined by the wind vector and the voxel motion vector ; is the trend momentum, which is continuously updated according to the trend momentum before time t; Realize the registration with the original hidden layer in the three-dimensional grid through the Warp operation, and output the gate g t is jointly determined by H t ' and and output The derivation expression of the 3dCLSTM+WindGRU network at the l-th layer at time t+1 is as follows: where \(l\in(1,2,\cdots,L)\), the tensors respectively represent the instantaneous cumulus motion state and the cumulus motion trend momentum after wind vector correction, and are transmitted through the WindGRU unit; are the hidden state and the storage state at the previous moment respectively; \(o\) t represents the output gate of the 3d-ConvLSTM basic module.

2. The three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to claim 1, characterized in that: The depth, height, and width of the three-dimensional space are all 64.

3. The three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to claim 1, wherein: The process of constructing the dataset for this model structure is as follows: Step a1: Obtain a ground-based cloud image and perform downsampling on the ground-based cloud image to obtain an image with a resolution of 64×64. Step a2: Perform distortion correction, strong light point removal, and image correction operations on the downsampled image to extract the cumulus region. Step a3: According to the geographical location and imaging time of the ground-based imaging device, combine the pixel values of the cumulus region image to calculate the cloud base height and cloud cluster thickness as three-dimensional cumulus modeling parameters for voxel three-dimensional cumulus modeling. Each voxel is assigned a cloud particle density as voxel attribute information. The spatial range of the three-dimensional cumulus is 64×64×64. Step a4: Organize the three-dimensional voxel cumulus data into a three-dimensional voxel cumulus time series dataset to form a 6D tensor (S, L, C, D, H, W) as the input of the spatio-temporal prediction model; where S represents the sequence length, L represents the number of sequences, C represents the number of channels, and D, H, W respectively represent the depth, height, and width of the three-dimensional space. Step a5: Obtain the time range T and spatial range R of the cumulus dataset, expand the spatial range, denoted as R*, and extract the wind speed and wind direction information matching the spatio-temporal range T and R* from the derived motion wind products. Step a6: Obtain the wind speed and wind direction distribution in the entire R* range through Kriging spatial interpolation. Step a7: Extract the wind speed and wind direction values within the region R from the calculation results in the R* range, denoted as Wind=(WindSp, WindDir), where WindSp represents the wind speed and WindDir represents the wind direction in the horizontal direction; make a wind vector dataset matching the cumulus dataset, and the dataset is organized into a 4D tensor (S, L, WindSp_S, WindDir_S) for storage; where S represents the sequence length, L represents the number of sequences, WindSp_S represents the sequence length of the wind speed value, and WindDir_S represents the sequence length of the wind direction value.

4. The three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to claim 1, characterized in that: In the first to third layers, downsampling with max pooling is introduced between layers for dimensionality reduction; in the fourth to sixth layers, upsampling with trilinear interpolation is introduced between layers to restore to the original data size.

5. The three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to claim 3, characterized in that: The spatial range represented by R is a rectangular area composed of A(lon min , lat min ), B(lon max , lat min ), C(lon max , lat max ), D(lon min , lat max ); where lon min represents the minimum longitude, lon max represents the maximum longitude, lat min represents the minimum latitude, and lat max represents the maximum latitude; The spatial expansion range represented by R* is from A * (lon min -r, lat min -r), B * (lon max +r, lat min -r), C * (lon max +r, lat max +r), D * (lon min -r, lat max +r) to form a rectangular area, where r represents the degrees of longitude and latitude for expansion.

6. A training method for the three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to any one of claims 1-5, characterized in that: ① Divide the three-dimensional cumulus data set and the wind vector data set into a training set and a test set according to a ratio of 7:3 respectively. Input the three-dimensional cumulus training set into the 3dCLSTM in the form of a 6D tensor, and input the wind vector training set into the WindGRU unit in the form of a 4D tensor to prepare for network training; ② Hyperparameter settings of the network: the batch size is 4, the kernel size is 3, and the hidden state is 130; ③ Adopt the mode of scheduled sampling for training, and stop the training process after 300 epochs. All experiments are carried out using 4 GPUs on Nvidia Tesla V100 16GB, and a trained model is obtained.

7. The three-dimensional cumulus ultra-short-term spatio-temporal prediction model training method according to claim 6, characterized in that: When training the model, obtain 3000 ground-based cloud images, with an imaging time interval of 10 minutes between every two ground-based cloud images. Divide the 3000 ground-based cloud images into 500 groups, each group containing 6 consecutive time-series images, and downsample all the images to obtain images with a resolution of 64×64.

8. An electronic device, characterized in that, It includes: One or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the functions of the three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to any one of claims 1-5.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the functions of the three-dimensional cumulus ultra-short-term spatio-temporal prediction model structure according to any one of claims 1-5.

Images