Space-time convolutional neural network lightning forecasting method based on double-source observation data

Abstract

The invention provides a time-space convolutional neural network lightning forecasting method based on double-source observation data. The method comprises the following steps: constructing a radar prediction network, constructing a double-source prediction network based on a convolutional neural network, preparing a data set, training the radar prediction network, training the double-source prediction network, performing double-source data acquisition and processing, and predicting future lightning by using the double-source prediction network. According to the method, the internal relation of the radar feature data and the lightning data in the space-time dimension is fully integrated, the defect that the traditional image channel layer direct fusion mode is poor in characterization capability on a complex meteorological scene is effectively overcome, and therefore a solid data basis is provided for more accurate meteorological prediction. Moreover, the time sequence convolutional neural network is adopted for lightning prediction, and the number of model parameters is effectively reduced while the time sequence data processing capability is maintained, so that the overhead of training and reasoning is remarkably reduced, and the meteorological condition can be predicted and evaluated more quickly and accurately.

Description

Technical Field

[0001] The present invention relates to the field of weather forecasting, and more particularly, to a spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data. Background Art

[0002] Lightning, as a common discharge phenomenon in nature, originates from the layered structure of the net charge inside the cloud. During the process of mutual collision and rebound of ice-phase substances of different scales in the cloud, opposite-polarity charges are carried. These charged particles are separated under the action of gravity and air flow, forming positive and negative charge regions. When the electric field strength between the positive and negative charge regions reaches the air breakdown threshold, a discharge phenomenon in the atmosphere occurs, namely lightning. Lightning causes extensive harm to human society. It threatens human life safety, can cause outdoor personnel to be struck by lightning and injured or killed, and may also trigger secondary disasters such as building fires and explosions. In terms of social resources, it has an obvious impact on power and communication systems, causing line short circuits, equipment damage, and signal and data transmission failures. In the agricultural field, it will damage crops and agricultural facilities; in the aviation industry, it may cause the aircraft's on-board equipment and flight control systems to malfunction, endangering the lives of personnel. China is located in temperate and subtropical regions, and thunderstorm activities are very frequent. The annual maximum number of thunderstorm days in 21 provincial capital cities across the country exceeds 50 days, and the maximum reaches 134 days. In some local areas of Yunnan, Hainan, Guangdong, Guangxi, and Tibet, it exceeds 90 days (References 1-3). Farmland, open areas, etc. have become the main disaster-bearing environments for lightning-caused casualties. In terms of the current scientific and technological level, humans do not yet have the ability to directly intervene in the formation of lightning. However, if lightning weather can be accurately and quickly forecasted, sufficient time can be gained for disaster prevention and mitigation. Relevant departments can immediately draw up emergency plans, orderly carry out emergency preparations, shut down equipment at risk, and protect key facilities from damage; ordinary people can also terminate non-essential outdoor activities in advance and actively evacuate to safe areas, thereby significantly reducing the threat of lightning disasters to personal safety and the damage to property. In summary, achieving accurate lightning forecasting has extremely crucial value and far-reaching significance.

[0003] Existing lightning forecasting methods mainly focus on the following aspects: extrapolation methods based on observation data, numerical weather prediction systems based on physical models, and methods based on machine learning.

[0004] Both of the two early methods, namely the numerical weather prediction system based on observational data extrapolation and the physical model, have their own characteristics and limitations. The method based on observational data extrapolation is based on the assumption of the continuation of the change law of some characteristics of the object target (Reference 4). However, for lightning forecasting, only using real-time lightning monitoring data to extrapolate the lightning activity in the future, the forecasting effect is not good due to the randomness and discreteness of lightning activity. During systematic thunderstorm processes, by using continuous evolution characteristics such as radar reflectivity, echo top height, and vertical liquid water content, the correlation between them and lightning activity can be statistically analyzed for extrapolation forecasting, and the warning effect has been improved to a certain extent. However, the extrapolation time range is limited. Usually, the warning results are relatively accurate within the next 30 minutes. For long-term forecasting, the forecasting performance is seriously affected by error accumulation. The numerical weather prediction system based on physical models predicts the weather by using numerical calculations to solve the equations according to atmospheric physics knowledge under the set initial and boundary conditions, such as the WRF model in the United States (References 5-6) and the CMA-MESO mesoscale weather numerical prediction system in China. However, in lightning prediction, due to the complex atmospheric electrical and microphysical mechanisms of lightning generation, it is difficult to directly model. It is necessary to build a correlation model or parameterization scheme related to lightning by using its output elements. For example, Sousa et al. (Reference 7) established a lightning forecasting diagnostic formula and applied it to the WRF model to obtain lightning forecasting products in the Portuguese region. The PR92 parameterization scheme proposed by Price and Rind et al. (Reference 8) gives the relationship formula between lightning frequency and thunderstorm cloud top height for different types of thunderstorms. However, these methods rely on the understanding of the electrical processes of thunderstorms for prediction, and the forecasting performance of the models is different under different thunderstorm types, and a large number of adjustments need to be made for different situations.

[0005] Lightning forecasting methods based on machine learning have gradually emerged with the development of technology. Deep learning models have the ability to extract features and model complex non-linear scenarios, with obvious advantages compared to traditional methods. For example, Sebastian Brodeh et al. proposed an end-to-end lightning prediction method based on CNN, which directly predicts lightning events using geostationary satellite images (Reference 9); LightNet by Geng et al. (Reference 10) is based on the ConvLSTM module, combines simulation and observational data, and improves long-term prediction by fusing features through a dual encoder. However, most of these deep learning-based methods are constructed based on recurrent neural networks and their variants, and there are problems of large training and inference burdens.

[0006] Generally speaking, the existing lightning forecasting methods have the following deficiencies:

[0007] (1) The extrapolation method based on observational data mainly relies on historical data. However, due to the rapid change characteristics of the thunderstorm system, its forecasting accuracy for long-term requirements is significantly limited, and the forecasting effect is poor in the initial and extinction stages of lightning.

[0008] (2) The numerical model prediction system based on physical models predicts future meteorological changes by simulating the atmospheric state. There are still deviations in the specific details of the prediction in the spatial and temporal domains. The prediction performance fundamentally depends on the microphysical scheme adopted, and more complex and accurate microphysical schemes are needed to improve the prediction ability;

[0009] (3) The existing lightning warning methods based on machine learning perform better than traditional methods. However, due to the complexity of the network architecture and special design requirements, there are often problems such as high consumption of computing resources, long inference and training times, and increased computational burden with the improvement of spatio-temporal resolution, and they still cannot fully meet the actual needs of efficient warning in the industry.

[0010] References:

[0011] [1] Wang Juan, Chen Yun. Analysis of the lightning distribution characteristics in China from 2009 to 2012 [J]. Meteorology, 2015, 41(2): 160 - 170.

[0012] [2] Pang Bo, Zhu Biao, Lai Huihuang, et al. Statistical test of station thunderstorm days based on lightning monitoring data and its evolution characteristics research [J]. Plateau Meteorology, 2023, 42(6).

[0013] [3] Chen Xiaoxin, Yu Xiaoding, Wang Xiuming. Research on large-scale derecho events in China: Spatiotemporal distribution, environmental background and morphological characteristics of convective systems [J]. Acta Meteorologica Sinica, 2022, 80(1): 67 - 81.

[0014] [4] Dixon M, Wiener G. TITAN: Thunderstorm identification, tracking, analysis, and nowcasting - A radar-based methodology [J]. Journal of Atmospheric and Oceanic Technology, 1993, 10(6): 785 - 797.

[0015] [5] Skamarock W C, Klemp J B, Dudhia J. Prototypes for the WRF (Weather Research and Forecasting) model [C] / / Preprints, Ninth Conf. Mesoscale Processes, J11–J15, Amer. Meteorol. Soc., Fort Lauderdale, FL. 2001, 1.

[0016] [6]Xu L, Zhang Y, Wang F, et al. Coupling of electrification and discharge processes with WRF model and its preliminary verification[J]. Chinese J. Atmos. Sci, 2012, 36: 1041 - 1052.

[0017] [7]Sousa J F, Fragoso M, Mendes S, et al. Statistical–dynamical modeling of the cloud-to-ground lightning activity in Portugal[J]. Atmospheric Research, 2013, 132: 46 - 64.

[0018] [8Price C, Rind D. A simple lightning parameterization for calculating global lightning distributions[J]. Journal of Geophysical Research: Atmospheres, 1992, 97(D9): 9919 - 9933.

[0019] [9]Shi X, Chen Z, Wang H, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting[J]. Advances in Neural Information Processing Systems, 2015, 28.

[0020]

[10] Geng Y,Li Q,Lin T,et al.Lightnet:A dual spatiotemporal encodernetwork model for lightning prediction[C] / / Proceedings of the 25th ACMSIGKDDInternational Conference on Knowledge Discovery&Data Mining.2019:2439-2447. Summary of the Invention

[0021] The present invention provides a spatiotemporal convolutional neural network lightning prediction method based on dual-source observation data to solve the technical problems existing in the above-mentioned prior art.

[0022] To achieve the above object, the present invention provides a spatiotemporal convolutional neural network lightning prediction method based on dual-source observation data, which includes:

[0023] S1: Construct a radar prediction network,

[0024] The radar prediction network sequentially includes a first encoder, a first time feature extraction module, and a first decoder,

[0025] S2: Construct a dual-source prediction network based on a convolutional neural network

[0026] The dual-source prediction network includes a second encoder, a second time feature extraction module, a dual-source feature fusion layer, and a second decoder connected in sequence,

[0027] S3: Prepare a data set

[0028] The data set includes a radar training data set and a dual-source data training data set. The radar training data set is radar feature data collected by a radar. The dual-source data training data set includes radar feature data collected by a radar and lightning data collected by a lightning locator in the same space-time. The number of frames and the size of the image frames included in each sample in the radar training data set and the dual-source data training data set are equal,

[0029] S4: Train the radar prediction network

[0030] Input the radar training data set into the radar prediction network, calculate the loss function MSE, and use the stochastic gradient descent method to optimize the value of the loss function MSE to train the radar prediction network to obtain a trained radar prediction network,

[0031] The calculation formula of the loss function MSE is as follows:

[0032]

[0033] Q is the total number of training samples in the radar training dataset, y i represents the true target value of the i-th training sample, represents the predicted output of the i-th training sample,

[0034] S5: Train the dual-source prediction network

[0035] Input the radar feature data in the dual-source data training dataset into the radar prediction network, input the lightning data in the dual-source data training dataset into the dual-source prediction network. The first time feature extraction module of the radar prediction network outputs the first feature map, and the second time feature extraction module of the dual-source prediction network outputs the second feature map. The first feature map and the second feature map are fused in the dual-source feature fusion layer and then input into the second decoder.

[0036] Calculate the loss function BCEWithLogitsLoss, and use the stochastic gradient descent method to optimize the loss function BCEWithLogitsLoss to train the dual-source prediction network, obtaining the trained dual-source prediction network.

[0037] The calculation formula of the BCEWithLogitsLoss function value is as follows:

[0038]

[0039] where T is the total number of samples in the dual-source data training dataset, x j is the probability value predicted for the j-th sample, z j is the true label of the j-th sample, and pos_weight is the proportion coefficient of positive examples.

[0040] S6: Conduct dual-source data acquisition and processing

[0041] Collect dual-source data samples and process them so that the number of frames and the size of the image frames contained in the dual-source data samples are the same as each training sample in the radar training dataset. The dual-source data samples are radar feature data and lightning data collected in the same space-time.

[0042] S7: Use the dual-source prediction network to predict future lightning

[0043] Input the dual-source data samples obtained in step S6 into the dual-source prediction network for future lightning prediction.

[0044] In one embodiment of the present invention, the first encoder is formed by sequentially connecting N convolutional blocks in series. Each convolutional block of the first encoder includes a convolutional layer, a normalization layer, and an activation function. The first time feature extraction module is formed by sequentially connecting M feature processing blocks. Each feature processing block includes a first branch, a second branch, and a channel adjustment convolutional layer. The first branch is formed by sequentially connecting three convolutional layers. The second branch is composed of a 1x1 convolutional layer, an average pooling layer, a fully connected layer, and a 1x1 convolutional layer. The input of the channel adjustment convolutional layer in each feature processing block is connected to the output of the first branch and the output of the second branch therein. The first decoder includes P1 convolutional blocks connected in series and a readout layer. Each convolutional block of the first decoder includes a convolutional layer, a normalization layer, and an activation function.

[0045] In one embodiment of the present invention, the second encoder and the second time feature extraction module are the same as the first encoder and the first time feature extraction module respectively. The second decoder includes P2 convolutional blocks connected in series and a readout layer. Each convolutional block of the second decoder includes a convolutional layer, a normalization layer, and an activation function.

[0046] In one embodiment of the present invention, N is 4, M is 8, P1 is 4, P2 is 4, and pos_weight is 20.

[0047] In one embodiment of the present invention, among the 4 convolutional blocks of the first encoder, the output feature map channels of each convolutional block are 16, the convolutional block size is 3×3, the stride of the first and third convolutional blocks is 1, and the stride of the second and fourth convolutional blocks is 2.

[0048] In one embodiment of the present invention, in the first feature processing block of the first time feature extraction module, the convolutional kernel of the first convolutional layer in the first branch is 1×1, the stride is 1, the output feature map channel is 160, the convolutional kernel of the second convolutional layer in the first branch is 5x5, the stride is 1, the padding is 2, the convolutional kernel of the third convolutional layer in the first branch is 7x7, the stride is 1, the padding is 9, and the dilation coefficient is 3;

[0049] The convolutional kernels of the two 1x1 convolutional layers in the second branch of the first feature processing block are both 1×1, the stride is 1, the input channel of the channel adjustment convolutional layer in the first feature processing block is 160, the output channel is 32, the convolutional kernel is 1×1, and the stride is 1;

[0050] In the 2nd to 7th feature processing blocks of the first-time feature extraction module, the convolution kernels of the first convolutional layer in the first branch are all 1×1, the stride is 1, the output feature map channels are all 32, the convolution kernel size of the second convolutional layer in the first branch is 5x5, the stride is 1 pixel, the padding is 2, the convolution kernel size of the third convolutional layer in the first branch is 7x7, the stride is 1, the padding is 9, and the dilation coefficient is 3. The convolution kernels of the two 1x1 convolutional layers in the second branch are all 1×1, the stride is 1, the input channels of the channel adjustment convolutional layer are 32, the output channels are 32, the convolution kernel is 1×1, and the stride is 1;

[0051] In the channel adjustment convolutional layer of the 8th feature processing block in the first-time feature extraction module, the input channels are 32, the output channels are 160, the convolution kernel is 1×1, and the stride is 1.

[0052] In an embodiment of the present invention, the input channel number of the convolutional layer in the 4 convolutional blocks of the first decoder is 8, the output channel number is 8, the convolution kernel size is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the first decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the first decoder is 2;

[0053] The input channel number of the readout layer of the first decoder is mapped from 8 to 1, the convolution kernel is 1×1, and the stride is 1.

[0054] In an embodiment of the present invention, the input channel number of the convolutional layer in the 4 convolutional blocks of the second decoder is 16, the output channel number is 16, the convolution kernel is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the second decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the second decoder is 2;

[0055] The input channel number of the readout layer of the second decoder is mapped from 16 to 1, the convolution kernel is 1×1, and the stride is 1.

[0056] In an embodiment of the present invention, the radar training dataset includes at least 50,000 samples and each sample contains 30 frames of 500×500 radar feature data. The first 10 frames of data in the radar training dataset are used as data input, the acquisition time of the first 10 frames of data in the radar training dataset is 60 minutes, and the last 20 frames in the radar training dataset are used as the true radar labels for predicting the next two hours.

[0057] The dual-source data training dataset includes at least 50,000 samples. Each sample contains 30 frames of dual-source data with a size of 500×500. The first 10 frames of data in the dual-source data training dataset are used as data input. The acquisition time of the first 10 frames of data in the dual-source data training dataset is 60 minutes. The last 20 frames are the true lightning labels to be predicted for the next two hours. The time resolution of the lightning data and radar feature data in the dual-source data training dataset is 6 minutes, and the spatial resolution is 5 km×5 km.

[0058] In an embodiment of the present invention, the dual-source data samples collected in step S6 are data with a time step of 6 minutes for 1 hour before the prediction time. The time period for predicting future lightning in step S7 is from the prediction time for the next 0-2 hours.

[0059] The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data provided by the present invention has the following advantages:

[0060] (1) It fully integrates the internal connection between radar feature data and lightning data in the spatio-temporal dimension, can make full use of the advantages of the two types of data to improve the forecasting effect, and effectively makes up for the defect that the traditional direct fusion method of the image channel layer has poor representation ability for complex meteorological scenes, thus providing a solid data foundation for more accurate meteorological prediction.

[0061] (2) It uses a temporal convolutional neural network for lightning prediction, replaces the traditional RNN network's cyclic input with a convolutional structure. While maintaining the ability to process time series data, it effectively reduces the number of model parameters and the computational complexity, thus significantly reducing the training and inference overhead, enabling it to forecast and evaluate meteorological conditions more quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0063] Figure 1 It is a flowchart of the spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to an embodiment of the present invention;

[0064] Figure 2 It is a schematic structural diagram of a radar prediction network according to an embodiment of the present invention;

[0065] Figure 3 It is a schematic diagram of training a dual-source prediction network according to an embodiment of the present invention;

[0066] Figure 4a Schematic diagram of real lightning data of the first embodiment of the present invention;

[0067] Figure 4b Schematic diagram of lightning prediction results of the first embodiment of the present invention;

[0068] Figure 5a Schematic diagram of real lightning data of the second embodiment of the present invention;

[0069] Figure 5b Schematic diagram of lightning prediction results of the second embodiment of the present invention;

[0070] Figure 6a Schematic diagram of real lightning data of the third embodiment of the present invention;

[0071] Figure 6b Schematic diagram of lightning prediction results of the third embodiment of the present invention;

[0072] Figure 7 Performance quantitative evaluation results of lightning prediction using the ConvLSTM model, PredRNN model, LightNet model, and the method of the present invention on the same test set respectively. Detailed implementation manners

[0073] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0074] First, the meanings of the terms involved in this case are explained.

[0075] Dual-source observation data: Dual-source observation data refers to radar feature data and lightning data in the same time and space.

[0076] Radar feature data can reflect information such as water vapor, cloud structure, and air flow movement in the atmosphere, and detects the position, speed, and characteristics of the target by emitting electromagnetic waves and receiving reflected waves, which helps to understand the formation, development, and movement of thunderstorm clouds. Lightning data records parameters such as the time, location, and intensity of lightning strikes, directly reflecting the characteristics of lightning activities. Combining these two types of data can more comprehensively and deeply master the atmospheric environment information related to lightning, providing a rich data basis for accurate lightning prediction.

[0077] Spatio-temporal convolutional neural network: It refers to a network that uses a convolutional neural network for spatio-temporal sequence prediction.

[0078] The spatio-temporal convolutional neural network extracts features by designing the convolutional kernel size, stride, and stacking convolutional layers, and constructs a time series by sliding the convolutional kernel. For example, dilated convolution can capture long-range dependencies by expanding the receptive field, overcoming the problems of gradient vanishing or explosion that are prone to occur when the RNN processes the sequence in order, as well as the difficulties in parallel processing and low efficiency. It mines temporal features in a structured manner.

[0079] The present invention mainly solves the following technical problems:

[0080] (1) The problem of the fusion method of radar and lightning dual-source data: There is a large correlation between the radar feature data and the occurrence of lightning. The radar can reflect the dynamic changes of cloud and atmospheric instability in real time through continuous spatio-temporal observations, and these factors are strongly correlated with the occurrence of lightning. The lightning observation data is discrete and discontinuous compared with the radar feature data, but the radar feature data provides continuous spatio-temporal information and can capture the changing trend of the future atmospheric environment more smoothly. Therefore, it has greater predictability than lightning. Due to the large differences in the distribution of radar feature data and lightning events in time and space, how to reasonably dock these two types of information in the model is a major problem faced by the prior art. The commonly used fusion method in the prior art is to perform fusion at the image channel layer, that is, directly stack the lightning and radar feature data in the channel dimension and input them into the network for convolution as different channels of the image. However, this method easily ignores the internal connections of the data in the spatial and temporal dimensions. The lightning and radar feature data often show their own changing laws at different spatio-temporal scales, and this fusion method cannot effectively integrate these spatio-temporal information, thus reducing the representation ability of complex meteorological scenarios. Therefore, the first key problem to be solved by the present invention is how to effectively fuse the radar feature data and lightning data to make full use of the advantages of both to improve the forecasting effect.

[0081] (2) The problem of optimizing the training and inference overhead using the temporal convolutional neural network: When traditional recurrent neural networks (RNNs) process temporal data, there are problems such as high computational complexity, long training time, and easy occurrence of gradient vanishing or gradient explosion, which makes the training and inference overhead large when dealing with large-scale data. Therefore, the second key problem to be solved by the present invention is how to effectively reduce the number of model parameters and computational complexity while maintaining the ability to process time series data, thereby significantly optimizing the training and inference overhead and enabling it to forecast and evaluate meteorological conditions more quickly and accurately.

[0082] Figure 1 is a flowchart of the spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to an embodiment of the present invention. As Figure 1 shown, the spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data provided by the present invention includes:

[0083] S1: Construct a radar prediction network,

[0084] The radar prediction network sequentially includes a first encoder, a first temporal feature extraction module, and a first decoder. As Figure 2 shown in the structural schematic diagram of the radar prediction network according to an embodiment of the present invention, the first encoder is sequentially connected in series by N convolutional blocks. Each convolutional block of the first encoder includes a convolutional layer, a normalization layer, and an activation function. The first temporal feature extraction module is sequentially connected by M feature processing blocks. Each feature processing block includes a first branch, a second branch, and a channel adjustment convolutional layer. The first branch is sequentially connected by three convolutional layers. The second branch consists of a 1x1 convolutional layer, an average pooling layer, a fully connected layer, and a 1x1 convolutional layer. The input of the channel adjustment convolutional layer in each feature processing block is connected to the output of the first branch and the output of the second branch therein. The first decoder includes P1 serially connected convolutional blocks and a readout layer. Each convolutional block of the first decoder includes a convolutional layer, a normalization layer, and an activation function.

[0085] S2: Construct a dual-source prediction network based on a convolutional neural network

[0086] The dual-source prediction network includes a second encoder, a second temporal feature extraction module, a dual-source feature fusion layer, and a second decoder connected in sequence.

[0087] The second encoder and the second temporal feature extraction module are the same as the first encoder and the first temporal feature extraction module respectively. The second decoder includes P2 serially connected convolutional blocks and a readout layer. Each convolutional block of the second decoder includes a convolutional layer, a normalization layer, and an activation function.

[0088] S3: Prepare a dataset

[0089] The dataset includes a radar training dataset and a dual-source data training dataset. The radar training dataset is radar feature data collected by a radar. The radar feature data contains multiple element features. The role of the radar prediction network is to decode a radar multi-element feature image containing multiple element features and use a real radar image to supervise the training to ensure that the radar features extracted by the first temporal feature extraction module in the radar prediction network are reasonable and effective. The dual-source data training dataset includes radar feature data collected by a radar and lightning data collected by a lightning locator in the same time and space. The number of frames and the size of the image frames included in each sample in the radar training dataset and the dual-source data training dataset are equal.

[0090] S4: Train the radar prediction network

[0091] Input the radar training dataset into the radar prediction network, calculate the loss function MSE, and use the stochastic gradient descent method to optimize the value of the loss function MSE to train the radar prediction network, obtaining a trained radar prediction network.

[0092] The calculation formula of the loss function MSE is as follows:

[0093]

[0094] Q is the total number of training samples in the radar training dataset, and y i represents the true target value of the i-th training sample, and represents the predicted output of the i-th training sample.

[0095] The first encoder in the radar prediction network is used to receive the radar training dataset, that is, the collected continuous multi-frame radar image sequence. The first encoder extracts spatial features from the continuous multi-frame radar image sequence, then learns the temporal dimension evolution law of the features through the first time feature extraction module, and finally generates the final radar prediction image sequence through the first decoder. The MSE loss is calculated between the radar prediction image sequence and the true radar image sequence for gradient backpropagation to update each parameter in the radar prediction network.

[0096] The dimension of the continuous multi-frame radar image sequence is represented as (T, C, H, W), where T represents the time dimension of the radar image, C represents different channels of the radar image, and H and W represent the width and height of the radar image in the spatial dimension respectively. The following operations are performed on the continuous multi-frame radar image sequence in each convolutional activation block: The convolutional layer (Conv2d layer) receives the input radar image sequence and performs a convolutional operation in the (H, W) dimension to obtain the first feature sequence. The first feature sequence is input to the normalization layer (GroupNorm layer) for normalization operation to obtain the second feature sequence. The second feature sequence undergoes a non-linear transformation through the activation function and then outputs the third feature sequence. The third feature sequence is used as the input for the next convolutional activation block, and the above operations are performed in a loop. The above process is represented by the following formula:

[0097] h s = σ(GroupNorm(Conv2d(h s-1 )))

[0098] where s = 2, 3... N, N represents the number of convolutional blocks in the first encoder, the Conv2d() function represents the operation in the convolutional layer, the GroupNorm() function represents the operation in the normalization layer, σ represents the activation function, and h s-1 and h s represent the feature sequences output by the (s - 1)-th and s-th layers respectively. The final output of the first encoder is the feature sequence h N .

[0099] It can be seen that in each convolutional block, the input sequence of consecutive multiple frames of radar images first undergoes a convolutional operation in the (H, W) dimension through a convolutional layer to extract relevant features in the spatial dimension. The feature sequence after the convolutional operation is input into a normalization layer for normalization, and the normalized feature sequence undergoes a non-linear transformation through an activation function, serving as the output of the convolutional block. This is the processing flow of one convolutional block. The entire first encoder is composed of N convolutional blocks connected in series in sequence. Each convolutional block receives the feature sequence output by the previous convolutional block as input. Each feature sequence represents the spatial features output by the corresponding convolutional block. Each convolutional block after the first convolutional block further extracts features in sequence to obtain higher-level spatial features. The finally obtained feature sequence is output by the activation function of the last convolutional block, that is, h N 。

[0100] h N is a multi-channel feature map containing spatial features, and h N is sent to the first temporal feature extraction module to learn the evolution law of features in the temporal dimension. In the first temporal feature extraction module, h N is processed through M feature processing blocks. Each feature processing block includes a first branch, a second branch, and a channel adjustment convolutional layer. In the first feature processing block, h N is respectively input into the first branch and the second branch for processing. Each subsequent feature processing block receives the output of the previous feature processing block and is respectively sent into the first branch and the second branch inside itself for processing. The final processing result of the first temporal feature extraction module is output by the channel adjustment convolutional layer of the last feature processing block.

[0101] The processing process of the first branch is as follows: The first branch is composed of three convolutional layers connected in series, and uses small kernel depth convolution, dilated depth convolution, and 1×1 convolution for convolution respectively to simulate large kernel convolution. The small kernel depth convolution extracts features for local pixels within each size window in the T×C dimension with a relatively small convolutional kernel size. The dilated depth convolution expands the receptive field of the convolutional kernel by introducing a dilation rate and extracts the transformation law of features in the H×W dimension on a wider scale. Finally, through a 1×1 convolution, the information of each channel extracted is integrated and transformed. Summarizing the above steps, it can be summarized into the following expression:

[0102] s d =Conv 1×1 (Conv DW-D Conv DW (h N ))

[0103] where s dThe intra-frame static time features extracted for the first branch, Conv DW is the small kernel depth convolution, DW is the convolution kernel of the small kernel depth convolution, Conv DW-D is the dilated depth convolution, DW-D is the convolution kernel of the dilated depth convolution, Conv 1×1 is the 1×1 convolution.

[0104] The processing process of the second branch is as follows: The second branch is a global attention extraction architecture in the TxC dimension for inter-frame dynamic attention extraction. The second branch no longer focuses on the information changes within the feature map, but performs attention-based feature extraction on the entire TxC dimension. Specifically, it is implemented by referring to the classic SE attention mechanism module in terms of structure design. For the input ((T×C),H,W) vector, a compression operation is performed on the feature map of each channel through an average pooling layer (AvgPool layer), and the feature map of each channel is compressed into a single value (i.e., global average pooling), representing the global information of each channel. The formula for this step is expressed as:

[0105]

[0106] where c indicates the feature map of the c-th channel, z c corresponds to the value after global average pooling. At this time, the feature dimension changes to ((T×C),1,1). This vector will sequentially pass through two 1x1 convolutions and two fully connected layers, and a ReLU function (an activation function) is used in the middle to introduce non-linearity. The formula for this step is expressed as follows:

[0107] s c = σ(ReLU(W2·Conv 1*1 (ReLU(W1·Conv 1*1 (z c )))))

[0108] where W and W2 respectively represent the two fully connected layers, σ represents the Sigmoid function, s cRepresent the weights of each channel in the final output. The main function of this stage is to map the input global information to a higher-dimensional space through the fully connected layer, so that it can learn more complex and rich feature representations. Through the first convolutional-fully connected layer, the radar prediction network can capture the non-linear relationships between features, and due to the introduction of the ReLU activation function, the radar prediction network can learn more sparse and discriminative features. Through the second convolutional-fully connected layer, the high-dimensional feature representation is mapped back to the original number of channels to correspond to the original feature map. Finally, the learned weights are normalized to between 0 and 1 through the Sigmoid function, and these weights will be used as the output of the inter-frame dynamic attention, and multiplied by the result extracted in the first part to adjust the channel weights as the result vector of the temporal attention extraction.

[0109] After that, the results output by the first branch and the second branch are connected with the feature sequence h N for skip connection as the output of the final temporal feature extraction module. The formula is as follows, where h N represents the vector output by the spatial encoding, s d is the intra-frame static temporal feature, s c is the inter-frame dynamic attention weight, and h out is the feature to be decoded extracted by the spatio-temporal module.

[0110]

[0111] Then, through the first decoder similar to the first encoder structure, upsampling and feature decoding are realized, and the predicted radar image sequence is output:

[0112]

[0113] Finally, calculate the mean square error loss MSE between the real radar image sequence o T and the predicted radar sequence Optimize the value of the loss function MSE using the stochastic gradient descent method to obtain the finally trained predicted radar image sequence.

[0114] The above is the pre-training process of the radar prediction network. The predicted radar sequence is only used as an intermediate result of the present invention. More importantly, the present invention is to obtain the feature map h out containing the future radar information features therein. After the radar prediction network is trained, all its parameters are saved and packaged as a radar feature extractor

[0115] S5: Train the dual-source prediction network

[0116] Input the radar feature data in the dual-source data training dataset into the radar prediction network, and input the lightning data in the dual-source data training dataset into the dual-source prediction network. The first time feature extraction module of the radar prediction network outputs the first feature map, and the second time feature extraction module of the dual-source prediction network outputs the second feature map. Fuse the first feature map and the second feature map in the dual-source feature fusion layer, and then input it into the second decoder, as Figure 3 shown in the schematic diagram of training the dual-source prediction network according to an embodiment of the present invention.

[0117] Fusing the first feature map and the second feature map in the dual-source feature fusion layer means performing feature splicing on the first feature map and the second feature map in the channel dimension, and then inputting it into the second decoder, and finally decoding it into a predicted lightning sequence. This predicted sequence also calculates the binary cross-entropy loss with the lightning data in the dual-source data training dataset and propagates the gradient to update the parameters. The above process is detailed as calculating the loss function BCEWithLogitsLoss, using the stochastic gradient descent method to optimize the loss function BCEWithLogitsLoss to train the dual-source prediction network, and obtaining the trained dual-source prediction network,

[0118] The calculation formula of the BCEWithLogitsLoss function value is as follows:

[0119]

[0120] where T is the total number of samples in the dual-source data training dataset, x j is the predicted probability value of the j-th sample, z j is the true label of the j-th sample, and pos_weight is the proportion coefficient of positive examples,

[0121] S6: Perform dual-source data acquisition and processing

[0122] Collect dual-source data samples and process them so that the number of frames and the size of the image frames contained in the dual-source data samples are the same as each training sample in the radar training dataset. The dual-source data samples are radar feature data and lightning data collected in the same space-time,

[0123] S7: Use the dual-source prediction network to predict future lightning

[0124] Input the dual-source data samples obtained in step S6 into the dual-source prediction network for future lightning prediction.

[0125] In one embodiment of the present invention, N is 4 (the first encoder is formed by sequentially connecting 4 convolutional blocks in series), M is 8 (the first temporal feature extraction module includes 8 feature processing blocks), P1 is 4 (the first decoder includes 4 convolutional blocks connected in series and a readout layer), P2 is 4 (the second decoder includes 4 convolutional blocks connected in series and a readout layer), and pos_weight is 20 (the proportion coefficient of positive examples is 20).

[0126] In one embodiment of the present invention, among the 4 convolutional blocks of the first encoder, the output feature map channels of each convolutional block are 16, the convolutional block size is 3×3 (the convolutional block size is set to 3×3 pixels, the same below), the stride of the first and third convolutional blocks is 1 (the stride of the first and third convolutional blocks is set to 1 pixel, the same below), and the stride of the second and fourth convolutional blocks is 2.

[0127] In one embodiment of the present invention, in the first feature processing block of the first temporal feature extraction module, the convolution kernel of the first convolutional layer in the first branch is 1×1, the stride is 1, and the output feature map channels are 160 (the output feature map channels are set to 160, the same below). The convolution kernel of the second convolutional layer in the first branch is 5x5, the stride is 1, and the padding is 2 (the padding is set to 2 pixels, the same below). The convolution kernel of the third convolutional layer in the first branch is 7x7, the stride is 1, the padding is 9, and the dilation coefficient is 3;

[0128] The convolution kernels of the two 1x1 convolutional layers in the second branch of the first feature processing block are both 1×1, the stride is 1. The input channels of the channel adjustment convolutional layer in the first feature processing block are 160, the output channels are 32, the convolution kernel is 1×1, and the stride is 1;

[0129] In the 2nd to 7th feature processing blocks of the first temporal feature extraction module, the convolution kernels of the first convolutional layer in the first branch are all 1×1, the strides are all 1, and the output feature map channels are all 32. The convolution kernel sizes of the second convolutional layer in the first branch are all 5x5, the strides are all 1 pixel, the padding is all 2, the convolution kernel sizes of the third convolutional layer in the first branch are all 7x7, the strides are all 1, the padding is all 9, and the dilation coefficients are all 3. The convolution kernels of the two 1x1 convolutional layers in the second branch are both 1×1, the stride is 1. The input channels of the channel adjustment convolutional layer are 32, the output channels are 32, the convolution kernel is 1×1 pixel, and the stride is 1;

[0130] In the channel adjustment convolutional layer of the 8th feature processing block of the first temporal feature extraction module, the input channels are 32, the output channels are 160, the convolution kernel is 1×1, and the stride is 1.

[0131] In an embodiment of the present invention, the number of input channels of the convolutional layers in the 4 convolutional blocks of the first decoder is 8, the number of output channels is 8, the convolutional kernel size is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the first decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the first decoder is 2;

[0132] The number of input channels of the readout layer of the first decoder is mapped from 8 to 1, the convolutional kernel is 1×1, and the stride is 1.

[0133] As Figure 2 shown is a schematic structural diagram of the radar prediction network according to an embodiment of the present invention. Based on the various parameters in the above-mentioned first encoder, first time feature extraction module, and first decoder,

[0134] In an embodiment of the present invention, the number of input channels of the convolutional layers in the 4 convolutional blocks of the second decoder is 16, the number of output channels is 16, the convolutional kernel is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the second decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the second decoder is 2;

[0135] The number of input channels of the readout layer of the second decoder is mapped from 16 to 1, the convolutional kernel is 1×1, and the stride is 1.

[0136] In an embodiment of the present invention, the radar training data set includes at least 50,000 samples, and each sample contains 30 frames of radar feature data of 500×500. The first 10 frames of data in the radar training data set are used as data input, the acquisition time of the first 10 frames of data in the radar training data set is 60 minutes, and the last 20 frames in the radar training data set are used as the true radar labels for predicting the next two hours,

[0137] The dual-source data training data set includes at least 50,000 samples, each sample contains 30 frames of dual-source data of 500×500. The first 10 frames of data in the dual-source data training data set are used as data input, the acquisition time of the first 10 frames of data in the dual-source data training data set is 60 minutes, the last 20 frames are the true lightning labels for predicting the next two hours, and the time resolution of the lightning data and the radar feature data in the dual-source data training data set is 6 minutes, and the spatial resolution is 5km×5km.

[0138] In an embodiment of the present invention, the dual-source data samples collected in step S6 are data with a time step of 6 minutes for 1 hour before the prediction time, and the prediction period for future lightning in step S7 is from the prediction time for the next 0-2 hours.

[0139] A specific experimental process of lightning forecasting using the spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data provided by the present invention is as follows:

[0140] 1. Simulation conditions:

[0141] The hardware test platform for the simulation experiment: The CPU is an Intel(R) Xeon(R) Gold 6240 CPU @ 2.60 GHz, the memory is 192 GB, the GPU is an NVIDIA RTX 3090, and the software platform is Ubuntu 18.04.

[0142] The training sample set and the test sample set come from the radar feature data and lightning data during the flood seasons from 2019 to 2023 provided by the Yunnan Meteorological Disaster Prevention and Mitigation Technology Center. The data acquisition range is the radar data and lightning data across the whole territory of Yunnan Province. The acquisition duration of each data sample is 3 hours. The first 1 hour is used as data input, and the real radar / lightning labels for the next 2 hours to be predicted are used for supervised training. After data processing, the training set contains a total of 98,732 sample pairs, and the test sample set contains a total of 24,683 sample pairs.

[0143] 2. Simulation content and result analysis

[0144] Using the method of the present invention, on the dual-source data set of Yunnan Province processed, the constructed radar prediction network and the dual-source prediction network are trained respectively. The radar feature map is obtained by using the trained prediction network, and the future lightning occurrence situation is predicted in combination with the trained dual-source prediction network.

[0145] Figures 4a to 6b For the comparison between the schematic diagram of the real lightning data and the schematic diagram of the lightning prediction result of the present invention, Figure 4a 、 Figure 5a 、 Figure 6a are the schematic diagrams of the real lightning data. The horizontal and vertical coordinates respectively represent the longitude and latitude of the lightning distribution. A red dot represents one occurrence of real lightning. Figure 4b 、 Figure 5b 、 Figure 6b are the schematic diagrams of the lightning prediction results for the corresponding time periods ( Figure 4b the upper left, upper right, lower left, and lower right figures of Figure 4a correspond to the upper left, upper right, lower left, and lower right figures of Figures 4a to 6b respectively, and the same below). The color represents the predicted lightning occurrence probability (0, 100]. From light blue to dark red indicates that the lightning occurrence probability gradually increases. From Figure 7The following are the quantitative evaluation results of the prediction performance of the ConvLSTM model, PredRNN model, LightNet model, and the method of the present invention on the same test set. It can be seen that the ETS score of the method of the present invention is 0.14794, which is improved compared with the ConvLSTM model, PredRNN model, and LightNet model. The evaluation result is the best, thus verifying the effectiveness of the present invention.

[0146] The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data provided by the present invention has the following advantages:

[0147] (1) It fully integrates the internal connection between radar feature data and lightning data in the spatio-temporal dimension, can make full use of the advantages of the two types of data to improve the forecasting effect, and effectively makes up for the defect that the traditional direct fusion method of image channel layers has poor representation ability for complex meteorological scenarios, thus providing a solid data basis for more accurate meteorological prediction.

[0148] (2) It uses a temporal convolutional neural network for lightning prediction, replaces the traditional RNN network's cyclic input with a convolutional structure, while maintaining the ability to process time series data, effectively reducing the number of model parameters and the computational complexity, thus significantly reducing the training and inference overhead, enabling it to forecast and evaluate meteorological conditions more quickly and accurately.

[0149] Those of ordinary skill in the art can understand that: The drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0150] Those of ordinary skill in the art can understand that: The modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be correspondingly changed to be located in one or more devices different from the present embodiment. The modules of the above embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0151] Finally, it should be noted that: The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: They can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data, characterized in that, Including: S1: Construct a radar prediction network The radar prediction network sequentially includes a first encoder, a first temporal feature extraction module, and a first decoder. S2: Construct a dual-source prediction network based on a convolutional neural network The dual-source prediction network includes a second encoder, a second temporal feature extraction module, a dual-source feature fusion layer, and a second decoder connected in sequence. S3: Prepare a dataset The dataset includes a radar training dataset and a dual-source data training dataset. The radar training dataset is radar feature data collected by a radar. The dual-source data training dataset includes radar feature data collected by a radar and lightning data collected by a lightning locator in the same time and space. The number of frames and the size of the image frames in each sample of the radar training dataset and the dual-source data training dataset are equal. S4: Train the radar prediction network Input the radar training dataset into the radar prediction network, calculate the loss function MSE, and use the stochastic gradient descent method to optimize the value of the loss function MSE to train the radar prediction network, obtaining a trained radar prediction network. The calculation formula of the loss function MSE is as follows: Q is the total number of training samples in the radar training dataset, y i represents the true target value of the i-th training sample, represents the predicted output of the i-th training sample, S5: Train the dual-source prediction network Input the radar feature data in the dual-source data training dataset into the radar prediction network, and input the lightning data in the dual-source data training dataset into the dual-source prediction network. Output the first feature map by the first temporal feature extraction module of the radar prediction network, and output the second feature map by the second temporal feature extraction module of the dual-source prediction network. Fuse the first feature map and the second feature map in the dual-source feature fusion layer, and then input it into the second decoder. Calculate the loss function BCEWithLogitsLoss, and use the stochastic gradient descent method to optimize the loss function BCEWithLogitsLoss to train the dual-source prediction network, obtaining a trained dual-source prediction network. The calculation formula of the value of the BCEWithLogitsLoss loss function is as follows: where T is the total number of samples in the dual-source data training dataset, x j is the predicted probability value of the j-th sample, z j is the true label of the j-th sample, and pos_weight is the proportion coefficient of positive examples S6: Perform dual-source data collection and processing Collect dual-source data samples and process them so that the size of the image frames in the dual-source data samples is the same as that of each training sample in the radar training dataset. The dual-source data samples are radar feature data and lightning data collected in the same time and space before the moment to be predicted. S7: Use the dual-source prediction network to predict future lightning Input the dual-source data samples obtained in step S6 into the dual-source prediction network for future lightning prediction.

2. The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 1, wherein, The first encoder is formed by sequentially connecting N convolutional blocks in series. Each convolutional block of the first encoder includes a convolutional layer, a normalization layer, and an activation function. The first temporal feature extraction module is formed by sequentially connecting M feature processing blocks. Each feature processing block includes a first branch, a second branch, and a channel adjustment convolutional layer. The first branch is formed by sequentially connecting three convolutional layers. The second branch is composed of a 1x1 convolutional layer, an average pooling layer, a fully connected layer, and a 1x1 convolutional layer. The input of the channel adjustment convolutional layer in each feature processing block is connected to the output of the first branch and the output of the second branch therein. The first decoder includes P1 series-connected convolutional blocks and a readout layer. Each convolutional block of the first decoder includes a convolutional layer, a normalization layer, and an activation function.

3. The spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 2, wherein, The second encoder and the second temporal feature extraction module are the same as the first encoder and the first temporal feature extraction module respectively. The second decoder includes P2 series-connected convolutional blocks and a readout layer. Each convolutional block of the second decoder includes a convolutional layer, a normalization layer, and an activation function.

4. The spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 3, wherein, N is 4, M is 8, P1 is 4, P2 is 4, and pos_weight is 20.

5. The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 4, characterized in that Among the 4 convolutional blocks of the first encoder, the output feature map channels of each convolutional block are 16, the convolutional block size is 3×3, the stride of the first and third convolutional blocks is 1, and the stride of the second and fourth convolutional blocks is 2.

6. The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 4, wherein In the first feature processing block of the first temporal feature extraction module, the convolutional kernel of the first convolutional layer in the first branch is 1×1, the stride is 1, the output feature map channels are 160, the convolutional kernel of the second convolutional layer in the first branch is 5x5, the stride is 1, the padding is 2, the convolutional kernel of the third convolutional layer in the first branch is 7x7, the stride is 1, the padding is 9, and the dilation coefficient is 3; In the second branch of the first feature processing block, the convolutional kernels of the two 1x1 convolutional layers are both 1×1, the stride is 1. The input channels of the channel adjustment convolutional layer in the first feature processing block are 160, the output channels are 32, the convolutional kernel is 1×1, and the stride is 1; In the 2nd to 7th feature processing blocks of the first temporal feature extraction module, the convolutional kernels of the first convolutional layer in the first branch are all 1×1, the strides are all 1, the output feature map channels are all 32, the convolutional kernel sizes of the second convolutional layer in the first branch are all 5x5, the strides are all 1 pixel, the padding is all 2, the convolutional kernel sizes of the third convolutional layer in the first branch are all 7x7, the strides are all 1, the padding is all 9, the dilation coefficients are all 3, the convolutional kernels of the two 1x1 convolutional layers in the second branch are both 1×1, the stride is 1, the input channels of the channel adjustment convolutional layer are 32, the output channels are 32, the convolutional kernel is 1×1 pixel, and the stride is 1; In the channel adjustment convolutional layer of the 8th feature processing block of the first temporal feature extraction module, the input channels are 32, the output channels are 160, the convolutional kernel is 1×1, and the stride is 1.

7. The spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 4, characterized in that The number of input channels of the convolutional layers in the 4 convolutional blocks of the first decoder is 8, the number of output channels is 8, the kernel size is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the first decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the first decoder is 2; The number of input channels of the readout layer of the first decoder is mapped from 8 to 1, the kernel is 1×1, and the stride is 1.

8. The spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 4, characterized in that The number of input channels of the convolutional layers in the 4 convolutional blocks of the second decoder is 16, the number of output channels is 16, the kernel is 3×3, the stride is 1, and the padding is 1; the first and third convolutional blocks of the second decoder include PixelShuffle layers for upsampling, and the upsampling factor in the PixelShuffle layer of the second decoder is 2; the number of input channels of the readout layer of the second decoder is mapped from 16 to 1, the kernel is 1×1, and the stride is 1.

9. The spatio-temporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 1, characterized in that The radar training dataset includes at least 50,000 samples and each sample contains 30 frames of radar feature data of 500×500. The first 10 frames of data in the radar training dataset are used as data input, the acquisition time of the first 10 frames of data in the radar training dataset is 60 minutes, and the last 20 frames in the radar training dataset are used as the true radar labels for the two-hour prediction. The dual-source data training dataset includes at least 50,000 samples, each sample contains 30 frames of dual-source data of 500×500. The first 10 frames of data in the dual-source data training dataset are used as data input, the acquisition time of the first 10 frames of data in the dual-source data training dataset is 60 minutes, the last 20 frames are the true lightning labels for the two-hour prediction, and the time resolution of the lightning data and the radar feature data in the dual-source data training dataset is 6 minutes, and the spatial resolution is 5km×5km.

10. The spatiotemporal convolutional neural network lightning forecasting method based on dual-source observation data according to claim 1, wherein, The dual-source data samples collected in step S6 are data with a time step of 6 minutes for 1 hour before the prediction time. The time period for predicting future lightning in step S7 is from the prediction time to the next 0-2 hours.

Images