Lightning potential forecasting method and system based on random forest

Abstract

The invention relates to a lightning potential forecasting method and system based on a random forest. The method comprises the following steps: (1) collecting and preprocessing multi-source meteorological data; (2) feature extraction: cloud picture texture and shape features are extracted from satellite data, and echo intensity, top height and vertical development speed features are extracted from radar data; (3) constructing and optimizing a model; and (4) early warning thunder and lightning. The method has the advantages that the random forest serves as a typical representation of the ensemble learning algorithm, and the core of the random forest is to integrate prediction results of a plurality of decision trees. According to the integration mode based on Bagging, multi-dimensional feature information can be fully integrated, and the overfitting problem which is likely to happen to a single decision tree is effectively restrained. In a thunder and lightning potential forecasting scene, the algorithm can accurately capture a complex nonlinear relationship between meteorological elements and thunder and lightning potential by adaptively learning association between a large amount of historical meteorological data and thunder and lightning events, so that the forecasting accuracy is remarkably improved.

Description

Technical Field

[0001] This invention relates to a method and system for predicting lightning potential based on random forests, belonging to the field of meteorology. Background Technology

[0002] Lightning, a common and highly destructive natural phenomenon, often accompanies severe convective weather. The disasters it causes pose a serious threat to power systems, communication facilities, aerospace, building safety, and personal safety, not only causing power outages, communication network paralysis, and the spread of forest fires, but also leading to tragedies such as casualties. Therefore, accurate lightning potential forecasting is crucial for proactively deploying preventative measures and strengthening the safety of society and daily life. Currently, traditional lightning forecasting mainly relies on two methods: meteorological forecasting and numerical weather prediction. Meteorological forecasting depends on forecasters using data such as weather maps, satellite cloud images, and radar echoes, combined with long-term practical experience, to comprehensively analyze atmospheric circulation patterns, weather system evolution, and the characteristics of various meteorological elements, thereby determining the likelihood of lightning occurrence and its affected area. However, this method has significant limitations: the forecasting process is highly subjective, with significant differences in the analysis and judgment of different forecasters; and it is difficult to quantitatively evaluate the forecast results, with forecast accuracy greatly reduced under complex weather conditions. Numerical weather prediction simulates and extrapolates future weather conditions by solving atmospheric motion equations. However, this method has extremely stringent requirements for the accuracy of initial and boundary conditions; even minute errors can amplify exponentially over time. Furthermore, the formation mechanism of lightning involves complex physicochemical processes, and current numerical models have inherent deficiencies in their descriptions and parameterization schemes of these processes, making it difficult to achieve the accuracy standards required for practical applications in predicting the timing, spatial location, and intensity of lightning occurrences. In recent years, with the in-depth application of machine learning technology in meteorology, lightning forecasting methods based on this technology have emerged. However, facing the inherent high dimensionality, nonlinearity, and noise interference of meteorological data, existing machine learning models generally suffer from insufficient generalization ability and large fluctuations in forecast accuracy. Therefore, developing an innovative lightning potential forecasting technology to improve forecast accuracy and reliability has become a crucial issue that urgently needs to be addressed in the field of meteorological disaster prevention and mitigation.

[0003] Lightning forecasting is a challenging task, with difficulties mainly manifested in the following aspects: First, the physical processes of lightning occurrence are complex. Lightning formation involves multiple complex physical processes, including convection, water vapor condensation, and charge separation and accumulation. These processes interact and occur at different temporal and spatial scales, making it difficult for existing observation methods and numerical models to achieve comprehensive, high-precision simulation. Second, the meteorological conditions affecting lightning occurrence and development are highly variable. Lightning occurrence is extremely sensitive to meteorological conditions; even minor changes in meteorological elements can lead to significant differences in the probability and intensity of lightning. The uneven and rapidly changing distribution of atmospheric meteorological elements such as temperature, humidity, and wind fields in time and space makes high-density, high-precision real-time monitoring difficult, resulting in significant errors in the judgment of lightning occurrence conditions. Third, lightning activity exhibits significant spatiotemporal localization characteristics. Lightning often concentrates in the core area of ​​convective cells (horizontal scale < 10 km) and is closely related to local environmental factors such as orographic uplift and urban heat islands. This locality makes it extremely difficult for lightning forecasts to achieve the required spatial resolution, as conventional meteorological observation networks and forecasting models struggle to accurately capture the specific location and extent of lightning events.

[0004] Lightning potential forecasting has become a key technology for lightning forecasting and early warning due to its unique advantages: On the one hand, this technology has significant early warning capabilities, enabling risk assessment to be initiated when weather systems first show signs of activity. Compared to traditional forecasting models, it can issue warnings hours or even days in advance, allowing more preparation time for emergency response. On the other hand, trend forecasting is relatively accurate. Potential forecasting focuses on analyzing large-scale weather systems and circulation patterns, accurately depicting the evolution trend of weather systems. It can help meteorologists and relevant decision-making departments understand the evolution trend of the entire weather situation, grasp the overall development direction of weather systems, and thus better allocate resources and deploy emergency response measures.

[0005] In recent years, machine learning technology has opened up new avenues for lightning forecasting. By leveraging machine learning algorithms to mine the spatiotemporal correlation features of massive amounts of meteorological data, these algorithms can autonomously learn the nonlinear patterns of lightning occurrence, significantly improving the accuracy of predictions regarding lightning probability, intensity, and location. Furthermore, compared to traditional numerical weather prediction models, machine learning models require far less computation and resources, and their forecast accuracy and lead time have, in some scenarios, approached or even surpassed those of traditional numerical weather prediction, gradually becoming a core supporting technology for refined meteorological services. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention proposes a lightning potential forecasting method based on the random forest algorithm. This technique fully leverages the superior performance of the random forest algorithm in high-dimensional data processing, nonlinear relationship mining, and anti-overfitting, integrating key meteorological elements and data characteristics of lightning formation to construct a high-precision forecasting model. By quantitatively analyzing complex physical processes such as atmospheric convection, water vapor phase change, and charge accumulation, it achieves accurate predictions of lightning occurrence probability, spatiotemporal distribution, and intensity. This technology can provide relatively accurate lightning early warning information for key industries such as power, aviation, and communications, enhancing society's comprehensive defense capabilities and emergency response levels against lightning disasters.

[0007] This invention provides a method and system for predicting lightning potential based on random forest. The technical solution of this invention is as follows:

[0008] A lightning potential prediction method based on random forest includes the following steps:

[0009] (1) Collect and preprocess multi-source meteorological data: acquire upper-air sounding data, ground meteorological observation data, satellite remote sensing data, radar observation data and historical lightning observation data, and perform outlier removal, missing value filling and standardization through a three-level data quality control process;

[0010] (2) Feature extraction: Initial feature variables are selected based on the lightning formation mechanism, the importance of features is evaluated and redundant features are removed using the random forest algorithm, cloud image texture and shape features are extracted from satellite data, and echo intensity, top height and vertical development speed features are extracted from radar data.

[0011] (3) Model construction and optimization: A random forest model was constructed, and the parameters of the number of decision trees, the maximum depth of a single tree, and the minimum number of split samples in a node were optimized by ten-fold cross-validation. The area under the AUC-ROC curve and the F1 value were used as the joint evaluation criteria of multiple indicators.

[0012] (4) Lightning warning: Input real-time meteorological data into the trained model, output the probability of lightning occurrence, dynamically adjust the classification probability threshold to balance the hit rate and false alarm rate, and issue a warning signal when the probability exceeds the preset threshold.

[0013] The specific steps (1) are as follows:

[0014] 1.1 Data Acquisition: The upper-air sounding data includes geopotential height, temperature, humidity, wind speed, and wind direction data at different altitude levels; the ground meteorological observation data includes air temperature, air pressure, relative humidity, wind speed, wind direction, and precipitation data; the satellite remote sensing data includes cloud top temperature, cloud top height, and cloud structure data; the radar observation data includes radar reflectivity factor, radial velocity, and vertical liquid water content data; and the historical lightning observation data includes the time, location, and intensity of lightning occurrences, constructing a multi-source heterogeneous basic data pool.

[0015] 1.2 Implement three-level data quality control for the basic data pool:

[0016] First, through threshold verification and statistical analysis, outliers that exceed the physical constraints are identified and eliminated;

[0017] Secondly, a hybrid imputation model is constructed. For missing values ​​in time series data, a Long Short-Term Memory (LSTM) network is used for time series prediction and imputation. For missing values ​​in spatial data, the inverse distance weighting method is used to weight and fuse the spatial mean of neighboring monitoring stations.

[0018] Finally, based on the Z-score normalization method, the multi-source data are normalized to a unified dimension space, eliminating the interference of magnitude differences on subsequent analysis and providing a high-quality data base for model training.

[0019] Step (2) specifically refers to:

[0020] 2.1 Feature Selection:

[0021] Based on meteorological principles and lightning formation mechanisms, we initially screened characteristic variables closely related to lightning occurrence, including electrification environment parameters, dynamic triggering factors, and microphysical indicators. We then used the random forest algorithm to calculate the degree of influence of each feature on lightning occurrence, combined with correlation analysis, to remove redundant and unimportant features and retain key features.

[0022] 2.2 Feature Extraction:

[0023] For satellite remote sensing data, extract the texture and shape features of cloud images; for radar observation data, extract the spatial distribution features of echo intensity, echo top height, and echo vertical development speed features to mine potential lightning-related information in the data.

[0024] Step (3) specifically refers to:

[0025] 3.1 Model Construction: Construct a random forest model and set the following parameters: adjust the number of decision tree base classifiers, dynamically set the feature selection dimension when splitting nodes, constrain the maximum depth of the tree structure, and define the minimum sample threshold required for node splitting and leaf nodes; build the basic model framework through the preset layout of the parameter space.

[0026] 3.2 Parameter Optimization: Cross-validation was used to iterate through parameter combinations to optimize the model parameters. The preprocessed dataset was divided into training and validation sets in a 7:3 ratio. Candidate models were generated on the training set. The area under the AUC-ROC curve was used to measure the model's ability to distinguish between positive and negative samples. The F1 score was combined to balance the model's precision and recall, and a multi-metric joint evaluation system was constructed. During the iterative optimization process, the parameter configuration scheme that enabled the model's overall performance to reach the preset level was selected.

[0027] 3.3 Model Training and Feature Analysis: Based on parameter combination, the training set data is used to drive the training of the random forest model. The model adaptively learns the correspondence between massive historical meteorological data and lightning events, and establishes a nonlinear mapping relationship between meteorological characteristics and lightning potential. After training, the quantitative ranking results of feature importance are output, and the key meteorological disaster-causing factors affecting the occurrence of lightning are analyzed, providing interpretable basis for model decision-making.

[0028] The specific steps (4) are as follows: meteorological data are input into a trained random forest model, and the model outputs the probability of lightning occurring in the area within a specific time period in the future; according to user needs, a probability threshold is set. When the model output probability is greater than the threshold, it is determined that there is lightning potential in the area and a lightning warning signal is issued; when the output probability is less than or equal to the threshold, it is determined that there is no lightning potential in the area, and meteorological data monitoring and analysis continue, and the prediction model input is dynamically updated to realize intelligent monitoring and early warning of lightning disasters throughout the entire process.

[0029] In step 2.1, when constructing the decision tree, the Random Forest algorithm uses different bootstrap sampling techniques to sample the training set. This Random Forest algorithm evaluates factor importance based on Out-of-Body (OOB) samples. For a Random Forest model, assuming a factor becomes a random number, the significance of that factor in the model is evaluated by comparing the difference in OOB error before and after the change. The factor importance metric is the Mean Difference in Accuracy (MDA).

[0030] MDA=∑(errOOB n -eerOOB o ) / N tree ……………………(1)

[0031] Among them, errOOB n This is the OOB error after changing the factor, errOOBo This is the OOB error before the factor was changed. The more significant the decrease in MDA, the more important the corresponding factor.

[0032] In step 3.1, the initial value of the feature diversity control parameter of the random forest model is set to... M is the total number of characteristics;

[0033] In step 3.2, when optimizing the model parameters, the model changes the classification result by adjusting the classification probability threshold. The optimal classification probability threshold is the point on the ROC curve that is close to 100% POD and 0% POD, at which point the area under the ROC curve, AUC, is 1. Using the ten-fold cross-validation method, all training sets are divided into ten equal parts, and nine parts are used for training each time, with the remaining part used for testing. The probability threshold is divided into 100 equidistant intervals from 0 to 1. Through cross-testing, the optimal classification probability threshold of the model is determined with AUC-ROC as the optimization objective.

[0034] A lightning potential forecasting system based on a random forest-based lightning potential forecasting method includes:

[0035] Multi-source data acquisition module, used to acquire high-altitude, ground, satellite and radar data;

[0036] The data processing module is used to perform outlier detection, missing value imputation, and standardization.

[0037] The feature extraction and evaluation module is used to extract key meteorological features and evaluate their importance; the model training module is used to build a random forest model and optimize its parameters.

[0038] The prediction and early warning module is used to output the probability of lightning occurrence and generate early warning signals.

[0039] The advantages of this invention are:

[0040] (1) High prediction accuracy: Random forest, as a typical representative of ensemble learning algorithms, is based on integrating the prediction results of multiple decision trees. This unique ensemble mode can fully integrate multi-dimensional feature information and effectively suppress the overfitting problem that is prone to occur with a single decision tree. In the lightning potential forecasting scenario, by adaptively learning the correlation between a large amount of historical meteorological data and lightning events, the algorithm can accurately capture the complex nonlinear relationship between meteorological elements and lightning potential, thereby significantly improving the forecast accuracy.

[0041] (2) Adaptive High-Dimensional Data: Lightning potential forecasting involves numerous meteorological elements, which together constitute a complex high-dimensional feature space. The Random Forest algorithm, with its adaptive selection mechanism based on feature importance, can accurately extract key disaster-causing factor combinations from massive datasets without complex data dimensionality reduction operations. This characteristic not only greatly improves data processing efficiency but also preserves the original feature information of the data, ensuring the accuracy and reliability of the forecast model.

[0042] (3) Strong model interpretability: The decision tree, a component of random forest, has good interpretability. By analyzing the nodes and branches of the decision tree, we can understand which meteorological elements play a key role in lightning potential forecasting and how these elements affect the forecast results, which helps meteorologists understand the mechanism of lightning occurrence.

[0043] (4) Quantitative assessment of feature importance: Based on the Gini index decay or permutation importance assessment method, random forest can effectively quantify the contribution of each feature to the forecast results. Based on these quantitative results, meteorologists can identify the key meteorological elements affecting lightning potential, thereby optimizing the allocation of observation resources in a targeted manner, rationally adjusting the physical process parameters in the meteorological model, and further improving the performance and accuracy of the forecast model.

[0044] (5) Excellent generalization ability: Random forests use a bootstrap sampling method to randomly extract multiple subsets from the original training data, which are then used to train different decision trees. Finally, the prediction results of these decision trees are combined to arrive at the final forecast conclusion. This training method enables the model to maintain stable forecast performance under different datasets and meteorological conditions. By reducing model variance, random forests effectively improve their stability and reliability, providing strong technical support for the business applications of this type of model. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the random forest lightning prediction model of the present invention.

[0046] Figure 2 This is a measured lightning distribution diagram according to an embodiment of the present invention.

[0047] Figure 3 This is a measured lightning distribution and lightning potential prediction map of the present invention (red dots represent measured lightning distribution, and colored shading represents the potential for lightning to occur).

[0048] Figure 4 This is a measured lightning distribution and lightning potential forecast map of an embodiment of the present invention (red dots represent measured lightning distribution, and colored shading represents the potential for lightning to occur).

[0049] Figure 5This is a measured lightning distribution and lightning potential forecast map of an embodiment of the present invention (red dots represent measured lightning distribution, and colored shading represents the potential for lightning to occur).

[0050] Figure 6 This is a measured lightning distribution and lightning potential forecast map of an embodiment of the present invention (red dots represent measured lightning distribution, and colored shading represents the potential for lightning to occur).

[0051] Figure 7 This is a schematic diagram of the lightning potential prediction system based on random forest of the present invention. Detailed Implementation

[0052] The present invention will be further described below with reference to specific embodiments, and the advantages and features of the present invention will become clearer as a result. However, these embodiments are merely exemplary and do not constitute any limitation on the scope of the present invention. Those skilled in the art should understand that modifications or substitutions can be made to the details and form of the technical solutions of the present invention without departing from the spirit and scope of the present invention, but all such modifications and substitutions fall within the protection scope of the present invention.

[0053] See Figures 1 to 7 This invention relates to a method for predicting lightning potential based on random forest, comprising the following steps:

[0054] (1) Collect and preprocess multi-source meteorological data: acquire upper-air sounding data, ground meteorological observation data, satellite remote sensing data, radar observation data and historical lightning observation data, and perform outlier removal, missing value filling and standardization through a three-level data quality control process;

[0055] (2) Feature extraction: Initial feature variables are selected based on the lightning formation mechanism, the importance of features is evaluated and redundant features are removed using the random forest algorithm, cloud image texture and shape features are extracted from satellite data, and echo intensity, top height and vertical development speed features are extracted from radar data.

[0056] (3) Model construction and optimization: A random forest model was constructed, and the parameters of the number of decision trees, the maximum depth of a single tree, and the minimum number of split samples in a node were optimized by ten-fold cross-validation. The area under the AUC-ROC curve and the F1 value were used as the joint evaluation criteria of multiple indicators.

[0057] (4) Lightning warning: Input real-time meteorological data into the trained model, output the probability of lightning occurrence, dynamically adjust the classification probability threshold to balance the hit rate and false alarm rate, and issue a warning signal when the probability exceeds the preset threshold.

[0058] The specific steps (1) are as follows:

[0059] 1.1 Data Acquisition: The upper-air sounding data includes geopotential height, temperature, humidity, wind speed, and wind direction data at different altitude levels; the ground meteorological observation data includes air temperature, air pressure, relative humidity, wind speed, wind direction, and precipitation data; the satellite remote sensing data includes cloud top temperature, cloud top height, and cloud structure data; the radar observation data includes radar reflectivity factor, radial velocity, and vertical liquid water content data; and the historical lightning observation data includes the time, location, and intensity of lightning occurrences, constructing a multi-source heterogeneous basic data pool.

[0060] 1.2 Implement three-level data quality control for the basic data pool:

[0061] First, through threshold verification and statistical analysis, outliers that exceed the physical constraints are identified and eliminated;

[0062] Secondly, a hybrid imputation model is constructed. For missing values ​​in time series data, a Long Short-Term Memory (LSTM) network is used for time series prediction and imputation. For missing values ​​in spatial data, the inverse distance weighting method is used to weight and fuse the spatial mean of neighboring monitoring stations.

[0063] Finally, based on the Z-score normalization method, the multi-source data are normalized to a unified dimension space, eliminating the interference of magnitude differences on subsequent analysis and providing a high-quality data base for model training.

[0064] Step (2) specifically refers to:

[0065] 2.1 Feature Selection:

[0066] Based on meteorological principles and lightning formation mechanisms, a preliminary screening of characteristic variables closely related to lightning occurrence was conducted, including electrification environment parameters, dynamic triggering factors, and microphysical indicators. These electrification environment parameters include the Swiss Thunderstorm Index (SWISS), Severe Weather Threat Index (SWEAT), K Index (KI), Schaeffler Stability Index (SI), Lift Index (LI), Total Index (TT), Best Convective Effective Potential Energy (BCAPE), Convective Effective Potential Energy (CAPE), Downdraft Convective Effective Potential Energy (DCAPE), and Modified Depth Convective Index (MDCI).

[0067] The dynamic triggering factors mentioned include storm relative helicity (SRH), vorticity, divergence, and vertical wind shear; the microphysical indicators mentioned include ice crystal concentration derived from dual-polarization radar parameters KDP.

[0068] The random forest algorithm is used to calculate the influence of each feature on lightning occurrence. Combined with correlation analysis (such as Pearson correlation coefficient), redundant and unimportant features are removed, retaining key features (such as the Swiss Thunderstorm Index (SWISS), Convective Effective Potential Energy (CAPE), and Vertical Wind Shear). The random forest algorithm can assess the importance of factors based on out-of-bottle (OOB) samples. For a random forest model, assuming a certain factor is changed to a random number, the significance of that factor in the model is evaluated by comparing the difference in OOB error before and after the change.

[0069] 2.2 Feature Extraction:

[0070] For satellite remote sensing data, extract texture features (such as gray-level co-occurrence matrix features) and shape features (such as perimeter, area, and circularity) of cloud images; for radar observation data, extract spatial distribution features of echo intensity, echo top height, and echo vertical development speed features to mine potential lightning-related information in the data.

[0071] Step (3) specifically refers to:

[0072] 3.1 Model Construction: When constructing decision trees, the Random Forest algorithm uses different bootstrap sampling techniques to sample the training set. Taking the k-th tree as an example, approximately one-third of the samples do not participate in the generation of the k-th tree. These samples are called the OOB samples of the k-th tree. OOB samples can estimate the error rate of the random forest samples outside the training set, i.e., the generalization error of the model. The OOB error is an unbiased estimate of the generalization error of the random forest, and its result is similar to that of k-fold cross-validation, which requires a large amount of computation.

[0073] Construct a random forest model and set the following parameters: adjust the number of decision tree base classifiers, dynamically set the feature selection dimension when splitting nodes, constrain the maximum depth of the tree structure, and define the minimum sample threshold required for node splitting and leaf nodes; build the basic model framework through the preset layout of the parameter space.

[0074] 3.2 Parameter Optimization: Cross-validation was used to iterate through parameter combinations to optimize the model parameters. The preprocessed dataset was divided into training and validation sets in a 7:3 ratio. Candidate models were generated on the training set. The area under the AUC-ROC curve was used to measure the model's ability to distinguish between positive and negative samples. The F1 score was combined to balance the model's precision and recall, and a multi-metric joint evaluation system was constructed. During the iterative optimization process, the parameter configuration scheme that enabled the model's overall performance to reach the preset level was selected.

[0075] 3.3 Model Training and Feature Analysis: Based on parameter combination, the training set data is used to drive the training of the random forest model. The model adaptively learns the correspondence between massive historical meteorological data and lightning events, and establishes a nonlinear mapping relationship between meteorological characteristics and lightning potential. After training, the quantitative ranking results of feature importance are output, and the key meteorological disaster-causing factors affecting the occurrence of lightning are analyzed, providing interpretable basis for model decision-making.

[0076] The specific steps (4) are as follows: meteorological data are input into a trained random forest model, and the model outputs the probability of lightning occurring in the area within a specific time period in the future; according to user needs, a probability threshold is set. When the model output probability is greater than the threshold, it is determined that there is lightning potential in the area and a lightning warning signal is issued; when the output probability is less than or equal to the threshold, it is determined that there is no lightning potential in the area, and meteorological data monitoring and analysis continue, and the prediction model input is dynamically updated to realize intelligent monitoring and early warning of lightning disasters throughout the entire process.

[0077] In step 2.1, when constructing the decision tree, the Random Forest algorithm uses different bootstrap sampling techniques to sample the training set. This Random Forest algorithm evaluates factor importance based on Out-of-Body (OOB) samples. For a Random Forest model, assuming a factor becomes a random number, the significance of that factor in the model is evaluated by comparing the difference in OOB error before and after the change. The factor importance metric is the Mean Difference in Accuracy (MDA).

[0078] MDA=∑(errOOB n -eerOOB o ) / N tree ……………………(1)

[0079] Among them, errOOB n This is the OOB error after changing the factor, errOOB o This is the OOB error before the factor was changed. The more significant the decrease in MDA, the more important the corresponding factor.

[0080] In step 3.1, the initial value of the feature diversity control parameter of the random forest model is set to... M is the total number of characteristics;

[0081] In step 3.2, when optimizing the model parameters, the model changes the classification result by adjusting the classification probability threshold. The optimal classification probability threshold is a point on the ROC curve close to 100% POD and 0% POD, at which point the area under the ROC curve (AUC) = 1. A ten-fold cross-validation method is used, dividing the entire training set into ten equal parts, with nine parts used for training each time and the remaining part for testing. The probability threshold is divided into 100 equidistant intervals from 0 to 1. Through cross-testing, the optimal classification probability threshold of the model is determined with AUC-ROC as the optimization objective. Since lightning is an extreme weather phenomenon, lightning events are relatively rare in the samples, resulting in a large proportion of samples without lightning and a severe class imbalance. Random undersampling without replacement is used to equalize the number of samples in both classes, thus achieving balance when building the random forest model.

[0082] It also includes the step of model validation, including:

[0083] (1) Generalization ability assessment: The generalization ability of the model is assessed by using the built-in test index OOB error.

[0084] (2) Classification performance verification.

[0085] Table 1 shows the consistency of lightning predictions in the random forest model.

[0086]

[0087] Table 1

[0088] The prediction results for binary events can be categorized using a 2×2 consistency table, as shown in Table 1. Based on the confusion matrix, seven evaluation metrics were calculated to validate the prediction results. These metrics include: Probability of Detection (POD, also known as hit rate), Probability of False Alarms (POFD, also known as false alarm rate), Correct Rate (PC, also known as accuracy), False Alarm Rate (FAR), Critical Success Index (CSI), Fair Threat Score (ETS), and Heidke Skill Score (HSS).

[0089]

[0090] Where n = h + f + m + c, h random = (h+f)×(h+m) / (h+m+f+c), where h is the number of hits, m is the number of missed reports, f is the number of false alarms, and c is the number of correct predictions.

[0091] Model capability evaluation: The model's performance was tested using an independent test set. Evaluation metrics and 95% bootstrap confidence intervals (CIs) were calculated for the hourly lightning potential prediction model from 0 to 6 hours. If the cross-test results are consistent with the training set, and the model's performance gradually improves as the lightning event approaches its trigger time, it indicates that the model has high predictive ability and reliable short-term forecast lead time gain.

[0092] Random forest prediction models can ensure a high hit rate by selecting the optimal probability threshold, which comprehensively considers both the hit rate and the false alarm rate, while controlling the false alarm rate as much as possible, and ultimately determine the objective prediction results.

[0093] Because lightning is an extreme weather event with a relatively low probability of occurrence, most of the test samples consisted of lightning-free events. The test samples also exhibited severe class imbalance, resulting in numerous false alarms. Given the significant hazards of lightning, actual operational early warning strategies typically adhere to the principle of "better to err on the side of caution than to miss an alarm," allowing for a certain number of false alarms while ensuring a high hit rate.

[0094] Example verification: On June 27, 2023, a relatively strong thunderstorm moved from northwestern Shandong towards the Shandong Peninsula. Figure 2 (The image shows the measured lightning distribution for the entire day of June 27, 2023.) Overall, the model accurately predicted the lightning strike areas, with a high degree of overlap between the predicted high-probability lightning areas and the measured dense lightning areas. The model's predicted lightning movement was generally consistent with the actual movement. False alarms occurred at some times; analysis revealed that the development and movement of thunderstorm clouds deviated from the actual lightning strike areas at these times, leading to significant false alarms in the model's forecast. This demonstrates that the model can effectively predict lightning strike areas in individual case forecasts. Based on this, a trained random forest lightning potential forecasting model was used to verify the predictions for this event. The results show that the dense lightning areas generally fell within the regions with a 70% or higher probability of lightning occurrence, especially the lightning distribution at 04, 05, and 06 UTC (Universal Time). Figures 3 to 5 , Figure 3 The measured lightning distribution (red dots) and lightning potential forecast (colored shading) are shown for 04 UTC (Universal Time) on June 27, 2023. Figure 4 This is a map showing the measured lightning distribution (red dots) and lightning potential forecast (colored shading) as of 05 UTC (Universal Time) on June 27, 2023. Figure 5 The data shows the observed lightning distribution (red dots) and lightning potential forecast (colored shading) as of 06 UTC on June 27, 2023. The observed lightning strike areas correspond well with the predicted lightning strike areas. However, the deviation is larger at 07 UTC. Figure 6(June 27, 2023, 07UTC: Actual situation and forecast): The observed area of ​​dense lightning was located near Jiaonan-Rizhao, while the potential forecast area was in the Yantai-Weihai area (sporadic lightning occurred in this area, so it is not a completely false alarm).

[0095] The present invention also relates to a lightning potential forecasting system based on random forest, comprising: a multi-source data acquisition module 1 for acquiring upper-air, ground, satellite, and radar data; a data processing module 2 for performing outlier detection, missing value imputation, and standardization; a feature extraction and evaluation module 3 for extracting key meteorological features and evaluating their importance; a model training module 4 for constructing a random forest model and optimizing parameters; and a prediction and early warning module 5 for outputting the probability of lightning occurrence and generating early warning signals.

[0096] The multi-source data acquisition module integrates upper-air sounding, ground observation, satellite remote sensing, radar, and historical lightning data to construct a multi-source heterogeneous data pool, covering all environmental factors involved in lightning formation (such as dynamic, thermal, and microphysical conditions). It supports the dynamic acquisition of real-time meteorological data, providing a data foundation for short-term early warning, solving the problems of single data and insufficient local capture in traditional forecasts, and improving the spatial resolution and timeliness of forecasts.

[0097] The data processing module employs a three-level quality control process, including outlier removal: ensuring data conforms to physical constraints through threshold verification and statistical analysis (e.g., removing abnormal temperature and humidity values); intelligent missing value imputation: using LSTM interpolation for time-series data (capturing temporal dependencies); and using the inverse distance weighting method for spatial data (fusing information from neighboring sites); and standardization: employing Z-score normalization to eliminate dimensional differences and improve model convergence efficiency. This process provides high-quality input data, significantly reduces noise interference, and lays the foundation for model accuracy.

[0098] The feature extraction and evaluation module has a dual optimization mechanism:

[0099] 1. Initial screening based on physical mechanisms: Key features are screened based on lightning formation mechanisms (such as the electrification environment parameter CAPE, dynamic factor, vertical wind shear, etc.).

[0100] 2. Data-driven refinement: Utilize random forest feature importance assessment to quantify feature contribution, and combine correlation analysis (Pearson coefficient) to remove redundant features.

[0101] It achieves automatic dimensionality reduction while retaining key disaster-causing factor combinations; it outputs a ranking of feature importance (e.g., "the Swiss Thunderstorm Index (SWISS) has the highest weight") to help meteorologists understand the physical mechanisms.

[0102] The model training module employs 10-fold cross-validation and optimizes hyperparameters such as the number of decision trees and maximum depth. A multi-metric evaluation system using AUC-ROC and F1 scores is constructed to balance hit rate and false alarm rate. Random undersampling is used to balance lightning / non-lightning samples, preventing the model from biased towards the majority class. The optimal threshold, close to "100% hit rate + 0% false alarm rate," is found on the ROC curve. Multiple decision trees and cross-validation are integrated to improve generalization ability. Dynamic threshold adjustment is supported to adapt to different industry needs.

[0103] The prediction and early warning module issued an early warning signal when the probability exceeded a preset threshold, and supported customized early warnings by region / time period, reducing the cost of manual judgment.

[0104] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lightning potential prediction method based on random forest, characterized in that, Includes the following steps: (1) Collect and preprocess multi-source meteorological data: acquire upper-air sounding data, ground meteorological observation data, satellite remote sensing data, radar observation data and historical lightning observation data, and perform outlier removal, missing value filling and standardization through a three-level data quality control process; (2) Feature extraction: Initial feature variables are selected based on the lightning formation mechanism, the importance of features is evaluated and redundant features are removed using the random forest algorithm, cloud image texture and shape features are extracted from satellite data, and echo intensity, top height and vertical development speed features are extracted from radar data. (3) Model construction and optimization: A random forest model was constructed, and the parameters of the number of decision trees, the maximum depth of a single tree, and the minimum number of split samples in a node were optimized by ten-fold cross-validation. The area under the AUC-ROC curve and the F1 value were used as the joint evaluation criteria of multiple indicators. (4) Lightning warning: Input real-time meteorological data into the trained model, output the probability of lightning occurrence, dynamically adjust the classification probability threshold to balance the hit rate and false alarm rate, and issue a warning signal when the probability exceeds the preset threshold.

2. The lightning potential prediction method based on random forest according to claim 1, characterized in that, The specific steps (1) are as follows: 1.1 Data Acquisition: The upper-air sounding data includes geopotential height, temperature, humidity, wind speed, and wind direction data at different altitude levels; the ground meteorological observation data includes air temperature, air pressure, relative humidity, wind speed, wind direction, and precipitation data; the satellite remote sensing data includes cloud top temperature, cloud top height, and cloud structure data; the radar observation data includes radar reflectivity factor, radial velocity, and vertical liquid water content data; and the historical lightning observation data includes the time, location, and intensity of lightning occurrences, constructing a multi-source heterogeneous basic data pool. 1.2 Implement three-level data quality control for the basic data pool: First, through threshold verification and statistical analysis, outliers that exceed the physical constraints are identified and eliminated; Secondly, a hybrid imputation model is constructed, which uses a Long Short-Term Memory (LSTM) network to impute missing values ​​in time series data. For missing values ​​in spatial data, the spatial mean of neighboring monitoring stations is weighted and fused using the inverse distance weighting method. Finally, based on the Z-score normalization method, the multi-source data are normalized to a unified dimension space, eliminating the interference of magnitude differences on subsequent analysis and providing a high-quality data base for model training.

3. The lightning potential prediction method based on random forest according to claim 2, characterized in that, Step (2) specifically refers to: 2.1 Feature Selection: Based on meteorological principles and lightning formation mechanisms, we preliminarily screened characteristic variables closely related to lightning occurrence, including electrification environment parameters, dynamic triggering factors, and microphysical indicators. The random forest algorithm is used to calculate the impact of each feature on lightning occurrence. Combined with correlation analysis, redundant features are removed and key features are retained. 2.2 Feature Extraction: For satellite remote sensing data, extract the texture and shape features of cloud images; For radar observation data, we extract the spatial distribution characteristics of echo intensity, echo top height, and echo vertical development velocity characteristics to mine potential lightning-related information in the data.

4. The lightning potential prediction method based on random forest according to claim 3, characterized in that, Step (3) specifically refers to: 3.1 Model Construction: Construct a random forest model and set the following parameters of the model, including adjusting the number of decision tree base classifiers, dynamically setting the feature selection dimension when splitting nodes, constraining the maximum depth of the tree structure, and defining the minimum sample threshold required for node splitting and leaf nodes; build the basic model framework through the preset layout of the parameter space. 3.2 Parameter Optimization: Cross-validation was used to iterate through parameter combinations to optimize the model parameters. The preprocessed dataset was divided into training and validation sets in a 7:3 ratio. Candidate models were generated on the training set. The area under the AUC-ROC curve was used to measure the model's ability to distinguish between positive and negative samples. The F1 score was combined to balance the model's precision and recall, and a multi-metric joint evaluation system was constructed. During the iterative optimization process, the parameter configuration scheme that enabled the model's overall performance to reach the preset level was selected. 3.3 Model Training and Feature Analysis: Based on parameter combination, the training set data is used to drive the training of the random forest model. The model adaptively learns the correspondence between massive historical meteorological data and lightning events, and establishes a nonlinear mapping relationship between meteorological characteristics and lightning potential. After training, the quantitative ranking results of feature importance are output, and the key meteorological disaster-causing factors affecting the occurrence of lightning are analyzed, providing interpretable basis for model decision-making.

5. The lightning potential prediction method based on random forest according to claim 4, characterized in that, The specific steps (4) are as follows: meteorological data are input into a trained random forest model, and the model outputs the probability of lightning occurring in the area within a specific time period in the future; according to user needs, a probability threshold is set. When the model output probability is greater than the threshold, it is determined that there is lightning potential in the area and a lightning warning signal is issued; when the output probability is less than or equal to the threshold, it is determined that there is no lightning potential in the area, and meteorological data monitoring and analysis continue, and the prediction model input is dynamically updated to realize intelligent monitoring and early warning of lightning disasters throughout the entire process.

6. The lightning potential prediction method based on random forest according to claim 3, characterized in that, In step 2.1, when constructing the decision tree, the Random Forest algorithm uses different bootstrap sampling techniques to sample the training set. This Random Forest algorithm evaluates factor importance based on Out-of-Body (OOB) samples. For a Random Forest model, assuming a factor becomes a random number, the significance of that factor in the model is evaluated by comparing the difference in OOB error before and after the change. The factor importance metric is the Mean Difference in Accuracy (MDA). MDA=Σ(errOOB n -eerOOB o ) / N tree ……………………(1) Among them, errOOB n This is the OOB error after changing the factor, errOOB o This is the OOB error before the factor was changed. The more significant the decrease in MDA, the more important the corresponding factor.

7. The lightning potential prediction method based on random forest according to claim 6, characterized in that, In step 3.1, the initial value of the feature diversity control parameter of the random forest model is set to... M is the total number of characteristics; In step 3.2, when optimizing the model parameters, the model changes the classification result by adjusting the classification probability threshold; the optimal classification probability threshold is the point on the ROC curve that is close to 100% POD and 0% POD, at which point the area under the ROC curve AUC = 1; the ten-fold cross-validation method is used to divide all training sets into ten equal parts, and nine of them are used for training each time, with the remaining part used for testing. The probability threshold was divided into 100 equally spaced intervals from 0 to 1. Through cross-testing, the optimal classification probability threshold of the model was determined with AUC-ROC as the optimization objective.

8. A lightning potential forecasting system based on the random forest method of any one of claims 1 to 7, characterized in that, include: Multi-source data acquisition module, used to acquire high-altitude, ground, satellite and radar data; The data processing module is used to perform outlier detection, missing value imputation, and standardization. The feature extraction and evaluation module is used to extract key meteorological features and evaluate their importance; The model training module is used to build a random forest model and optimize its parameters. The prediction and early warning module is used to output the probability of lightning occurrence and generate early warning signals.

Images