A thunderstorm nowcasting method and system based on time constraint multi-modal learning

Abstract

This invention discloses a lightning nowcasting method and system based on time-constrained multimodal learning, belonging to the field of lightning nowcasting technology. The method first acquires multi-source meteorological data and performs spatiotemporal alignment, rasterization, and normalization on each data point to construct multivariate spatiotemporal input samples. Then, different modal data are independently encoded, and cross-variable spatiotemporal dependency features with temporal constraints are learned. Next, the spatiotemporal dependency features are modeled for the evolution of each forecast lead time to obtain a log-probability map of lightning predictions for multiple future times. During the training phase, spatial priors are injected into the log-probability map based on historical lightning occurrence frequencies, and dynamic focus loss is used to enhance the learning of rare and difficult lightning samples. Finally, activation processing yields a lightning occurrence probability map for future time series. This invention exhibits strong rare lightning event identification capabilities, good false alarm suppression, strong spatiotemporal generalization ability, and good interpretability of prediction results.

Description

Technical Field

[0001] This invention belongs to the field of lightning nowcasting technology, specifically relating to a lightning nowcasting method and system based on time-constrained multimodal learning. Background Technology

[0002] Lightning is a common and highly hazardous weather phenomenon during severe convective weather. It is characterized by its suddenness, short duration, scattered spatial distribution, and rapid evolution. It can easily have adverse effects on aviation operations, power transmission, outdoor work, transportation, and emergency rescue. Therefore, it is of great significance to conduct accurate forecasts of the location and probability of lightning in the short term.

[0003] Current lightning nowcasting methods typically rely on satellite cloud imagery, weather radar echoes, historical lightning observation data, or numerical weather prediction data. With the development of deep learning methods, the fusion of multi-source meteorological data can improve lightning forecast accuracy to some extent. However, lightning occurrence is influenced by multiple factors, including convective system development, cloud top thermal structure, precipitation particle phase, environmental instability energy, and historical discharge activity. Different data sources exhibit significant differences in spatial resolution, temporal resolution, observation methods, and statistical distribution, making it difficult to effectively coordinate and utilize multimodal information.

[0004] Furthermore, most existing multimodal lightning forecasting models directly fuse features based on the statistical correlation between variables, lacking explicit constraints on the temporal relationship between historical meteorological conditions and future lightning occurrences. This easily introduces unreasonable temporal sequences or unstable correlations into the prediction process. At the same time, the proportion of positive lightning samples in the spatiotemporal grid is relatively low, and the number of background areas is far greater than the number of lightning occurrence areas. Conventional loss functions are easily dominated by a large number of easily classified negative samples, causing the model to have problems such as missed lightning warnings, low prediction probabilities, or a large number of false alarms.

[0005] Therefore, there is an urgent need for a lightning nowcasting method and system that can uniformly organize multi-source meteorological observation data in time and space, learn cross-variable time and space dependencies in the direction of propagation from historical information to current information, and perform unbalanced optimization for rare lightning events. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a lightning nowcasting method and system based on time-constrained multimodal learning, which aims to solve the problems of insufficient accuracy and poor generalization ability of existing technologies in the fusion of multi-source meteorological observation data, modeling of the time-series dependency relationship between historical meteorological conditions and future lightning occurrences, and prediction of rare lightning events.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A lightning nowcasting method based on time-constrained multimodal learning includes the following steps: S1. Acquire historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and perform spatiotemporal alignment, rasterization, and normalization according to the time series to construct multimodal spatiotemporal input data; S2. Each variable in the multimodal spatiotemporal input data is independently encoded to obtain the variable feature representation in the shared latent space; S3. Construct a spatiotemporal attention interaction module with temporal constraints based on variable feature representation. The query feature and key feature are obtained by learningable projection from the variable feature representation, and the value feature is obtained by temporal unidirectional convolution that only receives information from the current time and its historical time, so as to obtain the fused temporal dependency feature. S4. Input the time-dependent features into the forecast-time decoding module. The Transformer decoder, convolutional long short-term memory network and temporally modulated deformable convolution are used to generate the lightning prediction log odds map corresponding to each forecast time. During the model training stage, a spatial occurrence prior is constructed based on the lightning labels in the training samples. The spatial occurrence prior is injected into the lightning prediction log odds map in an additive bias manner. At the same time, the focus parameters are dynamically adjusted according to the training process and the model parameters are updated using dynamic focus loss. S5. The log probability map of lightning prediction output by the trained model is processed by Sigmoid activation to obtain the probability map of lightning occurrence corresponding to multiple forecast lead times in the future.

[0008] As a further preferred embodiment of the present invention, the historical radar observation data includes radar vertical cumulative liquid water content data, the satellite infrared observation data includes satellite infrared 10.7-micron brightness temperature data, the historical lightning observation data includes lightning point detection data observed by the geosynchronous lightning imager or historical lightning grid data formed by lightning point detection data, and the environmental meteorological data includes at least one of convective effective potential energy, convective suppression energy, and near-surface air temperature.

[0009] As a further preferred embodiment of the present invention, the spatiotemporal alignment, rasterization, and normalization processing in S1 includes: resampling radar vertical cumulative liquid water content data and satellite infrared brightness temperature data to the target spatial grid; projecting environmental meteorological data to the target spatial grid using bilinear interpolation; rasterizing lightning point detection data into a lightning occurrence raster map according to a preset time interval and resampling it to the target spatial grid; and normalizing each input variable based on training set statistics to construct multimodal spatiotemporal input data; the multimodal spatiotemporal input data is represented as follows: in, Indicates the current moment. Indicates the number of historical observation time steps. Indicates the number of input channels. and These represent the height and width of the target space grid, respectively. Representing historical moments The corresponding multivariate meteorological feature map, among which, .

[0010] Future lightning tag data will be represented as follows: in, Indicates the number of time steps in the future forecast; Indicates future time The corresponding real lightning tag image, among which, For any spatial grid location, a label value of 1 indicates that lightning occurred at that location within the corresponding forecast period, and a label value of 0 indicates that lightning did not occur at that location within the corresponding forecast period. As a further preferred embodiment of the invention, the spatiotemporal attention interaction process of the independent variable encoding in S2 and the temporal sequence constraint in S3 satisfies: in, Indicates time At that time Continuous Multimodal spatiotemporal input data consisting of historical time steps; This represents a feature embedding function consisting of an independent encoder for each variable and a feature projection layer; This represents the feature representation of multimodal variables in the shared latent space obtained after independent variable encoding and feature projection; , and These represent query features, key features, and value features, respectively. and These represent the learnable projection matrices used to generate query features and key features, respectively; This indicates a temporal one-way convolution operation, which uses only the input features from that time and previous times when calculating the value features at any historical moment; This represents the number of channels sharing the latent feature space; Key features Transpose of; This indicates the normalized attention weight calculation operation; This represents the temporal dependency feature obtained after time-sequence constrained spatiotemporal attention interaction; This indicates the first forecast in the lead time decoding process. The time-dependent features invoked by each forecast lead time. All forecast lead times share the aforementioned time-dependent features, i.e. ,in, .

[0011] As a further preferred embodiment of the present invention, the forecast-by-forecast lead time decoding process in S4 is for the first... The forecast lead time is implemented, among which... The above time-series dependency features As the first Timing dependency characteristics of forecast lead times ,Right now and to Execution and the Time-series enhancement processing corresponding to each forecast lead time yields enhanced features. ; will enhance features With the Hidden states corresponding to each forecast lead time The concatenation and convolutional fusion are performed along the channel dimension, and the convolutional fusion result is combined with temporal dependent features. The common input to the Transformer decoder yields the first... Decoding features corresponding to each forecast lead time Decode features Hidden state and memory state The input decoder is convolved with a Long Short-Term Memory network to update the result. Hidden states corresponding to each forecast lead time and memory state Then hide the state Input a time-modulated deformable convolution and output a feature decoder to obtain future time-series data. The corresponding lightning prediction log odds plot, the process satisfies: in, This represents the temporal dependency feature obtained after time-sequence constrained spatiotemporal attention interaction; Indicates the first The timing dependency characteristics of each forecast lead time call, and ; Representing time-dependent features After the first The enhanced features obtained after time-series enhancement processing corresponding to each forecast lead time; This indicates a splicing operation along the channel dimension; Indicates the convolution fusion operation; This indicates a Transformer decoding operation; Indicates the first Decoding features corresponding to each forecast lead time; and They represent the first The hidden and memory states corresponding to each forecast lead time; and They represent the first The updated hidden state and updated memory state corresponding to each forecast lead time; when hour, and These represent the initial hidden state and the initial memory state during the decoding process, respectively. Indicates the decoded end volume Long Short-Term Memory (LSTM) network operations; Indicates based on forecast lead time Temporal modulation deformable convolution operation with modulation convolution sampling offset and modulation coefficients; This indicates the output feature decoding operation; Indicates the current moment; Indicates future time The corresponding log odds chart for lightning prediction.

[0012] As a further preferred embodiment of the present invention, the spatial prior constraint modeling in S4 includes: constructing a spatial lightning occurrence frequency map based on the lightning labels of each spatial location in the training set under all future forecast lead times; performing logarithmic compression and standardization on the spatial lightning occurrence frequency map to obtain a standardized spatial occurrence prior; and injecting the standardized spatial occurrence prior as an additive bias into the lightning prediction log probability map. The spatial prior is frozen after the preset preheating training phase and used for subsequent model training. The process satisfies the following: in, Represents spatial location within the training set The average frequency of lightning occurrence at that location Indicates the number of training samples. Indicates the number of time steps in the future forecast. Indicates the first The training sample at the th ... Forecast lead time and spatial location The real lightning tag at the location, This indicates that lightning has occurred. This indicates that no lightning occurred. This indicates that the space after logarithmic compression has prior knowledge. This represents a spatial frequency sensitivity parameter. This indicates the spatial prior amplitude adjustment parameter. Represents the set of all predicted spatial grid locations. Represents a set The total number of predicted spatial grid locations. and Let represent the mean and standard deviation of the spatial prior at all spatial grid locations, respectively. This indicates that the standardized spatial priors have occurred. Indicates the first The sample at the th Forecast lead time and spatial location The original lightning prediction log odds at the location, This represents the log-probability of lightning prediction after a priori occurrence in the injected space. ， This represents the injection intensity parameter that indicates a priori occurrence in the space.

[0013] As a further preferred embodiment of the present invention, the model training employs a dynamic focus loss function that combines log-odds space prior injection with dynamic focus modulation. The dynamic focus loss function dynamically adjusts the focus parameters according to the current training epoch, and calculates the predicted probability corresponding to the true class based on the log-odds of lightning prediction after the prior injection occurs in the injection space. This process then calculates the pixel-level dynamic focus loss and the batch-averaged dynamic focus loss, satisfying the following conditions: in, Indicates the current training round. Indicates the total number of training rounds. This indicates the focus parameter corresponding to the current training round. and These represent the initial and final values ​​of the focus parameter, respectively. This represents the Sigmoid activation function. This represents the probability of lightning occurrence after spatial prior correction. Indicates the first The sample at the th Forecast lead time and spatial location The actual lightning tag at the location; This represents the corrected predicted probability corresponding to the true category. Represents the category balance coefficient. Indicates the first The sample at the th Forecast lead time and spatial location Pixel-level dynamic focus loss at the location, This represents the sample index in the training batch. Indicates batch size, Represents the set of all predicted spatial grid locations. Represents a set The total number of predicted spatial grid locations. This represents the batch-averaged dynamic focus loss used for backpropagation to update model parameters.

[0014] As a further preferred embodiment of the invention, it further includes a step of performing target-oriented contribution attribution for lightning prediction results at a specified forecast lead time and a specified spatial location. Target-oriented contribution attribution involves backpropagating the prediction response corresponding to the specified forecast lead time and specified spatial location to historical multimodal input data to obtain the contribution of each variable and each historical time point to the prediction response. Among these, neurons... To neurons The contribution level satisfies: in, Indicates allocation to neurons The correlation value, Indicated by neurons Reverse allocation to neurons The correlation value, Represents neurons For neurons The contribution value, Indicates the relationship with neurons Indexes of other neurons in the same layer, Indicates the first Input variables in each sample The cumulative correlation value, Indicates the number of samples that participated in the contribution attribution. Indicates input variables Global prediction contribution score, Indicates input variables Normalized prediction contribution This indicates the total number of input variables. This represents the summation index when traversing all input variables during the normalization summation process.

[0015] A lightning nowcasting system based on time-constrained multimodal learning is used to execute the above-mentioned lightning nowcasting method based on time-constrained multimodal learning. It includes a multimodal data processing module, a variable independent encoding module, a time-constrained spatiotemporal interaction module, a forecast-by-forecast time-decoding module, an imbalanced optimization training module, and a lightning probability output module. The multimodal data processing module is used to acquire historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and to perform spatiotemporal alignment, rasterization, and normalization on the data to construct multimodal spatiotemporal input data; The independent variable encoding module is used to independently encode each variable in the multimodal spatiotemporal input data and output variable feature representations; The temporal sequence constraint spatiotemporal interaction module is used to perform attention interaction based on query features, key features, and value features obtained by temporal unidirectional convolution, and output temporal dependent features. The forecast-time-by-forecast decoding module is used to generate the lightning prediction log odds map for each forecast time through a Transformer decoder, a convolutional long short-term memory network, a time-modulated deformable convolution, and an output feature decoder. The unbalanced optimization training module is used to construct a spatial occurrence prior during the training phase, inject the spatial occurrence prior as an additive bias into the lightning prediction log probability map, and update the model parameters using dynamic focus loss that dynamically adjusts the focus parameters. The lightning probability output module is used to perform Sigmoid activation processing on the log probability map of lightning predictions output by the trained model to generate lightning occurrence probability maps for multiple future forecast lead times.

[0016] As a further preferred embodiment of the present invention, the time-sequence constrained spatiotemporal interaction module includes a query feature generation unit, a key feature generation unit, a temporal unidirectional convolutional value feature generation unit, and an attention fusion unit; the forecast-by-forecast-time-decoding module includes a feature fusion unit, a Transformer decoding unit, a convolutional long short-term memory state update unit, a temporally modulated deformable convolutional prediction unit, and an output feature decoding unit; and the imbalanced optimization training module includes a lightning occurrence frequency statistics unit, a spatial occurrence prior standardization unit, a prediction log probability bias injection unit, and a dynamic focus loss calculation unit.

[0017] The beneficial effects of this invention are as follows: 1. This invention integrates historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and performs unified spatiotemporal alignment, rasterization, and normalization on data from different sources. This enables different modal data to participate in lightning nowcasting under a unified time series and spatial grid. Subsequently, the spatial structure features of different data modalities are extracted through a variable-independent encoder, and the features of each variable are mapped to a shared latent space. Then, a time-series dependent features related to future lightning occurrences are obtained through a time-series constrained spatiotemporal attention interaction mechanism. This fully utilizes the complementary information between radar observation features, cloud top thermal structure, historical discharge activity, and environmental instability conditions, which helps to improve the accuracy and spatiotemporal generalization ability of lightning nowcasting.

[0018] 2. This invention utilizes temporal unidirectional convolution to generate value features during the attention interaction process, ensuring that the feature interaction at the current moment only uses historical observation information from the current moment and previous moments, without introducing observation information from future moments. This imposes a temporal constraint on the information propagation direction between historical weather conditions and future lightning occurrences. This approach can reduce the interference of non-temporally ordered feature associations on prediction results, enhance the model's ability to learn the predictive dependency between the evolution of thunderstorm systems and future lightning occurrences, and improve the stability of prediction results under different weather processes.

[0019] 3. This invention inputs time-dependent features into a forecast-time-by-forecast decoding module, employing a Transformer decoder, a convolutional long short-term memory network, and a time-modulated deformable convolution for recursive prediction. The Transformer decoder establishes long-distance correlations between different features, the convolutional long short-term memory network maintains state continuity across different forecast times, and the time-modulated deformable convolution adjusts the convolution sampling position and modulation coefficients according to the forecast time to adapt to the movement, development, attenuation, and deformation processes of thunderstorm systems under different forecast times. This enhances the model's ability to characterize the evolution of thunderstorms across multiple forecast times and mitigates the problem of prediction accuracy decay as forecast times increase.

[0020] 4. This invention addresses the problems of a low proportion of lightning occurrence samples in the spatial grid, a large number of background samples without lightning, and differences in lightning occurrence frequency at different spatial locations in the training set. During the model training phase, a spatial occurrence prior is constructed based on the lightning occurrence frequency at each spatial grid location in the training samples. The standardized spatial occurrence prior is then injected into the lightning prediction log probability using an additive bias. Simultaneously, the focus parameters are dynamically adjusted according to the training progress, and dynamic focus loss is used to perform differentiated optimization for samples of different difficulty levels. This allows the model to maintain its ability to learn the overall sample distribution in the early stages of training and enhance its focus on rare and difficult lightning samples in the later stages of training, which helps to improve lightning detection capabilities and suppress false alarms in the background area.

[0021] 5. This invention also employs a goal-oriented prediction contribution attribution mechanism to backpropagate the corresponding prediction response to historical multimodal input data for lightning prediction results with specified forecast lead time and specified spatial location, thereby obtaining the relative prediction contributions of different input variables and different historical times to the prediction results. This can assist in analyzing the main observation information relied upon by the model when generating specified lightning forecast results, and improve the interpretability and analyzability of lightning nowcasting results. Attached Figure Description

[0022] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is an overall flowchart of the present invention; Figure 2 This is a schematic diagram of the overall structure of the lightning nowcasting network based on multi-source meteorological observation data in this invention; Figure 3 This is a schematic diagram of the cross-variable spatiotemporal attention interaction module with temporal sequence constraints in this invention. Figure 4 This is a schematic diagram of the structure of the forecast-time decoding module in this invention; Figure 5 This is a schematic diagram illustrating the prior injection of spatial lightning occurrence and the optimization of dynamic focus loss in this invention. Detailed Implementation

[0023] like Figures 1 to 5 As shown, this invention discloses a lightning nowcasting method and system based on time-constrained multimodal learning. The flowchart of the method is as follows. Figure 1As shown, this method aims to address the shortcomings of existing technologies in terms of accuracy and poor generalization ability in multi-source meteorological observation data fusion, modeling the temporal dependency relationship between historical meteorological conditions and future lightning occurrences, and predicting rare lightning events. First, this invention acquires historical radar vertical cumulative liquid water content data, satellite infrared cloud top brightness temperature data, historical lightning observation data, and environmental meteorological data. Data from different sources are then spatiotemporally aligned, rasterized, and normalized according to a unified time interval and spatial grid to form multimodal spatiotemporal data for the model input. Next, a variable-independent encoding module is used to extract feature representations of different meteorological variables. A temporal-constrained spatiotemporal attention interaction mechanism is used to model the spatiotemporal correlation between different variables. Specifically, temporal unidirectional convolution restricts the current-moment features to utilize only the observation information of the current moment and its historical moments to extract temporally ordered dependency features related to future lightning occurrences. Subsequently, the time-dependent features are input into the forecast-by-forecast decoding module. A Transformer decoder, a convolutional long short-term memory network, and a time-modulated deformable convolution are used to recursively model the development, movement, and deformation processes of thunderstorm systems under different forecast lead times, generating a lightning prediction log-probability map for multiple future forecast lead times. Further, during model training and optimization, a spatial occurrence prior is constructed based on the lightning occurrence frequency at each spatial grid location in the training set. This standardized spatial occurrence prior is injected into the lightning prediction log-probability map with an additive bias. Simultaneously, a dynamic focus loss function, whose focus parameters are dynamically adjusted during training, is used. Backpropagation is employed to learn the model parameters end-to-end, enhancing the ability to identify rare and difficult lightning samples and suppressing false alarms in the background region. Finally, the lightning prediction log-probability map output by the trained model is processed with Sigmoid activation to generate lightning occurrence probability maps for multiple future forecast lead times. This invention comprehensively considers the complementary representation relationship between multi-source meteorological observation data, the temporal constraints in the evolution of thunderstorm systems, and the uneven spatial distribution of lightning events. It can adapt to the needs of lightning nowcasting under different geographical regions, different forecast lead times, and different severe convective weather conditions. Furthermore, it obtains the relative contributions of different historical variables and historical moments to the forecast results through prediction contribution attribution analysis, thereby improving the accuracy, spatiotemporal generalization ability, and interpretability of lightning nowcasting results.

[0024] A lightning nowcasting method based on time-constrained multimodal learning includes the following steps: S1. Acquire historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and perform spatiotemporal alignment, rasterization, and normalization on the data according to the time series to obtain processed multimodal spatiotemporal input data; S2. Each variable in the processed multimodal spatiotemporal input data is independently encoded to obtain the feature representation of the multimodal variables in the shared latent space; S3. Based on the multimodal variable feature representation, constrain the spatiotemporal attention interaction of execution time sequence to obtain temporal dependency features related to future lightning occurrences; S4. Input the time-dependent features into the forecast-time-by-forecast decoding module to perform time evolution modeling, and optimize the model parameters based on spatial lightning occurrence priors and dynamic focus loss function during the model training stage to obtain the lightning prediction log probability map corresponding to each future forecast time. S5. Perform Sigmoid activation processing on the lightning prediction log probability map to generate lightning occurrence probability maps corresponding to multiple future forecast lead times.

[0025] The historical radar observation data in S1 includes radar vertical cumulative liquid water content data, the satellite infrared observation data includes satellite infrared 10.7-micron brightness temperature data, the historical lightning observation data includes lightning point detection data observed by the geosynchronous lightning imager or historical lightning grid data formed by the lightning point detection data, and the environmental meteorological data includes at least one of convective effective potential energy, convective suppression energy, and near-surface air temperature.

[0026] The spatiotemporal alignment process includes: resampling historical radar observation data and satellite infrared observation data to the target spatial grid; projecting environmental meteorological data to the target spatial grid using bilinear interpolation; rasterizing lightning point detection data into a lightning occurrence raster map at preset time intervals and resampling it to the target spatial grid; and performing normalization processing on each input variable according to the training set statistics.

[0027] In one embodiment, the historical observation window length is set to... The future forecast window length is The number of input variable channels is The target space grid height and width are respectively and The processed multimodal spatiotemporal input data and future lightning tag data are represented as follows: in, Indicates the current moment; surface Showing time At that time Continuous Multimodal spatiotemporal input data consisting of historical time steps; Representing historical moments The corresponding multivariate meteorological feature map, among which, ; Indicates the future Real lightning tag data corresponding to each forecast lead time; Indicates future time The corresponding real lightning tag image, among which, For any spatial grid location, a label value of 1 indicates that lightning occurred at that location within the corresponding forecast period, while a label value of 0 indicates that no lightning occurred at that location within the corresponding forecast period.

[0028] In S2, a variable-independent encoder is used to extract features from different modal data. Specifically, independent convolutional encoders are used to process radar vertical cumulative liquid water content data, satellite infrared brightness temperature data, historical lightning grid data, and environmental meteorological data, so that different data modalities are mapped to a unified latent feature space while preserving their respective spatial structure characteristics. Let the first... The data for each input variable within the historical observation window are represented as follows: ,in, Then the first The independent encoding process for each input variable is represented as follows: Furthermore, the latent feature representations corresponding to each input variable are concatenated along the feature channel dimension and mapped to a unified latent space through a feature projection layer to obtain the multimodal variable feature representation, specifically as follows: in, Indicates the relationship with the first Each input variable corresponds to an independent convolutional encoder; Indicates the first Input variables in continuous The latent feature representation obtained by independent encoding within each historical time step; This indicates a splicing operation along the feature channel dimension; This represents the feature projection operation that maps the concatenated features corresponding to each input variable to a unified latent feature space. This represents a feature embedding function composed of independent encoders for each variable, a feature concatenation operation, and a feature projection layer. This represents the feature representation of multimodal variables in the shared latent space obtained after independent variable encoding and feature projection; Indicates the number of historical observation time steps. This indicates the number of input variable channels. In this embodiment, each input variable corresponds to one input channel. Therefore, It also represents the total number of input variables. Indicates the index of the input variable. This represents the number of channels in the unified latent feature space. and These represent the height and width of the target space grid, respectively.

[0029] In S3, cross-variable spatiotemporal attention interaction processing with temporal constraints is performed on multimodal latent features. In this attention interaction mechanism, query features and key features are used to describe the relationships between different variables, different historical moments, and different spatial locations. Value features are obtained through temporal unidirectional convolution, ensuring that the current moment features only utilize information from the current moment and its historical moments, without incorporating observation information from future moments. The specific processing procedure is as follows: in, This represents the feature representation of multimodal variables in the shared latent space obtained after independent variable encoding and feature projection; , and These represent query features, key features, and value features, respectively. and These represent the learnable projection matrices used to generate query features and key features, respectively; This indicates a temporal unidirectional convolution operation, which, when generating value features at any historical moment, only receives input features from that moment and the moments before it; This represents the number of channels sharing the latent feature space; Key features Transpose of; This indicates the normalized attention weight calculation operation; This represents the temporal dependency feature obtained after time-sequence constrained spatiotemporal attention interaction; This indicates the first forecast in the lead time decoding process. The time-series dependency characteristics of each forecast lead time are shared by all forecast lead times. Therefore... ,in, .

[0030] The temporal sequence constraint is used to restrict the propagation of information from historical moments to the current moment, so that historical meteorological conditions, current convective structures, and historical lightning activity can all participate in future lightning forecasting; the temporal sequence constraint is used to learn the temporally ordered dependencies that contribute to the prediction of future lightning occurrence, rather than to identify real physical causal relationships based on intervention.

[0031] Specifically, let This represents the forecast lead time index, where, The above time-series dependency features As the first Timing dependency characteristics of forecast lead times ,Right now ; for time-dependent features Execution and the Time-series enhancement processing corresponding to each forecast lead time yields enhanced features. ;make and They represent the first The hidden state and memory state corresponding to the forecast lead time, then the th... The decoding process for each forecast lead time satisfies: in, This represents the temporal dependency feature obtained after time-sequence constrained spatiotemporal attention interaction; Indicates the first The timing dependency characteristics of each forecast lead time call, and ; Representing time-dependent features After the first The enhanced features obtained after time-series enhancement processing corresponding to each forecast lead time; This indicates a splicing operation performed along the feature channel dimension; Indicates the convolution fusion operation; This indicates a Transformer decoding operation; Indicates the first Decoding features corresponding to each forecast lead time; This represents the Long Short-Term Memory (LSTM) network operation at the decoding end. ； and They represent the first The hidden and memory states corresponding to each forecast lead time; and They represent the first The updated hidden state and updated memory state corresponding to each forecast lead time; when hour, and These represent the initial hidden state and the initial memory state during the decoding process, respectively. Indicates based on forecast lead time Temporally modulated deformable convolution operations that adjust the convolution sampling offset and modulation coefficients ； Indicates output feature decoding operation ； Indicates the current time ； Indicates future time The corresponding lightning prediction log-odds plot. For the th in the training batch... One sample, Represents the logarithmic probability diagram of lightning prediction In spatial location The element at that position, among which, This represents the sample index in the training batch. This indicates the lead time index for future forecasts. .

[0032] As a further preferred embodiment of the present invention, S4 also constructs a spatial lightning occurrence prior during the model training phase. Specifically, during a preset preheating training phase, the actual lightning labels of each spatial grid position in the training samples under all future forecast lead times are cumulatively statistically analyzed to obtain a spatial lightning occurrence frequency map. Subsequently, the spatial lightning occurrence frequency map is sequentially subjected to logarithmic compression and standardization to obtain a standardized spatial occurrence prior. After the preset preheating training phase ends, the standardized spatial occurrence prior is frozen to remain unchanged in subsequent training phases, and it is injected as an additive bias into the original lightning prediction log probability map output by the forecast lead-by-lead decoding module to obtain a lightning prediction log probability map corrected by the spatial occurrence prior.

[0033] After completing the spatial occurrence prior injection, the sigmoid activation function is used to convert the corrected lightning prediction log odds into lightning occurrence probabilities, and the corrected prediction probabilities corresponding to the true categories are obtained based on the real lightning labels. Further, the focus parameters are dynamically adjusted according to the current training round, pixel-level dynamic focus loss is calculated, and the pixel-level dynamic focus loss corresponding to all samples, all future forecast lead times, and all predicted spatial grid positions within a training batch is averaged to obtain the batch-averaged dynamic focus loss. The construction of the spatial lightning occurrence prior, the correction of the prediction log odds, and the calculation of the dynamic focus loss satisfy the following: in, Represents spatial location within the training set The average frequency of lightning occurrence at that location; Indicates the number of training samples; Indicates the number of time steps in the future forecast; Indicates the first The training sample at the th ... Forecast lead time and spatial location The actual lightning tag in the location, among which, This indicates that lightning has occurred. This indicates that no lightning occurred; This indicates that the space after logarithmic compression has prior knowledge; This represents a spatial frequency sensitivity parameter. This indicates the spatial prior amplitude adjustment parameter; Represents the set of all predicted spatial grid locations; This indicates the number of predicted spatial grid locations in the set; and These represent the mean and standard deviation of the spatial occurrence prior at all predicted spatial grid locations, respectively; This indicates that the standardized space has prior occurrences; Indicates the first The sample at the th Forecast lead time and spatial location The original lightning prediction log odds at the location; This represents the log-probability of lightning prediction after prior occurrence in the injected standardized space; This represents the injection intensity parameter that indicates a priori occurrence in the space.

[0034] Indicates the current training round; Indicates the total number of training rounds; This indicates the focus parameter corresponding to the current training round; and These represent the initial and final values ​​of the focus parameter, respectively. This represents the Sigmoid activation function; This represents the probability of lightning occurrence after spatial prior correction; This represents the corrected predicted probability corresponding to the true category; Indicates the category balance coefficient; Indicates the first The sample at the th Forecast lead time and spatial location Pixel-level dynamic focus loss at the location; Indicates batch size, This represents the sample index in the training batch. Indicates the first in the training batch The sample at the th Forecast lead time and spatial location The actual lightning tag at the location; This represents the batch-averaged dynamic focus loss used for backpropagation to update model parameters.

[0035] During model training, a pre-training phase is used to statistically analyze real lightning labels and construct a spatial lightning occurrence frequency map. After the pre-training phase, the spatial lightning occurrence frequency map is logarithmically compressed and standardized to obtain a standardized spatial occurrence prior, which is maintained unchanged in subsequent training phases. For each training batch, the standardized spatial occurrence prior is injected as an additive bias into the original lightning prediction log-probability map, and the focus parameters are updated according to the current training round. The batch average dynamic focus loss is calculated, and then the model parameters are updated using the backpropagation method.

[0036] In S5, the log probability of lightning prediction after spatial prior correction is input into the Sigmoid activation function to obtain the probability of lightning occurrence for each sample at multiple forecast leads and at each predicted spatial grid location. For the first... The sample, the first Forecast lead time and spatial location The probability of lightning occurring satisfies: in, Indicates the first The sample at the th Forecast lead time and spatial location The log-probability of lightning prediction after spatial prior correction. This represents the Sigmoid activation function. This indicates the probability of the corresponding lightning strike. Indicates the number of time steps in the future forecast. This represents the set of all predicted spatial grid locations. The lightning occurrence probabilities are arranged according to each spatial grid location to obtain a lightning occurrence probability map corresponding to each future forecast period. The pixel value of each spatial grid location in the lightning occurrence probability map represents the predicted probability of lightning occurring at that location within the corresponding forecast period.

[0037] As a further preferred embodiment of the invention, it further includes performing target-oriented prediction contribution attribution on lightning prediction results for a specified forecast lead time and a specified spatial location. Specifically, it selects the first lightning occurrence probability from a lightning occurrence probability map corresponding to multiple future forecast leads. One forecast lead time and specified spatial location The corresponding predicted response is used as the attribution target, where, , The attribution target is backpropagated along the trained prediction network to historical multimodal input data, and correlation values ​​are assigned layer by layer according to the contribution of each layer of neurons to the attribution target, so as to obtain the predictive contribution of different input variables and different historical time points to the attribution target.

[0038] In the process of layer-by-layer reverse allocation, the neurons are allocated... The correlation value is based on the neuron For neurons The contribution values ​​of neurons are proportionally allocated to obtain the results. Reverse allocation to neurons The correlation value, the process satisfies: Through the aforementioned layer-by-layer reverse allocation process, the correlation value corresponding to the specified predicted response is propagated to the historical multimodal input data. For each sample participating in the contribution attribution, the correlation values ​​are summarized according to the input variables and their corresponding historical moments to obtain the contribution of each input variable and each historical moment to the specified predicted response; further, the correlation values ​​of the same input variable at each historical moment and each spatial location are accumulated to obtain the first... Input variables in each sample Cumulative correlation value .

[0039] For participation contribution attribution For each sample, the cumulative correlation value of the same input variable is averaged to obtain the input variable. The global prediction contribution score, the process satisfies: Furthermore, the global prediction contribution scores of each input variable are normalized to obtain the input variables. The normalized predicted contribution rate satisfies the following process: in, Indicates allocation to neurons The correlation value, Indicated by neurons Reverse allocation to neurons The correlation value, Represents neurons For neurons The contribution value, Indicates the relationship with neurons Indexes of other neurons in the same layer, Indicates the first Input variables in each sample The cumulative correlation value, Indicates the number of samples that participated in the contribution attribution. Indicates input variables Global prediction contribution score, This indicates the total number of input variables. This represents the summation index during the normalization summation process, traversing all input variables. Indicates input variables The normalized prediction contribution.

[0040] Predict the contribution based on the normalization of each input variable. It is possible to determine the impact of different historical meteorological variables on the first One forecast lead time and specified spatial location The relative contribution of correlation values ​​to lightning prediction results can be determined; based on the distribution of correlation values ​​along the historical time dimension, the historical time range can be identified. The relative contribution of different historical moments to the lightning prediction results is analyzed to determine the historical observation information on which the model is based when generating a specified lightning prediction result.

[0041] The above methods can be used to obtain the relative predictive contributions of historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data to the specified lightning prediction results at different historical moments, which can be used to assist in the analysis of the prediction basis of the lightning nowcasting model.

[0042] It also includes a lightning nowcasting system based on time-constrained multimodal learning, comprising a multimodal data processing module, a variable independent encoding module, a time-constrained spatiotemporal interaction module, a forecast-by-forecast-lead-time decoding module, an imbalanced optimization training module, and a lightning probability output module. The multimodal data processing module acquires historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and performs spatiotemporal alignment, rasterization, and normalization on the data to construct multimodal spatiotemporal input data. The variable independent encoding module independently encodes each variable in the multimodal spatiotemporal input data and outputs variable feature representations. The time-constrained spatiotemporal interaction module is used to base its system on query features, key features, and time-series data. The value features obtained from unidirectional convolution undergo attention interaction to output temporally dependent features; the forecast-by-forecast decoding module generates lightning prediction log-probability maps for each forecast time through a Transformer decoder, a convolutional long short-term memory network, a temporally modulated deformable convolution, and an output feature decoder; the imbalanced optimization training module constructs a spatial occurrence prior during the training phase, injects the spatial occurrence prior as an additive bias into the lightning prediction log-probability map, and updates the model parameters using a dynamic focus loss that dynamically adjusts the focus parameters; the lightning probability output module performs Sigmoid activation processing on the lightning prediction log-probability map output by the trained model to generate lightning occurrence probability maps for multiple future forecast timeframes.

[0043] The temporal sequence constraint spatiotemporal interaction module includes a query feature generation unit, a key feature generation unit, a temporal unidirectional convolutional value feature generation unit, and an attention fusion unit. The forecast-by-forecast-time decoding module includes a feature fusion unit, a Transformer decoding unit, a convolutional long short-term memory state update unit, a temporally modulated deformable convolutional prediction unit, and an output feature decoding unit. The imbalanced optimization training module includes a lightning occurrence frequency statistics unit, a spatial occurrence prior standardization unit, a prediction log probability bias injection unit, and a dynamic focus loss calculation unit.

[0044] The training process of a deep learning network includes model training, model validation, and model testing. The model training phase includes forward propagation and backpropagation. Based on the training samples, the dynamic focus loss between the lightning prediction results and the actual lightning labels is calculated. The learnable parameters in the variable-independent encoding module, the temporal-constrained spatiotemporal interaction module, and the forecast-by-forecast-time decoding module are updated using backpropagation. After the pre-set warm-up training phase, the standardized space occurrence prior, obtained by accumulating and statistically processing the actual lightning labels corresponding to each future forecast time based on the training samples through logarithmic compression and standardization, is frozen and used for prediction log-probability correction in subsequent training phases.

[0045] In the model validation phase, the validation set is used to evaluate the lightning nowcasting performance of the model corresponding to different training rounds, and the optimal model parameters are selected according to the preset evaluation indicators. The evaluation indicators include one or more of the following: probability detection rate, false alarm rate, critical success index, extended critical success index, area under the curve, and spatial structure similarity. Based on the validation results, one or more of the following parameters are adjusted: learning rate, historical observation window length, forecast time steps, number of feature channels, spatial occurrence prior injection intensity parameter, and dynamic focus loss parameter.

[0046] During the model testing phase, independent test samples are used to verify the performance of the trained lightning nowcasting model. The model outputs lightning occurrence probability maps corresponding to multiple forecast lead times and generates lightning occurrence forecast results based on preset thresholds. This is to evaluate the model's robustness, generalization ability, and forecast accuracy under unknown severe convective weather processes, different geographical regions, and different thunderstorm evolution conditions.

[0047] The lightning nowcasting method based on time-constrained multimodal learning of this invention can be applied to scenarios such as severe convective weather monitoring, lightning disaster early warning, and operational meteorological forecasting. By collaboratively processing historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data under different geographical regions, weather processes, and time scales through a deep learning network, it can generate lightning occurrence probability maps corresponding to multiple future forecast times, improving the accuracy and stability of lightning occurrence location and timing predictions. This method is of great significance for fields such as aviation operation safety assurance, power transmission line protection, outdoor operation risk early warning, transportation scheduling, forest fire prevention, and emergency rescue. The lightning nowcasting method of this invention has the advantages of strong multimodal information fusion capability, clear temporal constraints, good rare lightning event identification capability, strong spatiotemporal generalization capability, and good interpretability of prediction results, which can meet the lightning nowcasting needs of different regions and under different severe convective weather conditions. Furthermore, this method can be deployed on existing meteorological data processing platforms, operational forecast servers, or computing devices with deep learning inference capabilities, providing an efficient intelligent prediction tool for lightning disaster monitoring and early warning operations.

[0048] The specific implementation of this invention provides a technical solution for lightning nowcasting based on multi-source meteorological observation information for those skilled in meteorological forecasting, disaster early warning, and intelligent environmental data processing. This helps improve the availability and reliability of lightning prediction results under complex and severe convective weather conditions. By jointly modeling radar echo structure, satellite cloud top thermal characteristics, historical discharge activity, and environmental convection conditions, and utilizing techniques such as time-sequence constrained spatiotemporal attention interaction, forecast-by-forecast time-lead time decoding, spatial occurrence priors, and dynamic focus loss, the model's ability to characterize the development, movement, deformation, and lightning occurrence processes of thunderstorm systems can be improved. This provides more valuable predictive information for lightning disaster risk identification and early warning decisions.

[0049] The forecasting technology of this invention can also be applied to other scenarios related to severe convective weather risks, such as lightning risk alerts for airport take-off and landing and air route operations, lightning risk warnings for power transmission lines and substations, safety assurance for large-scale outdoor activities, meteorological risk management for wind farms and photovoltaic power stations, and short-term disaster early warning and decision support for urban emergency management departments. In the above application scenarios, users can generate corresponding lightning warning results based on the lightning occurrence probability map corresponding to different future forecast lead times, combined with preset probability thresholds, risk area boundaries, or business rules.

[0050] The lightning nowcasting method based on time-constrained multimodal learning of this invention can uniformly organize, extract features, model time dependencies, and predict probabilities from multi-source meteorological observation data, providing technical support for lightning disaster monitoring, early warning, and risk management. Through this invention, users can obtain lightning occurrence probability predictions for multiple forecast lead times and analyze the relative contributions of different historical observation variables to specified prediction results by combining prediction contribution attribution results, thereby improving the analysability and operational application value of lightning forecast results.

[0051] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A lightning nowcasting method based on time-constrained multimodal learning, characterized in that, Includes the following steps: S1. Acquire historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and perform spatiotemporal alignment, rasterization, and normalization according to the time series to construct multimodal spatiotemporal input data; S2. Each variable in the multimodal spatiotemporal input data is independently encoded to obtain the variable feature representation in the shared latent space; S3. Construct a spatiotemporal attention interaction module with temporal constraints based on variable feature representation. The query feature and key feature are obtained by learningable projection from the variable feature representation, and the value feature is obtained by temporal unidirectional convolution that only receives information from the current time and its historical time, so as to obtain the fused temporal dependency feature. S4. Input the time-dependent features into the forecast-time decoding module. The Transformer decoder, convolutional long short-term memory network and temporally modulated deformable convolution are used to generate the lightning prediction log odds map corresponding to each forecast time. During the model training stage, a spatial occurrence prior is constructed based on the lightning labels in the training samples. The spatial occurrence prior is injected into the lightning prediction log odds map in an additive bias manner. At the same time, the focus parameters are dynamically adjusted according to the training process and the model parameters are updated using dynamic focus loss. S5. The log probability map of lightning prediction output by the trained model is processed by Sigmoid activation to obtain the probability map of lightning occurrence corresponding to multiple forecast lead times in the future.

2. The lightning nowcasting method based on time-constrained multimodal learning according to claim 1, characterized in that: Historical radar observation data includes radar vertical cumulative liquid water content data; satellite infrared observation data includes satellite infrared 10.7-micron brightness temperature data; historical lightning observation data includes lightning point detection data observed by the Geosynchronous Lightning Imager or historical lightning grid data formed from lightning point detection data; and environmental meteorological data includes at least one of convective available potential energy, convective suppression energy, and near-surface air temperature.

3. The lightning nowcasting method based on time-constrained multimodal learning according to claim 2, characterized in that: The spatiotemporal alignment, rasterization, and normalization processes in S1 include: resampling radar vertical cumulative liquid water content data and satellite infrared brightness temperature data to the target spatial grid; projecting environmental meteorological data to the target spatial grid using bilinear interpolation; rasterizing lightning point detection data into a lightning occurrence raster map at preset time intervals and resampling it to the target spatial grid; and normalizing each input variable based on training set statistics to construct multimodal spatiotemporal input data. The multimodal spatiotemporal input data is represented as follows: in, Indicates the current moment. Indicates the number of historical observation time steps. Indicates the number of input channels. and These represent the height and width of the target space grid, respectively. Representing historical moments The corresponding multivariate meteorological feature map, among which, ; Future lightning tag data will be represented as follows: in, Indicates the current moment. Indicates the number of time steps in the future forecast. and These represent the height and width of the target space grid, respectively. Indicates future time The corresponding real lightning tag image, among which, For any spatial grid location, a label value of 1 indicates that lightning occurred at that location within the corresponding forecast period, while a label value of 0 indicates that no lightning occurred at that location within the corresponding forecast period.

4. The lightning nowcasting method based on time-constrained multimodal learning according to claim 3, characterized in that: The spatiotemporal attention interaction process between the independent variable encoding in S2 and the temporal sequence constraint in S3 satisfies: in, Indicates time At that time Continuous Multimodal spatiotemporal input data consisting of historical time steps; This represents a feature embedding function consisting of an independent encoder for each variable and a feature projection layer; This represents the feature representation of multimodal variables in the shared latent space obtained after independent variable encoding and feature projection; , and These represent query features, key features, and value features, respectively. and These represent the learnable projection matrices used to generate query features and key features, respectively; This indicates a temporal one-way convolution operation, which uses only the input features from that time and previous times when calculating the value features at any historical moment; This represents the number of channels sharing the latent feature space; Key features transpose; This indicates the normalized attention weight calculation operation; This represents the temporal dependency feature obtained after time-sequence constrained spatiotemporal attention interaction; This indicates the first forecast in the lead time decoding process. The time-dependent features invoked by each forecast lead time; all forecast lead times share the aforementioned time-dependent features, therefore... ,in, , This indicates the number of time steps in the future forecast.

5. The lightning nowcasting method based on time-constrained multimodal learning according to claim 4, characterized in that: The forecast-by-forecast time-lead time decoding process in S4 is aimed at the first The forecast lead time is implemented, among which... Temporal dependency features As the first Timing dependency characteristics of forecast lead times ,Right now and to Execution and the Time-series enhancement processing corresponding to each forecast lead time yields enhanced features. ; will enhance features With the Hidden states corresponding to each forecast lead time The data is concatenated along the channel dimension, then fused via convolution, and combined with temporally dependent features. The common input to the Transformer decoder yields the first... Decoding features corresponding to each forecast lead time Decode features Hidden state and memory state The input decoder is convolved with a Long Short-Term Memory network to update the result. Hidden states corresponding to each forecast lead time and memory state Then hide the state Input a time-modulated deformable convolution and output a feature decoder to obtain future time-series data. The corresponding lightning prediction log odds plot, the process satisfies: in, Indicates the first The timing dependency characteristics of each forecast lead time call, and ; Representing temporal dependency features After the first The enhanced features obtained after time-series enhancement processing corresponding to each forecast lead time; This indicates a splicing operation along the channel dimension; Indicates the convolution fusion operation; This indicates a Transformer decoding operation; Indicates the first Decoding features corresponding to each forecast lead time; and They represent the first The hidden and memory states corresponding to each forecast lead time; and They represent the first The updated hidden state and updated memory state corresponding to each forecast lead time; when hour, and These represent the initial hidden state and the initial memory state during the decoding process, respectively. This represents the Long Short-Term Memory (LSTM) network operation at the decoding end; Indicates based on forecast lead time Temporal modulation deformable convolution operation with modulation convolution sampling offset and modulation coefficients; This indicates the output feature decoding operation; Indicates the current time; Indicates future time The corresponding log-odds plot for lightning prediction.

6. The lightning nowcasting method based on time-constrained multimodal learning according to claim 5, characterized in that: The spatial prior constraint modeling in S4 includes: constructing a spatial lightning occurrence frequency map based on the lightning labels of each spatial location in the training set under all future forecast lead times; performing logarithmic compression and standardization on the spatial lightning occurrence frequency map to obtain a standardized spatial occurrence prior; and injecting the standardized spatial occurrence prior as an additive bias into the lightning prediction log probability map. The spatial prior is frozen after the pre-set preheating training phase and used for subsequent model training. The process satisfies the following: in, Represents spatial location within the training set The average frequency of lightning occurrence at that location Indicates the number of training samples. Indicates the number of time steps in the future forecast. Indicates the first The training sample at the th ... Forecast lead time and spatial location The real lightning tag at the location, This indicates that lightning has occurred. This indicates that no lightning occurred. This indicates that the space after logarithmic compression has prior knowledge. This represents a spatial frequency sensitivity parameter. This indicates the spatial prior amplitude adjustment parameter. Represents the set of all predicted spatial grid locations. Represents a set The total number of predicted spatial grid locations; and Let represent the mean and standard deviation of the spatial prior at all spatial grid locations, respectively. This indicates that the standardized spatial priors have occurred. This represents the sample index in the training batch. Indicates the first The sample at the th Forecast lead time and spatial location The original lightning prediction log odds at the location, This represents the log-probability of lightning prediction after a priori occurrence in the injected space. ， This represents the injection intensity parameter that indicates a priori occurrence in the space.

7. The lightning nowcasting method based on time-constrained multimodal learning according to claim 6, characterized in that: The model training employs a dynamic focus loss function that combines log-odds space prior injection with dynamic focus modulation. This dynamic focus loss function dynamically adjusts the focus parameters based on the current training epoch and calculates the predicted probability corresponding to the true class based on the log-odds of lightning prediction after the prior injection in the space. This process then calculates the pixel-level dynamic focus loss and the batch-averaged dynamic focus loss, satisfying the following conditions: in, Indicates the current training round. Indicates the total number of training rounds. This indicates the focus parameter corresponding to the current training round. and These represent the initial and final values ​​of the focus parameter, respectively. Indicates the first in the training batch The sample at the th Forecast lead time and spatial location The real lightning tag at the location; This represents the Sigmoid activation function. This represents the probability of lightning occurrence after spatial prior correction. ， This represents the corrected predicted probability corresponding to the true category. Represents the category balance coefficient. Indicates the first The sample at the th Forecast lead time and spatial location Pixel-level dynamic focus loss at the location, This represents the sample index in the training batch. Indicates batch size, Represents the set of all predicted spatial grid locations. Represents a set The total number of predicted spatial grid locations. This represents the batch-averaged dynamic focus loss used for backpropagation to update model parameters.

8. A lightning nowcasting method based on time-constrained multimodal learning according to claim 7, characterized in that: It also includes a step of performing goal-oriented contribution attribution for lightning prediction results at specified forecast lead times and specified spatial locations. Goal-oriented contribution attribution involves backpropagating the prediction response corresponding to the specified forecast lead time and specified spatial location to historical multimodal input data to obtain the contribution of each variable and each historical time point to the prediction response. Among these, neurons... To neurons The contribution level satisfies: in, Indicates allocation to neurons The correlation value, Indicated by neurons Reverse allocation to neurons The correlation value, Represents neurons For neurons The contribution value, Indicates the relationship with neurons Indexes of other neurons in the same layer, Indicates the first Input variables in each sample The cumulative correlation value, Indicates the number of samples that participated in the contribution attribution. Indicates input variables Global prediction contribution score, Indicates input variables Normalized prediction contribution This indicates the total number of input variables. This represents the summation index when traversing all input variables during the normalization summation process.

9. A lightning nowcasting system based on time-constrained multimodal learning, used to execute a lightning nowcasting method based on time-constrained multimodal learning as described in any one of claims 1-8, characterized in that: It includes a multimodal data processing module, a variable independent encoding module, a time-sequence constraint spatiotemporal interaction module, a forecast-by-forecast-time-decoding module, an imbalanced optimization training module, and a lightning probability output module; The multimodal data processing module is used to acquire historical radar observation data, satellite infrared observation data, historical lightning observation data, and environmental meteorological data, and to perform spatiotemporal alignment, rasterization, and normalization on the data to construct multimodal spatiotemporal input data; The independent variable encoding module is used to independently encode each variable in the multimodal spatiotemporal input data and output variable feature representations; The temporal sequence constraint spatiotemporal interaction module is used to perform attention interaction based on query features, key features, and value features obtained by temporal unidirectional convolution, and output temporal dependent features. The forecast-time-by-forecast decoding module is used to generate the lightning prediction log odds map for each forecast time through a Transformer decoder, a convolutional long short-term memory network, a time-modulated deformable convolution, and an output feature decoder. The unbalanced optimization training module is used to construct a spatial occurrence prior during the training phase, inject the spatial occurrence prior as an additive bias into the lightning prediction log probability map, and update the model parameters using dynamic focus loss that dynamically adjusts the focus parameters. The lightning probability output module is used to perform Sigmoid activation processing on the log probability map of lightning predictions output by the trained model to generate lightning occurrence probability maps for multiple future forecast lead times.

10. A lightning nowcasting system based on time-constrained multimodal learning according to claim 9, characterized in that: The temporal sequence constraint spatiotemporal interaction module includes a query feature generation unit, a key feature generation unit, a temporal unidirectional convolutional value feature generation unit, and an attention fusion unit. The forecast-by-forecast-time-decoding module includes a feature fusion unit, a Transformer decoding unit, a convolutional long short-term memory state update unit, a temporally modulated deformable convolutional prediction unit, and an output feature decoding unit. The imbalanced optimization training module includes a lightning occurrence frequency statistics unit, a spatial occurrence prior standardization unit, a prediction log probability bias injection unit, and a dynamic focus loss calculation unit.

Images