Method for lightning location and frequency estimation based on attention mechanism and bias calibration

Abstract

This invention discloses a collaborative estimation method for lightning location and frequency based on attention mechanisms and bias calibration, belonging to the fields of meteorological disaster monitoring and deep learning technology. The method includes: constructing a spatiotemporally aligned multi-source mapping dataset based on multi-channel data from geostationary meteorological satellites and ground-based lightning location network data; constructing a feature-enhanced U-shaped network FE UNet, which integrates a channel feature enhancement module, a dual-attention residual module, and an attention gating module, and is trained using a mask loss function based on Gaussian priors to enhance the spatial perception capability of sparse lightning features; and performing post-processing statistical bias calibration on the lightning frequency map output by the model, recovering the physical magnitude through binning statistics and linear fitting. This invention solves the problems of intensity underestimation and inaccurate spatial positioning in traditional methods for sparse lightning estimation, significantly improving the accuracy and reliability of lightning monitoring, and can be widely applied in the field of wide-area all-weather meteorological disaster early warning.

Description

Technical Field

[0001] This invention belongs to the field of meteorological disaster monitoring and deep learning technology, specifically involving a collaborative estimation method for lightning location and frequency based on attention mechanism and bias calibration. Background Technology

[0002] Real-time and accurate parameter estimation of lightning activity is crucial for building a meteorological disaster early warning system. Current lightning monitoring technologies mainly suffer from the following problems:

[0003] Limitations of ground-based radar networks: Existing ground-based radar networks are limited by terrain obstruction effects, resulting in significant monitoring blind spots in remote areas such as oceans, plateaus, and complex mountains, leading to gaps in global-scale geospatial data.

[0004] Computational bottlenecks of numerical models: Numerical weather forecasting models are limited by huge computational loads, making it difficult to meet the requirements of high resolution while responding quickly to the instantaneous dynamics of the atmospheric environment, and thus failing to meet the needs of real-time disaster visualization.

[0005] The limitations of existing deep learning methods: Traditional convolutional neural networks (CNNs) or U-Net models often lead to intensity bias or structural blurring when dealing with extremely sparse disaster events like lightning. Specifically:

[0006] Insufficient feature extraction: The thermodynamic physical meaning contained in multi-channel satellite data was not fully explored, resulting in a lack of physical consistency in the estimation results.

[0007] Intensity Underestimation: Since most areas in the image are zero-value backgrounds, conventional loss functions (such as MSE) will cause the model to tend to output smooth low values, making it difficult to capture the core of extremely high-frequency thunderstorms.

[0008] Oversmoothing: During the downsampling process of deep networks, sparse lightning features are easily "diluted" by background noise, resulting in inaccurate spatial localization.

[0009] Therefore, there is an urgent need for a wide-area lightning parameter estimation method that can integrate physical priors, effectively handle sparse targets, and calibrate intensity biases. Summary of the Invention

[0010] The purpose of this invention is to solve the problems of large strength deviation and structural ambiguity in the existing technology when dealing with sparse disaster targets.

[0011] The lightning location and frequency co-estimation method based on attention mechanism and bias calibration of the present invention includes the following steps: Step 1: Construct a spatiotemporally aligned multi-source mapping dataset based on geostationary meteorological satellite data and lightning location network data; Step 2: Construct a feature-enhanced U-shaped network FE UNet based on the encoder-decoder structure to output a two-dimensional gridded lightning frequency distribution map; Step 3: Train the Feature Enhanced U-Net (FE UNet) model based on the multi-source mapping dataset to obtain the trained Feature Enhanced U-Net (FE UNet); The physical characteristics of the geostationary meteorological satellite to be estimated are input into the trained feature-enhanced U-shaped network FEUNet, which outputs a two-dimensional gridded lightning frequency distribution map as the result of the joint estimation of lightning location and frequency.

[0012] Furthermore, the geostationary meteorological satellite physical characteristics input in step 3 include multi-band brightness temperature observations, dual-channel brightness temperature difference, and three-spectral brightness temperature difference; The multi-band brightness temperature observations include brightness temperature data obtained from Band 8, Band 9, Band 10, and Band 13 channels; Band 8 refers to the 8th band of the satellite, Band 9 refers to the 9th band of the satellite, Band 10 refers to the 10th band of the satellite, and Band 13 refers to the 13th band of the satellite. The dual-channel brightness temperature difference includes BTD0910, BTD1013, and BTD1513, where BTD0910 = Band 10 brightness temperature - Band 9 brightness temperature, BTD1013 = Band 10 brightness temperature - Band 13 brightness temperature, and BTD1513 = Band 15 brightness temperature - Band 13 brightness temperature. Band 15 refers to the 15th band of the satellite. The brightness temperature difference of the three spectra is expressed as (Band11 brightness temperature - Band13 brightness temperature) - (Band13 brightness temperature - Band15 brightness temperature), where Band 13 refers to the 13th band of the satellite.

[0013] Furthermore, the multi-source mapping dataset mentioned in step 1 includes geostationary meteorological satellite physical features obtained from geostationary meteorological satellite data, and a two-dimensional gridded lightning frequency distribution map obtained from lightning location network data; wherein, the process of obtaining the two-dimensional gridded lightning frequency distribution map based on lightning location network data is as follows: Lightning location network data includes the time of occurrence. and latitude and longitude coordinates The discrete lightning event sequence is aligned with the time window and projected into a grid coordinate system consistent with the satellite image. The gridding process assigns each lightning event to a unique spatial grid point and accumulates the count through a linear mapping from latitude and longitude to image coordinates, thereby generating a two-dimensional gridded lightning frequency distribution map. The two-dimensional gridded lightning frequency distribution map is used as a label during the training of the Feature Enhancement U-Net (FE UNet).

[0014] Furthermore, the Feature Enhancement U-Net (FE UNet) includes a Channel Feature Enhancement Module (CFE Block), an encoder, a decoder, an Attention Gated (AG) Module, and a Lightning Feature Enhancement Structure (LE Block). The Channel Feature Enhancement (CFE) Block expands the dimensionality of the input features. Multiple encoders perform downsampling at each stage, and the decoder performs upsampling at each stage. An attention-gated (AG) module is integrated at the jump connection between the encoder and decoder. The output of the final decoder is connected to the Lightning Feature Enhancement (LE) Block.

[0015] Furthermore, both the encoder and decoder employ a dual attention residual module (DARB). The dual attention residual module (DARB) adopts a three-segment cascaded feature evolution structure, which includes: a residual convolutional block, a mask space attention unit (MAM), and a channel attention module (CAM). The residual convolutional block contains a main path and bypass jump connections. The input features are processed by continuous convolution operations in the main path to extract local features, and then fused with the original features passed by the bypass jump connections element by element. After the fused features are activated by the ReLU function, they are sequentially passed through the Mask Space Attention Unit (MAM) and the Channel Attention Unit (CAM) to perform global spatial context awareness and secondary calibration of the feature weights in the channel dimension, respectively.

[0016] Furthermore, the Channel Feature Enhancement (CFE) Block consists of two intermediate processing layers connected in series with the Channel Attention (CAM) Block. The input physical features are sequentially processed through two intermediate processing layers containing 3×3 convolution, batch normalization, and ReLU activation functions for high-dimensional mapping, and then the Channel Attention (CAM) Block performs adaptive calibration of the importance of physical channels.

[0017] Furthermore, the channel attention module (CAM) includes parallel global max pooling and global average pooling layers, as well as a multilayer perceptron (MLP) with shared weights. The input features are processed by the global max pooling and global average pooling layers to generate two spatially compressed feature vectors. The two feature vectors are independently input into the MLP with shared weights for dimensionality reduction and dimensionality increase. Subsequently, the two channel feature vectors output by the MLP are fused element-wise and normalized using the sigmoid activation function to generate a channel weight tensor. Finally, the channel weight tensor is multiplied channel-wise with the input features of the channel attention module (CAM) to obtain the calibrated output features.

[0018] Furthermore, the Masked Spatial Attention Unit (MAM) flattens the input features into one-dimensional sequence features in the spatial dimension. Then, the one-dimensional sequence features are linearly mapped to generate query Q, key K, and value V features, which are then split and fed into a multi-head attention mechanism for processing. Within each attention head, the matrix product of the query and key transposes is first calculated to evaluate the global correlation between features. A learnable mask bias matrix is ​​then superimposed on this, followed by normalization using the Softmax function to obtain attention weights. Finally, the attention weights are multiplied by the value features to complete the weighted fusion of information. Subsequently, each attention head... The results from the force head are reassembled and integrated along the channel dimension, and restored to the initial spatial resolution of the Mask Space Attention Unit (MAM) through reshaping and bilinear interpolation upsampling operations. Then, the features input to the MAM are added element-wise through a long-range residual connection to obtain the fused features. Finally, the fused features enter the Feedforward Neural Network (FFN), which undergoes linear expansion that doubles the channel dimension, GELU activation function processing, Dropout operation, and channel compression. Finally, the features are added together with the local residual connections at the end to generate and output the final feature map.

[0019] Furthermore, the attention-gated (AG) module uses the deep features upsampled by the decoder as a gating signal to perform pixel-level weighted filtering on the shallow features of the encoder in the skip connection, so as to filter the background noise in the non-convection region. The specific selection process is as follows: First, the shallow features from the encoder are concatenated with the corresponding upsampled deep features from the decoder. Second, the concatenated features are passed through a 1×1 convolutional layer and a sigmoid activation function to generate a spatial attention weight map α with values ​​ranging from 0 to 1. Next, the input shallow features from the encoder are multiplied element-wise with the spatial attention weight map α to obtain purified shallow features after suppressing background noise. Finally, the purified shallow features are concatenated with the input upsampled deep features from the decoder again as the input to the next level of the decoder.

[0020] Furthermore, the lightning feature enhancement structure LE Block adopts a deep cascaded architecture, which sequentially includes a first intermediate processing layer, a channel attention module CAM, a second intermediate processing layer, and an output convolutional layer; Both the first and second intermediate processing layers are composed of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function connected in series. The output convolutional layer is a 1×1 convolutional layer; The high-dimensional features input to the LE Block are sequentially processed by feature mapping in the first intermediate processing layer, adaptive calibration of channel weights in the channel attention module CAM, and depth smoothing and dealiasing in the second intermediate processing layer. Finally, they are mapped by the output convolutional layer into a single-channel continuous lightning frequency estimate.

[0021] Furthermore, the feature-enhanced U-shaped network FE UNet is trained based on a Gaussian prior mask loss function; First, the two-dimensional gridded lightning frequency distribution map is transformed into a continuous Gaussian density field to generate a binary attention mask. Then, a weight coefficient α is assigned to the background region when calculating the loss. w , satisfying 0 < α w ≤ 0.1 forces the Feature Enhancement U-Net (FE UNet) to focus on the core region of thunderstorms.

[0022] The lightning location and frequency co-estimation method of the present invention further includes post-processing statistical bias calibration of the two-dimensional gridded lightning frequency distribution map output in step 3, as follows: A positive sample subset is obtained by setting a detection threshold to filter the lightning frequency distribution map output by the feature-enhanced U-Net (FE UNet). The actual lightning frequency was divided into multiple intensity intervals using a binning statistical method. The correspondence between the mean actual frequency and the mean model estimate in each interval was calculated to establish a mapping trend. A linear calibration function was fitted based on the least squares method, and the lightning frequency distribution map output by the feature-enhanced U-Net (FE UNet) was calibrated. The process of establishing the mapping trend also includes: calculating the 95% confidence interval of the model output in each intensity interval based on the extracted positive sample subset; using the distribution width of the 95% confidence interval as a statistical stability evaluation index to verify the high confidence of the model in extracting the relative intensity relationship features of lightning activity, thereby providing a statistical premise for constructing a linear calibration function from the estimation space to the physical space.

[0023] Beneficial Effects: This invention constructs a deep learning framework incorporating physical priors to collaboratively estimate the location and frequency of lightning from multi-channel geostationary meteorological satellite data. By integrating a dual-domain attention mechanism to adaptively filter physically relevant spectral features, it enhances spatial perception of sparse convection cores and suppresses background noise. Simultaneously, a mask loss function based on Gaussian priors forces the model to focus on high-value thunderstorm regions. Finally, a post-processing calibration module corrects systematic intensity biases in the regression task, thereby achieving high-precision lightning parameter estimation with all-weather, wide-area coverage.

[0024] 1. Significantly improved the spatial localization accuracy and regression accuracy of sparse lightning events. Thanks to the synergistic effect of the dual-attention residual module and attention gating mechanism in the feature-enhanced U-shaped network architecture, this invention effectively overcomes the defect of traditional convolutional networks where features are easily diluted by background noise when processing extremely sparse targets. Full-image evaluation data based on the test set shows that the root mean square error (RMSE) of this invention is reduced to 0.4413, the Pearson correlation coefficient (PCC) is increased to 0.4594, the critical success index (CSI) reaches 0.3863, and the false alarm rate (FAR) is only 0.1278. Compared with the traditional U-Net model, this invention significantly improves the spatial capture capability of small convection cores while maintaining an extremely low false alarm rate.

[0025] 2. Effectively solves the problem of systematic strength underestimation in deep learning regression tasks, achieving accurate recovery of physical quantities. To address the common problem of zero-value inflation in sparse data training, which leads to conservative and smooth model outputs, this invention employs a two-pronged approach: "training-side mask loss" and "back-end statistical calibration." The mask mean squared error loss function at the training end prevents gradient vanishing, while the back-end statistical bias calibration strategy successfully corrects the magnitude of the errors. Experimental data show that after linear calibration, the model's mean bias error (MBE) is significantly optimized from the original -2.0555 to 0.0743, essentially achieving unbiased estimation. Simultaneously, the coefficient of determination (R²) in the positive sample region increases from 0.1030, verifying that the model can accurately map compressed feature response values ​​back to the true physical frequency range.

[0026] 3. Possesses excellent physical interpretability and multi-band feature adaptive capability. The channel feature enhancer and channel attention module design in this invention enable the model to adaptively adjust the weights of different satellite bands based on atmospheric thermodynamics principles. Permutation feature importance analysis shows that the model can automatically identify the high importance of the mid-to-low tropospheric water vapor brightness-temperature difference (the difference between Band 9 and Band 10) and cloud top features (Band 13). This is highly consistent with the physical basis for judging atmospheric instability and cloud top height in meteorology, thus ensuring that the inversion results are not merely a fitting of image textures, but rather a retrieval of meteorological parameters with physical meaning.

[0027] 4. Enabled seamless, wide-area, all-weather meteorological disaster monitoring. This invention establishes a nonlinear mapping relationship from multi-channel geostationary meteorological satellite data to ground-based lightning data, overcoming the limitations of ground radar stations due to terrain obstruction and deployment range. This method can effectively fill monitoring gaps in areas such as oceans, plateaus, and complex mountains, and its computational efficiency is far higher than that of numerical weather prediction models. It can provide real-time, high-resolution (2km level) and wide-area coverage lightning parameter estimation for digital earth systems. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall architecture of the feature-enhanced U-shaped network FE UNet of the present invention.

[0029] Figure 2 This is a schematic diagram of the core attention mechanism unit structure.

[0030] Figure 3 This is a schematic diagram of the internal structure of the dual attention residual block (DARB).

[0031] Figure 4 This is a schematic diagram of the attention-gated (AG) module.

[0032] Figure 5 The flowchart shows the construction process of the attention mask based on Gaussian prior, where (a) is the discrete lightning event distribution map, (b) is the continuous density response field map generated by Gaussian convolution, and (c) is the binary attention mask map generated by threshold truncation.

[0033] Figure 6 This is a regression trend analysis chart of positive sample strength.

[0034] Figure 7 This is a schematic diagram of linear fitting of the calibration function.

[0035] Figure 8 This is a comparison chart of the intensity response trends before and after calibration. Detailed Implementation

[0036] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0037] Example 1 The proposed method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration includes the following three steps: Step 1: Data Preprocessing and Multi-Source Mapping Construction Based on Himawari-8 / Himawari-9 satellite data in NETCDF format and lightning location network data (VLF-LLN), a multi-source coupled dataset containing the physical characteristics of geostationary meteorological satellites and the corresponding lightning location network data (VLF-LLN) is constructed to establish the nonlinear mapping relationship between the two. The physical characteristics of geostationary meteorological satellites include multi-band brightness temperature observations, dual-channel brightness temperature difference (BTD), and tri-spectral brightness temperature difference (TTD), as shown in Table 1; The multi-band brightness temperature observations include brightness temperature data obtained from the Band 8 (6.2μm), Band 9 (6.9μm), Band 10 (7.3μm), and Band 13 (10.4μm) channels of the Himawari 8 / 9 satellite, which represent the water vapor distribution and cloud top height characteristics of the upper, middle, and lower troposphere, respectively. The dual-channel brightness temperature difference (BTD) includes BTD0910 (reflecting the relative cloud top height at approximately 450 hPa in the mid-troposphere), BTD1013 (reflecting the relative cloud top height at approximately 600 hPa in the lower troposphere), and BTD1513 (reflecting the optical thickness of the cloud). The three-spectral brightness temperature difference (TTD) is (Band11 brightness temperature - Band13 brightness temperature) - (Band13 brightness temperature - Band15 brightness temperature), used to characterize the cloud top phase state; The physical characteristics of geostationary meteorological satellites constitute a multi-channel input tensor that includes atmospheric thermal structure, water vapor distribution, and cloud microphysical properties.

[0038] Table 1. Characteristics and Definitions of Multi-Channel Geostationary Meteorological Satellites

[0039] Lightning Location Network (VLF-LLN) data includes lightning location and lightning frequency; lightning location refers to the precise latitude and longitude coordinates of a lightning event recorded by a ground-based location network, and lightning frequency refers to the total number of lightning events that occur cumulatively within a 2 km resolution grid and a specific time window (10 minutes).

[0040] Lightning Location Network Data (VLF-LLN) includes the time of occurrence. and latitude and longitude coordinates The discrete event sequence is first aligned with a time window for the lightning event: for each satellite observation time. Extracting time window All lightning events within the image are collected and then projected onto a 2 km grid coordinate system consistent with the satellite image through a linear mapping from latitude and longitude to image coordinates. The gridding process assigns each lightning event to a unique spatial grid point and accumulates the counts through the linear mapping from latitude and longitude to image coordinates, thereby generating a two-dimensional gridded lightning frequency distribution map. To suppress the interference of extreme outliers on the stability of model training, the frequency map is numerically truncated to obtain the final two-dimensional gridded lightning frequency distribution map as the model label, thus constructing a spatiotemporally highly matched training sample set.

[0041] Step 2: Construct the deep learning model FE UNet (Feature Enhancement U-Net).

[0042] A feature-enhanced convolutional neural network based on an encoder-decoder structure is constructed, and the model is trained using a mask loss function based on Gaussian prior. The network input includes a Channel Feature Enhancement (CFE) Block, which maps the original physical data into high-signal-to-noise ratio high-dimensional features through continuous convolution and a Channel Attention (CAM) Block, thereby enhancing the feature response while maintaining unified physical dimensions.

[0043] The backbone unit of the feature-enhanced U-shaped network employs a Dual Attention Residual Block (DARB), which integrates a Masked Attention Module (MAM) and a Channel Attention Module (CAM). The MAM introduces spatial mask priors and learnable biases, forcing feature extraction to focus on convection-active regions, while the CAM dynamically calibrates the weights of each physical channel. An Attention Gated (AG) module is integrated at the skip connections between the encoder and decoder, using upsampled semantic features from the skip connections as gating signals to perform pixel-level filtering of the AG module features, suppressing background noise in non-convective regions. At the network's end, a Lightning Feature Enhancement (LE) Block is incorporated, employing multi-layer convolution for depth smoothing and de-aliasing. Figure 1The overall data flow of the Feature Enhancement U-Net (FE UNet) is illustrated. The input multi-channel satellite physical features are processed by the Channel Feature Enhancer (CFE) and then enter the encoder path; the encoder contains multi-level dual attention residual blocks (DARB); the decoder path receives features from skip connections through attention gating (AG) and restores the resolution through DARB; finally, a lightning frequency estimation map is output through the Lightning Feature Enhancement Structure (LE Block).

[0044] like Figure 3 As shown, the Dual Attention Residual Module (DARB) adopts a three-segment cascaded feature evolution structure, which includes: residual convolutional block, mask space attention unit (MAM), and channel attention module (CAM). The residual convolutional block contains a main path and bypass jump connections. The input features are processed through continuous convolution operations in the main path to extract local features, which are then fused element-wise with the original features passed through the bypass jump connections. The fused features are activated by the ReLU function and then sequentially passed through the Mask Space Attention Unit (MAM) and the Channel Attention Unit (CAM) to perform global spatial context awareness and secondary calibration of the feature weights in the channel dimension, respectively.

[0045] Figure 2 The structure of the Mask Space Attention Module (MAM) is shown, which includes learnable bias terms. , The Masked Attention Unit (MAM) employs a spatial masking mechanism to focus on sparse features. It first receives the input feature map, and for larger feature maps, it performs adaptive downsampling to reduce computational overhead. Size downsampling to The spatial dimension is then flattened into a one-dimensional sequence format. This one-dimensional sequence feature is used to generate query Q, key K, and value V features through linear mapping, and then split into two groups along the channel dimension, which are fed into two independent attention heads for processing: the first group is Q1, K1, V1, and the second group is Q2, K2, V2. Within each attention head, the module first calculates the matrix product of the query and the corresponding key transpose, i.e., Q1 × With Q2× This is used to assess the global correlation between features, and then a dedicated learnable mask bias matrix is ​​superimposed on each feature. and The attention weights are then normalized using the Softmax function, and finally multiplied by the corresponding value features V1 and V2 to complete the weighted fusion of information. The results of each attention head are concatenated and integrated along the channel dimension, and then restored to the initial spatial resolution of the module through reshaping and bilinear interpolation upsampling operations. Next, they are added to the original input features via a long-range residual connection. The fused features enter the feedforward neural network (FFN), undergoing linear expansion that doubles the channel dimension, GELU activation function processing, Dropout operation, and channel compression. Finally, they are added together with the local residual connection at the end of the module to generate and output the feature map.

[0046] like Figure 2 As shown, the Channel Attention Module (CAM) Block includes parallel global max pooling and global average pooling layers, as well as a multilayer perceptron (MLP) with shared weights. The input features are processed by the global max pooling and global average pooling layers to generate two spatially compressed feature vectors. The two feature vectors are then independently input into the MLP with shared weights for dimensionality reduction and dimensionality increase, respectively. Subsequently, the two channel feature vectors output by the MLP are fused element-wise and normalized using the sigmoid activation function to generate a channel weight tensor. Finally, the channel weight tensor is multiplied channel-wise with the input features of the CAM Block to obtain the calibrated output features.

[0047] Figure 4 The demonstration shows that the AG module uses deep features upsampled by the decoder as a gating signal to perform pixel-level weighted filtering of shallow encoder features in skip connections, generating a spatial attention weight map α to filter background noise in non-convective regions. The specific filtering process is as follows: First, the shallow features from the encoder are concatenated with the corresponding deep features upsampled by the decoder. Second, the concatenated features are passed sequentially through a 1×1 convolutional layer and a sigmoid activation function to generate a spatial attention weight map α with values ​​ranging from 0 to 1. Next, the input shallow encoder features are multiplied element-wise with the spatial attention weight map α to obtain purified shallow features after suppressing background noise. Finally, the purified shallow features are concatenated again with the input deep features upsampled by the decoder as input for the next stage of the decoder.

[0048] The lightning feature enhancement structure LE Block adopts a deep cascaded architecture, which includes a first intermediate processing layer, a channel attention module (CAM), a second intermediate processing layer, and an output convolutional layer. The first and second intermediate processing layers are both composed of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function connected in series. The output convolutional layer is a 1×1 convolutional layer. The high-dimensional features input to the LE Block are sequentially processed by feature mapping in the first intermediate processing layer, adaptive calibration of channel weights in the channel attention module (CAM), and depth smoothing and dealiasing in the second intermediate processing layer. Finally, they are mapped by the 1×1 output convolutional layer into a single-channel continuous lightning frequency estimate.

[0049] During the model training phase, a masked mean squared error loss function based on Gaussian prior is employed. This method first transforms the discrete lightning frequency matrix into a spatially continuous Gaussian density response field, and then constructs a binary attention mask to distinguish between active and background regions. This mask is used to reduce the weight contribution of the background region when calculating the loss, forcing the model to focus on the sparse thunderstorm core region during optimization, thus solving the problem of low prediction values ​​caused by gradient updates dominated by zero-value background.

[0050] Step 3, Post-processing statistical bias calibration To address the systematic underestimation of strength in deep learning models during regression tasks, a statistical bias calibration module is introduced to correct the prediction results.

[0051] Step 3.1: First, set a detection threshold to screen positive samples from the original frequency map output by the model and eliminate zero-value background interference.

[0052] Step 3.2: Then, the binning statistical method is used to divide the actual frequency into multiple intensity intervals, and the correspondence between the mean of the actual value and the mean of the model estimate in each interval is calculated, thereby establishing the mapping trend from the estimation space to the physical space.

[0053] Step 3.3 involves fitting the optimal linear calibration function using the least squares method and then applying this function to perform a secondary transformation on the model's original output, restoring the compressed feature response values ​​to the true physical magnitude. This step eliminates systematic numerical biases while preserving spatial structural characteristics, achieving precise quantification of lightning frequency and intensity.

[0054] Example 2 This embodiment uses Himawari-8 geostationary meteorological satellite observation data and VLF-LLN ground-based lightning location data to illustrate the specific implementation process of the present invention. The lightning location and frequency co-estimation method based on attention mechanism and bias calibration of the present invention includes the following steps: Step 1, Dataset Construction and Preprocessing Eight physical channel data from the Himawari-8 satellite were selected as input features, specifically: Band8 (6.2 μm), Band9 (6.9 μm), Band10 (7.3 μm), Band13 (10.4 μm), as well as the calculated brightness temperature differences BTD0910, BTD1013, BTD1513 and the tri-spectral difference (TTD).

[0055] Lightning events recorded by the VLF-LLN system were used as labels. The time window for lightning data was set to 10 minutes prior to the satellite observation time. The lightning locations were mapped onto a 2-kilometer resolution grid consistent with the satellite imagery through latitude and longitude projection, generating a gridded frequency map with a size of 128×128. To ensure training stability, the frequency map was numerically truncated, with an upper limit threshold set to 40.

[0056] Step 2: Constructing and training the Feature-Enhanced U-Net model (FE). Input: Receives tensors with dimensions [Batch, 8, 128, 128], where Batch represents the batch size (i.e., the number of samples fed into the model in a single training iteration), 8 represents the number of input channels, and 128 represents the spatial dimension of the input tensor (i.e., the height and width of the gridded image).

[0057] Feature enhancement: The input data first passes through the Channel Feature Enhancement (CFE) Block, which uses two 3×3 convolutions and a channel attention module to expand the feature dimension to 32.

[0058] Encoding and Decoding: The encoder contains 4 levels of downsampling, and the decoder contains 4 levels of upsampling. Each level of upsampling and downsampling uses a dual-attention residual block (DARB) for feature extraction. At skip connections, an attention-gated AG module is used, employing a spatial attention map generated by a sigmoid activation function to filter shallow features.

[0059] Loss function optimization: During training, a masked mean squared error loss function based on Gaussian prior is used. First, a Gaussian convolution is performed on the ground truth labels to generate a mask. When calculating the loss, the weight coefficient α of the background region (Mask=0) is adjusted. w Set the weight to 0.05 and the weight of the non-background region to 1 to force the model to focus on the sparse thunderstorm core.

[0060] Step 3. Post-processing statistical bias calibration To objectively reveal the mapping relationship between model estimates and true values, this invention designs a bias diagnosis process based on positive sample binning statistics, which specifically includes three steps: (1) Positive Sample Selection The primary prerequisite for calibration is to eliminate zero-value background and focus on lightning events successfully captured by the model. The test set is defined as a collection of pixel-level data pairs. ,in, , Representing coordinates The true frequency and estimated value of each location. The index set of successfully detected positive samples. The results obtained from D are: ; in, This is the detection threshold.

[0061] (2) Bin-wise Statistical Profiling The actual frequency is divided into M intervals. For each interval, extract the subset of samples that fall into the k-th interval. And calculate the mean of the true values ​​within that interval. Compared with the mean of the estimates At the same time, calculate the 95% confidence interval of the estimated value. To assess volatility: ; ; ; in, The number of samples in the subset. This represents the estimated standard deviation within the subset.

[0062] (3) Calibration Modeling Based on the statistically obtained points, the mean of the true values The independent variable is the mean of the estimated value. As the dependent variable, the least squares method is used to fit the optimal mapping function. That is, to search make Minimize. The final calibration is performed only for the effective estimation region: ; The trained feature-enhanced U-shaped network model is used to predict the test set to obtain the original estimated values. Then perform the following calibration steps: Step A: Set the detection threshold Filter out ≥0.5 effective pixels; Step B: Subsequently, a binning statistical method is used to divide the true frequency into multiple intensity intervals. For each interval, a subset of positive samples falling within that interval is extracted, and the mean of the true frequency and the mean of the model estimate within that interval are calculated respectively; at the same time, the 95% confidence interval (95% CI) of the model estimate within that interval is calculated. Step C: Fit a linear calibration formula using the least squares method. Based on the training results of the dataset in this embodiment, the best fitting mapping function is: ; Step D: Apply the formula to the original output of the model to obtain the final calibrated co-estimation results of lightning location and frequency, thereby restoring the true physical intensity magnitude.

[0063] Figure 5 The image in the image clearly shows the cloud distribution characteristics against the background of Himawari-8 Band 9 (water vapor) brightness temperature. Figure 5 (a) in the diagram is a discrete lightning event distribution map; Figure 5 (b) in the figure is the continuous density response field generated by Gaussian convolution; Figure 5 (c) in the diagram is a binary attention mask generated by threshold truncation, which is used for weighting the loss function.

[0064] Figure 6 This is used to diagnose systematic biases in the model (before calibration) through binning statistics. The dashed line (Ideal y=x) in the figure represents the ideal unbiased prediction baseline; the blue solid line (Mean Pred) represents the mean trend of the original model output. This figure visually shows that the original model output is significantly lower than the ideal line, proving the necessity of introducing post-processing calibration. Figure 6 The extremely narrow shaded area surrounding the blue solid line represents the 95% confidence interval of the model estimate. Even in the extremely high-frequency range, this confidence interval remains narrow and steep, without feature collapse. This extremely high statistical stability is the core physical and mathematical basis for establishing the use of least squares for linear fitting (step 3.3).

[0065] Figure 7 The process of establishing the mapping function in the post-processing calibration module is shown. The scatter points represent the correspondence between the true mean (Y-axis) and the estimated mean (X-axis) in different bin intervals; the dashed line represents the best linear calibration function (y=kx+b) obtained by least squares fitting, which establishes the mapping parameters from the estimation space to the physical space.

[0066] Figure 8The diagram illustrates the beneficial effects of post-processing calibration following the implementation of this invention. The blue curve represents the original model response before calibration, showing saturation and underestimation. The red curve represents the final estimated trend after linear calibration. The black dashed line represents the ideal diagonal. The high degree of overlap between the red curve and the black dashed line proves that this invention successfully recovered the true physical magnitude of lightning frequency.

Claims

1. A method for co-estimating lightning location and frequency based on attention mechanism and bias calibration, characterized in that, Includes the following steps: Step 1: Construct a spatiotemporally aligned multi-source mapping dataset based on geostationary meteorological satellite data and lightning location network data; Step 2: Construct a feature-enhanced U-shaped network FE UNet based on the encoder-decoder structure to output a two-dimensional gridded lightning frequency distribution map; Step 3: Train the Feature Enhanced U-Net (FE UNet) model based on the multi-source mapping dataset to obtain the trained Feature Enhanced U-Net (FE UNet); The physical characteristics of the geostationary meteorological satellite to be estimated are input into the trained feature-enhanced U-shaped network FE UNet, which outputs a two-dimensional gridded lightning frequency distribution map as the result of the joint estimation of lightning location and frequency.

2. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 1, characterized in that, The multi-source mapping dataset mentioned in step 1 includes geostationary meteorological satellite physical characteristics obtained from geostationary meteorological satellite data and a two-dimensional gridded lightning frequency distribution map obtained from lightning location network data. The process of obtaining a two-dimensional gridded lightning frequency distribution map based on lightning location network data is as follows: Lightning location network data includes discrete lightning event sequences with occurrence times and latitude and longitude coordinates. The lightning events are aligned with time windows and projected into a grid coordinate system consistent with satellite images. The gridding process assigns each lightning event to a unique spatial grid point through a linear mapping from latitude and longitude to image coordinates and accumulates the counts, thereby generating a two-dimensional gridded lightning frequency distribution map.

3. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 2, characterized in that, The physical characteristics of the geostationary meteorological satellite include multi-band brightness temperature observations, dual-channel brightness temperature difference, and three-spectral brightness temperature difference; the multi-band brightness temperature observations include brightness temperature data obtained from the satellite's Band 8, Band 9, Band 10, and Band 13 channels; Band 8 refers to the satellite's 8th band, Band 9 refers to the satellite's 9th band, Band 10 refers to the satellite's 10th band, and Band 13 refers to the satellite's 13th band; The dual-channel brightness temperature difference includes BTD0910, BTD1013, and BTD1513, where BTD0910 = Band 10 brightness temperature - Band 9 brightness temperature, BTD1013 = Band 10 brightness temperature - Band 13 brightness temperature, and BTD1513 = Band 15 brightness temperature - Band 13 brightness temperature. Band 15 refers to the 15th band of the satellite. The brightness temperature difference of the three spectra is expressed as (Band11 brightness temperature - Band13 brightness temperature) - (Band13 brightness temperature - Band15 brightness temperature), where Band 11 refers to the 11th band of the satellite; The two-dimensional gridded lightning frequency distribution map in the multi-source mapping dataset is used as the label during the training of the feature-enhanced U-shaped network FEUNet.

4. The method for co-estimating lightning location and frequency based on attention mechanism and bias calibration according to claim 1, characterized in that, The Feature Enhancement U-Net (FE UNet) includes a Channel Feature Enhancement Module (CFE Block), an encoder, a decoder, an Attention Gated (AG) Module, and a Lightning Feature Enhancement Structure (LE Block). The Channel Feature Enhancement (CFE) Block expands the dimensionality of the input features. Multiple encoders perform downsampling at each stage, and the decoder performs upsampling at each stage. An attention-gated (AG) module is integrated at the jump connection between the encoder and decoder. The output of the final decoder is connected to the Lightning Feature Enhancement (LE) Block.

5. The lightning location and frequency co-estimation method based on attention mechanism and bias calibration according to claim 4, characterized in that, Both the encoder and decoder employ a dual attention residual module (DARB). The dual attention residual module (DARB) adopts a three-segment cascaded feature evolution structure, which includes: a residual convolutional block, a mask space attention unit (MAM), and a channel attention module (CAM). The residual convolutional block contains a main path and bypass jump connections. The input features are processed by continuous convolution operations in the main path to extract local features, and then fused with the original features passed by the bypass jump connections element by element. After the fused features are activated by the ReLU function, they are sequentially passed through the Mask Space Attention Unit (MAM) and the Channel Attention Unit (CAM) to perform global spatial context awareness and secondary calibration of the feature weights in the channel dimension, respectively.

6. The lightning location and frequency co-estimation method based on attention mechanism and bias calibration according to claim 4, characterized in that, The Channel Feature Enhancement Module (CFE Block) consists of two intermediate processing layers connected in series with the Channel Attention Module (CAM). The input physical features are sequentially processed through two intermediate processing layers containing 3×3 convolution, batch normalization and ReLU activation function for high-dimensional mapping, and then the channel attention module CAM performs adaptive calibration of the importance of physical channels. The channel attention module (CAM) includes parallel global max pooling and global average pooling layers, as well as a multilayer perceptron (MLP) with shared weights. The input features are processed by the global max pooling and global average pooling layers to generate two spatially compressed feature vectors. These two feature vectors are then independently input into the MLP for dimensionality reduction and expansion, respectively. Subsequently, the two channel feature vectors output by the MLP are element-wise added and fused, and then normalized using a sigmoid activation function to generate a channel weight tensor. Finally, the channel weight tensor is multiplied channel-wise by the input features of the CAM to obtain the calibrated output features.

7. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 5, characterized in that, The Mask Space Attention Unit (MAM) performs global spatial context awareness and feature optimization on the input features, and finally outputs a feature map. The specific processing flow is as follows: First, the input features are flattened into one-dimensional sequence features in the spatial dimension. Then, the one-dimensional sequence features are linearly mapped to generate query Q, key K, and value V features, which are then split and fed into a multi-head attention mechanism for processing. Within each attention head, the matrix product of the query and key is calculated to evaluate the global correlation between features. Then, a learnable mask bias matrix is ​​superimposed, and attention weights are obtained by normalization using the Softmax function. Subsequently, the attention weights are multiplied with the value V features to complete information weighted fusion. After that, the results of each attention head are reassembled and integrated in the channel dimension. The initial spatial resolution of the mask space attention unit (MAM) is restored through reshaping and bilinear interpolation upsampling operations. Then, the features are added element-wise with the features input to the mask space attention unit (MAM) through a long-range residual connection to obtain the fused features. Finally, the fused features enter the feedforward neural network (FFN), which undergoes linear expansion that doubles the channel dimension, GELU activation function processing, Dropout operation, and channel compression. Combined with the addition of the local residual connections at the end, a feature map is finally generated and output.

8. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 5, characterized in that, The attention-gated (AG) module uses the deep features upsampled by the decoder as a gating signal to perform pixel-level weighted filtering on the shallow features of the encoder in the skip connection, so as to filter the background noise in the non-convection region. The specific selection process is as follows: First, the shallow features from the encoder are concatenated with the corresponding upsampled deep features from the decoder. Second, the concatenated features are passed through a 1×1 convolutional layer and a sigmoid activation function to generate a spatial attention weight map α with values ​​ranging from 0 to 1. Next, the input shallow features from the encoder are multiplied element-wise with the spatial attention weight map α to obtain purified shallow features after suppressing background noise. Finally, the purified shallow features are concatenated with the input upsampled deep features from the decoder again as the input to the next level of the decoder.

9. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 4, characterized in that, The lightning feature enhancement structure LE Block adopts a deep cascaded architecture, which sequentially includes a first intermediate processing layer, a channel attention module CAM, a second intermediate processing layer, and an output convolutional layer. Both the first and second intermediate processing layers consist of a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function connected in series; the output convolutional layer is a 1×1 convolutional layer; the high-dimensional features input to the LE Block are sequentially processed by the feature mapping of the first intermediate processing layer, the adaptive calibration of the channel weights of the channel attention module CAM, and the depth smoothing and dealiasing of the second intermediate processing layer, and then mapped by the output convolutional layer into a single-channel continuous lightning frequency estimate.

10. The method for co-estimation of lightning location and frequency based on attention mechanism and bias calibration according to claim 1, characterized in that, The feature-enhanced U-shaped network FE UNet is trained based on the mask loss function of Gaussian prior, and the two-dimensional gridded lightning frequency distribution map output in step 3 is post-processed for statistical bias calibration. During training, the two-dimensional gridded lightning frequency distribution map is first transformed into a continuous Gaussian density field to generate a binary attention mask. Then, a weight coefficient α is assigned to the background region when calculating the loss. w , satisfying 0 < α w ≤ 0.1 forces the Feature Enhancement U-Net (FE UNet) to focus on the core region of thunderstorms; The post-processing statistical bias calibration steps are as follows: a detection threshold is set to screen positive samples in the lightning frequency distribution map output by the Feature Enhancement U-Net (FE UNet) to obtain a subset of positive samples; the actual lightning frequency is divided into multiple intensity intervals using a binning statistical method, and the correspondence between the mean of the actual frequency and the mean of the model estimate in each interval is calculated to establish a mapping trend; A linear calibration function was fitted using the least squares method, and the lightning frequency distribution map output by the model was calibrated. When establishing the mapping trend, the method further includes: calculating the 95% confidence interval of the model output in each intensity interval based on the extracted positive sample subset; The distribution width of the 95% confidence interval was used as a statistical stability evaluation index to verify the high confidence level of the model in extracting the relative intensity relationship features of lightning activity.

Images