An artificial intelligence-based hail prediction method, device and electronic equipment

Abstract

The embodiment of the application discloses a hail prediction method and device based on artificial intelligence and electronic equipment; the embodiment of the application can encode radar echo sequences and text prompt strings to obtain text embedding features and multi-dimensional radar sequence visual embedding features; the text embedding features are cross-modal aligned with the multi-dimensional radar sequence visual embedding features to obtain semantic enhancement features; the multi-dimensional radar sequence visual embedding features are conditionally modulated by using affine parameters to obtain modulated visual features, and the modulated visual features are fused with the semantic enhancement features to obtain cross-modal fusion features; the cross-modal fusion features are time-series modeled to obtain a hail prediction result, and the accuracy and robustness of early hail identification are improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, specifically to an artificial intelligence-based hail prediction method, device, and electronic device. Background Technology

[0002] Meteorological disaster monitoring and early warning are of great significance to agricultural production, power grid safety, and transportation operations. Hail, as a typical severe convective disaster, is characterized by its sudden onset, significant spatial locality, and rapid evolution. In modern meteorological observation systems, satellite cloud images, reanalysis data, and numerical weather prediction products can provide macroscopic circulation and radiation / thermal information, but they are usually difficult to analyze the fine structure and rapid evolution of condensates within convective systems. Ground-based weather radar, with its advantages of high spatiotemporal resolution, continuous observation, and sensitivity to condensates, is an important data source for hail monitoring and short-term early warning.

[0003] During the incipient stage of hail, radar echoes are often characterized by weak intensity, small scale, fragmented structure, and rapid changes. Furthermore, differences in terrain, climate, weather systems, and radar systems across different regions can lead to significant shifts in the statistical distribution of echoes, resulting in two prominent issues:

[0004] 1. Modeling the early evolution of weak signals is difficult: relying solely on two-dimensional spatial features is easily overwhelmed by background noise or non-convective echoes, making it difficult to stably accumulate precursor information within a long time window.

[0005] 2. Regional heterogeneity leads to insufficient generalization: The echo intensity distribution, morphology and noise characteristics vary significantly across regions. A single time-series backbone network is prone to performance degradation when the training / deployment regions are inconsistent, resulting in missed and false alarms.

[0006] Existing research often improves representation capabilities by increasing network structural complexity (such as ConvLSTM, TCN, Transformer, or video encoders), but it still struggles to fundamentally solve the coupling problem of "weak features + regional distribution bias." In particular, Transformer-type models exhibit quadratic complexity growth with sequence length, hindering efficient deployment in business scenarios with minute-level updates and long window durations. Therefore, there is an urgent need for a hail prediction (early hail identification) method that can model time series data with linear complexity and explicitly incorporate regional / seasonal / weather contextual semantics to enhance cross-domain robustness. Summary of the Invention

[0007] This application proposes an artificial intelligence-based hail prediction method, apparatus, electronic device, and storage medium, which can improve the robustness of hail identification.

[0008] This application provides an artificial intelligence-based hail prediction method, including:

[0009] The radar echo sequence is acquired, and the radar echo sequence is processed to obtain a time-series sequence sample and a sequence-level label corresponding to the time-series sequence sample.

[0010] Obtain the metadata set corresponding to the time series sample, and convert the metadata set into a text prompt string;

[0011] The text prompt string is encoded to obtain text embedding features, and affine parameters for conditional modulation are generated based on the text embedding features.

[0012] Each frame of radar image in the time sequence sample is encoded to obtain multidimensional radar sequence visual embedding features;

[0013] The text embedding features are aligned across modalities with the multidimensional radar sequence visual embedding features to obtain semantically enhanced features;

[0014] The visual embedding features of the multidimensional radar sequence are conditionally modulated using the affine parameters to obtain modulated visual features, and the modulated visual features are fused with the semantic enhancement features to obtain cross-modal fusion features.

[0015] The cross-modal fusion features are then used for time-series modeling to obtain hail prediction results.

[0016] Accordingly, embodiments of this application also provide an artificial intelligence-based hail prediction device, including:

[0017] The data processing unit is used to acquire radar echo sequences, process the radar echo sequences, and obtain time-series sequence samples. and the sequence-level labels corresponding to the time-series sequence samples;

[0018] A conversion unit is used to obtain a metadata set corresponding to the time series sample and convert the metadata set into a text prompt string.

[0019] The first encoding unit is used to encode the text prompt string to obtain text embedding features, and generate affine parameters for conditional modulation based on the text embedding features.

[0020] The second encoding unit is used for encoding the time-series sequence samples. Each frame of radar image in the image is encoded to obtain multidimensional radar sequence visual embedding features;

[0021] An alignment unit is used to perform cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantically enhanced features;

[0022] The modulation unit is used to conditionally modulate the visual embedding features of the multidimensional radar sequence using the affine parameters to obtain modulated visual features, and then fuse the modulated visual features with the semantic enhancement features to obtain cross-modal fusion features.

[0023] The temporal modeling unit is used to perform temporal modeling on the cross-modal fusion features to obtain hail prediction results.

[0024] In some embodiments, the first encoding unit may include:

[0025] The first encoding subunit is used to encode the text prompt string using a preset hail prediction model to obtain a word-level embedding matrix. and global convergence vector;

[0026] The first parsing subunit is used to parse the text prompt string to obtain the word index set corresponding to each phrase in the text prompt string;

[0027] The first aggregation subunit is used to aggregate the set of word indexes corresponding to each phrase to obtain the text embedding features;

[0028] The first alignment subunit is used to align the global convergence vector and the text embedding features in spatial dimensions through the first projection network and the second projection network to obtain global semantic features.

[0029] The first generation subunit is used to input the global semantic features into the first affine parameter generation network and the second affine parameter generation network respectively to generate scaling parameters and translation parameters for conditional modulation.

[0030] In some embodiments, the second encoding unit may include:

[0031] Dividing into sub-units for each frame of radar image in the time-series sample. The image is divided into blocks to obtain at least one image block;

[0032] The first mapping subunit is used to flatten the at least one image block and map it to the initial embedding space through the first linear projection layer to obtain the image block embedding sequence of each frame of radar image.

[0033] The second encoding subunit is used to encode the image block embedding sequence using a preset hail recognition model to obtain image block encoding features;

[0034] The second mapping subunit is used to map the image block coding features to the shared latent space through the second linear projection layer to obtain the visual block features of each frame of radar image;

[0035] The first stacking sub-unit is used to divide the visual block features of each frame of radar image into... The multidimensional radar sequence visual embedding features are obtained by stacking them along the time dimension.

[0036] In some embodiments, the alignment unit may include:

[0037] The first acquisition subunit is used to acquire the mapped text embedding features corresponding to the text embedding features;

[0038] The second acquisition subunit is used to acquire the visual embedding sub-feature corresponding to each dimension of the radar sequence visual embedding feature in the multi-dimensional radar sequence visual embedding feature;

[0039] The first calculation subunit is used to calculate the similarity between the mapped text embedding feature and each visual embedding sub-feature at the current encoding resolution for each of the L encoding resolutions.

[0040] The second computational subunit is used to calculate the attention distribution based on the similarity;

[0041] The first aggregation subunit is used to aggregate the attention distribution with the visual embedding features of each dimension of radar sequence using a preset hail prediction model to obtain the semantic enhancement features.

[0042] In some embodiments, the modulation unit may include:

[0043] The normalization subunit is used to normalize the radar sequence visual embedding features of each coding resolution in the multidimensional radar sequence visual embedding features to obtain normalized features.

[0044] A modulation subunit is used to conditionally modulate the normalized features using the scaling and translation parameters to obtain modulated visual features.

[0045] The fusion subunit is used to fuse the modulated visual features with the semantic enhancement features to obtain the cross-modal fusion features.

[0046] In some embodiments, the timing modeling unit may include:

[0047] The temporal modeling subunit is used to input the cross-modal fusion features of the corresponding encoding resolution along the time dimension into the Mamba selective state space module for temporal dependency modeling, and obtain the temporal modeling output features:

[0048] The upsampling subunit is used to upsample the temporal modeling output features of each coding resolution to the spatial grid resolution, so as to obtain the upsampled temporal modeling output features.

[0049] The pooling subunit is used to perform global pooling of the upsampled temporal modeling output features along the spatial and temporal dimensions to obtain a sequence-level description vector.

[0050] Prediction subunit, used to convert the sequence-level description vector Input the preset hail prediction model to obtain the hail prediction results.

[0051] In some embodiments, the AI-based hail prediction device may further include:

[0052] Acquire the hail prediction model to be trained, the text prompt training set, and the radar echo training set;

[0053] The hail prediction model to be trained is used to perform prediction processing on the text prompt training set and the radar echo training set to obtain image block coding training features, global semantic training features and model training results.

[0054] Global alignment loss information is constructed based on the image block encoding training features and the global semantic training features;

[0055] Construct model training loss information based on the model training results;

[0056] The hail prediction model to be trained is adjusted based on the global alignment loss information and the model training loss information to obtain a preset hail prediction model.

[0057] Accordingly, this application also provides an electronic device, which includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the artificial intelligence-based hail prediction methods provided in this application.

[0058] Accordingly, this application also provides a storage medium storing a computer program, which, when executed by a processor, implements any of the artificial intelligence-based hail prediction methods provided in this application.

[0059] This application embodiment can acquire radar echo sequences, process the radar echo sequences to obtain time-series sequence samples and corresponding sequence-level labels; acquire metadata sets corresponding to the time-series sequence samples and convert the metadata sets into text prompt strings; encode each frame of radar image in the time-series sequence samples to obtain multi-dimensional radar sequence visual embedding features; perform cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantic enhancement features; conditionally modulate the multi-dimensional radar sequence visual embedding features using affine parameters to obtain modulated visual features, and fuse the modulated visual features with the semantic enhancement features to obtain cross-modal fusion features; perform time-series modeling on the cross-modal fusion features to obtain hail prediction results, thereby improving the accuracy and robustness of early hail identification. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of a scenario for the hail prediction method based on artificial intelligence provided in an embodiment of this application;

[0062] Figure 2 This is a flowchart illustrating the artificial intelligence-based hail prediction method provided in an embodiment of this application;

[0063] Figure 3 This is a schematic diagram of a scenario for the hail prediction method based on artificial intelligence provided in an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of the structure of the artificial intelligence-based hail prediction device provided in the embodiments of this application;

[0065] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. However, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0067] This application proposes an AI-based hail prediction method, which can be executed by an AI-based hail prediction device, and this device can be integrated into an electronic device with a screensaver function. The electronic device may include at least one of a terminal and a server. That is, the AI-based hail prediction method can be executed by either a terminal or a server.

[0068] The terminal may include, but is not limited to, smartphones, tablets, laptops, personal computers (PCs), smart home appliances, wearable electronic devices, VR / AR devices, in-vehicle terminals, intelligent voice interaction devices, etc.

[0069] Among them, the server can be an interconnection server between multiple heterogeneous systems or a backend server, or it can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc.

[0070] In one embodiment, such as Figure 1 The hail prediction device based on artificial intelligence can be integrated into computer devices such as terminals or servers to implement the thunderstorm prediction method based on deep learning technology proposed in this application. Specifically, the server 11 or terminal 10 can acquire radar echo sequences, process the radar echo sequences to obtain time-series sequence samples and corresponding sequence-level labels; acquire metadata sets corresponding to the time-series sequence samples and convert the metadata sets into text prompt strings; encode each frame of radar image in the time-series sequence samples to obtain multi-dimensional radar sequence visual embedding features; perform cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantic enhancement features; conditionally modulate the multi-dimensional radar sequence visual embedding features using affine parameters to obtain modulated visual features, and fuse the modulated visual features with the semantic enhancement features to obtain cross-modal fusion features; perform time-series modeling on the cross-modal fusion features to obtain hail prediction results.

[0071] The following will provide a detailed description of each example. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0072] This application will describe embodiments from the perspective of an AI-based hail prediction device that can be integrated into an electronic device.

[0073] like Figure 1 As shown, an artificial intelligence-based hail prediction method is provided, and the specific process includes:

[0074] 101. Obtain the radar echo sequence, perform data processing on the radar echo sequence to obtain time series sequence samples and the corresponding sequence-level labels.

[0075] In some embodiments, radar observation data of the target area is acquired. The radar observation data includes: a radar mosaic product generated by mosaicking observation data from multiple weather radar stations, or a radar echo product obtained by parsing raw volume scan data output from a single-station weather radar. The radar mosaic product can cover a wider spatial area and is suitable for monitoring and early warning of large-scale hail weather; the single-station radar echo product has higher spatiotemporal resolution and more detailed local structural information, making it suitable for refined identification of specific areas.

[0076] In some embodiments, radar mosaics or single-station radar echo products (radar echo sequences) of the target area are acquired. Using composite reflectivity as the main input channel, each sample is constructed into a fixed-length time series, i.e., a time series sample:

[0077]

[0078] in, Refers to time series samples, This indicates the sample size. H and W refer to the sample's length and height.

[0079] In some embodiments, uniform preprocessing is performed on radar data, including but not limited to: regrinding / projection unification, time alignment and missing data processing, removal of negative and invalid values, and normalization to a uniform dynamic range, to ensure that data from different regions are in a learnable, unified input space. Simultaneously, sequence-level labels are established for each sample.

[0080]

[0081] in, Indicates early hail samples, This indicates a non-hail sample.

[0082] 102. Obtain the metadata set corresponding to the time series sample and convert the metadata set into a text prompt string.

[0083] Extract the metadata set corresponding to the sample from radar data or operational records:

[0084]

[0085] in, Represents regional / geographical information. Indicates season or month information. It indicates the weather background (such as semantic descriptions of cold vortex, shear, topographic effects, etc.). This indicates the observation / recording time period. Based on a predefined prompt template function, the metadata is deterministically populated into a structured text prompt string:

[0086]

[0087] Then, the prompt string is parsed into a set of phrases. This is used for subsequent phrase-level semantic-visual alignment.

[0088] The prompt template function is a predefined, rule-based mapping function, denoted as Template(M). It fills the input metadata set M (containing information such as region, season, weather background, and time period) into a complete natural language description, i.e., a structured text prompt string S, according to a fixed sentence template.

[0089] 103. Encode the text prompt string to obtain text embedding features, and generate affine parameters for conditional modulation based on the text embedding features.

[0090] In some embodiments, the affine parameters include scaling parameters. Translation parameters .

[0091] In some embodiments, the step "for the text prompt string" Encode to obtain text embedding features "and generate affine parameters for conditional modulation based on text embedding features", including:

[0092] Using a pre-set hail prediction model to analyze text prompt strings Encode to obtain a word-level embedding matrix. and global convergence vector ;

[0093] The text prompt string is parsed to obtain the phrases within it. Corresponding lexical index set ;

[0094] Each phrase Corresponding lexical index set Aggregate to obtain text embedding features ;

[0095] Global convergence vector and text embedding features Spatial dimension alignment is performed using a first projection network and a second projection network to obtain global semantic features. ;

[0096] global semantic features The inputs are respectively fed into the first affine parameter generation network. Second Affine Parameter Generation Network Generate scaling and translation parameters for conditional modulation. , .

[0097] The preset hail prediction model is a visual-language model for early hail identification. This visual-language model can be a Convolutional Neural Network (CNN), a De-Convolutional Network (DN), a Deep Convolutional Inverse Graphics Network (DCIGN), a Region-based Convolutional Network (RCNN), or a Faster Region-based Convolutional Network (Faster RCNN), etc.

[0098] The structure of some pre-defined hail prediction models can be as follows: Figure 3 As shown.

[0099] In some embodiments, a frozen text encoder in a preset hail prediction model is used. The prompt text is encoded to obtain a word-level embedding matrix. With global convergence vector :

[0100]

[0101] in, This refers to pooling operations. The word-level embedding matrix is ​​a real matrix with dimension 1. . It can represent the number of lexical units. It can represent the embedding dimension.

[0102] In some embodiments, a frozen text encoder in a pre-trained vision-language model is employed. The text prompt string S is encoded to obtain a word-level embedding matrix. The dimension of this matrix is ,in This indicates the number of tokens obtained after word segmentation of the text prompt string. This represents the dimension of the lexical embedding vector output by the pre-trained text encoder.

[0103] lexical embedding matrix Perform pooling operations along the word dimension to obtain the global convergence vector. This global convergence vector The dimension is , which is used to characterize the global semantic information of the text prompt string.

[0104] Then, for each phrase corresponding Index Set Perform phrase-level aggregation:

[0105]

[0106] To share latent space dimensions with visual branches Alignment, introducing two layers of MLP projection:

[0107]

[0108]

[0109] In some embodiments, the text global convergence vector Passing through the first linear layer in sequence Nonlinear activation function Second linear layer The transformation yields the projected global semantic features. This feature is mapped to the same shared latent space as the visual branch, and is used for subsequent conditional modulation parameter generation and global alignment loss calculation.

[0110] In some embodiments, the aggregated embedding vector of the j-th phrase Passing through the first linear layer in sequence Nonlinear activation function Second linear layer The transformation yields the projected phrase semantic features. This feature is mapped to the same shared latent space as the visual branch, and is used for subsequent fine-grained phrase-patch alignment, enabling phrase semantics to guide attention to visual chunks.

[0111] In some embodiments, from global semantics Generate affine parameters for conditional modulation:

[0112]

[0113] in, , Will be broadcast to time dimension The model uses conditional normalization modulation of visual features to explicitly inject prior information such as region and weather background into the model.

[0114] In some embodiments, the projected global semantic features The parameters are fed into the first affine parameter generation network and the second affine parameter generation network, respectively, to generate scaling parameters for conditional modulation. Translation parameters The scaling and translation parameters will be broadcast to the temporal and image block dimensions of the radar sequence visual embedding features, and used to scale and translate the normalized visual features element by element, thereby achieving adaptive correction of the visual representation by the text prior.

[0115] 104. Encode each frame of radar image in the time-series sample to obtain multi-dimensional radar sequence visual embedding features.

[0116] In some embodiments, the step "on time series samples" Each frame of radar image in the sequence is encoded to obtain "multi-dimensional radar sequence visual embedding features", which may include:

[0117] For each frame of radar image in the time-series sample Perform image block partitioning to obtain at least one image block. ;

[0118] Divide at least one image into blocks After flattening, it passes through the first linear projection layer Mapped to the initial embedding space This yields the image block embedding sequence for each frame of radar image. ;

[0119] Image blocks are embedded using a pre-defined hail recognition model. Encode the image to obtain image block coding features. ;

[0120] Through the second linear projection layer Encode features by dividing the image into blocks Mapped to the shared latent space, each radar image is obtained. Visual segmentation features ;

[0121] Each frame of radar image Visual segmentation features Stacking along the time dimension yields multidimensional radar sequence visual embedding features. .

[0122] In some embodiments, for each frame of radar image Perform non-overlapping patch partitioning and linear projection:

[0123]

[0124]

[0125] in, This represents the radar image in frame t. Image segmentation refers to the process of dividing an entire image into multiple non-overlapping local image blocks. This indicates the number of image blocks. This represents the block side length. C represents the number of input channels. This represents the feature matrix after differentiation.

[0126] in, The linear projection weight matrix refers to the matrix that maps the pixel vector of each block to the embedding space.

[0127] In some embodiments, The frozen visual encoder of the pre-set hail prediction model is input. Obtain image block coding features The images are then projected onto a shared latent space to obtain individual radar frames. Visual segmentation features :

[0128]

[0129]

[0130] in, Refers to the spatial projection weight matrix.

[0131] Then, all time steps are stacked to obtain the multidimensional radar sequence visual embedding features:

[0132]

[0133] In some embodiments, a multi-scale pyramid representation can also be constructed along the patch dimension. Hierarchical representation is achieved through downsampling / pooling and a lightweight MLP to simultaneously capture fine-grained texture and high-level structural semantics. Specifically, based on the L encoding resolutions of a pre-defined hail recognition model, time-series samples are processed. Each frame of the radar image is encoded to obtain multidimensional radar sequence visual embedding features. ;

[0134] 105. Cross-modal alignment of text embedding features with multidimensional radar sequence visual embedding features yields semantically enhanced features.

[0135] In some embodiments, the step of "aligning the text embedding features with the radar sequence visual embedding features across modalities to obtain semantically enhanced features" may include:

[0136] Obtaining text embedding features Corresponding mapped text embedding features ;

[0137] Obtain the visual embedding sub-features corresponding to each dimension of the radar sequence visual embedding features in the multi-dimensional radar sequence visual embedding features. ;

[0138] For each of the L encoding resolutions, compute the mapped text embedding features. Visual embedding sub-features at the current encoding resolution Similarity between ;

[0139] Based on similarity Calculate attention distribution ;

[0140] The attention distribution is achieved using a pre-defined hail prediction model. Visual embedding features of radar sequences in various dimensions The semantic enhancement features are obtained by aggregation. .

[0141] At each scale Above, computational phrase embeddings and vision The similarity is normalized to the attention distribution:

[0142]

[0143] use By semantically weighting visual features, semantically enhanced features are obtained:

[0144]

[0145] 106. Conditionally modulate the visual embedding features of multidimensional radar sequences using affine parameters to obtain modulated visual features, and then fuse the modulated visual features with semantic enhancement features to obtain cross-modal fusion features.

[0146] In some embodiments, affine parameters include scaling parameters and translation parameters. , ;

[0147] In some embodiments, the step "conditionally modulates the visual embedding features of the multidimensional radar sequence using affine parameters to obtain modulated visual features" The modulated visual features are then fused with the semantically enhanced features to obtain cross-modal fusion features. ", can include:

[0148] Visual embedding features of multidimensional radar sequences Visual embedding features of radar sequences at various coding resolutions Normalization is performed to obtain normalized features;

[0149] Conditional modulation of normalized features is performed using scaling and translation parameters to obtain modulated visual features. ;

[0150] Modulated visual features With semantic enhancement features Perform fusion to obtain cross-modal fusion features .

[0151] Specifically, visual embedding features of radar sequences First normalize, then use text-generated data. Perform conditional modulation:

[0152]

[0153] in, Refers to normalization processing. Visual characteristics after fingering.

[0154] Subsequently, the modulated visual features With semantic enhancement features To perform fusion (e.g., element-wise addition):

[0155]

[0156] in, Refers to the semantic enhancement features at scale l and time step t. Feature fusion operation

[0157] This refers to stacking operations along the time dimension.

[0158] In some embodiments, the modulated visual features at scale l and time step t are... With semantic enhancement features Perform fusion (preferably element-wise addition) to obtain cross-modal fusion features. Fuse cross-modal features across T time steps. Stacked along the time dimension, forming a complete temporal feature sequence. Its dimensions are This is used for subsequent time-series Mamba modeling.

[0159] This step enables adaptive correction of visual representations by interpretable priors such as "region / season / weather background," thereby alleviating the generalization problem caused by cross-regional distribution shift.

[0160] 107. Perform time-series modeling on the cross-modal fusion features to obtain hail prediction results.

[0161] In some embodiments, the step "temporally modeling the cross-modal fusion features to obtain hail prediction results" ", can include:

[0162] For each encoding resolution, the cross-modal fusion features of the corresponding resolution are... Temporal dependency modeling is performed by inputting the Mamba selective state-space module along the time dimension, and the temporal modeling output features are obtained. :

[0163] Temporal modeling of each coding resolution outputs features Upsampling to spatial grid resolution yields the upsampled temporal modeling output features. ;

[0164] The upsampled temporal modeling output features are then globally pooled along the spatial and temporal dimensions to obtain sequence-level description vectors. :

[0165] sequence-level description vectors Input a preset hail prediction model to obtain hail prediction results. .

[0166] Specifically, for each scale Along the time dimension Temporal modeling is performed by applying the Mamba Selective State Space Module (SSM), which involves fusing cross-modal features at the l-th scale. Input to the corresponding Mamba selective state space module Temporal modeling is performed along the time dimension to obtain the output features of the temporal modeling. :

[0167]

[0168] Among them, the Selective State Space Model (SelectiveSSM) is the core component of the Mamba framework. It is a new type of sequence modeling architecture designed to solve the efficiency bottleneck problem of traditional sequence models (such as RNN and Transformer) when processing long sequences.

[0169] in, This refers to the output of the l-th scale time-series modeling features. The term refers to the Mamba selective state-space module at scale l. T refers to the number of time steps; in this application, T=7 means that each sample contains 7 radar images. The l-th scale refers to the number of image blocks. The spatial resolution varies at different scales; coarse-scale images have fewer blocks, while fine-scale images have more. d refers to the feature dimension.

[0170] In some embodiments, the multi-scale output is upsampled back to grid resolution and global pooled to obtain the sequence description vector, which is the multi-scale temporal modeling output. The spatial resolution of the original radar image is restored through upsampling, and then fused to obtain the spatiotemporal feature tensor. Its dimensions are :

[0171]

[0172] in, The spatiotemporal feature tensor after upsampling. Upsampling operation.

[0173] In some embodiments, an early hail probability is output by a classifier, and then a hail prediction result can be obtained from the early hail probability:

[0174]

[0175] That is, the sequence-level description vector The input is fed into a classifier, mapped by the classifier, and activated by a sigmoid function. The output is the probability of early hail detection. The early hail identification probability p is compared with a preset threshold τ. If p > τ, it is predicted as an early hail sample; otherwise, it is predicted as a non-hail sample.

[0176] In some embodiments, the method proposed in this application further includes:

[0177] Acquire the hail prediction model to be trained, the text prompt training set, and the radar echo training set;

[0178] The hail prediction model to be trained is used to perform prediction processing on the text prompt training set and the radar echo training set to obtain image block encoding training features. Global semantic training features and model training results ;

[0179] Feature training based on image block encoding and global semantic training features Constructing global alignment loss information ;

[0180] Based on model training results Construct model training loss information;

[0181] The pre-set hail prediction model is obtained by adjusting the hail prediction model to be trained based on the global alignment loss information and the model training loss information.

[0182] In some embodiments, the step "training features based on image block encoding" and global semantic training features Constructing global alignment loss information ", can include:

[0183] The hail prediction model to be trained is used to train features by encoding image blocks. Pooling is performed to obtain pooled image block training features. ;

[0184] Based on a pre-defined global alignment loss function, global semantic training features are used. Image block encoding training features after pooling Perform consistency constraint calculations to obtain global alignment loss information. .

[0185] Specifically, to enhance global semantic consistency, a global alignment loss (CLIP style contrast consistency constraint) is introduced to align the global cue representation with the frame-level converged representation:

[0186]

[0187] in, This application calculates global text features. With each frame's visual features The dot product similarity, after temperature coefficient After scaling, the data is normalized to a probability distribution using Softmax, and then averaged over all frames after taking the negative logarithm to obtain the global alignment loss. Minimizing this loss can help maintain consistency between the global semantics of the text and the visual semantics of each frame in the sequence within the shared space.

[0188] In some embodiments, for imbalanced and weak precursor signals, the present invention employs sequence-level Focal Loss:

[0189]

[0190] in, It is the predicted probability of the i-th sample. The early hail recognition probability is output by the model. Let be the true label of the i-th sample. Let B represent the batch size, i.e., the number of samples in each training batch. For each of the B samples in a batch, calculate the focus loss. Then, the average value is taken to obtain the sequence-level focus loss. This loss is achieved through the balance factor. To address the imbalance between positive and negative samples, focus parameters are used. The loss weights of easily classified samples are reduced, causing the model to focus more on early hail samples that are difficult to classify. The balancing factor is used to balance the weights of positive and negative samples, typically set to 0.25 or adaptively adjusted according to the sample distribution. The focusing parameter controls the strength of the modulation factor, typically set to 2; a larger value results in a more significant reduction in weighting for easily classified samples.

[0191] The binary Focal form is as follows:

[0192]

[0193] The total loss is:

[0194]

[0195] The total loss function consists of a weighted sum of three terms: the first term is the sequence-level focus loss. The first term is used to handle class imbalance and focuses on early hail samples that are difficult to classify; the second term is the global alignment loss. The first term is used to constrain the consistency between the global semantics of the text and the visual semantics of each frame in the shared space; the third term is a weight decay regularization term. This is used to prevent overfitting. The balance coefficient is adjusted... , , The relative importance of each component can be controlled. During training, only the parameters of the cross-modal alignment module, conditional modulation module, temporal Mamba module, and classifier are optimized, while the parameters of the pre-trained text encoder and visual encoder remain frozen.

[0196] During training, the pre-trained text encoder and visual encoder are frozen, and only the cross-modal alignment module, conditional modulation module, Temporal Mamba module, and classification / localization head parameters are optimized.

[0197] In one embodiment, such as Figure 3 As shown, metadata including Location, Weather, Time, Status, Radar information, Max / Min can be input into a preset hail prediction model. Additionally, radar sequences can also be input. Then, the metadata is processed using a predefined cue template function to generate text cue strings, and a set of phrases is parsed out. The extracted phrases are then fed into the "encoding module" (a frozen pre-trained text encoder) to obtain word-level embeddings and phrase-level embeddings. These embeddings will be used for subsequent cross-modal alignment and conditional modulation.

[0198] For radar sequences, the segmented radar image sequence can be input into a frozen pre-trained visual encoder to obtain visual segmentation features. Then, a multi-scale pyramid representation is constructed along the spatial dimension of the visual features. Hierarchical representation is achieved through downsampling / pooling and a lightweight MLP to simultaneously capture fine-grained texture and high-level structural semantics. Inside the multi-scale denoiser, spatial resolution is restored through upsampling, and more macroscopic features are extracted through downsampling to form a feature pyramid.

[0199] Optionally, visual features can be further transformed into visual tags aligned with text "terms" to facilitate similarity calculation. Furthermore, phrase embeddings and visual tags are finely aligned across multiple scales to obtain semantically enhanced features. Simultaneously, the global semantic generation affine parameters (γ, β) extracted from the text branch are used to conditionally modulate the visual features, and then the modulated features are fused with the semantically enhanced features to obtain cross-modal fusion features.

[0200] Optionally, the fused features are input along the time dimension into the Mamba selective state space module for linear time-varying temporal modeling to capture the long-range dependencies of hail evolution. After the temporal modeling output, spatial details are recovered through upsampling, and then the early hail probability is output through a classifier.

[0201] In some embodiments, Figure 3 The areas marked "frozen" indicate that the parameters of the pre-trained text encoder and visual encoder remain unchanged during training. Figure 3 The blue arrows indicate the data flow direction during forward propagation. An end-to-end training approach is adopted, optimizing the parameters of the cross-modal alignment module, conditional modulation module, temporal Mamba module, and classifier. The loss function includes sequence-level focus loss, global alignment loss, and a regularization term.

[0202] This application proposes an artificial intelligence-based hail prediction method. This method acquires radar echo sequences, processes the radar echo sequences to obtain time-series sequence samples and their corresponding sequence-level labels; acquires metadata sets corresponding to the time-series sequence samples and converts these metadata sets into text prompt strings; performs visual encoding on each frame of the radar image in the time-series sequence samples to obtain multi-dimensional radar sequence visual embedding features; performs cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantic enhancement features; conditionally modulates the multi-dimensional radar sequence visual embedding features using affine parameters to obtain modulated visual features, and fuses these modulated visual features with the semantic enhancement features to obtain cross-modal fusion features; and performs temporal modeling on the cross-modal fusion features to obtain hail prediction results, improving the accuracy and robustness of early hail identification.

[0203] Furthermore, this application proposes PGMamba, a cue-guided framework that integrates a temporal Mamba model for early hail identification. This method encodes region, season, case recording time, and other relevant information into textual cues and establishes an accurate correspondence between meteorological semantics and radar echoes. The representation fusing radar spatiotemporal features and semantic cues is input into a selective state-space model of Mamba, thereby achieving the accumulation and enhancement of hail signals in the temporal dimension. Experiments on multi-region datasets demonstrate that PGMamba significantly outperforms existing methods in classification accuracy and region generalization ability, while generating more interpretable prediction results.

[0204] To better implement the AI-based hail prediction method provided in this application, one embodiment also provides an AI-based hail prediction device, which can be integrated into an electronic device. The meanings of the terms used are the same as in the product testing and verification methods described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0205] In one embodiment, an artificial intelligence-based hail prediction device is provided, which can be integrated into an electronic device, such as a computer. Figure 4 As shown, the AI-based hail prediction device includes: a data processing unit 201, a conversion unit 202, a first encoding unit 203, a second encoding unit 204, an alignment unit 205, a modulation unit 206, and a timing modeling unit 207, as detailed below:

[0206] Data processing unit 201 is used to acquire radar echo sequences, perform data processing on the radar echo sequences, and obtain time-series sequence samples. and the sequence-level labels corresponding to the time-series sequence samples;

[0207] The conversion unit 202 is used to obtain the metadata set corresponding to the time series sample and convert the metadata set into a text prompt string;

[0208] The first encoding unit 203 is used to encode the text prompt string to obtain text embedding features, and generate affine parameters for conditional modulation based on the text embedding features.

[0209] The second encoding unit 204 is used to encode the time sequence sample. Each frame of radar image in the image is encoded to obtain multidimensional radar sequence visual embedding features;

[0210] Alignment unit 205 is used to perform cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantically enhanced features;

[0211] The modulation unit 206 is used to conditionally modulate the visual embedding features of the multidimensional radar sequence using the affine parameters to obtain modulated visual features, and to fuse the modulated visual features with the semantic enhancement features to obtain cross-modal fusion features.

[0212] The temporal modeling unit 207 is used to perform temporal modeling on the cross-modal fusion features to obtain hail prediction results.

[0213] In some embodiments, the first encoding unit 203 may include:

[0214] The first encoding subunit is used to encode the text prompt string using a preset hail prediction model to obtain a word-level embedding matrix. and global convergence vector;

[0215] The first parsing subunit is used to parse the text prompt string to obtain the word index set corresponding to each phrase in the text prompt string;

[0216] The first aggregation subunit is used to aggregate the set of word indexes corresponding to each phrase to obtain the text embedding features;

[0217] The first alignment subunit is used to align the global convergence vector and the text embedding features in spatial dimensions through the first projection network and the second projection network to obtain global semantic features.

[0218] The first generation subunit is used to input the global semantic features into the first affine parameter generation network and the second affine parameter generation network respectively to generate scaling parameters and translation parameters for conditional modulation.

[0219] In some embodiments, the second encoding unit 204 may include:

[0220] Dividing into sub-units for each frame of radar image in the time-series sample. The image is divided into blocks to obtain at least one image block;

[0221] The first mapping subunit is used to flatten the at least one image block and map it to the initial embedding space through the first linear projection layer to obtain the image block embedding sequence of each frame of radar image.

[0222] The second encoding subunit is used to encode the image block embedding sequence using a preset hail recognition model to obtain image block encoding features;

[0223] The second mapping subunit is used to map the image block coding features to the shared latent space through the second linear projection layer to obtain the visual block features of each frame of radar image;

[0224] The first stacking sub-unit is used to divide the visual block features of each frame of radar image into... The multidimensional radar sequence visual embedding features are obtained by stacking them along the time dimension.

[0225] In some embodiments, the alignment unit 205 may include:

[0226] The first acquisition subunit is used to acquire the mapped text embedding features corresponding to the text embedding features;

[0227] The second acquisition subunit is used to acquire the visual embedding sub-feature corresponding to each dimension of the radar sequence visual embedding feature in the multi-dimensional radar sequence visual embedding feature;

[0228] The first calculation subunit is used to calculate the similarity between the mapped text embedding feature and each visual embedding sub-feature at the current encoding resolution for each of the L encoding resolutions.

[0229] The second computational subunit is used to calculate the attention distribution based on the similarity;

[0230] The first aggregation subunit is used to aggregate the attention distribution with the visual embedding features of each dimension of radar sequence using a preset hail prediction model to obtain the semantic enhancement features.

[0231] In some embodiments, the modulation unit 206 may include:

[0232] The normalization subunit is used to normalize the radar sequence visual embedding features of each coding resolution in the multidimensional radar sequence visual embedding features to obtain normalized features.

[0233] A modulation subunit is used to conditionally modulate the normalized features using the scaling and translation parameters to obtain modulated visual features.

[0234] The fusion subunit is used to fuse the modulated visual features with the semantic enhancement features to obtain the cross-modal fusion features.

[0235] In some embodiments, the timing modeling unit 207 may include:

[0236] The temporal modeling subunit is used to input the cross-modal fusion features of the corresponding encoding resolution along the time dimension into the Mamba selective state space module for temporal dependency modeling, and obtain the temporal modeling output features:

[0237] The upsampling subunit is used to upsample the temporal modeling output features of each coding resolution to the spatial grid resolution, so as to obtain the upsampled temporal modeling output features.

[0238] The pooling subunit is used to perform global pooling of the upsampled temporal modeling output features along the spatial and temporal dimensions to obtain a sequence-level description vector.

[0239] Prediction subunit, used to convert the sequence-level description vector Input the preset hail prediction model to obtain the hail prediction results.

[0240] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0241] The accuracy of hail prediction can be improved by using the hail prediction device described above.

[0242] This application also provides an electronic device, which may include a terminal or a server. For example, the electronic device may serve as an AI-based hail prediction terminal, which may be a computer, etc.; or, for example, a computer device may serve as a server, such as an AI-based hail prediction server. Figure 5 As shown, it illustrates the structural diagram of the terminal involved in the embodiments of this application, specifically:

[0243] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 5The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0244] The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0245] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0246] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0247] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0248] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0249] The radar echo sequence is acquired, and the radar echo sequence is processed to obtain a time-series sequence sample and a sequence-level label corresponding to the time-series sequence sample.

[0250] Obtain the metadata set corresponding to the time series sample, and convert the metadata set into a text prompt string;

[0251] The text prompt string is encoded to obtain text embedding features, and affine parameters for conditional modulation are generated based on the text embedding features.

[0252] Each frame of radar image in the time sequence sample is encoded to obtain multidimensional radar sequence visual embedding features;

[0253] The text embedding features are aligned across modalities with the multidimensional radar sequence visual embedding features to obtain semantically enhanced features;

[0254] The visual embedding features of the multidimensional radar sequence are conditionally modulated using the affine parameters to obtain modulated visual features, and the modulated visual features are fused with the semantic enhancement features to obtain cross-modal fusion features.

[0255] The cross-modal fusion features are then used for time-series modeling to obtain hail prediction results.

[0256] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0257] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0258] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0259] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute steps in any of the artificial intelligence-based hail prediction methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0260] The radar echo sequence is acquired, and the radar echo sequence is processed to obtain a time-series sequence sample and a sequence-level label corresponding to the time-series sequence sample.

[0261] Obtain the metadata set corresponding to the time series sample, and convert the metadata set into a text prompt string;

[0262] The text prompt string is encoded to obtain text embedding features, and affine parameters for conditional modulation are generated based on the text embedding features.

[0263] Each frame of radar image in the time sequence sample is encoded to obtain multidimensional radar sequence visual embedding features;

[0264] The text embedding features are aligned across modalities with the multidimensional radar sequence visual embedding features to obtain semantically enhanced features;

[0265] The visual embedding features of the multidimensional radar sequence are conditionally modulated using the affine parameters to obtain modulated visual features, and the modulated visual features are fused with the semantic enhancement features to obtain cross-modal fusion features.

[0266] The cross-modal fusion features are then used for time-series modeling to obtain hail prediction results.

[0267] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0268] Since the computer program stored in the storage medium can execute the steps in any of the artificial intelligence-based hail prediction methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the artificial intelligence-based hail prediction methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0269] The above provides a detailed description of an artificial intelligence-based hail prediction method, apparatus, electronic device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A hail prediction method based on artificial intelligence, characterized in that, include: The radar echo sequence is acquired, and the radar echo sequence is processed to obtain a time-series sequence sample and a sequence-level label corresponding to the time-series sequence sample. Obtain the metadata set corresponding to the time series sample, and convert the metadata set into a text prompt string; The text prompt string is encoded to obtain text embedding features, and affine parameters for conditional modulation are generated based on the text embedding features. Specifically, this includes: encoding the text prompt string using a preset hail prediction model to obtain a word-level embedding matrix and a global convergence vector; parsing the text prompt string to obtain a set of word indices corresponding to each phrase in the text prompt string; aggregating the set of word indices corresponding to each phrase to obtain the text embedding features; aligning the global convergence vector and the text embedding features spatially through a first projection network and a second projection network to obtain global semantic features; and inputting the global semantic features into a first affine parameter generation network and a second affine parameter generation network to generate scaling and translation parameters for conditional modulation. Each frame of radar image in the time sequence sample is encoded to obtain multidimensional radar sequence visual embedding features; The text embedding features are aligned across modalities with the multidimensional radar sequence visual embedding features to obtain semantically enhanced features; The visual embedding features of the multidimensional radar sequence are conditionally modulated using the affine parameters to obtain modulated visual features, and the modulated visual features are fused with the semantic enhancement features to obtain cross-modal fusion features. The cross-modal fusion features are then used for time-series modeling to obtain hail prediction results.

2. The method according to claim 1, characterized in that, The process of encoding each frame of radar image in the time-series sample to obtain multi-dimensional radar sequence visual embedding features includes: Each frame of radar image in the time sequence sample is divided into image blocks to obtain at least one image block; After flattening the at least one image block, it is mapped to the initial embedding space through the first linear projection layer to obtain the image block embedding sequence of each frame of radar image; The image block embedding sequence is encoded using a preset hail recognition model to obtain image block encoding features; The image block coding features are mapped to the shared latent space through the second linear projection layer to obtain the visual block features of each frame of radar image; The visual block features of each radar image frame are stacked along the time dimension to obtain the multidimensional radar sequence visual embedding features.

3. The method according to claim 2, characterized in that, The method further includes: Based on the L encoding resolutions of the preset hail recognition model, each frame of radar image in the time sequence sample is encoded to obtain multi-dimensional radar sequence visual embedding features. The step of aligning the text embedding features with the radar sequence visual embedding features across modalities to obtain semantically enhanced features includes: Obtain the mapped text embedding features corresponding to the text embedding features; Obtain the visual embedding sub-feature corresponding to each dimension of the radar sequence visual embedding feature in the multi-dimensional radar sequence visual embedding feature; For each of the L encoding resolutions, calculate the similarity between the mapped text embedding feature and each visual embedding sub-feature at the current encoding resolution; The attention distribution is calculated based on the similarity. The semantic enhancement features are obtained by aggregating the attention distribution with the visual embedding features of each dimension of radar sequence using a preset hail prediction model.

4. The method according to claim 3, characterized in that, The affine parameters include scaling parameters and translation parameters; The affine parameters are used to conditionally modulate the visual embedding features of the multidimensional radar sequence to obtain modulated visual features. These modulated visual features are then fused with the semantic enhancement features to obtain cross-modal fusion features, including: The radar sequence visual embedding features at each coding resolution in the multidimensional radar sequence visual embedding features are normalized to obtain normalized features. The normalized features are conditionally modulated using the scaling and translation parameters to obtain modulated visual features. The modulated visual features are fused with the semantic enhancement features to obtain the cross-modal fusion features.

5. The method according to claim 4, characterized in that, The step of performing time-series modeling on the cross-modal fusion features to obtain hail prediction results includes: For each encoding resolution, the cross-modal fusion features of the corresponding resolution are input along the time dimension into the Mamba selective state-space module for temporal dependency modeling, obtaining the temporal modeling output features: The temporal modeling output features at each coding resolution are upsampled to the spatial grid resolution to obtain the upsampled temporal modeling output features; The upsampled temporal modeling output features are then globally pooled along the spatial and temporal dimensions to obtain a sequence-level description vector: The sequence-level description vector is input into a preset hail prediction model to obtain hail prediction results.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Acquire the hail prediction model to be trained, the text prompt training set, and the radar echo training set; The hail prediction model to be trained is used to perform prediction processing on the text prompt training set and the radar echo training set to obtain image block coding training features, global semantic training features and model training results. Global alignment loss information is constructed based on the image block encoding training features and the global semantic training features; Construct model training loss information based on the model training results; The hail prediction model to be trained is adjusted based on the global alignment loss information and the model training loss information to obtain a preset hail prediction model.

7. The method according to claim 6, characterized in that, The step of constructing global alignment loss information based on the image block encoding training features and the global semantic training features includes: The image block encoding training features are pooled using the hail prediction model to be trained, resulting in pooled image block training features. Based on a preset global alignment loss function, consistency constraint calculation is performed on the global semantic training features and the pooled image block encoding training features to obtain the global alignment loss information.

8. A hail prediction device based on artificial intelligence, characterized in that, include: A data processing unit is used to acquire radar echo sequences, perform data processing on the radar echo sequences, and obtain time-series sequence samples and sequence-level labels corresponding to the time-series sequence samples. A conversion unit is used to obtain a metadata set corresponding to the time series sample and convert the metadata set into a text prompt string. The first encoding unit is used to encode the text prompt string to obtain text embedding features, and generate affine parameters for conditional modulation based on the text embedding features. Specifically, it includes: encoding the text prompt string using a preset hail prediction model to obtain a word-level embedding matrix and a global convergence vector; parsing the text prompt string to obtain a set of word indexes corresponding to each phrase in the text prompt string; aggregating the set of word indexes corresponding to each phrase to obtain the text embedding features; aligning the global convergence vector and the text embedding features in spatial dimensions through a first projection network and a second projection network to obtain global semantic features; and inputting the global semantic features into a first affine parameter generation network and a second affine parameter generation network to generate scaling and translation parameters for conditional modulation. The second encoding unit is used to encode each frame of radar image in the time sequence sample to obtain multi-dimensional radar sequence visual embedding features. An alignment unit is used to perform cross-modal alignment between the text embedding features and the multi-dimensional radar sequence visual embedding features to obtain semantically enhanced features; The modulation unit is used to conditionally modulate the visual embedding features of the multidimensional radar sequence using the affine parameters to obtain modulated visual features, and then fuse the modulated visual features with the semantic enhancement features to obtain cross-modal fusion features. The temporal modeling unit is used to perform temporal modeling on the cross-modal fusion features to obtain hail prediction results.

9. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the artificial intelligence-based hail prediction method according to any one of claims 1 to 7.

Images