Tropical cyclone strength prediction method and tropical cyclone strength prediction system based on multi-modal data

Abstract

The invention relates to a tropical cyclone strength prediction method and a tropical cyclone strength prediction system based on multi-modal data. According to the method, the tropical cyclone strength prediction is carried out by integrating the multi-modal data and extracting and integrating related characteristics between the multi-modal data. The method comprises the following steps: firstly, acquiring a multi-modal data set, dividing the multi-modal data set according to tropical cyclone batches and time, extracting data features from the divided multi-modal data, and integrating the data features to obtain integrated multi-modal fusion features; carrying out coding and decoding processing on the integrated multi-modal fusion features to obtain a future meteorological feature sequence; carrying out calculation processing on the future meteorological feature sequence by adopting a self-attention mechanism, and obtaining an aggregated long-term global feature; and finally, extracting wind speed information from the gathered long-term global feature sequence to obtain a tropical cyclone strength prediction value. Compared with the prior art, the method has the advantages of high accuracy, good timeliness, strong robustness and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of tropical cyclone intensity prediction, and in particular to a tropical cyclone intensity prediction method based on multi-modal data and a system thereof. Background Art

[0002] Accurate prediction of tropical cyclone intensity is crucial for disaster preparedness and relief work. The prediction of tropical cyclone intensity not only depends on basic meteorological data such as wind speed and air pressure, but also needs to consider environmental variables such as temperature, relative humidity, wind component and relative vorticity, so as to accurately predict the future intensity of tropical cyclones. In this way, more effective emergency plans and decision-making support can be provided for disaster prevention and mitigation departments, and rescue materials and personnel can be reasonably deployed, thereby reducing economic losses.

[0003] However, due to the heterogeneity of tropical cyclones, existing tropical cyclone intensity prediction methods still face the problem of effectively utilizing and integrating multimodal data, which directly affects the accurate capture of cyclone intensity changes, resulting in low final prediction accuracy, especially under rapidly changing meteorological conditions, where the prediction error is large. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a tropical cyclone intensity prediction method and system based on multimodal data.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to one aspect of the present invention, there is provided a method for predicting tropical cyclone intensity based on multimodal data, comprising the following steps:

[0007] S1. Obtain a multimodal data set and divide it according to tropical cyclone batches and time to obtain divided multimodal data;

[0008] S2, extracting data features from the divided multimodal data and integrating them to obtain integrated multimodal fusion features;

[0009] S3, encoding and decoding the integrated multi-modal fusion features to obtain future meteorological feature sequences;

[0010] S4, using the self-attention mechanism to calculate and process the future meteorological feature sequence to obtain the aggregated long-term global features;

[0011] S5. Extract wind speed information from the aggregated long-term global feature sequence to obtain the tropical cyclone intensity forecast value.

[0012] As a preferred technical solution, the multimodal data set in S1 includes tropical cyclone numerical data, reanalysis data, and satellite cloud images;

[0013] The reanalysis data includes wind components and atmospheric data.

[0014] As a preferred technical solution, the specific calculation formula for extracting data features from multimodal data in S2 is:

[0015]

[0016] A=BatchNorm(ReLU(Conv(BatchNorm(Sigmoid(Attn(Input))))))

[0017] Among them, f tr (*) indicates transformation operation, Represents the numerical data of tropical cyclones, f fu (*) indicates fusion wind speed operation, represents the wind component, f 3d (*) is a 3D convolution operation, represents atmospheric data, represents the density map data, f 2d (*) is a 2D convolution operation, Represents two-dimensional satellite cloud image data; Input represents the input of the wind component, A represents the spatial attention value, BatchNorm(*) represents the batch normalization operation, ReLU(*) represents the ReLU activation function, Conv(*) represents the convolution operation, Sigmoid(*) represents the Sigmoid activation function, and Attn(*) represents the spatial attention.

[0018] As a preferred technical solution, density map data is set in the process of extracting data features, specifically, the area is divided according to the current tropical cyclone shape, the data within the area is marked as the current tropical cyclone intensity, and the data outside the area is marked as 0.

[0019] As a preferred technical solution, the specific calculation formula for integrating the extracted data features in S2 is:

[0020] F fusion =fb([α⊙fb(F1),β⊙fb(F2),

[0021] γ⊙fb(F3),δ⊙fb(F4)])

[0022] Among them, fb(*) is the fusion block; F1 is the feature extracted from the tropical cyclone numerical data; F2 is the feature extracted from the wind speed data in the reanalysis data; F3 is the feature extracted from the atmospheric data in the reanalysis data; F4 is the feature extracted from the two-dimensional satellite cloud image data; ⊙ is the element multiplication; F fusionis a multimodal fusion feature; α, β, γ, δ are trainable weight coefficients used to control the importance of different data.

[0023] As a preferred technical solution, the specific calculation formula for encoding and decoding the multimodal fusion features in S3 is:

[0024] H en =f en (F fusion )

[0025] H de =f de (H en )

[0026] F pred =Concat(H en ,H de )

[0027] Among them, f en (*) indicates encoder, f de (*) indicates decoder, F fusion represents multimodal fusion features, H en represents the encoded features, H de represents the decoded features, F pred Represents the future meteorological characteristic sequence, and Concat(*) represents concatenation.

[0028] As a preferred technical solution, the specific calculation formula for the aggregated long-term global features in S4 is:

[0029] F focus =Att(F pred )

[0030] Among them, Att(*) represents the self-attention mechanism, F pred represents the future meteorological characteristic sequence, F focus Represents the aggregation of long-term global features.

[0031] According to another aspect of the present invention, there is provided a tropical cyclone intensity prediction system based on multimodal data, the system adopting the tropical cyclone intensity prediction method based on multimodal data as described above, the system comprising a feature fusion module, a main prediction module, a feature focusing module, and an intensity prediction module;

[0032] The feature fusion module includes a plurality of feature extraction blocks, each of which is connected to a first fusion block, and each of the first fusion blocks is connected to a second fusion block. The feature extraction block is used to extract corresponding features from input modal data. The first fusion block is used to fuse corresponding features of different dimensions to obtain aggregate features, and the aggregate features are multiplied by trainable weight coefficients to obtain weighted features. The second fusion block is used to fuse the weighted features to obtain integrated multimodal fusion features.

[0033] The main prediction module includes an encoder and a decoder, which captures and utilizes the long-term correlation between sequences through the encoder-decoder structure, performs prediction in a complex dynamic system, and generates a sequence of predicted future meteorological characteristics;

[0034] The feature focusing module includes a self-attention mechanism, which generates aggregated long-term global features through the self-attention mechanism;

[0035] The intensity prediction module includes four convolution blocks and two linear layers, which processes feature sequences to extract wind speed information, generate and output tropical cyclone intensity prediction values.

[0036] As a preferred technical solution, the process of the feature fusion module processing multimodal fusion features is: first, corresponding feature extraction blocks are used to extract corresponding features from different data, and then the first fusion block is used to fuse corresponding features of different dimensions to obtain aggregated features, and the aggregated features are multiplied by trainable weight coefficients to obtain weighted features, and finally the weighted features are integrated through the second fusion block to obtain multimodal fusion features.

[0037] As a preferred technical solution, the specific calculation process of the intensity prediction module to obtain the tropical cyclone intensity prediction value is:

[0038] I1=MaxPool(ReLU(Conv(F focus )))

[0039] I2 = MaxPool(ReLU(Conv(I1)))

[0040] I3 = MaxPool(ReLU(Conv(I2)))

[0041] I4 = MaxPool(ReLU(Conv(I3)))

[0042] I5=Linear(Dropout(I4))

[0043] I out =Linear(Dropout(I5))

[0044] Among them, MaxPool(*) is the maximum pooling operation, ReLU(*) is the ReLU activation function, Conv(*) is the convolution operation, and F focus represents the aggregation of long-term global features, Linear(*) is a linear layer, Dropout(*) is a dropout function, I1 represents the features extracted by the first convolution block of the intensity estimation module, I2 represents the features extracted by the second convolution block of the intensity estimation module, I3 represents the features extracted by the third convolution block of the intensity estimation module, I4 represents the features extracted by the fourth convolution block of the intensity estimation module, I5 represents the features converted by the first linear block of the intensity estimation module, and I out Represents the tropical cyclone intensity forecast value that is the final output of the intensity estimation module.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The present invention integrates multimodal data and extracts and integrates the relevant features between them to predict the intensity of tropical cyclones. The method first obtains a multimodal data set and divides it according to tropical cyclone batches and time, then extracts data features from the divided multimodal data and integrates them to obtain integrated multimodal fusion features; then encodes and decodes the integrated multimodal fusion features to obtain a future meteorological feature sequence; a self-attention mechanism is used to calculate and process the future meteorological feature sequence to obtain aggregated long-term global features; finally, wind speed information is extracted from the aggregated long-term global feature sequence, that is, a tropical cyclone intensity prediction value is obtained, and tropical cyclone intensity prediction is realized, and the correlation between multimodal data is fully explored to improve the accuracy of tropical cyclone intensity prediction.

[0047] 2. The present invention captures and utilizes the long-term correlation between sequences by encoding and decoding the integrated multimodal fusion features, performs predictions in complex dynamic systems, generates predicted future meteorological feature sequences, generates aggregated long-term global features through a self-attention mechanism, and processes feature sequences to extract wind speed information, thereby generating and outputting tropical cyclone intensity prediction values, thereby improving the timeliness of tropical cyclone intensity predictions.

[0048] 3. The present invention first uses the corresponding feature extraction blocks to extract corresponding features from different data, then uses the first fusion block to fuse the corresponding features of different dimensions to obtain aggregate features, multiplies the aggregate features by the trainable weight coefficient to obtain weighted features, and finally integrates the weighted features through the second fusion block to obtain multimodal fusion features. This makes the tropical cyclone intensity prediction method highly robust and can adapt to the tropical cyclone intensity prediction needs in different application scenarios.

[0049] 4. In the present invention, the integrated multimodal fusion features are input into the main prediction module, captured through encoding and decoding, and the future meteorological feature sequence is generated and output to the feature focusing module. Through the self-attention mechanism in the feature focusing module, the aggregated long-term global features are generated and output; the encoding and decoding steps associated with the self-attention mechanism are used to solve the problem of long-term dependency attenuation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the steps of a tropical cyclone intensity prediction method based on multimodal data in the present invention;

[0051] Figure 2 It is a structural schematic diagram of the feature fusion module in the present invention;

[0052] Figure 3 It is a schematic diagram of the structure of the main prediction module in the present invention;

[0053] Figure 4 It is a structural schematic diagram of the feature focusing module in the present invention;

[0054] Figure 5 It is a structural schematic diagram of the strength estimation module in the present invention;

[0055] Figure 6 Schematic diagram of comparison between the actual value of tropical cyclone intensity in the embodiment and the predicted value of tropical cyclone intensity obtained based on the method of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0057] Example

[0058] In this embodiment, a tropical cyclone intensity prediction system based on multimodal data is used, and a tropical cyclone intensity prediction method based on multimodal data is applied to predict the intensity of tropical cyclones, aiming to overcome the challenges faced by the prior art in the effective use and integration of multimodal data. The method steps are as follows: Figure 1 shown.

[0059] In this embodiment, a tropical cyclone intensity prediction system based on multimodal data includes a feature fusion module, a main prediction module, a feature focusing module and an intensity estimation module. The feature fusion module includes a feature extraction block, a first fusion block, a trainable weight coefficient and a second fusion block. The features of different data are extracted through the feature extraction block, the importance of different features is adjusted through the trainable weight coefficient, and the extracted features are finally integrated through the second fusion block; the main prediction module includes an encoder and a decoder, which captures and utilizes the long-term correlation between sequences through the encoder-decoder structure, performs predictions in complex dynamic systems, and generates a sequence of predicted future meteorological features; the feature focusing module includes a self-attention mechanism, which generates aggregated long-term global features through the self-attention mechanism; the intensity prediction module includes four convolution blocks and two Linear layers, which extract wind speed information by processing feature sequences, and generates and outputs tropical cyclone intensity prediction values.

[0060] In this embodiment, for the tropical cyclone intensity prediction system based on multimodal data, its training process includes the following steps: first, data preprocessing is performed to improve the training efficiency and performance stability of the model and reduce overfitting; then the model training phase is entered, covering forward propagation and back propagation, and the model parameters are optimized through the loss function to complete the learning process; finally, the model verification phase is performed, using cross-validation and performance evaluation to ensure the robustness and accuracy of the model to unknown data.

[0061] In this embodiment, the tropical cyclone intensity prediction system based on multimodal data is trained on a remote server equipped with an Intel(R) Xeon(R) Silver 4210 CPU@2.20GHz, 64GB main memory and Ubuntu 18.04 operating system. TM The GeForce RTX3090 GPU is used as an accelerator to improve the efficiency of batch training. The method is implemented using Python 3.9 and machine learning or deep learning software packages, including Scikit-image0.18.3, OpenCV 4.5.3, and PyTorch 1.9.0.

[0062] In this embodiment, when training the tropical cyclone intensity prediction system based on multimodal data, the batch size is set to 16, the learning rate is 0.001, the optimizer uses the adaptive moment estimation optimizer (Adaptive Moment Estimation, Adam), the loss function uses the mean squared error (MSE), and the training cycle is set to 100. After the system model is trained, the following tropical cyclone intensity prediction method based on multimodal data is applied to predict the tropical cyclone intensity.

[0063] In this embodiment, a tropical cyclone intensity prediction method based on multimodal data is applied, and the steps include:

[0064] S1. Obtain a multimodal data set; and divide it according to tropical cyclone batches and time to obtain divided multimodal data;

[0065] S2, input the divided multimodal data into the feature fusion module, extract the data features and integrate them to obtain the integrated multimodal fusion features;

[0066] S3, input the integrated multimodal fusion features into the main prediction module, and obtain the future meteorological feature sequence through encoding and decoding processing;

[0067] S4, input the future meteorological feature sequence into the feature focusing module, and obtain the aggregated long-term global features through the self-attention mechanism;

[0068] S5. The aggregated long-term global features are input into the intensity estimation module, and the feature sequence is processed to extract the wind speed information to obtain the tropical cyclone intensity prediction value.

[0069] In this embodiment, the feature fusion module includes a feature extraction block, a first fusion block, a trainable weight coefficient, and a second fusion block; the module structure is as follows: Figure 2 As shown, the module formula is as follows:

[0070] F fusion =fb([α⊙fb(F1),β⊙fb(F2),

[0071] γ⊙fb(F3),δ⊙fb(F4)])

[0072] α+β+γ+δ=1

[0073] Where fb(*) represents the fusion block, F1 represents the features extracted from the tropical cyclone numerical data, F2 represents the features extracted from the wind speed data in the reanalysis data, F3 represents the features extracted from the atmospheric data in the reanalysis data, F4 represents the features extracted from the two-dimensional satellite cloud image data, ⊙ represents element multiplication, and F fusionis the output of the feature fusion module, i.e., the multimodal fusion feature. α, β, γ, and δ are trainable weight coefficients used to control the importance of different data. The initial value of the trainable weight coefficient is assigned as It indicates that the four types of data are equally important, and are dynamically adjusted during training according to the importance of different data.

[0074] In this embodiment, the feature extraction block is used to adopt different feature extraction methods for different features, so as to obtain features of different types of data.

[0075] In this embodiment, the input of the feature extraction block is: different types of data, including tropical cyclone numerical data, reanalysis data and satellite cloud images. Different feature extraction methods are used to extract high-level features from different types of data, and the first fusion block is used to fuse them, and the weighted data is used as the input of the second fusion block.

[0076] In this embodiment, the calculation formula for feature extraction of different data is:

[0077]

[0078] Among them, f tr (*) indicates transformation operation, Indicates numerical data, f fu (*) indicates fusion wind speed operation, represents the wind component, f 3d (*) is a 3D convolution operation, represents atmospheric data, represents the density map data, f 2d (*) is a two-dimensional convolution operation, Represents two-dimensional satellite cloud image data.

[0079] In this embodiment, f fu The calculation formula of (*) is as follows:

[0080]

[0081] A=BatchNorm(ReLU(Conv(BatchNorm(Sigmoid(Attn(Input))))))

[0082] Among them, Input represents the input of the wind component, A represents the spatial attention value, BatchNorm(*) represents the batch normalization operation, ReLU(*) represents the ReLU activation function, Conv(*) represents the convolution operation, Sigmoid(*) represents the Sigmoid activation function, and Attn(*) represents the spatial attention.

[0083] In this embodiment, the calculation formula of Attn(*) is as follows:

[0084] F avg =AvgPool(Input)

[0085] F max =MaxPool(Input)

[0086] A s =Sigmoid(Conv(Concat(F avg ,F max )))

[0087] Attn(Input)=A s ⊙Input

[0088] Where Input represents input data, AvgPool(*) represents average pooling operation, MaxPool(*) represents maximum pooling operation, Sigmoid(*) represents Sigmoid activation function, A s Represents the spatial attention map.

[0089] In this embodiment, the spatial attention method uses the wind speed grid and its residual with the center position as input, and enhances the contribution of high-correlation areas in intensity prediction by dynamically adjusting the weight distribution of each position, thereby improving the accuracy of the prediction results.

[0090] In this embodiment, the first fusion block and the second fusion block include convolution operations and linear operations, which are used to fuse features extracted from different data. The input includes features extracted from tropical cyclone numerical data, reanalysis data, and satellite cloud images. The feature extraction module extracts features from multimodal data of different dimensions, and uses additional trainable weight coefficients to adjust the importance of each feature. The fused features are used as input to the main prediction module for subsequent processing. The module formula is as follows:

[0091] F fusion =fb([α⊙fb(F1),β⊙fb(F2),

[0092] γ⊙fb(F3),δvfb(F4)])

[0093] α+β+γ+δ=1

[0094] Where fb(*) represents the fusion block, F1 represents the features extracted from the tropical cyclone numerical data, F2 represents the features extracted from the wind speed data in the reanalysis data, F3 represents the features extracted from the atmospheric data in the reanalysis data, F4 represents the features extracted from the two-dimensional satellite cloud image data, ⊙ represents element multiplication, and F fusionis a multimodal fusion feature, α, β, γ, δ are trainable weight coefficients used to control the importance of different data, and the initial value of the trainable weight coefficient is assigned as It indicates that the four types of data are equally important, and are dynamically adjusted during training according to the importance of different data.

[0095] In this embodiment, the trainable weight coefficient is automatically adjusted through training to control the importance of different data.

[0096] In this embodiment, the main prediction module includes an encoder and a decoder, and the structure is as follows: Figure 3 As shown, the encoder is used to process multimodal fusion features, and the decoder is used to reconstruct spatiotemporal features from hidden vectors. The encoder and the decoder are combined to build the main prediction module to realize the processing of multimodal fusion features and output the future meteorological feature sequence.

[0097] In this embodiment, the main prediction module uses multimodal fusion features as input, captures and utilizes long-term correlations between sequences through an encoder-decoder structure, performs predictions in complex dynamic systems and improves the accuracy of predictions, generates and outputs future meteorological feature sequences, and serves as input to the feature focusing module for further processing. The formula of this module is as follows:

[0098] H en =f en (F fusion )

[0099] H de =f de (H en )

[0100] F pred =Concat(H en ,H de )

[0101] Among them, f de (*) indicates decoder, f en (*) indicates encoder, F fusion represents multimodal fusion features, H en represents the encoded features, H de represents the decoded features, Concat(*) represents concatenation, F pred Represents a sequence of future meteorological characteristics.

[0102] In this embodiment, the main prediction module can capture and utilize the long-term correlation between sequences and make predictions in complex dynamic systems, realize the effective processing of multimodal fusion data, and output future meteorological characteristic sequences.

[0103] In this embodiment, the feature focusing module includes a self-attention mechanism, such as Figure 4 As shown in Figure 1, the self-attention mechanism is used to capture the feature correlation within the time step. The feature focusing module takes the future meteorological feature sequence as input, captures the feature correlation within the time step through the self-attention mechanism to solve the problem of long-term dependency decay, and generates aggregated long-term global features as the input of the intensity estimation module for further processing. The formula of this module is as follows:

[0104] F focus =Att(F pred )

[0105] Among them, Att(*) represents the self-attention mechanism, F pred represents the future meteorological characteristic sequence, F focus Represents the aggregation of long-term global features.

[0106] In this embodiment, the feature focusing module can capture the feature correlation within the time step through the attention mechanism to solve the problem of long-term dependency decay and output aggregated long-term global features.

[0107] In this embodiment, the intensity estimation module includes four convolution blocks and two linear layers, and its structure is as follows: Figure 5 As shown. The convolutional block and the Linear layer are used to specifically process the feature sequence to extract wind speed information. This module is used to specifically process the feature sequence to extract wind speed information and output the tropical cyclone intensity forecast value. The formula of this module is as follows:

[0108] I1=MaxPool(ReLU(Conv(F focus )))

[0109] I2 = MaxPool(ReLU(Conv(I1)))

[0110] I3 = MaxPool(ReLU(Conv(I2)))

[0111] I4 = MaxPool(ReLU(Conv(I3)))

[0112] I5=Linear(Dropout(I4))

[0113] I out =Linear(Dropout(I5))

[0114] Among them, MaxPool(*) is the maximum pooling operation, ReLU(*) is the ReLU activation function, Conv(*) is the convolution operation, Linear(*) is the linear layer, Dropout(*) is the drop function, and F focusrepresents the aggregated long-term global features, I1 represents the features extracted by the first convolution block of the intensity estimation module, I2 represents the features extracted by the second convolution block of the intensity estimation module, I3 represents the features extracted by the third convolution block of the intensity estimation module, I4 represents the features extracted by the fourth convolution block of the intensity estimation module, I5 represents the features converted by the first linear block of the intensity estimation module, and I out Represents the tropical cyclone intensity forecast value that is the final output of the intensity estimation module.

[0115] In this embodiment, the module input is the aggregated long-term global features, and outputs the tropical cyclone intensity prediction value. The intensity estimation module can specifically process the feature sequence to extract wind speed information and output the tropical cyclone intensity prediction value.

[0116] In this embodiment, the result of predicting the intensity of tropical cyclones is as follows: Figure 6 As shown in the figure, this result graph shows the actual change in wind speed of Tropical Cyclone Mindulle around September 24, 2021, and the prediction results of the "Tropical Cyclone Intensity Prediction Method Based on Multimodal Data". Among them, the blue solid line represents the observed real tropical cyclone intensity value. The values ​​in the figure show that the tropical cyclone intensity increases from 13m / s to 20m / s, indicating that the tropical cyclone intensity gradually increases. The green dotted line represents the prediction result of the "Tropical Cyclone Intensity Prediction Method Based on Multimodal Data". According to the result graph, it can be seen that the overall trend is consistent with the actual wind speed change trend, especially at t+24h, the prediction result and the actual result almost coincide. This shows that the prediction method based on multimodal data has excellent performance and practicality in the short-term prediction of tropical cyclone intensity.

[0117] In this embodiment, the tropical cyclone intensity prediction method used can be applied to weather forecasting and meteorological analysis to accurately predict the intensity of tropical cyclones, helping meteorological departments to issue early warnings and improve the accuracy and timeliness of predictions. The method can provide more accurate disaster intensity information to governments and emergency management departments, thereby helping them to formulate emergency plans, arrange evacuation plans, and deploy rescue forces, thereby reducing the loss of human life and property caused by disasters.

[0118] In this embodiment, the tropical cyclone intensity prediction method has the advantages of high precision and strong robustness, and can meet the needs of tropical cyclone intensity prediction in different application scenarios.

[0119] In this embodiment, a new tropical cyclone intensity prediction solution is provided for technical personnel in related fields, which is helpful to promote the development of tropical cyclone intensity prediction technology. The tropical cyclone intensity prediction method based on multimodal data will significantly improve the prediction accuracy and reliability of cyclone intensity under complex meteorological conditions, thereby playing a key role in weather forecasting, disaster response and other fields. By providing more accurate cyclone intensity prediction data, a more reliable basis can be provided for scientific research and emergency decision-making, which is crucial to reducing disaster losses and improving decision-making efficiency. Provide more effective emergency plans and decision-making support for disaster prevention and mitigation departments, rationally deploy rescue materials and personnel, and thus reduce economic losses.

[0120] In addition, the method in this embodiment can also be extended to other fields, such as marine meteorology, agricultural climate assessment, etc., in which accurate intensity prediction is also of great value.

[0121] In summary, this embodiment applies an advanced and highly adaptable tropical cyclone intensity prediction method and system, which can improve the accuracy of meteorological forecasts and bring new opportunities for applications in multiple fields.

[0122] The tropical cyclone intensity prediction method in this embodiment can not only efficiently perform intensity prediction tasks, but also provide solid technical support for research and application in related fields. Through this method, users can obtain clearer and more accurate tropical cyclone intensity prediction results, thereby providing more effective support for scientific research and decision-making.

[0123] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A tropical cyclone intensity prediction method based on multimodal data, characterized in that: The following steps are involved: S1. Obtain a multimodal data set and divide it according to tropical cyclone batches and time to obtain divided multimodal data; S2, extracting data features from the divided multimodal data and integrating them to obtain integrated multimodal fusion features; S3, encoding and decoding the integrated multi-modal fusion features to obtain future meteorological feature sequences; S4, using the self-attention mechanism to calculate and process the future meteorological feature sequence to obtain the aggregated long-term global features; S5. Extract wind speed information from the aggregated long-term global feature sequence to obtain the tropical cyclone intensity forecast value.

2. The method for predicting tropical cyclone intensity based on multimodal data according to claim 1, characterized in that: The multimodal dataset in S1 includes tropical cyclone numerical data, reanalysis data and satellite cloud images; The reanalysis data includes wind components and atmospheric data.

3. The method for predicting tropical cyclone intensity based on multimodal data according to claim 2, characterized in that: The specific calculation formula for extracting data features from multimodal data in S2 is: f fu (*)=Input°A A=BatchNorm(ReLU(Conv(BatchNorm(Sigmoid(Attn(Input)))))) Among them, f tr (*) indicates transformation operation, represents the numerical data of tropical cyclones, f fu (*) indicates fusion wind speed operation, represents the wind component, f 3d (*) is a 3D convolution operation, represents atmospheric data, represents the density map data, f 2d (*) is a 2D convolution operation, Represents two-dimensional satellite cloud image data; Input represents the input of the wind component, A represents the spatial attention value, BatchNorm(*) represents the batch normalization operation, ReLU(*) represents the ReLU activation function, Conv(*) represents the convolution operation, Sigmoid(*) represents the Sigmoid activation function, and Attn(*) represents the spatial attention.

4. The method for predicting tropical cyclone intensity based on multimodal data according to claim 3, characterized in that: The density map data is set in the process of extracting data features, specifically, the area is divided according to the current tropical cyclone shape, the data in the area is marked as the current tropical cyclone intensity, and the data outside the area is marked as 0.

5. The method for predicting tropical cyclone intensity based on multimodal data according to claim 1, characterized in that: The specific calculation formula for integrating the extracted data features in S2 is: F fusion =fb([α⊙fb(F1),β⊙fb(F2), γ⊙fb(F3),δ⊙fb(F4)]) Among them, fb(*) is the fusion block; F1 is the feature extracted from the tropical cyclone numerical data; F2 is the feature extracted from the wind speed data in the reanalysis data; F3 is the feature extracted from the atmospheric data in the reanalysis data; F4 is the feature extracted from the two-dimensional satellite cloud image data; ⊙ is the element multiplication; F fusion is a multimodal fusion feature; α, β, γ, δ are trainable weight coefficients used to control the importance of different data.

6. The method for predicting tropical cyclone intensity based on multimodal data according to claim 1, characterized in that: The specific calculation formula for encoding and decoding the multimodal fusion features in S3 is: H en =f en (F fusion ) H de =f de (H en ) F pred =Concat(H en ,H de ) Among them, f en (*) indicates encoder, f de (*) indicates decoder, F fusion represents multimodal fusion features, H en represents the encoded features, H de represents the decoded features, F pred Represents the future meteorological characteristic sequence, and Concat(*) represents concatenation.

7. The method for predicting tropical cyclone intensity based on multimodal data according to claim 1, characterized in that: The specific calculation formula of the aggregated long-term global features in S4 is: F focus =That(F pred ) Among them, Att(*) represents the self-attention mechanism, F pred represents the future meteorological characteristic sequence, F focus Represents the aggregation of long-term global features.

8. A tropical cyclone intensity prediction system based on multimodal data, characterized in that: The system works by adopting a tropical cyclone intensity prediction method based on multimodal data as described in any one of claims 1 to 7, and the system includes a feature fusion module, a main prediction module, a feature focus module, and an intensity prediction module; The feature fusion module includes a plurality of feature extraction blocks, each of which is connected to a first fusion block, and each of the first fusion blocks is connected to a second fusion block. The feature extraction block is used to extract corresponding features from input modal data. The first fusion block is used to fuse corresponding features of different dimensions to obtain aggregate features, and the aggregate features are multiplied by trainable weight coefficients to obtain weighted features. The second fusion block is used to fuse the weighted features to obtain integrated multimodal fusion features. The main prediction module includes an encoder and a decoder, which captures and utilizes the long-term correlation between sequences through the encoder-decoder structure, performs prediction in a complex dynamic system, and generates a sequence of predicted future meteorological characteristics; The feature focusing module includes a self-attention mechanism, which generates aggregated long-term global features through the self-attention mechanism; The intensity prediction module includes four convolution blocks and two linear layers, which processes feature sequences to extract wind speed information, generate and output tropical cyclone intensity prediction values.

9. The tropical cyclone intensity prediction system based on multimodal data according to claim 7, characterized in that: The process of the feature fusion module processing multimodal fusion features is as follows: first, corresponding feature extraction blocks are used to extract corresponding features from different data, and then the first fusion block is used to fuse corresponding features of different dimensions to obtain aggregated features, and the aggregated features are multiplied by trainable weight coefficients to obtain weighted features, and finally the weighted features are integrated through the second fusion block to obtain multimodal fusion features.

10. The tropical cyclone intensity prediction system based on multimodal data according to claim 7, characterized in that: The specific calculation process of the intensity prediction module to obtain the tropical cyclone intensity prediction value is as follows: I1=MaxPool(ReLU(Conv(F focus ))) I2 = MaxPool(ReLU(Conv(I1))) I3 = MaxPool(ReLU(Conv(I2))) I4 = MaxPool(ReLU(Conv(I3))) I5=Linear(Dropout(I4)) I out =Linear(Dropout(I5)) Among them, MaxPool(*) is the maximum pooling operation, ReLU(*) is the ReLU activation function, Conv(*) is the convolution operation, and F focus represents the aggregation of long-term global features, Linear(*) is a linear layer, Dropout(*) is a dropout function, I1 represents the features extracted by the first convolution block of the intensity estimation module, I2 represents the features extracted by the second convolution block of the intensity estimation module, I3 represents the features extracted by the third convolution block of the intensity estimation module, I4 represents the features extracted by the fourth convolution block of the intensity estimation module, I5 represents the features converted by the first linear block of the intensity estimation module, and I out Represents the tropical cyclone intensity forecast value that is the final output of the intensity estimation module.

Images