Typhoon track and intensity prediction method based on multimodal fusion network model
By constructing a multimodal fusion network model and combining data with the principles of physics, the problem of inaccurate typhoon path and intensity prediction in existing technologies has been solved, and more accurate typhoon path and intensity predictions have been achieved.
Patent Information
- Application Number
- CN202511100642.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-07
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-08-07
AI Technical Summary
Existing deep learning methods ignore the impact of physical background fields on typhoon development in typhoon path and intensity predictions, resulting in inaccurate predictions.
A multimodal fusion network model is constructed, combining data with the principles of physics, to achieve the prediction of typhoon path and intensity through wind field cross-feature fusion, potential height spatiotemporal feature extraction, multi-scale feature fusion, and generative adversarial networks and scalar long short-term memory units.
A more comprehensive understanding of the typhoon formation mechanism has improved the accuracy of typhoon path and intensity forecasts, alleviated the problem of long-term reliance on information attenuation, and enhanced the ability to model wind field motion characteristics.
Smart Images
Figure CN120598141B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to meteorological monitoring technology, and in particular to a typhoon path and intensity prediction method based on a multimodal fusion network model. Background Art
[0002] Typhoons are common in many parts of the world, affecting nearly all tropical regions. Typhoons are among the most dangerous and destructive natural phenomena on Earth, associated with extreme winds, flooding, heavy rain, huge waves, and devastating storm surges, causing significant economic losses. With the intensification of global climate change, the frequency and intensity of typhoons are likely to increase further, placing higher demands on typhoon forecasting globally, especially in tropical and subtropical regions. Therefore, accurate predictions of typhoon paths and intensities are extremely important, as they are directly related to the formulation and implementation of disaster prevention and mitigation strategies.
[0003] In recent years, deep learning has seen its application in meteorology continue to expand due to its powerful nonlinear modeling capabilities. Data-driven deep learning methods can mine implicit information from multi-source time series data, providing an effective approach for forecasting typhoon paths and intensities. However, the trajectory and intensity of a typhoon are affected by multiple factors, including the background field, internal thermal processes, and atmospheric dynamic processes at the time of typhoon development. Relying solely on multi-source time series data to predict path and intensity often overlooks the impact of the physical background field on the typhoon's path and intensity during typhoon development. Currently, some deep learning methods incorporate satellite cloud imagery into typhoon path and intensity forecasts. Based on two-dimensional convolutional neural networks, they extract spatiotemporal features from satellite cloud imagery. However, relying on cloud information captured by satellite cloud imagery makes it difficult to fully represent complex atmospheric dynamic processes. Summary of the Invention
[0004] Purpose of the invention: In response to the above problems, the purpose of the present invention is to provide a typhoon path and intensity prediction method based on a multimodal fusion network model. By fusing data from different modes and combining it with the physical principles and constraints of atmospheric motion, a multimodal fusion network model driven by both data and physics is proposed for typhoon path and intensity prediction, so as to further improve the prediction performance of typhoon path and intensity.
[0005] Technical solution: The present invention provides a typhoon path and intensity prediction method based on a multimodal fusion network model, comprising the following steps:
[0006] Step 1: Obtain the reanalysis data of the typhoon's location coordinates, central wind speed, central pressure, and wind field and geopotential height in the typhoon's central area, preprocess them according to a uniform time resolution, and divide the preprocessed data into a training set and a test set in proportion;
[0007] Step 2: Build a multimodal fusion network model driven by both data and physics. The multimodal fusion network model includes a wind field cross-feature fusion module, a potential height spatiotemporal feature extraction module, a multi-scale feature fusion module, and a fusion network module.
[0008] Step 3: Construct a loss function for the multimodal fusion network model, input the training set data into the multimodal fusion network model, calculate the loss, and update the parameters of the multimodal fusion network model through back propagation to obtain a trained multimodal fusion network model.
[0009] Step 4: Input the data in the test set into the trained multimodal fusion network model and output the typhoon path and intensity.
[0010] Furthermore, the fusion network module includes a generator and a discriminator, the generator includes an encoder and a decoder, the discriminator includes an encoder, and both the encoder and the decoder are composed of scalar long short-term memory units.
[0011] Furthermore, step 3 includes:
[0012] Input the U component and V component of the wind field into the wind field cross feature fusion module, where the U component is input into the first wind field cross branch and the V component is input into the second wind field cross branch;
[0013] In the first wind field cross branch, the U component is input into the first feature transformation block to obtain the first feature , expressed as:
[0014] ,
[0015] Where, Indicates that the convolution kernel size is 3D convolution, Represents the U component, and SAM represents the spatial attention mechanism operation;
[0016] Input the first feature into the second feature transformation block to obtain the second feature , expressed as:
[0017] ,
[0018] Input the second feature into the third feature transformation block to obtain the third feature , expressed as:
[0019] ,
[0020] Input the V component to the fourth feature transformation block to obtain the fourth feature , expressed as:
[0021] ,
[0022] Where, Represents the V component;
[0023] Input the fourth feature into the fifth feature transformation block to obtain the fifth feature , expressed as:
[0024] ,
[0025] Input the fifth feature into the sixth feature transformation block to obtain the sixth feature , expressed as:
[0026] ,
[0027] Cross-fuse the first feature and the fourth feature to obtain the first fusion feature , expressed as:
[0028] ,
[0029] Cross-fuse the second feature and the fifth feature, and then combine them with the first fusion feature obtained in the previous order Perform separation and fusion to obtain the second fusion feature , expressed as:
[0030] ,
[0031] In the formula, the symbol + represents the separation and fusion operation;
[0032] The third feature is cross-fused with the sixth feature, and then separated and fused with the second fused feature obtained in the previous sequence to obtain the third fused feature, which is expressed as:
[0033] ,
[0034] The third fusion features of different time steps are spliced to obtain feature O, which is expressed as:
[0035] ,
[0036] Where, Represents the third fusion feature of the g-th time step, g=1,2,3,4, represents the transposed convolutional layer, Represents a splicing operation;
[0037] Among them, the structures of the first to sixth feature transformation blocks are the same, all consisting of convolution blocks and spatial attention mechanisms.
[0038] Furthermore, step 3 also includes:
[0039] The geopotential height characteristics of the typhoon center area are input into the geopotential height spatiotemporal feature extraction module to obtain four feature maps of different scales, which are recorded as input1, input2, input3 and input4 respectively; the geopotential height spatiotemporal feature extraction module adopts a 3D U-Net structure.
[0040] Furthermore, step 3 also includes:
[0041] The feature maps of four different scales are input into the multi-scale feature fusion module, and the feature maps of different scales are uniformly scaled and fused. The process is as follows:
[0042] For feature map input1, use convolution kernel size The transposed convolution layer is operated three times, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q1;
[0043] For feature map input2, use a convolution kernel size of The transposed convolution layer is operated twice, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q2;
[0044] For feature map input3, use a convolution kernel size of The transposed convolution layer is operated once, which is processed by the transposed convolution operation, batch normalization layer and activation layer, and then processed by the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q3;
[0045] For feature map input4, use a convolution kernel size of The convolution layer of is operated once, in which the convolution operation, batch normalization layer and activation layer are processed to obtain the feature map Q4;
[0046] The feature maps Q1 to Q4 are concatenated and then input into the efficient channel attention network to obtain the feature map Q5. The efficient channel attention network is expressed as:
[0047] ,
[0048] in, Represents the input, that is, the concatenated feature map, represents the Sigmod function, represents a one-dimensional convolution operation, Represents the adaptive average pooling operation; k represents the convolution kernel size, and the formula is:
[0049] ,
[0050] Where, Represents the nonlinear mapping function from the number of channels C to the convolution kernel size k, Indicates the closest Odd number, and b are constants, C is the number of channels;
[0051] The feature map Q5 is then processed by a three-dimensional convolution layer, a batch normalization operation, and a ReLU activation function layer to obtain the final extracted feature map Q.
[0052] Furthermore, step 3 also includes:
[0053] After the typhoon one-dimensional data passes through the embedding layer, it is input into the generator together with the feature map Q5 and the feature map Q. The process is as follows:
[0054] Input the typhoon position coordinates, central wind speed and central air pressure data into the embedding layer for embedding operation to obtain fused data;
[0055] The fusion data, feature map Q5 and feature map Q are integrated through the fully connected layer to obtain the feature , the features Input to the scalar long short-term memory unit of the encoder, the formulas are:
[0056] Forget Gate: ,
[0057] Input Gate: ,
[0058] Output Gate: ,
[0059] Unit input: ,
[0060] Unit Status: ,
[0061] Normalizer status: ,
[0062] Hidden state: ,
[0063] in 、 、 Represent the outputs of the input gate, forget gate, and output gate respectively, 、 、 、 is the weight of the corresponding correlation matrix, 、 、 are the cell states that are fed together to the next time step, Represents the time step t The unit input, Represents the time step t The input vector, 、 、 、 represents the deviation vector, state n Is used to control the unit state Normalization of represents the Sigmoid activation function, Represents an infinitely small number;
[0064] The output of the encoder is combined with the hidden state Input to the decoder, and the output of the decoder is the output result of the generator.
[0065] Furthermore, the formula of the loss function is:
[0066] ,
[0067] in, represents the adjustable weight coefficient for different losses, represents the adversarial loss in the generative adversarial network, represents the L2 loss, 、 represent the encoder losses of wind field and geopotential height respectively, Represents the physical loss function based on wind field motion.
[0068] Furthermore, the expressions of each loss function are:
[0069] The adversarial loss expression in the generative adversarial network is:
[0070] ,
[0071] In the formula, x represents real data, z represents random noise, represents the value function of the generated adversarial network, which is the objective function of the adversarial game, E represents the mathematical expectation, and p data (x) represents the distribution of real data samples, represents the potential distribution of noise in the latent space, G represents the generator, and D represents the discriminator;
[0072] The expression of L2 loss is:
[0073] ,
[0074] in, and Represent the true value and predicted value of typhoon path and intensity respectively, Indicates the total number of sequences;
[0075] The expression of encoder loss is:
[0076] ,
[0077] in, It is the basic data of wind field and geopotential height in the center of typhoon. It refers to the prediction sequence obtained by the wind field cross feature fusion module and the potential height spatiotemporal feature extraction module. are the length, width and number of channels of the wind field variable map or the geopotential height variable map, respectively. is the predicted length, is the L1 norm function;
[0078] The formula of the physical loss function based on wind field motion is:
[0079] ,
[0080] in, , , represents the kinematic viscosity coefficient, represents the V component of the wind field, represents the U component of the wind field, They are right and The first derivative of They are right and The second derivative of yes right and The first and second derivatives of .
[0081] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:
[0082] 1. This invention predicts typhoon paths and intensities by jointly driving a multimodal fusion network based on data and physics. It constructs a field cross-feature fusion module, a potential height spatiotemporal feature extraction module, a multi-scale feature fusion module, and a fusion network module based on a generative adversarial network and scalar long short-term memory units. These modules extract and fuse features based on different background field characteristics, providing a more comprehensive understanding of the typhoon formation mechanism and fully utilizing the spatiotemporal characteristics of multi-source environmental variables.
[0083] 2. Using a fusion network module of generative adversarial networks and scalar long short-term memory units can alleviate the problem of long-term dependent information decay and enhance the modeling ability of wind field motion characteristics using physical constraints;
[0084] 3. Applying this invention to typhoon trajectory and intensity can more comprehensively learn the typhoon path and intensity characteristics, more accurately describe the typhoon movement path and change trend, and improve the accuracy of typhoon path and intensity prediction. It is an effective way to further improve the application of deep learning methods in the field of typhoon measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0085] Figure 1 This is a flow chart of the typhoon path and intensity prediction method based on the multimodal fusion network model;
[0086] Figure 2 This is the structural diagram of the multimodal fusion network model;
[0087] Figure 3 This is a structural diagram of the wind field cross-feature fusion module;
[0088] Figure 4 Schematic diagram of the structure of SAM;
[0089] Figure 5 Schematic diagram of the structure of the transposed convolution layer;
[0090] Figure 6 This is a schematic diagram of the structure of the potential height spatiotemporal feature extraction module;
[0091] Figure 7 It is a structural diagram of the multi-scale feature fusion module;
[0092] Figure 8 Schematic diagram of the structure of the scalar long short-term memory unit. DETAILED DESCRIPTION
[0093] In order to make the purpose, technical solutions and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings and embodiments.
[0094] Combine Figure 1 The typhoon path and intensity prediction method based on the multimodal fusion network model described in this embodiment includes the following steps:
[0095] Step 1: Obtain the typhoon's position coordinates, central wind speed and central pressure, as well as the reanalysis data of the wind field and geopotential height in the typhoon center area, and preprocess the data according to a uniform time resolution. The preprocessed data are divided into a training set and a test set in proportion.
[0096] The typhoon's position coordinates, central wind speed, and central pressure are obtained at each time point. The position coordinates include latitude and longitude, and the wind speed includes a U component and a V component. Latitude, longitude, wind speed, and pressure are one-dimensional data. After being embedded through an embedding layer, they are input into the generator and discriminator for adversarial training. All data is preprocessed to a uniform temporal resolution. The data is organized in chronological order, with each frame representing a fixed time point (every 6 hours). When constructing training samples, fixed-length subsequences are continuously extracted from this time series, each time interval being equal. The time intervals between both the observed and predicted sequences are consistent, so the temporal resolution is implicitly unified during sample generation.
[0097] Step 2: Build a multimodal fusion network model driven by data and physics, combining Figure 2 As shown in the figure, the multimodal fusion network model includes a wind field cross-feature fusion module (WCFM), a geopotential height spatiotemporal feature extraction module (GSTFM), a multi-scale feature fusion module (MSFM), and a fusion network module based on a generative adversarial network (GAN) and a scalar long short-term memory unit (LSTM).
[0098] Furthermore, the fusion network module includes a generator and a discriminator, the generator includes an encoder and a decoder, the discriminator includes an encoder, and both the encoder and the decoder are composed of scalar long short-term memory units.
[0099] Step 3: Construct the loss function of the multimodal fusion network model, input the training set data into the multimodal fusion network model, calculate the loss and update the parameters of the multimodal fusion network model through back propagation to obtain the trained multimodal fusion network model.
[0100] Furthermore, step 3 includes:
[0101] Combine Figure 3 , inputting the U component and V component of the wind field into a wind field cross feature fusion module, the wind field cross feature fusion module including a first wind field cross branch and a second wind field cross branch, wherein the U component is input into the first wind field cross branch, and the V component is input into the second wind field cross branch;
[0102] In the first wind field cross branch, the U component is input into the first feature transformation block to obtain the first feature , expressed as:
[0103] ,
[0104] Where, Indicates that the convolution kernel size is 3D convolution, Represents the U component, SAM represents the spatial attention mechanism operation, which includes the sequentially connected convolution layer, normalization layer, ReLU activation function layer, convolution layer, normalization layer and Sigmoid activation function generation layer. The specific structure diagram is as follows Figure 4 As shown;
[0105] Input the first feature into the second feature transformation block to obtain the second feature , expressed as:
[0106] ,
[0107] Input the second feature into the third feature transformation block to obtain the third feature , expressed as:
[0108] ,
[0109] Input the V component to the fourth feature transformation block to obtain the fourth feature , expressed as:
[0110] ,
[0111] Where, Represents the V component;
[0112] Input the fourth feature into the fifth feature transformation block to obtain the fifth feature , expressed as:
[0113] ,
[0114] Input the fifth feature into the sixth feature transformation block to obtain the sixth feature , expressed as:
[0115] ,
[0116] Cross-fuse the first feature and the fourth feature to obtain the first fusion feature , expressed as:
[0117] ,
[0118] Cross-fuse the second feature and the fifth feature, and then combine them with the first fusion feature obtained in the previous order Perform separation and fusion to obtain the second fusion feature , expressed as:
[0119] ,
[0120] In the formula, the symbol + represents the separation and fusion operation;
[0121] The third feature is cross-fused with the sixth feature, and then separated and fused with the second fused feature obtained in the previous sequence to obtain the third fused feature, which is expressed as:
[0122] ,
[0123] The third fusion features of different time steps are spliced to obtain features, which are expressed as:
[0124] ,
[0125] Where, Represents the third fusion feature of the g-th time step, g=1,2,3,4, Represents a splicing operation; Represents the transposed convolution layer, which includes three submodules. Each submodule consists of a transposed convolution, a ReLU activation function, and a pooling layer. Figure 5 As shown;
[0126] Among them, the structures of the first to sixth feature transformation blocks are the same, all consisting of convolution blocks and spatial attention mechanisms.
[0127] A wind field feature cross enhancement module (WCFM) is constructed to extract and fuse the features of the U and V components of the wind field through convolution operations and spatial attention mechanism (SAM).
[0128] Wind speed consists of a U component and a V component. Given their physical correlation and dynamic coupling within the wind speed vector field, a wind field cross-feature fusion module is constructed to fuse and extract wind field information. While extracting features from the U and V components of the reanalysis data, a dual-branch fusion is also performed. A phased feature fusion strategy is employed to construct a complete picture of the wind field's background.
[0129] In the feature fusion stage, a three-level progressive fusion mechanism is designed. In each level of feature extraction, the U and V components of the wind speed are transformed through convolution blocks respectively, and then spatial attention weighting is realized through SAM. The primary fusion layer constructs basic interaction features through the element-wise product of U and V branch features. The subsequent fusion layer introduces the residual connection between the previous fusion features and the current level features, splices the fusion features of different time steps along the time dimension, and realizes upsampling recovery through the transposed convolution layer.
[0130] A potential height spatiotemporal feature extraction module (GSTFM) is constructed to enhance the structural feature expression capability of the input potential height at different time scales. The GSTFM adopts a 3D U-Net structure for feature extraction. Global features are extracted by downsampling layer by layer in the encoding stage, while local details are gradually restored in the decoding stage. Skip connections allow low-level detail information to be combined with high-level global information, so that both detail information and global structure can be retained during the decoding process, thereby improving the integrity of feature expression.
[0131] After 3D U-Net processing, the network can output feature map networks of four different scales, corresponding to input1, input2, input3 and input4 in the MSFM module, and input them into the multi-scale fusion module for feature enhancement, thereby realizing hierarchical extraction of the spatiotemporal evolution characteristics of meteorological elements, effectively learning the potential patterns and trends of the early geopotential heights of typhoon sequences, strengthening the ability to capture early reanalysis data information, and improving the analysis of potential patterns and trends in time series.
[0132] Furthermore, step 3 also includes:
[0133] The geopotential height characteristics of the typhoon center area are input into the geopotential height spatiotemporal feature extraction module, combined with Figure 6 As shown in the figure, four feature maps of different scales are obtained, which are recorded as input1, input2, input3 and input4 respectively; the potential height spatiotemporal feature extraction module adopts a 3D U-Net structure, which includes 9 convolution blocks arranged in sequence, and the number of channels c of each convolution block is 16, 32, 64, 128, 128, 64, 32, 16, and 16 respectively.
[0134] Furthermore, step 3 also includes:
[0135] The feature maps of four different scales are input into the multi-scale feature fusion module, and the feature maps of different scales are uniformly scaled and fused. Figure 7 , the process is:
[0136] For feature map input1, use convolution kernel size The transposed convolution layer is operated three times, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q1;
[0137] For feature map input2, use a convolution kernel size of The transposed convolution layer is operated twice, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q2;
[0138] For feature map input3, use a convolution kernel size of The transposed convolution layer is operated once, which is processed by the transposed convolution operation, batch normalization layer and activation layer, and then processed by the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q3;
[0139] For feature map input4, use a convolution kernel size of The convolution layer of is operated once, in which the convolution operation, batch normalization layer and activation layer are processed to obtain the feature map Q4;
[0140] Concatenate the feature maps Q1 to Q4, corresponding to Figure 7 The concatenate in is then input into the efficient channel attention network to obtain the feature map Q5. The efficient channel attention network is expressed as:
[0141] ,
[0142] in, Represents the input, that is, the concatenated feature map, represents the Sigmod function, represents a one-dimensional convolution operation, Represents the adaptive average pooling operation; k represents the convolution kernel size, representing the coverage of cross-channel interaction, and the formula is:
[0143] ,
[0144] Where, Represents the nonlinear mapping function from the number of channels C to the convolution kernel size k, Indicates the closest Odd number, and b are constants, C is the number of channels;
[0145] The feature map Q5 is then processed by a three-dimensional convolution layer, a batch normalization operation, and a ReLU activation function layer to obtain the final extracted feature map Q.
[0146] Furthermore, step 3 also includes:
[0147] After the typhoon one-dimensional data passes through the embedding layer, it is input into the generator together with the feature map Q5 and the feature map Q. The process is as follows:
[0148] Input the typhoon position coordinates, central wind speed and central pressure data into the embedding layer to obtain fused data;
[0149] The fused data, feature map Q5 and feature map Q are input into the scalar long short-term memory unit of the encoder, combined with Figure 8 As shown, the formula is:
[0150] Forget Gate: ,
[0151] Input Gate: ,
[0152] Output Gate: ,
[0153] Unit input: ,
[0154] Unit Status: ,
[0155] Normalizer status: ,
[0156] Hidden state: ,
[0157] in 、 、 Represent the outputs of the input gate, forget gate, and output gate respectively, 、 、 、 is the weight of the corresponding correlation matrix, 、 、 are the cell states that are fed together to the next time step, Represents the time step t The unit input, Represents the time step t The input vector, 、 、 、 represents the deviation vector, state n Is used to control the unit state Normalization of represents the Sigmoid activation function, Represents an infinitely small number to prevent errors in calculations;
[0158] The output and hidden state obtained by the encoder are input into the decoder to obtain the output result of the generator.
[0159] In the generator, the fusion data, feature map Q5 and feature map Q are flattened by the linear layer respectively, and then the features are added. The result of the addition is used as the input gate. Input to LSTM. LSTM mainly includes two inputs, one is One is , It is an initialized all-zero matrix that gradually accumulates information in the time dimension, representing the model's understanding or feature expression of the input sequence so far.
[0160] During training, the generator and discriminator compete against each other. The generator generates fake data, while the discriminator determines whether the data comes from a real dataset or the generator. As training progresses, the data generated by the generator becomes increasingly realistic, and the discriminator becomes increasingly adept at distinguishing real and fake data. Through this adversarial training, the generator is able to produce highly realistic data samples. The generator's goal is to generate realistic data by inputting random noise. It typically receives a random vector as input, processes it through a neural network, and outputs a fake data sample. The generator's goal is to confuse the discriminator, causing it to make an incorrect judgment and mistake the generated fake data for real data.
[0161] The discriminator's task is to determine whether the input data is real or generated. It receives real data and generated data as input and outputs a probability value indicating authenticity. The discriminator's goal is to distinguish real data from fake data as accurately as possible. The discriminator's output is typically a probability value between 0 and 1, with values close to 1 indicating real data and values close to 0 indicating generated data.
[0162] Furthermore, the formula of the loss function is:
[0163] ,
[0164] in, represents the adjustable weight coefficient for different losses, represents the adversarial loss in the generative adversarial network, represents the L2 loss, 、 represent the encoder losses of wind field and geopotential height respectively, Represents the physical loss function based on wind field motion.
[0165] Furthermore, the expressions of each loss function are:
[0166] The adversarial loss expression in the generative adversarial network is:
[0167] ,
[0168] In the formula, x represents real data, z represents random noise, Represents the value function of the generative adversarial network, which is the objective function of the adversarial game. Under the current configuration of generator G and discriminator D, the discriminator is able to distinguish between real and generated samples. It is also a simplified expression of the following formula. For the generator G, its goal is to minimize this loss function, that is, to deceive the discriminator and let D(G(z))→1 (make the generated samples as realistic as possible); for the discriminator D, its goal is to maximize the adversarial loss function, that is, to distinguish between real data and generated data as much as possible, and let D(x)→1 (real samples are judged to be true); E represents the mathematical expectation, p data (x) represents the distribution of real data samples, represents the potential distribution of noise in the latent space, G represents the generator, and D represents the discriminator;
[0169] The expression of L2 loss is:
[0170] ,
[0171] in, and Represent the true value and predicted value of typhoon path and intensity respectively, Indicates the total number of sequences;
[0172] The expression of encoder loss is:
[0173] ,
[0174] in, It is the basic data of wind field and geopotential height. It refers to the prediction sequence obtained by the wind field cross feature fusion module and the potential height spatiotemporal feature extraction module. are the length, width and number of channels of the image respectively, is the predicted length, is the L1 norm function;
[0175] The formula of the physical loss function based on wind field motion is:
[0176] ,
[0177] in, , , represents the kinematic viscosity coefficient, represents the V component of the wind field, represents the U component of the wind field, They are right and The first derivative of They are right and The second derivative of yes right and The first and second derivatives of .
[0178] The U and V components of wind speed are highly correlated, representing different wind speed information in orthogonal directions. Analyzing the dynamic behavior of these two components and their mutual influence reflects the wave propagation characteristics in the wind field, better describing the motion trajectory and changing trends of typhoons. Physical constraints are incorporated into a multimodal fusion network model based on a generative adversarial network and scalar long short-term memory, and the Burgers equation is used to establish a physical loss function based on wind field motion.
[0179] Step 4: Input the data in the test set into the trained multimodal fusion network model and output the typhoon path and intensity.
[0180] For example, taking a 6-hour interval as a time point, the typhoon-related information of the first eight time points (including path, intensity, wind field and geopotential height field reanalysis data) is input into the entire multimodal fusion network model. By generating adversarial networks and scalar long short-term memory units, the laws of typhoon characteristics are continuously learned, and the typhoon path and intensity of the next four time points are predicted.
[0181] In order to further verify the effectiveness of the typhoon path and intensity prediction method based on the multimodal fusion network model described in the present invention and the accuracy of typhoon prediction, the following examples are used to illustrate.
[0182] Comparative experimental data for typhoon paths and typhoon intensity are shown in Tables 1 and 2, respectively. Table 1 compares the results of the present invention's method with 11 prior art deep learning methods for typhoon path prediction. The MAE objective evaluation metric (MAE) is used to objectively evaluate typhoon path prediction accuracy. A smaller MAE value indicates a higher accuracy in path prediction.
[0183] Table 2 compares the results of the present invention's method with seven prior art deep learning methods for typhoon intensity prediction. Typhoon intensity is closely related to wind speed and central pressure, and can be predicted by predicting wind speed and central pressure. The first four methods only predict typhoon wind speed, while the last three methods predict both wind speed and central pressure simultaneously. The objective evaluation index for typhoon intensity prediction, MAE, is used to objectively evaluate the accuracy of typhoon intensity prediction. The smaller the MAE objective evaluation index value, the higher the accuracy of wind speed and central pressure prediction. Compared with other deep learning methods, the present invention can better learn the basic characteristics of a typhoon and its background field characteristics, thereby enabling accurate typhoon path and intensity prediction, improving the application value of the present method.
[0184] Table 1 Comparison results of MAE indicators for typhoon track prediction
[0185]
[0186] Table 2 Comparison results of MAE indicators for typhoon intensity prediction
[0187]
[0188] Based on the method of the present invention, path predictions were performed for five different typhoon cases: Typhoon Noru (1705) in 2017, Typhoons Mangkhut (1822) and Kongrey (1825) in 2018, and Typhoons Lingling (1913) and Faxai (1915) in 2019. The prediction results show that the trajectories of both Kongrey and Faxai exhibited a gradual shift from northwest to southeast, with Faxai's trajectory showing a more pronounced turning point. The prediction results indicate that the present invention's predictions of path turning points have minimal deviation from the true path and are capable of accurately predicting the typhoon's overall trajectory. Throughout its life cycle, Typhoon Lingling exhibited a trend of moving from the ocean to the northwest and rapidly crossing land. Comparing the predicted path with the true path shows that the present invention was able to accurately predict the typhoon's northward movement and landfall location. Typhoon Noru's path was relatively complex, undergoing multiple significant transitions throughout its life cycle, resulting in a complex and tortuous true path. In the mid-term and final forecasts for NORU, the model captured the motion trends near the turning point well. Finally, based on the forecast results for Typhoon MANGKHUT, as a typical westward-moving typhoon, MANGKHUT's overall motion trend is relatively stable. The forecast results for the five typhoons above show that this method is able to closely match their actual paths, demonstrating its stability and reliability in typhoon path prediction tasks. Therefore, it can be seen that compared with other deep learning methods, the method described in the present invention can better and more accurately predict typhoon paths.
Claims
1. A typhoon path and intensity prediction method based on a multimodal fusion network model, characterized in that: The following steps are involved: Step 1: Obtain the reanalysis data of the typhoon's location coordinates, central wind speed, central pressure, and wind field and geopotential height in the typhoon's central area, preprocess them according to a uniform time resolution, and divide the preprocessed data into a training set and a test set in proportion; Step 2: Build a multimodal fusion network model driven by both data and physics. The multimodal fusion network model includes a wind field cross-feature fusion module, a potential height spatiotemporal feature extraction module, a multi-scale feature fusion module, and a fusion network module. Step 3: Construct a loss function for the multimodal fusion network model, input the training set data into the multimodal fusion network model, calculate the loss, and update the parameters of the multimodal fusion network model through back propagation to obtain a trained multimodal fusion network model. Step 4: Input the data in the test set into the trained multimodal fusion network model to output the typhoon path and intensity; Step 3 includes: Input the U component and V component of the wind field into the wind field cross feature fusion module, where the U component is input into the first wind field cross branch and the V component is input into the second wind field cross branch; In the first wind field cross branch, the U component is input into the first feature transformation block to obtain the first feature , expressed as: ， Where, Indicates that the convolution kernel size is 3D convolution, Represents the U component, and SAM represents the spatial attention mechanism operation; Input the first feature into the second feature transformation block to obtain the second feature , expressed as: ， Input the second feature into the third feature transformation block to obtain the third feature , expressed as: ， Input the V component to the fourth feature transformation block to obtain the fourth feature , expressed as: ， Where, Represents the V component; Input the fourth feature into the fifth feature transformation block to obtain the fifth feature , expressed as: ， Input the fifth feature into the sixth feature transformation block to obtain the sixth feature , expressed as: ， Cross-fuse the first feature and the fourth feature to obtain the first fusion feature , expressed as: ， Cross-fuse the second feature and the fifth feature, and then combine them with the first fusion feature obtained in the previous order Perform separation and fusion to obtain the second fusion feature , expressed as: ， In the formula, the symbol + represents the separation and fusion operation; The third feature is cross-fused with the sixth feature, and then separated and fused with the second fused feature obtained in the previous sequence to obtain the third fused feature, which is expressed as: ， The third fusion features of different time steps are spliced to obtain feature O, which is expressed as: ， Where, Represents the third fusion feature of the g-th time step, g=1,2,3,4, represents the transposed convolutional layer, Represents a splicing operation; Among them, the structures of the first to sixth feature transformation blocks are the same, all consisting of convolution blocks and spatial attention mechanisms.
2. The typhoon path and intensity prediction method based on the multimodal fusion network model according to claim 1 is characterized in that: The fusion network module includes a generator and a discriminator. The generator includes an encoder and a decoder. The discriminator includes an encoder. Both the encoder and the decoder are composed of scalar long short-term memory units.
3. The typhoon path and intensity prediction method based on the multimodal fusion network model according to claim 2 is characterized in that: Step 3 also includes: The geopotential height characteristics of the typhoon center area are input into the geopotential height spatiotemporal feature extraction module to obtain four feature maps of different scales, which are recorded as input1, input2, input3 and input4 respectively; the geopotential height spatiotemporal feature extraction module adopts a 3D U-Net structure.
4. The typhoon path and intensity prediction method based on a multimodal fusion network model according to claim 3 is characterized in that: Step 3 also includes: The feature maps of four different scales are input into the multi-scale feature fusion module, and the feature maps of different scales are uniformly scaled and fused. The process is as follows: For feature map input1, use convolution kernel size The transposed convolution layer is operated three times, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q1; For feature map input2, use a convolution kernel size of The transposed convolution layer is operated twice, each time through the transposed convolution operation, batch normalization layer and activation layer, and then through the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q2; For feature map input3, use a convolution kernel size of The transposed convolution layer is operated once, which is processed by the transposed convolution operation, batch normalization layer and activation layer, and then processed by the 3D convolution layer, batch normalization layer and activation layer to obtain the feature map Q3; For feature map input4, use a convolution kernel size of The convolution layer of is operated once, in which the convolution operation, batch normalization layer and activation layer are processed to obtain the feature map Q4; The feature maps Q1 to Q4 are concatenated and then input into the efficient channel attention network to obtain the feature map Q5. The efficient channel attention network is expressed as: ， in, Represents the input, that is, the feature map after the concatenation of feature maps Q1 to Q4, represents the Sigmod function, represents a one-dimensional convolution operation, Represents the adaptive average pooling operation; k represents the convolution kernel size, and the formula is: ， Where, Represents the nonlinear mapping function from the number of channels C to the convolution kernel size k, Indicates the closest Odd number, and b are constants, C is the number of channels; The feature map Q5 is then processed by a three-dimensional convolution layer, a batch normalization operation, and a ReLU activation function layer to obtain the final extracted feature map Q.
5. The typhoon path and intensity prediction method based on the multimodal fusion network model according to claim 4 is characterized in that: Step 3 also includes: After the typhoon one-dimensional data passes through the embedding layer, it is input into the generator together with the feature map Q5 and the feature map Q. The process is as follows: Input the typhoon position coordinates, central wind speed and central pressure data into the embedding layer for embedding operation to obtain fused data; The fusion data, feature map Q5 and feature map Q are integrated through the fully connected layer to obtain the feature , the features Input to the scalar long short-term memory unit of the encoder, the formulas are: Forget Gate: , Input Gate: , Output gate: , Unit input: , Unit Status: , Normalizer status: , Hidden state: , in 、 、 Represent the outputs of the input gate, forget gate, and output gate respectively, 、 、 、 is the weight of the corresponding correlation matrix, 、 、 are the cell states that are fed together to the next time step, Represents the time step t The unit input, Represents the time step t The input vector, 、 、 、 represents the deviation vector, state n Is used to control the unit state Normalization of represents the Sigmoid activation function, Represents an infinitely small number; The output of the encoder and hidden state Input to the decoder, and the output of the decoder is the output result of the generator.
6. The typhoon path and intensity prediction method based on a multimodal fusion network model according to claim 1, characterized in that: The formula of the loss function is: ， in, represents the adjustable weight coefficient for different losses, represents the adversarial loss in the generative adversarial network, represents the L2 loss, 、 represent the encoder losses of wind field and geopotential height respectively, Represents the physical loss function based on wind field motion.
7. The typhoon path and intensity prediction method based on a multimodal fusion network model according to claim 6, characterized in that: The expressions of each loss function are: The adversarial loss expression in the generative adversarial network is: ， In the formula, x represents real data, z represents random noise, represents the value function of the generative adversarial network, which is the objective function of the adversarial game. E represents the mathematical expectation, p data (x) represents the distribution of real data samples, represents the potential distribution of noise in the latent space, G represents the generator, and D represents the discriminator; The expression of L2 loss is: ， in, and Represent the true value and predicted value of typhoon path and intensity respectively, Indicates the total number of sequences; The expression of encoder loss is: ， in, It is the basic data of wind field and geopotential height in the center of typhoon. It refers to the prediction sequence obtained by the wind field cross feature fusion module and the potential height spatiotemporal feature extraction module. are the length, width and number of channels of the wind field variable map or the geopotential height variable map, respectively. is the predicted length, is the L1 norm function; The formula of the physical loss function based on wind field motion is: ， in, , , represents the kinematic viscosity coefficient, represents the V component of the wind field, represents the U component of the wind field, They are right and The first derivative of They are right and The second derivative of yes right and The first and second derivatives of .