Typhoon rapid enhancement trend prediction method fused with multi-source satellite data

Abstract

The invention relates to the technical field of meteorological prediction, in particular to a typhoon rapid enhancement trend prediction method fusing multi-source satellite data, which comprises the following steps: constructing a cloud picture data set, and training a CNN spatial feature extractor by using the cloud picture data set; constructing a space-time sequence data set, and inputting the remote sensing cloud picture sequence into a CNN spatial feature extractor to obtain an image feature tensor; training a multi-modal fusion feature extractor by using the image feature tensor and the corresponding typhoon related parameter sequence to obtain the multi-modal fusion feature extractor; and constructing an auxiliary sequence data set, splicing the multi-modal fusion feature and the auxiliary sequence feature to obtain a classification training feature, training the LSTM network, and finally outputting a confidence coefficient that whether the sample is rapidly enhanced or not. According to the method, the satellite remote sensing cloud picture and various meteorological data are subjected to multi-stage coding fusion, an information source is provided for predicting whether the typhoon intensity is rapidly enhanced or not, and the prediction precision of rapid enhancement trend judgment of the typhoon intensity is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological forecasting, and in particular to a method for forecasting a typhoon rapid intensification trend by integrating multi-source satellite data. Background Art

[0002] Tropical cyclones (typhoons) are one of the most destructive weather systems on Earth. Over the past half century, the forecasts of typhoon tracks and intensities have improved significantly, but the ability to predict the rapid intensification (RI) of typhoon intensity is still insufficient. Previous studies on the rapid intensification of typhoons focused on using predictive variables as features, such as the SHIPS series of statistical typhoon intensity forecasts, or by analyzing the climate characteristics and spatiotemporal characteristics of tropical cyclones (such as monsoons, months, longitude and latitude, pressure changes, monthly average water temperature fields, etc.), using statistics to find some laws as an important basis for judging whether tropical cyclones will rapidly intensify. However, the research on predictive variables lacks the exploration of original data, and some of the laws found can only be used as auxiliary means, making it difficult to quickly and accurately predict whether a typhoon will rapidly intensify. With the continuous development of my country's meteorological satellites, the measurement of relevant meteorological data has become more accurate and complete. Therefore, applying relevant technologies of artificial intelligence to identify the trend of rapid intensification of typhoon intensity is a good idea to solve this problem.

[0003] With the rapid development of machine learning and deep learning technologies in the field of artificial intelligence, they have been widely used in finance, agriculture, production and other fields. For the identification of rapid increase in typhoon intensity in the meteorological field, computer vision technology and time series prediction technology can be combined to solve this problem. In 2020, Ching-Yuan Bai et al. from National Taiwan University proposed an attention-based deep learning method from the perspective of data science, using ConvLSTM to identify rapid typhoon intensification. ConvLSTM uses a convolutional neural network to associate cloud map features with typhoon intensity in the first stage, and uses a long short-term memory network LSTM in the second stage to use time information to smooth typhoon features.

[0004] According to the statistics of typhoons in the northwest Pacific region in recent years, the positive samples of typhoons with rapid intensity increase account for less than 10% of all samples. Therefore, when training the model, it is necessary to consider the impact of uneven sample distribution on the identification of the trend of rapid intensity increase of typhoons. The previous methods only used satellite remote sensing cloud images and corresponding typhoon intensity data for RI prediction, and alleviated the problem of uneven sample distribution by resampling, but also brought two new problems: (1) Only based on the single data feature source of satellite remote sensing cloud images, RI prediction cannot provide sufficient information; (2) The time series prediction model based only on LSTM uses the autoregressive prediction method, which has the problem of cumulative error. Summary of the invention

[0005] The purpose of the present invention is to use satellite remote sensing cloud images and various meteorological data for multi-stage coding fusion to jointly provide an information source for predicting whether the typhoon intensity will increase rapidly, thereby solving the problem of insufficient information provided by the original method. To this end, a typhoon rapid intensification trend prediction method integrating multi-source satellite data is provided to alleviate the problem of cumulative errors.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] A typhoon rapid intensification trend prediction method integrating multi-source satellite data is implemented by a rapid intensification trend discrimination model, wherein the rapid intensification trend discrimination model includes a CNN spatial feature extractor, a multimodal fusion feature extractor, and an LSTM classification network;

[0008] The method comprises the following steps:

[0009] Step 1: construct a cloud image dataset, use the cloud image dataset to train the CNN spatial feature extractor, and complete the training of the CNN spatial feature extractor;

[0010] Step 2, constructing a spatiotemporal sequence data set, wherein the spatiotemporal sequence data set includes a plurality of consecutive remote sensing cloud image sequences at the same time interval and typhoon-related parameters at the corresponding moment of each remote sensing cloud image; inputting the remote sensing cloud image sequence in the spatiotemporal sequence data set into a trained CNN spatial feature extractor to obtain an image feature tensor; using the typhoon-related parameters corresponding to the remote sensing cloud image in the spatiotemporal sequence data set and the image feature tensor output by the CNN spatial feature extractor as inputs to a multimodal fusion feature extractor, training the multimodal fusion feature extractor, and obtaining a multimodal fusion feature extractor;

[0011] Step 3: construct an auxiliary sequence data set, concatenate the multimodal fusion features corresponding to the spatiotemporal sequence data set with its corresponding auxiliary sequence features to obtain classification training features, use the classification training features as the input of the LSTM classification network, train the LSTM network, and finally output the confidence level of whether the sample is rapidly enhanced.

[0012] Step 4, for the real-time remote sensing cloud images and typhoon-related parameters of the typhoon to be measured, pre-process the remote sensing images and typhoon-related parameters according to the optimal path of the typhoon at the current time and the historical time, obtain the remote sensing cloud image sequence and the numerical sequence data corresponding to the typhoon-related parameters, and obtain the auxiliary data of the corresponding time, input these data into the rapid enhancement trend discrimination model, and finally obtain the confidence of whether the typhoon intensity is rapidly increasing.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] The present invention mainly solves the problems of slow judgment speed, low accuracy, reliance on statistical laws, and unsuitability for large-scale meteorological data in traditional typhoon rapid intensification judgment methods, effectively avoids the subjectivity of meteorological personnel's predictions, alleviates the problems of uneven sample distribution and data missing, and optimizes by integrating typhoon satellite remote sensing cloud images and multi-dimensional numerical features, thereby improving the accuracy of rapid intensification trend judgment.

[0015] Compared with the current traditional method of judging the trend of rapid increase in typhoon intensity, the present invention avoids the subjectivity of business forecasters and can maintain stable and efficient work, thereby reducing the tedious forecasting work of business personnel and improving the efficiency of judging the trend of rapid increase in typhoon intensity.

[0016] Compared with the existing technology, the present invention adopts multi-source data plus artificial intelligence technology to solve the problem of typhoon RI trend identification, and proposes a multimodal fusion attention time series model framework. This framework designs a novel multimodal fusion feature extractor based on multiple attention mechanisms, which can more accurately extract typhoon RI features from multiple data dimensions and improve the accuracy of typhoon RI trend identification.

[0017] Due to the problems of missing data and unbalanced data distribution in multi-source data, the multi-stage training method with different data dimensions can be compatible with the subsequent typhoon data input of more dimensions and years, and has good scalability.

[0018] The present invention proposes the concept of rapid intensification of weak typhoons. By introducing intensity gradient parameters, the accuracy of trend discrimination when the typhoon is rapidly intensified and in a critical state can be better improved, and the intelligence of meteorological services is further promoted. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a fast enhanced trend discrimination model training according to an embodiment of the present invention;

[0021] Figure 2 Schematic diagram of the network structure of the first unimodal self-attention network and the second unimodal self-attention network according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the network structure of a dual-channel cross-attention network according to an embodiment of the present invention;

[0023] Figure 4 Schematic diagram of the network structure of the multimodal fusion attention network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0025] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance, or implying any such actual relationship or order between these entities or operations. In addition, the terms "connected", "connected", etc. can be directly connected between elements, or indirectly connected via other elements.

[0026] The present invention is achieved through the following technical solutions: Figure 1 As shown, a typhoon rapid intensification trend prediction method integrating multi-source satellite data is implemented by a rapid intensification trend discrimination model, and the rapid intensification trend discrimination model includes a CNN spatial feature extractor, a multimodal fusion feature extractor, and an LSTM classification network. The method includes the following steps:

[0027] Step 1: Build a cloud image dataset and use the cloud image dataset to train the CNN spatial feature extractor to complete the training of the CNN spatial feature extractor.

[0028] The cloud image dataset contains several data without time series information, and the format of any data is <remote sensing cloud image, intensity>, and there is no time series information between any two data. The remote sensing cloud image of the best typhoon path obtained by satellite, each remote sensing cloud image has the typhoon intensity (abbreviated as intensity) at the corresponding time. The typhoon intensity refers to the maximum wind speed in the typhoon center at the corresponding time of the remote sensing cloud image, which comes from the babj file.

[0029] The cloud image dataset is used as a training dataset for the CNN spatial feature extractor. In this embodiment, the typhoon vortex center is clipped to expand to the surrounding areas according to the typhoon optimal path information. The remote sensing cloud image with a resolution of 400*400 (longitude and latitude) is automatically scaled when input into the CNN spatial feature extractor to meet the input size requirements of different CNN spatial feature extractors. In this embodiment, the CNN spatial feature extractor uses the ResNet50 model, and the image size of the remote sensing cloud image is 224*224. The trained CNN spatial feature extractor can identify typhoon image features from the received remote sensing cloud image. Each input remote sensing cloud image is outputted with a typhoon image feature of size 1×1×2048 through the global average pooling layer of the CNN spatial feature extractor, and then mapped and reduced to an image feature tensor of 1×512 dimensions through a linear layer. .

[0030] Step 2, constructing a spatiotemporal sequence data set, wherein the spatiotemporal sequence data set includes a plurality of consecutive remote sensing cloud image sequences at the same time interval and typhoon-related parameters at the corresponding moment of each remote sensing cloud image; inputting the remote sensing cloud image sequence in the spatiotemporal sequence data set into a trained CNN spatial feature extractor to obtain an image feature tensor; using the typhoon-related parameters corresponding to the remote sensing cloud image in the spatiotemporal sequence data set and the image feature tensor output by the CNN spatial feature extractor as inputs to a multimodal fusion feature extractor, training the multimodal fusion feature extractor, and obtaining a multimodal fusion feature extractor.

[0031] The step 2 specifically includes the following steps:

[0032] Step 2-1, construct a spatiotemporal series dataset.

[0033] The spatiotemporal sequence data set includes a sequence of remote sensing cloud images with multiple consecutive same time intervals and typhoon-related parameters at the corresponding moment of each remote sensing cloud image, wherein the remote sensing cloud image is obtained in the same way as the remote sensing cloud image in step 1, and is a remote sensing cloud image of the best path of the typhoon obtained by satellite; the typhoon-related parameters at the corresponding moment of the remote sensing cloud image include intensity, overburden cloud top, temperature, humidity, and a rapid enhancement mark is carried for the remote sensing cloud image sequence. In this embodiment, the overburden cloud top is the number of overburden cloud tops within 100km×500km of the typhoon center; the temperature and humidity are from the temperature and humidity profile of the polar orbit satellite data, the temperature is the mean of the maximum temperature of the middle layer M of the typhoon, and the humidity is the mean of the maximum humidity of the bottom layer M of the typhoon (default M=10); the rapid enhancement mark is calculated based on the babj file to determine whether the typhoon intensity is rapidly enhanced within 12 hours or 24 hours after the last typhoon moment in the sequence data, 1 means it will happen, and 0 means it will not happen. In this embodiment, if the current remote sensing cloud image time is T, then if the typhoon intensity at the corresponding time T+12 hours is equal to or exceeds 7m / s, it is a rapid enhancement and is marked as 1, otherwise it is marked as 0.

[0034] The spatiotemporal series dataset described in this solution consists of three phases of datasets, namely the first phase dataset D1, the second phase dataset D2, and the third phase dataset D3, where:

[0035] The first stage data set D1 includes several sequence data with time series information. The format of each sequence data is: Figure 1 ,Remote sensing cloud Figure 2 ,...,remote sensing cloud map N},{intensity 1,intensity 2,...,intensity N},fast enhancement mark>, and the remote sensing cloud in the same sequence data Figure 1 ~Remote sensing cloud image N is a sequence with continuous time intervals from the same typhoon number. N is the number of remote sensing cloud images collected in a sequence data. The rapid enhancement mark in a sequence data is the rapid enhancement mark corresponding to remote sensing cloud image N. The time interval of each data in the sequence data is 6 hours, and the sequence data length N defaults to 4.

[0036] For example, combined with Table 1, the time interval for collecting remote sensing cloud images is set to 6 hours, and the number of data items N=4, then the first sequence data can be expressed as: Figure 1 ,Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 },{Intensity 1,Intensity 2,Intensity 3,Intensity 4},Rapid Enhancement Flag>, the corresponding collection time is 1997-02-02-00, 1997-02-02-06, 1997-02-02-12, 1997-02-02-18, indicating 4 consecutive time data with a time interval of 6 hours from 0:00 on February 2, 1997 to 18:00 on February 2, 1997, and these 4 data belong to the same typhoon number 9702; the rapid enhancement flag of the remote sensing cloud image corresponding to the most recent time 1997-02-02-18 is 1, so the rapid enhancement flag of the first data sequence is 1.

[0037] Next, the second sequence data can be expressed as: <{Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 , remote sensing cloud image 5}, {intensity 2, intensity 3, intensity 4, intensity 5}, rapid enhancement mark>, the corresponding time is 1997-02-02-06, 1997-02-02-12, 1997-02-02-18, 1997-02-03-00, indicating 4 consecutive time data with a time interval of 6 hours from 6:00 on February 2, 1997 to 0:00 on February 3, 1997, and these 4 data belong to the same typhoon number 9702; the rapid enhancement mark of the remote sensing cloud image corresponding to the latest time 1997-02-03-00 is 0, and the rapid enhancement mark of the second sequence is 0. And so on, no more details.

[0038] Table 1 Parameters of the first stage dataset D1

[0039] In the process of constructing a spatiotemporal sequence data set, if the number of any sequence data is less than N (N=4), or multiple consecutive equally spaced data are missing, the method of copying the most recent parameters is used to complete them. For example, if the current moment is the initial moment 1997-02-02-00, then copy N-1 parameters of the initial moment 1997-02-02-00 to complete the sequence data <{ Remote Sensing Cloud Figure 1 ,Remote sensing cloud Figure 1 ,Remote sensing cloud Figure 1 ,Remote sensing cloud Figure 1 },{intensity 1, intensity 1, intensity 1, intensity 1}, fast enhancement mark>; For example, if the current time is 1997-02-02-06, then copy N-2 parameter completion sequence data at 1997-02-02-06<{remote sensing cloud Figure 1 ,Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 2 },{intensity1,intensity2,intensity2,intensity2},fast enhancement mark>.

[0040] The second stage data set D2 includes several sequence data with time series information. The format of each sequence data is: Figure 1 ,Remote sensing cloud Figure 2 ,...,remote sensing cloud image N},{intensity 1,intensity 2,...,intensity N},{uprush cloud top 1,uprush cloud top 2,...,uprush cloud top N},fast enhancement mark>, and the remote sensing cloud in the same sequence data Figure 1 ~Remote sensing cloud image N is a sequence with continuous time intervals from the same typhoon number, and the rapid enhancement mark is the rapid enhancement mark corresponding to remote sensing cloud image N.

[0041] For example, combined with Table 2, the time interval for collecting remote sensing cloud images is set to 6 hours, and the number of data items N=4, then the first sequence data can be expressed as: Figure 1 ,Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 },{Intensity 1, Intensity 2, Intensity 3, Intensity 4},{Upward Cloud Top 1, Upward Cloud Top 2, Upward Cloud Top 3, Upward Cloud Top 4}, Rapid Enhancement Mark>, the corresponding times are 2013-06-10-00, 2013-06-10-06, 2013-06-10-12, 2013-06-10-18, indicating 4 consecutive time data with a time interval of 6 hours from 0:00 on June 10, 2013 to 18:00 on June 10, 2013, and all 4 belong to the same typhoon number 1308; the rapid enhancement mark is 1.

[0042] Next, the second sequence data can be expressed as: <{Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 , remote sensing cloud image 5}, {intensity 2, intensity 3, intensity 4, intensity 5}, {uprush cloud top 2, uprush cloud top 3, uprush cloud top 4, uprush cloud top 5}, rapid enhancement mark>, the corresponding time is 2013-06-10-06, 2013-06-10-12, 2013-06-10-18, 2013-06-11-00, indicating 4 consecutive time data with a time interval of 6 hours from 6:00 on June 10, 2013 to 0:00 on June 11, 2013, and these 4 data all belong to the same typhoon number 1308; the rapid enhancement mark is 0; and so on, no longer repeated.

[0043] Table 2 Parameters of the second stage dataset D2

[0044] The third stage data set D3 includes several sequence data with time series information. The format of each sequence data is: Figure 1 ,Remote sensing cloud Figure 2 ,...,remote sensing cloud image N},{intensity 1,intensity 2,...,intensity N},{uprush cloud top 1,uprush cloud top 2,...,uprush cloud top N},{temperature 1,temperature 2,...,temperature N},{humidity 1,humidity 2,...,humidity N},fast enhancement mark>, and the remote sensing cloud in the same sequence data Figure 1 ~Remote sensing cloud image N is a sequence with continuous time intervals from the same typhoon number, and the rapid enhancement mark is the rapid enhancement mark corresponding to remote sensing cloud image N.

[0045] For example, combined with Table 3, the time interval for collecting remote sensing cloud images is set to 6 hours, and the number of data items N=4, then the first sequence data can be expressed as: Figure 1 ,Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 },{Intensity 1, Intensity 2, Intensity 3, Intensity 4},{Cloud top 1, Cloud top 2, Cloud top 3, Cloud top 4},{Temperature 1, Temperature 2, Temperature 3, Temperature 4},{Humidity 1, Humidity 2, Humidity 3, Humidity 4},Rapid enhancement mark>, the corresponding time is 2022-08-28-00, 2022-08-28-06, 2022-08-28-12, 2022-08-28-18, indicating 4 consecutive time data with a time interval of 6 hours from 0:00 on August 28, 2022 to 18:00 on August 28, 2022, and these 4 data belong to the same typhoon number 2211; the rapid enhancement mark is 1.

[0046] Next, the second sequence data can be expressed as: <{Remote sensing cloud Figure 2 ,Remote sensing cloud Figure 3 ,Remote sensing cloud Figure 4 , remote sensing cloud image 5},{intensity 2, intensity 3, intensity 4, intensity 5},{uprush cloud top 2, uprush cloud top 3, uprush cloud top 4, uprush cloud top 5},{temperature 2, temperature 3, temperature 4, temperature 5},{humidity 2, humidity 3, humidity 4, humidity 5}, rapid enhancement mark>, the corresponding time is 2022-08-28-06, 2022-08-28-12, 2022-08-28-18, 2022-08-29-00, indicating 4 consecutive time data with a time interval of 6 hours from 6:00 on August 28, 2022 to 0:00 on August 29, 2022, and these 4 belong to the same typhoon number 2211; the rapid enhancement mark is 0; and so on, no further details are given.

[0047] Table 3 Parameters of the third stage dataset D3

[0048] Each sequence data in the numerical sequence data set is aligned with the typhoon number by the acquisition time, where the remote sensing cloud image component is recorded as <pic1,pic2,...,pic N >; The intensity component is recorded as <speed1,speed2,...,speed N >; The upward cloud top component is recorded as <ot1,ot2,...,ot N >; The temperature component is recorded as <temp1,temp2,...,temp N >; humidity component is recorded as <vapor1,vapor2,...,vapor N >.

[0049] It should be noted that the reason for conducting three-stage data collection is that the data sets come from different years. For example, the cloud top and temperature and humidity profile data could not be obtained in 2005, which would lead to data missing. Therefore, this scheme gradually adds data features through three-stage data collection, which can alleviate data missing while ensuring the performance of model training.

[0050] Step 2-2, input the remote sensing cloud image sequence in the spatiotemporal sequence data set into the trained CNN spatial feature extractor to obtain an image feature tensor; use the typhoon-related parameters corresponding to the remote sensing cloud image in the spatiotemporal sequence data set and the image feature tensor output by the CNN spatial feature extractor as the input of the multimodal fusion feature extractor, train the multimodal fusion feature extractor, and obtain the multimodal fusion feature extractor.

[0051] The step 2-2 specifically includes the following steps:

[0052] Step 2-2-1, input the remote sensing cloud image sequence in the first stage data set D1 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity of the remote sensing cloud image at the corresponding moment in the first stage data set D1 and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the first stage multimodal fusion feature extractor MMF1.

[0053] See also Figure 1 First, N remote sensing cloud images of each sequence data in the first stage data set D1 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N > Then, the N intensity components in the first stage dataset D1 are <speed1,speed2,...,speed N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Next, the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >As multimodal training data, the multimodal fusion feature extractor is trained to obtain the first-stage multimodal fusion feature extractor MMF1.

[0054] The multimodal fusion feature extractor includes two single-modal self-attention networks, a dual-channel cross-attention network, and a multimodal fusion attention network. <p1,p2,...,p N >After inputting a unimodal self-attention network to extract encoding features, the image side features are obtained; the numerical feature tensor NE = <n1,n2,...,n N >After inputting into another unimodal self-attention network for encoding feature extraction, the numerical side features are obtained. Then the image side features and the numerical side features are input into the dual-channel cross attention network for encoding, and the information from the image side features to the numerical side features and from the numerical side features to the image side features is fused to obtain the image mixed features VEE and the numerical mixed features NEE. The image mixed features VEE and the numerical mixed features NEE are input into the multimodal fusion attention network together, and finally the first-stage multimodal fusion feature extractor MMF1 is obtained.

[0055] For more details, see Figure 2 , the two unimodal self-attention networks are defined as the first unimodal self-attention network and the second unimodal self-attention network, and the network structures of these two unimodal self-attention networks are the same.

[0056] The first unimodal self-attention network includes a self-attention layer Self, a fusion layer Add1, a feedforward network FF, and a fusion layer Add2. The image feature tensor VE= <p1,p2,...,p N >After inputting the self-attention layer Self, the first image feature F1 is obtained; in the fusion layer Add1, the image feature tensor VE= <p1,p2,...,p N >Fuse with the first image feature F1 to obtain the second image feature F2; input the second image feature F2 into the feedforward network FF to obtain the third image feature F3; fuse the second image feature F2 and the third image feature F3 in the fusion layer Add2 to obtain the image side feature F4.

[0057] The second unimodal self-attention network includes a self-attention layer Self, a fusion layer Add1, a feedforward network FF, and a fusion layer Add2. The numerical feature tensor NE = <n1,n2,...,n N >After inputting the self-attention layer Self, the first numerical feature G1 is obtained; in the fusion layer Add1, the numerical feature tensor NE = <n1,n2,...,n N >Fuse it with the first numerical feature G1 to obtain the second numerical feature G2; input the second numerical feature G2 into the feedforward network FF to obtain the third numerical feature G3; fuse the second numerical feature G2 and the third numerical feature G3 in the fusion layer Add2 to obtain the numerical side feature G4.

[0058] See also Figure 3 The dual-channel cross-attention network includes a first channel cross-attention network and a second channel cross-attention network, and the network structures of the cross-attention networks of these two channels are the same.

[0059] The first channel cross attention network includes a cross attention layer Cross1, a fusion layer Add1, a self-attention layer Self1, a fusion layer Add2, a feedforward network FF1, and a normalization layer Norm1; the second channel cross attention network includes a cross attention layer Cross2, a fusion layer Add3, a self-attention layer Self2, a fusion layer Add4, a feedforward network FF2, and a normalization layer Norm2. The image side feature F4 is input into the cross attention layer Cross1 and the cross attention layer Cross2 respectively, and the numerical side feature G4 is input into the cross attention layer Cross1 and the cross attention layer Cross2 respectively. The cross attention layer Cross1 outputs the first image side information feature W1, and the cross attention layer Cross2 outputs the first numerical side information feature H1. In the fusion layer Add1, the image side feature F4 and the first image side information feature W1 are fused to obtain the second image side information feature W2; the second image side information feature W2 is input into the self-attention layer Self1 to obtain the third image side information feature W3; in the fusion layer Add2, the second image side information feature W2 and the third image side information feature W3 are fused to obtain the fourth image side information feature W4; the fourth image side information feature W4 is input into the feedforward network FF1, and after normalization processing in the normalization layer Norm1, the image mixed feature VEE is obtained. In the fusion layer Add3, the numerical side feature G4 and the first numerical side information feature H1 are fused to obtain the second numerical side information feature H2; the second numerical side information feature H2 is input into the self-attention layer Self2 to obtain the third numerical side information feature H3; in the fusion layer Add4, the second numerical side information feature H2 and the third numerical side information feature H3 are fused to obtain the fourth numerical side information feature H4; the fourth numerical side information feature H4 is input into the feedforward network FF2, and after normalization in the normalization layer Norm2, the numerical mixed feature NEE is obtained.

[0060] See also Figure 4 The multimodal fusion attention network includes a first multi-head attention layer, a second multi-head attention layer, a third multi-head attention layer, a fourth multi-head attention layer, a fusion layer Add1, a normalization layer Norm1, a feedforward network FeedForward, and a fusion normalization layer Add2&Norm2. The first multi-head attention layer performs image-image multimodal feature fusion, the second multi-head attention layer performs image-value multimodal feature fusion, the third multi-head attention layer performs value-image multimodal feature fusion; and the fourth multi-head attention layer performs data-value multimodal feature fusion.

[0061] Image mixed feature VEE as key K VEE , value V VEE and query Q VEE , numerical hybrid feature NEE as key K NEE , value V NEE and query Q NEE , the key, value, and query are fed into four multi-head attention layers, each of which is calculated as follows:

[0062] First multi-head attention layer (image-image):

[0063] Second multi-head attention layer (image-value):

[0064] The third multi-head attention layer (value-image):

[0065] Fourth multi-head attention layer (value-value):

[0066] in, They are the features output by the first, second, third and fourth multi-head attention layers respectively.

[0067] The features output by the four multi-head attention layers are processed through a series of processes including the fusion layer Add1, the normalization layer Norm1, the feedforward network FeedForward, and the fusion normalization layer Add2&Norm2 to form multimodal fusion features.

[0068] The multimodal fusion feature extractor is initialized with random parameters and inputs each sequence data in the first stage data set D1 in turn. For any sequence data , and obtain the multimodal fusion features:

[0069] in, represents the multimodal fusion feature corresponding to the x-th sequence data in the first stage dataset D1, The format is:

[0070] in, Represents the multimodal fusion features corresponding to the i-th remote sensing cloud image and the numerical data of typhoon-related parameters in the sequence data x, i=1,2,...,N, N is the number of remote sensing cloud images in each sequence data;

[0071] Fusion of multimodal features The chronological order is used as the input of the LSTM classification network and joint training is performed until all sequence data in the first-stage data set D1 are trained, and the first-stage multimodal fusion feature extractor MMF1 is obtained.

[0072] Step 2-2-2, input the remote sensing cloud image sequence in the second stage data set D2 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity of the remote sensing cloud image at the corresponding moment in the second stage data set D2, the cloud top and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the second stage multimodal fusion feature extractor MMF2.

[0073] Taking the parameters of the multimodal fusion feature extractor MMF1 in the first stage as the initial value, the N remote sensing cloud images of each sequence data in the second stage data set D2 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N >

[0074] The N intensity components in the second stage data set D2 <speed1,speed2,...,speed N >、N upward rushing cloud tops <ot1,ot2,...,ot N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Set the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >As multimodal training data, train the multimodal fusion feature extractor. For any sequence data , and obtain the multimodal fusion features:

[0075] in, Represents the multimodal fusion feature corresponding to the x-th sequence data in the second stage data set D2. The chronological order is used as the input of the LSTM classification network and joint training is performed until all sequence data in the second-stage data set D2 are trained, and the second-stage multimodal fusion feature extractor MMF2 is obtained.

[0076] Step 2-2-2 aims to further train the multimodal fusion feature extractor trained in step 2-2-1 to make up for the problem of missing training data and improve the model recognition accuracy.

[0077] Step 2-2-3, input the remote sensing cloud image sequence in the third stage data set D3 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity, overburden cloud top, temperature and humidity profile of the remote sensing cloud image at the corresponding moment in the third stage data set D3 and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the third stage multimodal fusion feature extractor MMF3.

[0078] Taking the parameters of the second-stage multimodal fusion feature extractor MMF2 as the initial values, the N remote sensing cloud images of each sequence data in the third-stage data set D3 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N >

[0079] The N intensity components in the third stage data set D3 <speed1,speed2,...,speed N >、N upward cloud top components <ot1,ot2,...,ot N >, N temperature components <temp1,temp2,...,temp N > and N humidity components <vapor1,vapor2,...,vapor N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Set the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >As multimodal training data, train the multimodal fusion feature extractor. For any sequence data , and obtain the multimodal fusion features:

[0080] in, Represents the multimodal fusion feature corresponding to the xth sequence data in the third stage data set D3. The chronological order is used as the input of the LSTM classification network and joint training is performed until all sequence data in the third-stage data set D3 are trained, and the third-stage multimodal fusion feature extractor MMF3 is obtained.

[0081] Step 2-2-3 aims to further train the multimodal fusion feature extractor trained in step 2-2-4 to make up for the problem of missing training data and improve the model recognition accuracy.

[0082] Step 2 constructs a three-stage data set and conducts three-stage training on the multimodal fusion feature extractor. After the first stage of training, the model is able to predict whether a typhoon will rapidly intensify given the typhoon intensity numerical sequence data and typhoon cloud map sequence data. The second and third stage trainings are supplemented with the upper cloud top numerical sequence data and temperature and humidity profile sequence data, respectively, adding more features for prediction, which can improve the accuracy of rapid enhancement predictions.

[0083] In step 2-2, during the training of the multimodal fusion feature extractor, the multimodal fusion feature extractor is trained using the standard cross entropy loss function:

[0084] in, represents the cross entropy loss function; It indicates the probability that sample i is a positive sample (enhanced) after the rapid enhancement trend discrimination model output (output by the rapid enhancement trend discrimination model Obtained through softmax normalization); It indicates the probability that sample i is output as a negative sample (not enhanced) after the rapid enhancement trend discrimination model; y i represents the fast enhancement label of sample i (positive sample is 1, negative sample is 0); Ls represents the total amount of sequence data in the s-th stage dataset, s=1,2,3, L1 is the number of data in the first-stage dataset D1, L2 is the number of data in the second-stage dataset D2, and L3 is the number of data in the third-stage dataset D3.

[0085] Step 3: construct an auxiliary sequence data set, concatenate the multimodal fusion features corresponding to the spatiotemporal sequence data set with its corresponding auxiliary sequence features to obtain classification training features, use the classification training features as the input of the LSTM classification network, train the LSTM network, and finally output the confidence level of whether the sample is rapidly enhanced.

[0086] The auxiliary sequence data set includes a life cycle identifier and an intensity gradient parameter, wherein the life cycle identifier represents whether the typhoon is in an ascending period or a descending period within a certain period of time, the life cycle identifier of the ascending period is 1, and the life cycle identifier of the descending period is 0. Specifically, the life cycle of a typhoon in a certain sequence can be determined according to the intensity change of the entire historical sequence of the current typhoon. If the typhoon intensity at the current moment is equal to or higher than the highest intensity of the entire historical sequence of the current typhoon, the life cycle of the sequence is determined to be an ascending period; if the typhoon intensity at the current moment is lower than the highest intensity of the entire historical sequence of the current typhoon, the life cycle of the sequence is determined to be a descending period. For the typhoon initial stage sequence, the default life cycle of the typhoon is an ascending period.

[0087] The intensity gradient parameter is used in the rising period of the typhoon life cycle, and the initial prediction value is calculated twice within the range near the threshold. Assuming that the current time is t, the difference between the typhoon intensity at time t and the typhoon intensity in the previous 3 hours, 6 hours, and 9 hours is calculated respectively, and recorded as [t~t-3, t~t-6, t~t-9]. According to the typhoon rapid enhancement standard in this embodiment (12 hours ≥7m / s), if t~t-3≥2m / s, t~t-6≥3m / s, t~t-9≥5m / s, the corresponding intensity gradient parameters are recorded as 1, otherwise recorded as 0. For example, if the typhoon intensity difference [t~t-3, t~t-6, t~t-9] is [1,3,0], the corresponding intensity gradient parameter is [0,1,0]; for another example, if the typhoon intensity difference [t~t-3, t~t-6, t~t-9] is [3,0,0], the corresponding intensity gradient parameter is [1,0,0].

[0088] The auxiliary sequence dataset constructed in this step is also in the form of a three-stage dataset. For the first-stage dataset D1 in step 2-2-1, a first-stage auxiliary sequence dataset U1 corresponding to the time in the first-stage dataset D1 is constructed based on the babj file; the first-stage auxiliary sequence dataset U1 contains several auxiliary data, and the format of each auxiliary data is: <life cycle identifier, intensity gradient parameter>, and corresponds to the time of the sequence data in the first-stage dataset D1.

[0089] Similarly, a second-stage auxiliary sequence dataset U2 corresponding to the time in the second-stage dataset D2 is constructed; and a third-stage auxiliary sequence dataset U3 corresponding to the time in the third-stage dataset D3 is constructed.

[0090] Next, each auxiliary data <life cycle identifier, intensity gradient parameter> is recorded as the auxiliary sequence feature 。

[0091] 将第一阶段数据集D1中任意一条序列数据对应的多模态融合特征对应时间的辅助序列特征<supx>进行拼接：

[0092] 其中，表示与第一阶段数据集D1中任意一条序列数据x对应的多模态融合特征与其对应时间的辅助序列特征进行拼接后，得到的分类训练特征；

[0093] 将第一阶段数据集D1中所有的序列数据对应的多模态融合特征和与其对应时间的辅助序列特征都完成拼接后，得到第一分类训练特征集合I1：

[0094] 其中，x=1,2,...,L1，L1表示第一阶段数据集D1中序列数据的数量；

[0095] 同理，将第二阶段数据集D2中所有的序列数据对应的多模态融合特征和与其对应时间的辅助序列特征都完成拼接后，得到第二分类训练特征集合I2：

[0096] 其中，x=1,2,...,L2，L2表示第二阶段数据集D2中序列数据的数量；

[0097] 同理，将第三阶段数据集D3中所有的序列数据对应的多模态融合特征和与其对应时间的辅助序列特征都完成拼接后，得到第三分类训练特征集合I3：

[0098] 其中，x=1,2,...,L3，L3表示第三阶段数据集D3中序列数据的数量。

[0099] 最后，使用第一分类训练特征集合I1、第二分类训练特征集合I2、第三分类训练特征集合I3依次或共同作为LSTM分类网络的输入，对LSTM分类网络进行训练，使得LSTM网络最终能输出样本是否快速增强的置信度。

[0100] 步骤3对LSTM分类网络进行训练的过程中，通过对正负样本施以不同程度的权重，即通过重加权的方法调整训练过程中不同类别的损失来重新平衡类别使模型得到一个更好的效果：

[0101] 其中，表示加权损失函数；表示赋予正样本的损失权重，表示赋予负样本的损失权重；表示样本i经过快速增强趋势判别模型输出为正样本（增强）的概率（由快速增强趋势判别模型输出经过softmax归一化得到）；表示样本i经过快速增强趋势判别模型输出为负样本（不增强）的概率；yi表示样本i的快速增强标签（正样本为1，负样本为0）；L表示三个阶段数据集中序列数据的总量，L=L1+L2+L3，L1为第一阶段数据集D1中的数据数量，L2为第二阶段数据集D2中的数据数量，L3为第三阶段数据集D3中的数据数量。

[0102] 步骤4，对于实时到达的待测台风遥感云图和台风相关参数，根据该台风当前时刻和历史时刻的最佳路径对其遥感图像和台风相关参数进行预处理，得到遥感云图序列及其台风相关参数对应的数值序列数据，并获取对应时间的辅助数据，将这些数据输入所述快速增强趋势判别模型，最终得到台风强度是否快速增强的置信度。

[0103] 根据实施标准，即12小时内台风强度增强大于或等于7m / s记为快速增强，则在设置置信度时，可以根据验证数据集数据分布，计算RI判别阈值（本实施例中设置为0.55），即输出的置信度若大于该RI判别阈值，则判定未来12小时台风发生强度快速增强；反之，台风不发生强度快速增强。

[0104] 以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以权利要求的保护范围为准。

Claims

1. A typhoon rapid intensification trend prediction method integrating multi-source satellite data, characterized by: This is achieved through a fast enhanced trend discrimination model, which includes a CNN spatial feature extractor, a multimodal fusion feature extractor, and an LSTM classification network; The method comprises the following steps: Step 1: construct a cloud image dataset, use the cloud image dataset to train the CNN spatial feature extractor, and complete the training of the CNN spatial feature extractor; Step 2, constructing a spatiotemporal sequence data set, wherein the spatiotemporal sequence data set includes a plurality of consecutive remote sensing cloud image sequences at the same time interval and typhoon-related parameters at the corresponding moment of each remote sensing cloud image; inputting the remote sensing cloud image sequence in the spatiotemporal sequence data set into a trained CNN spatial feature extractor to obtain an image feature tensor; using the typhoon-related parameters corresponding to the remote sensing cloud image in the spatiotemporal sequence data set and the image feature tensor output by the CNN spatial feature extractor as inputs to a multimodal fusion feature extractor, training the multimodal fusion feature extractor, and obtaining a multimodal fusion feature extractor; Step 3: construct an auxiliary sequence data set, concatenate the multimodal fusion features corresponding to the spatiotemporal sequence data set with its corresponding auxiliary sequence features to obtain classification training features, use the classification training features as the input of the LSTM classification network, train the LSTM network, and finally output the confidence level of whether the sample is rapidly enhanced.

2. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 1 is characterized in that: In step 1, the cloud image data set contains several data without time series information, wherein the format of any data is <remote sensing cloud image, intensity>, and there is no time series information between any two data; the cloud image data set is used to train the CNN spatial feature extractor so that the CNN spatial feature extractor can output the image feature tensor corresponding to each remote sensing cloud image 。 3. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 1, characterized in that: In step 2, the step of constructing a spatiotemporal series data set specifically includes: The spatiotemporal series dataset consists of three phases of datasets, namely the first phase dataset D1, the second phase dataset D2, and the third phase dataset D3, where: The first-stage data set D1 includes several sequence data with time series information. The format of each sequence data is: <{remote sensing cloud image 1, remote sensing cloud image 2, ..., remote sensing cloud image N}, {intensity 1, intensity 2, ..., intensity N}, rapid enhancement mark>, and remote sensing cloud image 1 to remote sensing cloud image N in the same sequence data are sequences with continuous time intervals from the same typhoon number; The second-stage data set D2 includes several sequence data with time series information. The format of each sequence data is: <{remote sensing cloud image 1, remote sensing cloud image 2, ..., remote sensing cloud image N}, {intensity 1, intensity 2, ..., intensity N}, {uprush cloud top 1, uprush cloud top 2, ..., uprush cloud top N}, rapid enhancement mark>, and remote sensing cloud image 1 to remote sensing cloud image N in the same sequence data are sequences with continuous time intervals from the same typhoon number; The third-stage data set D3 includes several sequence data with time series information. The format of each sequence data is: <{remote sensing cloud image 1, remote sensing cloud image 2, ..., remote sensing cloud image N}, {intensity 1, intensity 2, ..., intensity N}, {uprush cloud top 1, uprush cloud top 2, ..., uprush cloud top N}, {temperature 1, temperature 2, ..., temperature N}, {humidity 1, humidity 2, ..., humidity N}, rapid enhancement mark>, and remote sensing cloud image 1 to remote sensing cloud image N in the same sequence data are sequences with continuous time intervals from the same typhoon number.

4. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 3 is characterized in that: In the step 2, the remote sensing cloud image sequence in the spatiotemporal sequence data set is input into the trained CNN spatial feature extractor to obtain an image feature tensor; the typhoon-related parameters corresponding to the remote sensing cloud image in the spatiotemporal sequence data set and the image feature tensor output by the CNN spatial feature extractor are used as inputs of the multimodal fusion feature extractor, and the multimodal fusion feature extractor is trained to obtain the multimodal fusion feature extractor, which specifically includes: Step 2-2-1, input the remote sensing cloud image sequence in the first stage data set D1 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity of the remote sensing cloud image at the corresponding moment in the first stage data set D1 and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the first stage multimodal fusion feature extractor MMF1; Step 2-2-2, input the remote sensing cloud image sequence in the second stage data set D2 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity of the remote sensing cloud image at the corresponding moment in the second stage data set D2, the cloud top and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the second stage multimodal fusion feature extractor MMF2; Step 2-2-3, input the remote sensing cloud image sequence in the third stage data set D3 into the trained CNN spatial feature extractor to obtain the image feature tensor; take the intensity, overburden cloud top, temperature and humidity profile of the remote sensing cloud image at the corresponding moment in the third stage data set D3 and the image feature tensor output by the CNN spatial feature extractor as input, train the multimodal fusion feature extractor, and obtain the third stage multimodal fusion feature extractor MMF3.

5. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 4 is characterized in that: The specific steps of step 2-2-1 are: Taking the random parameters of the multimodal fusion feature extractor as the initial value, the N remote sensing cloud images of each sequence data in the first stage data set D1 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N > The N intensity components in the first stage data set D1 <speed1,speed2,...,speed N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Set the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >Used as multimodal training data, the multimodal fusion feature extractor is trained to obtain the first-stage multimodal fusion feature extractor MMF1; The specific steps of step 2-2-2 are: Taking the parameters of the multimodal fusion feature extractor MMF1 in the first stage as the initial value, the N remote sensing cloud images of each sequence data in the second stage data set D2 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N > The N intensity components in the second stage data set D2 <speed1,speed2,...,speed N >、N upward rushing cloud tops <ot1,ot2,...,ot N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Set the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >Used as multimodal training data, the multimodal fusion feature extractor is trained to obtain the second-stage multimodal fusion feature extractor MMF2; The specific steps of step 2-2-3 are: Taking the parameters of the second-stage multimodal fusion feature extractor MMF2 as the initial values, the N remote sensing cloud images of each sequence data in the third-stage data set D3 are <pic1,pic2,...,pic N > Input the CNN spatial feature extractor trained in step 1 to obtain the image feature tensor VE of dimension N×512= <p1,p2,...,p N > The N intensity components in the third stage data set D3 <speed1,speed2,...,speed N >、N upward cloud top components <ot1,ot2,...,ot N >, N temperature components <temp1,temp2,...,temp N > and N humidity components <vapor1,vapor2,...,vapor N >Converted to a numerical feature tensor NE of dimension N×512 through the embedding layer = <n1,n2,...,n N >; Set the image feature tensor VE= <p1,p2,...,p N > and the numerical feature tensor NE = <n1,n2,...,n N >Used as multimodal training data, the multimodal fusion feature extractor is trained to obtain the third-stage multimodal fusion feature extractor MMF3; The multimodal fusion feature extractor includes two single-modal self-attention networks, a dual-channel cross-attention network, and a multimodal fusion attention network; The image feature tensor VE = <p1,p2,...,p N >After inputting a unimodal self-attention network to extract encoding features, the image side features are obtained; the numerical feature tensor NE = <n1,n2,...,n N >After inputting into another unimodal self-attention network for encoding feature extraction, the numerical side features are obtained; Then the image side features and the numerical side features are jointly input into the dual-channel cross attention network for encoding, and the information from the image side features to the numerical side features and from the numerical side features to the image side features is fused to obtain the image mixed features VEE and the numerical mixed features NEE; the image mixed features VEE and the numerical mixed features NEE are jointly input into the multimodal fusion attention network to finally obtain the multimodal fusion features.

6. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 5, characterized in that: The image feature tensor VE= <p1,p2,...,p N >After inputting a unimodal self-attention network to extract encoding features, the image side features are obtained; the numerical feature tensor NE = <n1,n2,...,n N >After inputting another unimodal self-attention network to extract the encoding features, the steps of obtaining the numerical side features include: The two unimodal self-attention networks are respectively a first unimodal self-attention network and a second unimodal self-attention network, and the network structures of the two unimodal self-attention networks are the same; The first single-modal self-attention network includes a self-attention layer Self, a fusion layer Add1, a feedforward network FF, and a fusion layer Add2; the image feature tensor VE= <p1,p2,...,p N >After inputting the self-attention layer Self, the first image feature F1 is obtained; in the fusion layer Add1, the image feature tensor VE= <p1,p2,...,p N >Fuse with the first image feature F1 to obtain the second image feature F2; input the second image feature F2 into the feedforward network FF to obtain the third image feature F3; fuse the second image feature F2 and the third image feature F3 in the fusion layer Add2 to obtain the image side feature F4; The second unimodal self-attention network includes a self-attention layer Self, a fusion layer Add1, a feedforward network FF, and a fusion layer Add2; the numerical feature tensor NE = <n1,n2,...,n N >After inputting the self-attention layer Self, the first numerical feature G1 is obtained; in the fusion layer Add1, the numerical feature tensor NE = <n1,n2,...,n N >Fuse it with the first numerical feature G1 to obtain the second numerical feature G2; input the second numerical feature G2 into the feedforward network FF to obtain the third numerical feature G3; fuse the second numerical feature G2 and the third numerical feature G3 in the fusion layer Add2 to obtain the numerical side feature G4.

7. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 6, characterized in that: The step of inputting the image side features and the numerical side features into a dual-channel cross attention network for encoding, fusing the information from the image side features to the numerical side features, and from the numerical side features to the image side features, to obtain the image mixed feature VEE and the numerical mixed feature NEE specifically includes: The dual-channel cross-attention network includes a first channel cross-attention network and a second channel cross-attention network, and the network structures of the cross-attention networks of the two channels are the same; The first channel cross attention network includes a cross attention layer Cross1, a fusion layer Add1, a self-attention layer Self1, a fusion layer Add2, a feedforward network FF1, and a normalization layer Norm1; the second channel cross attention network includes a cross attention layer Cross2, a fusion layer Add3, a self-attention layer Self2, a fusion layer Add4, a feedforward network FF2, and a normalization layer Norm2; the image side feature F4 is respectively input into the cross attention layer Cross1 and the cross attention layer Cross2, and the numerical side feature G4 is respectively input into the cross attention layer Cross1 and the cross attention layer Cross2, the cross attention layer Cross1 outputs a first image side information feature W1, and the cross attention layer Cross2 outputs a first numerical side information feature H1; the image side feature F4 and the first image side information feature W1 are fused in the fusion layer Add1 to obtain a second image side information feature W2; the second image The side information feature W2 is input into the self-attention layer Self1 to obtain the third image side information feature W3; the second image side information feature W2 and the third image side information feature W3 are fused in the fusion layer Add2 to obtain the fourth image side information feature W4; the fourth image side information feature W4 is input into the feedforward network FF1, and after normalization processing in the normalization layer Norm1, the image mixed feature VEE is obtained; the numerical side feature G4 and the first numerical side information feature H1 are fused in the fusion layer Add3 to obtain the second numerical side information feature H2; the second numerical side information feature H2 is input into the self-attention layer Self2 to obtain the third numerical side information feature H3; the second numerical side information feature H2 and the third numerical side information feature H3 are fused in the fusion layer Add4 to obtain the fourth numerical side information feature H4; the fourth numerical side information feature H4 is input into the feedforward network FF2, and after normalization processing in the normalization layer Norm2, the numerical mixed feature NEE is obtained.

8. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 7, characterized in that: The step of inputting the image mixed feature VEE and the numerical mixed feature NEE into the multimodal fusion attention network to finally obtain the multimodal fusion feature specifically includes: The multimodal fusion attention network includes a first multi-head attention layer, a second multi-head attention layer, a third multi-head attention layer, a fourth multi-head attention layer, a fusion layer Add1, a normalization layer Norm1, a feedforward network Feed Forward, and a fusion normalization layer Add2&Norm2; wherein the first multi-head attention layer performs image-image multimodal feature fusion, the second multi-head attention layer performs image-value multimodal feature fusion, and the third multi-head attention layer performs value-image multimodal feature fusion; the fourth multi-head attention layer performs data-value multimodal feature fusion; Image mixed feature VEE as key K VEE , value V VEE and query Q VEE , numerical hybrid feature NEE as key K NEE , value V NEE and query Q NEE , the key, value, and query are fed into four multi-head attention layers, each of which is calculated as follows: First multi-head attention layer: Second multi-head attention layer: Third multi-head attention layer: Fourth multi-head attention layer: in, They are the features output by the first, second, third and fourth multi-head attention layers respectively; The features output by the four multi-head attention layers are processed by the fusion layer Add1, the normalization layer Norm1, the feedforward network FeedForward, and the fusion normalization layer Add2&Norm2 to form multimodal fusion features; The multimodal fusion feature extractor is initialized with random parameters and inputs each sequence data in the first stage data set D1 in turn. For any sequence data , and obtain the multimodal fusion features: in, represents the multimodal fusion feature corresponding to the x-th sequence data in the first stage dataset D1, The format is: in, Represents the multimodal fusion features corresponding to the i-th remote sensing cloud image and the numerical data of typhoon-related parameters in the sequence data x, i=1,2,...,N, N is the number of remote sensing cloud images in each sequence data; Fusion of multimodal features As the input of the LSTM classification network, joint training is performed until all sequence data in the first-stage data set D1 are trained, and the first-stage multimodal fusion feature extractor MMF1 is obtained; Similarly, taking the parameters of the first-stage multimodal fusion feature extractor MMF1 as the initial values, the second-stage data set D2 is used to train the multimodal fusion feature extractor. For any sequence data , and obtain the multimodal fusion features: in, Represents the multimodal fusion feature corresponding to the x-th sequence data in the second stage data set D2; multimodal fusion feature As the input of the LSTM classification network, joint training is performed until all sequence data in the second-stage data set D2 are trained, and the second-stage multimodal fusion feature extractor MMF2 is obtained; Similarly, taking the parameters of the second-stage multimodal fusion feature extractor MMF2 as the initial values, the third-stage dataset D3 is used to train the multimodal fusion feature extractor. For any sequence data , and obtain the multimodal fusion features: in, Represents the multimodal fusion feature corresponding to the x-th sequence data in the third stage data set D3; multimodal fusion feature As the input of the LSTM classification network, joint training is performed until all sequence data in the third-stage data set D3 are trained, and the third-stage multimodal fusion feature extractor MMF3 is obtained.

9. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 8, characterized in that: The auxiliary sequence data set constructed in step 3 is: For the first-stage data set D1 in step 2-2-1, a first-stage auxiliary sequence data set U1 corresponding to the time in the first-stage data set D1 is constructed based on the babj file; the first-stage auxiliary sequence data set U1 includes several auxiliary data, each of which has the format of <life cycle identifier, intensity gradient parameter>, and corresponds to the time of the sequence data in the first-stage data set D1; Similarly, construct the second-stage auxiliary sequence dataset U2 corresponding to the time in the second-stage dataset D2; Similarly, construct the third-stage auxiliary sequence dataset U3 corresponding to the time in the third-stage dataset D3; Each auxiliary data <life cycle identifier, intensity gradient parameter> is recorded as the form of auxiliary sequence feature 。 10. The method for predicting the typhoon rapid intensification trend by integrating multi-source satellite data according to claim 9, characterized in that: In step 3, after the multimodal fusion features corresponding to the spatiotemporal sequence data set are spliced ​​with the corresponding auxiliary sequence features, classification training features are obtained, and the classification training features are used as the input of the LSTM classification network to train the LSTM network so that the LSTM network can finally output the confidence of whether the sample is rapidly enhanced, which specifically includes: Any sequence data in the first stage data set D1 Corresponding multimodal fusion features Auxiliary sequence features corresponding to time x >To do splicing:​ in, It represents the classification training features obtained by concatenating the multimodal fusion features corresponding to any sequence data x in the first-stage data set D1 with the auxiliary sequence features of its corresponding time; After the multimodal fusion features corresponding to all the sequence data in the first stage data set D1 and the auxiliary sequence features corresponding to the time are concatenated, the first classification training feature set I1 is obtained: Where x=1,2,...,L1, L1 represents the number of sequence data in the first stage data set D1; Similarly, after the multimodal fusion features corresponding to all the sequence data in the second stage data set D2 and the auxiliary sequence features corresponding to the time are concatenated, the second classification training feature set I2 is obtained: Where x=1,2,...,L2, L2 represents the number of sequence data in the second-stage data set D2; Similarly, after the multimodal fusion features corresponding to all the sequence data in the third stage data set D3 and the auxiliary sequence features corresponding to the time are concatenated, the third classification training feature set I3 is obtained: Where x=1,2,...,L3, L3 represents the number of sequence data in the third stage data set D3; Finally, the first classification training feature set I1, the second classification training feature set I2, and the third classification training feature set I3 are used sequentially or together as inputs of the LSTM classification network to train the LSTM classification network so that the LSTM network can ultimately output the confidence level of whether the sample is rapidly enhanced.

Images