Transform architecture-based rainfall prediction method

Abstract

The invention discloses a rainfall prediction method based on a Transform architecture, and the method comprises the steps: constructing a Transform rainfall model, carrying out the full extraction of the features of different time scales, enabling the interior of the Transform rainfall model to be provided with a multi-scale channel module, a Transform module and a prediction head module, carrying out the time sequence processing and abnormal value processing of the rainfall data of a laser radar through the Transform rainfall model, and carrying out the prediction of the abnormal value of the rainfall data of the laser radar. According to the method, a data set with reasonable data distribution is provided, and a multi-scale channel attention mechanism is provided after the data set is provided and is used for extracting features in a channel time space, performing comprehensive experimental verification on the data set and completing rainfall prediction of a target area; the method is composed of three key parts, namely a multi-scale channel module, a Transform module and a prediction head module, the multi-scale channel module is introduced for a similar three-dimensional structure of laser radar data, the feature extraction capability of the Transform is improved, the problem of memory deterioration in a long-time sequence is solved, and feature fusion is realized by allocating different weights to two channels.

Description

Technical Field

[0001] The present invention relates to the technical field of rainfall prediction, and in particular to a rainfall prediction method based on a transformer architecture. Background Art

[0002] Rainfall, snowfall and other phenomena are very common in nature. Rainfall plays a very important role in agricultural development, ecosystem maintenance, power generation and water conservancy projects. In terms of agricultural production and urban development, people have realized the impact of rainfall prediction on people's lives. In this context, the development of automated rainfall prediction technology is particularly important, especially for heavy rain and rainstorms in extreme weather conditions. Improving the accuracy and timeliness of early warnings has become a priority. The research and development of localized rainfall monitoring equipment has received strong support. By promoting the localization process, the ability to independently obtain and analyze meteorological data has been enhanced, providing more reliable protection for urban disaster prevention and emergency decision-making. The formation of rainfall is regulated by a variety of key factors, including water vapor transport, convective triggering conditions and cloud microphysical processes. Physical mechanisms such as water vapor flux convergence, lifting condensation height and ice crystal effect jointly determine the occurrence and development of precipitation.

[0003] The rainfall prediction in Chinese patent 202210218269.1 mainly relies on meteorological radar and satellite observations, and uses extrapolation to predict future rainfall conditions by analyzing the degree of change of physical quantities over time. The extrapolation method relies on horizontal radar echoes and has difficulty in characterizing the complex process of cloud water phase change and convection development in the vertical direction. Therefore, the present invention proposes a rainfall prediction method based on the transformer architecture to solve the problems existing in the prior art. Summary of the Invention

[0004] In response to the above problems, the purpose of the present invention is to propose a rainfall prediction method based on the transformer architecture. This method, based on the transformer architecture, targets the three-dimensional structure of lidar data, introduces a multi-scale channel module, improves the feature extraction capability of the transformer, solves the memory decay problem in long time series, and realizes feature fusion by assigning different weights to the dual channels.

[0005] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: a rainfall prediction method based on transformer architecture, comprising the following steps;

[0006] Step 1: Construct a Transformer rainfall model to fully extract features at different time scales. The Transformer rainfall model is internally configured with a multi-scale channel module, a Transformer module, and a prediction head module.

[0007] Step 2: Use the Transformer rainfall model to perform time series processing and outlier processing on the lidar rainfall data, and propose a dataset with reasonable data distribution;

[0008] Step 3: After proposing the dataset, a multi-scale channel attention mechanism is proposed to extract features in channel time and space;

[0009] Step 4: Comprehensive experimental verification of the dataset to complete rainfall prediction in the target area.

[0010] Further improvements are as follows: in step 1, by constructing a Transformer rainfall model, different meteorological variables including temperature, humidity, air pressure, wind speed and precipitation are organized into time series data in chronological order, and different features are normalized to ensure that each feature of the model input is at the same scale. For data at different time scales, time series data slicing is performed to construct suitable input sequences respectively.

[0011] A further improvement is that in step one, through the multi-scale channel module and multiple attention heads, the model learns dependencies of different time scales from different subspaces, so that the model can adaptively process information of different time granularities.

[0012] A further improvement is that in step 1, the dependency between any two time points in the time series is modeled through the Transformer module, while paying attention to the information of the entire sequence, thereby improving the ability to capture rainfall patterns.

[0013] Further improvements are: in the step one, the prediction head module is responsible for providing time series prediction. Rainfall prediction is usually based on time series data, predicting the rainfall situation at a future time point. The prediction head module is responsible for outputting the prediction results of continuous time. After the prediction head module outputs the original prediction value, the output is adjusted to an actually available value.

[0014] Further improvements are as follows: in step 2, the lidar rainfall data is cleaned to remove missing values, duplicate data and noise. For missing data, interpolation and adjacent data are used to fill in the missing data to ensure data integrity. By statistically identifying and removing outliers from the data, the lidar data contains abnormal data caused by equipment errors, and statistical methods are used to eliminate data points that do not conform to the normal pattern.

[0015] Further improvements are as follows: in step three, features of different time scales and spatial scales are captured through a multi-scale channel attention mechanism, and by weighting channel features of different scales, the model can adaptively learn features of multiple scales for rainfall prediction, and the multi-scale spatial and temporal features weighted by the channel attention mechanism are fused. The fusion of spatial information and temporal information is more consistent in time and space, which helps to better understand the changing trends and patterns of rainfall.

[0016] Further improvements are as follows: in step 4, historical rainfall data provided by lidar and weather station equipment are collected to ensure that the data covers different temporal and spatial scales, which facilitates the verification of the model under various environmental conditions. The time series cross-validation method is used to ensure that the model can cope with rainfall prediction tasks in different time periods and weather conditions, and to verify the performance of the multi-scale channel attention mechanism at different spatial and temporal scales. By comparing the prediction results of multi-scale features with those of single-scale features, the accuracy of the multi-scale mechanism for rainfall prediction is proved.

[0017] Further improvements are as follows: in the data set of step 2, the real rainfall data obtained is divided into training set data and test set data, all the synthetic rainfall data is used as training set data, the model is trained using the training set data, and the model is verified using the test set data. During the training process, the model parameters are adjusted and the training is repeated until the model converges and passes the verification, and the training is completed.

[0018] Further improvements are: using the trained rainfall prediction model to predict rainfall in the target area; collecting the current real rainfall image of the target area, inputting the collected real rainfall image into the trained rainfall prediction model to obtain the corresponding rainfall intensity value, and realizing rainfall prediction for the target area.

[0019] The beneficial effects of the present invention are as follows: the present invention consists of three key parts: a multi-scale channel module, a Transformer module and a prediction head module. Aiming at the three-dimensional structure of lidar data, the multi-scale channel module is introduced to improve the feature extraction capability of the Transformer and solve the memory decay problem in long time series. Feature fusion is achieved by assigning different weights to the dual channels. The variable time series processed by the multi-scale channel module is input into the Transformer network as a token, and features are further extracted through multi-head attention and cohesive structure. Finally, the extracted features are used by the prediction head module to predict rainfall conditions in the future. The Transformer model adopts an encoding and decoding structure, which can capture richer and more complex features. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a flow chart of the steps of the present invention;

[0022] Figure 2 This is the architecture diagram of the multi-scale channel module of the present invention. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that, unless otherwise specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0025] In file 202210218269.1, a rainfall prediction network model is constructed, which includes an encoder and a decoder. The encoder includes four feature balance extraction modules connected in sequence, and the decoder also includes four feature balance extraction modules connected in sequence. The outputs of the first, second, and third feature balance extraction modules of the encoder are also connected to the inputs of the third, second, and first feature balance extraction modules of the decoder, respectively. When a radar echo map is input, the future rainfall trend is effectively predicted, which overcomes the problems of huge resources required and global information loss in existing rainfall prediction methods. However, it is difficult to depict the complex process of cloud water phase change and convection development in the vertical direction. In the present invention, combined with the global modeling capabilities of deep learning methods, it is expected to break through the bottleneck of precipitation prediction in complex terrain. LiDAR overcomes the limitation of traditional weather radar, which only provides horizontal structural information, through light detection and ranging technology. It can quickly and accurately obtain vertical structural information of the atmosphere's extinction coefficient and relative humidity, providing richer data support for precipitation forecasting. Deep learning models can better capture the changing trends of time series and the correlation between variables, and have significant advantages in reducing the uncertainty of precipitation forecasting. The long short-term memory (LSTM) network model is used for rainfall forecasting, which can effectively capture the dependencies in long time series by controlling the flow of information, avoiding the common gradient vanishing or gradient exploding problems of RNN.

[0026] Example 1

[0027] according to Figure 1 、 Figure 2 , as shown, this embodiment provides a rainfall prediction method based on transformer architecture, comprising the following steps:

[0028] Step 1: Construct a Transformer rainfall model to fully extract features at different time scales. The Transformer rainfall model is internally configured with a multi-scale channel module, a Transformer module, and a prediction head module.

[0029] Step 2: Use the Transformer rainfall model to perform time series processing and outlier processing on the lidar rainfall data, and propose a dataset with reasonable data distribution;

[0030] Step 3: After proposing the dataset, a multi-scale channel attention mechanism is proposed to extract features in channel time and space;

[0031] Step 4: Comprehensive experimental verification of the dataset to complete rainfall prediction in the target area.

[0032] In step 1, by constructing a Transformer rainfall model, different meteorological variables including temperature, humidity, air pressure, wind speed and precipitation are organized into time series data in chronological order. Different features are normalized to ensure that each feature of the model input is at the same scale. For data at different time scales, time series data are sliced ​​and suitable input sequences are constructed respectively.

[0033] In step one, through the multi-scale channel module and multiple attention heads, the model learns dependencies of different time scales from different subspaces, enabling the model to adaptively process information of different time granularities.

[0034] In step 1, the Transformer module is used to model the dependency between any two time points in the time series, while focusing on the information of the entire sequence to improve the ability to capture rainfall patterns. The process of the Transformer and attention mechanism is described as follows: Assume there is a sequence X = [x1, x2, ..., x n ], which represents the i-th element of the sequence. For each element, the Query, Key, and Value are calculated respectively. Secondly, the attention weight is calculated by performing a dot product operation on the Query and Key, and the Softmax function is used for normalization. Finally, the weighted sum is calculated by combining the attention weight with the corresponding Value to obtain Out. i , this process can be described by the following formula;

[0035] Q=W Q *x i ,K=W K *x i ,V=W V *x i

[0036]

[0037] out i =sum(attn i *V j ), j = 1, 2, ..., n

[0038] In the field of time series prediction, most transformer-based architectures are based on the two-dimensionality of data, extracting data features and making predictions.

[0039] In step one, the prediction head module is responsible for providing time series predictions. Rainfall predictions are usually based on time series data, predicting rainfall conditions at a future time point. The prediction head module is responsible for outputting the prediction results for continuous time. After the prediction head module outputs the original prediction value, it adjusts the output to an actually available value.

[0040] In step 2, the lidar rainfall data is cleaned to remove missing values, duplicate data, and noise. Missing data are filled using interpolation and neighboring data to ensure data integrity. The lidar data contains abnormal data caused by equipment errors, and statistical methods are used to eliminate data points that do not conform to the normal pattern.

[0041] In step three, the multi-scale channel attention mechanism is used to capture features of different time scales and spatial scales. By weighting the channel features of different scales, the model can adaptively learn features of multiple scales for rainfall prediction. The multi-scale spatial and temporal features weighted by the channel attention mechanism are fused. The fusion of spatial information and temporal information is more temporally and spatially consistent, which helps to better understand the changing trends and patterns of rainfall. Feature fusion is achieved by assigning different weights to multiple channels. The encoder-only structure of the inverted cohesive Transformer is adopted. The variable time series processed by the truncated center module is input as a token into the Transformer network. The multi-head attention and cohesive structure are used to further extract features. Finally, the prediction head module uses the extracted features to predict rainfall conditions in the future.

[0042] In step 4, historical rainfall data provided by lidar and meteorological station equipment are collected to ensure that the data covers different spatiotemporal scales, which facilitates the verification of the model under various environmental conditions. The time series cross-validation method is used to ensure that the model can cope with rainfall prediction tasks in different time periods and weather conditions, and to verify the performance of the multi-scale channel attention mechanism at different spatial and temporal scales. By comparing the prediction results of multi-scale features with those of single-scale features, the accuracy of the multi-scale mechanism in rainfall prediction is demonstrated.

[0043] In the data set of step 2, the real rainfall data obtained is divided into training set data and test set data. All synthetic rainfall data is used as training set data. The model is trained using the training set data, and the model is verified using the test set data. During the training process, the model parameters are adjusted and trained repeatedly until the model converges and passes the verification, completing the training.

[0044] Rainfall is predicted for the target area using the trained rainfall prediction model. The current real rainfall image of the target area is collected and input into the trained rainfall prediction model to obtain the corresponding rainfall intensity value, thereby realizing rainfall prediction for the target area. In practical applications, the rainfall intensity is predicted by using the images in the video stream of the road surveillance camera in the target area as input. In addition, residents in the target area use mobile devices such as mobile phones to shoot and upload rainfall images to the cloud, and the prediction model deployed in the cloud predicts rainfall intensity.

[0045] Example 2

[0046] like Figure 2 As shown, the two-dimensional input data (relative humidity and extinction coefficient) are combined into a three-dimensional data structure. Assume that the input data Where T represents the time dimension, N represents the vertical height, and C represents the number of channels. During the processing, global pooling is not directly used to solve the problem of channel dependency, because simple global pooling is difficult to capture the complex nonlinear dependency between channels.

[0047] In order to extract multi-channel information more effectively, a multi-scale pooling mechanism is proposed to gradually extract multi-scale spatial information. The applicability of multi-scale information is also verified in neural network models such as UNET. Specifically, multi-layer average pooling is used to compress at different scales. Each layer of pooling operation further reduces the spatial dimension, allowing the model to capture more detailed channel features at different scales. The specific steps are: the spatial dimension is compressed by the first layer of average pooling. Then the second layer of average pooling is used to compress the spatial dimension into Similarly, after three layers of average pooling, the global spatial information is gradually pooled to four scales. Compared with single-layer global pooling, this multi-scale information extraction mechanism can learn convolutional features more carefully and enhance the sensitivity of feature fusion.

[0048] The final step is to map the output back to the original channel dimension. This allows the model to use the rescaled channel features. This approach allows it to more comprehensively capture the channel dependencies of the lidar data across the entire channel range and strengthen key channels. This adaptive mechanism enables the model to dynamically adjust the importance of different channels, effectively enhancing the accuracy and robustness of predictions.

[0049] In the multi-scale pooling process, the multi-scale channel module has different correlations in the spatial dimension. The correlation of the closer parts in the spatial dimension is stronger, while the correlation of the farther parts is weaker. The channel attention mechanism implemented by the multi-scale channel module is not limited to the local receptive field of the convolution filter, but a global attention across all channels. Finally, the multi-scale channel module can improve the model's robustness to noise in long time series prediction, while better capturing the nonlinear relationship between channels in lidar data.

[0050] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A rainfall prediction method based on transformer architecture, characterized in that: The following steps are included: Step 1: Construct a Transformer rainfall model to fully extract features at different time scales. The Transformer rainfall model is internally configured with a multi-scale channel module, a Transformer module, and a prediction head module. Step 2: Use the Transformer rainfall model to perform time series processing and outlier processing on the lidar rainfall data, and propose a dataset with reasonable data distribution; Step 3: After proposing the dataset, a multi-scale channel attention mechanism is proposed to extract features in channel time and space; Step 4: Comprehensive experimental verification of the dataset to complete rainfall prediction in the target area.

2. The rainfall prediction method based on the transformer architecture according to claim 1, characterized in that: In step 1, a Transformer rainfall model is constructed to organize different meteorological variables, including temperature, humidity, air pressure, wind speed, and precipitation, into time series data in chronological order. Different features are normalized to ensure that each feature of the model input is at the same scale. For data at different time scales, time series data is sliced ​​and suitable input sequences are constructed respectively.

3. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In step one, through the multi-scale channel module and multiple attention heads, the model learns dependencies of different time scales from different subspaces, so that the model can adaptively process information of different time granularities.

4. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In step 1, the dependency between any two time points in the time series is modeled through the Transformer module, while focusing on the information of the entire sequence to improve the ability to capture rainfall patterns.

5. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In step one, the prediction head module is responsible for providing time series prediction. Rainfall prediction is usually based on time series data, predicting the rainfall conditions at a future time point. The prediction head module is responsible for outputting the prediction results for continuous time. After the prediction head module outputs the original prediction value, it adjusts the output to an actually available value.

6. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In the second step, the lidar rainfall data is cleaned to remove missing values, duplicate data and noise. Missing data is filled using interpolation and adjacent data to ensure data integrity. The lidar data contains abnormal data caused by equipment errors, and statistical methods are used to eliminate data points that do not conform to the normal pattern.

7. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In step three, features of different time and spatial scales are captured through a multi-scale channel attention mechanism. By weighting channel features of different scales, the model can adaptively learn features of multiple scales for rainfall prediction. The multi-scale spatial and temporal features weighted by the channel attention mechanism are fused. The fusion of spatial and temporal information is more consistent in time and space, which helps to better understand the changing trends and patterns of rainfall.

8. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In step 4, historical rainfall data provided by lidar and weather station equipment is collected to ensure that the data covers different spatiotemporal scales, which facilitates the verification of the model under various environmental conditions. A time series cross-validation method is used to ensure that the model can cope with rainfall prediction tasks in different time periods and weather conditions, and to verify the performance of the multi-scale channel attention mechanism at different spatial and temporal scales.

9. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: In the data set of step 2, the real rainfall data obtained is divided into training set data and test set data. All the synthetic rainfall data is used as training set data. The model is trained using the training set data, and the model is verified using the test set data.

10. The rainfall prediction method based on transformer architecture according to claim 1, characterized in that: The trained rainfall prediction model is used to predict rainfall in the target area. The real rainfall image of the target area is collected and input into the trained rainfall prediction model to obtain the corresponding rainfall intensity value, thereby realizing rainfall prediction for the target area.

Images