High-resolution rainfall analysis generation method based on multi-source modal fusion deep learning

Abstract

The present invention discloses a method for generating high-resolution rainfall analysis based on multi-source modal fusion deep learning, the method comprising: obtaining a multi-physical quantity data set based on a multi-source data set related to rainfall conditions, and establishing a dimensionless multi-modal feature factor library; performing multi-dimensional non-equal weight processing on the loss function, and constructing a multi-scale spatial conversion model MF-ST-Unet; finally generating an hourly regional high-resolution rainfall analysis data set after model training; based on the trained MF-ST-Unet model, obtaining a high-resolution rainfall analysis data set applied to cover the entire area through transfer learning knowledge distillation. The present invention can quickly and effectively generate uniform grid rainfall analysis with high spatial resolution, and is more objective and accurate than traditional methods; fully considering the multi-scale characteristics of rainfall conditions and the differences in precipitation levels, achieving high-resolution objective rainfall data acquisition, improving rainfall detection capabilities and disaster prevention accuracy, and having extremely strong application value.

Description

Technical Field

[0001] The present invention belongs to the field of high-resolution rainfall situation generation, and in particular to a high-resolution rainfall situation analysis generation method based on multi-source modal fusion deep learning. Background Art

[0002] High-resolution rainfall observation can help improve the accuracy and timeliness of flood forecasting in small and medium-sized river basins. Through high-resolution observation of rainfall, the temporal and spatial distribution of precipitation can be more accurately grasped, the amount and intensity of precipitation can be monitored in real time, and the changes in the precipitation formation process can be tracked and predicted more accurately. It helps forecasters to more accurately estimate the possibility of floods and predict the flood response of the river basin. By comprehensively monitoring the temporal and spatial changes of rainfall, the parameters of the flood forecasting model can be adjusted in time, and the accuracy and reliability of the hydrological model can be improved, thereby improving the accuracy and timeliness of flood forecasting. In addition, high-resolution rainfall observation can also help identify and monitor local heavy rainfall and local extreme rainfall, issue early warning information in a timely manner, strengthen monitoring and early warning work in small and medium-sized river basins, and improve flood risk management and prevention capabilities. Therefore, high-resolution rainfall observation plays a very important role in flood forecasting in small and medium-sized river basins, and can effectively improve the accuracy and forecasting capabilities of flood forecasting.

[0003] The existing rainfall stations are unevenly distributed, and the input fields with existing resolution cannot meet the requirements of high-precision meteorological and hydrological models. In addition, the input fields of numerical forecasting models such as meteorology and hydrology are all grid data, that is, grid data is more suitable for the input of numerical forecasting models. Low-resolution and unevenly distributed station data cannot be directly input into numerical forecasting models. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning to improve rainfall monitoring capabilities and disaster prevention accuracy.

[0005] Technical solution: The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning of the present invention comprises the following steps:

[0006] Normalize the multi-source data sets related to rainfall conditions to obtain a multi-physical quantity data set; based on the multi-source data sets, perform spatial modal statistics on the data representing hourly precipitation of different magnitudes to obtain a multi-modal feature factor library, and perform adaptive coding to obtain the feature vector of the multi-modal feature factor library, and further establish a dimensionless multi-modal feature factor library;

[0007] For the data-intensive sub-regions in the multi-source dataset, based on the differences in precipitation levels and spatial distribution characteristics, the loss function is processed in multiple dimensions with non-equal weights, and a multi-scale spatial transformation model MF-ST-Unet is constructed and model training is carried out. The MF-ST-Unet model includes an input layer, an encoding module, a decoding module and a fusion module. The input layer combines the multi-physical quantity dataset, the dimensionless multi-modal feature factor library and the rainfall station observation data to obtain the model input features. In the encoding module, the convolutional neural network CNN is first used to extract and fuse the feature information of different data sources and spatial fields of different scales of the model input features to obtain global information. Then the global information is introduced into the multi-scale spatial transformation encoding module, and the spatial information at different scales is captured through multi-layer encoding to obtain the encoded feature field. In the decoding module, the spatial resolution is restored through the upsampling process, and the encoded feature field is refined and extracted using convolution to obtain the decoded feature field. In the fusion module, the global information and the decoded feature field are weighted with non-equal weights through the jump connection layer to obtain the fused feature output field. After model training, a high-resolution rainfall analysis dataset for each hour in the region is finally generated.

[0008] Based on the trained MF-ST-Unet model, a high-resolution rainfall analysis dataset covering the entire area is obtained through transfer learning knowledge distillation.

[0009] Furthermore, multi-source datasets related to rainfall conditions include regional hourly rainfall monitoring data, hourly precipitation grid datasets fused with precipitation products, higher-resolution radar data, multi-channel satellite data, radar and satellite warning frequency datasets, reanalysis meteorological element datasets, and geographic elevation data.

[0010] Furthermore, the multimodal characteristic factor library includes the historical average mode of hourly precipitation, the spatial distribution mode of precipitation and frequency of convective precipitation, the spatial distribution mode of precipitation and frequency of short-term heavy rainfall, the spatial distribution mode of precipitation and frequency of extreme precipitation, the frequency distribution mode of radar warning, the frequency distribution mode of satellite warning and the geographic elevation distribution mode.

[0011] Furthermore, the method for establishing a dimensionless multimodal feature factor library is:

[0012] (1) First, based on multi-source data sets, spatial modal statistics are performed on data representing hourly precipitation of different magnitudes. The spatial modal distribution of precipitation and frequency of convective precipitation, short-term heavy rainfall, and extreme precipitation, as well as the distribution modality of radar and satellite warning frequency are introduced. The geographic elevation distribution modality is considered, and through adaptive label coding, various spatial distribution characteristics of precipitation are learned. The feature learning process formula is as follows:

[0013] ,

[0014] in, is the encoding feature, , is the number of spatially distributed modal features in the multimodal feature factor library; is the input data, i.e., the spatial distribution modal features in the multimodal feature factor library; is the adaptive coding learning feature mapping function, Is with input Related parameters;

[0015] (2) After obtaining the encoded features of the multimodal feature factor library, integrate them into an overall feature vector:

[0016] ,

[0017] in, is the feature vector of the multimodal feature factor library;

[0018] (3) Perform dimensionless processing on the feature vectors of the multimodal feature factor library to form a dimensionless multimodal feature factor library .

[0019] Furthermore, the loss function is processed in multiple dimensions with unequal weights to obtain a unequal weight multidimensional loss function, which is expressed as:

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] in, is a non-equal weight multidimensional loss function; , , They are the weight coefficients of the root mean square error RMSE, spatial correlation coefficient PCC, and peak signal-to-noise ratio PSNR indicators; , is the number of rainfall stations involved in the RMSE calculation in the model training area; Output results for the model during training; It is the spatial average value of the model output results in the model training area during the training process; is the precipitation observation value at the rain gauge station; is the spatial average of precipitation observations at rain gauge stations within the model training area; , The number of grid points involved in PSNR calculation in the model training area; The pixel value corresponding to the model output result during the training process; is the pixel value corresponding to the precipitation observation value of the rain gauge station; The maximum pixel value in the model output image during training and the precipitation observation image of the rain gauge station.

[0025] Furthermore, the construction method of the multi-scale spatial transformation model MF-ST-Unet is as follows:

[0026] Input layer: The model receives multi-physics datasets , dimensionless multimodal feature factor library And rainfall station observation data , integrated at the input layer to form the model input features ;

[0027] Convolutional neural network encoding module: use CNN to input features to the model The feature information of spatial fields of different scales is extracted, the activation function is applied to introduce nonlinearity, and the spatial resolution is reduced through the pooling layer to extract global information; the convolution feature extraction is as follows:

[0028] ,

[0029] in, is the convolution feature field, i.e., global information, Encoding operations for convolutional neural networks;

[0030] Multi-scale spatial transformation encoding module: convolution feature field Introduce the multi-scale spatial conversion module:

[0031] ,

[0032] in, For the spatial transform coding operation, we first use a sliding window to transform After being divided into multiple sub-regions, these sub-regions are sorted into a sequence in order to form a new feature , and transform it into a basic unit sequence through linear transformation, the expression is as follows:

[0033] ,

[0034] in, is the basic unit sequence, is the weight matrix, is the bias term; the multi-head attention mechanism is used to calculate the correlation between basic units, and then the feedforward neural network is used to perform nonlinear transformation on each basic unit. The training process is stabilized by layer normalization, and finally the encoded feature field is generated. ;

[0035] Decoding module: restore spatial resolution through upsampling operation and extract features through convolution:

[0036] ,

[0037] in, is the decoded feature field, Perform network decoding operations for U rows;

[0038] Fusion module: introduces a skip connection layer to transform the convolutional feature field generated in the encoding module And the decoded feature field Perform weighted average;

[0039] ,

[0040] in, is the fused feature output field, It is the fusion module function.

[0041] Furthermore, after the model is trained using the training set, the model parameters are adjusted on the validation set based on the gradient descent method by adjusting the loss function in the opposite direction of the gradient, and the loss function is continuously iterated to minimize the loss function and verify the multi-source modal fusion effect. The specific parameter update rules are as follows:

[0042] ,

[0043] in, represents the updated model parameters; is a non-equal weight multidimensional loss function; Represents the current model parameters; is the learning rate; is the gradient of the non-equal-weight multidimensional loss function with respect to the current model parameters.

[0044] Furthermore, in knowledge distillation, the cross entropy is used to calculate the difference between the migrated MF-ST-Unet model and the trained MF-ST-Unet model. The loss function is obtained by weighted calculation of the original cross entropy loss and the soft label cross entropy loss between the migrated MF-ST-Unet model and the trained MF-ST-Unet model. The expression is as follows:

[0045] ,

[0046] in, is the new cross entropy loss, is the original cross entropy loss, is the soft label cross entropy loss, It is a hyperparameter that balances the original cross entropy loss and the soft label cross entropy loss.

[0047] The high-resolution rainfall analysis generation system based on multi-source modal fusion deep learning of the present invention comprises:

[0048] The data processing unit is used to normalize the multi-source data sets related to the rainfall conditions to obtain a multi-physical quantity data set; based on the multi-source data sets, spatial modal statistics are performed on the data representing hourly precipitation of different magnitudes to obtain a multi-modal feature factor library, and adaptive encoding is performed to obtain a feature vector of the multi-modal feature factor library, and further establish a dimensionless multi-modal feature factor library;

[0049] The model building and training unit is used to perform multi-dimensional non-equal weight processing on the loss function for the data-intensive sub-regions in the multi-source data set based on the differences in precipitation levels and spatial distribution characteristics, construct the multi-scale spatial transformation model MF-ST-Unet, and carry out model training; the MF-ST-Unet model includes an input layer, an encoding module, a decoding module, and a fusion module. The input layer combines the multi-physical quantity data set, the dimensionless multi-modal feature factor library, and the rain gauge observation data to obtain the model input features. In the encoding module, the convolutional neural network CNN is first used to extract and fuse the feature information of different data sources and different scales of the model input features to obtain global information, and then the global information is introduced into the multi-scale spatial transformation encoding module. The spatial information at different scales is captured through multi-layer encoding to obtain the encoded feature field; in the decoding module, the spatial resolution is restored through the upsampling process, and the encoded feature field is refined and extracted using convolution to obtain the decoded feature field; in the fusion module, the global information and the decoded feature field are weighted with non-equal weights through the jump connection layer to obtain the fused feature output field; after model training, a high-resolution rainfall analysis data set for each hour in the region is finally generated;

[0050] The analysis unit is used to obtain a high-resolution rainfall analysis dataset covering the entire area through transfer learning knowledge distillation based on the trained MF-ST-Unet model.

[0051] The electronic device of the present invention comprises:

[0052] A memory storing executable program code;

[0053] a processor coupled to the memory;

[0054] The processor calls the executable program code stored in the memory to execute the steps of the high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning.

[0055] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows: (1) Based on an advanced deep learning scheme, the present invention makes full use of multi-source observation data that is highly correlated with the rainfall field, fully considers the multi-scale characteristics of the rainfall field, establishes a multi-modal feature factor library for severe convective precipitation, and effectively extracts the spatially refined distribution structure of the rainfall field, thereby obtaining a refined rainfall analysis field with high spatial resolution. Compared with the traditional method of linear or nonlinear interpolation downscaling based on station data, the data coverage is more comprehensive, the data mining is more sufficient, and the high-resolution data obtained is more objective; (2) After extracting the main spatial modal features based on CNN, considering the discontinuity and suddenness of the rainfall field, from the perspective of the physical mechanism of precipitation formation, that is, it is the result of the interaction of multi-scale systems, feature extraction, processing and fusion are performed at different scales, and richer and more complex spatial features are extracted on the basis of improving the global perception range of the data, so as to fully mine the data and reduce the overfitting of the rainfall analysis caused by local heavy rainfall events. Risk, which helps to improve the performance and generalization ability of the model; (3) In the training process of the MF-ST-Unet model, a multi-head attention mechanism is introduced. According to the data features extracted by the convolution and pooling processes, the attention weights of the spatial positions of the rainfall feature fields are further adjusted through the attention mechanism. Then, through the feature weighting and feature fusion process, the representation features are further optimized, making the model more flexible and adaptable, fully mining the information of multi-source complex data, filtering out noise and anomalies in the data, enhancing the stability of the model, and improving the efficiency, accuracy and interpretability of the model when processing large-scale data; (4) In addition, a new loss function is introduced. Based on the local and jump characteristics of the rainfall field, comprehensive evaluation indicators such as root mean square error (RMSE), spatial correlation coefficient (PCC), and peak signal-to-noise ratio (PSNR) are used to comprehensively evaluate precipitation intensity and frequency to construct a non-equal weight multidimensional loss function, making the model more accurate, thereby improving the accuracy of the high-resolution rainfall analysis field. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the method of the present invention;

[0057] Figure 2 It is a schematic diagram of adaptive encoding and dimensionless transformation of multimodal feature factor library;

[0058] Figure 3 It is a schematic diagram of the MF-ST-Unet model structure;

[0059] Figure 4 Schematic diagram of optimizing the MF-ST-Unet model using a multi-dimensional unequal-weighted loss function. DETAILED DESCRIPTION

[0060] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0061] The spatial resolution of existing radar data sets and satellite data sets is much higher than that of rain gauge data. For example, the spatial resolution of ordinary weather radar is 1km or 0.25km, the spatial resolution of water conservancy rain radar is 0.03km, and the spatial resolution of satellite data sunflower 9 is 2km. By integrating multi-source data such as rain gauges, satellites, and radars, a high-quality rain analysis field (3km) that meets high-resolution water forecasting can be generated, thereby improving the accuracy of flood forecasting for small and medium-sized rivers. The present invention constructs a high-resolution rain analysis generation method based on multi-source modal fusion deep learning, and introduces non-uniformly distributed rain gauge monitoring data, radar data, and multi-channel satellite data into a multi-modal feature factor library such as spatial distribution modes such as historical precipitation of different magnitudes, heavy precipitation frequency from multiple data sources, and geographic elevation distribution modes, and adopts a new multi-dimensional non-equal weight loss function to train and learn the spatial distribution characteristics of rain conditions, so as to better capture the spatial distribution characteristics of rain conditions from different data sources, achieve high-resolution objective rain data acquisition, and improve rain monitoring capabilities and disaster prevention accuracy.

[0062] like Figure 1 As shown, the high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning of the present invention specifically includes the following steps:

[0063] S1. Obtain multi-source data sets related to rainfall conditions, including: regional hourly rainfall monitoring data, hourly precipitation grid data sets integrated with precipitation products, higher-resolution radar data, multi-channel satellite data, radar and satellite warning frequency data sets, reanalysis meteorological element data sets (total precipitation, 850hPa horizontal wind, 850hPa humidity, 850hPa temperature, 500hPa geopotential height, 500hPa temperature) and geographic elevation data. Clean the data based on historical extreme value detection and spatial consistency principles, identify and eliminate incomplete and inaccurate data, and normalize the data of different physical quantities in the multi-source data sets to ensure the stability and accuracy of the model operation after data input. The normalization formula is as follows:

[0064] ,

[0065] in, It is the original data, i.e., the data of different physical quantities in the multi-source data set; is the minimum value of the original data, is the maximum value of the original data, It is the normalized data. The normalized data of different physical quantities are combined into a multi-physical quantity data set , and divided into training set and validation set.

[0066] S2. Based on the multi-source data set in S1, perform spatial modal statistics on the data representing hourly precipitation of different magnitudes to form a multimodal feature factor library, and perform adaptive encoding to establish a dimensionless multimodal feature factor library. The library is consistent with the data set in S1 and is divided into a training set and a validation set. The multimodal feature factor library includes: historical average mode of hourly precipitation, spatial distribution mode of precipitation and frequency of convective precipitation (5mm / h), spatial distribution mode of precipitation and frequency of short-term heavy rainfall (20mm / h), spatial distribution mode of precipitation and frequency of extreme precipitation (threshold of 90% percentile), radar warning frequency distribution mode, satellite warning frequency distribution mode and geographic elevation distribution mode. Figure 2 As shown, specifically including:

[0067] S21. First, based on multi-source data sets, spatial modal statistics are performed on data showing different magnitudes of hourly precipitation. The locality, sporadicity and extremeness of convective precipitation are considered. The spatial modal distribution of precipitation and frequency of convective precipitation, short-term heavy rainfall and extreme precipitation, as well as the distribution modality of radar and satellite warning frequency are introduced. The geographic elevation distribution modality is considered. Through adaptive identification coding, various spatial distribution characteristics of precipitation are learned to maximize the capture of effective information and improve the model training efficiency and performance. The feature learning process formula is as follows:

[0068] ,

[0069] in, is the encoding feature, , is the number of spatially distributed modal features in the multimodal feature factor library; is the input data, i.e., the spatial distribution modal features in the multimodal feature factor library; is the adaptive coding learning feature mapping function, Is with input The goal of this function is to adjust the parameters by learning , so that the encoded features More adaptable to model data distribution.

[0070] S22. After obtaining the encoding features of the multimodal feature factor library, integrate them into an overall feature vector:

[0071] ,

[0072] in, is the feature vector of the multimodal feature factor library.

[0073] S23, performing dimensionless processing on the feature vector of the multimodal feature factor library to form a dimensionless multimodal feature factor library , to ensure the stability in model construction, the dimensionless processing method is the same as the normalization algorithm in step S1.

[0074] S24, using the same data set partitioning scheme as in step S1, the dimensionless multimodal feature factor library is divided into a training set and a validation set.

[0075] S3. Considering the uneven characteristics of regional observation density, for specific sub-regions with dense data in multi-source data sets, based on the differences in precipitation levels and spatial distribution characteristics, the loss function is processed in multiple dimensions with non-equal weights, and a multi-scale spatial transformation U-net model (i.e., MF-ST-Unet model) is constructed that integrates regional hourly rainfall monitoring data, radar satellite remote sensing data (including higher-resolution radar data, multi-channel satellite data, radar and satellite warning frequency data sets), multimodal feature factor library, geographic elevation data, and reanalysis meteorological element data sets, and model training is carried out. The MF-ST-Unet model mainly includes an input layer, an encoding module, a decoding module, and a fusion module. The input layer data includes the multi-physical quantity data set in step S1 , the dimensionless multimodal feature factor library in step S2 and rainfall station observation data In the encoding module, a convolutional neural network (CNN) is first used to train the multi-physics dataset. , dimensionless multimodal feature factor library and rainfall station observation data The feature information is extracted and fused, and the spatial resolution is reduced through the pooling layer to extract the global information; the global information is introduced into the multi-scale spatial conversion coding module, and the spatial information at different scales is captured through multi-layer coding. In the decoding module, the spatial resolution is restored through the upsampling process, and the spatial feature field of the decoding module part is refined and extracted using convolution. In the fusion module, the global information extracted in the encoding process and the convolution feature field extracted in the decoding process are weighted with non-equal weights through the jump connection layer, and the high-precision spatial information is supplemented and fused to improve the accuracy of spatial distribution feature extraction, and the fused feature output field is obtained; after model training, a complete and accurate output field is finally generated, that is, a high-resolution rainfall analysis dataset for each hour in the region. Specifically including:

[0076] S31. According to the particularity of precipitation data, taking into full account the local characteristics of short-term heavy precipitation, an innovative loss function is introduced, namely the non-equal weight multidimensional loss function. This loss function integrates multiple evaluation indicators such as root mean square error (RMSE), spatial correlation coefficient (PCC), peak signal-to-noise ratio (PSNR), etc., comprehensively considers precipitation intensity and frequency, and more comprehensively measures the performance of the model. The non-equal weight multidimensional loss function is constructed as follows:

[0077] ,

[0078] ,

[0079] ,

[0080] ,

[0081] in, is a non-equal weight multidimensional loss function; , , They are the weight coefficients of the root mean square error (RMSE), spatial correlation coefficient (PCC), and peak signal-to-noise ratio (PSNR) indicators; , is the number of rainfall stations involved in the RMSE calculation in the model training area; Output results for the model during training; It is the spatial average value of the model output results in the model training area during the training process; is the precipitation observation value at the rain gauge station; is the spatial average of precipitation observations at rain gauge stations within the model training area; , The number of grid points involved in PSNR calculation in the model training area; The pixel value corresponding to the model output result during the training process; is the pixel value corresponding to the precipitation observation value of the rain gauge station; The maximum pixel value in the model output image during training and the precipitation observation image of the rain gauge station.

[0082] By adjusting the weight coefficient, the influence of various indicators in the training process can be balanced, so that the model can better adapt to the characteristics of the precipitation field and obtain a more accurate high-resolution rainfall distribution. In addition, in order to avoid the homogenization of the rainfall analysis field caused by conventional interpolation, the present invention only considers the local rainfall characteristics (the range radius is limited to 3km) in the calculation process of the evaluation index, and does not perform interpolation. The inverse distance weight method is used to calculate the difference between the high-resolution grid data and the rainfall station within 3km. If there is no observation data within 3km, this point will not be evaluated and calculated, and will not participate in the calculation of the loss function.

[0083] S32, using the non-equal weighted multidimensional loss function obtained in step S31, constructing a MF-ST-Unet model that integrates multi-source data sets and multi-modal feature factor libraries, such as Figure 3 As shown in the figure, the model mainly includes input layer, encoding module, decoding module and fusion module. The specific steps are as follows:

[0084] Input layer: The model receives the multi-physics data set obtained in step S1 , the dimensionless multimodal feature factor library obtained in step S2 And rainfall station observation data , integrated at the input layer to form the model input features .

[0085] The encoding module includes: convolutional neural network encoding module (CNN) and multi-scale spatial transformation encoding module (ST).

[0086] Convolutional Neural Network Encoding Module (CNN): Use CNN to input features to the model The feature information of different data sources and different scales of spatial fields is extracted, the activation function is applied to introduce nonlinearity, and the spatial resolution is reduced through the pooling layer to extract global information. Convolution feature extraction is as follows:

[0087] ,

[0088] in, is the convolution feature field, i.e., global information, Encoding operations for convolutional neural networks;

[0089] Multi-scale spatial transformation coding module (ST): Convolution feature field Introducing a multi-scale spatial transformation coding module;

[0090] ,

[0091] in, For the spatial transform coding operation, we first use a sliding window to transform After being divided into multiple sub-regions, these sub-regions are sorted into a sequence in order to form a new feature , and transform it into a basic unit sequence through linear transformation, the expression is as follows:

[0092] ,

[0093] in, is the basic unit sequence, is the weight matrix, is the bias term. The multi-head attention mechanism is used to calculate the correlation between basic units, and then the feedforward neural network is used to perform nonlinear transformation on each basic unit. The training process is stabilized by layer normalization, and finally the encoded feature field is generated. ;

[0094] The multi-scale spatial transform coding module (ST) process is repeated multiple times to form a multi-layer multi-scale spatial transform coding module.

[0095] Decoding module: restores spatial resolution through upsampling operations and extracts features using convolution;

[0096] ,

[0097] in, is the decoded feature field, Decoding operation for U-type network.

[0098] Fusion module: introduces a skip connection layer to transform the convolutional feature field generated in the encoding module And the decoded feature field Perform weighted average;

[0099] ,

[0100] in, is the fused feature output field, It is the fusion module function;

[0101] It contains information extracted from both the encoding module and the decoding module, and fully integrates spatial feature information at different scales.

[0102] S33, training the model of step S32, introducing a regularization process to avoid overfitting of the model, thereby improving the generalization performance and robustness of the model, and generating a more complete, accurate, and higher-resolution rainfall analysis field, i.e., an hourly regional high-resolution rainfall analysis data set, such as Figure 4After the model is trained with the training set, based on the gradient descent method, the model parameters are adjusted in the opposite direction of the gradient of the non-equal weight multidimensional loss function on the validation set, and the non-equal weight multidimensional loss function is minimized through continuous iteration to verify the multi-source modal fusion effect, making the model stable, efficient, and optimal. The specific parameter update rules are as follows:

[0103] ,

[0104] in, is a non-equal weight multidimensional loss function; Represents the current model parameters; is the learning rate; is the gradient of the non-equal-weighted multidimensional loss function to the current model parameters, which represents the changing trend of the non-equal-weighted multidimensional loss function relative to the current model parameters, that is, the impact of the current parameters on the model performance in the non-equal-weighted multidimensional loss function; Represents the updated model parameters.

[0105] S4. Based on the MF-ST-Unet model trained in step S3, the migrated MF-ST-Unet model is obtained through transfer learning knowledge distillation. The real-time high-resolution (3km*3km) spatial rainfall analysis field is generated through the migrated MF-ST-Unet model, that is, a high-resolution rainfall analysis data set applied to the overall hourly range is obtained. In knowledge distillation, the cross entropy is used to calculate the difference between the migrated MF-ST-Unet model and the MF-ST-Unet model trained in S3. The new cross entropy loss function is obtained by weighted calculation of the original cross entropy loss and the cross entropy loss (i.e., soft label cross entropy loss) between the migrated MF-ST-Unet model and the MF-ST-Unet model trained in S3. Its expression is as follows:

[0106] ,

[0107] in, is the new cross entropy loss, is the original cross entropy loss, is the soft label cross entropy loss, It is a hyperparameter that balances the original cross entropy loss and the soft label cross entropy loss.

[0108] The deep learning high-resolution rainfall analysis generation method improved based on multi-source modal fusion described in the present invention can quickly and effectively generate uniform grid rainfall analysis with high spatial resolution, and is more objective and accurate than traditional methods; by combining multi-source data and a multi-modal feature factor library, by improving the loss function, the deep learning neural network model is optimized in a targeted manner, the multi-scale characteristics of rainfall and the differences in precipitation levels are fully considered, high-resolution objective rainfall data acquisition is achieved, the rainfall detection capability and disaster prevention accuracy are improved, and the method has extremely strong application value.

[0109] The system corresponding to the method of the present invention includes:

[0110] The data processing unit is used to normalize the multi-source data sets related to the rainfall conditions to obtain a multi-physical quantity data set; based on the multi-source data sets, spatial modal statistics are performed on the data representing hourly precipitation of different magnitudes to obtain a multi-modal feature factor library, and adaptive encoding is performed to obtain a feature vector of the multi-modal feature factor library, and further establish a dimensionless multi-modal feature factor library;

[0111] The model building and training unit is used to perform multi-dimensional non-equal weight processing on the loss function for the data-intensive sub-regions in the multi-source data set based on the differences in precipitation levels and spatial distribution characteristics, construct the multi-scale spatial transformation model MF-ST-Unet, and carry out model training; the MF-ST-Unet model includes an input layer, an encoding module, a decoding module, and a fusion module. The input layer combines the multi-physical quantity data set, the dimensionless multi-modal feature factor library, and the rain gauge observation data to obtain the model input features. In the encoding module, the convolutional neural network CNN is first used to extract and fuse the feature information of different data sources and different scales of the model input features to obtain global information, and then the global information is introduced into the multi-scale spatial transformation encoding module. The spatial information at different scales is captured through multi-layer encoding to obtain the encoded feature field; in the decoding module, the spatial resolution is restored through the upsampling process, and the encoded feature field is refined and extracted using convolution to obtain the decoded feature field; in the fusion module, the global information and the decoded feature field are weighted with non-equal weights through the jump connection layer to obtain the fused feature output field; after model training, a high-resolution rainfall analysis data set for each hour in the region is finally generated;

[0112] The analysis unit is used to obtain a high-resolution rainfall analysis dataset covering the entire area through transfer learning knowledge distillation based on the trained MF-ST-Unet model.

[0113] The electronic device of the present invention comprises:

[0114] A memory storing executable program code;

[0115] a processor coupled to the memory;

[0116] The processor calls the executable program code stored in the memory to execute the steps of the high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning.

Claims

1. A high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning is characterized by: The following steps are involved: Normalize the multi-source data sets related to rainfall conditions to obtain a multi-physical quantity data set; based on the multi-source data sets, perform spatial modal statistics on the data representing hourly precipitation of different magnitudes to obtain a multi-modal feature factor library, and perform adaptive coding to obtain the feature vector of the multi-modal feature factor library, and further establish a dimensionless multi-modal feature factor library; For the data-intensive sub-regions in the multi-source dataset, based on the differences in precipitation levels and spatial distribution characteristics, the loss function is processed in multiple dimensions with non-equal weights, and a multi-scale spatial transformation model MF-ST-Unet is constructed and model training is carried out. The MF-ST-Unet model includes an input layer, an encoding module, a decoding module and a fusion module. The input layer combines the multi-physical quantity dataset, the dimensionless multi-modal feature factor library and the rainfall station observation data to obtain the model input features. In the encoding module, the convolutional neural network CNN is first used to extract and fuse the feature information of different data sources and spatial fields of different scales of the model input features to obtain global information. Then the global information is introduced into the multi-scale spatial transformation encoding module, and the spatial information at different scales is captured through multi-layer encoding to obtain the encoded feature field. In the decoding module, the spatial resolution is restored through the upsampling process, and the encoded feature field is refined and extracted using convolution to obtain the decoded feature field. In the fusion module, the global information and the decoded feature field are weighted with non-equal weights through the jump connection layer to obtain the fused feature output field. After model training, a high-resolution rainfall analysis dataset for each hour in the region is finally generated. Based on the trained MF-ST-Unet model, a high-resolution rainfall analysis dataset covering the entire area is obtained through transfer learning knowledge distillation.

2. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: Multi-source datasets related to rainfall conditions include regional hourly rainfall monitoring data, hourly precipitation grid datasets fused with precipitation products, higher-resolution radar data, multi-channel satellite data, radar and satellite warning frequency datasets, reanalysis meteorological element datasets, and geographic elevation data.

3. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: The multimodal characteristic factor library includes the historical average mode of hourly precipitation, the spatial distribution mode of precipitation and frequency of convective precipitation, the spatial distribution mode of precipitation and frequency of short-term heavy rainfall, the spatial distribution mode of precipitation and frequency of extreme precipitation, the frequency distribution mode of radar warning, the frequency distribution mode of satellite warning and the geographic elevation distribution mode.

4. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: The method to establish a dimensionless multimodal feature factor library is: (1) First, based on multi-source data sets, spatial modal statistics are performed on data representing hourly precipitation of different magnitudes. The spatial modal distribution of precipitation and frequency of convective precipitation, short-term heavy rainfall, and extreme precipitation, as well as the distribution modality of radar and satellite warning frequency are introduced. The geographic elevation distribution modality is considered, and through adaptive label coding, various spatial distribution characteristics of precipitation are learned. The feature learning process formula is as follows: ， in, is the encoding feature, , is the number of spatially distributed modal features in the multimodal feature factor library; is the input data, i.e., the spatial distribution modal features in the multimodal feature factor library; is the adaptive coding learning feature mapping function, Is with input Related parameters; (2) After obtaining the encoded features of the multimodal feature factor library, integrate them into an overall feature vector: ， in, is the feature vector of the multimodal feature factor library; (3) Perform dimensionless processing on the feature vectors of the multimodal feature factor library to form a dimensionless multimodal feature factor library .

5. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: The loss function is processed in multiple dimensions with unequal weights to obtain a unequal weight multidimensional loss function, which is expressed as: ， ， ， ， in, is a non-equal weight multidimensional loss function; , , They are the weight coefficients of the root mean square error RMSE, spatial correlation coefficient PCC, and peak signal-to-noise ratio PSNR indicators; , is the number of rainfall stations involved in the RMSE calculation in the model training area; Output results for the model during training; It is the spatial average value of the model output results in the model training area during the training process; is the precipitation observation value at the rain gauge station; is the spatial average of precipitation observations at rain gauge stations within the model training area; , The number of grid points involved in PSNR calculation in the model training area; The pixel value corresponding to the model output result during the training process; is the pixel value corresponding to the precipitation observation value of the rain gauge station; The maximum pixel value in the model output image during training and the precipitation observation image of the rain gauge station.

6. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: The construction method of the multi-scale spatial transformation model MF-ST-Unet is: Input layer: The model receives multi-physics datasets , dimensionless multimodal feature factor library And rainfall station observation data , integrated at the input layer to form the model input features ; Convolutional neural network encoding module: use CNN to input features to the model The feature information of spatial fields of different scales is extracted, the activation function is applied to introduce nonlinearity, and the spatial resolution is reduced through the pooling layer to extract global information; the convolution feature extraction is as follows: ， in, is the convolution feature field, i.e., global information, Encoding operations for convolutional neural networks; Multi-scale spatial transformation encoding module: convolution feature field Introduce the multi-scale spatial conversion module: ， in, For the spatial transform coding operation, we first use a sliding window to transform After being divided into multiple sub-regions, these sub-regions are sorted into a sequence in order to form a new feature , and transform it into a basic unit sequence through linear transformation, the expression is as follows: ， in, is the basic unit sequence, is the weight matrix, is the bias term; the multi-head attention mechanism is used to calculate the correlation between basic units, and then the feedforward neural network is used to perform nonlinear transformation on each basic unit. The training process is stabilized by layer normalization, and finally the encoded feature field is generated. ; Decoding module: restore spatial resolution through upsampling operation and extract features through convolution: ， in, is the decoded feature field, Perform network decoding operations for U rows; Fusion module: introduces a skip connection layer to transform the convolutional feature field generated in the encoding module And the decoded feature field Perform weighted average; ， in, is the fused feature output field, It is the fusion module function.

7. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: After training the model with the training set, the model parameters are adjusted on the validation set based on the gradient descent method by adjusting the loss function in the opposite direction of the gradient, and the loss function is minimized through continuous iteration to verify the multi-source modal fusion effect. The specific parameter update rules are as follows: ， in, represents the updated model parameters; is a non-equal weight multidimensional loss function; Represents the current model parameters; is the learning rate; is the gradient of the non-equal-weight multidimensional loss function with respect to the current model parameters.

8. The high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning according to claim 1 is characterized in that: In knowledge distillation, the cross entropy is used to calculate the difference between the migrated MF-ST-Unet model and the trained MF-ST-Unet model. The loss function is obtained by weighted calculation of the original cross entropy loss and the soft label cross entropy loss between the migrated MF-ST-Unet model and the trained MF-ST-Unet model. The expression is as follows: ， in, is the new cross entropy loss, is the original cross entropy loss, is the soft label cross entropy loss, It is a hyperparameter that balances the original cross entropy loss and the soft label cross entropy loss.

9. A high-resolution rainfall analysis and generation system based on multi-source modal fusion deep learning is characterized by: include: A data processing unit, used for normalizing multi-source data sets related to rainfall conditions to obtain multi-physical quantity data sets; Based on multi-source data sets, spatial modal statistics are performed on data representing hourly precipitation of different magnitudes to obtain a multimodal feature factor library, which is then adaptively encoded to obtain feature vectors of the multimodal feature factor library, and a dimensionless multimodal feature factor library is further established. The model building and training unit is used to perform multi-dimensional non-equal weight processing on the loss function for the data-intensive sub-regions in the multi-source data set based on the differences in precipitation levels and spatial distribution characteristics, construct the multi-scale spatial transformation model MF-ST-Unet, and carry out model training; the MF-ST-Unet model includes an input layer, an encoding module, a decoding module, and a fusion module. The input layer combines the multi-physical quantity data set, the dimensionless multi-modal feature factor library, and the rain gauge observation data to obtain the model input features. In the encoding module, the convolutional neural network CNN is first used to extract and fuse the feature information of different data sources and different scales of the model input features to obtain global information, and then the global information is introduced into the multi-scale spatial transformation encoding module. The spatial information at different scales is captured through multi-layer encoding to obtain the encoded feature field; in the decoding module, the spatial resolution is restored through the upsampling process, and the encoded feature field is refined and extracted using convolution to obtain the decoded feature field; in the fusion module, the global information and the decoded feature field are weighted with non-equal weights through the jump connection layer to obtain the fused feature output field; after model training, a high-resolution rainfall analysis data set for each hour in the region is finally generated; The analysis unit is used to obtain a high-resolution rainfall analysis dataset covering the entire area through transfer learning knowledge distillation based on the trained MF-ST-Unet model.

10. An electronic device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the steps of the high-resolution rainfall analysis generation method based on multi-source modal fusion deep learning as described in any one of claims 1 to 8.

Images