Satellite remote sensing rainfall data fusion method and system based on deep learning
Through the satellite remote sensing precipitation data fusion method based on deep learning, using a combination of multiple precipitation methods, the problem of insufficient precision and coverage of precipitation data in the prior art is solved, and the fusion of precipitation data with high precision and high spatial and temporal resolution is achieved.
Patent Information
- Application Number
- CN202411725822.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-04-15
AI Technical Summary
There is a lack of estimation methods that can be combined with multiple precipitation methods in the prior art, resulting in insufficient accuracy, coverage and spatial and temporal resolution of precipitation data.
Using a satellite remote sensing precipitation data fusion method based on deep learning, the data set of training deep learning models, including rainmeters, satellite data, etc., is used to fusion of multi-source satellite remote sensing precipitation data.
It improves the accuracy and spatiotemporal resolution of precipitation data, enhances the coverage of data, and can efficiently and accurately output satellite remote sensing precipitation data products.
Smart Images
Figure CN119830199A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a satellite remote sensing precipitation data fusion method and system based on deep learning, belonging to the technical field of meteorological satellite remote sensing. Background Art
[0002] Precipitation is an important part of the global water cycle and heat cycle, and dominates the earth's weather, climate, hydrology, ecology and other aspects. The global energy and water cycle exchange drives most hydrological models and has a direct impact on the atmospheric system cycle. Therefore, accurately obtaining high-precision precipitation data is an important task in the field of meteorology. Currently, the commonly used methods of rainfall include: rain gauges, ground-based rainfall radars, numerical weather forecasts and satellite remote sensing, but these methods have their own advantages and disadvantages.
[0003] Rain gauges are usually built based on ground observation stations, and the precipitation data obtained is very close to the actual precipitation situation. However, the establishment of ground stations is usually very sparse, especially in some remote mountainous areas, where the distribution of rain gauges is even more sparse, and the spatial representativeness of the obtained data is poor. Ground-based precipitation radars can usually obtain precipitation data with high temporal and spatial resolution of hundreds of kilometers around the station, but radar data are easily affected by clutter and system noise, and the precipitation measurement accuracy is usually lower than that of ground rain gauges. Under certain initial values and boundary conditions, numerical weather forecasts use large computers to simulate weather processes in combination with the actual atmospheric conditions to achieve forecasts of atmospheric motion and weather phenomena for a period of time in the future. However, since the current numerical forecast system cannot fully describe the actual state of the atmosphere, the precipitation data predicted by the model has a certain deviation from the actual situation. Obtaining precipitation data based on satellite remote sensing has played an important role in the global precipitation distribution, and the obtained precipitation data has a high temporal and spatial resolution. However, due to the limitations of the current satellite precipitation inversion algorithm, the accuracy of precipitation acquisition still needs to be improved. Although a single precipitation acquisition method can also be used to estimate precipitation, research has found that precipitation data obtained based on a combination of multiple precipitation methods has more advantages in accuracy, coverage, and spatiotemporal resolution. Summary of the invention
[0004] The technical problem solved by the present invention is: in view of the problem that the current prior art lacks an estimation method that can utilize a combination of multiple precipitation modes, a satellite remote sensing precipitation data fusion method and system based on deep learning are proposed.
[0005] The present invention solves the above technical problems by the following technical solutions:
[0006] A satellite remote sensing precipitation data fusion method based on deep learning, comprising:
[0007] Build a dataset for training deep learning models;
[0008] Build a deep learning model including input layer, hidden layer, and fully connected layer;
[0009] Randomly divide the training deep learning model data set to obtain two parts of randomly divided data sets, perform data set conversion on one part of the randomly divided data sets to obtain a converted training set with the same time and longitude and latitude;
[0010] The converted training set is used as the input data of the deep learning model, and the output data of the deep learning model is compared with the randomly divided data set;
[0011] When the comparison results meet the validation set error range, the training is stopped and the precipitation data fusion model is generated.
[0012] The training deep learning model data set includes rain gauge precipitation data, satellite data, geosynchronous orbit satellite passive infrared precipitation data, and earth low-Earth orbit passive microwave precipitation data; the rain gauge precipitation data is collected according to the statistics of the target meteorological station, and the geosynchronous orbit satellite passive infrared precipitation data and earth low-Earth orbit passive microwave precipitation data are collected according to meteorological satellites;
[0013] Satellite data include GPM satellite DPR data, Sentinel-1 satellite GRDH data, and CloudSat satellite CPR data, which are collected based on the satellite data transmission network.
[0014] The method for dividing the training deep learning model data set is:
[0015] Randomly divide the training deep learning model data set into data set X and data set Y;
[0016] Transform the dataset X through the time-space matching algorithm to obtain the dataset X′ with the same time, longitude and latitude;
[0017] The dataset X′ and dataset Y are randomly divided into training set and validation set according to the corresponding relationship Y.
[0018] The training set and validation set account for 75% and 25% of the dataset for training the deep learning model, respectively.
[0019] The deep learning model is obtained by coupling the full connection layer according to the spatiotemporal matching model, using the training set as the input data of the full connection layer, the verification set Y as the ideal value of the full connection layer output, and the full connection layer setting as the optimization target. The full connection layer setting is iteratively adjusted by minimizing the loss function micro-optimization.
[0020] The minimization loss function is:
[0021]
[0022] Where f(x) is the output of the fully connected layer, x is the xth neuron in the fully connected layer, and e is the eigenvalue of the fully connected layer.
[0023] The method of training set X′ through the space-time matching algorithm is:
[0024] X′=s(w×X+b), where w is the time weight, b is the bias, and s is the weight function related to the distance.
[0025] The deep learning model includes an input layer, a hidden layer, and a fully connected layer, wherein:
[0026] The training set X′ obtained by the spatiotemporal matching algorithm is used as the input data of the fully connected layer. According to the input data, neurons related to the number of collected precipitation data types and a bias are set in the input layer of the deep learning model. Different numbers of hidden layers are set, and the number of neurons in each hidden layer is set respectively, and a bias is set for each layer to ensure that the neurons in the input layer, each hidden layer, and the fully connected layer are not connected to each other, and the connection between neurons in each layer is achieved by setting weight values.
[0027] Taking the fully connected layer setting as the optimization target, the method of iteratively adjusting the fully connected layer setting by minimizing the loss function micro-optimization is:
[0028] Collect input data and output data of each layer in the deep learning model, transfer data according to the activation function, compare the difference between the expected input data and the output data, generate an error signal based on the difference, and transfer it back layer by layer to correct the weights between neurons in each layer until the difference between the expected input data and the output data meets the error range.
[0029] The active and passive satellite remote sensing precipitation data fusion system includes a data set acquisition module, a learning model building module, a data set conversion module, and a model training module, among which:
[0030] The data set collection module collects various rainfall data and satellite data to generate data sets for training deep learning models;
[0031] The learning model construction module constructs a deep learning model including an input layer, a hidden layer, and a fully connected layer as the initial learning model;
[0032] The data set conversion module randomly divides the data set for training the deep learning model, obtains two parts of the randomly divided data set, performs data set conversion on one part of the randomly divided data set, obtains the converted training set of the same time and longitude and latitude, and sends the converted training set and the other part of the randomly divided data set to the model training module;
[0033] The model training module uses the converted training set as the input data of the initial learning model, compares the output data of the deep learning model with the randomly divided data set, and obtains the comparison result. When the comparison result meets the error range of the validation set, the training is stopped and the precipitation data fusion model is generated.
[0034] The advantages of the present invention compared with the prior art are:
[0035] The present invention provides a method for fusion of satellite remote sensing precipitation data based on deep learning. Based on deep learning model training and optimization, it fully integrates multiple active and passive satellite remote sensing precipitation data. According to this feature, a data set for model training is constructed, and a spatiotemporal matching system is used to perform spatiotemporal matching on multi-source satellite remote sensing precipitation data. Secondly, a fully connected layer is connected after the spatiotemporal matching system, and the fully connected layer is trained with a large amount of data by minimizing the loss function and the Levenberg-Marquardt minimization algorithm. Finally, the deep learning model can solve the problem of the difficulty of fusion of active and passive satellite remote sensing precipitation data, and can output satellite remote sensing precipitation data products efficiently and with high precision. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A structural diagram of the deep learning model provided by the present invention;
[0037] Figure 2 This is a graph of the test results of the model output data provided by the present invention. DETAILED DESCRIPTION
[0038] A satellite remote sensing precipitation data fusion method based on deep learning integrates the advantages of multi-source satellite precipitation detection, constructs a data set for model training, and uses a space-time matching system to perform space-time matching on multi-source satellite remote sensing precipitation data. The powerful nonlinear fitting of deep learning is used to solve the problem of difficulty in fusing active and passive satellite remote sensing precipitation data, and can output satellite remote sensing precipitation fusion data products efficiently and with high precision.
[0039] The satellite remote sensing precipitation data fusion method based on deep learning has the following specific steps:
[0040] Build a dataset for training deep learning models;
[0041] Build deep learning models;
[0042] Divide the training set and validation set according to the training deep learning model data set;
[0043] The divided training set is used as the input data of the deep learning model, and the output data of the deep learning model is compared with the validation set;
[0044] When the comparison results meet the validation set error range, the training is stopped and the precipitation data fusion model is generated.
[0045] The datasets for training deep learning models include rain gauge precipitation data, satellite data, geosynchronous orbit satellite passive infrared precipitation data, and low-Earth orbit passive microwave precipitation data; rain gauge precipitation data is collected based on statistics from the target meteorological station, and geosynchronous orbit satellite passive infrared precipitation data and low-Earth orbit passive microwave precipitation data are collected based on meteorological satellites;
[0046] Satellite data include GPM satellite DPR data, Sentinel-1 satellite GRDH data, and CloudSat satellite CPR data, which are collected based on the satellite data transmission network.
[0047] The method for dividing the data set for training deep learning models is:
[0048] Randomly divide the training deep learning model data set into data set X and data set Y;
[0049] Transform the dataset X through the time-space matching algorithm to obtain the dataset X′ with the same time, longitude and latitude;
[0050] The dataset X′ and dataset Y are randomly divided into training set and validation set according to the corresponding relationship Y.
[0051] The training set and validation set account for 75% and 25% of the dataset for training the deep learning model, respectively.
[0052] The deep learning model is obtained by coupling the fully connected layer according to the spatiotemporal matching model. The training set is used as the input data of the fully connected layer, the validation set Y is used as the ideal value of the fully connected layer output, the fully connected layer setting is used as the optimization target, and the fully connected layer setting is iteratively adjusted by minimizing the loss function micro-optimization.
[0053] The deep learning model is obtained by coupling the fully connected layer according to the spatiotemporal matching model. The training set is used as the input data of the fully connected layer, the validation set Y is used as the ideal value of the fully connected layer output, the fully connected layer setting is used as the optimization target, and the fully connected layer setting is iteratively adjusted by minimizing the loss function micro-optimization.
[0054] The method of training set X′ through the space-time matching algorithm is:
[0055] X′=s(w×X+b)
[0056] Where w is the time weight, b is the bias, and s is the weight function related to the distance.
[0057] The deep learning model includes input layer, hidden layer, and fully connected layer, where:
[0058] The training set X′ obtained by the spatiotemporal matching algorithm is used as the input data of the fully connected layer. According to the input data, neurons related to the number of collected precipitation data types and a bias are set in the input layer of the deep learning model. Different numbers of hidden layers are set, and the number of neurons in each hidden layer is set respectively, and a bias is set for each layer to ensure that the neurons in the input layer, each hidden layer, and the fully connected layer are not connected to each other, and the connection between neurons in each layer is achieved by setting weight values.
[0059] Taking the fully connected layer setting as the optimization target, the method of iteratively adjusting the fully connected layer setting by minimizing the loss function micro-optimization is:
[0060] Collect input data and output data of each layer in the deep learning model, transfer data according to the activation function, compare the difference between the expected input data and the output data, generate an error signal based on the difference, and transfer it back layer by layer to correct the weights between neurons in each layer until the difference between the expected input data and the output data meets the error range.
[0061] The active and passive satellite remote sensing precipitation data fusion system includes a data set acquisition module, a learning model building module, a data set conversion module, and a model training module, among which:
[0062] The data set collection module collects various rainfall data and satellite data to generate data sets for training deep learning models;
[0063] The learning model construction module constructs a deep learning model including an input layer, a hidden layer, and a fully connected layer as the initial learning model;
[0064] The data set conversion module randomly divides the data set for training the deep learning model, obtains two parts of the randomly divided data set, performs data set conversion on one part of the randomly divided data set, obtains the converted training set of the same time and longitude and latitude, and sends the converted training set and the other part of the randomly divided data set to the model training module;
[0065] The model training module uses the converted training set as the input data of the initial learning model, compares the output data of the deep learning model with the randomly divided data set, and obtains the comparison result. When the comparison result meets the error range of the validation set, the training is stopped and the precipitation data fusion model is generated.
[0066] The following is further described in conjunction with the accompanying drawings and preferred embodiments of the specification:
[0067] In the current embodiment, if Figure 1 , Figure 2 As shown in FIG. 1 , the satellite remote sensing precipitation data fusion method based on deep learning includes the following steps:
[0068] Step 1: In order to realize the fusion of active and passive satellite remote sensing precipitation data based on deep learning, the rain gauge precipitation data of Hunan Meteorological Station in 2020, and the GPM satellite DPR data, Sentinel-1 satellite GRDH data, CloudSat satellite CPR data, Sunflower 8 geostationary satellite passive infrared precipitation data, and microwave imager GMI passive microwave precipitation data that passed through Hunan Province in 2020 were obtained. Then these data are used to construct a dataset A for training the deep learning model. A includes the rain gauge precipitation dataset Y used as the model output, and the GPM satellite DPR data, Sentinel-1 satellite GRDH data, CloudSat satellite CPR data, Sunflower 8 geostationary satellite passive infrared precipitation data, and microwave imager GMI passive microwave precipitation data used as the model input;
[0069] Step 2: Build a deep learning model coupled by the spatiotemporal matching system and the fully connected layer, input X in the training set into the spatiotemporal matching system, and convert X into a data set X′ of the same time and longitude and latitude through the spatiotemporal matching algorithm. X′ and Y are in a corresponding relationship. X′ and Y are randomly divided into a training set and a validation set, which account for 75% and 25% of the total data respectively;
[0070] Step 3: Connect the fully connected layer after the spatiotemporal matching system to build a complete deep learning model. Use the output of the spatiotemporal matching system in step 2 as the input of the fully connected layer, and use Y in the training set as the ideal value of the output of the fully connected layer. Fine-tune the entire fully connected layer by minimizing the loss function, thereby realizing the fusion of active and passive satellite remote sensing precipitation data.
[0071] Step 4: Use X′ in the validation set as the input of the deep learning model, calculate the corresponding output, and compare the error with Y in the validation set to test the accuracy of the method.
[0072] It is worth noting that in step 2, the training set X′ is input into the spatiotemporal matching system, and the following algorithm is used:
[0073] X′=s(w×X+b), where X′ is the result of spatiotemporal matching of X, w represents the time weight, b is the bias, and s is the weight function related to the distance.
[0074] In this embodiment, in step 3, the output of the spatiotemporal matching system in step 2 is used as the input of the fully connected layer, and Y in the training set is used as the ideal value of the output of the fully connected layer.
[0075] In this embodiment, the active and passive satellite remote sensing precipitation data fusion mainly includes: after inputting the multi-source satellite data into the time-space matching system, the output data is input into the fully connected layer. Since the data used to output the final active and passive satellite remote sensing precipitation data product are: GPM satellite DPR data after time-space matching, Sentinel-1 satellite GRDH data, CloudSat satellite CPR data, Sunflower 8 geostationary satellite passive infrared precipitation data, microwave imager GMI passive microwave precipitation data. There are 5 precipitation data in total, so the input layer is set to 5 neurons plus 1 bias, the number of hidden layers is set to 2 layers, the number of neurons in each layer is 3 and 2 respectively, and each layer includes 1 bias (this patent is intended to explain the idea, the number of hidden layers and the number of neurons need to be determined by experiments) The number of neurons in the output layer is 1 (rain gauge precipitation data), the neurons in each layer are not connected to each other, and the neurons between each layer are connected to each other through different weights. Then, according to the set activation function, it is passed backward layer by layer, and an error signal is generated by comparing the output data with the expected input data. The error signal is passed backward layer by layer, and the weights of each layer are continuously corrected. The activation function is the sigmiod function: f(x) is the output of the fully connected layer, x is the xth neuron in the fully connected layer, and e is the eigenvalue of the fully connected layer.
[0076] In this embodiment, the reverse transfer stage adopts the Levenberg-Marquardt minimization algorithm to achieve the fusion of active and passive satellite remote sensing precipitation data through a large amount of training.
[0077] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.
[0078] The contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.
Claims
1. A satellite remote sensing precipitation data fusion method based on deep learning, characterized in that include: Build a dataset for training deep learning models; Build a deep learning model including input layer, hidden layer, and fully connected layer; Randomly divide the training deep learning model data set to obtain two parts of randomly divided data sets, perform data set conversion on one part of the randomly divided data sets to obtain a converted training set with the same time and longitude and latitude; The converted training set is used as the input data of the deep learning model, and the output data of the deep learning model is compared with the randomly divided data set; When the comparison results meet the validation set error range, the training is stopped and the precipitation data fusion model is generated.
2. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 1, characterized in that: The training deep learning model data set includes rain gauge precipitation data, satellite data, geosynchronous orbit satellite passive infrared precipitation data, and low-Earth orbit passive microwave precipitation data; Rain gauge precipitation data are collected based on statistics from the target meteorological station, and geosynchronous orbit satellite passive infrared precipitation data and low-Earth orbit passive microwave precipitation data are collected based on meteorological satellites; Satellite data include GPM satellite DPR data, Sentinel-1 satellite GRDH data, and CloudSat satellite CPR data, which are collected based on the satellite data transmission network.
3. The satellite remote sensing precipitation data fusion method based on deep learning according to claim 1, characterized in that: The method for dividing the training deep learning model data set is: Randomly divide the training deep learning model data set into data set X and data set Y; Transform the dataset X through the time-space matching algorithm to obtain the dataset X′ with the same time, longitude and latitude; The dataset X′ and dataset Y are randomly divided into training set and validation set according to the corresponding relationship Y.
4. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 3, characterized in that: The training set and validation set account for 75% and 25% of the dataset for training the deep learning model, respectively.
5. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 3, characterized in that: The deep learning model is obtained by coupling the full connection layer according to the spatiotemporal matching model, using the training set as the input data of the full connection layer, the verification set Y as the ideal value of the full connection layer output, and the full connection layer setting as the optimization target. The full connection layer setting is iteratively adjusted by minimizing the loss function micro-optimization.
6. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 5, characterized in that: The minimization loss function is: Where f(x) is the output of the fully connected layer, x is the xth neuron in the fully connected layer, and e is the eigenvalue of the fully connected layer.
7. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 3, characterized in that: The method of training set X′ through the space-time matching algorithm is: X′=s(w×X+b) Where w is the time weight, b is the bias, and s is the weight function related to the distance.
8. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 5, characterized in that: The deep learning model includes an input layer, a hidden layer, and a fully connected layer, wherein: The training set X′ obtained by the spatiotemporal matching algorithm is used as the input data of the fully connected layer. According to the input data, neurons related to the number of collected precipitation data types and a bias are set in the input layer of the deep learning model. Different numbers of hidden layers are set, and the number of neurons in each hidden layer is set respectively, and a bias is set for each layer to ensure that the neurons in the input layer, each hidden layer, and the fully connected layer are not connected to each other, and the connection between neurons in each layer is achieved by setting weight values.
9. The method for fusion of satellite remote sensing precipitation data based on deep learning according to claim 8, characterized in that: Taking the fully connected layer setting as the optimization target, the method of iteratively adjusting the fully connected layer setting by minimizing the loss function micro-optimization is: Collect input data and output data of each layer in the deep learning model, transfer data according to the activation function, compare the difference between the expected input data and the output data, generate an error signal based on the difference, and transfer it back layer by layer to correct the weights between neurons in each layer until the difference between the expected input data and the output data meets the error range.
10. A fusion system for implementing the satellite remote sensing precipitation data fusion method according to claim 1, characterized in that: It includes data set acquisition module, learning model construction module, data set conversion module and model training module, among which: The data set collection module collects various rainfall data and satellite data to generate data sets for training deep learning models; The learning model construction module constructs a deep learning model including an input layer, a hidden layer, and a fully connected layer as the initial learning model; The data set conversion module randomly divides the data set for training the deep learning model, obtains two parts of the randomly divided data set, performs data set conversion on one part of the randomly divided data set, obtains the converted training set of the same time and longitude and latitude, and sends the converted training set and the other part of the randomly divided data set to the model training module; The model training module uses the converted training set as the input data of the initial learning model, compares the output data of the deep learning model with the randomly divided data set, and obtains the comparison result. When the comparison result meets the error range of the validation set, the training is stopped and the precipitation data fusion model is generated.