Quantitative precipitation estimation method based on hurdle-imdl framework

Abstract

The application discloses a kind of quantitative precipitation estimation methods based on Hurdle-IMDL framework, comprising: obtaining historical precipitation measurement data and meteorological satellite observation data;Based on Hurdle model, construct biased quantitative precipitation estimation probability model, revise the biased rainfall estimation probability submodel according to the IMDL method, into experience distribution to construct experience quantitative precipitation estimation probability model and derive its negative log-likelihood function;Build AI model, use negative log-likelihood function as loss function to optimize AI model;Meteorological satellite observation data of the region to be inverted is input into the AI model trained, to obtain the estimated value of the parameter of experience quantitative precipitation estimation probability model, then estimate precipitation according to conditional expectation.The application uses Hurdle model to solve zero inflation problem, and uses IMDL learning method to deal with long tail problem, to improve the inversion accuracy of strong to extreme precipitation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of quantitative remote sensing, quantitative precipitation estimation and artificial intelligence, and in particular to a quantitative precipitation estimation method based on a Hurdle-IMDL artificial intelligence learning framework. BACKGROUND

[0002] In recent years, artificial intelligence (AI) has been widely applied in quantitative remote sensing (QRS), significantly promoting the development of QRS. However, when the label distribution is imbalanced, conventional learning focuses on common samples, while underfitting occurs on rare samples, and the obtained inversion model misestimates rare values, leading to poor performance on extreme events. Among numerous environmental variables, precipitation distribution is the most imbalanced and obvious, and AI-based quantitative precipitation estimation (QPE) models significantly underestimate heavy precipitation.

[0003] In related research, existing methods can be roughly divided into four categories: the first category-decomposition learning. Under this learning paradigm, precipitation distribution imbalance is decomposed into zero inflation (precipitation samples are significantly less than non-precipitation samples) and long tail (strong precipitation samples are much less than weak precipitation samples), and the two parts are treated respectively. Zero inflation is usually solved by two-step modeling: first, detect whether precipitation occurs, then estimate precipitation intensity. The long tail is handled by data resampling, ensemble learning or introducing specific constraints to the objective function. This method still has significant shortcomings in strong precipitation inversion and cannot effectively solve the problem. The second category-cost sensitive learning. The mean square error (MSE) is decomposed into a non-precipitation term (observation = 0) and a precipitation term (observation > 0), the latter is further divided into an underestimation term (inversion < observation) and an overestimation term (inversion > observation), and each term is assigned a weight to adjust the model bias. This method improves the consistency of inversion and observation value distribution and improves the performance of strong precipitation detection; however, when the precipitation is greater than 15mm·h -1 , the model still shows significant shortcomings. The third category-generative learning. Some scholars have proposed a generative model for QPE, and the evaluation shows that the model has excellent ability to reconstruct the fine structure of precipitation space, which is closely related to extreme precipitation. At the same time, frequency distribution analysis shows that the number of strong precipitation samples inverted by generative learning is significantly more than that by conventional learning. The generative model shows the potential to enhance precipitation inversion, but its effectiveness still needs further verification. The fourth category-multitask learning. A collaborative multitask learning method enables detection and estimation tasks to promote each other. For the case where the precipitation rate is higher than 10mm·h -1 , the detection performance is improved by 4%. However, the analysis is limited to the case where the precipitation rate is higher than 10mm·h -1 , and lacks evaluation of strong to extreme precipitation inversion error, so the effectiveness of this method is not fully clear.

[0004] In summary, there is no clear and effective method to deal with the precipitation imbalance in QPE, so as to improve the inversion of heavy to extreme precipitation. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems existing in the related art.

[0006] One object of the present application is to provide a quantitative precipitation estimation method based on a Hurdle-IMDL framework, which uses a Hurdle model to solve the zero inflation problem, and uses an inversion model debiasing learning (IMDL) to deal with the problem of long-tailed distribution, so as to improve the inversion effect of heavy precipitation and even extreme precipitation.

[0007] In order to achieve the above-mentioned purpose, one aspect of the present application provides a quantitative precipitation estimation method based on a Hurdle-IMDL framework, comprising the following steps:

[0008] Obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set, wherein the precipitation labels of the data set are characterized by zero inflation and long-tailed distribution in the non-zero part;

[0009] Constructing a biased quantitative precipitation estimation probability model based on a Hurdle model, which consists of a rain area detection probability model and a biased rainfall estimation probability model; converting the biased rainfall estimation probability model into an ideal rainfall estimation probability model based on IMDL to obtain an ideal quantitative precipitation estimation probability model; introducing an empirical distribution into the ideal quantitative precipitation estimation probability model to construct an empirical quantitative precipitation estimation probability model and derive its negative log-likelihood function;

[0010] Constructing an AI model for fitting the empirical quantitative precipitation estimation probability model; training the AI model using the negative log-likelihood function as a loss function to enable the AI model to achieve optimal estimation of the parameters of the empirical quantitative precipitation estimation probability model;

[0011] Inputting meteorological satellite observation data of a region to be inverted into the trained AI model to obtain parameter estimates of the corresponding empirical quantitative precipitation estimation probability model, and then estimating the precipitation according to the conditional expectation.

[0012] A further preferred technical solution of the present application is to construct an empirical quantitative precipitation estimation probability model under the Hurdle-IMDL framework; specifically:

[0013] Constructing a biased quantitative precipitation estimation probability model based on a Hurdle model, which consists of a rain area detection probability model and a biased rainfall estimation probability model, and models precipitation occurrence and precipitation amount respectively to deal with the zero inflation data characteristics of the data set;

[0014] For the biased rainfall estimation probability model part, the IMDL method is adopted to establish a conversion relationship between the ideal rainfall estimation probability model and the biased rainfall estimation probability model, to convert the biased rainfall estimation probability model into the ideal rainfall estimation probability model according to the conversion relationship, to correct the biased quantitative precipitation estimation probability model into the ideal quantitative precipitation estimation probability model, and to learn the ideal quantitative precipitation estimation probability model from the long-tail distribution data without processing.

[0015] The ideal quantitative precipitation estimation probability model is obtained by replacing the biased rainfall estimation model in the biased quantitative precipitation estimation probability model with the ideal model.

[0016] As preferred, the Hurdle model for constructing the biased quantitative precipitation estimation probability model is composed of a rain area detection model and a biased rainfall estimation model, which respectively model the occurrence of rainfall and the precipitation amount, and specifically:

[0017] For the precipitation amount and the remote sensing signal , the conditional probability density function is represented as:

[0018] ;

[0019] wherein, represents the probability of no rainfall under the condition of a given remote sensing signal ; represents the probability of the precipitation amount taking a certain specific positive value under the condition of a given remote sensing signal , i.e. the biased rainfall estimation model.

[0020] The modeling process of the model is divided into two parts. The first part is to model the occurrence of rainfall, corresponding to the estimation ; and the second part is to model the precipitation amount, corresponding to the fitting .

[0021] As preferred, for the precipitation amount modeling part, a conversion relationship between the ideal rainfall estimation model and the biased rainfall estimation model is established, represented as:

[0022] ;

[0023] wherein, is a biased rainfall estimation probability model, is an ideal rainfall estimation probability model, is the probability distribution of the historical precipitation measurement data in the data set.

[0024] After replacing the biased rainfall estimation model with the ideal rainfall estimation model, the ideal quantitative precipitation estimation probability model is represented as:

[0025] .

[0026] As preferred, the empirical distribution is introduced into the ideal quantitative precipitation estimation probability model according to the practical scene, the empirical quantitative precipitation estimation probability model is constructed, and the negative log-likelihood function thereof is derived; specifically:

[0027] The biased rainfall estimation model in the biased quantitative precipitation estimation probability model is replaced by the ideal rainfall estimation model, and an ideal quantitative precipitation estimation probability model is obtained, and its expression is:

[0028] ;

[0029] Wherein, represents the parameter of the ideal rainfall estimation probability model; represents the parameter of , which is estimated from the data set by statistical method;

[0030] The negative log-likelihood of the empirical quantitative precipitation estimation probability model serves its fitting, and is expressed as:

[0031] ;

[0032] Each term is:

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] Wherein, is the number of labels in the data set, indicates an indicator function.

[0038] As preferred, based on the constructed ideal quantitative precipitation estimation probability model, the three parameters to be estimated are determined as , wherein is regarded as a hyperparameter and is given a fixed value;

[0039] The parameter is estimated by constructing an AI model, which is an improved version of the ordinary U-Net network, and an additional output module is added at the end of the network to estimate and at the same time; the improved network extracts deep features of meteorological satellite observation data ​and then processed by two branches, the first branch consisting of stacked 1x1 convolutional layers with a ReLU activation function following the first layer of convolutional layers, estimating parameters ; the second branch estimating by a parallel sequence ending with a Sigmoid activation function . The U-Net network is trained with the above negative log-likelihood function as the loss function.

[0040] As preferred, the precipitation is estimated according to conditional expectation, in particular:

[0041] .

[0042] Another aspect of the present application provides a non-transitory computer readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to execute the above-mentioned Hurdle-IMDL framework-based quantitative precipitation estimation method.

[0043] Still another aspect of the present application provides an electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus, the processor invokes the logical instructions in the memory to execute the above-mentioned Hurdle-IMDL framework-based quantitative precipitation estimation method.

[0044] Still another aspect of the present application provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program being executed by a processor, the computer executing the above-mentioned Hurdle-IMDL framework-based quantitative precipitation estimation method.

[0045] Beneficial effects: the Hurdle-IMDL framework-based quantitative precipitation estimation method of the present application proposes a learning framework-Hurdle-IMDL, wherein the Hurdle branch constructs a biased quantitative precipitation estimation probability model, and the IMDL corrects the biased rainfall estimation model in the biased quantitative precipitation estimation probability model, so that the AI model for estimating the parameters of the quantitative precipitation estimation probability model automatically corrects the tendency of heavy rain underestimation in the training stage, and outputs the already-debiased pixel-level precipitation intensity once in the inference stage, without additional resampling, weighting or post-processing, so as to obtain quantitative precipitation estimation results with complete spatial details, strong real-time performance and good generalization ability, and significantly improve the reliability of extreme precipitation detection and intensity estimation. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 Fig. 1 is a neural network structure diagram in Embodiment 1 of the present application;

[0047] Figure 2 Fig. 2 is the inversion result of each model of the first precipitation event in Embodiment 1 of the present application;

[0048] Figure 3 The inversion results of each model for the second precipitation event in Example 1 of the present application;

[0049] Figure 4 The comparative evaluation results of Hurdle-IMDL and the baseline model in Example 1 of the present application. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments, and they should not be understood as limiting the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application. In the description of the present application, it should be understood that the terms used are only for the purpose of description and should not be understood as indicating or implying relative importance.

[0051] The following will be described in conjunction with Figures 1-4 The Hurdle-IMDL framework-based quantitative precipitation estimation method provided by the present application is described.

[0052] Example 1: The present embodiment provides a Hurdle-IMDL framework-based quantitative precipitation estimation method, which specifically comprises the following steps:

[0053] S1, obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set.

[0054] In the present embodiment, a specific region (115.75°E–120.75°E, 28.0°N–33.0°N) is selected, and precipitation measurement data and observation data of the Japanese new-generation geostationary meteorological satellite Himawari-8 are collected for each summer (May–August) from 2016 to 2021. The region contains 5,407 precipitation measurement stations, which are densely distributed to ensure that reliable gridded precipitation products can be obtained by interpolation.

[0055] In standard mode, the Advanced Himawari Imager (AHI) onboard Himawari-8 provides full-disk data every 10 minutes. AHI contains 16 channels (band01–band16) covering visible, near-infrared, and far-infrared wavelengths. One infrared channel and five brightness temperature differences are selected as input features. A total of 4,883 samples of precipitation events are selected at time steps. Half-hour satellite data and hourly accumulated precipitation data are interpolated to a 0.05° × 0.05° grid (100 × 100 grid points) by the nearest neighbor interpolation method, resulting in 4,883 spatiotemporally matched samples. The samples for 2016–2019, 2020, and 2021 are used for training, validation, and testing, respectively. The statistics of the training samples show that the ratio of precipitation to non-precipitation grid points is approximately 3.6:1, in which ≈0.46 and , exhibiting significant zero-inflation and long-tail characteristics.

[0056] S2. Constructing the ideal quantitative precipitation estimation probability model according to the Hurdle model and IMDL.

[0057] The Hurdle model is a statistical model specifically designed to handle zero-inflation problems and is widely used in medical expenditure, species abundance counting, and insurance claims, and has been successfully applied in precipitation prediction. For the precipitation amount and remote sensing signal , the conditional probability density function (PDF) is expressed as:

[0058] (1).

[0059] where is the probability of no precipitation given the remote sensing signal ; is the probability of the precipitation amount taking a certain positive value given the remote sensing signal .

[0060] This biased quantitative precipitation estimation probability model contains two parts: a rain area detection model and a biased rainfall estimation model. The modeling process is accordingly divided into two parts. The first part is to model the occurrence of precipitation, corresponding to the estimation of ; the second part is to model the precipitation amount, corresponding to the fitting of .

[0061] Although the Hurdle model addresses the zero-inflation problem by separating zero values from positive values, the precipitation modeling using regular learning methods can still underestimate heavy precipitation due to the long-tailed distribution. To address this issue, the present embodiment employs the IMDL learning method to establish a conversion relationship between the ideal rainfall estimation model and the biased rainfall estimation model, and to learn the ideal rainfall estimation model from the unprocessed long-tailed distribution data.

[0062] When trained on a dataset with a long-tailed distribution, regular learning methods result in a biased rainfall estimation model. This phenomenon can be represented as:

[0063] (2);

[0064] In-depth analysis of equation (2). The forward describes how to generate remote sensing signals from environmental variables (e.g., precipitation , denoted as , i.e., the forward model. The inversion aims to calculate the precipitation from , denoted as , i.e., the inversion model. According to Bayes' theorem, the relationship between the forward model and the inversion model and the long-tailed dataset ( and ) is as follows:

[0065] (3);

[0066] As shown in equation (3), not only depends on the long-tailed dataset, but also depends on the forward model . Therefore, equation (2) can be further extended as:

[0067] (4);

[0068] Now consider an ideal scenario: an ideal balanced dataset, where and are uniformly distributed, denoted as and , respectively, and there is an ideal forward model . Under this ideal condition, the ideal rainfall estimation model obtained by regular learning has no bias, represented as:

[0069] (5);

[0070] According to Bayes' theorem, the following equation holds:

[0071] (6);

[0072] In practice, due to theoretical and technical limitations, perfectly balanced datasets are not available. However, there is one key invariance: the forward model is determined mainly by the physical process, independent of data acquisition and modeling, and thus is not affected by data and algorithms. This invariance can be expressed as:

[0073] (7);

[0074] Based on equations (3), (6), and (7), it can be derived that:

[0075] (8);

[0076] Considering and equation (8), it is further obtained that:

[0077] (9);

[0078] Combining the above derivation steps, it can be obtained that:

[0079] (10);

[0080] Since is uniformly distributed, the in the numerator and denominator can be removed, and finally it is obtained that:

[0081] (11);

[0082] Equation (11) establishes a conversion relationship between the ideal rainfall estimation model and the biased rainfall estimation model .

[0083] The goal of this embodiment is to learn from long-tail datasets using equation (11). Assuming is parameterized by , denoted as , given a long-tail dataset containing independent and identically distributed samples , the likelihood function is defined as:

[0084] (12);

[0085] Substituting equation (11) into equation (12), it is obtained that:

[0086] (13);

[0087] parameters The likelihood function is estimated by the following equation:

[0088] (14);

[0089] Here, denotes the parameters of the ideal rainfall estimation model that maximizes the likelihood function given the long-tailed distribution dataset. This process makes it possible to fit the ideal rainfall estimation model directly from the unprocessed long-tailed data. This learning method is called Inversion Model Debiasing Learning (IMDL).

[0090] The Hurdle model and IMDL together constitute the complete Hurdle-IMDL framework, and the ideal quantitative precipitation estimation probability model obtained after correcting the biased quantitative precipitation estimation probability model is denoted as:

[0091] (15).

[0092] S4, according to the practical scene, introduce the empirical distribution to the ideal quantitative precipitation estimation probability model, construct the empirical quantitative precipitation estimation probability model and derive its negative log-likelihood function.

[0093] In order to realize the practical application of the ideal quantitative precipitation estimation probability model, the empirical distribution is introduced and its negative log-likelihood function is derived. The log-normal distribution is not only widely used to characterize the distribution of , but also successfully applied to model the conditional distribution of under given satellite observation conditions. According to formulas (1) and (11), can be expressed as:

[0094] (16);

[0095] where denotes the parameters of the ideal rainfall estimation model; denotes the parameters of , which are estimated from the dataset by statistical methods.

[0096] The negative log-likelihood of the empirical quantitative precipitation estimation probability model To achieve its fitting target function, it has an analytical solution and can be decomposed into the sum of the following items:

[0097] (17);

[0098] ;

[0099] ;​

[0100] ;

[0101] ;

[0102] where, is the index of the label in the dataset, denotes the indicator function. is a correction term, which embodies the adjustment and guidance correction of IMDL to regular learning.

[0103] S5, constructing an AI model to fit the above experience quantitative precipitation estimation probability model; training the AI model using the negative log-likelihood function.

[0104] According to the above formula, the three parameters to be estimated are respectively Based on the observed phenomenon, the embodiment adopts a hybrid estimation scheme. For , it can be observed that if no range constraint is imposed on the output of the AI model, its estimated value will often diverge to infinity, leading to model failure; on the other hand, when the range constraint is imposed, all estimated values converge uniformly to the set upper limit. This phenomenon shows that the current IMDL is not sufficient to guide the differentiated dynamic estimation of . In order to solve this problem, is regarded as a hyperparameter and is assigned a fixed value, which is selected from a candidate set by an external optimization method (such as grid search). Although this hyperparameterization is a compromise, subsequent quantitative precipitation estimation results show that it is an effective strategy.

[0105] For the parameter , the embodiment adopts an improved U-Net network to estimate it simultaneously. U-Net was originally developed for medical image segmentation and has since been widely applied in the fields of computer vision, weather forecasting and remote sensing. Inspired by the Mixture Density Network, the embodiment extends the U-Net by adding an additional output module, Figure 1 which shows the overall design of the U-Net network of the embodiment. The improved U-Net network extracts deep features of meteorological satellite observation data , which are then processed by two branches, the first branch consisting of stacked 1x1 convolutional layers, with a ReLU activation function used after the first convolutional layer to estimate the parameter ; the second branch estimates by a parallel sequence ending with a Sigmoid activation function.

[0106] S6, inputting the meteorological satellite observation data of the area to be inverted into the trained network to obtain the corresponding With the estimated value, combined with the given value, the specific precipitation amount is determined by conditional expectation , whose expression is:

[0107] (18).

[0108] To evaluate the effectiveness and superiority of Hurdle-IMDL, five benchmarks are selected in this embodiment, which are:

[0109] Original mean square error (OMSE) represents the learning method with mean square error (MSE) as the objective function. Nonlinear weighted mean square error (NWMSE) corresponds to the cost-sensitive learning method. Linear weighted mean square error (LWMSE) is a cost-sensitive learning method designed for label distribution imbalance in computer vision tasks. Diffusion is a typical generative model. Multi-task collaborative deep learning framework (MTCF) corresponds to the multi-task learning method.

[0110] A hierarchical evaluation method is adopted, and 12 thresholds (0, 0.1, 0.5, 1, 2, 3, 5, 7, 10, 15, 20, 30 mm·h -1 ) are set to define 12 levels, each of which corresponds to whether the observed value is greater than or equal to the given threshold. The inversion error is measured by the root mean square error and the mean error. The range of RMSE is from 0 to +∞, and the smaller the value, the lower the error (the higher the accuracy). ME reflects the system bias: ME<0 indicates underestimation, ME>0 indicates overestimation, and the larger the absolute value, the stronger the bias. The detection performance is measured by the detection probability (POD), the false alarm rate (FAR), and the equitable threat score (ETS). The range of POD is from 0 to 1, and the higher the value, the stronger the detection ability and the less the missed detection. The range of FAR is also from 0 to 1, and the smaller the value, the less the false detection. The equitable threat score (ETS) is commonly used for the evaluation of extreme precipitation forecasts. Compared with POD or FAR, it provides a more comprehensive measure of detection performance. Its value range is from -1 / 3 to 1, and the higher the value, the better the model detection performance, while ETS≤0 indicates no prediction ability.

[0111] Two typical precipitation events are taken as case studies to evaluate the advantages of Hurdle-IMDL.

[0112] The first case, as shown in Figure 2 , occurred on July 2, 2021, 04:00 UTC, when the Meiyu front triggered continuous precipitation in the East China region. The precipitation belt showed a zonal distribution, with a peak intensity of 48.9 mm·h -1The precipitation intensity retrieved by OMSE and Diffusion is generally weak, with local maximum values ​​below 30 mm·h. -1 Although LWMSE and NWMSE generated yields higher than 30 mm·h -1 The values ​​were calculated, but areas of heavy precipitation were significantly underestimated. In contrast, OMSE, LWMSE, NWMSE, and Diffusion significantly overestimated values ​​above 1 mm·h⁻¹. -1 The Hurdle-IMDL method misclassifies many samples with no precipitation or light rain as moderate rain due to its inaccurate prediction of precipitation range. Overall, the Hurdle-IMDL provides the most reasonable estimate, capturing both the extent of the precipitation band and accurately estimating the precipitation intensity.

[0113] The second precipitation case is as follows Figure 3 As shown, this occurred at 06:00 UTC on July 7, 2021, by which time the Meiyu front had dissipated. Under conditions of high temperature and humidity, convective cells developed in multiple locations, producing sporadic precipitation, with localized intensity reaching 30 mm·h. -1 In this context, MTCF and Diffusion significantly overestimated the precipitation area, while OMSE, LWMSE, and NWMSE underestimated the precipitation intensity. This again demonstrates that Hurdle-IMDL best approximates the observed results in both spatial extent and precipitation intensity.

[0114] Based on the entire test set, a quantitative comparison was conducted between Hurdle-IMDL and various benchmark models, and the results are as follows: Figure 4 As shown, when the threshold is between 0 and 10 mm·h -1 In the meantime, the RMSE differences among the methods are relatively small. However, for heavier precipitation (threshold ≥15 mm·h), the differences are less pronounced. -1 The Hurdle-IMDL model showed a significantly lower RMSE than all baseline models. Regarding ME, most methods exhibited negative values, demonstrating a systematic underestimation, particularly of heavy precipitation. However, the Hurdle-IMDL model had the smallest absolute ME value among all methods, significantly reducing the underestimation. Diffusion was observed at thresholds between 0.1 and 5 mm·h. -1 The highest POD was observed, but it also showed the highest FAR, indicating an overestimation of samples with no precipitation and light rain. In contrast, the Hurdle-IMDL showed the highest POD at a threshold greater than or equal to 7 mm·h. -1 It reached its maximum POD during extremely heavy precipitation (threshold ≥ 20 mm·h), while its FAR only occurred during extremely heavy precipitation (threshold ≥ 20 mm·h). -1 The Hurdle-IMDL outperformed other methods in mitigating the underestimation of heavy to extreme precipitation, demonstrating its effectiveness. Finally, the Hurdle-IMDL consistently achieved higher ETS values ​​than all baseline models across all levels. The relative improvement became more significant as the threshold increased. Notably, for thresholds greater than or equal to 30 mm·h… -1In the case of the reference model, the ETS is zero (indicating that the detection ability is almost negligible), while the ETS of the Hurdle-IMDL always remains above 0.1. Overall, these results show that the Hurdle-IMDL can effectively deal with the unbalanced distribution of precipitation, enhancing the performance of the model inversion, especially in the case of strong to extreme precipitation.

[0115] Embodiment 2: The embodiment provides a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to execute a quantitative precipitation estimation method based on a Hurdle-IMDL framework, the method comprising the following steps:

[0116] Obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set, the precipitation labels of the data set being zero-inflated and the non-zero part being characterized by a long-tail distribution;

[0117] Constructing a biased quantitative precipitation estimation probability model based on a Hurdle model, which consists of a rain area detection probability model and a biased rainfall estimation probability model; converting the biased model into an ideal model based on an IMDL to obtain an ideal quantitative precipitation estimation probability model; introducing an empirical distribution into the ideal quantitative precipitation estimation probability model according to a practical scenario to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof;

[0118] Building an AI model to fit the above-mentioned empirical quantitative precipitation estimation probability model; training the AI model by taking the negative log-likelihood function as a loss function;

[0119] Inputting meteorological satellite observation data of a region to be inverted into the trained AI model to obtain an estimated value of the parameter of the above-mentioned empirical quantitative precipitation estimation probability model, and then performing precipitation estimation according to the conditional expectation.

[0120] Embodiment 3: The embodiment provides an electronic device, which can include a processor, a communications interface, a memory and a communications bus, wherein the processor, the communications interface and the memory complete mutual communication through the communications bus. The processor can invoke a logical instruction in the memory to execute a quantitative precipitation estimation method based on a Hurdle-IMDL framework, the method comprising the following steps:

[0121] Obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set, the precipitation labels of the data set being zero-inflated and the non-zero part being characterized by a long-tail distribution;

[0122] A biased quantitative precipitation estimation probability model is constructed based on a Hurdle model, which is composed of a rain area detection probability model and a biased rainfall estimation probability model; the biased model is converted into an ideal model based on IMDL to obtain an ideal quantitative precipitation estimation probability model; an empirical distribution is introduced into the ideal quantitative precipitation estimation probability model according to a practical scene to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof;

[0123] An AI model is constructed to fit the above-mentioned empirical quantitative precipitation estimation probability model; the negative log-likelihood function is taken as a loss function to train the AI model;

[0124] Meteorological satellite observation data of a region to be inverted are input into the trained AI model to obtain parameter estimation values of the empirical quantitative precipitation estimation probability model, and then precipitation is estimated according to conditional expectation.

[0125] In addition, the logical instructions in the above-mentioned memory can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0126] Embodiment 4: The present embodiment provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute a quantitative precipitation estimation method based on a Hurdle-IMDL framework, the method comprising the following steps:

[0127] Obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set, the precipitation labels of the data set are zero-inflated and the non-zero part is characterized by a long-tailed distribution;

[0128] Based on the Hurdle model, a biased quantitative precipitation estimation probability model is constructed, which is composed of a rain area detection probability model and a biased rainfall estimation probability model; based on the IMDL, the biased model is converted into an ideal model to obtain an ideal quantitative precipitation estimation probability model; according to the practical scene, an empirical distribution is introduced into the ideal quantitative precipitation estimation probability model to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof;

[0129] An AI model is constructed to fit the above-described empirical quantitative precipitation estimation probability model; the negative log-likelihood function is taken as a loss function to train the AI model;

[0130] Meteorological satellite observation data of a region to be inverted are input into the trained AI model to obtain parameter estimation values of the empirical quantitative precipitation estimation probability model, and then the precipitation is estimated according to the conditional expectation.

[0131] The device embodiments described above are merely illustrative, wherein the units illustrated as separate components can or can not be physically separated, and the components illustrated as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0132] From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and the necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0133] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A quantitative precipitation estimation method based on Hurdle-IMDL framework, characterized in that, The method comprises the following steps: obtaining historical precipitation measurement data and meteorological satellite observation data to form a data set, wherein the precipitation label of the data set is zero-inflated and the non-zero part is characterized by a long-tailed distribution; constructing a biased quantitative precipitation estimation probability model based on a Hurdle model, which is composed of a rain area detection probability model and a biased rainfall estimation probability model; converting the biased rainfall estimation probability model into an ideal rainfall estimation probability model based on IMDL to obtain an ideal quantitative precipitation estimation probability model; introducing an empirical distribution into the ideal quantitative precipitation estimation probability model to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof; constructing an AI model for fitting the empirical quantitative precipitation estimation probability model; training the AI model by taking the negative log-likelihood function as a loss function; inputting meteorological satellite observation data of a region to be inverted into the trained AI model to obtain parameter estimates of the ideal quantitative precipitation estimation probability model, and then estimating the precipitation according to the conditional expectation.

2. The Hurdle-IMDL framework based quantitative precipitation estimation method according to claim 1, wherein, The biased quantitative precipitation estimation probability model based on the Hurdle model is composed of a rain area detection probability model and a biased rainfall estimation probability model; the biased rainfall estimation probability model is converted into an ideal rainfall estimation probability model based on IMDL to obtain an ideal quantitative precipitation estimation probability model; an empirical distribution is introduced into the ideal quantitative precipitation estimation probability model to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof; specifically: The biased quantitative precipitation estimation probability model based on the Hurdle model is composed of a rain area detection probability model and a biased rainfall estimation probability model, which models the occurrence of precipitation and the amount of precipitation, respectively, for dealing with the zero-inflated data characteristics of the data set; For the biased rainfall estimation probability model part, the IMDL method is used to establish a conversion relationship between the ideal rainfall estimation probability model and the biased rainfall estimation probability model, and the biased rainfall estimation probability model is converted into the ideal rainfall estimation probability model according to the conversion relationship, so as to correct the biased quantitative precipitation estimation probability model into the ideal quantitative precipitation estimation probability model, and learn the ideal quantitative precipitation estimation probability model from the long-tailed distribution data without processing.

3. The Hurdle-IMDL framework based quantitative precipitation estimation method according to claim 2, wherein, The biased quantitative precipitation estimation probability model based on the Hurdle model is specifically: For the precipitation amount and the remote sensing signal whose conditional probability density function is expressed as: ； wherein, denotes the probability that no precipitation occurs given the remote sensing signal ; and denotes the probability that the amount of precipitation given the remote sensing signal takes a certain positive value . The modeling process of the model is divided into two parts, the first part is to model the occurrence of precipitation, corresponding to the estimation ; the second part is to model the amount of precipitation, corresponding to the fitting .

4. The Hurdle-IMDL framework based quantitative precipitation estimation method of claim 3, wherein, For the precipitation modeling part of the biased quantitative precipitation estimation probability model, the conversion relationship between the ideal rainfall estimation probability model and the biased rainfall estimation probability model is established and expressed as: ； wherein, is a biased rainfall estimate probability model, is an ideal rainfall estimate probability model, is a probability distribution of historical precipitation measurement data in the dataset; According to the conversion relationship, the ideal rainfall estimation probability model is used to replace the biased rainfall estimation probability model to obtain the ideal quantitative precipitation estimation probability model, which is expressed as: 。 5. The Hurdle-IMDL framework based quantitative precipitation estimation method of claim 4, wherein, The empirical distribution is introduced into the ideal quantitative precipitation estimation probability model to construct an empirical quantitative precipitation estimation probability model and derive a negative log-likelihood function thereof; specifically: Taking a lognormal distribution as the random variable and The ideal precipitation probability model parameterization for the empirical distribution of ； wherein, parameters of the ideal rainfall estimate probability model; parameters of the ideal rainfall estimate probability model, parameters of the ideal rainfall estimate probability model, are estimated from the data set by statistical methods; Negative log-likelihood of ideal quantitative precipitation estimation probability model For the objective function used to estimate its parameters, denoted as: ； Each analytic term is respectively: ； ； ； ； wherein is the number of labels in the data set, denotes an indicator function.

6. The Hurdle-IMDL framework based quantitative precipitation estimation method of claim 5, wherein, Based on the constructed ideal quantitative precipitation estimation probability model, three parameters to be estimated are determined as wherein is regarded as the hyperparameter of the AI model and is given a fixed value; Parameters Estimation is performed by the constructed AI model, which is an improved U-Net network, with an additional output module added at the end of the network to estimate and simultaneously Improved u-net network extracts deep features of meteorological satellite observation data and then processed by two branches, the first branch consists of stacked 1x1 convolutional layers with ReLU activation function after the first layer of convolutional layers, estimated parameters ; the second branch estimates parameters through a parallel sequence ended with a Sigmoid activation function.

7. The Hurdle-IMDL framework based quantitative precipitation estimation method of claim 6, wherein, The precipitation is estimated according to the conditional expectation, which is specifically: 。 8. A non-transitory computer-readable storage medium, comprising: The computer instructions stored thereon cause the computer to execute the quantitative precipitation estimation method based on the Hurdle-IMDL framework according to any one of claims 1-7.

9. An electronic device, comprising: It comprises: The processor, the communication interface, the memory and the communication bus, wherein the processor, the communication interface, the memory complete the communication among each other through the communication bus, the processor calls the logic instruction in the memory, to execute the quantitative precipitation estimation method based on the Hurdle-IMDL framework in any one of claims 1-7.

10. A computer program product, characterised in that, The computer program product comprises a computer program, the computer program is stored on a non-transient computer readable storage medium, when the computer program is executed by the processor, the computer executes the quantitative precipitation estimation method based on the Hurdle-IMDL framework in any one of claims 1-7.

Images