A cloud condensation nucleus number concentration inversion method and system
By using machine learning models and Shapley value analysis in the prediction of cloud condensate number concentration, the problem of insufficient prediction interpretability and stability in the prior art is solved, and reliable prediction of cloud condensate number concentration in different regions is achieved.
Patent Information
- Application Number
- CN202411464818.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-02-25
- Estimated Expiration
- 2044-10-21
AI Technical Summary
The prior art has insufficient interpretability and stability in predicting cloud condensation nuclei concentrations, and it is difficult to simultaneously predict cloud condensation nuclei concentrations at oceans and land, especially at coastal junctions.
By acquiring and preprocessing oceanic and terrestrial meteorological site data, selecting, training and integrating machine learning models, combining Shapley values for interpretability analysis, a prediction model of cloud condensation nuclei concentration at coastal junctions is obtained to achieve cloud condensation in different regions Nuclear concentration prediction.
The interpretability and accuracy of cloud condensation nucleus concentration prediction is improved, the generalization ability of the model is enhanced, and reliable prediction of cloud condensation nucleus concentration at the junction of ocean, land and coast.
Smart Images
Figure CN118966035B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of atmospheric detection, and in particular relates to a cloud condensation nucleus number concentration inversion method and system. Background Art
[0002] Cloud condensation nuclei refer to aerosol particles that are activated to become clouds and fog droplets under supersaturated water vapor. Changes in the number concentration or properties of cloud condensation nuclei will lead to changes in cloud properties, which may affect the radiation budget of precipitation and clouds. Among the uncertainties in the global climate effective radiative forcing (ERF) related to aerosols, the aerosol indirect effect caused by aerosol-cloud interaction (ACI) is dominant and is usually difficult to observe. Therefore, it is of great scientific significance and practical value to propose a relatively simple and accurate inversion scheme to reduce the uncertainty of climate change predictions.
[0003] Optical parameters can indirectly reflect the size, shape, and composition of aerosols, and are relatively easy to obtain. Although many domestic and foreign researchers have used aerosol optical parameters to invert the number concentration of cloud condensation nuclei, in most studies, the parameterization schemes of each scholar have great uncertainty, and only one or several optical parameters are discussed in relation to the number concentration of cloud condensation nuclei, which inevitably ignores the role of some parameters in actual situations. The rational use of machine learning models may help solve this problem. At present, the use of machine learning models to study the number concentration of cloud condensation nuclei is still in its infancy, and in most cases it is a "black box" model, whose internal mechanism and results lack interpretability, and the role of each parameter in the calculation process is vague. In addition, existing models can usually only predict the number concentration of cloud condensation nuclei in a single area, such as the number concentration of cloud condensation nuclei in the ocean or land, but cannot simultaneously predict the number concentration of cloud condensation nuclei at the junction of the ocean and land, or even the coast.
[0004] Therefore, a method is needed that can enhance the interpretability and stability of machine learning models for predicting cloud condensation nucleus number concentration, have strong generalization ability, and improve prediction accuracy. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a cloud condensation nucleus number concentration inversion method and system, which can achieve reliable prediction of cloud condensation nucleus number concentration in different areas such as ocean, land, and coastal boundaries.
[0006] The present invention provides the following technical solutions:
[0007] In a first aspect, a cloud condensation nucleus number concentration inversion method is provided, comprising:
[0008] Obtain ocean and land meteorological station data and perform preprocessing;
[0009] Based on the preprocessed oceanic and land meteorological station data, model selection, model training, model integration and model optimization are carried out to obtain the oceanic and land cloud condensation nucleus number concentration prediction models;
[0010] The oceanic and land-based cloud condensation nucleus number concentration prediction models are used to invert the oceanic and land-based meteorological stations respectively, and the interpretability analysis is performed in combination with the Shapley value.
[0011] According to the prediction models of oceanic and land-based cloud condensation nuclei concentration and the results of interpretability analysis, the prediction model of cloud condensation nuclei concentration at the coastal junction is obtained.
[0012] Based on the oceanic and land-type cloud condensation nucleus number concentration prediction models and the cloud condensation nucleus number concentration prediction model at the coastal junction, the prediction of cloud condensation nucleus number concentration in different regions can be achieved.
[0013] Further, the meteorological site data includes optical data, meteorological element data and cloud condensation nucleus number concentration data;
[0014] The optical data include scattering coefficient, absorption coefficient, transmittance, backscattering coefficient, backscattering ratio, Åstrom index, asymmetry factor and single scattering albedo at three wavelengths, and the meteorological element data include relative humidity.
[0015] Further, the preprocessing includes identifying and removing outliers and missing values in the data, and performing spatiotemporal matching;
[0016] The identification and removal of outliers and missing values in the data includes: performing statistical analysis on the data, calculating the mean and standard deviation, defining data points that are more than 3 standard deviations away from the mean as outliers, and processing the outliers and missing values. The formula is as follows:
[0017] ;
[0018] In the formula, x i represents the original data points, NaN Indicates that the data is abnormal or missing and is removed;
[0019] The time-space matching includes: matching the data of different instruments at different meteorological stations in time series and eliminating other data. The formula is as follows:
[0020] ;
[0021] In the formula, T i , T j Indicates the original time corresponding to the data point,i , j represents the time series of different instruments, M Indicates mismatch and is removed.
[0022] Furthermore, the model selection and model training method includes:
[0023] Select appropriate tree models based on the completeness, accuracy and consistency of meteorological station data, and use each tree model as a baseline model;
[0024] The preprocessed optical data and meteorological element data are input as feature values, the cloud condensation nucleus number concentration is predicted as the target value, and each baseline model is trained.
[0025] Furthermore, the model integration method includes:
[0026] In the ensemble learning process of the model, the design dynamically adjusts the learning rate based on the model performance:
[0027] ;
[0028] In the formula, To dynamically adjust the learning rate, is the initial learning rate, and To adjust the parameters, t is the current iteration number, is the error improvement between two iterations;
[0029] In ensemble learning using multiple models, the contribution of each model is optimized by the following weight assignment formula:
[0030] ;
[0031] In the formula, is the weight of the model in the ensemble, MAE a For the a The mean absolute error of the model, MAE b For the b The mean absolute error of the model, n is the total number of models.
[0032] Furthermore, the model optimization method includes using a Bayesian optimization algorithm to adaptively adjust each model hyperparameter so that each evaluation index reaches the optimal value, as follows:
[0033] Observations are collected from the objective function through a Gaussian process, where the objective function is modeled as:
[0034] ;
[0035] In the formula, f ( x ) is the input The random function value at , GP(·) is the Gaussian process model, is the mean function, is a single vector in the input space, is another separate vector in the input space that is used to For comparison, is the covariance function, which defines the similarity between any two points in the input space;
[0036] Collect observation data from the objective function. At each iteration, the objective function is evaluated at the point that is most likely to improve the objective function. The posterior distribution of the probability model is updated using the collected observation data. The mean and variance are updated as follows:
[0037] ;
[0038] ;
[0039] In the formula, is the updated mean, is the updated variance, is the newly input observation point, is the observed input point set, is the kernel function calculated from a new input observation point To the observed input point set The covariance of each point in K ( X n , X n ) is the covariance matrix, which represents the similarity between observation points; is the noise variance, which indicates the noise level of the observed data; I is the identity matrix, whose dimensions are K same; Y n is the corresponding observed value; For the newly input observation point The autocovariance of is used to calculate the posterior variance; k ( X n , ) is the newly input observation point x With each observed input point set X n Through the kernel function k The calculated similarity vector;
[0040] Define the acquisition function to determine the next location to evaluate the objective function. EI ( x ) is as follows:
[0041] ;
[0042] In the formula, is a single vector in the input space of the Gaussian process surrogate model x The predicted mean of the objective function value, is the best value observed so far. is a non-negative adjustment parameter, is the cumulative distribution function of the standard normal distribution, is the observation point of the Gaussian process proxy model for the new input x The predicted standard deviation of is the probability density function of the standard normal distribution, Z To standardize the improvement amount;
[0043] By repeating the process of observing, updating the posteriori, and selecting new points using the acquisition function until the stopping condition is met, the most appropriate model is selected as the prediction model for the number concentration of oceanic and land-based cloud condensation nuclei according to the specified benchmark.
[0044] Furthermore, the method for inverting the ocean-type and land-type meteorological stations includes: using real-time acquired and pre-processed optical data and meteorological element data to input into the ocean-type and land-type cloud condensation nucleus number concentration prediction model, and outputting the cloud condensation nucleus number concentration;
[0045] The method for performing interpretability analysis in combination with Shapley value includes:
[0046] According to the Shapley value, explain the contribution, positive and negative impact of each feature on the model prediction results:
[0047] ;
[0048] ;
[0049] ;
[0050] In the formula, For a single feature, for The Shapley value of , S is the subset of features used in the model, val is a value function used to evaluate the contribution of S to the model prediction, val ( S ) is in SThe model output value under are all the features in the model, p is the total number of features, To represent the number of elements in S, ! is the factorial of the size of S, which means it does not include The number of feature combinations, Indicates adding features The change in the predicted value caused by is the prediction function of the model, is the value function defined for S, used to evaluate the feature set in The relative contribution of ( ) is represented as a The function of represents the probability measure on features other than S, W For the entire input dataset, yes about W The expected prediction value of , that is, the average prediction value of the entire data set, ( w ) represents the model for the input w The predicted output is is the intercept term of the model, also called the bias, Indicates l Input features, Indicates l The coefficients or weights of the features, Represents all input features and their respective coefficients The weighted sum of M Represents the total number of features in the model;
[0051] The interaction effect of each feature is explained in terms of Shapley value:
[0052] ;
[0053] In the formula, Representing two features in a machine learning model and The interaction effect or interaction of Indicates that it contains S And two characteristics and The predicted value of the model in the case of , When S Contains only , The function of one side,v ( S ) represents the subset of features used in the model S Contribution or model value to the model prediction output, excluding and The predicted value of the model when When positive, it indicates the characteristics and When they work together, they contribute more to the model prediction than when they work alone, indicating that there is a positive interaction between them; When negative, it indicates that the characteristic and When they appear together, their contribution to the model is less than the sum of their individual effects, indicating that there is redundancy or mutual inhibition between them.
[0054] Sort the calculated Shapley values and Shapley interaction values, and determine the Shapley main effect value based on each Shapley value and its corresponding Shapley interaction value;
[0055] The interpretability of the prediction models for oceanic and land-based cloud condensation nuclei number concentrations was analyzed based on multiple Shapley main effect values.
[0056] Furthermore, the oceanic and land-based cloud condensation nucleus number concentration prediction models are:
[0057] ;
[0058] ;
[0059] In the formula, P ( ) is the land cloud condensation nucleus number concentration prediction model, m is the number of models in the ensemble, is the number of condensation nuclei in the land-based cloud concentration prediction model assigned to the r The weight of the model satisfies , It is the first prediction model of the number concentration of condensation nuclei of land-based clouds. r Models; Q ( ) is the prediction model for the number concentration of cloud condensation nuclei over the sea, is the number of condensation nuclei in the oceanic cloud concentration prediction model assigned to the u The weight of the model satisfies , It is the first prediction model of oceanic cloud condensation nucleus number concentration. u Models, are all the features in the model, and p is the total number of features.
[0060] Furthermore, the cloud condensation nucleus number concentration prediction model at the coast junction is:
[0061] ;
[0062] ;
[0063] ;
[0064] In the formula, is the variable corresponding to the coastal station data according to the onshore Shapley sorting, For order, for After calibration, the sequence L is the distance between the land station and the coast, For land The average value of For the sea The average value of For the sea After calibration, the sequence is the variable corresponding to the offshore Shapley sorting of the coastal station data, For order, is a constant, if and If they are the same, then postpone. For land The average value of For the sea The average value of CCN coast is the cloud condensation nucleus number concentration prediction model at the coast junction. Trained machine learning models for onshore use.
[0065] In a second aspect, a cloud condensation nucleus number concentration inversion system is provided, comprising:
[0066] Data acquisition and preprocessing module, used to acquire and preprocess ocean and land meteorological station data;
[0067] The first model building module is used to perform model selection, model training, model integration and model optimization based on the preprocessed oceanic and land-type meteorological station data to obtain oceanic and land-type cloud condensation nucleus number concentration prediction models;
[0068] The inversion and analysis module is used to invert oceanic and land-based meteorological stations using oceanic and land-based cloud condensation nucleus number concentration prediction models, and to perform interpretability analysis in combination with Shapley values;
[0069] The second model building module is used to obtain a cloud condensation nucleus number concentration prediction model at the coastal junction based on the ocean-type and land-type cloud condensation nucleus number concentration prediction models and interpretability analysis results;
[0070] The prediction module is used to predict the cloud condensation nucleus number concentration in different regions based on the ocean-type and land-type cloud condensation nucleus number concentration prediction models and the cloud condensation nucleus number concentration prediction model at the coastal junction.
[0071] Compared with the prior art, the present invention has the following beneficial effects:
[0072] (1) The present invention performs model selection, model training, model integration and model optimization based on the pre-processed ocean and land meteorological station data to obtain ocean and land cloud condensation nucleus number concentration prediction models, and uses the ocean and land cloud condensation nucleus number concentration prediction models to invert the ocean and land meteorological stations respectively, and combines the Shapley value to perform interpretability analysis, thereby obtaining a cloud condensation nucleus number concentration prediction model at the coastal junction based on the ocean and land cloud condensation nucleus number concentration prediction models and the interpretability analysis results, and finally realizes the prediction of cloud condensation nucleus number concentration in different regions through the ocean and land cloud condensation nucleus number concentration prediction models and the cloud condensation nucleus number concentration prediction model at the coastal junction;
[0073] (2) The present invention uses the Shapley value interpretation method to analyze the impact of predictors on model output, including Shapley value, interaction effect value and main effect value, thereby explaining the complex relationship between the internal input and output of the machine learning model. This not only enhances the interpretability of the model, but also improves the accuracy of the cloud condensation nucleus number concentration prediction method and the reliability of the results;
[0074] (3) The present invention improves the accuracy and robustness of the model by integrating multi-source data and ensemble learning models. At the same time, it improves the generalization ability of the model, so that the model can adapt to the prediction needs in different regions and different climatic conditions around the world, and realize reliable prediction of cloud condensation nucleus number concentration in different regions such as the ocean, land, and coastal boundaries. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] Figure 1 is a schematic flow chart of the cloud condensation nucleus number concentration inversion method in Example 1 of the present invention;
[0076] Figure 2 is a comparison chart of the cloud condensation nucleus number concentration calculated by the method of Example 1 and the actual cloud condensation nucleus number concentration of the present invention;
[0077] Figure 3 is a scatter plot comparing the cloud condensation nucleus number concentration calculated by the method of Example 1 and the actually measured cloud condensation nucleus number concentration;
[0078] Figure 4 The method provided in Example 1 of the present invention uses Shapley value to analyze the importance ranking diagram of characteristic values of various parameters when predicting the number concentration of cloud condensation nuclei;
[0079] Figure 5 is a diagram constructed in a two-dimensional coordinate system using Shapley main effect values in Example 2 of the present invention;
[0080] Figure 6 It is a schematic diagram of the predicted value of cloud condensation nucleus number concentration at the coastal boundary made by the method provided in Example 1 of the present invention. DETAILED DESCRIPTION
[0081] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0082] Example 1
[0083] like Figure 1 As shown, this embodiment provides a cloud condensation nucleus number concentration inversion method, the steps are as follows:
[0084] Step S1, obtaining ocean and land meteorological station data and performing preprocessing.
[0085] The meteorological station data include optical data, meteorological element data and cloud condensation nucleus number concentration data; the optical data include scattering coefficient (BS), absorption coefficient (BA), transmittance (TR), backscattering coefficient (BBS), backscattering ratio (BSF), Angstrom index (AE), asymmetry factor (g) and single scattering albedo (SSA) at three wavelengths, and the meteorological element data include relative humidity (RH). The data set with larger data volume is divided into training set and test set according to 8:2, and the data set with smaller data volume is divided into training set and test set according to 7:3.
[0086] The preprocessing includes identifying and removing outliers and missing values in the data, and performing time-space matching.
[0087] The identification and removal of outliers and missing values in the data includes: performing statistical analysis on the data, calculating the mean and standard deviation, defining data points that are more than 3 standard deviations away from the mean as outliers, and processing the outliers and missing values. The formula is as follows:
[0088] ;
[0089] In the formula, x i represents the original data points, NaN Indicates that the data is abnormal or missing and is removed.
[0090] The time-space matching includes: matching the data of different instruments at different meteorological stations in time series and eliminating other data. The formula is as follows:
[0091] ;
[0092] In the formula, T i , T j Indicates the original time corresponding to the data point, i , j represents the time series of different instruments, M Indicates mismatch and is removed.
[0093] Step S2: Based on the preprocessed oceanic and land meteorological station data, model selection, model training, model integration and model optimization are performed to obtain oceanic and land cloud condensation nucleus number concentration prediction models.
[0094] Step S2.1, model selection and model training, the specific method includes:
[0095] Select appropriate tree models based on the completeness, accuracy and consistency of meteorological station data, and use each tree model as a baseline model;
[0096] The preprocessed optical data and meteorological element data are input as feature values, the cloud condensation nucleus number concentration is predicted as the target value, and each baseline model is trained.
[0097] Step S2.2: Model integration. The specific method includes:
[0098] In the ensemble learning process of the model, in order to optimize the convergence speed and accuracy of the machine learning model during training, a dynamic adjustment learning rate based on model performance is designed:
[0099] ;
[0100] In the formula, To dynamically adjust the learning rate, is the initial learning rate, and To adjust the parameters, t is the current iteration number, is the error improvement between two iterations;
[0101] In ensemble learning using multiple models, the contribution of each model is optimized by the following weight assignment formula:
[0102] ;
[0103] In the formula, is the weight of the model in the ensemble, MAE a For the a The mean absolute error of the model, MAE b For the b The mean absolute error of the model, For all models The sum of the values, n is the total number of models.
[0104] Step S2.3, model optimization, the specific method includes using the Bayesian optimization algorithm to adaptively adjust the hyperparameters of each model to optimize each evaluation index, the steps are as follows:
[0105] Observations are collected from the objective function through a Gaussian process, where the objective function is modeled as:
[0106] ;
[0107] In the formula, f ( x ) is the input The random function value at , GP(·) is the Gaussian process model, is the mean function, usually assumed to be zero (or some other constant), is a single vector in the input space, is another separate vector in the input space that is used to For comparison, is the covariance function, which defines the similarity between any two points in the input space;
[0108] Collect observation data from the objective function. At each iteration, the objective function is evaluated at the point that is most likely to improve the objective function. The posterior distribution of the probability model is updated using the collected observation data. The mean and variance are updated as follows:
[0109] ;
[0110] ;
[0111] In the formula, is the updated mean, is the updated variance, is the newly input observation point, is the observed input point set, is the kernel function calculated from a new input observation point To the observed input point set The covariance of each point inK ( X n , X n ) is the covariance matrix, which represents the similarity between observation points; is the noise variance, which indicates the noise level of the observed data; I is the identity matrix, whose dimensions are K same; Y n is the corresponding observed value; For the newly input observation point The autocovariance of is used to calculate the posterior variance; k ( X n , ) is the newly input observation point x With each observed input point set X n Through the kernel function k The calculated similarity vector;
[0112] Define the acquisition function to determine the next location to evaluate the objective function. EI ( x ) is as follows:
[0113] ;
[0114] In the formula, is a single vector in the input space of the Gaussian process surrogate model x The predicted mean of the objective function values, which reflects the best estimate of the model’s performance at that point without considering uncertainty; is the best value observed so far; is a non-negative adjustment parameter used to balance exploration and exploitation; is the cumulative distribution function (CDF) of the standard normal distribution; is the observation point of the Gaussian process proxy model for the new input x The prediction standard deviation indicates the uncertainty of the prediction value; is the probability density function (PDF) of the standard normal distribution, which is used to calculate the probability density of the random variable Z at a specific value; Z is the standardized improvement, which is a standard normal variable used to convert the improvement into a standard normal distribution.
[0115] Step S2.4, by repeating the process of observation, updating the posteriori, and selecting new points using the acquisition function until the stopping condition is met, the most appropriate model is selected as the oceanic and land-based cloud condensation nucleus number concentration prediction model based on the specified benchmark.
[0116] The prediction models for the number concentration of oceanic and land-based cloud condensation nuclei are:
[0117] ;
[0118] ;
[0119] In the formula, P ( ) is the land cloud condensation nucleus number concentration prediction model, m is the number of models in the ensemble, is the number of condensation nuclei in the land-based cloud concentration prediction model assigned to the r The weight of the model satisfies , It is the first prediction model of the number concentration of condensation nuclei of land-based clouds. r Models; Q ( ) is the prediction model for the number concentration of cloud condensation nuclei over the sea, is the number of condensation nuclei in the oceanic cloud concentration prediction model assigned to the u The weight of the model satisfies , It is the first prediction model of oceanic cloud condensation nucleus number concentration. u Models, are all the features in the model, and p is the total number of features.
[0120] Step S3: Use oceanic and land-based cloud condensation nucleus number concentration prediction models to invert oceanic and land-based meteorological stations respectively, and perform interpretability analysis in combination with Shapley values.
[0121] Step S3.1, inverting the oceanic and land-type meteorological stations, the specific method includes: using real-time acquired and pre-processed optical data and meteorological element data to input into the oceanic and land-type cloud condensation nucleus number concentration prediction model, and outputting the cloud condensation nucleus number concentration.
[0122] Step S3.2: Perform interpretability analysis in combination with Shapley value. The specific method includes:
[0123] Step S3.2.1, explain the contribution, positive and negative impact of each feature on the model prediction results based on the Shapley value:
[0124] ;
[0125] ;
[0126] ;
[0127] In the formula, For a single feature, for The Shapley value of , S is the subset of features used in the model, val is a value function used to evaluate the contribution of S to the model prediction, val ( S ) is in S The model output values below represent the average contribution of the feature to the model prediction as well as the positive and negative contributions; are all the features in the model, p is the total number of features, To represent the number of elements in S, ! is the factorial of the size of S, which means it does not include The number of feature combinations, Indicates adding features The change in the predicted value caused by is the prediction function of the model, is the value function defined for S, used to evaluate the feature set in The relative contribution of ( ) is represented as a The function of represents the probability measure on features other than S, W For the entire input dataset, yes about W The expected prediction value of , that is, the average prediction value of the entire data set, ( w ) represents the model for the input w The predicted output is is the intercept term of the model, also called the bias, Indicates l Input features, Indicates l The coefficients or weights of the features, Represents all input features and their respective coefficients The weighted sum of M Represents the total number of features in the model.
[0128] Calculate the Shapley value to evaluate the influence of each feature on the model prediction results. This process involves adding or removing each feature one by one and observing the changes in the model prediction results. By calculating the contribution of different feature combinations, the Shapley value can be considered as the average marginal contribution of each feature to the model prediction results. It can quantify the relative importance of each feature in the model decision and explain the prediction results.
[0129] Step S3.2.2: Explain the interaction of each feature based on the Shapley value:
[0130] ;
[0131] In the formula, Representing two features in a machine learning model and The interaction effect or interaction of Indicates that it contains S And two characteristics and The predicted value of the model in the case of , When S Contains only , The function of one side, v ( S ) represents the subset of features used in the model S Contribution or model value to the model prediction output, excluding and The predicted value of the model when When positive, it indicates the characteristics and When they work together, they contribute more to the model prediction than when they work alone, indicating that there is a positive interaction between them; When negative, it indicates that the characteristic and When they appear together, their contribution to the model is less than the sum of their individual effects, indicating that there is redundancy or mutual inhibition between them.
[0132] Step S3.2.3, sort the Shapley values calculated in step S3.2.1 and the Shapley interaction values calculated in step S3.2.2 respectively, and determine the Shapley main effect value according to each Shapley value and its corresponding Shapley interaction value.
[0133] Step S3.2.4: Perform interpretability analysis on the oceanic and land-based cloud condensation nucleus number concentration prediction models based on the multiple Shapley main effect values obtained in step S3.2.3.
[0134] Step S4: According to the oceanic and land-type cloud condensation nucleus number concentration prediction models and the interpretability analysis results, a cloud condensation nucleus number concentration prediction model at the coastal junction is obtained.
[0135] The prediction model of cloud condensation nucleus number concentration at the coast junction is:
[0136] ;
[0137] ;
[0138] ;
[0139] In the formula, is the variable corresponding to the coastal station data according to the onshore Shapley sorting, For order, for After calibration, the sequence L is the distance between the land station and the coast, For land The average value of For the sea The average value of For the sea After calibration, the sequence is the variable corresponding to the offshore Shapley sorting of the coastal station data, For order, is a constant, if and If they are the same, then postpone. For land The average value of For the sea The average value of CCN coast is the cloud condensation nucleus number concentration prediction model at the coast junction. Trained machine learning models for onshore use.
[0140] Step S5: based on the oceanic and land-type cloud condensation nucleus number concentration prediction models and the cloud condensation nucleus number concentration prediction model at the coastal junction, the cloud condensation nucleus number concentration in different regions is predicted.
[0141] Example 2
[0142] In this embodiment, 100 cases are randomly selected, and the cloud condensation nucleus number concentration is predicted at a typical supersaturation (SS) = 0.4% using the method provided in Example 1, and then compared with the actual cloud condensation nucleus number concentration.
[0143] Figure 2 The figure is a comparison chart of the cloud condensation nucleus number concentration calculated by the method provided in Example 1 and the actual cloud condensation nucleus number concentration. The dotted line is the cloud condensation nucleus number concentration calculated by the method provided in Example 1 (i.e., the predicted value), and the solid line is the actual cloud condensation nucleus number concentration observation result (i.e., the true value). Figure 2 It can be seen that the two values are very similar.
[0144] Figure 3 The scatter plot is a comparison of the cloud condensation nucleus number concentration calculated by the method provided in Example 1 and the measured cloud condensation nucleus number concentration, including evaluation indicators. Figure 3 It can be seen that the method of the present invention can accurately estimate the cloud condensation nucleus number concentration (R 2 =0.916, RMSE=31.439).
[0145] Figure 4 This is a ranking diagram of the importance of each optical data and meteorological element data using Shapley value analysis when predicting the number concentration of cloud condensation nuclei provided in Example 1, where the underlined B, G, and R of each data represent three wavelengths (blue light: 450nm, green light: 550nm, and red light: 700nm), such as BS_B represents the scattering coefficient at 450nm, and the underlined AE represents the Angstrom index obtained by dividing the two wavelengths. Figure 4 It can be seen that BS, BBS, BA, BSF, and AE are positively correlated with the cloud condensation nucleus concentration, while g and SSA are negatively correlated with the cloud condensation nucleus concentration in most cases. The main factors affecting the cloud condensation nucleus concentration are BS and AE, while SSA has a small effect, indicating that compared with the particle chemical composition, aerosol concentration and particle size play an important role in the cloud condensation nucleus concentration.
[0146] Based on the spatial distribution law of the obtained Shapley main effect values in the two-dimensional coordinate system, the complex dependency relationship between the predictor and the model output results is identified, thereby explaining the machine learning model.
[0147] like Figure 5 As shown in the figure, the Shapley main effect value is used to construct a diagram in a two-dimensional coordinate system. By analyzing the spatial distribution of all samples in the coordinate system, the dependence properties between the predictor and the model output results (such as linearity or nonlinearity, response threshold, etc.) can be identified, thereby providing an explanation for the model.
[0148] like Figure 6 As shown, the method provided in Example 1 is used to predict the cloud condensation nucleus number concentration at the coastal junction (30.43N°, 91.96W°).
[0149] Example 3
[0150] Based on the same inventive concept as Example 1, this embodiment provides a cloud condensation nucleus number concentration inversion system, comprising:
[0151] Data acquisition and preprocessing module, used to acquire and preprocess ocean and land meteorological station data;
[0152] The first model building module is used to perform model selection, model training, model integration and model optimization based on the preprocessed oceanic and land-type meteorological station data to obtain oceanic and land-type cloud condensation nucleus number concentration prediction models;
[0153] The inversion and analysis module is used to invert oceanic and land-based meteorological stations using oceanic and land-based cloud condensation nucleus number concentration prediction models, and to perform interpretability analysis in combination with Shapley values;
[0154] The second model building module is used to obtain a cloud condensation nucleus number concentration prediction model at the coastal junction based on the ocean-type and land-type cloud condensation nucleus number concentration prediction models and interpretability analysis results;
[0155] The prediction module is used to predict the cloud condensation nucleus number concentration in different regions based on the ocean-type and land-type cloud condensation nucleus number concentration prediction models and the cloud condensation nucleus number concentration prediction model at the coastal junction.
[0156] The specific functional implementation of each of the above modules can be found in the relevant contents of the method in Example 1 and will not be elaborated here.
[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0159] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0161] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A cloud condensation nucleus number concentration inversion method, characterized in that: include: Obtain ocean and land meteorological station data and perform preprocessing; Based on the preprocessed oceanic and land meteorological station data, model selection, model training, model integration and model optimization are carried out to obtain the oceanic and land cloud condensation nucleus number concentration prediction models; The oceanic and land-based cloud condensation nucleus number concentration prediction models are used to invert the oceanic and land-based meteorological stations respectively, and the interpretability analysis is performed in combination with the Shapley value. According to the prediction models of oceanic and land-based cloud condensation nuclei concentration and the results of interpretability analysis, the prediction model of cloud condensation nuclei concentration at the coastal junction is obtained. Based on the oceanic and land-type cloud condensation nucleus number concentration prediction models and the coastal junction cloud condensation nucleus number concentration prediction model, the cloud condensation nucleus number concentration in different regions can be predicted; The model integration method includes: In the ensemble learning process of the model, the design dynamically adjusts the learning rate based on the model performance: ； In the formula, To dynamically adjust the learning rate, is the initial learning rate, and To adjust the parameters, t is the current iteration number, is the error improvement between two iterations; In ensemble learning using multiple models, the contribution of each model is optimized by the following weight assignment formula: ； In the formula, is the weight of the model in the ensemble, MAE a For the a The mean absolute error of the model, MAE b For the b The mean absolute error of the model, n is the total number of models.
2. The cloud condensation nucleus number concentration inversion method according to claim 1, characterized in that: The meteorological station data include optical data, meteorological element data and cloud condensation nucleus number concentration data; The optical data include scattering coefficient, absorption coefficient, transmittance, backscattering coefficient, backscattering ratio, Åstrom index, asymmetry factor and single scattering albedo at three wavelengths, and the meteorological element data include relative humidity.
3. The cloud condensation nucleus number concentration inversion method according to claim 1, characterized in that: The preprocessing includes identifying and removing outliers and missing values in the data, and performing time-space matching; The identification and removal of outliers and missing values in the data includes: performing statistical analysis on the data, calculating the mean and standard deviation, defining data points that are more than 3 standard deviations away from the mean as outliers, and processing the outliers and missing values. The formula is as follows: ； In the formula, x i represents the original data points, NaN Indicates that the data is abnormal or missing and is removed; The time-space matching includes: matching the data of different instruments at different meteorological stations in time series and eliminating other data. The formula is as follows: ； In the formula, T i , T j Indicates the original time corresponding to the data point, i , j represents the time series of different instruments, M Indicates mismatch and is removed.
4. The cloud condensation nucleus number concentration inversion method according to claim 2, characterized in that: The method of model selection and model training includes: The tree models are selected based on the completeness, accuracy and consistency of the meteorological station data, and the selected tree models are used as baseline models; The preprocessed optical data and meteorological element data are input as feature values, the cloud condensation nucleus number concentration is predicted as the target value, and each baseline model is trained.
5. The cloud condensation nucleus number concentration inversion method according to claim 4, characterized in that: The model optimization method includes using a Bayesian optimization algorithm to adaptively adjust each model hyperparameter so that each evaluation index reaches the optimal value, as follows: Observations are collected from the objective function through a Gaussian process, where the objective function is modeled as: ； In the formula, f ( x ) is the input The random function value at , GP(·) is the Gaussian process model, is the mean function, is a single vector in the input space, is another separate vector in the input space that is used to For comparison, is the covariance function, which defines the similarity between any two points in the input space; Collect observation data from the objective function. At each iteration, the objective function is evaluated at the point that is most likely to improve the objective function. The posterior distribution of the probability model is updated using the collected observation data. The mean and variance are updated as follows: ； ； In the formula, is the updated mean, is the updated variance, is the newly input observation point, is the observed input point set, is the kernel function calculated from a new input observation point To the observed input point set The covariance of each point in K ( X n , X n ) is the covariance matrix, which represents the similarity between observation points; is the noise variance, which indicates the noise level of the observed data; I is the identity matrix, whose dimensions are K same; Y n is the corresponding observed value; For the newly input observation point The autocovariance of is used to calculate the posterior variance; k ( X n , ) is the newly input observation point x With each observed input point set X n Through the kernel function k The calculated similarity vector; Define the acquisition function to determine the next location to evaluate the objective function. EI ( x ) is as follows: ； In the formula, is a single vector in the input space of the Gaussian process surrogate model x The predicted mean of the objective function value, is the best value observed so far. is a non-negative adjustment parameter, is the cumulative distribution function of the standard normal distribution, is the observation point of the Gaussian process proxy model for the new input x The predicted standard deviation of is the probability density function of the standard normal distribution, Z To standardize the improvement amount; By repeating the process of observation, updating the posteriori, and selecting new points using the acquisition function until the stopping condition is met, oceanic and terrestrial cloud condensation nucleus number concentration prediction models are selected according to the specified benchmark.
6. The cloud condensation nucleus number concentration inversion method according to claim 1, characterized in that: The method for inverting ocean-type and land-type meteorological stations comprises: using real-time acquired and pre-processed optical data and meteorological element data to input into the ocean-type and land-type cloud condensation nucleus number concentration prediction model, and outputting the cloud condensation nucleus number concentration; The method for performing interpretability analysis in combination with Shapley value includes: According to the Shapley value, explain the contribution, positive and negative impact of each feature on the model prediction results: ； ； ； In the formula, For a single feature, for The Shapley value of , S is the subset of features used in the model, val is a value function used to evaluate the contribution of S to the model prediction, val ( S ) is in S The model output value under are all the features in the model, p is the total number of features, To represent the number of elements in S, ! is the factorial of the size of S, which means it does not include The number of feature combinations, Indicates adding features The change in the predicted value caused by is the prediction function of the model, is the value function defined for S, used to evaluate the feature set in The relative contribution of ( ) is represented as a The function of represents the probability measure on features other than S, W For the entire input dataset, yes about W The expected prediction value of , that is, the average prediction value of the entire data set, ( w ) represents the model for input w The predicted output is is the intercept term of the model, also called the bias, Indicates l input features, Indicates l The coefficients or weights of the features, Represents all input features and their respective coefficients The weighted sum of M Represents the total number of features in the model; The interaction effect of each feature is explained in terms of Shapley value: ； In the formula, Representing two features in a machine learning model and The interaction effect or interaction of Indicates that it contains S And two characteristics and The predicted value of the model in the case of , When S Contains only , The function of one side, v ( S ) represents the subset of features used in the model S Contribution or model value to the model prediction output, excluding and The predicted value of the model when When positive, it indicates the characteristics and When they work together, they contribute more to the model prediction than when they work alone, indicating that there is a positive interaction between them; When negative, it indicates that the characteristic and When they appear together, their contribution to the model is less than the sum of their individual effects, indicating that there is redundancy or mutual inhibition between them. Sort the calculated Shapley values and Shapley interaction values, and determine the Shapley main effect value based on each Shapley value and its corresponding Shapley interaction value; The interpretability of the prediction models for oceanic and land-based cloud condensation nuclei number concentrations was analyzed based on multiple Shapley main effect values.
7. The cloud condensation nucleus number concentration inversion method according to claim 1, characterized in that: The prediction models for the number concentration of oceanic and land-based cloud condensation nuclei are: ； ； In the formula, P ( ) is the land cloud condensation nucleus number concentration prediction model, m is the number of models in the ensemble, is the number of condensation nuclei in the land-based cloud concentration prediction model assigned to the r The weight of the model satisfies , It is the first prediction model of the number concentration of condensation nuclei of land-based clouds. r Models; Q ( ) is the prediction model for the number concentration of cloud condensation nuclei over the sea, is the number of condensation nuclei in the oceanic cloud concentration prediction model assigned to the u The weight of the model satisfies , It is the first prediction model of oceanic cloud condensation nucleus number concentration. u Models, are all the features in the model, and p is the total number of features.
8. The cloud condensation nucleus number concentration inversion method according to claim 1, characterized in that: The prediction model of cloud condensation nucleus number concentration at the coast junction is: ； ； ； In the formula, is the variable corresponding to the coastal station data according to the onshore Shapley sorting, For order, for After calibration, the sequence L is the distance between the land station and the coast, For land The average value of For the sea The average value of For the sea After calibration, the sequence is the variable corresponding to the offshore Shapley sorting of the coastal station data, For order, is a constant, if and If they are the same, then postpone. For land The average value of For the sea The average value of CCN coast is the cloud condensation nucleus number concentration prediction model at the coast junction. Trained machine learning models for onshore use.
9. A cloud condensation nucleus number concentration inversion system, characterized in that: include: Data acquisition and preprocessing module, used to acquire and preprocess ocean and land meteorological station data; The first model building module is used to perform model selection, model training, model integration and model optimization based on the preprocessed oceanic and land-type meteorological station data to obtain oceanic and land-type cloud condensation nucleus number concentration prediction models; The inversion and analysis module is used to invert oceanic and land-based meteorological stations using oceanic and land-based cloud condensation nucleus number concentration prediction models, and to perform interpretability analysis in combination with Shapley values; The second model building module is used to obtain a cloud condensation nucleus number concentration prediction model at the coastal junction based on the ocean-type and land-type cloud condensation nucleus number concentration prediction models and interpretability analysis results; The prediction module is used to predict the cloud condensation nucleus concentration in different regions based on the oceanic and land-type cloud condensation nucleus concentration prediction models and the cloud condensation nucleus concentration prediction model at the coastal junction; The model integration method includes: In the ensemble learning process of the model, the design dynamically adjusts the learning rate based on the model performance: ； In the formula, To dynamically adjust the learning rate, is the initial learning rate, and To adjust the parameters, t is the current iteration number, is the error improvement between two iterations; In ensemble learning using multiple models, the contribution of each model is optimized by the following weight assignment formula: ； In the formula, is the weight of the model in the ensemble, MAE a For the a The mean absolute error of the model, MAE b For the b The mean absolute error of the model, n is the total number of models.