Random forest-based Mie scattering laser radar aerosol extinction coefficient inversion method

Abstract

The invention discloses a random forest-based Mie scattering laser radar aerosol extinction coefficient inversion method. The method comprises the following steps of 1, Mie scattering laser radar data set construction and standardization processing; step 2, predicting an aerosol extinction coefficient by using a random forest; and step 3, optimizing random forest model parameters by a Bayesian method. The method can improve the precision and reliability of aerosol extinction coefficient inversion.

Description

Technical Field

[0001] The present invention relates to the field of atmospheric detection technology, and in particular to a method for inverting aerosol extinction coefficient using Mie scattering laser radar based on random forest. Background Art

[0002] The atmospheric extinction coefficient is a key parameter that describes the relative attenuation of light per unit distance in the atmosphere. It is often used to assess ambient air quality and atmospheric visibility. The aerosol extinction coefficient mainly reflects the absorption and scattering of light by aerosols. Studying the aerosol extinction coefficient helps predict the concentration of aerosols and particulate matter in the atmosphere, thereby reflecting the air quality. Mie scattering lidar is widely used in atmospheric detection. It has the advantages of simple structure, low cost, high automation and long detection distance. It is especially suitable for continuous monitoring of aerosol and molecular extinction coefficients in the atmosphere day and night. However, since the Mie scattering signal decays with the square of the detection distance, it is easily interfered by electrical noise and background noise, resulting in unsatisfactory detection accuracy. At present, the Klett and Fernald method is the main method for Mie scattering lidar to invert the aerosol extinction coefficient. However, because it relies on the signal of a single detection time, it is limited by factors such as signal-to-noise ratio, calibration height, and lidar ratio, and the inversion accuracy is limited. In particular, it is difficult for the current method to effectively capture the changing trend of the aerosol extinction coefficient under continuous detection. Summary of the invention

[0003] The purpose of the present invention is to propose a method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest, which can improve the accuracy and reliability of aerosol extinction coefficient inversion.

[0004] The technical solution adopted by the present invention is a method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest, comprising the following steps:

[0005] Step 1: Construction and standardization of Mie scattering lidar dataset; Step 2: The process of random forest prediction of aerosol extinction coefficient; Step 3: Bayesian method to optimize random forest model parameters.

[0006] The present invention is also characterized in that:

[0007] Step 1 is specifically as follows: Step 1.1: Construction of Mie scattering lidar dataset; Step 1.2: Standardization of Mie scattering lidar dataset.

[0008] Step 1.1 is as follows:

[0009] First, the Mie scattering lidar dataset consists of multiple feature parameters, which are defined as follows:

[0010] 1. Mie scattering lidar echo signal strength v i,j, which means at the i-th detection time t i Under the condition that at the jth detection distance r j The Mie scattering lidar echo signal strength value at , where:

[0011] i=1,2,…,n, represents the number of detection time points;

[0012] j=1,2,…,m, represents the number of detection distance points;

[0013] The collection form is v i = {v i,1 ,v i,2 ,…,v i,m}, that is, at the detection time t i The set of Mie scattering lidar echo signal strength values ​​at all detection distances;

[0014] 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric pressure value at the location, in the collective form of p i ={p i,1 ,p i,2 ,…,p i,m};

[0015] 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at . The collection form is c i ={c i,1 ,c i,2 ,…,c i,m}.

[0016] 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at . The set form is t i ={t i,1 ,t i,2 ,…,t i,m};

[0017] 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,ri,m};

[0018] Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j (j=1,2,…,m), specifically expressed as:

[0019] x i ={x i,1 ,x i,2 ,...,x i,m}

[0020] Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows:

[0021] x i,j = {v i,j ,p i,j ,c i,j ,r i,j ,t i,j}

[0022] Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time;

[0023] Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is:

[0024] x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1},{vi,2 ,p i,2 ,c i,2 ,r i,2 ,t i,2},…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m}}.

[0025] The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set:

[0026] y i ={y i,1 ,y i,2 ,…,y i,m}

[0027] Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset:

[0028] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0029] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0030] Step 1.2 is as follows:

[0031] The input data set x of the Mie scattering lidar data set S is standardized by the standardization formula. The expression of the processing process is as follows:

[0032]

[0033] Among them, x i,j Indicates that the data is concentrated at the detection time t i and detection distance r j The original input data value at, mean(x) represents the mean value of the input data set x, std(x) represents the standard deviation of the input data set x, x i ' ,j Indicates the data value after normalization.

[0034] Step 2 is as follows:

[0035] Step 2.1: Generate a data subset, specifically:

[0036] From the Mie scattering lidar dataset S constructed in step 1, random sampling with replacement is performed using the bootstrap method. The specific steps are as follows:

[0037] (1) Randomly draw n times to obtain a Mie scattering lidar data subset containing n samples, called s;

[0038] (2) Repeat this process k times to generate k Mie scattering lidar data subsets, denoted as s1, s2, ..., s k ;

[0039] (3)s k represents the kth data subset, k = 1, 2, ..., K, each data subset contains n sample data, s k The form is:

[0040]

[0041] in, represents the input feature vector of the jth sample in the kth data subset, represents the jth target value in the kth data subset, i.e., the aerosol extinction coefficient;

[0042] Step 2.2: Decision tree generation, specifically:

[0043] For each Mie scattering lidar data subset s k , k=1,2,…,K, perform the following operations:

[0044] (1) Feature and label extraction: From the subset s k Extract the feature matrix X k Extract and label vector Y k (aerosol extinction coefficient) extraction, providing input data for decision tree generation, where X k Contains Mie scattering lidar signal strength, air pressure, temperature, detection distance and detection time, etc. k represents the corresponding aerosol extinction coefficient; that is, Medium, Features is extracted into the matrix Xk, Extracted to Y k ;

[0045] (2) Decision tree construction: The feature matrix X k and label vector Y kAs input, the Classification and Regression Trees (CART) algorithm is used to generate a decision tree T k ; The CART algorithm recursively divides the subset s k , using Mean Square Error (MSE) as the splitting criterion to find the best splitting features and splitting points, thereby realizing the construction of the tree structure.

[0046] (3) Repeat steps (1) and (2) K times for subsets s1, s2, ..., s K Generate K independent decision trees T1, T2, ..., T K ;

[0047] Step 2.3: Random forest model generation, specifically:

[0048] The trained K decision trees T1, T2, ..., T K , combined into a random forest model T, defined as

[0049] T={T1,T2,...,T K};

[0050] Step 2.4: Generate prediction results.

[0051] Step 2.4 is as follows:

[0052] (1) Single decision tree prediction: For a given new input data, it is input into each decision tree T in the random forest in turn. k In this case, we get K independent prediction results y k , where y k Represents the decision tree T k The predicted aerosol extinction coefficient for the input data, k = 1, 2, …, K;

[0053] (2) Result integration: The prediction results of all decision trees {y1,y2,...,y K} Perform simple averaging to obtain the final prediction result of the random forest model T Right now:

[0054]

[0055] This value is used as the final prediction output of the random forest model for the aerosol extinction coefficient of the input data.

[0056] Step 3 is as follows:

[0057] Step 3.1: Data division, specifically:

[0058] For each subset sk Divide into training set, validation set and test set according to the following proportions, as follows:

[0059] (1) Training set (X k,train ,Y k,train ): 70%, used to train decision tree T k and the random forest model T;

[0060] (2) Validation set (X k,val ,Y k,val ): 20%, used for decision tree T k and parameter adjustment and selection of the random forest model T;

[0061] (3) Test set (X k,test ,Y k,test ): 10%, used to evaluate the decision tree T k and the final performance of the random forest model T;

[0062] Step 3.2, parameter selection, specifically:

[0063] In the process of adjusting the random forest model T based on the Bayesian optimization method, the following key parameters need to be selected as optimization objects:

[0064] (1) The number of trees n tree : The total number of decision trees contained in the random forest; (2) The maximum number of features n feature : The maximum number of features considered by each split node; (3) Maximum depth n depth : The maximum depth of a single decision tree to control the complexity of the model; (4) The minimum number of sample splits n split : The minimum number of samples required when splitting a node to avoid over-splitting; (5) The minimum number of sample leaves n leaf : The minimum number of samples contained in the leaf node to control the termination condition of the tree;

[0065] Step 3.3: Construct the objective function, specifically:

[0066] The coefficient of determination R 2 Build a black box function for the target In the validation set (X k,val ,Y k,val ), R 2 The calculation formula is:

[0067]

[0068] Among them, Y k,val is the true target value of the validation set, Y′ k,val is the predicted value of the random forest model T, Y k,avg Yesk,val The average value of , N is the number of elements in the validation set.

[0069] Step 3.4: Construct a Gaussian process, specifically:

[0070] Let parameter vector Contains the following 5 parameters to be optimized:

[0071] (1) The number of trees n tree ; (2) Maximum number of features n feature ; (3) Maximum depth n depth ; (4) Minimum number of sample splits n split ; (5) Minimum number of sample leaves n leaf ;

[0072] Parameter vector Its parameter space is Ψ; the objective function The coefficient of determination R 2 , expressed as:

[0073]

[0074] By building a Gaussian process regression model For parameter vector And the objective function Fitting, the process of Bayesian optimization parameters is to maximize the predicted mean of the Gaussian process model To find the optimal parameter combination:

[0075]

[0076] Step 3.5, select the acquisition function, specifically:

[0077] The upper confidence bound (UCB) is selected as the acquisition function to balance the weight of parameter utilization and exploration. The UCB acquisition function is defined as:

[0078]

[0079] in, is the predicted mean of the Gaussian process regression model, is the prediction standard deviation of the Gaussian process regression model, and λ is the control and Balance weight parameters;

[0080] Step 3.6, repeat the iteration, specifically:

[0081] In the Bayesian optimization process, steps 3.4 (Gaussian process construction) and 3.5 (acquisition function selection) are repeated until one of the following stop conditions is met, and it is considered that the best parameter combination has been found;

[0082] When the optimization process meets one of the above stop conditions, the current optimal parameter combination is returned. That is, the objective function The parameter set that reaches the maximum value can improve the performance of the random forest model.

[0083] In step 3.6, the stopping conditions are as follows:

[0084] (1) Convergence judgment: If after multiple iterations, the objective function If the value of changes very little, that is, the improvement is lower than the set threshold, it is considered that the optimization process has converged and the model performance cannot be further significantly improved.

[0085] (2) Maximum number of iterations: Set the maximum number of iterations (such as 100, 200, etc.). When the maximum number of iterations is reached, the iteration stops and the optimal parameter combination is returned.

[0086] (3) Maximization of the objective function: Bayesian optimization continuously predicts the objective function through the Gaussian process regression model And through the acquisition function (such as UCB), new parameter combinations are continuously explored. If the value of the objective function tends to be stable after several rounds of iterations and the predicted value reaches the maximum value, it is considered that the optimal parameter combination has been found.

[0087] The beneficial effects of the present invention are:

[0088] (1) The method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest in the present invention analyzes the detection principle of Mie scattering lidar and its temporal and spatial characteristics, extracts data features, and constructs training and test sets based on the original continuous detection data. A Mie scattering lidar inversion model is established, and the model parameters are optimized using the Bayesian optimization method, thereby improving the accuracy and reliability of aerosol extinction coefficient inversion.

[0089] (2) High-precision inversion: The method of the present invention improves the accuracy of aerosol extinction coefficient inversion by integrating multiple decision tree models, and has higher accuracy and stability than traditional methods.

[0090] (3) Strong robustness: The random forest model shows good robustness when facing high-dimensional and noisy data. It can effectively handle various uncertain factors in Mie scattering lidar signals and improve the generalization ability of the model.

[0091] (4) Automated parameter optimization: The method of the present invention uses the Bayesian optimization method to automatically optimize the key parameters of the random forest model, avoiding the tedious process of manual parameter adjustment and improving the performance and efficiency of the model.

[0092] (5) Strong adaptability: The method of the present invention can adapt to Mie scattering lidar data under different environmental and climatic conditions, and is suitable for a variety of atmospheric environment monitoring applications, with wide applicability and promotion value.

[0093] (6) This method uses the continuous observation data of Mie scattering lidar to extract characteristics such as signal intensity, atmospheric pressure, temperature, detection time and distance, and adjusts the model parameters through Bayesian optimization to construct a more stable and accurate aerosol extinction coefficient inversion model, thereby improving the accuracy and reliability of the inversion results. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 is the structure of the Mie scattering laser radar echo signal in the method of the present invention;

[0095] Figure 2 The process of predicting aerosol extinction coefficient by random forest in the method of the present invention;

[0096] Figure 3 is the approximation effect of LR and Klett under data set A;

[0097] Figure 4 is the approximation effect of KNN and Klett under data set A;

[0098] Figure 5 is the approximation effect of BP and Klett under data set A;

[0099] Figure 6 is the approximation effect of BPGA and Klett under data set A;

[0100] Figure 7 is the approximation effect of RF and Klett under data set A;

[0101] Figure 8 is the approximation effect of the method of the present invention and Klett under data set A;

[0102] Fig. 9 is the approximation effect of LR and Klett under data set B;

[0103] Fig.10 is the approximation effect of KNN and Klett under data set B;

[0104] Fig.11 is the approximation effect of BP and Klett under data set B;

[0105] Fig.12 is the approximation effect of RF and Klett under data set B;

[0106] Fig.13 is the approximation effect of BPGA and Klett under data set B;

[0107] Fig.14 is the approximation effect of the method of the present invention and Klett under data set B;

[0108] Fig.15 is the approximation effect of LR and Klett under data set C;

[0109] Fig.16 is the approximation effect of KNN and Klett under data set C;

[0110] Fig.17 is the approximation effect of BP and Klett under data set C;

[0111] Fig.18 is the approximation effect of BPGA and Klett under data set C;

[0112] Fig.19 is the approximation effect of RF and Klett under data set C;

[0113] Fig. 20 This is the approximation effect of the method of the present invention and Klett on data set C. DETAILED DESCRIPTION

[0114] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0115] The present invention provides a method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest, comprising the following steps:

[0116] Step 1: Construction and standardization of Mie scattering lidar dataset;

[0117] Step 1 is as follows:

[0118] Step 1.1: Mie scattering lidar dataset construction. Step 1.1 is as follows:

[0119] First, in the present invention, the Mie scattering lidar data set is composed of multiple characteristic parameters, which are specifically defined as follows:

[0120] 1. Mie scattering lidar echo signal strength v i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r jThe Mie scattering lidar echo signal strength value at , where:

[0121] i=1,2,…,n, represents the number of detection time points;

[0122] j=1,2,…,m, represents the number of detection distance points;

[0123] The collection form is v i = {v i,1 ,v i,2 ,…,v i,m}, that is, at the detection time t i The Mie scattering lidar echo signal strength value set at all detection distances below; the structure of the Mie scattering lidar echo signal is as follows Figure 1 shown.

[0124] 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric pressure value at the location, in the collective form of p i ={p i,1 ,p i,2 ,…,p i,m};

[0125] 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at . The collection form is c i ={c i,1 ,c i,2 ,…,c i,m}.

[0126] 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at . The set form is t i ={t i,1 ,t i,2 ,…,t i,m};

[0127] 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,r i,m};

[0128] Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j (j=1,2,…,m), specifically expressed as:

[0129] x i ={x i,1 ,x i,2 ,...,x i,m}

[0130] Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows:

[0131] x i,j = {v i,j ,p i,j ,c i,j ,r i,j ,t i,j}

[0132] Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time;

[0133] Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is:

[0134] x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1},{v i,2 ,pi,2 ,c i,2 ,r i,2 ,t i,2},…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m}}.

[0135] The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set:

[0136] y i ={y i,1 ,y i,2 ,…,y i,m}

[0137] Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset:

[0138] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0139] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0140] Step 1.2: Standardization of Mie scattering lidar dataset, as follows:

[0141] In order to reduce the computational overhead caused by the uncertainty of the numerical range of Mie scattering lidar signals, the present invention performs normalization processing on the input data set x of the Mie scattering lidar data set S by the following normalization formula. The expression of the processing process is as follows:

[0142]

[0143] Among them, x i,j Indicates that the data is concentrated at the detection time t i and detection distance r j The original input data value at, mean(x) represents the mean value of the input data set x, std(x) represents the standard deviation of the input data set x, x i ' ,j Indicates the data value after standardization;

[0144] This normalization process converts each data x i,j Convert to a standard normal distribution with a mean of 0 and a standard deviation of 1 to eliminate the computational overhead caused by inconsistency in the data range.

[0145] Step 2: The process of predicting aerosol extinction coefficient by random forest, such as Figure 2 As shown;

[0146] Step 2 is as follows:

[0147] Step 2.1: Generate a data subset, specifically:

[0148] From the Mie scattering lidar dataset S constructed in step 1, random sampling with replacement is performed using the bootstrap method. The specific steps are as follows:

[0149] (1) Randomly draw n times to obtain a Mie scattering lidar data subset containing n samples, called s;

[0150] (2) Repeat this process k times to generate k Mie scattering lidar data subsets, denoted as s1, s2, ..., s k ;

[0151] (3)s k represents the kth data subset, k = 1, 2, ..., K, each data subset contains n sample data, s k The form is:

[0152]

[0153] in, represents the input feature vector of the jth sample in the kth data subset, represents the jth target value in the kth data subset, namely the aerosol extinction coefficient.

[0154] Step 2.2: Decision tree generation, specifically:

[0155] For each Mie scattering lidar data subset s k , k=1,2,…,K, perform the following operations:

[0156] (1) Feature and label extraction: From the subset s k Extract the feature matrix X k Extract and label vector Y k (aerosol extinction coefficient) extraction, providing input data for decision tree generation, where X k Contains Mie scattering lidar signal strength, air pressure, temperature, detection distance and detection time, etc. krepresents the corresponding aerosol extinction coefficient; that is, Medium, Features is extracted into the matrix Xk, Extracted to Y k ;

[0157] (2) Decision tree construction: The feature matrix X k and label vector Y k As input, the Classification and Regression Trees (CART) algorithm is used to generate a decision tree T k ; The CART algorithm recursively divides the subset s k , using the mean square error (MSE) as the splitting criterion to find the best splitting features and splitting points, thereby realizing the construction of the tree structure.

[0158] (3) Repeat steps (1) and (2) K times for subsets s1, s2, ..., s K Generate K independent decision trees T1, T2, ..., T K ;

[0159] Step 2.3: Random forest model generation, specifically:

[0160] The trained K decision trees T1, T2, ..., T K , combined into a random forest model T, defined as

[0161] T={T1,T2,...,T K};

[0162] Step 2.4: Generate prediction results, specifically:

[0163] The random forest model T is used to predict the input data, which includes the following steps:

[0164] (1) Single decision tree prediction: For a given new input data, it is input into each decision tree T in the random forest in turn. k In this case, we get K independent prediction results y k , where y k Represents the decision tree T k The predicted aerosol extinction coefficient for the input data, k = 1, 2, …, K;

[0165] (2) Result integration: The prediction results of all decision trees {y1,y2,...,y K} Perform simple averaging to obtain the final prediction result of the random forest model T Right now:

[0166]

[0167] This value is used as the final prediction output of the random forest model for the aerosol extinction coefficient of the input data.

[0168] Step 3: Bayesian method optimizes random forest model parameters;

[0169] In order to improve the prediction performance of the random forest model T, the present invention uses the Bayesian optimization method to adjust its key parameters and find the best parameter combination to optimize the model performance. The specific steps are as follows:

[0170] Step 3.1: Data division, specifically:

[0171] For each subset s k Divide into training set, validation set and test set according to the following proportions, as follows:

[0172] (1) Training set (X k,train ,Y k,train ): 70%, used to train decision tree T k and the random forest model T;

[0173] (2) Validation set (X k,val ,Y k,val ): 20%, used for decision tree T k and parameter adjustment and selection of the random forest model T;

[0174] (3) Test set (X k,test ,Y k,test ): 10%, used to evaluate the decision tree T k and the final performance of the random forest model T;

[0175] Step 3.2, parameter selection, specifically:

[0176] In the process of adjusting the random forest model T based on the Bayesian optimization method, the following key parameters need to be selected as optimization objects:

[0177] (1) The number of trees n tree : The total number of decision trees contained in the random forest; (2) The maximum number of features n feature : The maximum number of features considered by each split node; (3) Maximum depth n depth : The maximum depth of a single decision tree to control the complexity of the model; (4) The minimum number of sample splits n split : The minimum number of samples required when splitting a node to avoid over-splitting; (5) The minimum number of sample leaves n leaf : The minimum number of samples contained in the leaf node to control the termination condition of the tree;

[0178] Step 3.3: Construct the objective function, specifically:

[0179] The coefficient of determination R 2 Build a black box function for the target In the validation set (X k,val ,Y k,val ), R 2 The calculation formula is:

[0180]

[0181] Among them, Y k,val is the true target value of the validation set, Y′ k,val is the predicted value of the random forest model T, Y k,avg Yes k,val The average value of , N is the number of elements in the validation set.

[0182] Step 3.4: Construct a Gaussian process, specifically:

[0183] Let parameter vector Contains the following 5 parameters to be optimized:

[0184] (1) The number of trees n tree ; (2) Maximum number of features n feature ; (3) Maximum depth n depth ; (4) Minimum number of sample splits n split ; (5) Minimum number of sample leaves n leaf ;

[0185] Parameter vector Its parameter space is Ψ; the objective function The coefficient of determination R 2 , expressed as:

[0186]

[0187] By building a Gaussian process regression model For parameter vector And the objective function Fitting, the process of Bayesian optimization parameters is to maximize the predicted mean of the Gaussian process model To find the optimal parameter combination:

[0188]

[0189] Step 3.5, select the acquisition function, specifically:

[0190] The upper confidence bound (UCB) is selected as the acquisition function to balance the weight of parameter utilization and exploration. The UCB acquisition function is defined as:

[0191]

[0192] in, is the predicted mean of the Gaussian process regression model, is the prediction standard deviation of the Gaussian process regression model, and λ is the control and Balance weight parameters;

[0193] Step 3.6, repeat the iteration, specifically:

[0194] In the Bayesian optimization process, steps 3.4 (Gaussian process construction) and 3.5 (acquisition function selection) are repeated until one of the following stop conditions is met, and it is considered that the best parameter combination has been found;

[0195] The stopping conditions are as follows:

[0196] (1) Convergence judgment: If after multiple iterations, the objective function If the value of changes very little, that is, the improvement is lower than the set threshold, the optimization process is considered to have converged and the model performance cannot be significantly improved further. (2) Maximum number of iterations: Set the maximum number of iterations (such as 100, 200, etc.). When the maximum number of iterations is reached, the iteration stops and returns the currently considered optimal parameter combination. (3) Maximization of the objective function: Bayesian optimization continuously predicts the objective function through the Gaussian process regression model And through the acquisition function (such as UCB), new parameter combinations are continuously explored. If the value of the objective function tends to be stable after several rounds of iterations and the predicted value reaches the maximum value, it is considered that the optimal parameter combination has been found.

[0197] When the optimization process meets one of the above stop conditions, the current optimal parameter combination is returned. That is, the objective function The parameter set that reaches the maximum value can improve the performance of the random forest model.

[0198] Experimental and error comparison results

[0199] In order to verify the effectiveness of the present invention, we conducted detailed experiments and compared them with existing methods. The experimental settings are as follows: The experimental data comes from the Mie scattering lidar system of the Atmospheric Remote Sensing Center of Xi'an University of Technology. The lidar samples at a frequency of 40 MHz, collecting 2500 points each time, with a sampling interval of 15 minutes. The experiments were conducted in Xi'an (34.233°N, 104.003°E) in 2013, 2014 and 2017, and atmospheric aerosol observations were continuously carried out.

[0200] According to the characteristics of Mie scattering lidar signals, three sets of continuous observation datasets with different weather conditions and signal-to-noise ratios (SNR) were selected. The detailed information of these datasets is shown in Table 1.

[0201] Table 1: Three datasets used in the experiment

[0202] serial number weather Signal-to-Noise Ratio (SNR) Data Similarity A Cloudy and windy Low Low B Cloudy, no wind high high C No clouds, no wind middle middle

[0203] In order to evaluate the ability of the method of the present invention in predicting the aerosol extinction coefficient from Mie scattering lidar signals, it is compared with other common fitting models. These comparison models include linear regression model (LinearRegression, LR), K-nearest NeighborsRegression model (KNN), back propagation neural network model (Back Propagation Neural Network, BP), BP neural network model optimized by genetic algorithm (BPNeural Network Optimized byGenetic Algorithm, BPGA), and random forest model (RandomForest, RF).

[0204] Through these comparisons, the purpose is to analyze the relative advantages of the method of the present invention in predictive performance. The experimental results are shown in Tables 2, 3, and 4. The data in the tables are the means and standard deviations of multiple experimental results.

[0205] Table 2 Experimental results of each model using dataset A

[0206] method <![CDATA[R 2 ]]> MSE MAE Linear Regression Model (LR) 8.664e-01±6.104e-02 3.580e-04±2.154e-04 14.720e-03±3.874e-03 K-nearest neighbor regression model (KNN) 9.564e-01±4.076e-02 1.226e-04±1.311e-04 6.922e-03±3.550e-03 Back propagation neural network model (BP) 9.473e-01±4.782e-02 1.480e-04±1.541e-04 7.932e-03±3.442e-03 Back propagation neural network model optimized by genetic algorithm (BPGA) 9.547e-01±4.154e-02 1.252e-04±1.301e-04 7.680e-03±4.959e-03 Random Forest Model (RF) 9.635e-01±4.102e-02 1.046e-04±1.317e-04 6.218e-03±3.033e-03 Method of the present invention 9.641e-01±3.349e-02 1.029e-04±1.097e-04 6.132e-03±3.008e-03

[0207] Table 3 Experimental results of each model using dataset B

[0208] method <![CDATA[R 2 ]]> MSE MAE Linear Regression Model (LR) 6.487e-01±6.365e-02 20.893e-04±10.626e-04 4.062e-02±10.067e-03 K-nearest neighbor regression model (KNN) 8.692e-01±6.033e-02 10.058e-04±7.067e-04 2.281e-02±7.056e-03 Back propagation neural network model (BP) 8.897e-01±6.520e-02 9.710e-04±9.517e-04 2.165e-02±8.582e-03 Back propagation neural network model optimized by genetic algorithm (BPGA) 8.962e-01±6.277e-02 9.082e-04±8.858e-04 2.076e-02±7.996e-03 Random Forest Model (RF) 9.011e-01±6.503e-02 8.465e-04±7.893e-04 1.976e-02±8.476e-03 Method of the present invention 9.466e-01±2.251e-02 3.664e-04±2.001e-04 1.400e-02±3.089e-03

[0209] Table 4 Experimental results of each model using dataset C

[0210] method <![CDATA[R 2 ]]> MSE MAE Linear Regression Model (LR) 7.258e-01±9.920e-04 1.279e-03±7.107e-05 30.033e-03±10.512e-04 K-nearest neighbor regression model (KNN) 9.530e-01±70.234e-04 8.083e-05±3.735e-05 5.183e-03±11.951e-04 Back propagation neural network model (BP) 9.520e-01±20.723e-04 2.231e-04±2.076e-06 9.017e-03±2.075e-04 Back propagation neural network model optimized by genetic algorithm (BPGA) 9.489e-01±200.962e-04 238.124e-04±1.375e-06 10.037e-03±24.904e-04 Random Forest Model (RF) 9.640e-01±4.358e-04 2.807e-05±2.758e-06 3.935e-03±4.408e-04 Method of the present invention 9.748e-01±6.964e-04 2.445e-05±4.353e-06 3.657e-03±4.890e-04

[0211] From the experimental results, the proposed method shows the best performance in all three datasets, not only in R 2 , MSE and MAE indicators, and showed more stable performance in terms of standard deviation. 2The values ​​in the three experiments were 0.9641, 0.9466 and 0.9748, which were the highest among all the models, showing the strongest fitting ability. At the same time, the MSE values ​​were 1.029e-04, 3.664e-04 and 2.445e-05, which were the lowest among all the models, indicating that the prediction error was the smallest. In addition, the MAE value of the method of the present invention was also the lowest, which was 6.132e-03, 1.400e-02 and 3.657e-03, respectively, showing extremely high prediction accuracy.

[0212] Standard deviation analysis shows that the method of the present invention not only has advantages in performance, but also performs stably. For example, the standard deviation of R2 is 3.349e-02, 2.251e-02 and 6.964e-04 in Table 1, Table 2 and Table 3, respectively, which are smaller than the standard deviation of most other models, especially compared with the linear regression model (LR) with large fluctuations. At the same time, in the standard deviation of MSE and MAE, the method of the present invention also shows a smaller fluctuation range, such as the MSE standard deviation in Table 2 is only 2.001e-04, indicating the stability of its prediction error.

[0213] In contrast, the random forest model (RF) in R 2 , MSE and MAE indicators are closely followed by the method of the present invention, and the performance is quite excellent, but its standard deviation is slightly higher, especially in R 2 The standard deviation is 4.358e-04, showing a certain instability. The K-nearest neighbor regression model (KNN) and the back propagation neural network model (BPGA) optimized by genetic algorithm also performed well in some experiments, but their standard deviations are often large, especially in MSE and MAE, showing the instability of the prediction results.

[0214] The linear regression model (LR) performed the worst in all experiments, regardless of R 2 , MSE or MAE, their values ​​and standard deviations all show that the model is not suitable for the task, especially in Table 2, R 2 It is only 0.6487, and the standard deviation reaches 6.365e-02, indicating that the prediction performance of this model is not only poor but also highly volatile.

[0215] In summary, the method of the present invention performs well in all indicators, and its standard deviation is small, showing stronger stability. Compared with other models, the method of the present invention not only has significant advantages in prediction accuracy and error control, but also has outstanding results in the reliability of the results. Its prediction results are closest to the aerosol extinction coefficient obtained by the Klett method. Under the three atmospheric conditions of continuous observation of Mie scattering lidar signals, the method of the present invention can accurately predict the aerosol extinction coefficient.

[0216] In order to further illustrate the effectiveness of the method of the present invention, the experiment demonstrated the approximation effect of the aerosol extinction coefficient curve inverted by the linear regression model (LR), K-nearest neighbors regression model (KNN), back propagation neural network model (BP), BP neural network model optimized by genetic algorithm (BPGA), random forest model (RF), and the method of the present invention and the Klett method under three atmospheric conditions.

[0217] Figures 3 to 8 The Mie lidar signal changes over time, with the observed cloud height ranging from 2 to 4 km and the weather being cloudy and windy. Figure 8 It is shown that the aerosol extinction coefficient distribution predicted by the method of the present invention is very close to the result inverted by the Klett method. Figure 7 The random forest (RF) model in also produced similar results, but some additional contour details appeared in its predictions. In addition, the profile predictions of RF in the cloud range were 1.5 to 2 km higher than those of the proposed method. Figure 3 Linear regression (LR), Figure 4 KNN, Figure 5 BP and Figure 6 The BPGA showed significant errors compared with the Klett method.

[0218] Figures 9 to 14 This reflects the cloudy and windless weather conditions. Fig.10 KNN, Fig.13 RF and Fig.14 The aerosol extinction coefficient distribution predicted by the method of the present invention is similar to the result of Klett method. Fig.10 The prediction profile of the KNN model shows a line at 1.5 to 2 km that is inconsistent with the vertical pattern of atmospheric extinction, which is an inaccurate prediction result. At the same time, at 3 km, Fig.14 The prediction curve of the method of the present invention shows an opposite trend compared to Klett, but the two are almost completely overlapped below 1.8 kilometers.

[0219] like Figures 15 to 20 As shown, under cloudless and windless weather conditions, Fig.17 BP in Fig. 20 BPGA in Fig.19 RF and Fig. 20 The aerosol extinction coefficient prediction of the method of the present invention is in good agreement with Klett's curve. Fig.16 The KNN model in shows prediction discontinuities at 6.5 km, mainly concentrated in areas with high signal noise. Fig.19 The RF model in shows an inflection point at 2 km and a discontinuity at 4.8 km, which is quite different from the Klett profile. Fig. 20 The predicted profiles of the method of the present invention are more consistent with those of the Klett method in high-noise and low-noise areas, and overlap often occurs.

[0220] In summary, from Figures 3 to 20 It can be seen that the method of the present invention is closest to the Klett method in predicting the extinction coefficient, followed by the RF and KNN models. Therefore, the method of the present invention has the largest R under the three weather conditions. 2 The lowest MSE and MAE values ​​can accurately predict the aerosol extinction coefficient.

[0221] Example 1

[0222] The method of inverting aerosol extinction coefficient by Mie scattering lidar based on random forest includes the following steps: step 1, construction and standardization of Mie scattering lidar data set; step 2: the process of predicting aerosol extinction coefficient by random forest; step 3, Bayesian method optimization of random forest model parameters.

[0223] Example 2

[0224] The method for inverting aerosol extinction coefficient by Mie scattering lidar based on random forest includes the following steps: step 1, construction and standardization of Mie scattering lidar data set; step 1.1: construction of Mie scattering lidar data set; step 1.2, standardization of Mie scattering lidar data set.

[0225] Step 2: The process of predicting aerosol extinction coefficient using random forest; Step 3: Bayesian method to optimize random forest model parameters.

[0226] Example 3

[0227] The method for inverting aerosol extinction coefficient by Mie scattering lidar based on random forest includes the following steps: Step 1, Mie scattering lidar data set construction and standardization processing; Step 1.1: Mie scattering lidar data set construction; Step 1.1 is specifically as follows: First, define multiple characteristic parameters of the Mie scattering lidar data set. Finally, construct the Mie scattering lidar data set S, including the input data set and the target data set:

[0228] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0229] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0230] Step 1.2: Standardization of Mie scattering lidar dataset.

[0231] Step 2: The process of predicting aerosol extinction coefficient using random forest; Step 3: Bayesian method to optimize random forest model parameters.

[0232] Example 4

[0233] The method for inverting aerosol extinction coefficient by Mie scattering lidar based on random forest includes the following steps: Step 1, Mie scattering lidar data set construction and standardization processing; Step 1.1: Mie scattering lidar data set construction; Step 1.1 specifically includes:

[0234] First, the Mie scattering lidar dataset consists of multiple feature parameters, which are defined as follows:

[0235] 1. Mie scattering lidar echo signal strength v i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The Mie scattering lidar echo signal strength value at , where:

[0236] i=1,2,…,n, represents the number of detection time points;

[0237] j=1,2,…,m, represents the number of detection distance points;

[0238] The collection form is v i = {v i,1 ,v i,2 ,…,v i,m}, that is, at the detection time t i The set of Mie scattering lidar echo signal strength values ​​at all detection distances;

[0239] 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric pressure value at the location, in the collective form of p i={p i,1 ,p i,2 ,…,p i,m};

[0240] 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at the location; the collection form is c i ={c i,1 ,c i,2 ,…,c i,m};

[0241] 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at; the set form is t i ={t i,1 ,t i,2 ,…,t i,m};

[0242] 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,r i,m};

[0243] Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j (j=1,2,…,m), specifically expressed as:

[0244] x i ={x i,1 ,x i,2 ,...,x i,m}

[0245] Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows:

[0246] x i,j = {v i,j ,p i,j ,c i,j ,r i,j,t i,j}

[0247] Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time;

[0248] Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is:

[0249] x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1},{v i,2 ,p i,2 ,c i,2 ,r i,2 ,t i,2},…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m}};

[0250] The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set:

[0251] y i ={y i,1 ,y i,2 ,…,y i,m}

[0252] Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset:

[0253] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0254] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0255] Step 1.2: Standardization of Mie scattering lidar dataset.

[0256] Step 2: The process of predicting aerosol extinction coefficient using random forest; Step 3: Bayesian method to optimize random forest model parameters.

[0257] Example 5

[0258] A method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forests includes the following steps: Step 1, construction and standardization of Mie scattering lidar dataset; Step 1 is specifically as follows: Step 1.1: construction of Mie scattering lidar dataset;

[0259] Step 1.1 is as follows:

[0260] First, the Mie scattering lidar dataset consists of multiple feature parameters, which are defined as follows:

[0261] 1. Mie scattering lidar echo signal strength v i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The Mie scattering lidar echo signal strength value at , where:

[0262] i=1,2,…,n, represents the number of detection time points;

[0263] j=1,2,…,m, represents the number of detection distance points;

[0264] The collection form is v i = {v i,1 ,v i,2 ,…,v i,m}, that is, at the detection time t i The set of Mie scattering lidar echo signal strength values ​​at all detection distances;

[0265] 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance rj The atmospheric pressure value at the location, in the collective form of p i ={p i,1 ,p i,2 ,…,p i,m};

[0266] 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at the location; the collection form is c i ={c i,1 ,c i,2 ,…,c i,m};

[0267] 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at; the set form is t i ={t i,1 ,t i,2 ,…,t i,m};

[0268] 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,r i,m};

[0269] Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j (j=1,2,…,m), specifically expressed as:

[0270] x i ={x i,1 ,x i,2 ,...,x i,m}

[0271] Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows:

[0272] x i,j = {v i,j ,pi,j ,c i,j ,r i,j ,t i,j}

[0273] Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time;

[0274] Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is:

[0275] x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1},{v i,2 ,p i,2 ,c i,2 ,r i,2 ,t i,2},…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m}};

[0276] The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set:

[0277] y i ={y i,1 ,y i,2 ,…,y i,m}

[0278] Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset:

[0279] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0280] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0281] Step 1.2: Standardization of Mie scattering lidar dataset.

[0282] Step 1.2 is as follows:

[0283] The input data set x of the Mie scattering lidar data set S is standardized by the standardization formula. The expression of the processing process is as follows:

[0284]

[0285] Among them, x i,j Indicates that the data is concentrated at the detection time t i and detection distance r j The original input data value at, mean(x) represents the mean value of the input data set x, std(x) represents the standard deviation of the input data set x, x i ' ,j Indicates the data value after normalization.

[0286] Step 2: The process of predicting aerosol extinction coefficient using random forest; Step 3: Bayesian method to optimize random forest model parameters.

[0287] Example 6

[0288] A method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forests includes the following steps: Step 1, construction and standardization of Mie scattering lidar dataset; Step 1 is specifically as follows: Step 1.1: construction of Mie scattering lidar dataset;

[0289] Step 1.1 is as follows: First, the Mie scattering lidar dataset consists of multiple feature parameters, which are specifically defined as follows:

[0290] 1. Mie scattering lidar echo signal strength v i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance rj The Mie scattering lidar echo signal strength value at , where:

[0291] i=1,2,…,n, represents the number of detection time points;

[0292] j=1,2,…,m, represents the number of detection distance points;

[0293] The collection form is v i = {v i,1 ,v i,2 ,…,v i,m}, that is, at the detection time t i The set of Mie scattering lidar echo signal strength values ​​at all detection distances;

[0294] 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric pressure value at the location, in the collective form of p i ={p i,1 ,p i,2 ,…,p i,m};

[0295] 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at the location; the collection form is c i ={c i,1 ,c i,2 ,…,c i,m};

[0296] 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at; the set form is t i ={t i,1 ,t i,2 ,…,t i,m};

[0297] 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,r i,m};

[0298] Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j (j=1,2,…,m), specifically expressed as:

[0299] x i ={x i,1 ,x i,2 ,...,x i,m}

[0300] Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows:

[0301] x i,j = {v i,j ,p i,j ,c i,j ,r i,j ,t i,j}

[0302] Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time;

[0303] Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is:

[0304] x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1},{v i,2 ,p i,2 ,c i,2,r i,2 ,t i,2},…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m}};

[0305] The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set:

[0306] y i ={y i,1 ,y i,2 ,…,y i,m}

[0307] Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset:

[0308] S={(x1,y1),(x2,y2),...,(x n ,y n )}

[0309] Where x={x1,x2,...,x n} is the input data set, y={y1,y2,...,y n} is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

[0310] Step 1.2: Standardization of Mie scattering lidar dataset. Step 1.2 is as follows: standardize the input dataset x of Mie scattering lidar dataset S by using the standardization formula. The expression of the processing process is as follows:

[0311]

[0312] Among them, x i,j Indicates that the data is concentrated at the detection time t i and detection distance r j The original input data value at, mean(x) represents the mean value of the input data set x, std(x) represents the standard deviation of the input data set x, x i ' ,j Indicates the data value after normalization.

[0313] Step 2: The process of predicting aerosol extinction coefficient by random forest; Step 2 is specifically as follows: Step 2.1: Generate data subset; Step 2.2: Generate decision tree; Step 2.3: Generate random forest model; Step 2.4: Generate prediction result.

[0314] Step 3: Bayesian method is used to optimize the parameters of the random forest model.

Claims

1. A method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest, characterized in that: The following steps are involved: Step 1: Construction and standardization of Mie scattering lidar dataset; Step 2: The process of predicting aerosol extinction coefficient by random forest; Step 3: Bayesian method is used to optimize the parameters of the random forest model.

2. The method for inverting aerosol extinction coefficient based on Mie scattering lidar based on random forest according to claim 1, characterized in that: Step 1 is as follows: Step 1.1: Mie scattering lidar dataset construction; Step 1.2: Standardization of Mie scattering lidar dataset.

3. The method for inverting aerosol extinction coefficient based on Mie scattering lidar based on random forest according to claim 2, characterized in that: Step 1.1 is as follows: First, the Mie scattering lidar dataset consists of multiple feature parameters, which are defined as follows:

1. Mie scattering lidar echo signal strength v i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The Mie scattering lidar echo signal strength value at , where: i=1,2,…,n, represents the number of detection time points; j=1,2,…,m, represents the number of detection distance points; The collection form is v i = {v i,1 ,v i,2 ,…,v i,m }, that is, at the detection time t i The set of Mie scattering lidar echo signal strength values ​​at all detection distances; 2. Atmospheric pressure p i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric pressure value at the location, in the collective form of p i ={p i,1 ,p i,2 ,…,p i,m }; 3. Atmospheric temperature c i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The atmospheric temperature value at the location; the collection form is c i ={c i,1 ,c i,2 ,…,c i,m }; 4. Detection time t i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection time value at; the set form is t i ={t i,1 ,t i,2 ,…,t i,m }; 5. Detection distance r i,j , which means at the i-th detection time t i Under the condition that at the jth detection distance r j The detection distance value at is used to calibrate the spatial position of each data point, and the collection form is r i = {r i,1 ,r i,2 ,…,r i,m }; Among them, at each detection time t i Next, construct a feature data set x i , the set contains m feature vectors, each feature vector corresponds to a detection distance r j , j = 1, 2, ..., m, which can be specifically expressed as: x i ={x i,1 ,x i,2 ,...,x i,m } Let x i,j Indicates that at the detection time t i and detection distance r j The set of eigenvalues ​​at is defined as follows: x i,j ={v i,j ,p i,j ,c i,j ,r i,j ,t i,j } Among them, v i,j is the detection time t i and detection distance r j Mie scattering lidar echo signal intensity at p i,j is the detection time t i and detection distance r j The atmospheric pressure at i,j is the detection time t i and detection distance r j The atmospheric temperature at r i,j is the detection time t i The jth detection distance point at time t i,j is the detection time t i When the jth detection distance r j The point in time; Therefore, the set x i At a specific detection time t i All detection distances r under j The complete set of feature vectors is: x i ={{v i,1 ,p i,1 ,c i,1 ,r i,1 ,t i,1 },{v i,2 ,p i,2 ,c i,2 ,r i,2 ,t i,2 },…,{v i,m ,p i,m ,c i,m ,r i,m ,t i,m }}; The Klett method is used to calculate each detection time t i The Mie scattering lidar echo signal strength value set v under i Perform inversion to obtain the corresponding aerosol extinction coefficient set: and i ={and i,1 ,and i,2 ,…,and i,m } Finally, the Mie scattering lidar dataset S is constructed, which includes the input dataset and the target dataset: S={(x1,y1),(x2,y2),...,(x n ,y n )} Where x={x1,x2,...,x n } is the input data set, y={y1,y2,...,y n } is the target dataset, i.e., the aerosol extinction coefficient, and the Mie scattering lidar dataset S is used to train and test the random forest inversion model.

4. The method for inverting aerosol extinction coefficient based on Mie scattering lidar based on random forest according to claim 3 is characterized in that: Step 1.2 is as follows: The input data set x of the Mie scattering lidar data set S is standardized by the standardization formula. The expression of the processing process is as follows: Among them, x i,j Indicates that the data is concentrated at the detection time t i and detection distance r j The original input data value at, mean(x) represents the mean value of the input data set x, std(x) represents the standard deviation of the input data set x, x i ' ,j Indicates the data value after normalization.

5. The method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest according to claim 3, characterized in that: Step 2 is as follows: Step 2.1: Generate a data subset, specifically: From the Mie scattering lidar dataset S constructed in step 1, use the bagging method to perform random sampling with replacement. The specific steps are: (1) Randomly draw n times to obtain a Mie scattering lidar data subset containing n samples, called s; (2) Repeat this process k times to generate k Mie scattering lidar data subsets, denoted as s1, s2, ..., s k ; (3)s k represents the kth data subset, k = 1, 2, ..., K, each data subset contains n sample data, s k The form is: in, represents the input feature vector of the jth sample in the kth data subset, represents the jth target value in the kth data subset, i.e., the aerosol extinction coefficient; Step 2.2: Decision tree generation, specifically: For each Mie scattering lidar data subset s k , k=1,2,…,K, perform the following operations: (1) Feature and label extraction: From the subset s k Extract the feature matrix X k Extract and label vector Y k Extract, provide input data for decision tree generation, where X k Contains Mie scattering lidar signal strength, air pressure, temperature, detection distance and detection time, etc. k represents the corresponding aerosol extinction coefficient; that is, Medium, Features is extracted into the matrix Xk, Extracted to Y k ; (2) Decision tree construction: The feature matrix X k and label vector Y k As input, the decision tree T is generated using the classification and regression tree algorithm k ; The CART algorithm recursively divides the subset s k , using mean square error as the splitting criterion to find the best splitting features and splitting points, thereby realizing the construction of the tree structure; (3) Repeat steps (1) and (2) K times for subsets s1, s2, ..., s K Generate K independent decision trees T1, T2, ..., T K ; Step 2.3: Random forest model generation, specifically: The trained K decision trees T1, T2, ..., T K , combined into a random forest model T, defined as T={T1,T2,...,T K }; Step 2.4: Generate prediction results.

6. The method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest according to claim 5, characterized in that: Step 2.4 is as follows: (1) Single decision tree prediction: For a given new input data, it is input into each decision tree T in the random forest in turn. k In this case, we get K independent prediction results y k , where y k Represents the decision tree T k The predicted aerosol extinction coefficient for the input data, k = 1, 2, …, K; (2) Result integration: The prediction results of all decision trees {y1,y2,...,y K } Perform simple averaging to obtain the final prediction result of the random forest model T Right now: This value is used as the final prediction output of the random forest model for the aerosol extinction coefficient of the input data.

7. The method for inverting aerosol extinction coefficient based on Mie scattering lidar based on random forest according to claim 6, characterized in that: Step 3 is as follows: Step 3.1: Data division, specifically: For each subset s k Divide into training set, validation set and test set according to the following proportions, as follows: (1) Training set (X k,train ,Y k,train ): 70%, used to train decision tree T k and the random forest model T; (2) Validation set (X k,val ,Y k,val ): 20%, used for decision tree T k and parameter adjustment and selection of the random forest model T; (3) Test set (X k,test ,Y k,test ): 10%, used to evaluate the decision tree T k and the final performance of the random forest model T; Step 3.2, parameter selection, specifically: In the process of adjusting the random forest model T based on the Bayesian optimization method, the following key parameters need to be selected as optimization objects: (1) The number of trees n tree : The total number of decision trees contained in the random forest; (2) Maximum number of features n feature : The maximum number of features considered for each splitting node; (3) Maximum depth n depth : The maximum depth of a single decision tree to control the complexity of the model; (4) Minimum number of sample splits n split : The minimum number of samples required when splitting a node to avoid over-splitting; (5) Minimum number of sample leaves n leaf : The minimum number of samples contained in the leaf node to control the termination condition of the tree; Step 3.3: Construct the objective function, specifically: The coefficient of determination R 2 Build a black box function for the target In the validation set (X k,val ,Y k,val ), R 2 The calculation formula is: Among them, Y k,val is the true target value of the validation set, Y′ k,val is the predicted value of the random forest model T, Y k,avg Yes k,val The average value of , N is the number of elements in the validation set; Step 3.4: Construct a Gaussian process, specifically: Let parameter vector Contains the following 5 parameters to be optimized: (1) The number of trees n tree ; (2) Maximum number of features n feature ; (3) Maximum depth n depth ; (4) Minimum number of sample splits n split ; (5) Minimum number of sample leaves n leaf ; Parameter vector Its parameter space is Ψ; the objective function The coefficient of determination R 2 , expressed as: By building a Gaussian process regression model For parameter vector And the objective function Fitting, the process of Bayesian optimization parameters is to maximize the predicted mean of the Gaussian process model To find the optimal parameter combination: Step 3.5, select the acquisition function, specifically: The upper confidence bound is selected as the acquisition function to balance the weight of parameter utilization and exploration. The UCB acquisition function is defined as: in, is the predicted mean of the Gaussian process regression model, is the prediction standard deviation of the Gaussian process regression model, and λ is the control and Balance weight parameters; Step 3.6, repeat the iteration, specifically: In the Bayesian optimization process, steps 3.4 and 3.5 are repeated until one of the following stopping conditions is met, and it is considered that the best parameter combination has been found; When the optimization process meets one of the above stop conditions, the current optimal parameter combination is returned. That is, the objective function The parameter set that reaches the maximum value can improve the performance of the random forest model.

8. The method for inverting aerosol extinction coefficient using Mie scattering lidar based on random forest according to claim 7, characterized in that: In step 3.6, the stopping conditions are as follows: (1) Convergence judgment: If after multiple iterations, the objective function If the value of changes very little, that is, the improvement is lower than the set threshold, it is considered that the optimization process has converged and the model performance cannot be significantly improved further; (2) Maximum number of iterations: Set the maximum number of iterations; when the maximum number of iterations is reached, the iteration stops and the optimal parameter combination is returned. (3) Maximization of the objective function: Bayesian optimization continuously predicts the objective function through the Gaussian process regression model And through the acquisition function; continuously explore new parameter combinations; if the value of the objective function tends to be stable after several rounds of iterations and the predicted value reaches the maximum value, it is considered that the optimal parameter combination has been found.

Images