A Spectral Weighted Prediction Method Based on Weight Coefficient
Through the spectral weighted prediction method based on the weight coefficient, the preferred model is selected and its weight coefficient is calculated, which solves the problem of low stability and accuracy of the spectral data of the portable near-infrared spectrometer, and improves the stability and accuracy of the prediction results.
Patent Information
- Application Number
- CN202210624408.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-02
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-02
AI Technical Summary
The spectral data collected by the portable near-infrared spectrometer have poor stability and low accuracy, resulting in unstable prediction results and low accuracy.
Using the spectral weighted prediction method based on weight coefficients, modeling through different spectral preprocessing methods and partial least squares method, the root mean square error and model determination coefficient of multiple spectral models are obtained, the preferred model is selected and its weight coefficient is calculated, and finally the Weighted averages to obtain the final predicted value.
Improve the prediction stability and accuracy of the portable near-infrared spectrometer and reduce prediction errors.
Smart Images

Figure CN115015163B_ABST
Abstract
Description
Technical Field
[0001] The invention mainly relates to the field of infrared spectroscopy, and more particularly to a spectral weighted prediction method based on weight coefficients. Background Art
[0002] With the development of micro-electromechanical technology, in recent years, portable near-infrared spectrometers have been widely used in various fields such as food, beverages, medicine, coal, etc. Compared with traditional large-scale Fourier transform spectroscopy analysis systems, they have the advantages of simple structure, low cost, easy to carry, and real-time visible results. At present, they have become a popular product in the field of spectroscopy. However, portable near-infrared spectrometers are easily affected by light sources, detectors, usage methods, environmental conditions, etc., and the collected spectral data has poor stability and low precision, which in turn causes problems such as unstable prediction results and low prediction accuracy.
[0003] In order to solve the above problems and improve the stability and accuracy of portable near-infrared spectroscopy analysis results, the industry currently adopts a method of optimizing the spectral data modeling work accordingly, mainly focusing on data source screening, preprocessing algorithm optimization, model screening algorithm optimization and other single model modeling optimization work based on partial least squares (PLS). This type of modeling algorithm is mainly suitable for high-precision Fourier near-infrared spectrometer data. For portable near-infrared spectrometers with low hardware resolution, these algorithms have poor applicability and play a small role in improving the prediction effect. Summary of the invention
[0004] The technical problems to be solved by the present invention are:
[0005] The spectral data collected by the portable near-infrared spectrometer has poor stability and low precision, which leads to unstable prediction results and low prediction accuracy.
[0006] The technical solution adopted by the present invention to solve the above problems is as follows:
[0007] A spectral weighted prediction method based on weight coefficients comprises the following steps:
[0008] Step 1: Obtain the original spectral data, and use partial least squares modeling to model the original spectral data using different spectral preprocessing methods to obtain a corresponding number of spectral models;
[0009] Step 2: Obtain the root mean square error RMSECV and model determination coefficient R of each spectral model 2 ;
[0010] Step 3: According to the root mean square error RMSECV and model determination coefficient R 2 Select multiple preferred models from the spectral models obtained in step 1;
[0011] Step 4: Obtain the weight coefficient of each optimal model;
[0012] Step 5: Predict the spectrum to be measured based on the selected optimal model, and perform weighted averaging on the prediction results of each optimal model according to the weight coefficient of the optimal model to obtain the final prediction value.
[0013] Furthermore, in step 1, the different spectral preprocessing methods include Gaussian smoothing, Gaussian derivative, SG smoothing, detrending, vector normalization SNV and LOG transformation.
[0014] Furthermore, in step 2, the root mean square error RMSECV and model determination coefficient R of each spectral model are obtained. 2 The methods include:
[0015] Take one sample X at a time from the M spectral data in the original sample i , use the remaining M-1 spectral sample data to build a model, and use the built model to predict the sample X that was taken out i The calibration value y i , get the predicted values of all samples And calculate the calibration value y of all samples i The average value y i,ave , according to the sample prediction value obtained And the calibration value y i The average value y i,ave Calculate the root mean square error RMSECV and model determination coefficient R of all models 2 .
[0016] Furthermore, the specific implementation method of step 3 is as follows:
[0017] According to the calculated root mean square error RMSECV and model determination coefficient R 2 Select 4 preferred models:
[0018] Preferred model T1: Based on the RMESCV direct screening method, the spectral model with the smallest RMESCV value among the spectral models is selected as the preferred model T1. If multiple models have the same minimum RMESCV value, one of them is randomly selected;
[0019] Preferred model T2: Based on R 2 Direct screening method, select R in the spectral model 2 The spectral model with the largest value is selected as the preferred model T2. If multiple models have the same maximum R 2 value, one of them is randomly selected;
[0020] Preferred model T3: Based on RMESCV / R 2Hybrid screening method: set the minimum RMESCV value of the spectral model to S, retain the spectral models with RMESCV values less than 1.1S, eliminate the rest of the models, and then select R from the retained models. 2 The spectral model with the largest value is selected as the preferred model T3. If multiple models have the same maximum R 2 value, one of them is randomly selected;
[0021] Preferred model T4: Based on R 2 / RMESCV hybrid screening method, setting the R 2 The maximum value is L, and R is retained 2 The spectral model with a value greater than 0.9L was eliminated, and the remaining models were then selected from the remaining models with the smallest RMESCV value as the preferred model T4. If multiple models had the same maximum RMESCV value, one of them was randomly selected;
[0022] The root mean square error RMESCV values of the optimal models T1, T2, T3, and T4 are denoted as S1, S, S3, and S4, respectively, and the model determination coefficient R 2 The values are recorded as L1, L2, L3, and L4 respectively.
[0023] Furthermore, the calculation method for obtaining the weight coefficient of each preferred model described in step 4 is:
[0024]
[0025] in They are the weight coefficients of the preferred model T1, the preferred model T2, the preferred model T3, and the preferred model T4 respectively.
[0026] Furthermore, the method for obtaining the final prediction value described in step 5 is as follows: the spectral data of the spectrum to be measured is set as H, which is used as the input of each priority model respectively, and the corresponding preprocessing of each preferred model is performed, and the preprocessed spectral data H1, H2, H3, H4 are multiplied by the linear fitting parameter Beta value of each preferred model to obtain the prediction result of each preferred model, and the prediction result of each preferred model is combined with its weight coefficient for weighted averaging, and finally the prediction result F of the integrated model is generated, which is the final prediction value of the spectrum to be measured; the specific calculation formula is:
[0027]
[0028] Among them, Beta1, Beta2, Beta3, and Beta4 are the linear fitting parameters of the optimal models T1, T2, T3, and T4 respectively.
[0029] Beneficial effects of the present invention:
[0030] Based on the PLS algorithm modeling, the present invention proposes a spectral weighted prediction method based on weight coefficient, selects several relatively stable and highly accurate spectral models, obtains the weight coefficients of the above spectral models for weighted prediction to improve the overall performance of the portable near-infrared spectrometer, thereby improving the prediction stability and accuracy of the portable near-infrared spectrometer and reducing the prediction error. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 The figure is a schematic diagram of the method steps of a spectral weighted prediction method based on weight coefficients described in the present invention. DETAILED DESCRIPTION
[0032] The present invention provides a spectral weighted prediction method based on weight coefficients, such as Figure 1 As shown, multiple preferred models are selected according to the root mean square error of the established spectral model and the model determination coefficient, the weight coefficient of each preferred model is calculated, the spectrum to be measured is preprocessed corresponding to the preferred model, the preferred model is used to predict the model to be measured to obtain multiple prediction results, and the predicted value of each preferred model is weighted averaged according to the weight coefficient to obtain the final predicted value of the spectrum to be measured.
[0033] Example:
[0034] Get the original spectral data, and perform data modeling on the original spectrum; Near infrared spectral data modeling mainly includes two parts, namely preprocessing and data linear fitting. The preprocessing method used in the present embodiment includes Gaussian smoothing, Gaussian derivative, SG smoothing, detrending, vector normalization SNV and LOG transformation, wherein, Gaussian smoothing half window width ∈ [1, 2, ..., 12], Gaussian derivative polynomial order ∈ [2, 4, 6], SG smoothing half window width ∈ [1, 2, ..., 12], polynomial order ∈ [1, 2, ..., 5]), detrending order ∈ [1, 2, ..., 4], vector normalization SNV and LOG changes, a total of 12 + 3 + 12 × 5 + 4 + 2 = 81 kinds. The data linear fitting method used is partial least squares, and the principal component interval ∈ [1, 2, ..., 12]. Based on the above combination, the number of modeling totals 81 × 12 = 972, and the spectral data and linear fitting parameter values after each model preprocessing are recorded respectively.
[0035] Obtain the root mean square error RMSECV and model determination coefficient R of each spectral model 2 ; If there are M original samples, then there are M corresponding original spectral data, and each time a sample X is taken out i (i-th sample, i = 1, 2, ..., M), use the remaining M-1 spectral samples to build a model, and use the built model to predict the sample X i The calibration value y i , get the predicted value of the sample At the same time, the calibration value y is calculated i The average value y i,ave , and then calculate the root mean square error RMSECV and model determination coefficient R for all the predicted values generated 2 .
[0036] Calculate the root mean square error RMSECV of all models. The root mean square error indicator actually reflects the deviation relationship between the model calibration value and the predicted value. The smaller the root mean square error, the smaller the deviation between the calibration value and the predicted value, and the better the model effect. The calculation method of the root mean square error RMSECV is:
[0037]
[0038] Calculate the model determination coefficient R for all models 2 The model determination coefficient reflects the percentage of the dependent variable's variable value controlled by the independent variable. The higher the percentage of the dependent variable's variable value controlled by the independent variable, that is, the higher the model coefficient, the better the model prediction effect. The model determination coefficient R 2 The calculation method is:
[0039]
[0040] Get the root mean square error RMSECV and model determination coefficient R of all models 2 After that, according to RMESCV and R 2 Select 4 preferred models. The specific selection method is as follows:
[0041] Preferred model T1: Based on the RMESCV direct screening method, the spectral model with the smallest RMESCV value among the spectral models is selected as the preferred model T1. If multiple models have the same minimum RMESCV value, one of them is randomly selected;
[0042] Preferred model T2: Based on R 2 Direct screening method, select R in the spectral model 2 The spectral model with the largest value is selected as the preferred model T2. If multiple models have the same maximum R 2 value, one of them is randomly selected;
[0043] Preferred model T3: Based on RMESCV / R 2 Hybrid screening method: set the minimum RMESCV value of the spectral model to S, retain the spectral models with RMESCV values less than 1.1S, eliminate the rest of the models, and then select R from the retained models. 2 The spectral model with the largest value is selected as the preferred model T3. If multiple models have the same maximum R 2 value, one of them is randomly selected;
[0044] Preferred model T4: Based on R 2 / RMESCV hybrid screening method, setting the R 2 The maximum value is L, and R is retained 2 The spectral model with a value greater than 0.9L was eliminated, and the remaining models were then selected from the remaining models with the smallest RMESCV value as the preferred model T4. If multiple models had the same maximum RMESCV value, one of them was randomly selected;
[0045] The root mean square error RMESCV values of the optimal models T1, T2, T3, and T4 are denoted as S1, S, S3, and S4, respectively, and the model determination coefficient R 2 The values are recorded as L1, L2, L3, and L4 respectively.
[0046] Root mean square error RMESCV and model determination coefficient R 2 The two indicators represent the performance of each spectral model under different combinations of preprocessing methods and modeling methods from different dimensions. When calculating the weight coefficient of the preferred model, both indicators are introduced into the calculation to maximize the stability and accuracy of the model. At the same time, when characterizing the performance of the spectral model, the smaller the RMESCV value, the better. That is, the preferred model with a smaller RMESCV value should have a higher weight coefficient, and R 2 The larger the value, the better, that is, R 2 The larger the value of the preferred spectral model, the higher the weight coefficient should be, and the calculation formula of the weight coefficient is obtained:
[0047]
[0048] in are the weight coefficients of the preferred models T1, T2, T3, and T4 respectively.
[0049] Set the original spectrum data of the spectrum to be measured as H, perform preprocessing corresponding to each preferred model, multiply the preprocessed spectrum data H1, H2, H3, H4 with the linear fitting parameter Beta value of each preferred model to obtain the prediction results of each preferred model, and combine the prediction results of each preferred model with its weight coefficient for weighted average, and finally generate the prediction result F of the integrated model, which is the final prediction value of the spectrum to be measured. The specific calculation formula is:
[0050] .
Claims
1. A spectral weighted prediction method based on weight coefficients, It is characterized in that The following steps are involved: Step 1: Obtain the original spectral data, and use partial least squares modeling to model the original spectral data using different spectral preprocessing methods to obtain a corresponding number of spectral models; Step 2: Obtain the root mean square error RMSECV and model determination coefficient R of each spectral model 2 ; Step 3: According to the root mean square error RMSECV and model determination coefficient R 2 Select multiple preferred models from the spectral models obtained in step 1; the specific implementation method is as follows: According to the calculated root mean square error RMSECV and model determination coefficient R 2 Select 4 preferred models: Preferred Model T 1 :Based on the RMESCV direct screening method, the spectral model with the smallest RMESCV value among the spectral models is selected as the preferred model T 1 , if multiple models have the same minimum RMESCV value, one of them is randomly selected; Preferred Model T 2 : Based on R 2 Direct screening method, select R in the spectral model 2 The spectral model with the largest value is selected as the preferred model T 2 , if multiple models have the same maximum R 2 value, one of them is randomly selected; Preferred Model T 3 :Based on RMESCV / R 2 Hybrid screening method: set the minimum RMESCV value of the spectral model to S, retain the spectral models with RMESCV values less than 1.1S, eliminate the rest of the models, and then select R from the retained models. 2 The spectral model with the largest value is selected as the preferred model T 3 , if multiple models have the same maximum R 2 value, one of them is randomly selected; Preferred Model T 4 : Based on R 2 / RMESCV hybrid screening method, setting the R 2 The maximum value is L, and R is retained 2 The spectral model with a value greater than 0.9L was removed, and the remaining models were then selected as the preferred model T from the remaining models with the smallest RMESCV value. 4 , if multiple models have the same minimum RMESCV value, one of them is randomly selected; The preferred model T 1 、T 2 、T 3 、T 4 The root mean square error RMESCV values are denoted as S 1 , S 2 , S 3 , S 4 , model determination coefficient R 2 The values are denoted as L 1 , L 2 , L 3 , L 4 ; Step 4: Obtain the weight coefficient of each optimal model: in The preferred model T 1 , Optimal Model T 2 , Optimal Model T 3 , Optimal Model T 4 The weight coefficient of Step 5: Predict the spectrum to be measured based on the selected optimal model, and perform weighted averaging on the prediction results of each optimal model according to the weight coefficient of the optimal model to obtain the final prediction value.
2. A spectral weighted prediction method based on weight coefficient according to claim 1, It is characterized in that In step 1, the different spectral preprocessing methods include Gaussian smoothing, Gaussian derivative, SG smoothing, detrending, vector normalization SNV and LOG transformation.
3. A spectral weighted prediction method based on weight coefficient according to claim 1, It is characterized in that In step 2, the root mean square error RMSECV and model determination coefficient R of each spectral model are obtained. 2 The methods include: Take one sample X at a time from the M spectral data in the original sample i , use the remaining M-1 spectral sample data to build a model, and use the built model to predict the sample X that was taken out i The calibration value y i , get the predicted values of all samples And calculate the calibration value y of all samples i The average value y i,ave , according to the sample prediction value obtained And the calibration value y i The average value y i,ave Calculate the root mean square error RMSECV and model determination coefficient R of all models 2 .
4. A spectral weighted prediction method based on weight coefficients according to any one of claims 1 to 3, It is characterized in that The method for obtaining the final prediction value described in step 5 is as follows: set the spectral data of the spectrum to be measured as H, use it as the input of each preferred model, perform preprocessing corresponding to each preferred model, and convert the preprocessed spectral data H into 1 , H 2 , H 3 , H 4 The prediction results of each preferred model are obtained by multiplying the linear fitting parameter Beta value of each preferred model, and the prediction results of each preferred model are combined with their weight coefficients for weighted average to finally generate the prediction result F of the integrated model, which is the final prediction value of the spectrum to be measured; the specific calculation formula is: Among them, Beta 1 , Beta 2 , Beta 3 , Beta 4 The preferred model T 1 、T 2 、T 3 、T 4 The linear fitting parameters of .
