Variable selection method of near infrared spectrum, chemical component detection method and computer equipment

Abstract

The invention belongs to the technical field of near infrared spectroscopy, and particularly relates to a variable selection method of near infrared spectroscopy, a chemical component detection method and computer equipment. According to the method, the extreme weight is punished, so that the situation that a certain variable has too high weight is avoided, the valuable low-weight variable combination is influenced and eliminated, the variable combination effect is considered, and the accuracy and stability of the prediction model are remarkably improved. The technical problem that the model prediction performance is low due to the fact that an existing variable selection method endows some variables with too high weights and ignores high correlation between continuous variables in spectral data is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of near infrared spectroscopy, and in particular relates to a near infrared spectroscopy variable selection method, a chemical composition detection method and a computer device. Background Art

[0002] Near infrared spectroscopy is widely used in qualitative and quantitative analysis in the fields of food, environment, agriculture, petrochemicals, pharmaceuticals and biomedicine due to its advantages such as simplicity, rapidity, non-invasiveness, pollution-free, low cost, no need for special pretreatment, large amount of information, and simultaneous analysis of multiple components. In near infrared spectroscopy quantitative analysis, it is necessary to combine the multivariate calibration method in chemometrics to establish a calibration model between the target chemical components and near infrared spectroscopy data. Since spectral data is the collective response of all components in the analyte and is susceptible to noise, near infrared spectra are usually composed of broad, weak, non-specific and overlapping bands and some irrelevant variables. These uninformative variables will affect the quality of the model and lead to a decrease in the model's predictive ability. With the development of near infrared spectroscopy technology and chemometrics, variable selection has become a key step in near infrared spectroscopy multivariate calibration, and its role can be summarized in three aspects: (1) improving the model's predictive ability; (2) providing faster and more cost-effective predictors by reducing the curse of dimensionality; and (3) making the model simpler and more interpretable.

[0003] In view of the importance of variable selection, a variety of variable selection methods have been developed at home and abroad, including stepwise selection, genetic algorithm (GA), particle swarm optimization (PSO), simulated annealing (SA), uninformative variable elimination (UVE) and other classic algorithms.

[0004] With the development of model population analysis (MPA), some new variable selection methods have been proposed, such as Monte Carlo based UVE (MCUVE), competitive adaptive reweighted sampling (CARS), and bootstrap soft shrinkage (BOSS).

[0005] Although the above methods have been proven to be effective in the near-infrared field, they are mostly focused on selecting fewer characteristic variables when selecting variables, which may lead to the loss of useful wavelength information, and the risk of overfitting of the model when applied to different data sets cannot be ignored. In addition, the variable selection method based on the MPA idea is a process of selecting variables separately, which gives too high weights to certain variables and ignores the high correlation between continuous variables in spectral data. Given that near-infrared spectra have the characteristics of high collinearity, each chemical group response has a certain width, and the combination of variables has a great influence on the prediction performance of the model. A single low-weight variable is not very important, but when multiple low-weight variables are combined together, the robustness of the model can be enhanced, so that the model can obtain good prediction performance. In addition, the original near-infrared spectrum contains interference information such as baseline drift and noise. Although most of the interference can be eliminated by preprocessing, the coefficient of variation of some spectral points is still large. If these points are selected as characteristic variables, the stability of the model will be significantly affected. Summary of the invention

[0006] The purpose of the present invention is to provide a near-infrared spectroscopy variable selection method, a chemical composition detection method and a computer device to solve the technical problem that the existing variable selection method assigns too high weights to certain variables, so that some important low-weight variable combinations are eliminated in the variable selection process, resulting in low model prediction performance.

[0007] In order to solve the above technical problems, the present invention provides a technical solution of a method for selecting variables of near infrared spectroscopy: a method for selecting variables of near infrared spectroscopy, the method comprising:

[0008] S1. Obtain near infrared spectrum variable data and content data of target chemical components of the test sample;

[0009] S2, obtaining n different variable subsets based on the near infrared spectrum variable data, and there is no repeated variable in each variable subset;

[0010] S3, establishing a regression model between each variable subset and the content data of the target chemical component;

[0011] Obtaining new weights of each variable according to the regression coefficient of the regression model, and then performing penalty processing on outliers in the new weights to update the new weights;

[0012] Performing weighted sampling with replacement in the spectral variable space to obtain n variable subsets, and removing duplicate variables within each variable subset; the probability of each variable being selected is obtained according to the new weight;

[0013] S4, return to S3 and execute again until the number of variables in the variable subset meets the requirement;

[0014] S5, selecting the variables in the variable subset corresponding to the regression model with the best model prediction accuracy evaluation index as the final selected variables;

[0015] Or, the variables in the variable subset corresponding to the regression model with the best evaluation index are recorded, and then the set number of times is re-executed from S2, and the finally selected variable is determined according to the number of times each variable is recorded.

[0016] The beneficial effect of the above technical solution is that the technical solution of the variable selection method of the near-infrared spectrum of the present invention belongs to an improved invention. The present invention avoids excessive weights of a certain variable by punishing extreme weights (i.e. outliers), so that valuable low-weight variable combinations are affected and eliminated, taking into account the variable combination effect, and significantly improving the accuracy and stability of the prediction model. The present invention solves the technical problem that the existing variable selection method assigns too high weights to certain variables and ignores the high correlation between continuous variables in spectral data, resulting in low model prediction performance.

[0017] Furthermore, the outlier in the new weight is penalized by multiplying the outlier by a penalty factor less than 1.

[0018] Furthermore, the outlier is detected by the interquartile range method on the new weight: when the new weight is greater than Upper or less than Lower, the new weight is considered to be an outlier; Upper and Lower are obtained according to the following formula:

[0019] Upper=Q3+k×(Q3-Q1)

[0020] Lower=Q1-k×(Q3-Q1)

[0021] Among them, Q1 is the first quartile of the new weight, Q3 is the third quartile of the new weight, and k is the outlier coefficient.

[0022] Furthermore, the outlier value is an outlier value in the new weight after averaging processing.

[0023] Furthermore, the near-infrared spectrum variable data is the near-infrared spectrum variable data after removing variables whose coefficient of variation is greater than a coefficient of variation threshold.

[0024] Furthermore, n different variable subsets are obtained in S2 by performing n uniform samplings with replacement in the spectral variable space and removing duplicate variables within each variable subset.

[0025] Furthermore, the number of variables in the variable subset satisfies the requirement that the number of variables in the variable subset is 1.

[0026] Furthermore, the method of obtaining the new weight of each variable according to the regression coefficient of the regression model is: obtaining the new weight of each variable according to the regression coefficient of the better regression model; the better regression model is a regression model with better evaluation indicators among the regression models.

[0027] Furthermore, the better regression model is obtained in the following manner: a regression model with better evaluation index is selected as the better regression model according to a set ratio.

[0028] Furthermore, the regression model is a partial least squares regression model.

[0029] Furthermore, the calculation formula of the new weight is:

[0030]

[0031] Where n is the number of regression models, ω i is the new weight of the i-th variable, b i,k is the absolute value of the normalized regression coefficient of the ith variable in the kth regression model.

[0032] The present invention also provides a technical solution for a computer device: a computer device, comprising a processor, wherein the processor is used to execute a computer program to implement the steps of the variable selection method for near-infrared spectroscopy as described above.

[0033] The present invention also provides a technical solution for a chemical component detection method using near infrared spectroscopy: a chemical component detection method using near infrared spectroscopy, the method comprising obtaining near infrared spectral data of a sample to be tested; selecting spectral variables in the near infrared spectral data of the sample to be tested according to a target chemical component, and inputting the selected spectral variables into a trained prediction model to obtain content data of the target chemical component. The method for selecting spectral variables in the near infrared spectral data of the sample to be tested according to the target chemical component comprises:

[0034] S1. Obtain near infrared spectrum variable data and content data of target chemical components of the test sample;

[0035] S2, obtaining n different variable subsets based on the near infrared spectrum variable data, and there is no repeated variable in each variable subset;

[0036] S3, establishing a regression model between each variable subset and the content data of the target chemical component;

[0037] Obtaining new weights of each variable according to the regression coefficient of the regression model, and then performing penalty processing on outliers in the new weights to update the new weights;

[0038] Performing weighted sampling with replacement in the spectral variable space to obtain n variable subsets, and removing duplicate variables within each variable subset; the probability of each variable being selected is obtained according to the new weight;

[0039] S4, return to S3 and execute again until the number of variables in the variable subset meets the requirement;

[0040] S5, selecting the variables in the variable subset corresponding to the regression model with the best model prediction accuracy evaluation index as the final selected variables;

[0041] Or, the variables in the variable subset corresponding to the regression model with the best evaluation index are recorded, and then the set number of times is re-executed from S2, and the finally selected variable is determined according to the number of times each variable is recorded.

[0042] The beneficial effect of the above technical solution is that the technical solution of the chemical composition detection method of near-infrared spectroscopy of the present invention belongs to an improved invention. The present invention avoids excessive weights of a certain variable by punishing extreme weights (i.e. outliers), so that valuable low-weight variable combinations are affected and eliminated, taking into account the variable combination effect, and significantly improving the accuracy and stability of the prediction model. The present invention solves the technical problem that the existing variable selection method assigns too high weights to certain variables and ignores the high correlation between continuous variables in spectral data, resulting in low model prediction performance.

[0043] Furthermore, the outlier in the new weight is penalized by multiplying the outlier by a penalty factor less than 1.

[0044] Furthermore, the outlier is detected by the interquartile range method on the new weight: when the new weight is greater than Upper or less than Lower, the new weight is considered to be an outlier; Upper and Lower are obtained according to the following formula:

[0045] Upper=Q3+k×(Q3-Q1)

[0046] Lower=Q1-k×(Q3-Q1)

[0047] Among them, Q1 is the first quartile of the new weight, Q3 is the third quartile of the new weight, and k is the outlier coefficient.

[0048] Furthermore, the outlier value is an outlier value in the new weight after averaging processing.

[0049] Furthermore, the near-infrared spectral data is the near-infrared spectral data after removing variables whose coefficient of variation is greater than a coefficient of variation threshold.

[0050] The present invention also provides a technical solution for a computer device: a computer device, comprising a processor, wherein the processor is used to execute a computer program to implement the steps of the chemical composition detection method using near-infrared spectroscopy as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A method flow chart of an embodiment of a method for selecting variables of near infrared spectroscopy of the present invention;

[0052] Figure 2 A schematic diagram of RMSEP of different variable selection algorithms in a tobacco leaf test data set in an embodiment of the near infrared spectroscopy variable selection method of the present invention;

[0053] Figure 3 The R values ​​of different variable selection algorithms in the tobacco leaf test data set of the variable selection method embodiment of the near infrared spectrum of the present invention are shown in FIG. test 2 Schematic diagram;

[0054] Figure 4 The BOSS and OR-WVCBS predicted stability diagrams of sample 1 in the embodiment of the near infrared spectroscopy variable selection method of the present invention;

[0055] Figure 5 The BOSS and OR-WVCBS predicted stability diagrams of sample 2 in the embodiment of the near infrared spectroscopy variable selection method of the present invention;

[0056] Figure 6 This is the BOSS and OR-WVCBS predicted stability diagram of sample 3 in the embodiment of the near infrared spectroscopy variable selection method of the present invention. DETAILED DESCRIPTION

[0057] The present invention punishes extreme weights (i.e., outliers) to avoid excessive weights of a certain variable, so that valuable low-weight variable combinations are affected and eliminated, taking into account the variable combination effect, and significantly improving the accuracy and stability of the prediction model. The present invention solves the technical problem that the existing variable selection method gives excessive weights to certain variables and ignores the high correlation between continuous variables in spectral data, resulting in low model prediction performance.

[0058] Variable selection method embodiment of near infrared spectroscopy:

[0059] A variable selection method for near infrared spectroscopy, namely, Outlier Removal with Weighted Variable Combination Bootstrapping Selection (OR-WVCBS) method. Figure 1As shown, the method includes:

[0060] S1. Obtain near infrared spectrum variable data and target chemical component content data of the test sample.

[0061] In this embodiment, tobacco leaves are used as test samples. The tobacco leaf near infrared spectrum data includes 1478 tobacco leaf samples. The near infrared spectrum wavelength of each sample is distributed in 1000-2500nm. AntarisⅡ Fourier transform near infrared spectrometer (Thermo Electron, USA) is used to measure the tobacco leaf diffuse reflectance near infrared spectrum data. Spectral acquisition conditions: scanning times 64 times; resolution: 8cm -1 ; The spectrum was collected twice, and the correlation coefficient threshold of the two spectra was > 0.99; the ambient temperature range was 20-27°C; the relative humidity was 30%-60%. The background spectrum was collected every 20 minutes.

[0062] The near-infrared spectra of the samples were preprocessed using multiplicative signal correction (MSC) and convolution smoothing (Savitzky-Golay filter, SG), where the SG parameters were first-order derivative, second-order polynomial, and window length of 17.

[0063] Then, the coefficient of variation (CV) of each spectral point was calculated, and the spectral variables with CV absolute value > 5% (i.e., coefficient of variation threshold) were removed. The coefficient of variation was calculated according to the following formula:

[0064]

[0065] Among them, CV is the coefficient of variation of the spectral variable, SD is the standard deviation of the test sample data under the spectral variable, and Mean is the mean value of the test sample data under the spectral variable.

[0066] 21 kinds of amino acids including Aspartic acid (Asp), L-Threonine (Thr), Serine (Ser), L-Asparagine (Asn), Glutamic acid (Glu), Glutamine (Gln), Glycine (Gly), Alanine (Ala), Valine (Val), Cystine (Cys), Methionine (Met), L-Isoleucine (Ile), Leucine (Leu), Tyrosine (Tyr), Phenylalanine (Phe), Γ-aminobutyric acid (Gaba), Lysine (Lys), Histidine (His), Tryptophan (Trp), Arginine (Arg), and Proline (Pro) were used as target components, and their chemical contents were determined by an amino acid analyzer.

[0067] The Hotelling T test combined with principal component analysis (PCA) was used to remove abnormal tobacco leaf samples, and the remaining samples were divided into a training set (1128×1557) and a test set (279×1557) using the Kennard-Stone (KS) algorithm.

[0068] S2, obtaining n different variable subsets according to the near infrared spectrum variable data of the test sample, and there is no repeated variable in each variable subset;

[0069] In this embodiment, 1000 samples are sampled in the variable space of the tobacco leaf training set samples by using the bootstrap sampling method (BSS) to generate 1000 different variable subsets, exclude duplicate variables, and assign the same initial weights to the selected variables.

[0070] S3. Establish a regression model between each variable subset and the content data of the target chemical component; calculate the model prediction accuracy evaluation index of each regression model.

[0071] New weights of various variables are obtained according to the regression coefficients of the regression model, and then outliers in the new weights are penalized to update the new weights.

[0072] According to the near-infrared spectrum variable data and the new weights, n different variable subsets are obtained, there is no repeated variable in each variable subset, and the probability of each variable being selected is obtained according to the new weights.

[0073] In this embodiment, the partial least squares (PLS) method is used to establish a regression model for each variable subset, and the root mean square error of cross validation (RMSECV) of the sub-model (i.e., a prediction accuracy evaluation index) is calculated.

[0074] The model corresponding to the first 10% (i.e., the set ratio) group variable subset with the smallest RMSECV is extracted as the best model. The regression coefficient (RC) of the best model is calculated, and the absolute value is taken and normalized vector summation is performed to obtain the new weight of the variable. The calculation formula of the new weight is:

[0075]

[0076] Where n is the number of regression models, ω i is the new weight of the i-th variable, b i,k is the absolute value of the normalized regression coefficient of the ith variable in the kth regression model.

[0077] The central variable weight is replaced by the mean of all variable weights within a fixed window length (i.e., averaging), and the interquartile range method is used to detect outliers, i.e., extreme weight variables. Extreme weights are penalized by multiplying them by a penalty factor less than 1. The interquartile range (IQR) is defined as the difference between the third quartile (Q3) and the first quartile (Q1), and is used to calculate the upper and lower limits of outlier detection. The upper and lower limits are calculated as follows:

[0078] Upper=Q3+k×(Q3-Q1)

[0079] Lower=Q1-k×(Q3-Q1)

[0080] Among them, Q1 is the first quartile of the new weight, Q3 is the third quartile of the new weight, and k is the outlier coefficient.

[0081] Through weighted bootstrap sampling, n samples are sampled in the spectral variable space to obtain n variable subsets, and repeated variables within each variable subset are removed; the probability of each variable being selected in the sampling is determined according to the new weight.

[0082] S4. Return to S3 and execute again until the number of variables in the variable subset meets the requirement.

[0083] Specifically, in this embodiment, the number of variables in the variable subset meets the legal person requirement: the number of variables in the variable subset is 1.

[0084] S5. Select the variables in the variable subset corresponding to the regression model with the best model prediction accuracy evaluation index as the final selected variables and record them.

[0085] In order to avoid the influence of algorithm randomness on the variable selection results during the operation of the algorithm, this embodiment repeats the above process 50 times, and uses the variables with a frequency of more than 60% recorded in S5 as the variables in the optimal variable combination, and as the characteristic variables of the target chemical components finally selected.

[0086] In order to illustrate the superiority of the OR-WVCBS variable selection method, the tobacco leaf near-infrared spectral data were subjected to characteristic variable extraction under the same conditions using five variable selection methods, namely MCUVE, CARS, BOSS, WVCBS (Weighted Variable Combination Bootstrapping Selection, i.e., the method of the present invention that does not eliminate abnormal spectral variables with higher coefficient of variation) and OR-WVCBS. Since each variable selection method has a certain randomness during the operation process, each of the above variable selection methods was run 50 times, and the prediction model of tobacco leaf amino acid content was established using PLS. The evaluation indicators of the prediction results of each modeling method are shown in Tables 1 to 21. The number of principal factors in the table is the number of principal factors of the PLS model, which determines the complexity and explanatory power of the PLS model; the number of variables is the number of optimal variable combinations extracted by the above variable selection methods; RMSECV is the root mean square error of cross validation; RMSEP (Root Mean Square Error of Prediction) is the root mean square error of prediction; R cv 2 is the fitting degree of the training set data; R test 2 is the fitting degree of the test set data. RMSEP and R of different variable selection algorithms in the tobacco test data set test 2 Respectively Figure 2 and Figure 3 shown.

[0087] Table 1 Comparison of prediction accuracy of tobacco leaf amino acid (Asp) content by different variable selection methods

[0088] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Asp PLS 10 1557 60.82 74.14 0.8136 0.7569 Asp MCUVE 8 565 63.13 77.31 0.7992 0.7357 Asp CARS 9 155 60.76 73.98 0.8140 0.7580 Asp BOSS 8 189 60.56 75.39 0.8152 0.7486 Asp WVCBS 11 357 55.78 65.53 0.8330 0.8034 Asp OR-WVCBS 13 295 54.31 64.81 0.8417 0.8078

[0089] Table 2 Comparison of prediction accuracy of tobacco leaf amino acid (Thr) content by different variable selection methods

[0090]

[0091]

[0092] Table 3 Comparison of prediction accuracy of tobacco leaf amino acid (Ser) content by different variable selection methods

[0093] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Ser PLS 12 1557 42.36 53.67 0.8347 0.7333 Ser MCUVE 9 756 45.32 53.87 0.8108 0.7313 Ser CARS 8 40 45.17 53.14 0.8121 0.7386 Ser BOSS 9 278 42.93 52.47 0.8303 0.7451 Ser WVCBS 12 509 32.17 37.87 0.8716 0.8690 Ser OR-WVCBS 12 499 33.19 38.06 0.8703 0.8677

[0094] Table 4 Comparison of prediction accuracy of tobacco leaf amino acid (Asn) content by different variable selection methods

[0095] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Asn PLS 9 1557 358.33 433.87 0.8444 0.7651 Asn MCUVE 8 443 352.04 423.6 0.8498 0.7761 Asn CARS 8 30 337.4 406.9 0.8621 0.7933 Asn BOSS 8 169 328.8 403.2 0.8690 0.7971 Asn WVCBS 13 480 301.0 311.4 0.9035 0.9010 Asn OR-WVCBS 13 448 300.6 301.9 0.9036 0.9032

[0096] Table 5 Comparison of prediction accuracy of tobacco leaf amino acid (Glu) content by different variable selection methods

[0097] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Glu PLS 11 1557 40.03 42.31 0.8525 0.8248 Glu MCUVE 8 673 42.45 44.41 0.8309 0.8071 Glu CARS 8 118 41.06 41.90 0.8449 0.8282 Glu BOSS 9 166 39.50 41.57 0.8564 0.8342 Glu WVCBS 12 311 35.71 38.55 0.8815 0.8661 Glu OR-WVCBS 11 292 36.33 39.09 0.8767 0.8623

[0098] Table 6 Comparison of prediction accuracy of tobacco leaf amino acid (Gln) content by different variable selection methods

[0099]

[0100]

[0101] Table 7 Comparison of prediction accuracy of tobacco leaf amino acid (Gly) content by different variable selection methods

[0102] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Gly PLS 14 1557 7.673 6.599 0.7725 0.7206 Gly MCUVE 8 479 8.541 7.123 0.6969 0.6744 Gly CARS 9 40 8.165 6.851 0.6916 0.6825 Gly BOSS 9 178 8.042 6.922 0.6926 0.6788 Gly WVCBS 13 387 6.004 6.037 0.7746 0.7661 Gly OR-WVCBS 13 325 5.907 5.797 0.7879 0.7683

[0103] Table 8 Comparison of prediction accuracy of tobacco leaf amino acid (Ala) content by different variable selection methods

[0104] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Ala PLS 13 1557 47.87 55.87 0.8298 0.7407 Ala MCUVE 9 699 51.81 58.06 0.8006 0.7200 Ala CARS 8 163 50.79 56.12 0.8084 0.7383 Ala BOSS 8 171 49.09 55.57 0.8210 0.7435 Ala WVCBS 12 335 44.53 48.32 0.8524 0.8233 Ala OR-WVCBS 13 320 44.57 47.93 0.8522 0.8261

[0105] Table 9 Comparison of prediction accuracy of tobacco leaf amino acid (Val) content by different variable selection methods

[0106] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Val PLS 12 1557 42.61 43.68 0.7626 0.7466 Val MCUVE 10 755 43.72 44.25 0.7501 0.7400 Val CARS 9 78 44.38 45.55 0.7424 0.7245 Val BOSS 9 268 42.83 43.38 0.7564 0.7539 Val WVCBS 12 466 40.35 41.38 0.7824 0.7727 Val OR-WVCBS 12 450 40.53 41.81 0.7805 0.7679

[0107] Table 10 Comparison of prediction accuracy of tobacco leaf amino acid (Cys) content using different variable selection methods

[0108]

[0109]

[0110] Table 11 Comparison of prediction accuracy of tobacco leaf amino acid (Met) content by different variable selection methods

[0111] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Met PLS 15 1557 4.308 4.346 0.3930 0.1828 Met MCUVE 15 510 4.280 4.287 0.4009 0.2049 Met CARS 9 35 4.510 4.338 0.3347 0.1857 Met BOSS 11 184 4.382 4.207 0.3721 0.2342 Met WVCBS 14 301 3.819 4.154 0.5216 0.3870 Met OR-WVCBS 14 288 3.809 4.144 0.5242 0.3902

[0112] Table 12 Comparison of prediction accuracy of tobacco leaf amino acid (Ile) content by different variable selection methods

[0113] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Ile PLS 15 1557 4.408 4.367 0.5519 0.5330 Ile MCUVE 11 469 4.401 4.454 0.5540 0.5141 Ile CARS 9 118 4.406 4.479 0.5530 0.5086 Ile BOSS 10 287 4.253 4.387 0.5834 0.5287 Ile WVCBS 13 501 3.521 3.977 0.6837 0.6126 Ile OR-WVCBS 13 451 3.703 4.059 0.6483 0.5966

[0114] Table 13 Comparison of prediction accuracy of tobacco leaf amino acid (Leu) content by different variable selection methods

[0115] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Leu PLS 9 1557 4.874 5.285 0.5346 0.5237 Leu MCUVE 8 443 4.996 5.305 0.5205 0.5200 Leu CARS 9 90 4.947 5.239 0.5319 0.5110 Leu BOSS 9 293 4.785 5.107 0.5553 0.5325 Leu WVCBS 14 517 4.466 4.665 0.6316 0.6092 Leu OR-WVCBS 14 486 4.531 4.697 0.6292 0.6046

[0116] Table 14 Comparison of prediction accuracy of tobacco leaf amino acid (Tyr) content using different variable selection methods

[0117] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Tyr PLS 8 1557 15.82 18.20 0.5333 0.4664 Tyr MCUVE 8 356 15.48 17.95 0.5528 0.4804 Tyr CARS 8 46 15.19 17.66 0.5697 0.4973 Tyr BOSS 9 205 15.04 17.63 0.5779 0.4990 Tyr WVCBS 15 472 13.99 16.89 0.6370 0.5400 Tyr OR-WVCBS 15 454 13.99 16.87 0.6369 0.5415

[0118] Table 15 Comparison of prediction accuracy of tobacco leaf amino acid (Phe) content by different variable selection methods

[0119] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Phe PLS 9 1557 34.08 43.68 0.8254 0.7349 Phe MCUVE 9 560 33.37 42.65 0.8326 0.7473 Phe CARS 8 26 32.74 40.18 0.8388 0.7756 Phe BOSS 8 191 33.21 42.16 0.8341 0.7530 Phe WVCBS 13 330 29.36 32.61 0.8688 0.8615 Phe OR-WVCBS 14 315 29.39 32.71 0.8685 0.8606

[0120] Table 16 Comparison of prediction accuracy of tobacco leaf amino acid (Gaba) content by different variable selection methods

[0121] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Gaba PLS 12 1557 26.98 29.45 0.8173 0.7493 Gaba MCUVE 9 531 28.62 30.09 0.7944 0.7383 Gaba CARS 8 118 27.66 29.51 0.8080 0.7483 Gaba BOSS 8 173 27.68 29.50 0.8077 0.7484 Gaba WVCBS 12 388 24.17 28.80 0.8430 0.8193 Gaba OR-WVCBS 12 376 24.32 28.78 0.8410 0.8195

[0122] Table 17 Comparison of prediction accuracy of tobacco leaf amino acid (Lys) content using different variable selection methods

[0123]

[0124]

[0125] Table 18 Comparison of prediction accuracy of tobacco leaf amino acid (His) content by different variable selection methods

[0126] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> His PLS 9 1557 27.86 36.34 0.8335 0.7007 His MCUVE 9 533 26.66 35.33 0.8473 0.7170 His CARS 8 40 25.63 33.99 0.8591 0.7381 His BOSS 8 149 25.43 34.37 0.8613 0.7323 His WVCBS 11 380 26.26 27.10 0.8512 0.8429 His OR-WVCBS 10 361 26.68 27.70 0.8464 0.8358

[0127] Table 19 Comparison of prediction accuracy of tobacco leaf amino acid (Trp) content by different variable selection methods

[0128] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Trp PLS 12 1557 32.85 40.79 0.7842 0.6528 Trp MCUVE 9 559 33.01 41.49 0.7790 0.6408 Trp CARS 7 60 32.41 40.66 0.7869 0.6550 Trp BOSS 9 253 31.98 40.17 0.7926 0.6634 Trp WVCBS 12 444 31.70 35.19 0.7990 0.7424 Trp OR-WVCBS 13 432 31.37 34.83 0.8010 0.7476

[0129] Table 20 Comparison of prediction accuracy of tobacco leaf amino acid (Arg) content by different variable selection methods

[0130] Components method Number of principal factors Number of variables RMSECV RMSEP <![CDATA[R cv 2 ]]> <![CDATA[R test 2 ]]> Arg PLS 13 1557 9.716 10.66 0.7035 0.6103 Arg MCUVE 8 633 9.433 10.62 0.7113 0.6134 Arg CARS 23 8 9.262 10.65 0.7217 0.6109 Arg BOSS 9 194 8.979 10.58 0.7384 0.6151 Arg WVCBS 11 336 8.565 9.604 0.7538 0.7137 Arg OR-WVCBS 11 313 8.670 9.667 0.7438 0.7074

[0131] Table 21 Comparison of prediction accuracy of tobacco leaf amino acid (Pro) content by different variable selection methods

[0132]

[0133]

[0134] It can be seen from Tables 1 to 21 above that the RMSECV and RMSEP values ​​of OR-WVCBS and WVCBS are always lower than those of other methods, while the fit of OR-WVCBS and WVCBS on both the training set and the test set is higher than that of other methods, that is, OR-WVCBS and WVCBS are better than other methods in various evaluation indicators.

[0135] In order to verify the stability of the variable selection method of this application, three tobacco leaf samples were selected, and the samples were repeatedly loaded and the near-infrared spectra were collected 10 times under the same environmental conditions as the tobacco leaf modeling set as the stability verification data set. The BOSS-PLS, WVCBS-PLS and OR-WVCBS-PLS near-infrared spectroscopy prediction models were used to predict the contents of 21 amino acids, and the RSD (Relative Standard Deviation) of the 10 prediction results was used to characterize the prediction stability of the model. The RSD values ​​of the prediction results of each indicator model are shown in Table 22. Figures 4 to 6 The amino acid indices with RSD>5% of BOSS-PLS and OR-WVCBS-PLS models in 3 samples are shown.

[0136] Table 22 Prediction stability results of BOSS-PLS, WVCBS-PLS and OR-WVCBS-PLS models

[0137]

[0138]

[0139] Computer equipment example:

[0140] A computer device includes a processor, the processor is used to execute a computer program to implement the steps of the variable selection method for near infrared spectroscopy as described above. The specific variable selection method for near infrared spectroscopy has been introduced in sufficient detail in the above embodiment of the variable selection method for near infrared spectroscopy, and will not be repeated here.

[0141] Chemical composition detection method embodiment of near infrared spectroscopy:

[0142] A chemical component detection method of near infrared spectroscopy, obtaining near infrared spectroscopy data of a sample to be tested; selecting spectral variables in the near infrared spectroscopy data of the sample to be tested according to a target chemical component, inputting the selected spectral variables into a trained prediction model to obtain content data of the target chemical component, the method of selecting spectral variables in the near infrared spectroscopy data of the sample to be tested according to the target chemical component is the same as the variable selection method of near infrared spectroscopy described above. The specific variable selection method of near infrared spectroscopy has been introduced in sufficient detail in the embodiment of the variable selection method of near infrared spectroscopy described above, and will not be repeated here.

[0143] Computer equipment example:

[0144] A computer device includes a processor, the processor is used to execute a computer program to implement the steps of the near infrared spectroscopy chemical component detection method as described above. The specific near infrared spectroscopy chemical component detection method has been introduced in sufficient detail in the above-mentioned near infrared spectroscopy chemical component detection method embodiment, and will not be repeated here.

[0145] The present invention has the following characteristics:

[0146] Compared with the existing variable selection methods, the outlier removal-weighted variable combination bootstrap selection proposed in the present invention takes into account the high collinearity of the near-infrared spectrum, takes into account the variable combination effect, reduces the risk of eliminating useful variables, and significantly improves the accuracy of the model. The abnormal spectral point removal step overcomes the technical defects that affect the prediction stability of the model and significantly improves the stability of the prediction model.

[0147] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions recorded in the aforementioned embodiments without creative work, or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A variable selection method for near infrared spectroscopy, characterized in that: The method includes: S1. Obtain near infrared spectrum variable data and content data of target chemical components of the test sample; S2, obtaining n different variable subsets based on the near infrared spectrum variable data, and there is no repeated variable in each variable subset; S3, establishing a regression model between each variable subset and the content data of the target chemical component; Obtaining new weights of each variable according to the regression coefficient of the regression model, and then performing penalty processing on outliers in the new weights to update the new weights; Performing weighted sampling with replacement in the spectral variable space to obtain n variable subsets, and removing duplicate variables within each variable subset; the probability of each variable being selected is obtained according to the new weight; S4, return to S3 and execute again until the number of variables in the variable subset meets the requirement; S5, selecting the variables in the variable subset corresponding to the regression model with the best model prediction accuracy evaluation index as the final selected variables; Or, the variables in the variable subset corresponding to the regression model with the best evaluation index are recorded, and then the set number of times is re-executed from S2, and the finally selected variable is determined according to the number of times each variable is recorded.

2. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The outlier in the new weight is penalized by multiplying the outlier by a penalty factor less than 1.

3. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The outlier is detected by the interquartile range method on the new weight: when the new weight is greater than Upper or less than Lower, the new weight is considered to be an outlier; Upper and Lower are obtained according to the following formula: Upper=Q3+k×(Q3-Q1) Lower=Q1-k×(Q3-Q1) Among them, Q1 is the first quartile of the new weight, Q3 is the third quartile of the new weight, and k is the outlier coefficient.

4. The variable selection method of near infrared spectroscopy according to claim 1, 2 or 3, characterized in that: The outlier value is an outlier value in the new weight after averaging.

5. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The near-infrared spectrum variable data is the near-infrared spectrum variable data after removing variables whose coefficient of variation is greater than a coefficient of variation threshold.

6. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The method of obtaining n different variable subsets in S2 is: performing n times of uniform sampling with replacement in the spectral variable space and removing repeated variables within each variable subset.

7. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The requirement that the number of variables in a variable subset meets is: the number of variables in a variable subset is 1.

8. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The method of obtaining the new weight of each variable according to the regression coefficient of the regression model is: obtaining the new weight of each variable according to the regression coefficient of the better regression model; the better regression model is the regression model with better evaluation indicators among the regression models.

9. The variable selection method of near infrared spectroscopy according to claim 8, characterized in that: The better regression model is obtained in the following manner: a regression model with better evaluation index is selected as the better regression model according to a set ratio.

10. The method for selecting variables of near infrared spectroscopy according to claim 1, 8 or 9, characterized in that: The regression model is a partial least squares regression model.

11. The variable selection method of near infrared spectroscopy according to claim 1, characterized in that: The calculation formula of the new weight is: oh i =∑ n k＝1 b i,k Where n is the number of regression models, ω i is the new weight of the i-th variable, b i,k is the absolute value of the normalized regression coefficient of the ith variable in the kth regression model.

12. A computer device comprising a processor, characterized in that: The processor is used to execute a computer program to implement the steps of the near-infrared spectroscopy variable selection method according to any one of claims 1 to 11.

13. A method for detecting chemical components using near infrared spectroscopy, the method comprising: Acquire near infrared spectrum data of the sample to be tested; A spectral variable is selected from the near-infrared spectral data of the sample to be tested according to the target chemical component, and the selected spectral variable is input into a trained prediction model to obtain the content data of the target chemical component. The method of selecting a spectral variable from the near-infrared spectral data of the sample to be tested according to the target chemical component includes: S1. Obtain near infrared spectrum variable data and content data of target chemical components of the test sample; S2, obtaining n different variable subsets based on the near infrared spectrum variable data, and there is no repeated variable in each variable subset; S3, establishing a regression model between each variable subset and the content data of the target chemical component; Obtaining new weights of each variable according to the regression coefficient of the regression model, and then performing penalty processing on outliers in the new weights to update the new weights; Performing weighted sampling with replacement in the spectral variable space to obtain n variable subsets, and removing duplicate variables within each variable subset; the probability of each variable being selected is obtained according to the new weight; S4, return to S3 and execute again until the number of variables in the variable subset meets the requirement; S5, selecting the variables in the variable subset corresponding to the regression model with the best model prediction accuracy evaluation index as the final selected variables; Or, the variables in the variable subset corresponding to the regression model with the best evaluation index are recorded, and then the set number of times is re-executed from S2, and the finally selected variable is determined according to the number of times each variable is recorded.

14. The chemical composition detection method of near infrared spectroscopy according to claim 13, characterized in that: The outlier in the new weight is penalized by multiplying the outlier by a penalty factor less than 1.

15. The chemical composition detection method of near infrared spectroscopy according to claim 13, characterized in that: The outlier is detected by the interquartile range method on the new weight: when the new weight is greater than Upper or less than Lower, the new weight is considered to be an outlier; Upper and Lower are obtained according to the following formula: Upper=Q3+k×(Q3-Q1) Lower=Q1-k×(Q3-Q1) Among them, Q1 is the first quartile of the new weight, Q3 is the third quartile of the new weight, and k is the outlier coefficient.

16. The chemical composition detection method of near infrared spectroscopy according to claim 13, 14 or 15, characterized in that: The outlier value is an outlier value in the new weight after averaging.

17. The chemical composition detection method of near infrared spectroscopy according to claim 13, characterized in that: The near infrared spectral data are the near infrared spectral data after removing variables whose coefficient of variation is greater than a threshold value of the coefficient of variation.

18. A computer device comprising a processor, characterized in that: The processor is used to execute a computer program to implement the steps of the chemical composition detection method using near-infrared spectroscopy as described in any one of claims 13 to 17.

Images