Multi-modal near infrared spectroscopy modeling classification method based on voting mechanism

Abstract

The application relates to a multi-modal near-infrared spectrum modeling classification method based on a voting mechanism, relates to the technical field of near-infrared spectrum detection, and is characterized in that: spectrum data is divided into a training set and a verification set, M kinds of pretreatment and N kinds of modeling methods are used, M*N models are established, K kinds of models with high optimal accuracy index of the verification set are selected to classify to-be-detected samples, K kinds of classification results of the same to-be-detected sample are saved as high-confidence results, the training set and the verification set are re-divided, models are established, models are selected to classify to-be-detected samples, another group of results is obtained, until each to-be-detected sample has at least one high-confidence result, the high-confidence results are analyzed based on the voting mechanism, and the classification results of each to-be-detected sample are obtained, so that the problem of low accuracy of near-infrared spectrum models in classification is solved, and the application is suitable for near-infrared spectrum detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of near-infrared spectroscopy detection technology, and particularly relates to a multi-modal near-infrared spectroscopy modeling classification method based on a voting mechanism. BACKGROUND

[0002] In near-infrared spectroscopy qualitative analysis, due to the limited amount of near-infrared spectroscopy detection data, the industry currently mainly uses traditional support vector machines, decision trees, linear discriminant methods, and cannot further use neural networks and other higher complexity algorithms. In the process of collecting near-infrared spectroscopy data, the data consistency is poor due to the influence of the surrounding environment and human operation, which results in that the stability and prediction accuracy of the near-infrared spectroscopy model in practical application are not ideal, especially in the classification of more than 5 categories, the spectral data crosstalk is large, and the defects are more obvious. SUMMARY

[0003] The present application solves the technical problem of providing a multi-modal near-infrared spectroscopy modeling classification method based on a voting mechanism to solve the problem of low accuracy of the near-infrared spectroscopy model in classification.

[0004] The technical solution adopted by the present application to solve the above technical problem is a multi-modal near-infrared spectroscopy modeling classification method based on a voting mechanism, comprising the following steps:

[0005] S01, obtaining spectral data of a plurality of samples of different categories and the categories of the samples;

[0006] S02, dividing the spectral data into a training set and a validation set;

[0007] S03, respectively pre-processing the spectral data in the training set;

[0008] S04, extracting principal components;

[0009] S05, respectively modeling using N methods to obtain M*N models;

[0010] S06, selecting the top K models in the M*N models in the order of the optimal accuracy indicators of the validation set from high to low as classification models;

[0011] S07, using the K classification models to classify the to-be-tested samples to obtain K results;

[0012] S08, saving the high-confidence results of the same to-be-tested sample classification results in the K results as consistent;

[0013] S09, re-dividing the training set and the validation set, and repeating steps S03 to S08 until all to-be-tested samples have at least one high-confidence result;

[0014] S10, based on the voting mechanism, analyzing the high confidence result to obtain the classification result of each sample to be tested.

[0015] Further, in S03, the M kinds of pretreatments include one or more of Gaussian smoothing, Gaussian derivation, SG smoothing, SG derivation, snv, detrending, standardization, MSC, centralization and baseline.

[0016] Further, in S05, the N kinds of methods include one or more of SVM, LDA, decision tree and neural network.

[0017] Further, the K is 3.

[0018] Further, in S02, the ratio of the training set and the validation set is 3 to 1.

[0019] Further, in S10, the voting mechanism is that: for the same sample to be tested, the classification result with the highest proportion of the classified categories in the high confidence result is the classification result of the sample to be tested, if the classification result with the highest proportion in the high confidence result of the same sample to be tested is multiple categories, then the category with the most sample quantity is taken as the classification result, if the sample quantities are equal, then the category with the earlier coding order in the model training is taken as the classification result.

[0020] The beneficial effects of the present application: the multi-modal near-infrared spectroscopy modeling classification method based on the voting mechanism, by dividing the spectral data into a training set and a validation set, and using M kinds of pretreatment and N kinds of modeling method, establishing M*N models, and selecting K kinds of models with the highest optimal accuracy rate index of the validation set to classify the samples to be tested, the K kinds of classification results of the same sample to be tested are consistent, which is saved as a high confidence result, the training set and the validation set are re-divided, the models are established, the models are selected to classify the samples to be tested, another group of results are obtained, until each sample to be tested has at least one high confidence result, based on the voting mechanism, the high confidence result is analyzed to obtain the classification result of each sample to be tested, thereby solving the problem of low accuracy rate when the near-infrared spectroscopy model is classified. BRIEF DESCRIPTION OF DRAWINGS

[0021] ATTACHMENT Figure 1 is the multi-modal near-infrared spectroscopy modeling classification method flowchart based on the voting mechanism of the present application. DETAILED DESCRIPTION

[0022] The multi-modal near-infrared spectroscopy modeling classification method based on the voting mechanism of the present application, as shown in the accompanying drawings, includes the following steps: Figure 1

[0023] ​S01, acquiring spectral data of a plurality of samples of different categories and the categories of the samples;

[0024] S02, dividing the spectral data into a training set and a validation set;

[0025] S03, performing M kinds of preprocessing on the spectral data in the training set respectively;

[0026] S04, extracting principal components;

[0027] S05, modeling by using N kinds of methods respectively, to obtain M*N kinds of models;

[0028] S06, selecting the top K models in the M*N kinds of models in terms of the optimal accuracy of the validation set from high to low, as the classification models;

[0029] S07, using the K classification models to classify the to-be-tested samples, to obtain K results;

[0030] S08, saving the high-confidence results of the same to-be-tested sample classification results in the K results as the same;

[0031] S09, re-dividing the training set and the validation set, and repeating steps S03 to S08 until all to-be-tested samples have at least one high-confidence result;

[0032] S10, based on a voting mechanism, analyzing the high-confidence results to obtain the classification results of each to-be-tested sample.

[0033] Further, in S03, the M kinds of preprocessing include one or more of Gaussian smoothing, Gaussian derivation, SG smoothing, SG derivation, snv, detrending, standardization, MSC, centralization, and baseline.

[0034] Further, in S05, the N kinds of methods include one or more of SVM, LDA, decision tree, and neural network.

[0035] Further, the K is 3.

[0036] Further, in step S02, the ratio of the training set to the validation set is 3 to 1.

[0037] Further, in S10, the voting mechanism is: for the same to-be-tested sample, the classification result of the category with the highest proportion in the high-confidence results is the classification result of the to-be-tested sample, if the classification result with the highest proportion in the high-confidence results of the same to-be-tested sample is multiple categories, then the category with the most sample quantity is taken as the classification result, if the sample quantities are also equal, then the category with the earlier coding order in the model training is taken as the classification result.

[0038] Embodiment:

[0039] Taking 7 categories as an example, respectively A, B, C, D, E, F, G categories, the number of samples is 377, then 377 samples are collected by the infrared spectrometer respectively, 377 spectrum data are obtained, and 76 spectrum data are selected as the detection sample set, and the remaining 301 spectrum data are used for model establishment. Specifically, the following steps are included:

[0040] S01, obtaining spectrum data of a plurality of samples of different categories and categories of samples;

[0041] Specifically, the spectrum data is 301, and the categories of the samples are 7, namely A, B, C, D, E, F, and G. In other embodiments, the number of spectrum data can be selected.

[0042] S02, dividing the spectrum data into a training set and a validation set;

[0043] Specifically, 301 spectrum data is divided into a training set and a validation set according to a ratio of 3:1. In other embodiments, a ratio of 3:1 can not be used, and a division ratio can be selected according to actual conditions.

[0044] S03, performing M kinds of preprocessing on the spectrum data in the training set;

[0045] Specifically, in this embodiment, M is 7, that is, 7 kinds of preprocessing, respectively Gaussian smoothing, Gaussian derivation, SG smoothing, SG derivation, snv, detrending and standardization. In other embodiments, one or more of Gaussian smoothing, Gaussian derivation, SG smoothing, SG derivation, snv, detrending, standardization, MSC, centering and baseline can be selected.

[0046] S04, extracting principal components;

[0047] Specifically, the principal component extraction algorithm is PCA, which aims to reduce the dimension of data, eliminate redundant information, and improve the correlation of data. In this embodiment, the data is reduced to 15 dimensions. In other embodiments, dimension reduction can be performed according to actual conditions.

[0048] S05, respectively using N kinds of methods for modeling, obtaining MN kinds of models;

[0049] Specifically, N is 2, that is, 2 kinds of methods are used for modeling, respectively SVM support vector machine and LDA linear discriminant analysis. In other embodiments, one or more of SVM, LDA, decision tree and neural network can be selected.

[0050] S06, selecting the top K models from high to low in the optimal accuracy index of the validation set in the M*N models as the classification model;

[0051] Specifically, 1≤K≤M*N, in the embodiment, there are 14 models in total, K is 3, that is, the top three models in the order from high to low of the optimal accuracy indicators of the validation set are selected as the classification models, in other embodiments, the top two, four, etc. can also be selected, as long as the optimal accuracy meets the requirements.

[0052] S07, classifying the to-be-tested samples by using the K classification models to obtain K results;

[0053] Specifically, in the embodiment, 76 to-be-tested samples are classified by using three classification models, that is, each model has a result for each to-be-tested sample, and the result includes A, B, C, D, E, F, and G.

[0054] S08, saving the same to-be-tested sample classification results in the K results as high-confidence results;

[0055] Specifically, for the same to-be-tested sample, if the classification results of the three models are consistent, the classification result is saved as a high-confidence result, and if they are inconsistent, the result is set to None.

[0056] S09, re-dividing the training set and the validation set, and repeating steps S03 to S08 until all to-be-tested samples have at least one high-confidence result;

[0057] Specifically, in S07, the three models may have inconsistent classification results for one or more to-be-tested samples, therefore, it is necessary to re-divide the training set and the validation set, repeat the steps and repeat steps S03 to S08, and then establish 14 models. The top three models in the order from high to low of the optimal accuracy indicators of the validation set in the 14 models may be different from the models in S06, and therefore the classification results of the 76 to-be-tested samples may also be different. Therefore, for the to-be-tested samples with inconsistent classification results, consistent classification results may be obtained. Therefore, the training set and the validation set are continuously re-divided, and steps S03 to S08 are repeatedly performed until all to-be-tested samples have at least one high-confidence result, and the number of repetitions is i. In the embodiment, i=3, that is, the classification results of the 76 to-be-tested samples are shown in Table 1.

[0058] Table 1

[0059]

[0060] S10, analyzing the high-confidence results based on a voting mechanism to obtain the classification results of each to-be-tested sample.

[0061] Specifically, the voting mechanism is: for the same to-be-tested sample, the classification result of the highest proportion of the classified categories in the high-confidence result of the to-be-tested sample is the classification result of the to-be-tested sample, if the highest proportion of the classification result of the same to-be-tested sample is multiple categories, then the category with the most sample quantity is taken as the classification result, if the sample quantities are also equal, then the category with the earlier coding order in the model training is taken as the classification result. For example, for the to-be-tested sample 1, the results of the three repetitions are all A, so the to-be-tested sample belongs to the A category, for the to-be-tested sample 2, the first result is null, the second and third results are both D, so the to-be-tested sample 2 belongs to the D category, for the to-be-tested sample 3, the first result is the C category, the second result is null, and the third result is the E category, so for the to-be-tested sample 3, the proportions of C and E in the high-confidence result are equal, both being one-third, therefore, the number of C category samples and the number of E category samples need to be confirmed, if the number of C category samples is higher than the number of E category samples, then the to-be-tested sample 3 belongs to the C category, if the number of C category samples is equal to the number of E category samples, then the coding order of the sample category in the model training is compared, the coding of C and E in the training is 3 and 5 respectively, 3 is before 5, that is, C is before E, so the to-be-tested sample 3 belongs to the C category, for the to-be-tested sample 76, the classification result A accounts for two-thirds, and the classification result F accounts for one-third, so the to-be-tested sample 76 belongs to the A category.

Claims

1. A multimodal near-infrared spectral modeling and classification method based on a voting mechanism, characterized in that, Includes the following steps: S01. Obtain spectral data of multiple samples of different categories and the category of the samples; S02. Divide the spectral data into a training set and a validation set; S03. Perform M types of preprocessing on the spectral data in the training set respectively; S04. Extract the main components; S05. Use N different methods to model, and obtain M*N models; S06. Select the top K models from the M*N models, ranked from highest to lowest, based on the optimal accuracy metrics on the validation set, and use them as classification models. S07. Use a K-type classification model to classify the test sample and obtain K results; S08. Save the results of the same test sample with consistent classification results among the K types of results as high-confidence results. S09. Redivide the training set and validation set, and repeat steps S03 to S08 until all test samples have at least one high-confidence result. S10. Based on the voting mechanism, analyze the high-confidence results to obtain the classification result for each test sample; The voting mechanism is as follows: for the same test sample, the classification result with the highest proportion of categories in its high confidence results is the classification result of the test sample. If the classification result with the highest proportion in the high confidence results of the same test sample is multiple categories, then the category with the largest number of samples in that category is taken as the classification result. If the number of samples is also equal, then the category with the earlier encoding order of the sample category in the model training is taken as the classification result.

2. The multimodal near-infrared spectral modeling and classification method based on a voting mechanism according to claim 1, characterized in that, In S03, the M types of preprocessing include one or more of Gaussian smoothing, Gaussian derivative, SG smoothing, SG derivative, SNV, detrending, standardization, MSC, centering, and baseline.

3. The multimodal near-infrared spectral modeling and classification method based on a voting mechanism according to claim 1, characterized in that, In S05, the N methods include one or more of SVM, LDA, decision tree, and neural network.

4. The multimodal near-infrared spectral modeling and classification method based on a voting mechanism according to claim 1, characterized in that, The value of K is 3.

5. The multimodal near-infrared spectral modeling and classification method based on a voting mechanism according to claim 1, characterized in that, In step S02, the ratio of the training set to the validation set is 3 to 1.

Images