A tea classification method based on near-infrared spectral feature selection and parameter optimization

Abstract

The present invention belongs to the technical field of tea classification, and discloses a tea classification method based on near-infrared spectral feature selection and parameter optimization. First, a near-infrared spectrometer is used to scan and obtain tea near-infrared spectral data; then, the original near-infrared spectral data is preprocessed by using the first-order derivative and multivariate scattering correction method; on the basis of mutual information screening, the mutual information value and the corresponding spectral data absorbance are fused in combination with information fusion technology, and the feature selection is performed by principal component analysis, and then random forest classification is used to calculate the tea classification accuracy; finally, a genetic algorithm is selected to screen out the best feature combination and hyperparameters according to the fitness function score and distinguish teas of different varieties and grades. The present invention integrates information fusion, mutual information feature selection, principal component feature selection, random forest classification and genetic algorithm optimization to screen out the best feature combination and hyperparameters, and realizes the discrimination of teas of different varieties and grades, and the screening and discrimination results are accurate and reliable.

Description

Technical Field

[0001] The invention belongs to the technical field of tea classification, and in particular relates to a tea classification method based on near-infrared spectrum feature selection and parameter optimization. Background Art

[0002] In the field of classification of tea varieties and grades, identification methods such as expert review, chemical analysis and image classification are often used. However, the expert review method is easily interfered by personal subjective factors, and has low accuracy and stability; the chemical analysis method requires professional institutions to identify the physical and chemical components of tea, which is expensive and time-consuming; the image classification method is difficult to distinguish the subtle texture differences of different grades of tea, and is prone to misjudgment.

[0003] Near-infrared spectroscopy technology obtains the characteristic information and internal structure of the sample based on the difference in the absorption wavelength of near-infrared light by different groups within the substance, and can make qualitative judgments quickly, non-destructively and accurately. However, near-infrared spectroscopy technology has the problem of high data feature dimensions, and the irrelevant and redundant information contained in the features interferes with the classification accuracy. In order to simply and quickly screen high-quality features, it is necessary to combine feature selection and parameter optimization methods to improve classification performance. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the present invention proposes a tea classification method based on near-infrared spectral feature selection and parameter optimization. The near-infrared spectral data is preprocessed by using the first-order derivative and multivariate scattering correction method, and then the gold and information fusion, mutual information and principal component analysis feature selection and random forest classification are performed, and then the fitness function of each iteration is calculated by using the genetic algorithm to screen out the best feature combination and hyperparameters and distinguish teas of different varieties and grades.

[0005] The technical solution of the present invention is as follows:

[0006] A tea classification method based on near infrared spectroscopy feature selection and parameter optimization comprises the following steps:

[0007] Step 1: Obtain tea near-infrared spectrum data;

[0008] Step 2: pretreatment of tea leaves by near infrared spectroscopy;

[0009] Step 3: Feature selection based on information fusion;

[0010] Step 4: Use the random forest algorithm to classify the data after feature selection and calculate the classification accuracy;

[0011] Step 5: Use genetic algorithm to find the best feature combination and hyperparameters.

[0012] Furthermore, the step 1 comprises the following steps:

[0013] Different tea samples were classified according to type, grade, price and quantity. A near-infrared analyzer was used to collect near-infrared spectral data of tea. 52 data were scanned for each variety and grade of tea, and each spectral data contained the absorbance corresponding to 800 wavelength points. The spectral data were randomly stratified and sampled to obtain training sets and validation sets.

[0014] Furthermore, the step 2 comprises the following steps:

[0015] The first-order derivative and multivariate scatter correction methods are used to perform data preprocessing operations on the measured original near-infrared spectral data in turn, and the preprocessed spectral data are calculated; the first-order derivative amplifies and smoothes the variation trend between spectra, and the multivariate scatter correction corrects the offset caused by the spectral baseline drift.

[0016] Furthermore, the step 3 comprises the following steps:

[0017] The mutual information between the features of the spectral data after preprocessing was calculated, and the features with the largest mutual information values ​​of the first m were selected. The mutual information values ​​and the corresponding spectral data absorbance were fused at the feature layer by combining information fusion technology, and the fused sample matrix was calculated. The principal component analysis method was then used to perform feature selection on the fused sample matrix, and finally the data after feature selection was calculated.

[0018] Furthermore, the step 3 uses the following formula to calculate the mutual information between the preprocessed spectral data features:

[0019]

[0020] Where x i is the absorbance value corresponding to each feature, y is the corresponding tea variety grade label, p(x i ), p(y) are the probabilities of the corresponding absorbance value and tea label, p(x i ,y) is the joint distribution probability of the corresponding absorbance value and tea label; the wavelength range is 1000-1800nm, the point interval is 1nm, and after the first-order derivative difference calculation, there are 799 mutual information data.

[0021] Furthermore, the step 3 uses the following formula to calculate the sample matrix after information fusion:

[0022] X' n×m =k1X n×m +k2I n×m

[0023] Where k1 and k2 are the weight coefficients of absorbance and mutual information, respectively, and they are set to 0.5 and 0.5, respectively. n×m , X n×mThey are the mutual information values ​​and spectral data absorbance corresponding to the first m features with the largest mutual information values, respectively, where n is the number of samples in the training set.

[0024] Furthermore, the step 3 uses the following formula to calculate the data after feature selection by principal component analysis:

[0025] Y n×q =X' n×m M m×q

[0026] Where M m×q The principal component analysis method is used to select the largest first q eigenvalues ​​after eigenvalue decomposition to form an eigenvector matrix. Further, the step 5 uses a genetic algorithm to find the best feature combination and hyperparameters, selects a binary encoding method to create chromosomes, and a large number of random chromosome individuals constitute an initialized population. The Gray code decoding method is used to map the binary value to the decimal solution space, and the fitness function is used to evaluate the quality of individuals in the population. The selection, crossover, and mutation methods of chromosomes are tournament selection method, multi-point crossover method, and basic bit mutation method, respectively. The above steps are repeated until the number of iterations reaches the set value.

[0027] Furthermore, the step 5 uses the following formula to calculate the binary-coded chromosome:

[0028]

[0029] Where L is the number of chromosomes, L MI , L PCA and L RF are the number of bits occupied by the hyperparameters set for mutual information, principal component analysis, and random forest in chromosomes, u MI , l MI ;u PCA , l PCA and u RF , l RF are the upper and lower limits of the number of features extracted by mutual information, the number of eigenvalues ​​of principal component analysis, and the number of random forest classifiers, respectively. eps is the genetic iteration accuracy. The value of each bit in the chromosome consists of 0 and 1.

[0030] Furthermore, the step 5 uses the following formula to calculate the fitness function:

[0031] FitV=Acc×α+σ×β

[0032] Where Acc is the accuracy of random forest algorithm after classification, σ is the standard deviation of the accuracy of each subclass, and α and β are the weight coefficients of accuracy and standard deviation respectively.

[0033] The beneficial effects of the present invention are as follows: the present invention integrates information fusion, mutual information feature selection, principal component feature selection, random forest classification and genetic algorithm optimization to screen out the best feature combination and hyperparameters, thereby achieving discrimination of teas of different varieties and grades, and the screening and discrimination results are accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the overall flow chart of the method of the present invention;

[0035] Figure 2 This is a near infrared spectrum data diagram after preprocessing of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments.

[0037] like Figure 1 As shown, a tea classification method based on near infrared spectroscopy feature selection and parameter optimization of the present invention comprises the following steps:

[0038] 1) Obtain tea near-infrared spectrum data

[0039] Different types and grades of Longjing spring tea from the six major production areas of Zhejiang Province in 2020 were selected. The types, grades, prices and quantities of tea samples are shown in Table 1. There are 6 types of Longjing tea: West Lake Longjing, Fuyang Longjing, Jinyun Longjing, Xinchang Longjing, Wuyi Longjing and Suichang Longjing. Among them, the grade of each Longjing is marked as A, B, C, D from high to low according to the market price, with a total of 24 subcategories, each of which contains 500g of Longjing tea of ​​the corresponding type and grade.

[0040] The SupNIR2720 near-infrared analyzer was selected, with a wavelength range of 1000-1800nm, a point interval of 1nm, a wavelength accuracy of 0.2nm, a resolution of 10nm, and a spectrum average of 30 times. After starting up, the instrument needs to be preheated for 30 minutes, and reference calibration is performed every 30 minutes.

[0041] The experiment collected spectral data of 6 kinds of Longjing tea in Table 1, of which each Longjing contains 4 grades. For each measurement, 10±0.1g of tea sample was weighed and placed on the sampling table. 52 spectral data were scanned for each grade of tea, totaling 1248 spectral data. Each spectral data contains the absorbance corresponding to 800 wavelength points. The spectral data were randomly stratified and sampled according to 7:3, and 873 training sets and 375 validation sets were obtained.

[0042] Table 1 Grade, price and quantity of Longjing tea

[0043]

[0044] 2) Tea Near Infrared Spectrum Pretreatment

[0045] The first-order derivative and multivariate scatter correction methods are used to preprocess the measured raw near-infrared spectral data in turn. The first-order derivative amplifies and smoothes the trend of changes between spectra, and the multivariate scatter correction corrects the offset caused by the spectral baseline drift. The preprocessed near-infrared spectral data (such as Figure 2 There are obvious characteristic peaks and data differences.

[0046] 3) Feature selection based on information fusion

[0047] Calculate the mutual information between the features of the preprocessed spectral data and select the first m features with the largest mutual information values. The following formula is used to calculate the mutual information between the features of the preprocessed spectral data:

[0048]

[0049] Where x i is the absorbance value corresponding to each feature, y is the corresponding tea variety grade label, p(x i ), p(y) are the probabilities of the corresponding absorbance value and tea label, p(x i ,y) is the joint distribution probability under the corresponding absorbance value and tea label. The wavelength range is 1000-1800nm, and the point interval is 1nm. After the first-order derivative difference calculation, there are 799 mutual information data.

[0050] Combined with information fusion technology, the mutual information value and the corresponding spectral data absorbance are fused at the feature layer to calculate the fused sample matrix. The following formula is used to calculate the sample matrix after information fusion:

[0051] X' n×m =k1X n×m +k2I n×m

[0052] Where k1 and k2 are the weight coefficients of absorbance and mutual information, respectively, and they are set to 0.5 and 0.5. n×m , X n×m They are the mutual information values ​​and spectral data absorbance corresponding to the first m features with the largest mutual information values, respectively, where n is the number of samples in the training set, 873.

[0053] The principal component analysis method is used to select features of the fused sample matrix, and finally the data after feature selection is calculated. The following formula is used to calculate the data after feature selection using the principal component analysis method:

[0054] Y n×q =X' n×m M m×q

[0055] Where M m×q The principal component analysis method is used to select the largest first q eigenvalues ​​after eigenvalue decomposition to form the eigenvector matrix.

[0056] 4) Use the random forest algorithm to classify the data after feature selection and calculate the classification accuracy.

[0057] 5) Feature combination and hyperparameter optimization based on genetic algorithm

[0058] A genetic algorithm is used to find the best feature combination and hyperparameters, binary coding is used to create chromosomes, and a large number of random chromosome individuals constitute the initialized population. Gray code decoding is used to map binary values ​​to decimal solution space, and fitness function is used to evaluate the quality of individuals in the population. The chromosome selection, crossover and mutation methods are tournament selection method, multi-point crossover method and basic bit mutation method respectively. The above steps are repeated until the number of iterations reaches the set value.

[0059] The binary-coded chromosomes are calculated using the following formula:

[0060]

[0061] Where L is the number of chromosomes, L MI , L PCA and L RF are the number of bits occupied by the hyperparameters set for mutual information, principal component analysis, and random forest in chromosomes, u MI , l MI ;u PCA , l PCA and u RF , l RF are the upper and lower limits of the number of features extracted from mutual information, the number of eigenvalues ​​of principal component analysis, and the number of random forest classifiers, which are set here as: 100, 600, 50, 100, 20, 50 respectively; eps is the genetic iteration accuracy, which is 1 here; the value of each bit in the chromosome consists of 0 and 1.

[0062] The fitness function is calculated using the following formula:

[0063] FitV=Acc×α+σ×β

[0064] Where Acc is the accuracy of random forest algorithm after classification, σ is the standard deviation of the accuracy of each subclass, α and β are the weight coefficients of accuracy and standard deviation, which are set to 0.8 and 0.2 respectively.

[0065] Finally, the classification effect of this embodiment is shown in Table 2. Combined with the tea classification method based on near infrared spectroscopy feature selection and parameter optimization proposed in the present invention, for the classification of Longjing tea in this embodiment, the calculated accuracy is 99.04%, the precision is 99.11%, and the recall is 99.06%.

[0066] Table 2 Longjing tea classification effect based on the present invention

[0067]

[0068] It can be seen that the tea classification method based on near-infrared spectral feature selection and parameter optimization proposed in the present invention can effectively select a better feature combination and set parameters, and finally obtain a better classification effect.

[0069] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modification and change made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A tea classification method based on near infrared spectroscopy feature selection and parameter optimization, characterized in that: The following steps are included: Step 1: Obtain tea near-infrared spectrum data; Step 2: pretreatment of tea leaves by near infrared spectroscopy; Step 3: Feature selection based on information fusion; Step 4: Use the random forest algorithm to classify the data after feature selection and calculate the classification accuracy; Step 5: Use genetic algorithm to find the best feature combination and hyperparameters; The step 5 uses a genetic algorithm to find the best feature combination and hyperparameters, selects a binary encoding method to create chromosomes, and a large number of random chromosome individuals constitute an initialized population. The Gray code decoding method is used to map the binary value to the decimal solution space, and the fitness function is used to evaluate the quality of individuals in the population. The selection, crossover, and mutation methods of chromosomes are respectively a tournament selection method, a multi-point crossover method, and a basic bit mutation method. The above steps are repeated until the number of iterations reaches the set value; The step 5 uses the following formula to calculate the binary-coded chromosome: ， In the formula L is the number of chromosomes, , and are the number of bits occupied by the hyperparameters set for mutual information, principal component analysis, and random forest in chromosomes, respectively. , ; , and , are the upper and lower limits of the number of features extracted by mutual information, the number of eigenvalues ​​of principal component analysis, and the number of random forest classifiers, respectively. eps For genetic iteration accuracy, the value of each bit in the chromosome consists of 0 and 1; The step 5 uses the following formula to calculate the fitness function: ， In the formula Acc is the accuracy of random forest algorithm classification, σ is the standard deviation of the accuracy of each subclass, , are the weight coefficients of accuracy and standard deviation respectively.

2. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 1 is characterized in that: The step 1 comprises the following steps: Different tea samples were classified according to type, grade, price and quantity. A near-infrared analyzer was used to collect near-infrared spectral data of tea. 52 data were scanned for each variety and grade of tea, and each spectral data contained the absorbance corresponding to 800 wavelength points. The spectral data were randomly stratified and sampled to obtain training sets and validation sets.

3. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 2 is characterized in that: The step 2 comprises the following steps: The first-order derivative and multivariate scatter correction methods are used to perform data preprocessing operations on the measured original near-infrared spectral data in turn, and the preprocessed spectral data are calculated; the first-order derivative amplifies and smoothes the variation trend between spectra, and the multivariate scatter correction corrects the offset caused by the spectral baseline drift.

4. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 3 is characterized in that: The step 3 comprises the following steps: The mutual information between the features of the spectral data after preprocessing was calculated, and the features with the largest mutual information values ​​of the first m were selected. The mutual information values ​​and the corresponding spectral data absorbance were fused at the feature layer by combining information fusion technology, and the fused sample matrix was calculated. The principal component analysis method was then used to perform feature selection on the fused sample matrix, and finally the data after feature selection was calculated.

5. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 4 is characterized in that: The step 3 uses the following formula to calculate the mutual information between the pre-processed spectral data features: ， In the formula is the absorbance value corresponding to each feature, y is the corresponding tea variety grade label, , are the probabilities corresponding to the absorbance value and the tea label, is the joint distribution probability corresponding to the absorbance value and the tea label; the wavelength range is 1000-1800nm, the point interval is 1nm, and after the first-order derivative difference calculation, there are 799 mutual information data.

6. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 4 is characterized in that: The step 3 uses the following formula to calculate the sample matrix after information fusion: ， In the formula , are the weight coefficients of absorbance and mutual information, which are set to 0.5 and 0.5 respectively. , They are the mutual information values ​​and spectral data absorbance corresponding to the first m features with the largest mutual information values, respectively, where n is the number of samples in the training set.

7. The tea classification method based on near infrared spectroscopy feature selection and parameter optimization according to claim 4 is characterized in that: The step 3 uses the following formula to calculate the data after feature selection using principal component analysis: ， In the formula To select the largest frontier after eigenvalue decomposition using principal component analysis q The eigenvalues ​​form the eigenvector matrix.

Images