A feature spectrum extraction method based on hyperspectral data dimensionality increase

Abstract

The application provides a feature spectrum extraction method based on hyperspectral data dimensionality increase, which is suitable for feature spectrum extraction of any object with hyperspectral reflectivity. The method comprises the following steps: using a spectrometer to collect hyperspectral radiance of a measured sample and a white board, and performing pretreatment on the hyperspectral radiance to obtain hyperspectral reflectivity of the measured sample; then, the one-dimensional reflectivity of the measured sample is increased in dimension by mathematical operation to obtain a two-dimensional matrix of the hyperspectral reflectivity, feature extraction is performed to obtain a feature vector, and the feature vector is applied to training and verification of a classification or regression model to obtain a high-precision classification or regression model; finally, the feature vector is mapped back to the hyperspectral wavelength to obtain a corresponding feature spectrum of the high-precision classification or regression model. Compared with a traditional dimensionality reduction method by reducing the number of spectral wavelengths, the method can maximize the information mining in the hyperspectral data, extract the feature spectrum corresponding to the problem, and improve the precision of the classification or regression model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hyperspectral data processing, and particularly relates to a feature spectrum extraction method based on dimensionality increase of hyperspectral data. BACKGROUND

[0002] Hyperspectral data refers to data collected in the visible light band, infrared band or other electromagnetic wave bands, which has hundreds or thousands of continuous spectral wavelength information. This data form provides researchers with a new observation method, enabling us to more comprehensively obtain sample information from multiple spectral dimensions. At present, hyperspectral data has been widely used in remote sensing, environmental monitoring, agriculture, geological exploration, medicine, industry and other fields.

[0003] Due to the problems of high dimensionality, large data volume and noise interference of hyperspectral data, the processing efficiency is low. In order to solve these problems, researchers use machine learning and statistics and other methods to process hyperspectral data, such as traditional principal component analysis and linear discriminant analysis. These methods reduce the number of hyperspectral wavelengths to improve processing efficiency, but do not completely extract the information in the hyperspectral data, making it difficult to further improve the accuracy of classification or regression models.

[0004] With the continuous development of computer technology and hardware, the amount of calculation required to process hyperspectral data is no longer a problem. How to better utilize these massive data to improve the ability of the observed target is the focus of the next research. In this background, an operation method opposite to dimensionality reduction is proposed - dimensionality increase of hyperspectral data. The hyperspectral data is increased from one-dimensional hyperspectral reflectivity curve to two-dimensional hyperspectral matrix, which maximizes the extraction of effective information in the hyperspectral data. This dimensionality increase method can effectively improve the display ability of hyperspectral data and improve the accuracy of classification or regression models. SUMMARY

[0005] The present application provides a feature spectrum extraction method based on dimensionality increase of hyperspectral data, which increases the hyperspectral data from one-dimensional reflectivity curve to two-dimensional matrix, maximizes the extraction of effective information in the hyperspectral data, and improves the accuracy of classification or regression models.

[0006] To solve the above technical problems, the present application adopts the following technical scheme, a feature spectrum extraction method based on dimensionality increase of hyperspectral data, which comprises the following steps:

[0007] a. Using a spectrometer to collect hyperspectral radiance of the measured sample and a white board;

[0008] b. Preprocessing the hyperspectral radiance of the measured sample and the white board;

[0009] c. Calculate the hyperspectral one-dimensional reflectivity of the measured sample using the hyperspectral radiance of the measured sample and the white board;

[0010] d. Dimensionality of the hyperspectral one-dimensional reflectivity of the measured sample, using the hyperspectral one-dimensional reflectivity of the measured sample and the white board;

[0011] e. Feature extraction of the hyperspectral two-dimensional matrix A, to obtain the feature vector B;

[0012] f. Apply the feature vector B to the training and verification of the classification or regression model, to obtain the corresponding high-precision classification or regression model;

[0013] g. Map the feature vector B back to the corresponding wavelength of the hyperspectral data, to obtain the characteristic spectrum of the corresponding high-precision classification or regression model.

[0014] Further, the mathematical operation in step d can be basic mathematical operations such as addition, subtraction, multiplication and division, or more complex mathematical operations.

[0015] Further, the number of wave bands used in step d for mathematical operation can be greater than or equal to two.

[0016] Further, the feature extraction of the hyperspectral two-dimensional matrix A in step e to obtain the feature vector B includes but is not limited to principal component analysis, random forest and mutual information method.

[0017] Advantages:

[0018] The advantages of the present application over the prior art are:

[0019] (1) The present application can maximize the effective information in the hyperspectral data, effectively improve the display ability of the hyperspectral data, and improve the accuracy of the classification or regression model.

[0020] (2) The dimensionality, mathematical operation and feature extraction steps in the method of the present application can be flexibly adjusted and combined according to the specific scene, and have good practicability and effect. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only intended to explain the preferred embodiments and are not considered as a limitation to the present application. Moreover, the same reference signs are used to indicate the same components throughout the drawings. In the drawings:

[0022] Figure 1 A flow chart of the feature spectrum extraction method based on hyperspectral data dimensionality increase disclosed by the embodiments of the present application;

[0023] Figure 2 A hyperspectral reflectance curve diagram of Cordyceps sinensis in Naqu three counties;

[0024] Figure 3 A hyperspectral one-dimensional reflectance curve of a Cordyceps sinensis sample;

[0025] Figure 4 A normalized spectral difference index visualization image of a Cordyceps sinensis sample. DETAILED DESCRIPTION

[0026] The technical solutions of the present application will be further described in detail below by means of the drawings and embodiments. Referring to Figure 1 The present application takes the production areas of Cordyceps sinensis in Naqu three counties (such as Baidai County, Suo County, and Jiali County) as embodiments.

[0027] In step S201, the hyperspectral radiance of all Cordyceps sinensis samples in Naqu three counties and a white plate is collected using a spectrometer, wherein the hyperspectral radiance including VNIR 108 wave bands and SWIR 288 wave bands is collected.

[0028] In step S202, the hyperspectral radiance of all Cordyceps sinensis samples in Naqu three counties is preprocessed.

[0029] In step S203, the hyperspectral one-dimensional reflectance of Cordyceps sinensis samples in Naqu three counties is calculated using the hyperspectral radiance of Cordyceps sinensis samples in Naqu three counties and the white plate.

[0030] Wherein, Figure 2 A hyperspectral reflectance curve diagram of Cordyceps sinensis in Naqu three counties.

[0031] In step S204, the hyperspectral one-dimensional reflectance of Cordyceps sinensis samples in Naqu three counties is dimensionality increased, and A(i,j) is obtained by performing mathematical operation on the reflectance of the i th wavelength and the j th wavelength of the hyperspectral one-dimensional reflectance. A(i,j) is calculated for N wavelengths of hyperspectral reflectance, and N×N hyperspectral two-dimensional matrix A is obtained, so as to realize the dimensionality increase of hyperspectral data from one-dimensional reflectance curve to two-dimensional matrix.

[0032] Wherein, the mathematical operation used is the difference between the two bands of hyperspectral reflectance divided by the sum of the two bands of hyperspectral reflectance, see formula (1), to obtain the spectral difference index (SDI).

[0033]

[0034] Wherein, Band1 represents the spectral reflectance data of one wavelength, and Band2 represents the spectral reflectance data of another wavelength.

[0035] Continue to normalize the spectral difference index using formula (2), the normalization method is shown in formula (2), and finally obtain the normalized spectral difference index (NSDI), which is the hyperspectral two-dimensional matrix A.

[0036]

[0037] Wherein, SDI represents the spectral difference index, SDI max represents the maximum value of the spectral difference index, SDI min represents the minimum value of the spectral difference index.

[0038] Specifically, the hyperspectral one-dimensional reflectance curve of a cordyceps sinensis sample can be seen in Figure 3 , and the normalized spectral difference index visualization image of a cordyceps sinensis sample can be seen in Figure 4 .

[0039] In step S205, the hyperspectral two-dimensional matrix A is subjected to feature extraction to obtain the feature vector B, and the extraction method is to use mutual information to calculate the mutual information between each feature and the target variable, to evaluate the correlation between the feature and the target variable, so as to select the feature with the highest correlation.

[0040] In step S206, the feature vector B is applied to the training and verification of the cordyceps sinensis naqu three county origin classification model, to obtain the high-precision classification model of cordyceps sinensis naqu three county origin.

[0041] In step S207, the feature vector B is mapped back to the corresponding wavelength of the hyperspectrum, to obtain the characteristic spectrum of the high-precision classification model of cordyceps sinensis naqu three county origin.

[0042] Through the embodiment, it can be seen that the feature spectrum extraction method provided by the application has good practicability and effect in the application scene of distinguishing the three counties of Naqu in Ophiocordyceps sinensis. Meanwhile, the steps of dimension increasing, mathematical operation and feature extraction in the method can be flexibly adjusted and combined according to specific scenes, and the universality and scalability are good.

[0043] Specifically, the random forest (RF), support vector machine (SVM), gradient boosting tree (XGBoost), naive Bayes (NBC) and logistic regression (LR) five kinds of machine learning models are used for training and testing on the hyperspectral data reflectivity, two-dimensional matrix A and two-dimensional feature matrix B of Ophiocordyceps sinensis from different production areas. Among the distribution of samples, there are 55 samples of Ophiocordyceps sinensis in Naqu three counties, 15 samples in Biecun county, 20 samples in Jiali county and 20 samples in Suo county, and the training set: test set is 8:1. The same samples are used for training and testing in training and testing. The result accuracy can be seen in table 1.

[0044] Table 1 verification accuracy of models using different data under different models

[0045]

[0046] The results show that the performance of the hyperspectral data reflectivity model is poor. When using the two-dimensional hyperspectral matrix A, the accuracy of the model is improved obviously, especially the SVM and LR models. After feature extraction on the two-dimensional hyperspectral matrix A, the accuracy of the model is improved when using the feature vector B. In future research, more complex machine learning models such as neural network will be combined, and the accuracy of the model will be further improved.

[0047] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the application. It should be understood that the above description is only a specific embodiment of the application and is not used to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. A feature spectrum extraction method based on hyperspectral data dimensionality increase, characterized in that, The method comprises: a. Using a spectrometer to collect hyperspectral radiance of the measured sample and a white board; b. Preprocessing the hyperspectral radiance of the measured sample and the white board; c. Using the hyperspectral radiance of the measured sample and the white board, calculating the hyperspectral one-dimensional reflectivity of the measured sample; d. Dimensionality of the hyperspectral one-dimensional reflectivity of the measured sample is increased, and A(i,j) is obtained by mathematical operation of the reflectivity of the i-th wavelength and the j-th wavelength of the hyperspectral one-dimensional reflectivity, A(i,j) is calculated for N wavelengths of hyperspectral reflectivity, and a N×N hyperspectral two-dimensional matrix A is obtained, realizing the dimensionality increase of hyperspectral data from one-dimensional reflectivity curve to two-dimensional matrix; e. Feature extraction is performed on the hyperspectral two-dimensional matrix A to obtain a feature vector B; f. The feature vector B is applied to the training and verification of a classification or regression model to obtain a corresponding high-precision classification or regression model; g. The feature vector B is mapped back to the corresponding wavelength of the hyperspectral to obtain the characteristic spectrum of the corresponding high-precision classification or regression model; Wherein, the mathematical operation is the difference between the reflectivity of two bands of hyperspectral divided by the sum of the reflectivity of two bands of hyperspectral, see formula (1), to obtain the spectral difference index (SDI); （1） wherein spectral reflectance data representing a wavelength, spectral reflectance data representing another wavelength; The spectral difference index is further normalized using formula (2), and the normalization method is shown in formula (2), and finally the normalized spectral difference index (NSDI) is obtained, that is, the hyperspectral two-dimensional matrix A; （2） wherein, representing a spectral difference index, representing a maximum value of the spectral difference index, representing a minimum value of the spectral difference index.

2. The method of claim 1, wherein: In step e, the feature extraction is performed on the hyperspectral two-dimensional matrix A to obtain the feature vector B, and the extraction method is selected from principal component analysis, random forest, and mutual information method.

Images