Method for removing abnormal spectrum in actual measurement spectrum curve based on support vector machine model

Abstract

The invention discloses a method for removing an abnormal spectrum in an actual measurement spectrum curve based on a support vector machine model. A threshold value is set by using the support vector machine classification method thought in the machine learning theory to remove the abnormal spectrum, automatic parameter optimization is carried out through a cross validation method to find out the optimal model parameter, then spectrum data are classified, the problems that due to the fact that the threshold value is set manually or is constantly adjusted, subjectivity is low, and efficiency is low are solved, the method can be applied to processing mass spectrum data in a large-scale mode, and precision and accuracy are effectively improved. A selected RBF kernel function has the advantages of being high in generalization ability and high in convergence rate. The parameter selection step for carrying out optimization on a penalty coefficient C and the interval parameter gamma in the RBF kernel function is further added, an SVM dichotomy algorithm model, namely, the support vector machine model, is built in combination with a training spectrum, and the final result of removing the abnormal spectrum is further optimized.

Description

technical field [0001] The invention relates to a method for removing abnormal spectra in measured spectral curves, in particular to a method for removing abnormal spectra by setting a threshold based on the principle of a support vector machine classification method. Background technique [0002] Spectral analysis technology is widely used in geological remote sensing, agriculture and forestry ecology, soil investigation, pollution monitoring, food testing, metrological chemistry and other fields. During the measurement process, due to the influence of instrument noise, external environmental interference, improper operation and other accidental factors, There are often abnormal spectra in the spectral data of , if these data are directly used for modeling analysis, it will inevitably affect the accuracy and stability of the model. Therefore, it is necessary to identify abnormal spectra and remove them. [0003] Many experts and scholars at home and abroad have done corres...

AI Technical Summary

Problems solved by technology

There are two defects in the above processing method. One is to compare the differences between spectra by using conventional similarity measurement methods such as the Mahalanobis distance, which cannot completely distinguish the spectral differences; the other is to artificially set the threshold or continuously adjust the threshold. The experience of the author is too subjective and the efficiency is not high, so it is difficult to apply to the processing of a large amount of spectral data

Images