Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A near-infrared anomaly spectral identification method based on knn

A technology for near-infrared spectroscopy and spectral identification, which is applied in the field of near-infrared anomaly spectral identification based on KNN, and can solve problems affecting the performance of near-infrared quantitative analysis models.

Active Publication Date: 2022-03-15
YANCHENG INST OF TECH
View PDF16 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0010] The purpose of the present invention is to solve the problem that the abnormal spectrum data affects the performance of the near-infrared quantitative analysis model, accurately and comprehensively better identify and eliminate the abnormal spectrum, thereby improving the accuracy and reliability of the near-infrared quantitative analysis prediction model and improving the model. The prediction accuracy of

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0026] Example 1: KNN abnormal spectrum identification method based on Euclidean distance metric

[0027] Euclidean distance KNN: Based on the K-nearest neighbor method of Euclidean distance, the Euclidean distance between the sample and its K-nearest neighbor sample is used as an anomaly measure;

[0028] Euclidean distance is the most commonly used distance measure and similarity measure between samples. The calculation of Euclidean distance is simple and fast, and it has advantages in calculation speed and algorithm implementation. It often becomes the preferred option in the case of similar performance. The outlier identification method based on Euclidean distance KNN is given below. On the one hand, it examines and verifies its ability to identify abnormal spectral data.

[0029] input: a The training set spectral data matrix X, n is the spectral sample, and p is the number of measured wavelength points.

[0030] 1) Select Euclidean distance as the similarity measure ...

Embodiment 2

[0038] Example 2: KNN abnormal spectrum identification method based on principal component space distance metric

[0039]Based on the PC-KNN method of principal component normalization space, the spectral data is subjected to principal component analysis, and on the basis of the data obtained after all principal components (PC) are standardized, the K-nearest neighbor method based on Euclidean distance is used.

[0040] Based on the similarity measure between samples based on Euclidean distance, the underlying assumption is that the variance of the sample distribution in each direction is basically equal. However, in practical applications, this condition may not necessarily be satisfied. The similarity measure between samples based on the Mahalanobis distance has no assumptions and requirements for the variance in each direction. However, the calculation of Mahalanobis distance is not suitable for occasions where small samples or data contain a large number of cross-correlat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a KNN-based near-infrared abnormal spectrum identification method, aiming at the problem that the existence of abnormal spectral data seriously affects the accuracy and reliability of the spectral analysis model in the near-infrared spectral analysis. The steps of the method include: selecting a similarity measure, selecting a hyperparameter k, calculating a distance measure between spectra, finding k-shortest distance samples, calculating a sample abnormal measure, sorting samples according to the abnormal measure, identifying and removing samples with high abnormal measure. The invention is mainly used for identifying and eliminating the abnormal spectrum in the construction of the near-infrared spectrum analysis model.

Description

technical field [0001] The invention relates to a KNN-based near-infrared abnormal spectrum identification method. Background technique [0002] Outliers are observed sample values ​​that are inconsistent with the majority of data patterns. Outlier identification is an integral part of any study based on empirical data. In many practical applications such as near-infrared spectroscopy, the data contains not only noise interference but also abnormal data, which leads to great deviations in the obtained model. The quality of training datasets, such as the existence of abnormal spectra, has become the key and bottleneck to improve the performance of near-infrared spectroscopy analysis models. In the detection of quality parameters based on near-infrared spectral analysis, the abnormal data includes abnormal spectral data and predicted abnormal measurement values ​​of quality parameters. Quality parameters are generally one-dimensional data, and the identification of outliers...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G01N21/359
CPCG01N21/359
Inventor 刘聪徐友武阳程
Owner YANCHENG INST OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products