An adaptive feature compression and credible identification method for high-throughput raman spectrum

Abstract

The application discloses a self-adaptive feature compression and credible identification method for high-throughput Raman spectrum, and belongs to the technical field of Raman spectrum data processing.The application constructs a spectrum quality evaluation mechanism based on a spectral line signal-to-noise ratio, a baseline drift degree, an abnormal peak interference, a peak shape integrity and a spectral line saturation state, introduces a double-threshold layered mechanism of a high-quality discrimination threshold and a low-quality discrimination threshold, divides the Raman spectrum into high-quality spectrum, repairable spectrum and low-quality spectrum, avoids the limitation that a traditional single-threshold method can only make binary judgment of effectiveness / ineffectiveness, and improves the fine degree of spectrum quality control; the repairable spectrum in the middle quality interval is not directly eliminated, but is subjected to enhanced adaptive pretreatment according to the quality defect type, and is subjected to quality re-inspection after the pretreatment, so that the repairable spectrum can re-enter an effective Raman spectrum data set, thereby reducing sample loss caused by excessive elimination and improving the effective utilization rate of high-throughput Raman spectrum data.

Description

Technical Field

[0001] This invention belongs to the field of Raman spectroscopy data processing technology, specifically relating to an adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy. Background Technology

[0002] Raman spectroscopy is an important non-destructive detection method for analyzing the molecular structure, chemical bond state, and material composition of samples based on changes in the scattered signal after the interaction of light and matter. It boasts advantages such as simple sample preparation, fast detection speed, rich information, and wide applicability, and has been widely applied in biomedical detection, drug screening, chemical analysis, environmental monitoring, and materials characterization. With the increasing automation of Raman detection equipment, high-throughput batch acquisition has become a significant development direction for Raman spectroscopy applications, generating a large amount of Raman spectroscopy data daily in related research and industrial settings. Due to the complex sources of samples and diverse testing conditions, different Raman spectra often exhibit significant differences in peak shape, peak intensity, background drift, and noise levels, resulting in high-throughput Raman spectroscopy data characterized by large volume, high dimensionality, inconsistent quality, and difficulty in interpretation.

[0003] In existing technologies, various machine learning methods such as support vector machines, random forests, and K-nearest neighbors have been proposed for Raman spectral identification and analysis to achieve spectral classification or component discrimination. However, these methods typically model raw spectra or high-dimensional data after simple preprocessing, lacking an effective screening mechanism for spectral quality differences in high-throughput scenarios. Low-quality spectra, such as abnormal peaks, baseline drift, spectral saturation, and low signal-to-noise ratio, are prone to entering subsequent analysis processes, affecting feature extraction and model recognition performance. Furthermore, existing dimensionality reduction methods often use fixed principal component numbers or empirically set parameters for feature compression, making it difficult to adaptively retain effective discrimination information based on the complexity of different batches and categories of Raman spectra. This can easily lead to the loss of key features or redundant noise residue, thus limiting the improvement of classification and recognition performance.

[0004] Meanwhile, existing technologies in high-throughput Raman spectroscopy identification typically focus more on the classification results themselves, while paying insufficient attention to the reliability of the identification results. For samples with similar composition, similar spectral peaks, or poor signal quality, existing methods often directly output classification labels, lacking a confidence determination mechanism based on category probability distribution. This makes it difficult to effectively distinguish between high-confidence and low-confidence identification results, and also makes it difficult to trigger subsequent rejection, re-examination, or manual confirmation processes. This means that these methods still have the risk of misjudgment in complex sample detection, fine identification, and practical online applications, affecting their engineering practicality and promotional value.

[0005] Therefore, there is an urgent need to provide an intelligent identification method for high-throughput Raman spectroscopy, which can first evaluate the spectral quality and remove abnormal spectra after batch acquisition of Raman spectra, and then combine an adaptive feature compression method to achieve effective characterization of high-dimensional spectra. Furthermore, an identification confidence judgment mechanism should be introduced in the classification and identification process to improve the utilization efficiency, identification accuracy, and reliability of high-throughput Raman spectroscopy data. Summary of the Invention

[0006] The purpose of this invention is to address the problems existing in the background technology by proposing an adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy. Firstly, this invention constructs a spectral quality assessment mechanism based on spectral line signal-to-noise ratio, baseline drift, abnormal peak interference, peak shape integrity, and spectral line saturation state to achieve automatic identification and removal of abnormal spectra. It then adaptively adjusts baseline correction, smoothing and denoising, wavenumber alignment, and intensity normalization parameters based on the quality score results to improve the standardization effect of Raman spectral data of different qualities. On this basis, an adaptive feature compression method based on the combined constraints of cumulative contribution rate and classification performance is adopted to dynamically determine the number of principal components to be retained. Furthermore, it integrates peak position, peak height, peak area, or peak width parameters of characteristic peaks to enhance the retention of key discriminative information. Simultaneously, an optimal identification model is determined by constructing a candidate classification model pool and a comprehensive evaluation mechanism. Finally, a confidence criterion is established by combining the maximum posterior probability, the difference between the probabilities of the first two classes, or the prediction entropy, enabling the rejection, re-examination, or manual confirmation process for low-confidence samples. Compared with traditional Raman spectroscopy identification methods, this invention can not only effectively solve the problems of uneven spectral quality, severe dimensional redundancy, insufficient feature utilization and poor reliability of identification results in high-throughput Raman spectroscopy processing, but also has the advantages of strong adaptability, high identification accuracy and strong engineering applicability, and can realize efficient, accurate and reliable identification of complex Raman spectral data.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] An adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy includes the following steps:

[0009] Step 1. Perform batch measurements on the samples to be tested using Raman spectroscopy acquisition equipment, and construct a two-dimensional spectral matrix based on the obtained raw Raman spectral data:

[0010]

[0011] in, Indicates the number of Raman spectra. This indicates the number of wavenumber sampling points corresponding to each spectrum. Indicates the first The spectrum in the first Intensity values ​​at each wavenumber sampling point;

[0012] Construct a two-dimensional spectral matrix with corresponding category label vectors for each row:

[0013]

[0014] in, Indicates the first Category labels corresponding to striped Raman spectra;

[0015] Step 2. Perform spectral quality assessment on the Raman spectrum dataset with category labels obtained in Step 1. Construct quality discrimination indexes based on spectral signal-to-noise ratio (SNR), baseline drift (BDF), abnormal peak interference (CRR), peak integrity (PSF), and spectral saturation state (SDR) to obtain a comprehensive quality score for each Raman spectrum.

[0016] Based on the mean and standard deviation of the overall quality scores of all Raman spectra in the current batch, high-quality discrimination thresholds and low-quality discrimination thresholds are constructed. Based on the high-quality discrimination thresholds and low-quality discrimination thresholds, Raman spectra are divided into high-quality spectra, repairable spectra, and low-quality spectra. Among them, high-quality spectra are subjected to conventional preprocessing, repairable spectra are subjected to enhanced adaptive preprocessing, and low-quality spectra are eliminated or triggered for reacquisition.

[0017] Step 3. Perform quality re-inspection on the repairable spectra after enhanced adaptive preprocessing; combine the high-quality spectra obtained in Step 2 and the repairable spectra that have passed the re-inspection after enhanced adaptive preprocessing to form an effective Raman spectrum dataset; perform baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization on the effective Raman spectrum dataset to obtain standardized Raman spectrum data;

[0018] Step 4. Adaptive feature compression method based on the joint constraints of cumulative contribution rate and classification performance is used to perform principal component analysis to reduce the dimensionality of the standardized Raman spectral data obtained in Step 3, and extract the peak position, peak height, peak area and peak width parameters of the characteristic peaks, and fuse them with the retained principal component features to obtain a compressed feature vector for classification and recognition.

[0019] Step 5. Input the compressed feature vector obtained in Step 4 into the candidate classification model pool for training and validation. The candidate classification model pool includes at least two different types of classification models. The training set and test set are divided by stratified sampling. A comprehensive evaluation index is constructed by combining cross-validation, accuracy, precision, recall, F1 score, AUC-ROC and training time to determine the optimal recognition model.

[0020] Step 6. Use the optimal recognition model determined in Step 5 to predict the category of the Raman spectrum to be identified, and output the corresponding category probability and visualization spectrum.

[0021] Furthermore, step 2 is specifically as follows:

[0022] 2.1 For each spectrum in the category-labeled Raman spectroscopy dataset obtained in step 1, construct a quality scoring function:

[0023]

[0024] in, Indicates the first Quality score of the spectrum Indicates the first The signal-to-noise ratio of the spectrum. Indicates the first The degree of baseline shift in the spectrum, Indicates the first The degree of interference from abnormal peaks in the spectrum. Indicates the first Peak shape integrity index of a spectrum Indicates the first The saturation distortion index of the spectrum Let be the corresponding weight coefficients, and satisfy:

[0025]

[0026] 2.2 To avoid the traditional single threshold method from directly rejecting Raman spectra that are in an intermediate quality state but can still be repaired through preprocessing, a dual threshold hierarchical discrimination mechanism is constructed based on the mean and standard deviation of the quality scores of all Raman spectra in the current batch;

[0027] 2.2.1 Calculate all Raman spectra ( Mean of quality score for (striped Raman spectra) and standard deviation And determine the high-quality discrimination threshold T. H and low quality discrimination threshold T L :

[0028]

[0029]

[0030]

[0031] in, and The preset adjustment coefficient, and > ;

[0032] 2.2.2 When the first Spectrum satisfies ≥ The spectrum was determined to be of high quality and underwent routine preprocessing; when the... Spectrum satisfies ≤ < The spectrum was determined to be repairable and subjected to enhanced adaptive preprocessing based on its quality defect type; when the... Spectrum satisfies < If the spectrum is identified as low-quality, it will be removed or a new acquisition will be triggered. Among them, the recoverable spectrum refers to the Raman spectrum that has certain noise, baseline drift, abnormal peaks, incomplete peak shape or saturation distortion trend, but may meet the requirements of subsequent analysis after enhanced preprocessing.

[0033] Preferably, in step 2.2.1, 0.5 ≤ ≤1.5, 1.5≤ ≤3.0.

[0034] Furthermore, in step 2.2.2, the high-quality spectrum is subjected to routine preprocessing, which includes baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization.

[0035] Furthermore, in step 2.2.2, the repairable spectrum is subjected to enhanced adaptive preprocessing according to the type of quality defect. The enhanced adaptive preprocessing process is as follows:

[0036] When the signal-to-noise ratio of the repairable spectrum is lower than the preset signal-to-noise ratio requirement, increase the smoothing and denoising window or increase the denoising intensity; when the baseline drift is higher than the preset baseline drift limit, increase the baseline correction intensity or adjust the baseline fitting parameters; when the abnormal peak interference is higher than the preset peak limit, use neighborhood median filtering or interpolation to correct the peak region; when the peak shape integrity index is lower than the preset peak shape integrity requirement, perform local smoothing and peak shape correction on the target characteristic peak region; when the saturation distortion index is higher than the preset saturation limit, remove the corresponding spectrum, reduce its weight, or trigger re-acquisition.

[0037] Furthermore, step 3 is specifically as follows:

[0038] 3.1 The quality of the repairable spectrum after enhanced adaptive preprocessing was rechecked;

[0039] Recalculate the quality score of the repairable spectrum after enhanced adaptive preprocessing. ,when ≥ or ≥ At that time, the repairable spectrum after enhanced adaptive preprocessing was considered acceptable, among which... Set the preset quality improvement threshold; otherwise, discard.

[0040] 3.2 Combine the high-quality spectra obtained in step 2 with the repairable spectra that have passed the re-examination after enhanced adaptive preprocessing to form an effective Raman spectrum dataset;

[0041] 3.3 Baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization are performed on the effective Raman spectrum dataset to obtain standardized Raman spectrum data.

[0042] Furthermore, step 4 is specifically as follows:

[0043] 4.1 Perform principal component analysis on the standardized Raman spectral data obtained in step 3 to obtain the eigenvalue sequence sorted by contribution from largest to smallest. For the number of candidate principal components p, its cumulative contribution rate Represented as:

[0044]

[0045] Where p represents the number of principal components retained, and w represents the total number of principal components. This represents the eigenvalue corresponding to the j-th principal component;

[0046] 4.2 Set a cumulative contribution rate threshold C0 and calculate the cumulative contribution rate corresponding to different numbers of candidate principal components p. When different candidate principal component numbers p satisfy When C ≥ C0, it is considered as a candidate to retain principal components;

[0047] 4.3 Obtain the number of all candidate principal components using the method in step 4.2, and construct principal component features based on all candidate principal component numbers; input the principal component features into the classification model for validation, and determine the final number of retained principal components p based on classification accuracy, F1 score, and AUC-ROC evaluation metrics. ∗ ;

[0048] 4.4 Extract the peak position, peak height, peak area, and peak width parameters of characteristic peaks from standardized Raman spectral data to construct the spectral peak feature vector. ; Combine the spectral peak eigenvectors with the top p values ​​sorted by contribution from largest to smallest. ∗ The principal component eigenvector is composed of 1 eigenvalues. The fusion process yields a compressed feature vector for classification and recognition:

[0049] .

[0050] Furthermore, in step 5, the candidate classification model pool includes at least two different types of classification models, including any two or more of the following classification models: Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), and Lightweight Gradient Boosting Machine. The compressed feature vectors obtained in step 4 are divided into training and test sets using stratified sampling, and cross-validation is performed on the training set to obtain the accuracy, precision, recall, F1 score, AUC-ROC, and training time of each candidate classification model.

[0051] Furthermore, the comprehensive evaluation index constructed in step 5 is as follows:

[0052]

[0053] Among them, S m This represents the overall evaluation score of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the recall rate of the m-th candidate classification model. This represents the F1 score of the m-th candidate classification model. This represents the area under the ROC curve (AUC-ROC) of the m-th candidate classification model. This represents the training time of the m-th candidate classification model. Let be the weight coefficient of the corresponding indicator, and satisfy:

[0054]

[0055] The candidate classification model with the highest comprehensive evaluation score is selected as the optimal recognition model.

[0056] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are as follows:

[0057] 1. This invention constructs a spectral quality assessment mechanism based on spectral line signal-to-noise ratio, baseline drift, abnormal peak interference, peak shape integrity, and spectral line saturation state. Furthermore, it introduces a dual-threshold stratification mechanism consisting of a high-quality discrimination threshold and a low-quality discrimination threshold to divide Raman spectra into high-quality spectra, repairable spectra, and low-quality spectra. This avoids the limitation of traditional single-threshold methods that can only make binary judgments of valid / invalid, and improves the precision of spectral quality control.

[0058] 2. This invention does not directly discard repairable spectra in the intermediate quality range. Instead, it performs enhanced adaptive preprocessing based on the type of quality defect and conducts quality re-inspection after preprocessing. This allows repairable spectra to re-enter the effective Raman spectral dataset, thereby reducing sample loss caused by excessive rejection and improving the effective utilization rate of high-throughput Raman spectral data.

[0059] 3. The present invention is an adaptive feature compression method based on the combined constraints of cumulative contribution rate and classification performance. It dynamically determines the number of principal components to be retained, and integrates the peak position, peak height, peak area or peak width parameters of the feature peaks. This effectively enhances the ability to retain key discriminative information of Raman spectroscopy and improves the feature expression effect and classification recognition accuracy.

[0060] 4. By associating the spectral quality stratification results with the preprocessing strategy, this invention can prevent low-quality spectra from entering the subsequent principal component analysis and classification model, while retaining medium-quality spectra that meet the requirements after repair. This reduces the impact of abnormal spectra, noise interference and baseline drift on model training and recognition results, and improves the stability and reliability of recognition results. Attached Figure Description

[0061] Figure 1 This is a flowchart of the spectral identification process according to an embodiment of the present invention;

[0062] Figure 2 This is a bar graph of principal component features extracted from the spectrum of an actual sample, as shown in an embodiment of the present invention.

[0063] Figure 3 This is a performance comparison chart of the various models used in the embodiments of the present invention;

[0064] Figure 4 This is a confusion matrix diagram of the test results of actual samples in an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0066] Example

[0067] An adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy includes the following steps:

[0068] Step 1. Using a Raman spectroscopy acquisition device, batch measurements of creatinine, glucose, and uric acid were performed. The number of sampling points for each spectrum was set to 1024 points to obtain raw Raman spectral data, which were then organized into a two-dimensional spectral matrix.

[0069]

[0070] in, Indicates the number of Raman spectra. This indicates the number of wavenumber sampling points corresponding to each spectrum. Indicates the first The spectrum in the first Intensity values ​​at each wavenumber sampling point; where uric acid is... Creatinine is Glucose is Simultaneously, construct the category label vector corresponding to each row of the two-dimensional spectral matrix:

[0071]

[0072] in, Indicates the first The category labels corresponding to the Raman spectra are shown below, where y corresponds to uric acid, creatinine, and glucose, respectively.

[0073] Step 2. Evaluate the spectral quality of the category-labeled Raman spectroscopy dataset obtained in Step 1;

[0074] 2.1 For each spectrum in the category-labeled Raman spectroscopy dataset obtained in step 1, construct a quality scoring function:

[0075]

[0076] in, Indicates the first Quality score of the spectrum Indicates the first The signal-to-noise ratio of the spectrum. Indicates the first The degree of baseline shift in the spectrum, Indicates the first The degree of interference from abnormal peaks in the spectrum. Indicates the first Peak shape integrity index of a spectrum Indicates the first The saturation distortion index of the spectrum Let be the corresponding weight coefficients, and satisfy:

[0077]

[0078] Here All values ​​are 0.2;

[0079] 2.2 To avoid the traditional single threshold method from directly rejecting Raman spectra that are in an intermediate quality state but can still be repaired through preprocessing, a dual threshold hierarchical discrimination mechanism is constructed based on the mean and standard deviation of the quality scores of all Raman spectra in the current batch;

[0080] 2.2.1 Calculate all Raman spectra ( Mean of quality score for (striped Raman spectra) and standard deviation And determine the high-quality discrimination threshold T. H and low quality discrimination threshold T L :

[0081]

[0082]

[0083]

[0084] in, and The preset adjustment coefficient, and > In the embodiments, ;

[0085] 2.2.2 When the first Spectrum satisfies ≥ The spectrum was determined to be of high quality and underwent routine preprocessing; when the... Spectrum satisfies ≤ < The spectrum was determined to be repairable and subjected to enhanced adaptive preprocessing based on its quality defect type; when the... Spectrum satisfies < Spectra deemed low-quality are discarded or reacquisition is triggered. Repairable spectra refer to Raman spectra exhibiting noise, baseline drift, abnormal peaks, incomplete peak shapes, or saturation distortion trends, but which may meet subsequent analysis requirements after enhanced preprocessing. The enhanced adaptive preprocessing process is as follows: when the signal-to-noise ratio (SNR) of a repairable spectrum is lower than the preset SNR requirement, the smoothing and denoising window is increased or the denoising intensity is improved; when the baseline drift exceeds the preset baseline drift limit, the baseline correction intensity is increased or the baseline fitting parameters are adjusted; when the abnormal peak interference exceeds the preset peak limit, neighborhood median filtering or interpolation is used to correct the peak region; when the peak shape integrity index is lower than the preset peak shape integrity requirement, local smoothing and peak shape correction are performed on the target characteristic peak region; when the saturation distortion index exceeds the preset saturation limit, the corresponding spectrum is discarded, weighted, or reacquisition is triggered.

[0086] Step 3. Recalculate the quality score of the repairable spectrum after enhanced adaptive preprocessing. ,when ≥ or ≥ At that time, the repairable spectrum after enhanced adaptive preprocessing was considered acceptable, among which... Set the preset quality improvement threshold; otherwise, discard; combine the high-quality spectra obtained in step 2 and the repairable spectra that have passed the re-inspection after enhanced adaptive preprocessing to form an effective Raman spectrum dataset; perform baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization on the effective Raman spectrum dataset to obtain standardized Raman spectrum data;

[0087] Step 4. Obtaining the compressed feature vector;

[0088] 4.1 Perform principal component analysis on the standardized Raman spectral data obtained in step 3 to obtain the eigenvalue sequence sorted by contribution from largest to smallest. For the number of candidate principal components p, its cumulative contribution rate Represented as:

[0089]

[0090] Where p represents the number of principal components retained, and w represents the total number of principal components. This represents the eigenvalue corresponding to the j-th principal component;

[0091] 4.2 Set a cumulative contribution rate threshold C0 and calculate the cumulative contribution rate corresponding to different numbers of candidate principal components p. When different candidate principal component numbers p satisfy When C ≥ C0, it is considered as a candidate to retain principal components;

[0092] 4.3 Obtain the number of all candidate principal components using the method in step 4.2, and construct principal component features based on all candidate principal component numbers; input the principal component features into the classification model for validation, and determine the final number of retained principal components p based on classification accuracy, F1 score, and AUC-ROC evaluation metrics. ∗ =28;

[0093] 4.4 Extract the peak position, peak height, peak area, and peak width parameters of characteristic peaks from standardized Raman spectral data to construct the spectral peak feature vector. The principal component eigenvector is formed by combining the spectral peak eigenvector with the top 28 eigenvalues ​​sorted by contribution from largest to smallest. The fusion process yields a compressed feature vector for classification and recognition:

[0094] ;

[0095] Figure 2The bar chart shows the principal component features. PCA14 and PCA2 principal components contribute most significantly to the classification, playing a dominant role in the model's discrimination process. This indicates that the spectroscopic information contained in these two principal components has a strong discriminative ability to distinguish different analytes. Besides the principal components, features such as PCA10, PCA12, PCA5, and PCA16 also have high weights. The main model classification does not rely primarily on a single variable but rather comprehensively utilizes the spectral differences reflected by multiple principal components. Therefore, PCA features can effectively retain information related to substance identification, providing support for the model to achieve stable classification.

[0096] Step 5. Input the compressed feature vector obtained in Step 4 into the candidate classification model pool for training and validation. In this embodiment, the classification models selected include Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), and Lightweight Gradient Boosting Machine classification models. The training set and test set are divided by stratified sampling. A comprehensive evaluation index is constructed by combining cross-validation and accuracy, precision, recall, F1 score, AUC-ROC, and training time to determine the optimal recognition model. The relevant test data values ​​are shown in Table 1 below.

[0097] Table 1 Evaluation results for each model

[0098] LR 0.3183 0.2282 0.3183 0.2621 0.4385 0.6127 SVM 0.9196 0.9296 0.9196 0.9184 0.9823 22.1599 RF 0.9598 0.9636 0.9598 0.9600 0.9984 35.1554 KNN 0.7363 0.7499 0.7363 0.7222 0.9119 0.8051 XGB 0.9823 0.9825 0.9823 0.9824 0.9994 10.6260 LGBM 0.9791 0.9796 0.9791 0.9792 0.9993 23.8348

[0099] Figure 3 This provides a more intuitive representation of the classification performance of different models on the test set. Overall, ensemble learning models performed better, with the XGBoost model standing out the most, achieving the best results across all evaluation metrics, with accuracy, F1 score, and AUC-ROC reaching 98.23%, 98.24%, and 99.94%, respectively. The LGBM and RF models also performed similarly, with accuracies of 97.91% and 95.98%, respectively, indicating that ensemble learning methods are well-suited for handling this type of complex spectral data. In contrast, the SVM model showed more stable overall performance, with an accuracy of 91.96%. The KNN model's accuracy was only 73.63%, which may be related to its sensitivity to feature scale and susceptibility to noise.

[0100] The comprehensive evaluation index constructed in step 5 is as follows:

[0101]

[0102] Among them, S m This represents the overall evaluation score of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the recall rate of the m-th candidate classification model. This represents the F1 score of the m-th candidate classification model. This represents the area under the ROC curve (AUC-ROC) of the m-th candidate classification model. This represents the training time of the m-th candidate classification model. Let be the weight coefficient of the corresponding indicator, and satisfy:

[0103]

[0104] In the embodiments, The value was set to 0.2, and the rest were set to 0.15. The candidate classification model with the highest comprehensive evaluation score was selected as the optimal recognition model. It was found that the extreme gradient boosting model has significant advantages.

[0105] Step 6. Use the extreme gradient boosting model determined in Step 5 to predict the category of the Raman spectra to be identified, including 210 glucose samples, 212 uric acid samples, and 200 creatinine samples. Output the corresponding category probabilities and visualized spectra.

[0106] Figure 4 As can be seen, the XGBoost model performs well in classifying all three types of samples. For glucose samples, 207 were correctly identified, achieving a classification accuracy of 98.6%; for uric acid samples, 206 were correctly classified, achieving an accuracy of 97.2%; and for creatinine samples, 198 were correctly identified, achieving an accuracy of 99.0%. Overall, the number of misclassified samples in each category is relatively small, with misclassifications mainly occurring between uric acid and glucose. This indicates that the XGBoost model can effectively distinguish the Raman spectral characteristics of different molecules, and the classification results are relatively stable.

[0107] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. An adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy, characterized in that, Includes the following steps: Step 1. Perform batch measurements on the samples using Raman spectroscopy acquisition equipment, and construct a two-dimensional spectral matrix based on the obtained Raman spectral data: in, Indicates the number of Raman spectra. This indicates the number of wavenumber sampling points corresponding to each spectrum. Indicates the first The spectrum in the first Intensity values ​​at each wavenumber sampling point; Construct a two-dimensional spectral matrix with corresponding category label vectors for each row: in, Indicates the first Category labels corresponding to striped Raman spectra; Step 2. Perform spectral quality assessment on the Raman spectral data with category labels obtained in Step 1. Construct quality discrimination indexes based on spectral signal-to-noise ratio, baseline drift, abnormal peak interference, peak integrity, and spectral saturation state to obtain a comprehensive quality score for each Raman spectrum. High-quality and low-quality discrimination thresholds were constructed based on the mean and standard deviation of the overall quality scores of all Raman spectra. Based on these thresholds, Raman spectra were classified into high-quality spectra, repairable spectra, and low-quality spectra. Repairable spectra were subjected to enhanced adaptive preprocessing, while low-quality spectra were either eliminated or triggered for reacquisition. Step 3. Perform quality re-inspection on the repairable spectra after enhanced adaptive preprocessing; combine the high-quality spectra obtained in Step 2 and the repairable spectra that have passed the re-inspection after enhanced adaptive preprocessing to form an effective Raman spectrum dataset; perform baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization on the effective Raman spectrum dataset to obtain standardized Raman spectrum data; Step 4. Adaptive feature compression method based on the joint constraints of cumulative contribution rate and classification performance is used to perform principal component analysis to reduce the dimensionality of the standardized Raman spectral data obtained in Step 3, and extract the peak position, peak height, peak area and peak width parameters of the characteristic peaks, and fuse them with the retained principal component features to obtain a compressed feature vector for classification and recognition. Step 5. Input the compressed feature vector obtained in Step 4 into the candidate classification model pool for training and validation. The candidate classification model pool includes at least two different types of classification models. The training set and test set are divided by stratified sampling. A comprehensive evaluation index is constructed by combining cross-validation, accuracy, precision, recall, F1 score, AUC-ROC and training time to determine the optimal recognition model. Step 6. Use the optimal recognition model determined in Step 5 to predict the category of the Raman spectrum to be identified, and output the corresponding category probability and visualization spectrum.

2. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 1, characterized in that, Step 2 is as follows: Step 2.1 For each spectrum in the category-labeled Raman spectroscopy dataset obtained in Step 1, construct a quality scoring function: in, Indicates the first Quality score of the spectrum Indicates the first The signal-to-noise ratio of the spectrum. Indicates the first The degree of baseline shift in the spectrum, Indicates the first The degree of interference from abnormal peaks in the spectrum. Indicates the first Peak shape integrity index of a spectrum Indicates the first The saturation distortion index of the spectrum Let be the corresponding weight coefficients, and satisfy: Step 2.2 Construct a dual-threshold hierarchical discrimination mechanism; Step 2.2.1 Calculate the mean quality score of all Raman spectra. and standard deviation And determine the high-quality discrimination threshold T. H and low quality discrimination threshold T L : in, and The preset adjustment coefficient, and > ; Step 2.2.2 When the first Spectrum satisfies ≥ It was determined to be a high-quality spectrum; when the first Spectrum satisfies ≤ < The spectrum was determined to be repairable and underwent enhanced adaptive preprocessing based on its quality defect type; when the... Spectrum satisfies < If the spectrum is identified as low-quality, it will be discarded or a re-acquisition will be triggered.

3. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 2, characterized in that, In step 2.2.1, 0.5 ≤ ≤1.5, 1.5≤ ≤3.

0.

4. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 2, characterized in that, In step 2.2.2, the process of performing enhanced adaptive preprocessing on the repairable spectrum according to the type of quality defect is as follows: When the signal-to-noise ratio of the repairable spectrum is lower than the preset signal-to-noise ratio requirement, increase the smoothing and denoising window or increase the denoising intensity; when the baseline drift is higher than the preset baseline drift limit, increase the baseline correction intensity or adjust the baseline fitting parameters; when the abnormal peak interference is higher than the preset peak limit, use neighborhood median filtering or interpolation to correct the peak region; when the peak shape integrity index is lower than the preset peak shape integrity requirement, perform local smoothing and peak shape correction on the target characteristic peak region; when the saturation distortion index is higher than the preset saturation limit, remove the corresponding spectrum, reduce its weight, or trigger re-acquisition.

5. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 1, characterized in that, Step 3 is as follows: Step 3.1 Perform a quality re-inspection on the repairable spectrum after enhanced adaptive preprocessing; Recalculate the quality score of the repairable spectrum after enhanced adaptive preprocessing. ,when ≥ or ≥ At that time, the repairable spectrum after enhanced adaptive preprocessing was deemed acceptable, among which... The preset quality improvement threshold; Otherwise, discard; Step 3.2 Combine the high-quality spectra obtained in Step 2 with the repairable spectra that have passed the re-examination after enhanced adaptive preprocessing to form an effective Raman spectrum dataset; Step 3.3 Perform baseline correction, smoothing and denoising, wavenumber alignment and intensity normalization on the effective Raman spectrum dataset to obtain standardized Raman spectrum data.

6. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 1, characterized in that, Step 4 is as follows: Step 4.1 Perform principal component analysis on the standardized Raman spectral data obtained in Step 3 to obtain the eigenvalue sequence sorted by contribution from largest to smallest. For the number of candidate principal components p, its cumulative contribution rate Represented as: Where p represents the number of principal components retained, and w represents the total number of principal components. This represents the eigenvalue corresponding to the j-th principal component; Step 4.2 Set the cumulative contribution rate threshold C0 and calculate the cumulative contribution rate corresponding to different numbers of candidate principal components p. When the number of principal components p satisfies When C ≥ C0, it is considered as a candidate to retain principal components; Step 4.3 Obtain the number of all candidate principal components using the method in Step 4.2, and construct principal component features based on all candidate principal component numbers; input the principal component features into the classification model for validation, and determine the final number of principal components p to be retained based on classification accuracy, F1 score, and AUC-ROC evaluation metrics. ∗ ; Step 4.4 Extract the peak position, peak height, peak area, and peak width parameters of characteristic peaks from the standardized Raman spectral data to construct the spectral peak feature vector. ; Combine the spectral peak eigenvectors with the top p values ​​sorted by contribution from largest to smallest. ∗ The principal component eigenvector is composed of 1 eigenvalues. The fusion process yields a compressed feature vector for classification and recognition: 。 7. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 1, characterized in that, In step 5, the candidate classification model pool includes at least two different types of classification models, including any two or more of the following classification models: logistic regression, support vector machine, random forest, K-nearest neighbors, extreme gradient boosting, and lightweight gradient boosting machine.

8. The adaptive feature compression and reliable identification method for high-throughput Raman spectroscopy according to claim 1, characterized in that, The comprehensive evaluation index constructed in step 5 is as follows: Among them, S m This represents the overall evaluation score of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the accuracy of the m-th candidate classification model. Let represent the recall rate of the m-th candidate classification model. This represents the F1 score of the m-th candidate classification model. Let AUC-ROC be the value of the m-th candidate classification model. This represents the training time of the m-th candidate classification model. Let be the weight coefficient of the corresponding indicator, and satisfy: The candidate classification model with the highest comprehensive evaluation score is selected as the optimal recognition model.

Images