A method for extracting baselines from NMR spectra based on improved DBSCAN
The improved DBSCAN algorithm clusters the nuclear magnetic spectrum data and automatically extracts the baseline, which solves the problem of inaccurate baseline settings in the prior art, and improves the accuracy and efficiency of nuclear magnetic spectrum analysis.
Patent Information
- Application Number
- CN202210508623.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-10
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2042-05-10
AI Technical Summary
The inaccurate reference line setting in the prior art in the nuclear magnetic resonance spectrum leads to errors in the analysis results, reducing the accuracy of substance detection, and the traditional methods are complex in operation and inefficient in efficiency.
The improved DBSCAN algorithm is used to cluster the nuclear magnetic spectrum data. By extracting the peak vertex coordinates and distance distribution matrix, setting the error allowable range of Eps and the value of minPts, the reference line is automatically extracted, simplifying the reference line setting process.
It realizes the rapid and accurate extraction of the reference line and characteristic peaks of the nuclear magnetic spectrum, avoids the problems of low efficiency, staggered peaks and missing peaks of manual selection, and improves the accuracy and efficiency of detection.
Smart Images
Figure CN114994115B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of nuclear magnetic resonance, and in particular to a method for extracting a nuclear magnetic spectrum baseline based on an improved DBSCAN. Background Art
[0002] The nuclear magnetic resonance spectrum is derived from the transition between the energy of atomic nuclei, that is, the sample is irradiated with electromagnetic waves of a certain radio frequency to make the atomic nuclei in a specific structural environment resonate and transition, and the signal peak and intensity when the nuclear magnetic resonance occurs are recorded to obtain the nuclear magnetic resonance spectrum. Nowadays, the nuclear magnetic resonance spectrum information (chemical shift of the peak, peak intensity, peak width and peak height, etc.) is usually applied to the molecular structure to analyze the structure of the material. Based on the spectrum, in the process of analyzing the structure of the material, it was found that most of the causes of errors in the analysis results were inaccurate baseline settings. The error in the spectrum baseline will make the peak intensity calculation in the spectrum inaccurate, thereby reducing the accuracy of material detection. Therefore, one of the keys to analyzing the material in terms of molecular structure based on the nuclear magnetic spectrum is to accurately set the baseline of the nuclear magnetic spectrum.
[0003] At present, the commonly used baseline setting methods in the spectrum mainly include: automatic iterative moving average method, wavelet transform, penalized least squares method, moving window smoothing method, etc. By reading a large number of documents, it is found that baseline selection methods based on nuclear magnetic spectrum have appeared one after another, but they all have certain limitations. Therefore, from the perspective of machine learning, the present invention proposes a nuclear magnetic spectrum baseline extraction method based on DBSCAN, which combines the data characteristics of nuclear magnetic spectrum with the clustering idea of DBSCAN algorithm, clusters the pre-processed nuclear magnetic data, and extracts the baseline according to the clustering results. The final experimental results are roughly consistent with the manually selected baseline and characteristic peaks.
[0004] Application publication number CN104458785B, a method for aligning and extracting peaks of nuclear magnetic resonance spectra, reading each nuclear magnetic resonance spectrum; using the peaks of the internal standard compound or the characteristic peaks of the predetermined compound to preliminarily calibrate each nuclear magnetic resonance spectrum; using the predetermined selected nuclear magnetic resonance spectrum as a reference spectrum and dividing it into several fixed intervals, calibrating the nuclear magnetic resonance spectrum to be calibrated in each fixed interval; integrating the corrected spectrum of each nuclear magnetic resonance spectrum in each fixed interval. In this method, the nuclear magnetic resonance spectrum needs to be calibrated multiple times, and the method involves multiple comparison and deletion operations, and its operation process is quite complicated. The method for extracting the baseline of the nuclear magnetic spectrum in the present invention focuses on the processing of data. It only needs to extract the peak vertex coordinates of the spectrum, and clustering can obtain the baseline value and the number of characteristic peaks using the improved DBSCAN algorithm. Its process and ideas are easy to understand. Summary of the invention
[0005] The present invention aims to solve the above problems of the prior art and proposes a method for extracting NMR spectrum baselines based on improved DBSCAN, which avoids the shortcomings of the traditional manual baseline setting and provides reliable data support for the subsequent component detection of lubricant additives.
[0006] The technical solution of the present invention is as follows:
[0007] A method for extracting a baseline of a nuclear magnetic spectrum based on an improved DBSCAN comprises the following steps:
[0008] Step 1: Preprocess the NMR data extracted from the NMR spectrometer, that is, retain the vertex coordinates of each peak in the spectrum and remove other useless coordinate points;
[0009] Step 2: Find the distance between the coordinates of each vertex in the data set and obtain the distance distribution matrix of the data set;
[0010] Step 3: Arrange the data in the distance distribution matrix from small to large, and use the inflection point of the curve in the distance ascending curve graph as the initial value of the neighborhood radius of the core parameter Eps of DBSCAN density-based noise application spatial clustering;
[0011] Step 4: Improve the parameter Eps of DBSCAN. The improvement mainly lies in setting an error tolerance range [minPeak, 2eps-minPeak] for Eps, where minPeak is the minimum peak value and eps is the inflection point obtained in step 3, and divide Eps into intervals;
[0012] Step 5: Use the endpoint values of each interval obtained in step 4 as the neighborhood radius Eps, and select minPts for clustering;
[0013] Step 6: If the number of outliers in the clustering result remains stable within a few continuous intervals, then the number of outliers is the optimal number of outliers, and the baseline interval is set according to the coordinates of the outliers.
[0014] Furthermore, the distance between the coordinates of each vertex in the data set is calculated to obtain the distance distribution matrix Dist of the data set. n*n , specifically including:
[0015] Dist n*n ={dist(i, j)|1≤i≤n, 1≤j≤n} (1)
[0016] dist(i,j) represents the distance between point i and j, n represents the total number of points in the data set, Dist n*n Represents a real symmetric matrix composed of dist(i,j), where the data in each row of the matrix are arranged from small to large.
[0017] Furthermore, the step 3: arranging the data in the distance distribution matrix from small to large, and taking the inflection point of the curve in the distance ascending curve graph as the initial value of the DBSCAN core parameter Eps;
[0018] From the curve growth rate, a slow growth rate indicates that the data density within the range is large, while a fast growth rate indicates that the data distribution within the range is sparse. The selection method of Eps is in line with the idea of the DBSCAN algorithm to divide clusters according to the distance between data.
[0019] Furthermore, the steps of the DBSCAN algorithm in step 4 are as follows: based on the selection of Eps by k-dist, the present invention sets an error tolerance range for Eps, the minimum value of which is the minimum peak value in the spectrum, and the maximum value is the difference between twice Eps and the minimum peak value. That is, the improvement of Eps lies in combining k-dist with regional division, and comprehensively considering all possible values of Eps.
[0020] Furthermore, the selection of minPts in step 5 is as follows: since the selection of minPts directly affects the quality of clustering, if the minPts value is too large, data that originally do not belong to the same cluster will be classified into the same cluster, and some outliers will not be identified; if the minPts value is too small, data that originally belonged to the same cluster will be divided into multiple clusters, and some core points or boundary points will also be identified as outliers; after repeated experiments on a large number of samples, minPts is set between [7,10].
[0021] The advantages and beneficial effects of the present invention are as follows:
[0022] The present invention realizes the extraction of the baseline of the nuclear magnetic spectrum with the help of the DBSCAN algorithm in the field of machine learning. The extraction of the baseline based on the vertex coordinates of each peak in the spectrum conforms to the idea of density-based clustering in the DBSCAN algorithm. The present invention improves the setting of the core parameter Eps of the algorithm by combining k-dist with regional division, and the outliers obtained in the experiment correspond to the characteristic peaks of the nuclear magnetic spectrum. A large number of experiments have verified that the baseline set by this method can quickly and accurately extract the information of the characteristic peaks, avoiding the low efficiency of manually selecting the baseline and the problems of staggered peaks and missed peaks that occur during manual recording. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 It is the raw data of the preferred embodiment provided by the present invention, the horizontal axis is ppm, the vertical axis is hz;
[0024] Figure 2 It is the peak apex data extracted after processing of the specific example of the present invention, the horizontal axis is ppm, the vertical axis is hz.
[0025] Figure 3 This is a distance ascending curve diagram of a specific example of the present invention, with the horizontal axis being the number of points and the vertical axis being the distance.
[0026] Figure 4 This is a graph of experimental results of DBSCAN clustering of a specific example of the present invention, with the horizontal axis being ppm and the vertical axis being hz.
[0027] Figure 5 The present invention is a flow chart of the NMR spectrum baseline extraction method based on improved DBSCAN. DETAILED DESCRIPTION
[0028] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.
[0029] The technical solution of the present invention to solve the above technical problems is:
[0030] like Figure 5 As shown, a method for extracting a baseline of a nuclear magnetic spectrum based on an improved DBSCAN comprises the following steps:
[0031] Step 1: Preprocess the NMR data extracted from the NMR spectrometer, that is, retain the vertex coordinates of each peak in the spectrum and eliminate other useless coordinate points.
[0032] like Figure 1 As shown in the figure, the amount of data in the NMR spectrum is large, and the baseline extraction of the NMR spectrum by this method is only related to the peak height, and has nothing to do with the peak width, peak intensity, chemical shift, etc. Therefore, before clustering the NMR data, we need to extract the peak vertex coordinates of each peak in the spectrum. The data distribution after extraction is as follows Figure 2 shown.
[0033] Step 2: Find the distance between each coordinate point in the data set and obtain the distance distribution matrix Dist of the data set n*n .
[0034] Dist n*n ={dist(i, j)|1≤i≤n, 1≤j≤n} (1)
[0035] dist(i,j) represents the distance between point i and j, n represents the total number of points in the data set, Dist n*n Represents a real symmetric matrix composed of dist(i,j), where the data in each row of the matrix are arranged from small to large.
[0036] Step 3: Arrange the data in the distance distribution matrix from small to large, such as Figure 3It can be clearly seen from the distance ascending curve that the curve grows slowly before a certain point, but after this point the curve shows an exponential growth trend. This inflection point is the initial value of the DBSCAN core parameter Eps (due to the large difference in the vertical coordinates of the data points in this experiment, in order to make the inflection point of the spectrum more obvious, some representative data are selected to obtain the distance ascending graph).
[0037] From the curve growth rate, a slow growth rate indicates that the data density in this range is large, while a fast growth rate indicates that the data distribution in this range is sparse. The selection method of Eps is consistent with the idea of DBSCAN algorithm to divide clusters according to the distance between data.
[0038] Step 4: Improve the parameter Eps of DBSCAN and set the value range of Eps to [minPeak, 2eps-minPeak], where minPeak is the minimum peak value and eps is the inflection point obtained in step 3, and divide Eps into intervals.
[0039] The initial value of eps (unit: 10 million) of this instance is 4.5, and the minimum peak is 0.088, so we divide its interval into [0.088, 1], [1, 2], …, [8, 8.912].
[0040] Step 5: Use the endpoint values of each interval obtained in step 4 as the neighborhood radius Eps of the experiment, and select minPts for clustering.
[0041] Selection of minPts: Since the selection of minPts directly affects the quality of clustering, if the minPts value is too large, data that originally do not belong to the same cluster will be classified into the same cluster, and some outliers will not be identified; if the minPts value is too small, data that originally belonged to the same cluster will be divided into multiple clusters, and some core points or boundary points will also be identified as outliers. After repeated experiments on a large number of samples, the present invention found that setting minPts between [7,10] has the best clustering effect.
[0042] In this sample, we select minPts as 8, and perform cluster analysis on the data in combination with each Eps in the Eps parameter set.
[0043] Step 6: If the number of outliers in the clustering result remains stable within a few continuous intervals, then the number of outliers is the optimal number of outliers, and the baseline interval is determined based on the coordinates of the outliers.
[0044] The present invention uses the core parameter minPts of DBSCAN as 8, and Eps (unit: 10 million) is selected as 0.088, 1, 2, 3, 4, 4.5 (initial value), 5, 6, 7, 8, 8.912 respectively. The experimental results are shown in Table 1:
[0045] Table 1 Experimental results
[0046]
[0047] It can be clearly seen from Table 1 that when Eps (unit: 10 million) is selected as 0.088, 1..., 4.5, the number of outliers changes greatly, and when the value is 5 and 6, the change is relatively gentle. When Eps is selected as 7, 8, 8.912, the number of outliers in the experiment is stable at 100, that is, 100 is the optimal number of outliers in this experiment. Based on this result, the present invention sets the baseline interval to between the maximum peak of non-outliers and the minimum peak of outliers. By comparing the baseline and the number of outliers of the experimental results with the baseline and the number of characteristic peaks calculated by traditional manual calculations, it is found that the experimental results are roughly consistent with the manually selected results, that is, the present invention can be applied to the baseline calculation of lubricating oil nuclear magnetic spectra.
[0048] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0049] The above embodiments should be understood to be only used to illustrate the present invention and not to limit the protection scope of the present invention. After reading the contents of the present invention, technicians can make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.
Claims
1. A method for extracting NMR spectrum baseline based on improved DBSCAN, characterized in that: The following steps are involved: Step 1: Preprocess the NMR data extracted from the NMR spectrometer, that is, retain the vertex coordinates of each peak in the spectrum and remove other useless coordinate points; Step 2: Find the distance between the coordinates of each vertex in the data set and obtain the distance distribution matrix of the data set; Step 3: Arrange the data in the distance distribution matrix from small to large, and use the inflection point of the curve in the distance ascending curve graph as the initial value of the neighborhood radius of the core parameter Eps of DBSCAN density-based noise application spatial clustering; Step 4: Improve the parameter Eps of DBSCAN. The improvement mainly lies in setting an error tolerance range [minPeak, 2eps-minPeak] for Eps, where minPeak is the minimum peak value and eps is the inflection point obtained in step 3, and divide Eps into intervals; Step 5: Use the endpoint values of each interval obtained in step 4 as the neighborhood radius Eps, and select minPts for clustering; Step 6: If the number of outliers in the clustering result remains stable within a few continuous intervals, then the number of outliers is the optimal number of outliers, and the baseline interval is set according to the coordinates of the outliers, and the baseline interval is set between the maximum peak of the non-outliers and the minimum peak of the outliers; The selection of minPts in step 5 is as follows: Since the selection of minPts directly affects the quality of clustering, if the minPts value is too large, data that do not originally belong to the same cluster will be classified into the same cluster, and some outliers will not be identified; If the minPts value is too small, data that originally belonged to the same cluster will be divided into multiple clusters, and some core points or boundary points will also be identified as outliers. After repeated experiments on a large number of samples, minPts is set between [7,10].
2. The method for extracting a nuclear magnetic spectrum baseline based on an improved DBSCAN according to claim 1, characterized in that: Step 2: Find the distance between the coordinates of each vertex in the data set and obtain the distance distribution matrix Dist of the data set n*n , including: Dist n*n ={dist(i,j)|1≤i≤n,1≤j≤n} (1) dist(i,j) represents the distance between point i and j, n represents the total number of points in the data set, Dist n*n Represents a real symmetric matrix composed of dist(i,j), where the data in each row of the matrix are arranged from small to large.
3. The method for extracting a nuclear magnetic spectrum baseline based on an improved DBSCAN according to claim 2, characterized in that: Step 3: Arrange the data in the distance distribution matrix from small to large, and use the inflection point of the curve in the distance ascending curve graph as the initial value of the DBSCAN core parameter Eps; From the curve growth rate, a slow growth rate indicates that the data density within the range is large, while a fast growth rate indicates that the data distribution within the range is sparse. The selection method of Eps is in line with the idea of the DBSCAN algorithm to divide clusters according to the distance between data.
4. The method for extracting a nuclear magnetic spectrum baseline based on an improved DBSCAN according to claim 3, characterized in that: The steps of the DBSCAN algorithm in step 4 are: based on the selection of Eps by k-dist, an allowable error range is set for Eps, the minimum value of the range is the minimum peak value in the spectrum, and the maximum value is the difference between twice Eps and the minimum peak value. That is, the improvement of Eps lies in combining k-dist with regional division to comprehensively consider all possible values of Eps.