Spectroscopic apparatus and method

Abstract

The present invention relates to a method for smoothing spectral data recorded by a spectrometer, the method comprising successively fitting (103) a plurality of spline curves to the spectral data, each spline curve having a different number of nodes. In each spline curve, the node position of each node except the end nodes is determined based on the fit of points of a previously fitted spline curve with fewer nodes among the spline curves to the spectral data. The method further comprises selecting (108) one of the spline curves as a smoothed data curve for the spectral data based on a model selection criterion.

Description

Technical Field

[0001] The present invention relates to a spectroscopic apparatus and method. The present invention is particularly useful in Raman spectroscopy, although the present invention may also be used in other forms of spectroscopy, for example narrow-line photoluminescence, fluorescence, cathodoluminescence, UV-visible (UVVis), nuclear magnetic resonance (NMR), mid-infrared (mid-IR) or near-infrared (NIR). Background Art

[0002] The Raman effect is the inelastic scattering of light by a sample. In Raman spectroscopy, a sample is illuminated by a monochromatic laser, and the scattered light is then dispersed by a dispersive device such as a diffraction grating in, for example, a monochromator to generate a spectrum known as a Raman spectrum. The Raman spectrum is detected by a detector such as a charge coupled device (CCD) to generate spectral data. Examples of Raman spectroscopy devices are known from U.S. Patent Nos. 5,442,438 and 5,510,894, which are incorporated herein by reference.

[0003] Different compounds have different characteristic Raman spectra. Therefore, the Raman effect can be used to analyze the compounds present in the sample.

[0004] Spectral data includes Raman spectra and noise. The presence of noise can affect the analysis of Raman signals. For example, if the analysis includes determining the area below a peak in the signal, the presence of noise can make it difficult to identify a suitable baseline (based on which to determine the area). Figure 1 is an example of spectral data containing Raman spectral peaks and noise. As can be seen, the presence of noise obscures the spectral peaks.

[0005] The Savitzky-Golay filter is a digital filter that can be applied to a set of digital data points to smooth the data, that is, to improve the signal-to-noise ratio without significantly distorting the signal. However, such smoothing techniques are not suitable for spectral signals with sharp peaks, because the Savitzky-Golay smoothing process tends to remove the sharp spectral peaks as well as unwanted noise. Summary of the invention

[0006] According to a first aspect of the present invention, there is provided a method for smoothing spectral data recorded by a spectrometer, the method comprising: successively fitting a plurality of spline curves to the spectral data, each spline curve having a different number of nodes, wherein, in each spline curve, the node position of each node except the endpoint nodes is determined based on the degree of fit between the points of a previously fitted spline curve with fewer nodes among the spline curves and the spectral data; and selecting one of the spline curves as a smoothed data curve for the spectral data based on a model selection criterion.

[0007] In this way, the spacing between nodes is adjusted based on the local information content of the spectral data, so that sharp spectral peaks are retained in the smooth data curve, while equally sharp noise is removed.

[0008] The term "spline curve" as used herein refers to a function defined by one or more polynomials in piecewise order. Each polynomial extends between two points referred to as nodes. The total number of nodes in each of the multiple splines can be between 2 (i.e., only two endpoint nodes) and N / K, where N is the total number of data points in the spectral data, and K is the minimum node separation allowed, in points. Typically, K is greater than 1 and preferably between 2 and 10. The endpoint nodes are the nodes at both ends of the spline curve, and all polynomials are fitted between these endpoint nodes.

[0009] The node positions may correspond to the positions of poorly fitted points of the previously fitted spline curves among the spline curves, as determined using the degree of fit. Poorly fitted points may be points having a fit that is not the best fit of all points, points that are one of the 50% worst fit points, and preferably one of the 10% worst fit points, and preferably the worst fit points, as determined by the degree of fit of the allowed positions for the node. One or more restrictions may be specified for the positions of the nodes. For example, the allowed positions of the node positions may be positions that are more than a predetermined number of data points (such as 1, 2, 3 or more) away from the nodes of the previously fitted spline curve. This ensures that there is a minimum node separation distance.

[0010] The model selection criterion may be a metric that increases with the likelihood of the fit of the spline to the spectral data, but balanced with a penalty for adding parameters in the form of knots, such as the Bayesian Information Criterion or the Akaike Information Criterion. In this way, overfitting is mitigated.

[0011] The splines of the plurality of splines may have 2 to N / K knots, and the method includes determining which spline of the plurality of splines results in an extreme value (a minimum value for BIC and AIC) of a model selection criterion.

[0012] The method may comprise successively fitting each of the plurality of spline curves to the spectral data, starting with the spline curve having the least number of nodes and ending with the spline curve having the greatest number of nodes. Each successively fitted spline curve may have (only) one more node than the immediately preceding spline curve fitted to the spectral data. The positions of the nodes added for each successively fitted spline curve may correspond to the positions of the poorly fitted points of the immediately preceding spline curve.

[0013] Alternatively, each successively fitted curve may add or subtract a random number of knots relative to the knots used by the immediately preceding fitted spline curve (although the requirement that the total number of knots must be between 2 and N / K may be retained).

[0014] The degree of fit of each point of the fitted spline curve can be based on the difference (e.g., residual) between the point and the corresponding point in the spectral data. However, it should be understood that the degree of fit of each point may not be based on the point alone, but may be based on the residuals between other points of the fitted spline curve (such as adjacent (or indirectly adjacent) points or all previous points)) and the corresponding points in the spectral data. For example, the node position can be determined by applying a smoothing algorithm (such as a Savitzky-Golay filter) to the residuals of all points of the previously fitted spline curve to generate a residual smooth curve, wherein the node position corresponds to the point of the maximum value on the residual smooth curve. In an alternative embodiment, the degree of fit of each point can be determined based on the cumulative sum of the residuals of the point when moving forward along these points. The node position corresponds to the point of the maximum value on the cumulative sum curve of the residuals. Taking the residuals of other points into account can be used to reduce the likelihood of the node being located at the peak. Positioning the node at or near the peak will cause the region to lack smoothness. Using such a degree of fit, the "worst" fitting point cannot be the point that causes the maximum residual with the spectral data, and therefore cannot be the point at the peak.

[0015] The method may include identifying the properties of the sample that generated the spectral data from the analysis of the smooth data curve. The method may include identifying one or more components present in the sample. The method may include determining the area under the curve of the smooth data curve to identify the properties (such as concentration) of the components in the sample. The method may include generating a graph of the sample based on the properties of the sample identified from the smooth data curve.

[0016] The method may include outputting the smoothed data curve as a spectrum representing the sample on a display, for example. For example, a user may be able to more easily visually identify one or more components present in the sample from the smoothed data curve.

[0017] The method may include searching a library of component spectra based on the smoothed data curve.

[0018] The method may include analyzing the smooth data curve to identify the properties of the sample, and controlling a process and / or further processing the sample based on the properties identified. For example, the process may be a manufacturing process. The sample may be a sample of one or more manufactured products, and the properties identified may be used to determine whether the manufactured product meets the required specifications. If the required specifications are not met, an adjustment process may be required so that the product is manufactured according to the required specifications. The sample may be a tissue sample and the process may be a treatment for a patient providing the tissue sample. The method may be used as a part of an inspection program (such as a safety program or a quality control program), which includes generating an alarm based on the analysis of the smooth data curve.

[0019] According to a second aspect of the invention, there is provided a processor arranged to perform the method of the first aspect of the invention.

[0020] According to a third aspect of the present invention, there is provided a spectrometer system comprising a spectrometer for obtaining spectral data from a sample and a processor according to the second aspect of the present invention.

[0021] According to a fourth aspect of the present invention there is provided a data carrier having instructions stored thereon which, when executed by a process, cause the process to perform the method of the first aspect of the present invention.

[0022] The data carrier may be a non-transitory data carrier, such as a volatile memory (eg RAM), a non-volatile memory (eg ROM), a flash memory and a data storage device (eg a hard disk, an optical disk), or a transient data carrier (eg an electronic signal or an optical signal). BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 shows spectral data recorded by a Raman spectrometer, including Raman spectra and noise;

[0024] Figure 2 is a flow chart illustrating an embodiment of the present invention;

[0025] Figure 3 Shown according to Figure 1 The smoothed data curve generated from the spectral data shown;

[0026] Figure 4a shows a plot of a sample based on analysis of unsmoothed spectral data generated from different points on the sample, and Figure 4b The corresponding graph is shown, in which the analysis is performed on the smoothed data curve of the spectral data;

[0027] Figure 5showing an overlay of the spectral data with a smoothed data curve determined using a first measure of fit, wherein node locations are shown; and

[0028] Figure 6 An overlay of the spectral data with a smoothed data curve determined using the second fit metric is shown, with node locations indicated. DETAILED DESCRIPTION

[0029] Reference Figure 2 , an embodiment of the present invention comprises receiving 101 spectral data recorded by a spectrometer, for example by a computer. The spectral data may be data generated by a Raman spectrometer. Figure 1 The spectral data of the type that can be received are shown, the spectral data including the number of counts for each wave number (Raman shift), which are measurements of the intensity of the light collected at that wave number. The spectral data includes spectral information and noise. The Raman spectrometer can be as described in WO 2012 / 156667, which is incorporated herein by reference.

[0030] The computer processes the spectral data to generate a smoothed data curve representing the spectral data without noise. The method of generating the smoothed data includes successively fitting 103 a plurality of spline curves having different numbers of nodes to the spectral data. Each spline curve in the plurality of spline curves has a number of nodes n ranging from 2 to N / K, where N is the total number of data points in the spectral data and K is the minimum separation distance between nodes in points. A model selection value is calculated 104 for each spline curve according to a model selection criterion. In this embodiment, the model selection value is a Bayesian Information Criterion (BIC) value calculated using the BIC given below:-

[0031]

[0032] in, is the maximum value of the likelihood function of the fitted spline, N is the number of data points in the spectral data, k is the number of parameters estimated by the spline, and A is the relative weight between the two terms of the BIC. For splines where each spline interval includes a fit of a cubic polynomial, k is taken to be 4n since the number of variables returned for each spline interval is 4. For the classical BIC, the value of A would be 2. However, the present invention includes embodiments where the value of A is between 0.5 and 10. Values ​​other than 2 may be reasonable since the BIC derivation assumes uniform noise, whereas the spectral data may not include uniform noise, and / or the number of variables added by each node may be different from 4.

[0033] Then, the spline curve having the lowest BIC value is returned 108 as the smoothed data curve of the spectral data.

[0034] In this embodiment, the algorithm begins by fitting 103 a first spline curve having two nodes located at the end points of the spectral data. The BIC value of the fitted first spline curve is calculated 104. If the number of nodes is determined 105 to be less than N / K, the worst fitting point of the first spline curve is determined 106 based on the fit.

[0035] In a first embodiment, the fit of each point is the residual between the point on the spline curve and the corresponding point in the spectral data. The point with the largest residual among the points considered as candidate locations for additional nodes is considered to be the worst fitting point. Then, the node position of the additional node of the next spline curve is assigned to the wavenumber position of the worst fitting point. A minimum separation distance for nodes can be set. Therefore, only points whose distance to existing nodes is outside a minimum threshold distance (number of data points) can be considered as candidate locations for additional nodes. The minimum separation distance can be user-selectable, but typically the distance will be greater than 0 (so that the two nodes will not be located at the same position). It has been found that a minimum separation distance of 3 is effective, i.e., K=3.

[0036] As with the first spline, a next spline with additional knots is fitted 103 to the spectral data and the BIC value of the fitted spline is calculated 104.

[0037] Then, the loop of 103 to 107 is repeated so that additional spline curves with different numbers of knots are fitted to the spectral data, and the BIC value is calculated until a spline curve with N / K knots has been fitted to the spectral data.

[0038] In step 108 , the spline curve having the lowest BIC value among the plurality of spline curves is output as a smoothed data curve of the spectral data.

[0039] The smoothed data curve can be displayed to a user and / or used to analyze the sample from which the spectral data was generated. Using the smoothed data curve to analyze and generate a representation of the sample can produce a clearer and / or more accurate representation of the information of interest.

[0040] Figure 4a and Figure 4b Graphs 201, 202 generated from the same Raman spectral data generated from a sample are shown. Graphs 201, 202 of spectral data are superimposed on a larger white light image of the sample. These graphs represent the area under the Raman peak between 989nm and 1019nm at different points on the sample. Figure 1 and Figure 3 Peaks of interest are shown. Figure 4a is a graph generated from the original spectral data, and Figure 4bis a graph generated from a smoothed data curve generated from spectral data. As can be seen, the two graphs are substantially different. In order to determine the Raman peak, a baseline is set corresponding to the background radiation source. However, noise in the signal may cause uncertainty in the position of the baseline. Inconsistency in baseline positioning may lead to changes in the area under the calculated peak and thus produce a non-representative drawing of the sample. Removing noise by smoothing the curve alleviates this problem and can produce a drawing that is more representative of the sample components.

[0041] Figure 5 Spectral data, a smoothed data curve generated according to the above embodiment, the residual between the smoothed data curve and the spectral data, and node locations (referred to as anchor points) are shown. As can be seen, for this example, a number of nodes are located at locations that coincide with spectral peaks. This can result in unnatural shapes (such as Figure 5 ). To mitigate this problem, different degrees of fit can be used to reduce the likelihood that the top of the spectral peak is identified as a node location. For example, the degree of fit of each point can be determined based on a smooth curve of residuals (referred to herein as a "residual smooth curve"). Such a residual smooth curve can be generated by applying a smoothing algorithm (such as a Savitzky-Golay filter) to the residuals of all points of a previously fitted spline curve. The node locations of the additional nodes can correspond to the points of the maximum values ​​on the residual smooth curve.

[0042] Figure 6 Spectral data and a smoothed data curve generated from the metric are shown, where the goodness of fit at each point depends on the other points as well as the residuals at the point of interest. Figure 6 Also shown are the residuals between the smoothed data curve and the spectral data, and the node locations (called anchor points). As can be seen, using this alternative fit can position the nodes away from the tops of the spectral peaks, and therefore make the peaks in the smoothed data curve smoother.

[0043] It will be appreciated that modifications and variations may be made to the embodiments described above without departing from the present invention as defined herein.

Claims

1. A method for smoothing spectral data recorded by a spectrometer, the method comprising: sequentially fitting a plurality of spline curves to the spectral data, each spline curve having a different number of nodes, wherein, in each spline curve, the node position of each node except the endpoint nodes is determined based on the degree of fit of points of a previously fitted spline curve having fewer nodes in the spline curve to the spectral data; and selecting one of the spline curves as a smoothed data curve for the spectral data based on a model selection criterion, Therein, the model selection criterion is a metric that increases with the likelihood of fit of the spline curve to the spectral data, but is balanced with a penalty for adding parameters in the form of knots.

2. The method according to claim 1, wherein: The node locations correspond to locations of points in the spline curve at which the fit of the previously fitted spline curve was poor, as determined using the measure of fit.

3. The method according to claim 2, wherein: The poorly fitted point is the point having the worst fit for the allowed positions of the node.

4. The method according to claim 1, wherein: The fit of each point of the spline curve is based on the residual between the point and a corresponding point in the spectral data.

5. The method according to claim 4, wherein: The fit of each point is based on the residuals between other points of the fitted spline curve and corresponding points in the spectral data.

6. The method according to claim 1, wherein: The spline curves in the plurality of spline curves have 2 to N / K nodes, and the method includes determining which of the plurality of spline curves results in an extreme value of the model selection criterion, wherein N is the number of data points in the spectral data, and K is the minimum separation distance between the nodes, in points.

7. The method according to claim 1, wherein: Sequentially fitting each of the plurality of spline curves to the spectral data includes starting with a spline curve having a fewest number of knots and ending with a spline curve having a great number of knots.

8. The method according to claim 7, wherein: Each successively fitted spline curve has one more node than the immediately previous spline curve fitted to the spectral data.

9. The method according to claim 8, wherein: The locations of the nodes added for each successively fitted spline curve correspond to the locations of the poorly fitted points of the immediately preceding spline curve.

10. The method according to claim 1, comprising: A property of the sample that generated the spectral data is identified from analysis of the smoothed data curve.

11. A processor arranged to perform the method according to any one of claims 1 to 10.

12. A spectrometer system comprising a spectrometer for obtaining spectral data from a sample and a processor according to claim 11.

13. A data carrier having instructions stored thereon which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 10.

Images