A general self-supervised learning Raman spectroscopy noise reduction method, apparatus and equipment

Abstract

This invention discloses a general self-supervised learning Raman spectral denoising method, comprising: Step 1: acquiring Raman data; Step 2: Raman data preprocessing, using an adaptive iterative reweighted penalized least squares method to remove the baseline from the Raman data, and standardizing the baseline-removed spectrum to obtain preprocessed spectral data; Step 3: processing the preprocessed spectral data using a downsampling strategy H(·) to obtain two noise-independent sub-spectral sets h1(·) and h2(·), each sub-spectral set including 10 sub-spectrums; Step 4: inputting the preprocessed spectral data and the corresponding two sub-spectral sets into a U-Net network f. θ (·), to obtain the denoised spectrum. This denoising method can effectively denoise various Raman data, and its self-supervised learning method does not require a complex and lengthy learning process.

Description

Technical Field

[0001] This invention relates to the field of spectral noise reduction, and more particularly to a general self-supervised learning Raman spectral noise reduction method, apparatus, and equipment. Background Technology

[0002] Raman spectroscopy (RS) is an optical modality for non-destructive and label-free molecular fingerprinting. However, only one in 10⁸ photons may undergo Raman scattering, meaning the Raman scattering signal is inherently weak. This makes it sensitive to sensor defects, necessitating signal enhancement and the elimination of instrument and background noise. Plasmon-based nanomaterials can further enhance Raman scattering, a technique known as surface-enhanced Raman spectroscopy (SERS), enabling intraoperative real-time imaging and specific detection of individual molecules. Thanks to its high specificity and sensitivity, Raman technology has been widely used in food testing, drug detection, and medical testing.

[0003] Over the past few decades, advancements in optical instruments such as lasers, filters, spectrometers, and detection devices have significantly improved spectral resolution and signal-to-noise ratio (SNR), driving the development of Raman spectroscopy. However, in detection scenarios with weak Raman signals, such as single-molecule detection and deep-penetration detection, noise from the sample itself and the instrument background often obscures the effective information in the spectrum. Improving experimental methods or optical devices is demanding and yields limited results. Therefore, efficient, economical, and universally applicable spectral noise reduction techniques, especially the development of advanced noise reduction algorithms, are crucial.

[0004] Currently, commonly used spectral denoising algorithms can be divided into moving window methods based on polynomial fitting and wavelet transform methods based on power spectrum. Moving window methods dynamically fit a polynomial to continuous data points in the spectrum according to the window size, such as moving averages and Savitzky-Golay (SG) filters. In contrast, wavelet transform methods use specific wavelet bases and introduce threshold coefficients to reconstruct the spectral signal while preserving key spectral features, such as wavelet threshold denoising (WTD). Although these traditional algorithms are very fast for Raman spectral denoising, parameters such as window size and polynomial order still need to be determined empirically during use. Furthermore, with the explosive growth of spectral data, fine-tuning parameters for each data point is extremely time-consuming, making real-time processing impossible.

[0005] In contrast, deep learning (DL) requires no manual parameter tuning and can automatically learn and extract high-level features from large datasets, integrating them into accurate predictions and decision-making. In spectral denoising, numerous studies have used simulated training data with added noise or background to achieve denoising effects. However, these models require a large number of ideal "clean-noise" spectral pairs for training. Such simulated data cannot cover all possible noise types in the real world, and the discrepancy between simulated and experimental data introduces uncertainty into the generalization ability of DL.

[0006] For Raman spectroscopy, real-world data (such as data from clinical applications) is limited, and corresponding labels (i.e., completely noise-free "clean" spectra) are often unavailable. A universal noise reduction strategy that can rapidly and stably reduce the noise of Raman and SERS spectra using a small amount of real-world data without relying on empirical parameter selection has not yet been developed and applied.

[0007] Current Raman spectroscopy noise reduction techniques have the following drawbacks:

[0008] 1. Traditional noise reduction algorithms rely on user experience to select parameters, which is almost ineffective for ultra-weak signal spectra.

[0009] 2. Traditional deep learning methods rely on simulated data for supervised training, which is time-consuming and cannot fully learn the characteristics of real noise and signals, nor can they perform noise reduction on individual spectra.

[0010] 3. The self-supervised loss function design is based on image data, lacks adaptive adjustment for spectral data, and lacks information retention terms.

[0011] 4. Deep learning methods are time-consuming to train and cannot achieve real-time noise reduction during the acquisition process.

[0012] Therefore, those skilled in the art are dedicated to developing a general self-supervised learning Raman spectroscopy noise reduction method. Summary of the Invention

[0013] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to perform fast and effective noise reduction processing on various Raman data.

[0014] To achieve the above objectives,

[0015] In a first aspect, the present invention provides a general self-supervised learning Raman spectroscopy noise reduction method, comprising,

[0016] Step 1: Obtain Raman data.

[0017] Step 2: Raman data preprocessing.

[0018] The baseline was removed from the Raman data using an adaptive iterative reweighted penalized least squares method, and the baseline-removed spectra were then standardized to obtain preprocessed spectral data.

[0019] Step 3: Use a downsampling strategy The preprocessed spectral data is processed to obtain two noise-independent sub-spectral sets. and Each sub-spectral set includes 10 sub-spectrums.

[0020] Step 4: Input the preprocessed spectral data and the corresponding two subsets of the spectrum into the U-Net network. The spectrum after noise reduction was obtained.

[0021] Furthermore, step 3 is specifically set as: downsampling strategy Slide the spectrum with a window size of 3, covering the 3 Raman shifts. Randomly select two different shifts and copy their corresponding intensity values ​​to create a subspectrum with the same dimensions as the original spectrum. Repeat this operation twice to obtain the final subspectral set. and .

[0022] Furthermore, the U-Net network It contains 5 modules.

[0023] The first module consists of two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and concatenation layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layer extract features, which are then passed to the second module. Simultaneously, the extracted features are passed to the fifth module after retaining low-level features through the copy and concatenation layer.

[0024] The second module consists of two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and concatenation layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layers extract features, which are then passed to the third module. Simultaneously, the extracted features are passed to the fourth module through the copy and concatenation layer, preserving low-level features.

[0025] The third module consists of two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The one-dimensional convolutional layers and ReLU activation layers extract features, and the extracted features are recovered through the upsampling layer and then passed to the fourth module.

[0026] The fourth module consists of two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The convolutional and ReLU activation layers concatenate the features passed from the second and third modules, and then the upsampling layer restores the concatenated features before passing them to the fifth module.

[0027] The fifth module consists of three one-dimensional convolutional layers and three ReLU activation layers. It concatenates the features passed from the first module with the features passed from the fourth module to reconstruct the spectrum.

[0028] Furthermore, the U-Net network The training method is as follows:

[0029] Stochastic gradient descent (SGD) was applied to the U-Net network. loss function for:

[0030]

[0031] In the formula:

[0032]

[0033]

[0034]

[0035] in: These are the weights of the corresponding loss terms. This indicates that the input Raman spectrum is an n-dimensional real vector. and This represents the two sub-spectral sets obtained by downsampling in step 2.

[0036] Furthermore, the learning rate of stochastic gradient descent (SGD) is 0.003.

[0037] Furthermore, the U-Net network The training process is converged based on the difference in the loss function between two iterations. When the difference in loss is less than 10... -4 The model stops training when the time comes.

[0038] Technical effect

[0039] 1. By constructing a deep learning denoising model, the U-net neural network is used to automatically learn the noise distribution and signal characteristics in real spectral data. Once established, the denoising model only requires the noisy spectrum as input. The deep learning model consists of multiple layers of neurons, effectively capturing complex structures and abstract features in complex real-world spectra. No parameter selection is required during use; the input spectrum is directly denoised, avoiding reliance on user experience and reducing subjective bias from manual parameter tuning. The denoising model adapts to different data types, providing stable denoising effects for spectral data with varying signal intensities.

[0040] 2. A self-supervised learning-based objective function for denoising is designed, utilizing the characteristics of the data itself for effective supervision to achieve denoising of a single spectrum. A regularization term is introduced into the objective function to emphasize the comprehensive preservation of effective information in one-dimensional spectral data. Shot noise is the main type of noise affecting the spectrum; this type of noise has a consistent impact on the entire spectrum and has a mean of zero. The self-supervised denoising model uses a downsampling strategy to construct noise-independent sub-spectral sets, with different sub-spectral sets mutually constrained and integrated using mean squared error loss. The self-supervised strategy uses only the input spectrum itself to construct supervision, where the downsampling strategy also considers data augmentation, enabling denoising of a single spectrum. No training process is required during use; the downsampling strategy constructs sub-spectral sets with prominent noise independence and identical distribution, highlighting potential bias interference and achieving more reliable and stable denoising results; it directly uses the real spectrum to construct supervision, fully learning the real noise distribution; and it achieves stable denoising of a single spectrum.

[0041] 3. In addition to commonly used regularization terms, this method incorporates a Mean Squared Error (MSE) term between the input and output, emphasizing the importance of information. The regularization term uses mean squared error to constrain the original and denoised spectra, preserving the effective information in the denoised spectrum. This added information regularization term achieves comprehensive information preservation. Simultaneously, this term improves the SNR of the denoised spectrum and reduces the difference in ED between the result and the reference spectrum, making this method more suitable for the characteristics of spectral data.

[0042] 4. An adaptive convergence criterion is used during training. The training process determines convergence based on the difference in the loss function between two iterations; training stops when the difference is less than 10⁻⁴. This method adaptively trains and denoises spectra of different quality levels, achieving real-time noise reduction.

[0043] 5. This invention utilizes a self-supervised learning strategy to apply deep learning to one-dimensional spectral data denoising, avoiding the reliance on parameter selection and experience in traditional denoising algorithms and the requirement of large amounts of training data for supervised learning. The process is simple and fast. The denoising effect of this method has been evaluated using multiple metrics, showing significant improvement at multiple characteristic peaks. The SNR improvement is significantly better than current traditional algorithms, and the denoising result is close to the average spectrum with good stability. Simultaneously, the peak position and half-width remain unchanged, proving that this method effectively denoises while preserving important information in the spectrum. The method requires no manual parameter selection; only the spectrum to be denoised needs to be input, facilitating integration with existing Raman spectroscopy analysis software for spectral processing. The method processes approximately 1.84 frames per second per spectrum, matching the commonly used testing time for Raman technology, and can be combined with Raman spectroscopy acquisition software to achieve real-time spectral denoising. The model is lightweight, approximately 1.76M, with a computational cost far less than other deep learning methods (e.g., CNN, 128.43M), making it easy to integrate with various software.

[0044] Secondly, the present invention provides a universal self-supervised learning Raman spectroscopy noise reduction device, characterized in that it comprises:

[0045] The data acquisition module is used to acquire Raman data;

[0046] The first calculation module is used for Raman data preprocessing;

[0047] The second computation module is used to input the preprocessed spectral data and the corresponding two subsets of the spectrum into the U-Net network. , to obtain the spectrum after noise reduction;

[0048] The noise reduction module is used to input the preprocessed spectral data and the corresponding two sub-spectral sets into the U-Net network. The spectrum after noise reduction was obtained.

[0049] Thirdly, the present invention provides a computer device, characterized in that it comprises:

[0050] Memory;

[0051] One or more processors are coupled to the memory;

[0052] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, the one or more applications being configured to perform the general self-supervised learning Raman spectroscopy noise reduction method as provided in the first aspect above.

[0053] Fourthly, the present invention provides a computer-readable storage medium storing program code, which can be called by a processor to execute the general self-supervised learning Raman spectroscopy noise reduction method provided in the first aspect.

[0054] Fifthly, the present invention provides a computer program, comprising a computer program that, when executed by a processor, implements the general self-supervised learning Raman spectroscopy noise reduction method provided in the first aspect.

[0055] It is understandable that the beneficial effects of the second, third, fourth, and fifth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here.

[0056] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0057] Figure 1 This is a flowchart of a preferred embodiment of the self-supervised learning Raman spectroscopy noise reduction method of the present invention;

[0058] Figure 2 This is a structural diagram of a self-supervised U-Net denoising model according to a preferred embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of a preferred embodiment of the downsampling strategy of the present invention;

[0060] Figure 4 This is a schematic diagram of the self-supervised noise reduction objective function of a preferred embodiment of the present invention;

[0061] Figure 5 This is a noise reduction result diagram of simulation data, Raman spectrum of ethanol, and open source HeLa cell Raman spectrum according to a preferred embodiment of the present invention;

[0062] Figure 6 This is a graph showing the change in spectral shape caused by different downsampling strategies in a preferred embodiment of the present invention;

[0063] Figure 7 This is a preferred embodiment of the present invention showing the characteristic peak intensities and Euclidean distances of various Raman data.

[0064] Figure 8 This is a noise reduction result diagram of deep penetration data according to a preferred embodiment of the present invention;

[0065] Figure 9 This is a preferred embodiment of the present invention showing the denoising results and Euclidean distance results of different denoising methods on different characteristic peaks of deep-penetrating data;

[0066] Figure 10 This is an analysis result diagram of the occlusion method according to a preferred embodiment of the present invention. Detailed Implementation

[0067] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0068] In the accompanying drawings, parts with the same structure are indicated by the same numerical designation, and components with similar structure or function are indicated by similar numerical designations.

[0069] Figure 1 This illustrates a general noise reduction method for Raman spectroscopy, including:

[0070] Acquire various Raman data, perform Raman data preprocessing, construct sub-spectral sets, and perform noise reduction using a self-supervised U-Net model.

[0071] The acquired Raman data used in the effect comparison can include: simulation data, open-source HeLa cell Raman spectra (Nat. Commun. 2024, 15 (1), 754), SERS spectra of ethanol molecules, and SERS spectra of GERRTs particles under interference from muscle tissue of different thicknesses. The data types cover Raman and SERS spectra and different detection scenarios, fully demonstrating the versatility of the method of this invention.

[0072] Raman data preprocessing: The acquired Raman data (excluding simulation data) typically includes the fluorescence background of the sample. The baseline is removed from the acquired spectra using the Automatic Iterative Removal of Pump-Laser Line from Raman Spectra (airPLS), and the baseline-removed spectra are then standardized to obtain preprocessed spectral data.

[0073] Constructing sub-spectral sets: Since shot noise has a uniform effect on the entire spectrum, this invention uses a downsampling strategy to process the preprocessed spectrum to obtain two noise-independent sub-spectral sets, each of which includes 10 sub-spectrals.

[0074] Self-supervised U-Net model denoising: The preprocessed spectrum and corresponding sub-spectral sets are constrained within the network to obtain the denoised spectrum. The U-Net network uses one-dimensional convolutional and pooling layers to construct network modules, with a stride of 3 to avoid spectral information loss, making it suitable for high-information-density spectral processing. The self-supervised denoising model uses a downsampling strategy to construct noise-independent sub-spectral sets, with mutual constraints between different sub-spectral sets and ensembled using mean squared error loss. We added a dedicated regularization term, using mean squared error to constrain the original spectrum and the denoised spectrum, thereby preserving effective information in the denoised spectrum. The self-supervised strategy uses only the input spectrum itself for supervision, where the downsampling strategy also considers data augmentation, enabling denoising of individual spectra. The method adaptively trains and denoises spectra of different quality. The training process uses the difference in the loss function between two iterations for convergence judgment; when the loss difference is <10⁻⁴, the model stops training, achieving real-time denoising.

[0075] Multi-metric evaluation: The proposed method is compared with commonly used SG filters, wavelet threshold denoising (WTD), and self-supervised CNN methods. Qualitative and quantitative evaluations are performed using the SNR, HWV, peak shape, and ED (exposure potential) of multiple characteristic peaks with their corresponding average spectra. An interpretability analysis of the model's denoising process is conducted using an occlusion method.

[0076] Network architecture Based on the U-Net framework ( Figure 2 ), through network weight parameters This method is used to denoise a single Raman spectrum. In practice, the input high-dimensional spectrum is first encoded into a low-dimensional tensor, and then reconstructed into the denoised spectrum. To achieve fast processing, a lightweight network consisting of only five modules is used: the first two modules form the encoder, and the fourth and fifth modules form the decoder.

[0077] The first module consists of two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and concatenation layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layer extract features, which are then passed to the second module. Simultaneously, the extracted features are passed to the fifth module after retaining low-level features through the copy and concatenation layer.

[0078] The second module consists of two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and concatenation layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layers extract features, which are then passed to the third module. Simultaneously, the extracted features are passed to the fourth module through the copy and concatenation layer, preserving low-level features.

[0079] The third module consists of two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The one-dimensional convolutional layers and ReLU activation layers extract features, and the extracted features are recovered through the upsampling layer and then passed to the fourth module.

[0080] The fourth module consists of two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The convolutional and ReLU activation layers concatenate the features passed from the second and third modules, and then the upsampling layer restores the concatenated features before passing them to the fifth module.

[0081] The fifth module consists of three one-dimensional convolutional layers and three ReLU activation layers. It concatenates the features passed from the first module with the features passed from the fourth module to reconstruct the spectrum.

[0082] The decoder is symmetrical to the encoder and reuses features from the encoder to focus on the location of characteristic peaks in the spectrum.

[0083] downsampling strategy Similar to the max pooling function ( Figure 3 ). In the spectrum, with window size Slide to cover the window. One Raman shift, Randomly select two different shifts and copy their corresponding intensity values ​​to create a subspectrum with the same dimensions as the original spectrum. Repeat this operation twice to obtain the final subspectral set. and Constraints can be applied between multiple sub-spectral sets to obtain features at different Raman shifts. Besides suppressing bias interference, this downsampling strategy also effectively removes zero-mean shot noise. Assuming the noise at each Raman shift is independent—that is, noise at different locations is uncorrelated—the noise in the two downsampled sub-spectral sets is independent and identically distributed. Using a mean squared error loss ensemble for training, this downsampling strategy converges to a noise-free signal more quickly.

[0084] This embodiment develops a general one-dimensional spectral data noise reduction loss function ( Figure 4 ). Indicating that the input Raman spectrum is an n-dimensional real vector, we first define the regression loss between the original spectrum and the denoised spectrum. Using mean squared error (MSE), Ensure key peak values:

[0085]

[0086] The input Raman spectrum is downsampled to compress noise; therefore, we use and Denoising of sub-spectrums is achieved by constructing constraints using MSE, and the calculation formula is as follows:

[0087]

[0088]

[0089] in, and This represents the two sub-spectral sets obtained by downsampling.

[0090] The objective function is defined as a combination of the three loss terms mentioned above, used to achieve significant denoising of the spectrum while avoiding distortion:

[0091]

[0092] in These are the weights of the corresponding loss terms, used to balance the noise reduction effect and maintain the spectral shape.

[0093] We use SGD as the gradient optimization method for the self-supervised U-Net denoising network with a learning rate of 0.003. To accommodate spectra of different qualities, model training stops when the loss function changes by less than 10⁻⁴.

[0094] The noise reduction effect of the noise reduction method provided in this embodiment is qualitatively and quantitatively analyzed using multiple evaluation indicators.

[0095] Signal-to-noise ratio (SNR): For a characteristic peak in the reference spectrum, the five wavenumbers closest to that peak are selected as the range for calculating the average peak intensity. The formula for calculating the average peak intensity is as follows:

[0096]

[0097] in, It is the signal intensity corresponding to the i-th Raman shift within the signal range. It is the number of Raman shifts contained within the signal range.

[0098] The frequency band without characteristic peaks in the spectrum is selected as the range of the noise standard deviation. The formula for calculating the noise standard deviation is:

[0099]

[0100] in This represents the intensity corresponding to the j-th Raman shift within the noise range. Indicates the average intensity within the noise range. This indicates the number of Raman shifts contained within the noise range.

[0101] The formula for calculating the signal-to-noise ratio of the characteristic peak is as follows:

[0102]

[0103] If the signal-to-noise ratio of the obtained characteristic peak is greater than 3, then a confidence level of 99% indicates that there is a characteristic peak at that location that is different from random noise.

[0104] Euclidean distance (ED): Define the high-quality spectrum (i.e., the spectrum of a high-concentration substance or the average spectrum) as the reference spectrum, and calculate the ED between the denoised spectrum and the reference spectrum, as follows:

[0105]

[0106] in, This represents the intensity corresponding to the i-th Raman shift from the reference spectrum. This represents the intensity corresponding to the i-th Raman shift in the noise-reduced spectrum.

[0107] Characteristic peak position: Find the Raman shift corresponding to the strongest signal in the characteristic peak range of the spectrum as the peak position, and compare the changes in the characteristic peak positions of the light after noise reduction with those of the reference spectrum.

[0108] Full Width Half Maximum (FWHM): Find the Raman shift at half the point of strongest signal of the corresponding characteristic peak in the spectrum, and calculate and compare the change in FWHM between the denoised spectrum and the reference spectrum.

[0109] This embodiment proposes a self-supervised U-Net denoising network that reduces noise in Raman spectra through downsampling, without requiring large amounts of training data or manual parameter settings. A single Raman spectrum is taken as input and subsampled to obtain two subsets of spectra. Thanks to self-supervised learning and downsampled data, our U-Net denoising network can quickly and effectively eliminate noise in the spectrum.

[0110] The performance of the proposed general denoising algorithm was verified using simulation data. Raman characteristic peaks were generated using the Lorentz function, with the main peak positions located at 600, 800, 1200, and 1270 cm⁻¹. Subsequently, Gaussian white noise with a noise level of 60 was added to the simulated spectrum. Figure 5 As can be seen, the small peak at 800 cm⁻¹ is masked, and the adjacent peaks (1200 and 1270 cm⁻¹) are difficult to distinguish. The spectrum after noise reduction using this algorithm consistently identifies four peaks and effectively separates the two peaks at 1200 and 1270 cm⁻¹. In the spectral no-signal region, 50 wavenumbers ( Figure 5 a(left), gray shading) was used to calculate The results showed that the signal-to-noise ratio (SNR) of each peak was significantly enhanced. Typically, the SNR is 3 ( Figure 5 a (right), the black dashed line) is considered to be the detection threshold for the characteristic Raman peak.

[0111] Raman spectrum of ethanol molecules ( Figure 5 b (left) The three main characteristic peaks at 879, 1050, and 1087 cm⁻¹ represent the symmetric stretching vibration of CCO, the antisymmetric stretching vibration, and the vibration of COH (light gray), respectively. Using the range of 78–245 cm⁻¹ ( Figure 5 b (left), gray shade) to calculate Our method consistently increases the signal-to-noise ratio at the three main peaks, such as... Figure 5 As shown in b (right). Therefore, both self-supervised learning and downsampling strategies enable the U-Net denoising network to operate effectively on a single spectrum.

[0112] Open-source spectra of HeLa cells were also used to further solidify the generalization ability and robustness of the proposed method. Raman shifts at 2800–3000 cm⁻¹ represent the CH signal of organic matter in the cells. The denoised spectral noise was significantly compressed, the characteristic peak at 2932 cm⁻¹ was more clearly identified, and the signal-to-noise ratio improved from 3.31 to 18.19 ± 1.94. Furthermore, the characteristic peak intensities and EDs from different data sources indicate that the denoising results of the proposed method are closer to the corresponding reference spectra (…). Figure 6 ).

[0113] Table 1 (ablation experiment of objective function) shows that all three loss terms of the objective function contribute to the significant noise reduction performance of our algorithm.

[0114] Table 1

[0115]

[0116] Furthermore, the impact of our downsampling strategy on the position, intensity, and width of each Raman peak is negligible (Table 2 (peak positions for different downsampling strategies), Table 3 (half-peak width for different downsampling strategies) and Figure 7 ).

[0117] Table 2

[0118]

[0119]

[0120] Table 3

[0121]

[0122] The effects of different noise conditions are shown in Table 4 (SNR and ED after spectral denoising with different noise levels), indicating that the proposed denoising method has strong generalization ability.

[0123] Table 4

[0124]

[0125] We evaluated the effectiveness of the algorithm in improving the detection of deep lesions, specifically the detection of SERS tags embedded in thick biological tissue. Figure 8 a). We used gap-enhanced resonance Raman probes (GERRTs) with an IR780 Raman reporter molecule as the signal source and embedded them into isolated porcine tissue. To acquire Raman spectra at different signal-to-noise ratios, porcine muscle tissue of varying thicknesses was used to simulate different noise levels. The overall tissue thickness was adjusted by adding tissue sections. Figure 8 b). A transmission Raman spectroscopy (GERRT) device was used to irradiate and collect signals from both sides of the tissue. The laser powers used were 100, 200, 400, and 600 mW, and the porcine muscle tissue thicknesses were 4, 6, 8, 10, 12, 14, 16, and 18 cm. GERRTs were placed on the excitation surface, and SERS spectra were collected by varying the laser power and increasing the tissue thickness on the GERRTs. A total of 160 SERS spectra were obtained, with a signal-to-noise ratio ranging from 0 to 190 (…). Figure 8 d).

[0126] GERRTs exhibit characteristic Raman peaks of IR780 at 520, 931, 1203, and 1580 cm⁻¹, which are clearly distinguishable from the background. Figure 8 c, light gray). 2000-2060 cm⁻¹ Figure 8 The Raman shift of (c, dark gray shade) was used to calculate Because light penetration through biological tissues undergoes exponential decay, even the strongest characteristic peak is significantly weakened with increasing tissue thickness. When the tissue thickness increased from 10 cm to 12 cm, we observed that the SNR of the characteristic peak at 1203 cm⁻¹ decreased from 10.1 ± 2.3 to 2.7 ± 1.2. Figure 8 e), which makes the signal undetectable.

[0127] To verify the robustness of this method to noise reduction of SERS spectra of different qualities, all spectra were subjected to three independent replicate experiments. Figure 8 The results show that, compared with the original spectra, the average SNR of all spectra is improved by approximately 401.57%. Meanwhile, the corresponding error bars are narrow (±18.71%), demonstrating the stability of the proposed method in the denoising task. This paper focuses particularly on denoising of ultra-weak signal spectra (SNR < 10), a challenging task with limited current research.

[0128] Finally, we compared the improvement in detection depth before and after using the U-Net denoising network. The characteristic peak at 1203 cm⁻¹ is the strongest Raman peak in GERRTs, and we used this peak to verify the denoising effect and optical penetration limit of the proposed method. The limiting depth of detection (LODT) is defined as the maximum tissue thickness corresponding to an SNR ≥ 3 at the characteristic peak of 1203 cm⁻¹. The LODT based on the original spectrum is 10 cm, while our denoising algorithm increases the LODT to over 18 cm. Figure 8 After using self-supervised U-Net for noise reduction, we further applied the deep penetration equation for fitting and estimated the LODT value to be approximately 19.53 cm. Since the distance between the abdomen and back of the human body is 15-30 cm, these results indicate that our method makes optical techniques more closely resemble real-world scenarios in biomedical deep detection.

[0129] Furthermore, using the deep penetration SERS data collected above, we compared the U-Net denoising algorithm with two commonly used denoising methods, SG and WTD, as well as a self-supervised CNN denoising algorithm. The SG algorithm is a typical moving window smoothing algorithm, while the WTD algorithm is a power spectrum estimation-based algorithm. SG uses an order of 3 and a window size of 5, while WTD employs a two-layer Daubechies wavelet basis (db3) and a VisuShrink threshold function. The self-supervised CNN consists of four blocks, each containing a convolutional layer with a kernel size of 2, a corrected linear unit, a pooling layer with a kernel size of 2, and finally three fully connected layers using the same self-supervised objective function as the method described above.

[0130] To more intuitively demonstrate the noise reduction effect, we defined the signal-to-noise ratio improvement (iSNR), as shown in the following formula:

[0131]

[0132] in, This represents the SNR of the spectrum after noise reduction. This represents the SNR of the undenoised spectrum.

[0133] Figure 9 The spectrum contained four characteristic peaks (521, 932, 1203, 1581 cm⁻¹), comprehensively demonstrating the denoising effects of different methods on the full spectrum. All spectra were subjected to three independent replicate experiments. Clearly, our method outperformed the other three methods on all characteristic peaks and exhibited uniform and reproducible denoising effects on a set of SERS spectra of different qualities at different locations. The iSNRs of the four characteristic peaks were 723.41±181.28%, 533.17±117.71%, 540.37±120.33%, and 617.66±160.13%, respectively. The iSNRs of the characteristic peaks after denoising by the three methods are shown in Table 5 (iSNR (%) of different denoising methods for deep penetration data).

[0134] Table 5

[0135]

[0136] SG uses an order of 3 and a window size of 5; WTD uses a wavelet basis of db3 and a decomposition level of 2.

[0137] The optimal result is indicated in bold.

[0138] For extremely weak signals, we Figure 9Table b clearly presents the results of different methods when the signal-to-noise ratio is <10. For such a challenging task, WTD's performance drops significantly, while SG and CNN models perform comparably. In contrast, our method achieves an improvement of over 500% on all feature peaks, demonstrating that its stability and noise reduction performance are significantly superior to other algorithms (see Table 6).

[0139] Table 6

[0140]

[0141] SG uses an order of 3 and a window size of 5; WTD uses a wavelet basis of db3 and a decomposition level of 2.

[0142] like Figure 10 As shown, we enhance the interpretability of our method by using an occlusion method to calculate the Raman shift contributing to spectral denoising. The occlusion window size used is 100, and the corresponding regions are zeroed out. The resulting occluded spectra are input into the training model based on the unoccluded spectra. The similarity between the occluded and original denoised spectra is calculated using the Pearson correlation coefficient (PCC): the smaller the absolute value of the PCC, the greater the difference between the two spectra, indicating that the contribution of certain regions is more important. We calculate the importance of spectra at different signal-to-noise ratios. Figure 10 The diagram presents three representative results. For SERS spectra with a signal-to-noise ratio (SNR) of 0-190, as the SNR increases, the CNN model focuses more on regions with higher Raman intensity, while the U-Net network consistently concentrates more on learning the positions of characteristic peaks. This explains the superior denoising performance of the proposed method. Notably, even for ultra-weak signal spectra with an SNR < 10, the method can automatically identify most of the characteristic peaks of interest to the model. This attention encourages the network to fully utilize and learn the patterns of key peaks (e.g., shape, half-width), thus preserving them well. Simultaneously, because noise uniformly affects the entire spectrum, the relative perturbations at characteristic peaks are weaker, which helps to better preserve peak values. Although the model is less interested in the signal-free regions at the ends of the spectrum (Raman shift > 1700 cm⁻¹), it eventually converges in the desired direction. The unique combination of self-supervised learning and downsampling strategies makes a significant contribution to stable denoising performance and high interpretability.

[0143] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A general self-supervised learning Raman spectroscopy noise reduction method, characterized in that: include, Step 1: Obtain Raman data; Step 2: Raman data preprocessing, The baseline of the Raman data was removed using an adaptive iterative reweighted penalized least squares method, and the spectrum after baseline removal was standardized to obtain preprocessed spectral data. Step 3: Use a downsampling strategy The preprocessed spectral data is processed to obtain two noise-independent sub-spectral sets. and Each sub-spectral set includes 10 sub-spectrums; Step 4: Input the preprocessed spectral data and the corresponding two subsets of the spectrum into the U-Net network. Obtain the spectrum after noise reduction. In step 3, the downsampling strategy Slide the spectrum with a window size of 3, covering the 3 Raman shifts. Randomly select two different shifts and copy their corresponding intensity values ​​to create a subspectrum with the same dimensions as the original spectrum. Repeat this operation twice to obtain the final subspectral set. and , The U-Net network The training method is as follows: Stochastic gradient descent (SGD) was applied to the U-Net network. loss function for: ； In the formula: in: These are the weights of the corresponding loss terms. This indicates that the input Raman spectrum is an n-dimensional real vector. and This represents the two sub-spectral sets obtained from downsampling in step 2. U-Net network The training process is converged based on the difference in the loss function between two iterations. When the difference in loss is less than 10... -4 The model stops training when the time comes.

2. The general self-supervised learning Raman spectroscopy noise reduction method as described in claim 1, characterized in that, The U-Net network It contains 5 modules. The first module includes two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and splice layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layers extract features, and the extracted features are passed to the second module. At the same time, the extracted features are passed to the fifth module by retaining low-level features through the copy and splice layer. The second module includes two one-dimensional convolutional layers, two ReLU activation layers, one max pooling layer, and one copy and splice layer. The one-dimensional convolutional layers, ReLU activation layers, and max pooling layers are used for feature extraction. The extracted features are passed to the third module, while the extracted features are passed to the fourth module by retaining low-level features through the copy and splice layer. The third module includes two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The one-dimensional convolutional layers and ReLU activation layers extract features, and the extracted features are restored through the upsampling layer and passed to the fourth module. The fourth module includes two one-dimensional convolutional layers, two ReLU activation layers, and one upsampling layer. The convolutional layers and ReLU activation layers concatenate the features passed from the second module with the features passed from the third module, and then the upsampling layer restores the concatenated features before passing them to the fifth module. The fifth module consists of three one-dimensional convolutional layers and three ReLU activation layers. It concatenates the features passed from the first module with the features passed from the fourth module to reconstruct the spectrum.

3. The general self-supervised learning Raman spectroscopy noise reduction method as described in claim 1, characterized in that, The learning rate of the stochastic gradient descent (SGD) method is 0.

003.

4. A computer device, characterized in that, include: Memory; One or more processors are coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, the one or more applications being configured to perform a general self-supervised learning Raman spectroscopy noise reduction method as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be called by a processor to execute the general self-supervised learning Raman spectroscopy noise reduction method as described in any one of claims 1 to 3.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the general self-supervised learning Raman spectroscopy noise reduction method as described in any one of claims 1 to 3.

Images