Raman denoising method, Raman denoising model, Raman imaging method and Raman imaging equipment

Abstract

The invention relates to a Raman denoising method. The method comprises the following steps: encoding an initial feature map of a Raman spectrum or a Raman image to obtain an encoded feature map; wherein the coding processing comprises the steps of performing feature extraction processing on the initial feature map to obtain a coding separation feature map, and performing feature fusion processing on the coding separation feature map to obtain a coding feature map; carrying out dimension integration and feature integration processing on the coding feature map to obtain an undertaking feature map; decoding the undertaking feature map to obtain a decoded feature map after noise of the corresponding Raman spectrum or Raman image is filtered; wherein the decoding processing comprises the steps of carrying out expansion processing on the coding feature map to obtain a sampling feature map, splicing the sampling feature map and the coding separation feature map to obtain a decoding splicing feature map, and carrying out feature extraction processing on the decoding splicing feature map to obtain a decoding feature map. According to the invention, the Raman spectrum or the Raman image can be denoised in consideration of the spatial dimension and the spectral dimension, and the denoising effect is strong.

Description

Technical Field

[0001] The invention relates to the technical field of spectrum measurement, and in particular to a Raman denoising method. Background Art

[0002] Modern spectroscopic methods include infrared spectroscopy, Raman spectroscopy, fluorescence spectroscopy, mass spectrometry and other methods. Among them, Raman spectroscopy analyzes samples at the molecular level and identifies the fingerprint spectral characteristics of their rotational or vibrational modes in a non-destructive and label-free manner, that is, Raman spectroscopy, which is a highly specific optical mode. Raman spectroscopy is mainly used to study the vibration and structural characteristics of molecular groups, and is widely used in many fields such as life sciences, medicine, and extraterrestrial material detection. Since each peak in the Raman spectrum has a one-to-one correspondence with the group of a chemical molecule, the type and number of chemical bonds contained in the chemical molecule can be inverted by analyzing specific peaks in the Raman spectrum. In biomedical research, Raman spectroscopy can obtain the composition of biochemical substances in specific structures in living cells of biological tissues in a non-destructive and label-free manner, such as the content and distribution of carbohydrates, proteins, lipids and nucleic acids, which makes Raman spectroscopy have great application potential in biomedical research.

[0003] Compared with Raman spectroscopy, the Raman images obtained by Raman imaging can contain both the chemical information and spatial distribution characteristics of the sample. The process of collecting the Raman spectrum of a specific site and recording the spatial information of the site, and arranging the collected Raman spectrum according to the corresponding spatial site to obtain a visualized three-dimensional data set is Raman imaging. For a simple single-layer sample, the data finally obtained by Raman imaging is three-dimensional, including two-dimensional plane space and the third-dimensional scattered light wavelength information, that is, spatial coordinate information and material spectrum information. Thanks to the advantages of Raman imaging such as non-destructive, label-free, and molecular imaging with subcellular resolution, many biomedical applications have been realized. Among them, Raman imaging of cells can reveal the relative concentration and spatial distribution information of substances such as nucleic acids, lipids and proteins in cells, thereby analyzing the biochemical composition state of cells, which is of great significance for analyzing cell differentiation.

[0004] The core principle of Raman spectroscopy and Raman imaging is Raman scattering, that is, when the incident laser is scattered by molecules, most of the scattered light has the same wavelength as the incident light, but a very small part of the scattered light has a different wavelength from the incident light. The change in wavelength is determined by the chemical structure of the scattering substance. Since the Raman scattering background generated by water is negligible, Raman spectroscopy or Raman imaging can be used to study biological samples, such as living cells, without interference from water.

[0005] However, the Raman scattering process is inherently weak. arrive Only one scattered photon may undergo Raman scattering among the scattered photons. The scarcity of scattering molecules makes the Raman signal very weak, which in turn increases the proportion of background noise in the overall signal. Taking the case of low-intensity Raman signals as an example, the detection of single molecules or ultra-low concentrations of analytes is susceptible to noise interference generated during the low-intensity signal acquisition process; similarly, for the identification of deep lesions inside biological tissues, deeper tissue penetration means a longer photon propagation path, and tissue heterogeneity causes Raman signals in deeper areas to be interfered by noise; in ultra-fast imaging, the acquisition time of each detection point is short, resulting in a low signal-to-noise ratio (SNR) of the Raman image. Due to similar application scenarios, the impact of noise has become non-negligible in the existing use of Raman scattering, and it is difficult to accurately obtain a high signal-to-noise ratio Raman image under complex noise interference. Summary of the invention

[0006] Based on this, the object of the present invention is to provide a Raman denoising method that can take into account the complex correlations in the spatial and spectral dimensions in spectral data.

[0007] A Raman denoising method, comprising: Performing encoding processing on the initial feature map of the Raman spectrum or Raman image to obtain an encoded feature map; wherein the encoding processing includes performing feature extraction processing on the initial feature map to obtain an encoded separation feature map, and performing feature fusion processing on the encoded separation feature map to obtain an encoded feature map; Performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; The inherited feature map is decoded to obtain a corresponding Raman spectrum or a Raman image with noise filtered to obtain a decoded feature map; wherein the decoding process includes expanding the inherited feature map to obtain a sampling feature map, splicing the sampling feature map with the coding separation feature map to obtain a decoding splicing feature map, and performing feature extraction on the decoding splicing feature map to obtain a decoding feature map.

[0008] Furthermore, the feature extraction processing of the initial feature map specifically includes: performing independent convolution processing on the features of the initial feature map of each channel to obtain multiple coded convolution feature maps, and performing point-by-point convolution processing on all the coded convolution feature maps to obtain coded separation feature maps.

[0009] Furthermore, the feature fusion processing of the coding separation feature map specifically includes: extracting the spatial dimension features of the coding separation feature map to obtain a spatial feature map; extracting the spectral dimension features of the coding separation feature map to obtain a spectral feature map; splicing the spatial feature map and the spectral feature map to obtain a coding splicing feature map; performing dimensionality reduction processing on the coding splicing feature map to obtain a first coding dimensionality reduction feature map; performing pooling processing on the first coding dimensionality reduction feature map to obtain a coding pooling feature map; performing attention allocation processing on the coding pooling feature map to obtain an attention feature map; multiplying the attention feature map with the first coding dimensionality reduction feature map to obtain a fused feature map with different weights.

[0010] Furthermore, the feature extraction processing of the decoded spliced ​​feature map specifically includes: performing independent convolution processing on the features of the decoded spliced ​​feature map of each channel to obtain multiple decoded convolution feature maps, and performing point-by-point convolution processing on all the decoded convolution feature maps to obtain a decoded feature map.

[0011] The present invention also provides a Raman denoising model, comprising: At least one encoding unit, used for encoding an initial feature map of a Raman spectrum or a Raman image to obtain an encoding feature map; wherein the encoding unit comprises an encoding separation module and a fusion module, the encoding separation module is used for performing feature extraction processing on the initial feature map to obtain an encoding separation feature map, and the fusion module is used for performing feature fusion processing on the encoding separation feature map to obtain an encoding feature map; A receiving unit, used for performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; At least one decoding unit is used to decode the inherited feature map to obtain a decoding feature map after the corresponding Raman spectrum or Raman image is filtered out of noise; wherein the decoding unit includes an upsampling module, a decoding connection module and a decoding separation module, the upsampling module is used to expand the inherited feature map to obtain a sampling feature map, the decoding connection module is used to splice the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and the decoding separation module is used to perform feature extraction on the decoding splicing feature map to obtain a decoding feature map.

[0012] Furthermore, the coding separation module is used to perform independent convolution processing on the features of the initial feature map of each channel to obtain multiple coding convolution feature maps, and perform point-by-point convolution processing on all the coding convolution feature maps to obtain coding separation feature maps.

[0013] Furthermore, the fusion module is used to extract the spatial dimension features of the coding separation feature map to obtain a spatial feature map; extract the spectral dimension features of the coding separation feature map to obtain a spectral feature map; splice the spatial feature map and the spectral feature map to obtain a coding splicing feature map; perform dimensionality reduction processing on the coding splicing feature map to obtain a first coding dimensionality reduction feature map; perform pooling processing on the first coding dimensionality reduction feature map to obtain a coding pooling feature map; perform attention allocation processing on the coding pooling feature map to obtain an attention feature map; multiply the attention feature map with the first coding dimensionality reduction feature map to obtain a fusion feature map with different weights.

[0014] Furthermore, the decoding separation module is used to perform independent convolution processing on the features of the decoding splicing feature map of each channel to obtain multiple decoding convolution feature maps, and perform point-by-point convolution processing on all the decoding convolution feature maps to obtain the decoding feature map.

[0015] The present invention also provides a Raman imaging method, comprising: Acquiring an initial characteristic map of a Raman spectrum or a Raman image; Performing encoding processing on the initial feature map of the Raman spectrum or Raman image to obtain an encoded feature map; wherein the encoding processing includes performing feature extraction processing on the initial feature map to obtain an encoded separation feature map, and performing feature fusion processing on the encoded separation feature map to obtain an encoded feature map; Performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; The inherited feature map is decoded to obtain a corresponding Raman spectrum or a decoding feature map after the Raman image is noise-filtered; wherein the decoding process includes expanding the encoding feature map to obtain a sampling feature map, splicing the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and performing feature extraction on the decoding splicing feature map to obtain a decoding feature map.

[0016] The present invention also provides a Raman imaging device, comprising: A Raman spectrometer, used to obtain an initial characteristic map of a Raman spectrum or a Raman image; At least one encoding unit, used for encoding an initial feature map of a Raman spectrum or a Raman image to obtain an encoding feature map; wherein the encoding unit comprises an encoding separation module and a fusion module, the encoding separation module is used for performing feature extraction processing on the initial feature map to obtain an encoding separation feature map, and the fusion module is used for performing feature fusion processing on the encoding separation feature map to obtain an encoding feature map; A receiving unit, used for performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; At least one decoding unit is used to decode the inherited feature map to obtain a decoding feature map after the corresponding Raman spectrum or Raman image is filtered out of noise; wherein the decoding unit includes an upsampling module, a decoding connection module and a decoding separation module, the upsampling module is used to expand the encoding feature map to obtain a sampling feature map, the decoding connection module is used to splice the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and the decoding separation module is used to perform feature extraction on the decoding splicing feature map to obtain a decoding feature map.

[0017] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the modules of the Raman denoising model of the present invention.

[0019] Figure 2 The figure is a schematic diagram of the operation flow of the Raman denoising model of the present invention.

[0020] Figure 3 This is an input image with noise when testing the Raman denoising model of the present invention.

[0021] Figure 4 for Figure 3 The image obtained after the input image is denoised by wavelet.

[0022] Figure 5 for Figure 3 The input image is obtained by SG filtering.

[0023] Figure 6 for Figure 3 The input image is processed by a one-dimensional U-Net.

[0024] Figure 7 for Figure 3 The input image is subjected to the Raman denoising model of the present invention to obtain a reconstructed image.

[0025] Figure 8 for Figure 3 The input image corresponds to an original image without noise.

[0026] Fig. 9 The input spectrum with noise when the Raman denoising model of the present invention is tested.

[0027] Fig.10 for Fig. 9 The input spectrum is obtained after wavelet denoising.

[0028] Fig.11 for Fig. 9The input spectrum is obtained after SG filtering.

[0029] Fig.12 for Fig. 9 The input spectrum is obtained after being processed by a one-dimensional U-Net.

[0030] Fig.13 for Fig. 9 The input spectrum is subjected to the Raman denoising model of the present invention to obtain a reconstructed spectrum.

[0031] Fig.14 for Fig. 9 The input spectrum corresponds to the original spectrum without noise. DETAILED DESCRIPTION

[0032] The applicant understands that the current Raman denoising method using deep convolutional networks can only perform denoising based on a single stage, and it is necessary to continuously set denoising parameters. It only performs spectral denoising on the Raman spectra in the Raman image one by one or only performs image denoising on the plane Raman image corresponding to a certain wave number. It does not consider the complex correlation between the spatial dimension and the spectral dimension in the Raman spectrum or Raman image stack, which may cause some important information to be lost during the reconstruction process, and the best denoising effect cannot be achieved, and the network generalization ability is weak. Therefore, the applicant tried to build a model using a three-dimensional convolutional network, while considering the independent characteristics of Raman spectra and Raman images in the spatial dimension and the spectral dimension; due to the increase in the number of parameters, the model may not be able to calculate or the computational cost is too high. For this reason, the applicant combines a fusion module to extract and fuse the features in the spectral dimension and the spatial dimension through pseudo-three-dimensional convolution, thereby reducing the number of calculation parameters while taking into account the complex correlation between the spatial and spectral dimensions in the spectral data.

[0033] The Raman imaging device includes a Raman spectrometer and a Raman denoising model. The Raman spectrometer obtains a Raman spectrum containing chemical information of a sample, and performs Raman imaging according to the Raman spectrum to obtain a Raman image. The Raman denoising model is connected to the Raman spectrometer, receives the Raman spectrum and the Raman image from the Raman spectrometer, denoises the Raman spectrum to obtain a reconstructed spectrum, and denoises the Raman image to obtain a reconstructed image.

[0034] See also Figure 1 The Raman denoising model includes at least one encoding unit, one receiving unit and at least one decoding unit. Figure 2 The schematic diagram of the process of running the Raman denoising model is shown in FIG. Perform encoding processing to obtain the encoding feature map , where the Raman spectrum or Raman image is used as the initial feature map , the feature map in the encoding unit adopts express, is the sub-number of the feature map in the encoding unit, is the serial number of the coding unit where the feature map is located; the receiving unit is used to encode the feature map Perform dimension integration and feature integration processing to obtain a receiving feature map, wherein the feature map in the receiving unit is adopted express, is the sub-sequence number of the feature map in the receiving unit; the decoding unit is used to decode the receiving feature map to obtain a decoded feature map, wherein the decoded feature map is a reconstructed spectrum obtained after the noise has been filtered out corresponding to the Raman spectrum, or a reconstructed image obtained after the noise has been filtered out corresponding to the Raman image, and the feature map in the decoding unit adopts express, is the sub-number of the feature map in the decoding unit, is the decoding unit number where the feature map is located.

[0035] The following is a detailed description using a Raman denoising model consisting of two encoding units, one receiving unit and two decoding units as an example.

[0036] The coding unit includes a coding dimension upgrading module, a coding separation module, a fusion module and a compression module.

[0037] Specifically, in the first encoding unit, the encoding dimension-upgrading module is used to encode the initial feature map Perform dimensionality increase processing to obtain the encoded dimensionality increase feature map Specifically, the encoding dimension-upgrading module includes a 1×1×1 convolutional layer, and the initial feature map The input channel number is 1, the height is 64, the width is 64, and the number of spectral bands is 500. , get the encoded dimension-upgraded feature map for .

[0038] The encoding separation module is used to encode the dimension-upgraded feature map Perform feature extraction to obtain the encoding separation feature map Specifically, the encoding separation module includes a channel-by-channel convolution layer and a point-by-point convolution layer. The number of convolution kernels of the channel-by-channel convolution layer is set to 16. Each convolution kernel corresponds to an input channel and performs independent convolution processing to obtain multiple encoding convolution feature maps. ; The point-by-point convolution layer encodes all convolution feature maps Perform point-by-point convolution to obtain the encoding separation feature map , the encoding separation feature map for .

[0039] In particular, assuming the size of the input feature map is ,in , , , They represent the number of feature channels, height, width, and spectral bands of the input feature map. Assume that the convolution kernel size of the input feature map is ,in is the dimension of the convolution kernel, , They represent the number of input channels and output channels of the convolution kernel respectively.

[0040] For existing modules using 3D convolution, calculate the number of parameters for:

[0041] For the encoding separation module of this application, the number of calculation parameters of the channel-by-channel convolution layer , the number of computational parameters of the point-by-point convolutional layer And the total calculation parameters They are:

[0042]

[0043]

[0044] We can therefore compare the computational cost of depthwise separable 3D convolution and standard 3D convolution:

[0045] From the above formula, it can be seen that the amount of calculation parameters of the coding separation module of the present application is greatly reduced compared with the existing module using three-dimensional convolution, and the same performance can be achieved at a lower calculation cost.

[0046] The fusion module is used to separate the encoding feature map Perform feature fusion processing to obtain a fused feature map Specifically, the fusion module includes a pseudo 3D convolution block, a connection layer, three 1×1×1 convolution layers, an adaptive average pooling layer, an attention layer and a fusion layer; the pseudo 3D convolution block extracts the encoding separation feature map The spatial dimension features of the spatial feature map are obtained , extract the encoding separation feature map The spectral dimension characteristics of the spectral feature map are obtained , wherein the spatial feature map for , the spectral characteristic diagram for ; The connection layer converts the spatial feature map And spectral characteristics Splice to get the encoding splicing feature map , the encoding splicing feature map for ; The coding splicing feature map After the first 1×1×1 convolutional layer, the dimension is reduced to obtain the first encoded dimension reduction feature map , the first encoding dimension reduction feature map for ; The adaptive average pooling layer reduces the dimension of the first encoding feature map Perform pooling to obtain the encoded pooling feature map , the encoded pooling feature map for ; The encoded pooling feature map After the second and third 1×1×1 convolutional layers and the attention layer, the attention feature map of the attention weight distribution of each channel is obtained. , where the attention layer uses the Sigmoid activation function and the attention feature map for ; The fusion layer reduces the dimension of the first encoding feature map And attention feature map Multiply to get a fusion feature map with different weights for each channel , the fusion feature map for .

[0047] The compression module is used to fusion feature map Compression processing is performed to obtain a compressed feature map Specifically, the compression module includes a 2×2×2 maximum pooling layer, and the fusion feature map The size of the compressed feature map after the maximum pooling layer is obtained by , the compressed feature map for .

[0048] The second encoding unit is connected to the first encoding unit and receives the compressed feature map from the first encoding unit. In the second encoding unit, the encoding dimension increasing module is used to compress the compressed feature map. Perform dimensionality increase processing to obtain the encoded dimensionality increase feature map Specifically, the encoding dimension-upgrading module includes a 1×1×1 convolutional layer to obtain the encoding dimension-upgrading feature map for .

[0049] The encoding separation module is used to encode the dimension-upgraded feature map Perform feature extraction to obtain the encoding separation feature map Specifically, the encoding separation module includes a channel-by-channel convolution layer and a point-by-point convolution layer. The number of convolution kernels of the channel-by-channel convolution layer is set to 16. Each convolution kernel corresponds to an input channel and performs independent convolution processing to obtain multiple encoding convolution feature maps. ; The point-by-point convolution layer encodes all convolution feature maps Perform point-by-point convolution to obtain the encoding separation feature map , the encoding separation feature map for .

[0050] The fusion module is used to separate the encoding feature map Perform feature fusion processing to obtain a fused feature map Specifically, the fusion module includes a pseudo 3D convolution block, a connection layer, three 1×1×1 convolution layers, an adaptive average pooling layer, an attention layer and a fusion layer; the pseudo 3D convolution block extracts the encoding separation feature map The spatial dimension features of the spatial feature map are obtained , extract the encoding separation feature map The spectral dimension characteristics of the spectral feature map are obtained , wherein the spatial feature map for , the spectral characteristic diagram for ; The connection layer converts the spatial feature map And spectral characteristics Splice to get the encoding splicing feature map , the encoding splicing feature map for ; The coding splicing feature map After the first 1×1×1 convolutional layer, the dimension is reduced to obtain the first encoded dimension reduction feature map , the first encoding dimension reduction feature map for ; The adaptive average pooling layer reduces the dimension of the first encoding feature map Perform pooling to obtain the encoded pooling feature map , the encoded pooling feature map for ; The encoded pooling feature map After the second and third 1×1×1 convolutional layers and the attention layer, the attention feature map of the attention weight distribution of each channel is obtained. , where the attention layer uses the Sigmoid activation function and the attention feature map for ; The fusion layer reduces the dimension of the first encoding feature map And attention feature map Multiply to get a fusion feature map with different weights for each channel , the fusion feature map for .

[0051] The compression module is used to fusion feature map Compression processing is performed to obtain a compressed feature map Specifically, the compression module includes a 2×2×2 maximum pooling layer, and the fusion feature map The size of the compressed feature map after the maximum pooling layer is obtained by , the compressed feature map for , the compressed feature map is used as the final encoded feature map.

[0052] The receiving unit includes at least one receiving dimension-upgrading module, a receiving separation module and at least one receiving dimension-downgrading module, wherein the number of receiving dimension-upgrading modules corresponds to the number of receiving dimension-downgrading modules. The receiving dimension-upgrading module includes a 1×1×1 convolutional layer for compressing the feature map. Perform dimensionality increase processing to obtain the dimensionality increase feature map , the inherited dimension-upgraded feature map for The specific structure of the receiving and separating module is the same as that of the encoding and separating module, and it receives the dimension-upgraded feature map. Perform feature extraction to obtain the separation feature map , the separation feature map for The concatenated dimension reduction module includes a 1×1×1 convolutional layer for concatenating the feature map Perform dimensionality reduction processing to obtain the feature map of dimensionality reduction , the inherited dimension reduction feature map for .

[0053] The decoding unit includes an up-sampling module, a decoding connection module, a decoding first dimensionality reduction module, a decoding separation module and a decoding second dimensionality reduction module.

[0054] Specifically, in the first decoding unit, the upsampling module includes a 2×2×2 upsampling layer for receiving the reduced dimension feature map. Perform expansion processing to obtain the sampling feature map , the sampling feature map for ; The decoding connection module includes a connection layer that samples the feature map Separate feature map from the second encoding unit Concatenate to get the decoded concatenated feature map , the decoded splicing feature map for ; The first dimensionality reduction module for decoding includes a 1×1×1 convolutional layer for decoding the spliced ​​feature map Perform dimensionality reduction processing to obtain the first decoding dimensionality reduction feature map , the first decoding dimension reduction feature map for ; The decoding separation module is used to decode the first dimension reduction feature map Perform feature extraction to obtain the decoding separation feature map Specifically, the encoding separation module includes a channel-by-channel convolution layer and a point-by-point convolution layer. The number of convolution kernels of the channel-by-channel convolution layer is set to 16. Each convolution kernel corresponds to an input channel and performs independent convolution processing to obtain multiple decoded convolution feature maps. ; The point-by-point convolution layer decodes all convolution feature maps Perform point-by-point convolution to obtain the decoding separation feature map , the decoding separation feature map for ; The second dimensionality reduction module for decoding includes a 1×1×1 convolutional layer for decoding separation feature map Perform dimensionality reduction processing to obtain the second decoding dimensionality reduction feature map , the second decoding dimension reduction feature map for .

[0055] The second decoding unit is connected to the first decoding unit and receives the second decoded dimensionality reduction feature map from the first decoding unit. In the second decoding unit, the decoding unit includes an upsampling module, a decoding connection module, a decoding first dimensionality reduction module, a decoding separation module and a decoding second dimensionality reduction module. The upsampling module includes a 2×2×2 upsampling layer to decode the second decoded dimensionality reduction feature map. Perform expansion processing to obtain the sampling feature map , the sampling feature map for ; The decoding connection module includes a connection layer that samples the feature map Separate feature map from the first encoding unit Concatenate to get the decoded concatenated feature map , the decoded splicing feature map for ; The first dimensionality reduction module for decoding includes a 1×1×1 convolutional layer for decoding the spliced ​​feature map Perform dimensionality reduction processing to obtain the first decoding dimensionality reduction feature map , the first decoding dimension reduction feature map for ; The decoding separation module is used to decode the first dimension reduction feature map Perform feature extraction to obtain the decoding separation feature map Specifically, the encoding separation module includes a channel-by-channel convolution layer and a point-by-point convolution layer. The number of convolution kernels of the channel-by-channel convolution layer is set to 16. Each convolution kernel corresponds to an input channel and performs independent convolution processing to obtain multiple decoded convolution feature maps. ; The point-by-point convolution layer decodes all convolution feature maps Perform point-by-point convolution to obtain the decoding separation feature map , the decoding separation feature map for ; The second dimensionality reduction module for decoding includes a 1×1×1 convolutional layer for decoding separation feature map Perform dimensionality reduction processing to obtain the second decoding dimensionality reduction feature map , the second decoding dimension reduction feature map for , the second decoding dimensionality reduction feature map is used as the final decoding feature map.

[0056] Since the composition structure of each coding unit is exactly the same, only the parameters of the coding feature map undergo the same logical changes, which will not be described here. It can be understood by those skilled in the art that when there are multiple coding units, the second coding unit compresses the feature map. Encoding process to obtain compressed feature map ; The third encoding unit compresses the feature map Encoding process to obtain compressed feature map , and so on, how many encoding units actually need to be set by the initial feature map The best denoising effect can be achieved by selecting the parameters and the denoising degree that the user wants to achieve.

[0057] Similarly, the composition structure of each decoding unit is exactly the same, except that the parameters of the decoding feature map undergo the same logical changes, which will not be repeated here. It should be noted that the number of decoding units should remain the same as the number of encoding units, and in the decoding connection module of the decoding unit, the decoding connection module will connect the decoding feature map obtained by the upsampling module of the decoding unit and the encoding feature map obtained by the encoding separation module of the encoding unit together, and the encoding unit to which it is connected and the decoding unit corresponding to the decoding connection module should be symmetrical with respect to the center of the receiving unit. Specifically, when the Raman denoising model has five encoding units, the decoding connection module of the first decoding unit should be connected to the fifth encoding unit, the decoding connection module of the second decoding unit should be connected to the fourth encoding unit, and so on.

[0058] After obtaining the Raman denoising model of the present application, in order to verify its practical application effect, it is trained and tested by the following method.

[0059] 169 high SNR spectral Raman images of breast cancer cells (including 1.4 million Raman spectra) are used as clear Raman images. The image size is 64×64×500, where 64 represents the spatial physical size of the Raman imaging area and 500 represents the relative wave number of the Raman image. The spectral range of the Raman image is arrive , the spatial resolution is 0.5μm.

[0060] The intensity of each Raman image depends on the sample characteristics, the acquisition environment, and the Raman spectrometer. Therefore, in order to remove the intensity non-uniformity between Raman images, the Raman image data is normalized to the maximum and minimum values:

[0061] in, and They represent the maximum and minimum intensity in the Raman image respectively. The size of the normalized Raman image is H×W×B, where H, W, and B represent its height, width, and the number of spectral bands, respectively.

[0062] The noise of Raman spectroscopy mainly includes shot noise, dark noise and readout noise. Among them, shot noise is related to the intrinsic quantum process in the process of photon generation. It is formed due to quantum fluctuations when photons are converted into electrons according to the photoelectric effect and obeys Poisson distribution. As a signal-dependent noise, it is proportional to the signal intensity and is also called multiplicative noise. When electrons are converted into electrical signals, readout noise and dark noise are introduced, which obey Gaussian distribution. Gaussian noise is an additive noise that is independent of signal intensity. In order to be more consistent with the noise distribution of real data, a noise model of mixed Poisson-Gaussian distribution is selected to generate a noise data set based on the high SNR spectral Raman image of breast cancer cells. The mathematical expression of the mixed Poisson-Gaussian noise model is:

[0063] in, is Poisson noise, weighted by signal strength; , is a constant that depends on the sensor and analog gain; is Gaussian noise, which is the remaining noise that is unrelated to the signal; is the random noise obtained by mixing. Mixed Poisson-Gaussian noise is added to 169 clear images to generate 169 noisy low SNR Raman images to form a data set. The Raman image after adding noise is linearly modeled as:

[0064] in, , They represent the noisy Raman image, the clean Raman spectrum image and the added mixed random noise respectively.

[0065] According to the ratio of 85:10:5 Divided into training set, validation set and test set. Among them, 144 hyperspectral Raman images are used as training set, 17 hyperspectral Raman images are used as validation set, and 8 hyperspectral Raman images are used as test set, which are input into the Raman denoising model to obtain the corresponding output results.

[0066] In order to better quantify the reconstruction effect and compare it with other reconstruction methods, the structural similarity is introduced ( SSIM ) and peak signal-to-noise ratio ( PSNR ) to evaluate the reconstructed image.

[0067] Two images x and y Between SSIM is defined as:

[0068] image y Relative to label image xThe PSNR is defined as:

[0069] See also Figures 3 to 8 , we can see the comparison between the noisy Raman image after different denoising processes and the noise-free original image. Specifically, the table headers are the noisy input Raman image, the Raman image after wavelet denoising, the Raman image after SG filtering, the Raman image after one-dimensional U-Net network denoising and the reconstructed image after Raman denoising of the present invention. Based on this, a quantitative evaluation comparison table of different reconstruction methods in the spatial dimension is established to more intuitively show the effect.

[0070] Table 1

[0071] Please refer to Table 1, which is a quantitative evaluation comparison table of different reconstruction methods in the spatial dimension. It can be seen that the SSIM of the reconstructed image of the Raman denoising model of the present invention reaches 0.9489, and the PSNR reaches 40.58dB. Compared with other denoising methods, it is closer to the original image and has a significant outstanding effect.

[0072] In addition, the mean square error ( MSE ) and signal-to-noise ratio ( SNR ) Evaluate the reconstructed spectrum a and b of MSE It is expressed as:

[0073] spectrum a and b of SNR It is expressed as:

[0074] See also Figures 9 to 14 , we can see the comparison between the noisy Raman spectrum after different denoising processes and the noise-free original spectrum. Specifically, the table headers are the noisy input Raman spectrum, the Raman spectrum after wavelet denoising, the Raman spectrum after SG filtering, the Raman spectrum after one-dimensional U-Net network denoising and the reconstructed spectrum after Raman denoising of the present invention. Based on this, a quantitative evaluation comparison table of different reconstruction methods in the spectral dimension is established to more intuitively show the effect.

[0075] Table 2

[0076] Please refer to Table 2, which is a quantitative evaluation comparison table of different reconstruction methods in the spectral dimension. It can be seen that the MSE of the reconstructed spectrum of the Raman denoising model of the present invention reaches 0.000019, and the SNR reaches 18.71dB, which is also better than other denoising methods.

[0077] The Raman of the present invention can realize overall denoising of the spectral axis and the spatial axis simultaneously from the two dimensions of spectrum and space, and compared with the denoising result of a single stage, the proposed method can achieve better denoising effects in the spectral and spatial dimensions without introducing any additional hardware. The low SNR cell Raman image can be reconstructed into a high SNR cell Raman image only through post-processing on the data end.

[0078] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A Raman denoising method, characterized in that: include: Performing encoding processing on the initial feature map of the Raman spectrum or Raman image to obtain an encoded feature map; wherein the encoding processing includes performing feature extraction processing on the initial feature map to obtain an encoded separation feature map, and performing feature fusion processing on the encoded separation feature map to obtain an encoded feature map; Performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; The inherited feature map is decoded to obtain a corresponding Raman spectrum or a Raman image with noise filtered to obtain a decoded feature map; wherein the decoding process includes expanding the inherited feature map to obtain a sampling feature map, splicing the sampling feature map with the coding separation feature map to obtain a decoding splicing feature map, and performing feature extraction on the decoding splicing feature map to obtain a decoding feature map.

2. The Raman denoising method according to claim 1, characterized in that: The feature extraction processing of the initial feature map specifically includes: performing independent convolution processing on the features of the initial feature map of each channel to obtain multiple coded convolution feature maps, and performing point-by-point convolution processing on all the coded convolution feature maps to obtain coded separation feature maps.

3. The Raman denoising method according to claim 2, characterized in that: The feature fusion processing of the coding separation feature map specifically includes: extracting the spatial dimension features of the coding separation feature map to obtain a spatial feature map; extracting the spectral dimension features of the coding separation feature map to obtain a spectral feature map; splicing the spatial feature map and the spectral feature map to obtain a coding splicing feature map; performing dimensionality reduction processing on the coding splicing feature map to obtain a first coding dimensionality reduction feature map; performing pooling processing on the first coding dimensionality reduction feature map to obtain a coding pooling feature map; performing attention allocation processing on the coding pooling feature map to obtain an attention feature map; multiplying the attention feature map with the first coding dimensionality reduction feature map to obtain a fused feature map with different weights.

4. The Raman denoising method according to claim 3, characterized in that: The feature extraction processing of the decoding splicing feature map specifically includes: performing independent convolution processing on the features of the decoding splicing feature map of each channel to obtain multiple decoding convolution feature maps, and performing point-by-point convolution processing on all the decoding convolution feature maps to obtain a decoding feature map.

5. A Raman denoising model, characterized in that: include: At least one encoding unit, used for encoding an initial feature map of a Raman spectrum or a Raman image to obtain an encoding feature map; wherein the encoding unit comprises an encoding separation module and a fusion module, the encoding separation module is used for performing feature extraction processing on the initial feature map to obtain an encoding separation feature map, and the fusion module is used for performing feature fusion processing on the encoding separation feature map to obtain an encoding feature map; A receiving unit, used for performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; At least one decoding unit is used to decode the inherited feature map to obtain a decoding feature map after the corresponding Raman spectrum or Raman image is filtered out of noise; wherein the decoding unit includes an upsampling module, a decoding connection module and a decoding separation module, the upsampling module is used to expand the inherited feature map to obtain a sampling feature map, the decoding connection module is used to splice the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and the decoding separation module is used to perform feature extraction on the decoding splicing feature map to obtain a decoding feature map.

6. The Raman denoising model according to claim 5, characterized in that: The coding separation module is used to perform independent convolution processing on the features of the initial feature map of each channel to obtain multiple coded convolution feature maps, and perform point-by-point convolution processing on all the coded convolution feature maps to obtain coded separation feature maps.

7. The Raman denoising model according to claim 6, characterized in that: The fusion module is used to extract the spatial dimension features of the coding separation feature map to obtain a spatial feature map; extract the spectral dimension features of the coding separation feature map to obtain a spectral feature map; splice the spatial feature map and the spectral feature map to obtain a coding splicing feature map; perform dimensionality reduction processing on the coding splicing feature map to obtain a first coding dimensionality reduction feature map; perform pooling processing on the first coding dimensionality reduction feature map to obtain a coding pooling feature map; perform attention allocation processing on the coding pooling feature map to obtain an attention feature map; multiply the attention feature map with the first coding dimensionality reduction feature map to obtain a fusion feature map with different weights.

8. The Raman denoising model according to claim 7, characterized in that: The decoding separation module is used to perform independent convolution processing on the features of the decoding splicing feature map of each channel to obtain multiple decoding convolution feature maps, and perform point-by-point convolution processing on all the decoding convolution feature maps to obtain a decoding feature map.

9. A Raman imaging method, characterized in that: include: Acquiring an initial characteristic map of a Raman spectrum or a Raman image; Performing encoding processing on the initial feature map of the Raman spectrum or Raman image to obtain an encoded feature map; wherein the encoding processing includes performing feature extraction processing on the initial feature map to obtain an encoded separation feature map, and performing feature fusion processing on the encoded separation feature map to obtain an encoded feature map; Performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; The inherited feature map is decoded to obtain a corresponding Raman spectrum or a decoding feature map after the Raman image is noise-filtered; wherein the decoding process includes expanding the encoding feature map to obtain a sampling feature map, splicing the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and performing feature extraction on the decoding splicing feature map to obtain a decoding feature map.

10. A Raman imaging device, characterized in that: include: A Raman spectrometer, used to obtain an initial characteristic map of a Raman spectrum or a Raman image; At least one encoding unit, used for encoding an initial feature map of a Raman spectrum or a Raman image to obtain an encoding feature map; wherein the encoding unit comprises an encoding separation module and a fusion module, the encoding separation module is used for performing feature extraction processing on the initial feature map to obtain an encoding separation feature map, and the fusion module is used for performing feature fusion processing on the encoding separation feature map to obtain an encoding feature map; A receiving unit, used for performing dimension integration and feature integration processing on the encoding feature map to obtain a receiving feature map; At least one decoding unit is used to decode the inherited feature map to obtain a decoding feature map after the corresponding Raman spectrum or Raman image is filtered out of noise; wherein the decoding unit includes an upsampling module, a decoding connection module and a decoding separation module, the upsampling module is used to expand the encoding feature map to obtain a sampling feature map, the decoding connection module is used to splice the sampling feature map with the encoding separation feature map to obtain a decoding splicing feature map, and the decoding separation module is used to perform feature extraction on the decoding splicing feature map to obtain a decoding feature map.

Images