Mass spectrum image super-resolution reconstruction method and device for single cells

Abstract

The invention discloses a mass spectrum image super-resolution reconstruction method and device for a single cell, and the method comprises the steps: carrying out the morphology representation of the single cell through an optical microscope, obtaining a single cell optical image, and obtaining a single cell low-resolution mass spectrum image through a mass spectrum imaging experiment; then registering the single-cell optical image and the single-cell low-resolution mass spectrum image by using a registration module; in order to prevent optical diffraction interference, filtering the registered single-cell optical image through guided filtering, and inputting the registered and filtered single-cell optical image and a single-cell mass spectrum imaging graph into a multi-modal fusion module to generate a mass spectrum super-resolution reconstruction image; and finally, in order to be closer to an image imaged by a real mass spectrum instrument, a process of imaging simulation to the real instrument is carried out, and after simulation, the super-resolution image can be closer to a real mass spectrum high-resolution image. According to the invention, high-spatial-resolution mass spectrum imaging can be realized by using the existing low-resolution mass spectrometer.

Description

Technical Field

[0001] This invention relates to the field of biochemical image processing technology, and in particular to a method and apparatus for super-resolution reconstruction of mass spectrometry images of single cells. Background Technology

[0002] Mass spectrometry imaging (MSI) is a powerful chemical analysis technique. Mass spectrometers use an ion source to scan the sample surface point by point, causing desorption / ionization of substances at each sampling point. The obtained ions are separated by a mass analyzer, allowing for the identification and determination of the types and intensities of ions at each sampling point. During this process, the MSI software simultaneously records the spatial location information of each sampling point. Finally, by integrating the ion intensities of ions with the same mass-to-charge ratio at each location, the distribution of that ion in the sample can be obtained. MSI's combination of mass spectrometry analysis and spatial information is widely used to observe the spatial distribution characteristics of various biomolecules (such as metabolites, lipids, peptides, and proteins) within single cells.

[0003] Among the characteristics of MSI technology, spatial resolution is one of its most critical evaluation metrics. Given the ubiquitous cellular heterogeneity within organisms, high spatial resolution MSI is essential for understanding the biological information determined by molecular distribution in complex samples. It can accurately locate and analyze subtle differences in molecular distribution, which are often closely related to cellular physiological states or disease progression. To achieve high spatial resolution, current methods generally employ a strategy of reducing the sampling spot size. While this improves spatial resolution, it also significantly increases analysis time, resulting in substantial data acquisition costs and excessively long computational processing times. Therefore, MSI analysis at high spatial resolution is relatively slow, especially for techniques like single-cell imaging. While single-cell imaging provides rapid, label-free, and localized chemical analysis at the single-cell and subcellular levels, making it an effective tool for studying cellular behavior mechanisms, the small size of single cells necessitates even higher spatial resolution. Achieving rapid imaging at high spatial resolution has become an urgent need and a developing trend in the field of mass spectrometry imaging.

[0004] In recent years, reference-based image super-resolution reconstruction methods have utilized additional image patches or images with high-resolution image textures and details to aid in the super-resolution reconstruction of a given low-resolution image. Using single-cell images from optical microscopes as reference images provides prior structural information about the single-cell images, enabling the recovery of more high-resolution details from limited low-resolution information and generating super-resolution images with better image quality. Summary of the Invention

[0005] The purpose of this application is to propose a method and apparatus for super-resolution reconstruction of mass spectrometry images for single cells, addressing the aforementioned technical problems. By using single-cell optical images as reference images, and using only a single mass spectrometry image and a single reference image, a high-resolution reconstructed mass spectrometry image with rich chemical information and detailed texture structure can be generated. This significantly reduces the conventional pixel-by-pixel scanning time by an order of magnitude, enabling high spatial resolution mass spectrometry imaging using existing low-resolution mass spectrometry instruments.

[0006] On the one hand, a method for super-resolution reconstruction of mass spectrometry images for single cells includes:

[0007] S1, Acquire low-resolution mass spectrometry image of a single cell. lr and single-cell optical images I op data;

[0008] S2, using interpolation methods, transforms the low-resolution mass spectrometry image of a single cell into... lr Upsampling and single-cell optical image I op Downsampled to the same pixel resolution, the downsampled optical image is input into the image registration module for registration, resulting in the registered single-cell optical image I. reg ;

[0009] S3, the registered single-cell optical image I is processed by the guided filtering module. reg Filtering is performed to obtain the registered and filtered reference image I. ref ;

[0010] S4, from single-cell low-resolution mass spectrometry image I lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr ;

[0011] S5, reconstruct the mass spectrometry super-resolution image I′ sr The final super-resolution reconstruction result I is obtained by simulating the imaging process with real instruments. sr .

[0012] Preferably, in step S1, a low-resolution mass spectrometry image of a single cell is acquired. lr and single-cell optical images I op The data processing is as follows:

[0013] S11, all cell sample culture and related operations are completed in a biosafety cabinet;

[0014] S12, the silicon wafer with adhered cells is dried at room temperature, and the cell morphology is characterized using a shape measurement laser microscopy system to obtain a single-cell optical image I. op ;

[0015] S13, mass spectrometry images of each pixel of a single cell are acquired using a mass spectrometry imager in point scanning mode. Based on the mass spectrometry images, low-resolution mass spectrometry images of single cells with different mass-to-charge ratio ion distributions are obtained. lr .

[0016] Preferably, in step S2, the downsampled optical image is input into the image registration module for registration to obtain the registered single-cell optical image I. reg Specifically, it includes:

[0017] The downsampled optical image is input into the image registration module; the image registration module is a spatial transformation network, and a conversion module consisting of four convolutional neural networks is connected after the registration module. The spatial transformation network uses nonlinear difference to perform affine transformation on the single-cell optical image and the single-cell low-resolution mass spectrometry image to obtain a mapping relationship, thereby realizing multimodal registration; the conversion module is used to eliminate the differences between modes.

[0018] Using registration loss L reg After a certain number of iterations of training, the network parameter weights of the registration module are retained, and the registered single-cell optical image I is output. reg ;

[0019] The registration loss L reg The calculation formula is as follows:

[0020] L reg =||I reg↓s -I lr || F

[0021] Among them, ||·|| F Represents the Frobenius norm; I lr This represents a low-resolution mass spectrometry image of a single cell; s is a downsampling factor used to downsample the single-cell optical image to the same pixel resolution as the low-resolution mass spectrometry image of the single cell; I reg↓s This indicates the image after downsampling the registered single-cell optical image by the downsampling factor s.

[0022] Preferably, S3 specifically includes:

[0023] A low-resolution mass spectrometry image of a single cell obtained through bicubic interpolation is used as the guide image. A guide filtering module filters the registered single-cell optical image to prevent interference from optical diffraction. The guide filtering module uses a structural loss function (L0). str The network parameters of the guided filtering module are optimized to output the registered and filtered reference image I. ref ;

[0024] The structural loss L str The calculation formula is:

[0025] I str =||I ref↓s -I lr || F

[0026] Among them, ||·|| F Represents the Frobenius norm; I lr This represents a low-resolution mass spectrometry image of a single cell; s is a downsampling factor used to downsample the registered and filtered image to the same pixel resolution as the low-resolution mass spectrometry image of the single cell; I ref↓s This indicates the image after downsampling the registered single-cell optical image by the downsampling factor.

[0027] Preferably, S4 specifically includes:

[0028] S41, from single-cell low-resolution mass spectrometry image I lr Randomly select n×n single-cell low-resolution mass spectrometry image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select a reference image patch I′ of size m×m at the corresponding position. ref , where the size of m is n×d;

[0029] S42, refer to image block I′ ref and single-cell low-resolution mass spectrometry image patch I′ lr As the input to the multimodal fusion module, the multimodal fusion module includes several levels of multimodal fusion modules connected in sequence, each level having the same structure; wherein, the first level of the multimodal fusion module uses reference image block I′ ref and single-cell low-resolution mass spectrometry image patch I′ lrAs input, before feeding the single-cell low-resolution mass spectrometry image into the convolutional neural network module, it is first upsampled using interpolation. A convolutional neural network module extracts features from both the single-cell low-resolution mass spectrometry image patch and the reference image patch, then merges the features, and passes them through another convolutional neural network module to obtain a new fused image. Each subsequent stage of the multimodal fusion module combines the fused image from the previous stage with the single-cell low-resolution mass spectrometry image I. lr and reference image I ref As input, it is processed through a four-layer convolutional neural network module after the last-stage multimodal fusion module;

[0030] S43, utilizing global network loss L total The network model is optimized; simultaneously, the network model parameters of the registration module and the multimodal fusion module are further adjusted through backpropagation. The registration accuracy and super-resolution reconstruction effect are improved iteratively a preset number of times. Each time, image patches are randomly selected again. After reaching the preset number of times, the super-resolution reconstructed image I′ is output. sr .

[0031] Preferably, the global network loss L total The calculation formula is:

[0032] L total =α1L str +α2L ion

[0033] L ion =||I′ sr↓s -I lr || F

[0034] Among them, ||·|| F L represents the Frobenius norm; str Indicates structural loss; I lr This represents the original low-resolution mass spectrometry image of a single cell, where s is a downsampling factor, and α1 and α2 are the weights of the loss function, respectively; I′ sr↓s This indicates that after reaching a preset number of iterations, the super-resolution reconstructed image will be output as the image downsampled by the sampling factor s.

[0035] Preferably, S5 specifically includes:

[0036] S51, for the super-resolution reconstructed image I′ sr Threshold segmentation is performed to separate single cells from the background, resulting in a cell mask. Poisson noise is then added to the single cell locations.

[0037] S52, threshold segmentation is used again to separate areas of high and low ion intensity within a single cell, resulting in an ion mask. For the high ion intensity regions, Gaussian noise is added on top of Poisson noise to obtain the final output image I. sr In this setting, pixel values ​​in high ion intensity regions are set to 1, while pixel values ​​in other regions are cleared to 0.

[0038] Preferably, the Poisson noise simulation instrument is affected by additional noise introduced by signal acquisition and sample preparation; the Gaussian noise simulation instrument is affected by noise introduced by sensors and electronic components.

[0039] On the other hand, a device for super-resolution reconstruction of mass spectrometry images of single cells includes:

[0040] The mass spectrometry and optical image acquisition module is used to acquire low-resolution mass spectrometry images of single cells. lr and single-cell optical images I op data;

[0041] The registration processing module is used to perform interpolation on low-resolution mass spectrometry images of single cells. lr Upsampling and single-cell optical image I op Downsampled to the same pixel resolution, the downsampled optical image is input into the image registration module for registration, resulting in the registered single-cell optical image I. reg ;

[0042] The guided filtering module is used to process the registered single-cell optical image I through the guided filtering module. reg Filtering is performed to obtain the registered and filtered reference image I. ref ;

[0043] The multimodal fusion processing module extracts low-resolution mass spectrometry images from single cells. lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr ;

[0044] The super-resolution reconstruction result acquisition module is used to obtain the mass spectrometry super-resolution reconstructed image I′. sr The final super-resolution reconstruction result I is obtained by simulating the imaging process with real instruments. sr .

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] (1) This invention uses single-cell optical images as reference images to extract rich structural information and combines them with the ion intensity information of the original mass spectrometry images to train a super-resolution reconstruction network specific to mass spectrometry images. Using only a single mass spectrometry image and a single reference image, a high-resolution reconstructed mass spectrometry image with rich chemical information and texture details is generated.

[0047] (2) This invention proposes a multimodal fusion module, which extracts shallow intuitive features and deep abstract features from mass spectrometry images and reference images through multimodal fusion modules at various levels (multi-level multi-module fusion modules are used, with the first few levels extracting shallow features and the later levels extracting deep features). Feature information at different depths is fused separately, thereby combining the structural information of single-cell optical images with the ion intensity information of the original mass spectrometry images.

[0048] (3) In this invention, two image blocks at the same position are selected from the mass spectrometry image and the single-cell optical image used as a reference for training the network each time. The computational load is small and the memory space required is small when using only small image blocks. It can also speed up the training speed and achieve faster imaging.

[0049] (4) The present invention reduces the conventional pixel-by-pixel scanning time by an order of magnitude, and high spatial resolution mass spectrometry imaging can be achieved using existing low-resolution mass spectrometry instruments. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic flowchart of a method and apparatus for super-resolution reconstruction of mass spectrometry images for single cells, according to an embodiment of this application.

[0052] Figure 2 This is a schematic diagram of the structure of a super-resolution reconstruction network according to an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the registration module in an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the conversion module in an embodiment of this application;

[0055] Figure 5This is a schematic diagram of the structure of the multimodal fusion module according to an embodiment of this application;

[0056] Figure 6 This is a simulation diagram of the actual instrument imaging effect in an embodiment of this application;

[0057] Figure 7 This is a super-resolution reconstruction result diagram of an embodiment of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0060] like Figure 1 As shown in the figure, this embodiment provides a method for super-resolution reconstruction of mass spectrometry images of single cells, which is applied in applications such as... Figure 2 The super-resolution reconstruction network shown (the network includes a registration module, a guided filtering module, and a multimodal fusion module) includes the following steps.

[0061] S1, Acquire low-resolution mass spectrometry image of a single cell. lr and single-cell optical images I op data.

[0062] Specifically, S1 is implemented as follows.

[0063] S11. The cell sample used was human cervical cancer HeLa cells (purchased from the Cell Bank of the Chinese Academy of Sciences). HeLa cells were grown in Duchenne Modified Eagle Medium (DMEM, HyClone) supplemented with 10% (v / v) fetal bovine serum, 1% penicillin, and 1% streptomycin. HeLa cells were transferred to 150 mm Falcon culture dishes, with fresh medium added every 2-3 days. Silicone wafers were prepared in advance, ultrasonically rinsed three times each with ultrapure water and ethanol, and sterilized before being used for cell culture. The wafers were placed in pre-prepared, high-temperature sterilized six-well plates, with cells spaced at 1×10⁻⁶ wells according to the required coverage. 3 -1×104 Cell / well density seeding of cells into wells and incubation at 37°C and 5% CO2 2 The cells were cultured for 24 hours under the specified conditions. Using dimethyl sulfoxide (DMSO) as the solvent, a certain mass of proflavin solid powder was weighed using an analytical balance to prepare a 50 mM proflavin stock solution, which was then stored at 4°C for later use. Once the cells had fully adhered and reached a satisfactory cell density, the culture medium was replaced with a solution containing 20 μM proflavin, and the cells were cultured for another 6 hours. After the drug culture was completed, the culture medium was removed from the wells, and the cells were washed three times with phosphate buffer solution. Then, the cells were fixed with 4% paraformaldehyde solution for 15 to 30 minutes. The fixative was then removed, and the cells were washed three times with 150 mM ammonium acetate solution (pH 7.4) to remove any inorganic salts introduced during the washing process.

[0064] S12, the silicon wafer with adhered cells was dried at room temperature, and then the cell morphology was characterized by a shape measurement laser microscopy system (VK-X250K, Keyence);

[0065] S13, mass spectrometry data were acquired using a commercially available TOF-SIMS6 mass spectrometer (INMTOF GmbH, Germany) in positive ion mode, with a mass range of 100–1,000 Da, a scan range of 75 μm × 75 μm, and a resolution of 64 pixels × 64 pixels. Prior to data acquisition, a 10 keV energy source was used. Cluster ion sources were used to analyze the cell surface. After the experiment, the ion source was used again... The cell surface was scanned using a cluster ion source. Then, the resolution was changed to 256 pixels × 256 pixels, and the experiment was repeated under the same conditions to obtain high-resolution and low-resolution single-cell low-resolution mass spectrometry images. The high-resolution single-cell low-resolution mass spectrometry image was used as a control.

[0066] S2, using interpolation methods, transforms the low-resolution mass spectrometry image of a single cell into... lr Upsampling and single-cell optical image I op Downsampled to the same pixel resolution, the downsampled optical image is input into the image registration module for registration, resulting in the registered single-cell optical image I. reg The structural diagram of the registration module is shown below. Figure 3 As shown.

[0067] Specifically, S2 is implemented as follows.

[0068] S21 utilizes a spatial transformation network to construct a registration module and a four-layer convolutional neural network to construct a conversion module. The spatial transformation network uses nonlinear interpolation to perform an affine transformation on the single-cell optical image and the mass spectrometry image to obtain a mapping relationship, achieving multimodal registration. The convolutional neural network module then eliminates the differences between modes. The spatial transformation network consists of two convolutional neural network modules, one flattening layer, and two fully connected layers. Each convolutional neural network module consists of a 3×3 convolutional layer, a 2×2 max-pooling layer, and a ReLU activation layer. The flattening layer converts the output data of the convolutional neural network module into a one-dimensional tensor input fully connected layer. The two fully connected layers have 32 and 6 nodes respectively, connected by a ReLU activation layer. The first three convolutional neural network modules of the four-layer module consist of a 3×3 convolutional layer and a ReLU activation layer. The last convolutional neural network module consists of a 3×3 convolutional layer to eliminate the differences between modes. A schematic diagram of the conversion module is shown below. Figure 4 As shown.

[0069] S22, using bicubic interpolation, the mass spectrometry image is upsampled and the single-cell optical image is downsampled to the same pixel resolution. The downsampled optical image is then input into the registration module, utilizing the registration loss L. reg After iteratively training the network a certain number of times, the network parameters and weights of the registration module are retained, and the registered single-cell optical image I is output. reg ;

[0070] The registration loss L reg The calculation formula is as follows:

[0071] L reg =||I reg↓s -I lr || F

[0072] Among them, ||·|| F Denotes the Frobenius norm, I lr This represents the original low-resolution mass spectrometry image of a single cell. 's' is a downsampling factor, here chosen as a factor of 4, used to downsample the registered image to the same pixel resolution as the low-resolution image. S3, to prevent interference from optical diffraction, uses a guided filtering module to filter the registered single-cell optical image I. reg Perform filtering to obtain the registered and filtered reference image I. ref .

[0073] Specifically, S3 is implemented as follows.

[0074] S31, Construct a guided filtering module. Guided filtering uses a guide image to guide the filtering of another input image. While removing noise, it can preserve edge and detail features. Its core idea is to establish a local linear relationship between the input image and the guide image, and to achieve the filtering processing of the input image by solving the coefficients of the linear model.

[0075] S32 places the filtering module after the registration module, using the bicubic interpolated low-resolution mass spectrometry image of a single cell as the guide image to filter the registered single-cell optical image to prevent interference from optical diffraction. The structural loss L... str Replacement registration loss L reg The network model is optimized, and the registered and filtered reference image I is output. ref .

[0076] Furthermore, the structural loss L str The calculation formula is:

[0077] L str =||I ref↓s -I lr || F

[0078] Among them, ||·|| F Denotes the Frobenius norm, I lr This represents the original low-resolution mass spectrometry image of a single cell. s is a downsampling factor, here chosen as a factor of 4, used to downsample the registered and filtered image to the same pixel resolution as the low-resolution image.

[0079] S4, from single-cell low-resolution mass spectrometry image I lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr .

[0080] Specifically, S4 is implemented as follows.

[0081] S41, from single-cell low-resolution mass spectrometry image I lr Randomly select a 32×32 image patch I′ lr Based on a super-resolution factor of 4x, from reference image I ref Select an image patch I′ of size 128×128 at the corresponding position. ref .

[0082] S42, Construct a multimodal fusion module. The first-level multimodal fusion module uses the registered and filtered reference image block I′. ref and single-cell low-resolution mass spectrometry image patch I′ lr As input, the single-cell low-resolution mass spectrometry image is upsampled using interpolation before being input into the convolutional neural network module. Features are extracted from the mass spectrometry image and the reference image separately by a convolutional neural network module, then the features are merged and passed through the same convolutional neural network module to obtain a new fused image. The first-level convolutional neural network module consists of a 3×3 convolutional layer and a ReLU activation layer. Each subsequent level of the multimodal fusion module takes the mass spectrometry image, the fused image, and the reference image from the previous level as input, and the rest is similar to the first level. Each convolutional neural network module consists of a 3×3 convolutional layer, a ReLU activation layer, and another 3×3 convolutional layer. After the last level of the multimodal fusion module, a four-layer convolutional neural network module, identical to step S2, is added; the first three convolutional neural network modules of the four-layer module consist of a 3×3 convolutional layer and a ReLU activation layer. The last convolutional neural network module consists of a 3×3 convolutional layer. A schematic diagram of the multimodal fusion module structure is shown below. Figure 5 As shown.

[0083] S43, utilizing global network loss L total The network model is optimized; simultaneously, the parameters of the registration module and the multimodal fusion network model are further adjusted through backpropagation, iterating a preset number of times to improve registration accuracy and super-resolution reconstruction effect, with image patches being randomly selected each time. After reaching the preset number of iterations, the super-resolution reconstructed image I′ is output. sr .

[0084] The global network loss L total The calculation formula is:

[0085] I total =α1L str +α2L ion

[0086] L ion =||I′ sr↓s -I lr || F

[0087] Among them, ||·|| F Denotes the Frobenius norm, I lr This represents the original low-resolution mass spectrometry image of a single cell. s is a downsampling factor, here chosen as 4x, and α1 and α2 are the weights of the loss function, respectively.

[0088] S5, In order to more closely approximate the image captured by a real mass spectrometer, the mass spectrometry super-resolution reconstructed image I′ is... sr The final super-resolution reconstruction result I is obtained by simulating the imaging process with real instruments. sr .

[0089] Specifically, the implementation of S5 is as follows: Figure 6 As shown, it includes the following steps.

[0090] S51, for the super-resolution reconstructed image I′ sr Threshold segmentation is performed to separate single cells from the background, resulting in a cell mask. Poisson noise is then added to the single cells to simulate additional noise introduced by the instrument due to signal acquisition, sample preparation, and other factors. The Poisson noise parameter λ is determined by the pixel value of the input image and is set to 255 times the pixel value.

[0091] In step S52, threshold segmentation is used again to separate areas with high and low ion intensity within a single cell. Pixel values ​​in high-ion-intensity regions are set to 1, while pixel values ​​in other regions are cleared to 0, resulting in an ion mask. In addition to the Poisson noise added in step S51, Gaussian noise is then added to the high-ion-intensity regions to simulate noise introduced by sensors, electronic components, etc., resulting in the final output image I. sr The Gaussian noise parameter has a mean μ of 0 and a variance σ. 2 Take 0.005.

[0092] To verify the effectiveness of the proposed method, HeLa cell mass spectrometry imaging experiments with pixel resolutions of 1 μm and 250 nm were performed using a commercially available TOF-SIMS6 mass spectrometer (INMTOF GmbH, Germany). Cell morphology was characterized using a shape measurement laser microscopy system (VK-X250K, Keyence, Japan). The proposed super-resolution reconstruction method for single-cell mass spectrometry images successfully reconstructed the original yellow mass spectrometry image of a 1 μm single cell.

[0093] The inter-cell resolution was improved by 4 times, and the resulting super-resolution reconstruction image was extremely close to the true 250nm high-resolution single-cell proto-yellow mass spectrometry image, such as... Figure 7 As shown.

[0094] This embodiment also discloses a device for super-resolution reconstruction of mass spectrometry images of single cells, characterized in that it includes:

[0095] Mass spectrometry and optical image acquisition module 801 is used to acquire low-resolution mass spectrometry images of single cells. lr and single-cell optical images I op data;

[0096] Registration processing module 802 is used to register low-resolution mass spectrometry images of single cells using interpolation methods. lr Upsampling and single-cell optical image I op Downsampled to the same pixel resolution, the downsampled optical image is input into the image registration module for registration, resulting in the registered single-cell optical image I. reg ;

[0097] Guided filtering processing module 803 is used to process the registered single-cell optical image I through the guided filtering module. reg Filtering is performed to obtain the registered and filtered reference image I. ref ;

[0098] Multimodal fusion processing module 804, from single-cell low-resolution mass spectrometry image I lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr ;

[0099] The super-resolution reconstruction result acquisition module 805 is used to obtain the mass spectrometry super-resolution reconstructed image I′. sr The final super-resolution reconstruction result I is obtained by simulating the imaging process with real instruments. sr .

[0100] The specific implementation of each module of a single-cell mass spectrometry image super-resolution reconstruction device is the same as that of a single-cell mass spectrometry image super-resolution reconstruction method, and will not be described again in this embodiment.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for super-resolution reconstruction of mass spectrometry images of single cells, characterized in that, include: S1, acquiring a low-resolution mass spectrometric image I of a single cell lr and a single-cell optical image I op data; S2, using an interpolation method, the single-cell low-resolution mass spectrum image I lr up-sampling and the single-cell optical image I op down-sampling to the same pixel resolution, inputting the down-sampled optical image into an image registration module for registration to obtain a registered single-cell optical image I reg ; S3, filtering the registered single-cell optical image I by a guided filtering module to obtain a filtered registered single-cell optical image I reg ref ;​ S4, from single-cell low-resolution mass spectrometry image I lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr ; S5, reconstructing the mass spectrum super-resolution image I' sr The final super-resolution reconstruction result I is output through the real instrument imaging simulation process sr .

2. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 1, characterized in that, In the S1, a low-resolution mass spectrum image I of a single cell is acquired lr and a single-cell optical image I op The process of the data is as follows: S11, all cell sample culture and related operations are completed in a biosafety cabinet; S12, the silicon wafer with adhered cells is air-dried at room temperature, and the cell morphology is characterized by a shape measuring laser microscopy system to obtain a single cell optical image I op ; S13, acquiring mass spectrum signals of each pixel point of the single cell in a point scanning mode by using the mass spectrum imager, and obtaining a low-resolution mass spectrum image I of the single cell based on the mass spectrum signals lr .

3. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 1, characterized in that, In S2, the down-sampled optical image is input into an image registration module for registration to obtain a registered single-cell optical image I reg , and specifically comprises: The downsampled optical image is input into the image registration module; the image registration module is a spatial transformation network, and a conversion module consisting of four convolutional neural networks is connected after the registration module. The spatial transformation network uses nonlinear difference to perform affine transformation on the single-cell optical image and the single-cell low-resolution mass spectrometry image to obtain a mapping relationship, thereby realizing multimodal registration; the conversion module is used to eliminate the differences between modes. using a registration loss L reg , the network parameter weights of the registration module are reserved after a certain number of iterations, and the registered single-cell optical image I reg is output The registration loss L reg The formula for calculating the registration loss L is as follows: L reg =||I reg↓s -I lr || F where || · || F denotes the Frobenius norm; I lr denotes a single-cell low-resolution mass-spectrometry image; s is a down-sampling factor used to down-sample the single-cell optical image to the same pixel resolution as the single-cell low-resolution mass-spectrometry image; I reg↓s denotes the image after down-sampling the registered single-cell optical image by the down-sampling factor s.

4. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 1, characterized in that, S3 specifically includes: With the single-cell low-resolution mass spectrum image after bicubic interpolation as a guide image, the registered single-cell optical image is filtered by a guide filtering module to prevent interference of optical diffraction; the guide filtering module uses a structure loss L str The network parameters of the guide filtering module are optimized, and a registered and filtered reference image I ref is output. The structural loss L str The calculation formula is: L str =||I ref↓s -I lr || F where || · || F denotes the Frobenius norm; I lr denotes a single-cell low-resolution mass-spectrometry image; s is a down-sampling factor used to down-sample the registered and filtered image to the same pixel resolution as the single-cell low-resolution mass-spectrometry image; I ref↓s denotes an image resulting from down-sampling the registered single-cell optical image by the down-sampling factor s.

5. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 1, characterized in that, S4 specifically includes: S41, randomly selecting a single-cell low-resolution mass spectrum image block I' of size n x n from the single-cell low-resolution mass spectrum image I lr S42, selecting a reference image block I' of size m x m from the reference image I lr corresponding to the position of the single-cell low-resolution mass spectrum image block I' according to the super-resolution multiple d ref S43, generating a high-resolution mass spectrum image block I''of size n x n x d from the reference image block I' ref ​ S42, the reference image block I' ref and single-cell low-resolution mass spectrum image block I' lr As the input of the multi-modal fusion module, the multi-modal fusion module includes several levels of multi-modal fusion modules connected in sequence, and each level of multi-modal fusion module has the same structure; wherein the first level of multi-modal fusion module takes the single-cell low-resolution mass spectrum image I ref and single-cell low-resolution mass spectrum image block I' lr As the input, the single-cell low-resolution mass spectrum image is up-sampled by using the interpolation method before being input into the convolutional neural network module; then the features are extracted from the single-cell low-resolution mass spectrum image block and the reference image block by a convolutional neural network module respectively, and then the features are combined, and then a new fusion image is obtained by a convolutional neural network module; each level of the multi-modal fusion module of the subsequent levels takes the fusion image of the previous level, the single-cell low-resolution mass spectrum image I lr and the reference image I ref As the input, the four-layer convolutional neural network module is used for processing after the last level of multi-modal fusion module. S43, utilize the global network loss L total Optimizing the network model; at the same time, further adjusting the network model parameters of the registration module and the multi-modal fusion module through back propagation, and iteratively improving the registration accuracy and super-resolution reconstruction effect for a preset number of times, randomly selecting an image block each time, and outputting a super-resolution reconstructed image I' after a preset number of times sr .

6. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 5, characterized in that, The global network loss L total The calculation formula is: L total = a1L str + a2L ion L ion =||I′ sr↓s -I lr || F where ||·||F F denotes the Frobenius norm; L str denotes the structure loss; I lr denotes the original single-cell low-resolution mass spectrum image, s is a down-sampling factor, and α1 and α2 are loss function weight values, respectively; I′ sr↓s denotes the image that will be output after the preset number of times of super-resolution reconstruction images are down-sampled by the down-sampling factor s.

7. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 1, characterized in that, S5 specifically includes: S51, on the super-resolution reconstructed image I' sr Threshold segmentation is performed to segment the single cells from the background to obtain a cell mask, and Poisson noise is added to the single cells. S52, again, by threshold segmentation, the single cell inside ion intensity high place and ion intensity low place are segmented, get ion mask, on the basis of adding poisson noise, high ion intensity area adds gaussian noise again, get final output image I sr ; wherein, high ion intensity area pixel value is 1, and the pixel value of the remaining area is 0.

8. The method for super-resolution reconstruction of mass spectrometry images for single cells according to claim 7, characterized in that, The Poisson noise simulator is affected by additional noise introduced by signal acquisition and sample preparation; the Gaussian noise simulator is affected by noise introduced by sensors and electronic components.

9. A device for super-resolution reconstruction of mass spectrometry images of single cells, characterized in that, include: a mass spectrometry image and an optical image acquisition module for acquiring a single cell low resolution mass spectrometry image I lr and a single cell optical image I op data; a registration processing module configured to register the single-cell low-resolution mass spectrometry image I lr up-sampling and inputting the up-sampled optical image into the image registration module for registration to obtain a registered single-cell optical image I op down-sampling to the same pixel resolution, inputting the down-sampled optical image into the image registration module for registration to obtain a registered single-cell optical image I reg ; The guided filtering module is used to process the registered single-cell optical image I through the guided filtering module. reg Filtering is performed to obtain the registered and filtered reference image I. ref ; The multimodal fusion processing module extracts low-resolution mass spectrometry images from single cells. lr Randomly select an n×n image patch I′ lr Based on the super-resolution factor d, from the reference image I ref Select an image patch I′ of size m×m at the corresponding position. ref Where m is of size n×d; the image block I′ lr and image block I′ ref Input the multimodal fusion module to generate a mass spectrometry super-resolution reconstructed image I′ sr ; The super-resolution reconstruction result acquisition module is used to obtain the mass spectrometry super-resolution reconstructed image I′. sr The final super-resolution reconstruction result I is obtained by simulating the imaging process with real instruments. sr .

Images