Three-dimensional mass spectrometry imaging method and device based on sparse sampling and deep generative model

Abstract

The application relates to the technical field of mass spectrum imaging, in particular to a three-dimensional mass spectrum imaging method and device based on sparse sampling and a deep generation model, wherein the method comprises the following steps: obtaining mass spectrum data by full sampling of part of tissue slices, obtaining simulated sparse sampling data by simulating a sparse sampling process on the mass spectrum data, inputting the mass spectrum data and the simulated sparse sampling data and training to obtain a deep generation model, then optimizing to obtain an optimal model structure and hyperparameter setting, generating the trained deep generation model, and reconstructing sparse sampling mass spectrum imaging to obtain complete three-dimensional mass spectrum imaging data. According to the embodiment of the application, three-dimensional mass spectrum imaging data can be obtained by combining the deep generation model and the sparse sampling method, high-resolution imaging is guaranteed, the required number of sampling pixels is reduced, the mass spectrum imaging process is accelerated, and high-resolution and high-efficiency three-dimensional mass spectrum imaging is realized, which is more accurate and reliable.

Description

Technical Field

[0001] This application relates to the field of mass spectrometry imaging technology, and in particular to a three-dimensional mass spectrometry imaging method and apparatus based on sparse sampling and depth generation models. Background Technology

[0002] Mass spectrometry imaging is an important technique that can characterize the spatial features of biomolecules and their biological activities in tissues and organs, contributing to the understanding of basic biology and diseases. Compared to two-dimensional imaging, three-dimensional imaging of tissues and organs allows for the mapping of the three-dimensional spatial distribution of biomolecules.

[0003] In related technologies, three-dimensional image reconstruction can be performed by collecting two-dimensional mass spectrometry imaging data of continuous tissue sections through methods such as desorption electrospray ionization three-dimensional mass spectrometry imaging and matrix-assisted laser desorption ionization three-dimensional mass spectrometry imaging.

[0004] However, in related technologies, two-dimensional mass spectrometry imaging analysis of tissue sections is time-consuming. As the imaging resolution increases, the imaging time increases dramatically. Furthermore, it is impossible to reduce the number of pixels to be sampled while avoiding damage to the imaging resolution, which reduces the efficiency of the mass spectrometry imaging process and the reliability of the imaging results. This issue urgently needs to be addressed. Summary of the Invention

[0005] This application provides a three-dimensional mass spectrometry imaging method and apparatus based on sparse sampling and a depth generation model to solve the problems in related technologies, such as the large time consumption of two-dimensional mass spectrometry imaging analysis of tissue slices, the sharp increase in imaging time when the imaging resolution is increased, and the inability to reduce the number of pixels to be sampled while avoiding damage to the imaging resolution, thus reducing the efficiency of the mass spectrometry imaging process and the reliability of the imaging results.

[0006] The first aspect of this application provides a three-dimensional mass spectrometry imaging method based on sparse sampling and a deep generation model, comprising the following steps: full sampling of a portion of tissue slices to acquire mass spectrometry data in full sampling mode, and preprocessing the mass spectrometry data to obtain processed mass spectrometry data; simulating a sparse sampling process using masks of different scales at a preset sampling ratio to obtain ion images acquired through simulated sparse sampling; converting the ion images acquired through simulated sparse sampling from single-channel images to three-channel images to obtain processed simulated sparse sampling data; using the mass spectrometry data as real samples and the processed simulated sparse sampling data as input to a deep generation model to train a deep generation model; further optimizing the reconstruction effect of the deep generation model according to different sparse sampling ratios and the network structure parameters of the model to obtain the optimal model structure and hyperparameter settings, and generating a trained deep generation model; and, according to actual needs, performing sparse sampling on subsequent continuous tissue slices at a preset ratio to acquire sparsely sampled mass spectrometry data, and reconstructing the sparsely sampled ion images using the trained deep generation model to obtain complete three-dimensional mass spectrometry imaging data.

[0007] Specifically, in one embodiment of this application, the conversion formula for the three-channel image is:

[0008]

[0009] in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

[0010] Optionally, in one embodiment of this application, the step of simulating a sparse sampling process by using masks of different scales with a preset sampling ratio to obtain an ion image obtained by simulated sparse sampling includes: during the sparse sampling process, the 0 and 1 positions of the mask are not uniformly distributed, and the positions of the sampled pixels are determined by the 0 and 1 positions in the mask.

[0011] Specifically, in one embodiment of this application, the loss function of the generator of the trained deep generative model is:

[0012]

[0013] Furthermore, the loss function of the discriminator is:

[0014]

[0015] Among them, L GLet G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate a three-channel ion image obtained through sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

[0016] A second aspect of this application provides a three-dimensional mass spectrometry imaging device based on sparse sampling and a depth generation model, comprising: a sampling module for performing full sampling on a portion of tissue slices to acquire mass spectrometry data in full sampling mode, and performing data preprocessing on the mass spectrometry data to obtain processed mass spectrometry data; a simulation module for simulating a sparse sampling process on the data using masks of different scales at a preset sampling ratio to obtain an ion image acquired by simulated sparse sampling; a conversion module for converting the ion image acquired by simulated sparse sampling from a single-channel image to a three-channel image to obtain processed simulated sparse sampling data; and a training module for processing the mass spectrometry data. Using real samples and the processed simulated sparse sampling data as input to the deep generation model, a deep generation model is trained. A generation module further optimizes the reconstruction effect of the deep generation model based on different sparse sampling ratios and the model's network structure parameters to obtain the optimal model structure and hyperparameter settings, generating the trained deep generation model. An imaging module performs sparse sampling on subsequent continuous tissue slices at a preset ratio according to actual needs, collects sparsely sampled mass spectrometry data, and reconstructs the sparsely sampled ion images using the trained deep generation model to obtain complete three-dimensional mass spectrometry imaging data.

[0017] Specifically, in one embodiment of this application, the conversion formula for the three-channel image is:

[0018]

[0019] in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

[0020] Optionally, in one embodiment of this application, the simulation module includes: during the implementation of sparse sampling, the 0 and 1 positions of the mask are not uniformly distributed, and the positions of the sampled pixels are determined by the 0 and 1 positions in the mask.

[0021] Specifically, in one embodiment of this application, the loss function of the generator of the trained deep generative model is:

[0022]

[0023] Furthermore, the loss function of the discriminator is:

[0024]

[0025] Among them, L G Let G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate a three-channel ion image obtained through sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

[0026] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional mass spectrometry imaging method based on sparse sampling and depth generation models as described in the above embodiments.

[0027] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model.

[0028] This application embodiment can acquire mass spectrometry data by fully sampling a portion of tissue slices in full sampling mode, and preprocess the mass spectrometry data to obtain processed mass spectrometry data; simulate a sparse sampling process using masks of different scales at a preset sampling ratio to obtain ion images acquired through simulated sparse sampling; convert the ion images acquired through simulated sparse sampling from single-channel images to three-channel images to obtain processed simulated sparse sampling data; use the mass spectrometry data as real samples and the processed simulated sparse sampling data as input to a deep generation model to train a deep generation model; according to different... The sparse sampling ratio and network structure parameters of the model are further optimized to improve the reconstruction effect of the deep generative model, resulting in the optimal model structure and hyperparameter settings, and generating a trained deep generative model. Based on actual needs, subsequent continuous tissue slices are sparsely sampled at a preset ratio to collect sparsely sampled mass spectrometry data. The trained deep generative model then reconstructs the ion images from the sparsely sampled data, obtaining complete three-dimensional mass spectrometry imaging data. This ensures high-resolution imaging while reducing the number of required sampling pixels, thereby accelerating the mass spectrometry imaging process and achieving high-resolution, high-efficiency, and more accurate and reliable three-dimensional mass spectrometry imaging. This solves the problems in related technologies, such as the high time consumption of two-dimensional mass spectrometry imaging analysis of tissue slices, the sharp increase in imaging time as the imaging resolution increases, and the inability to reduce the number of sampling pixels while avoiding damage to the imaging resolution, thus reducing the efficiency and reliability of the mass spectrometry imaging process.

[0029] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0030] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0031] Figure 1 This is a flowchart of a three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model, according to an embodiment of this application.

[0032] Figure 2 This is a schematic diagram of a programmable two-dimensional platform according to an embodiment of this application;

[0033] Figure 3 This is a schematic diagram illustrating the principle of a sparse sampling strategy according to an embodiment of this application;

[0034] Figure 4 This is a schematic diagram illustrating the reconstruction effect of an ion image based on a deep generative model and a sparse sampling strategy according to an embodiment of this application.

[0035] Figure 5 This is a schematic diagram of a fast three-dimensional mass spectrometry imaging process based on a depth generation model and sparse sampling strategy according to an embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the structure of a three-dimensional mass spectrometry imaging device based on a sparse sampling and depth generation model according to an embodiment of this application;

[0037] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0038] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0039] The following description, with reference to the accompanying drawings, describes a three-dimensional mass spectrometry imaging method and apparatus based on sparse sampling and a depth generation model, according to embodiments of this application. Addressing the issues raised in the background section regarding the time-consuming nature of two-dimensional mass spectrometry imaging analysis of tissue sections, the drastic increase in imaging time as the imaging resolution increases, and the inability to reduce the number of pixels requiring sampling while simultaneously compromising imaging resolution, thus reducing the efficiency and reliability of the mass spectrometry imaging process, this application provides a three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model. This method involves fully sampling a portion of the tissue sections to acquire mass spectrometry data in a full sampling mode, preprocessing the mass spectrometry data to obtain processed mass spectrometry data, simulating a sparse sampling process using masks of different scales at a preset sampling ratio to obtain an ion image acquired through simulated sparse sampling, and converting the simulated sparse sampling ion image from a single-channel image to a three-channel image to obtain processed... This method utilizes simulated sparse sampling data; mass spectrometry data is used as real samples, and the processed simulated sparse sampling data is used as input to a deep generative model to train the model. Based on different sparse sampling ratios and the model's network structure parameters, the reconstruction effect of the deep generative model is further optimized to obtain the optimal model structure and hyperparameter settings, generating the trained deep generative model. According to actual needs, subsequent continuous tissue slices are sparsely sampled at a preset ratio to collect sparsely sampled mass spectrometry data. The trained deep generative model is then used to reconstruct the ion images from the sparsely sampled data, obtaining complete three-dimensional mass spectrometry imaging data. This ensures high-resolution imaging while reducing the number of required sampling pixels, thereby accelerating the mass spectrometry imaging process and achieving high-resolution, high-efficiency, more accurate, and reliable three-dimensional mass spectrometry imaging. This solves the problems in related technologies, such as the high time consumption of two-dimensional mass spectrometry imaging analysis of tissue slices, the sharp increase in imaging time as the imaging resolution increases, and the inability to reduce the number of sampling pixels while avoiding damage to the imaging resolution, thus reducing the efficiency and reliability of the mass spectrometry imaging process.

[0040] Specifically, Figure 1 This is a flowchart illustrating a three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model, provided in an embodiment of this application.

[0041] like Figure 1 As shown, this three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model includes the following steps:

[0042] In step S101, a portion of the tissue slices are fully sampled to obtain mass spectrometry data in full sampling mode, and the mass spectrometry data is preprocessed to obtain processed mass spectrometry data.

[0043] It is understood that in the embodiments of this application, some tissue slices may be derived from frozen tissue serial slices, and the number of slices may be determined according to the slice size or imaging resolution. Data preprocessing of mass spectrometry data may involve obtaining ion image data corresponding to each mass-to-charge ratio and performing data preprocessing operations such as registration, background noise removal, TIC (Total Ion Count) normalization, and peak extraction.

[0044] In practice, frozen sections of biological tissues such as mouse brain, cerebellum, kidney, and liver can be obtained and numbered according to their spatial location. The first three sections from the consecutive tissue sections are then extracted in the order of their numbers. Mass spectrometry data is acquired in full sampling mode on a mass spectrometer in mzML format. The data is then converted using the mass spectrometry imaging data format and stored in Feather format on the computer to reduce memory usage and preserve data accuracy. The obtained data is then preprocessed, including background noise removal, registration, TIC normalization, peak extraction, and other data preprocessing operations.

[0045] In the registration process of three-dimensional mass spectrometry imaging, the tissue contour in the tissue slice can be fitted first, and the different slices can be matched according to the contour. The parameters required for the corresponding affine transformation can be calculated, and then the three-dimensional mass spectrometry imaging can be registered. Data preprocessing such as normalization and peak extraction can be implemented using Python according to conventional methods.

[0046] This application embodiment can perform full sampling on a portion of tissue slices to obtain mass spectrometry data in full sampling mode, and perform data preprocessing on the mass spectrometry data to obtain processed mass spectrometry data. By improving the mass spectrometry data processing process, the corresponding data accuracy and data quality are improved.

[0047] In step S102, the sparse sampling process is simulated by masks of different scales using a preset sampling ratio to obtain an ion image obtained by simulated sparse sampling.

[0048] It is understood that, in the embodiments of this application, the simulated sparse sampling process can be

[0049] I s =⊙I,

[0050] Among them, I s The data is obtained after sparse sampling, M is the mask, and I is the mass spectrometry data obtained after preprocessing and full sampling. The mask M consists of 0 and 1, where 0 represents no sampling and 1 represents sampling.

[0051] It should be noted that the preset sampling ratio is set by those skilled in the art based on the actual situation, and no specific limitation is made here.

[0052] For example, a programmable two-dimensional XY platform can be set up to automatically optimize the scanning path based on the mask M. The scanning mode is determined according to the distribution of sampling points, and sparse sampling is completed according to the optimized scanning mode.

[0053] like Figure 2 The diagram shown is a schematic representation of a programmable two-dimensional platform according to an embodiment of this application. The platform may include a microcontroller, a servo motor, a ball screw, and a computer program programmed into and capable of running on the microcontroller.

[0054] The microcontroller executes a scanning mode optimized based on the mask M. The programmable 2D moving platform includes servo motors and ball screws. There are two servo motors, serving as the power source for the X and Y axes respectively; the positioning accuracy of the servo motors is determined by the encoder resolution. There are also two ball screws, serving as the transmission devices for the X and Y axes respectively; the ball screws should be transmission type (T) with an accuracy class of 1.

[0055] The programmable two-dimensional moving platform can optimize the scanning mode according to the distribution of sampling points under the mask M. After optimization, the program is burned into the microcontroller. After the microcontroller with the program is burned into the servo motor establishes communication, the servo motor drives the ball screw, and the ball screw converts the rotation into high-precision linear motion of the X and Y axes, thereby realizing the high-precision two-dimensional movement of the platform.

[0056] The embodiments of this application can preset the sampling ratio and use masks of different scales to simulate the sparse sampling process of the data to obtain ion images obtained by simulating sparse sampling. By using the full-sample mass spectrometry imaging data obtained in the above steps, the required data information can be further provided for the implementation of the following steps.

[0057] Optionally, in one embodiment of this application, a sparse sampling process is simulated on the data using masks of different scales at a preset sampling ratio to obtain an ion image obtained by simulated sparse sampling, including: during the sparse sampling process, the 0 and 1 positions of the mask are not uniformly distributed, and the positions of the sampled pixels are determined by the 0 and 1 positions in the mask.

[0058] It is understood that in this embodiment, the mask determines the distribution of sampling points. The distribution of sampling points on a two-dimensional tissue slice uniformly conforms to a shape. The mask is randomly generated according to the shape setting and is not uniformly distributed. Under the action of the mask, the distribution shape of the sampling points includes, but is not limited to, square, linear, circular, etc., or a combination of several distributions. 0 in the mask can represent no sampling, and 1 can represent sampling. The positions of 0 and 1 in the mask determine the position of the sampling pixel.

[0059] like Figure 3The diagram shown is a schematic diagram of the principle of a sparse sampling strategy according to an embodiment of this application. The sparse sampling process can be simulated by a computer on the three-dimensional mass spectrometry imaging data collected by full sampling at a certain sampling ratio and according to the mask M of different scales.

[0060] In the implementation of sparse sampling in this application embodiment, the 0 and 1 positions of the mask are not uniformly distributed, and the position of the sampled pixel is determined by the 0 and 1 positions in the mask, thereby optimizing the specific implementation process of simulating sparse sampling and making the obtained results more comprehensive.

[0061] In step S103, the ion image obtained by simulated sparse sampling is converted from a single-channel image to a three-channel image to obtain processed simulated sparse sampling data.

[0062] It is understood that in this embodiment of the application, since the ion signal intensity value of the unsampled pixels in the simulated sparse sampling data is 0, while the ion signal intensity of some sampled pixels is weak and close to 0, the two cases are very close in value. Therefore, the ion image obtained by simulated sparse sampling can be converted from a single-channel image to a three-channel image for storage.

[0063] This application embodiment can convert the ion image obtained by simulated sparse sampling from a single-channel image to a three-channel image to obtain processed simulated sparse sampling data, so as to provide the required data for training the deep generative model.

[0064] Specifically, in one embodiment of this application, the conversion formula for the three-channel image is:

[0065]

[0066] in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

[0067] As can be seen from the above formula, ion images obtained by simulating sparse sampling can be... The image is converted from a single-channel image to a three-channel image. The data is stored to distinguish between the ion signals of unsampled pixels and the weaker ion signal intensity of some sampled pixels, thereby improving the accuracy of mass spectrometry data.

[0068] In step S104, mass spectrometry data is used as real samples, and processed simulated sparse sampling data is used as input to the deep generation model to train the deep generation model.

[0069] It is understood that the deep generative model in the embodiments of this application may consist of a generator and a discriminator, and has the ability to learn the distribution characteristics of data and infer unknown information based on known prior information.

[0070] The generator employs a Unet network structure, consisting of a contraction path and a dilation path. The contraction path, or encoding process, comprises four parts. Each part consists of two 3x3 convolutional operations, BatchNormalization, and ReLU activation. Between each part, a 2x2 max-pooling operation with a stride of 2 is performed for downsampling, doubling the number of feature channels after each downsampling. The dilation path, or decoding process, is symmetrical to the contraction path and also consists of four parts. Each part is preceded by an upsampling operation, and after each upsampling, the number of feature channels is halved. The contraction and dilation paths are connected by skip connections, reducing resolution loss and spatial location information loss during upsampling, resulting in better reconstruction performance.

[0071] The discriminator consists of five main parts: a convolution operation with a stride of 2, padding of 1, and a kernel size of 4×4; batch normalization and ReLU activation; followed by a convolution operation with a stride of 2, padding of 0, and a kernel size of 4×4, and a sigmoid activation operation.

[0072] In practice, the fully sampled data can be used as the real samples, and the processed simulated sparse sampled data can be used as the input to the deep generative model. The deep generative model will output the reconstruction result of the sparse sampled data, which is then fed into the discriminator network to calculate the loss function value. After multiple backpropagation updates to the model parameters, the model automatically learns how to reconstruct the mass spectrometry image obtained from sparse sampling. Specifically, small batch sizes and low learning rates are preferred for training the model, and during training, the weights of models with better reconstruction results are prioritized.

[0073] This application embodiment can use mass spectrometry data as real samples and processed simulated sparse sampling data as input to a deep generative model to train a deep generative model. Through model training, the distribution characteristics of the mass spectrometry imaging dataset are learned, further exploring the application capabilities of the deep generative model in three-dimensional mass spectrometry imaging, making the three-dimensional mass spectrometry imaging process more intelligent.

[0074] In step S105, the reconstruction effect of the deep generative model is further optimized according to different sparse sampling ratios and network structure parameters of the model to obtain the optimal model structure and hyperparameter settings, and generate the trained deep generative model.

[0075] It is understood that the network structure in the deep generative model in the embodiments of this application may include, but is not limited to, generative adversarial networks, variational autoencoders, U-net and its variant network models.

[0076] In actual implementation, the model reconstruction effect can be further optimized according to different sparse sampling ratios and network structure parameters of the model, so as to obtain the optimal model structure and hyperparameter settings. Priority is given to training the model by collecting simulated sparse sampling data with a low sampling ratio, such as 20% or 30%. Priority is given to network models consisting of generator networks and discriminator networks, and the generator network has an encoder-decoder structure as the deep production model network of this method.

[0077] The embodiments of this application can further optimize the reconstruction effect of the deep generative model according to different sparse sampling ratios and network structure parameters of the model, obtain the optimal model structure and hyperparameter settings, generate the trained deep generative model, and improve the imaging capability of the deep generative model for three-dimensional mass spectrometry by further optimizing the deep generative model obtained in the above steps, so as to make the reconstruction result of the obtained three-dimensional mass spectrometry more accurate.

[0078] Specifically, in one embodiment of this application, the loss function of the generator in the trained deep generative model is:

[0079]

[0080] Furthermore, the loss function of the discriminator is:

[0081]

[0082] Among them, L G Let G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate a three-channel ion image obtained through sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

[0083] It is understood that in the embodiments of this application, the number of images N in the function is related to the number of extracted mass-to-charge ratios. In the above formula, the model parameters are updated by performing operations on the loss function of the generator and the loss function of the discriminator of the deep generation model, thereby further realizing the automatic learning of the model for the reconstructed mass spectrum image obtained by sparse sampling.

[0084] In step S106, according to actual needs, sparse sampling is performed on subsequent continuous tissue slices at a preset ratio to collect sparsely sampled mass spectrometry data. The ion image of the sparsely sampled data is reconstructed by the trained depth generation model to obtain complete three-dimensional mass spectrometry imaging data.

[0085] It is understood that the three-dimensional mass spectrometry imaging in the embodiments of this application is applicable to mass spectrometry instruments of various principles and is not limited to the ionization method, ion fragmentation method, ion mass analysis method, etc. of the mass spectrometer.

[0086] It should be noted that the preset ratio is set by those skilled in the art based on the actual situation, and no specific limitation is made here.

[0087] In practice, a mask can be used to guide the mass spectrometer to perform sparse sampling on subsequent continuous tissue slices at a certain ratio, collect sparsely sampled mass spectrometry data, perform data preprocessing according to the above steps, and convert the single-channel mass spectrometry image into a three-channel mass spectrometry image. The processed sparsely sampled three-dimensional mass spectrometry image is then reconstructed using a trained deep generative model.

[0088] like Figure 4 The diagram illustrates the reconstruction effect of ion images based on a deep generative model and a sparse sampling strategy according to an embodiment of this application, including reconstruction results of ion images obtained from sparse sampling on a representative basis. The first column shows sparse sampling results for different layers of mouse brain tissue with the same mass-to-charge ratio of 837.087. The second column shows the true spatial distribution obtained from full sampling. The third column shows the ion spatial distribution map reconstructed by the model from the sparse sampling results in the first column. The fourth and fifth columns are magnified views of the same region from the second and third columns, respectively, showing the details of the reconstruction.

[0089] According to the actual needs, the embodiments of this application can perform sparse sampling on subsequent continuous tissue slices at a preset ratio, collect sparsely sampled mass spectrometry data, and reconstruct the ion image of the sparsely sampled data through a trained deep generation model to obtain complete three-dimensional mass spectrometry imaging data. This reduces the sampling time while ensuring resolution, and realizes high-resolution, high-efficiency three-dimensional mass spectrometry imaging, which is more accurate and reliable.

[0090] Specifically, such as Figure 5The diagram illustrates a rapid 3D mass spectrometry imaging process based on a depth generation model and a sparse sampling strategy, according to an embodiment of this application. It can be applied to mass spectrometers using ion sources that collect data in a point sampling mode for 3D mass spectrometry imaging. Specifically, full-sample mass spectrometry imaging is collected from the first three tissue slices. Sparse sampling is simulated on this dataset, and after preliminary pre-data processing, it serves as the training set for the model to train the depth generation model. Sparse sampling is then performed on the remaining tissue slices to obtain mass spectrometry images, which are then processed by reverse sampling and input into the model. The depth generation model performs data reconstruction to obtain the reconstruction result, thus completing the 3D mass spectrometry imaging.

[0091] The three-dimensional mass spectrometry imaging method based on sparse sampling and a deep generation model proposed in this application involves: acquiring mass spectrometry data by fully sampling a portion of tissue slices in a full sampling mode; preprocessing the mass spectrometry data to obtain processed mass spectrometry data; simulating a sparse sampling process using masks of different scales at a preset sampling ratio to obtain ion images acquired through simulated sparse sampling; converting the ion images acquired through simulated sparse sampling from single-channel images to three-channel images to obtain processed simulated sparse sampling data; and using the mass spectrometry data as real samples and the processed simulated sparse sampling data as input to a deep generation model for training. A deep generative model is obtained. Based on different sparse sampling ratios and the model's network structure parameters, the reconstruction effect of the deep generative model is further optimized to obtain the optimal model structure and hyperparameter settings, generating a trained deep generative model. According to actual needs, subsequent continuous tissue slices are sparsely sampled at a preset ratio to collect sparsely sampled mass spectrometry data. The trained deep generative model is then used to reconstruct the ion images from the sparsely sampled data, obtaining complete three-dimensional mass spectrometry imaging data. This ensures high-resolution imaging while reducing the number of required sampling pixels, thereby accelerating the mass spectrometry imaging process and achieving high-resolution, high-efficiency, and more accurate and reliable three-dimensional mass spectrometry imaging. This solves the problems in related technologies, such as the high time consumption of two-dimensional mass spectrometry imaging analysis of tissue slices, the sharp increase in imaging time as the imaging resolution increases, and the inability to reduce the number of sampling pixels while avoiding damage to the imaging resolution, thus reducing the efficiency and reliability of the mass spectrometry imaging process.

[0092] Next, referring to the accompanying drawings, a three-dimensional mass spectrometry imaging device based on a sparse sampling and depth generation model is described according to an embodiment of this application.

[0093] Figure 6 This is a block diagram of a three-dimensional mass spectrometry imaging device based on a sparse sampling and depth generation model according to an embodiment of this application.

[0094] like Figure 6As shown, the three-dimensional mass spectrometry imaging device 10 based on sparse sampling and depth generation model includes: a sampling module 100, a simulation module 200, a conversion module 300, a training module 400, a generation module 500, and an imaging module 600.

[0095] The sampling module 100 is used to perform full sampling on a portion of tissue slices, acquire mass spectrometry data in full sampling mode, and perform data preprocessing on the mass spectrometry data to obtain processed mass spectrometry data.

[0096] The simulation module 200 is used to simulate the sparse sampling process of data with masks of different scales at a preset sampling ratio to obtain the ion image obtained by simulated sparse sampling.

[0097] The conversion module 300 is used to convert the ion image obtained by simulated sparse sampling from a single-channel image to a three-channel image, so as to obtain the processed simulated sparse sampling data.

[0098] The training module 400 is used to train a deep generative model by taking mass spectrometry data as real samples and processed simulated sparse sampling data as input.

[0099] The generation module 500 is used to further optimize the reconstruction effect of the deep generative model based on different sparse sampling ratios and network structure parameters of the model, so as to obtain the optimal model structure and hyperparameter settings and generate the trained deep generative model.

[0100] The imaging module 600 is used to perform sparse sampling on subsequent continuous tissue slices at a preset ratio according to actual needs, collect sparsely sampled mass spectrometry data, and reconstruct the ion image of the sparsely sampled data through a trained depth generation model to obtain complete three-dimensional mass spectrometry imaging data.

[0101] Specifically, in one embodiment of this application, the conversion formula for the three-channel image is:

[0102]

[0103] in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

[0104] Optionally, in one embodiment of this application, the simulation module 200 includes: during the implementation of sparse sampling, the 0 and 1 positions of the mask are not uniformly distributed, and the positions of the sampled pixels are determined by the 0 and 1 positions in the mask.

[0105] Specifically, in one embodiment of this application, the loss function of the generator in the trained deep generative model is:

[0106]

[0107] Furthermore, the loss function of the discriminator is:

[0108]

[0109] Among them, L G Let G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate a three-channel ion image obtained through sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

[0110] It should be noted that the foregoing explanation of the three-dimensional mass spectrometry imaging method based on sparse sampling and depth generation model also applies to the three-dimensional mass spectrometry imaging device based on sparse sampling and depth generation model in this embodiment, and will not be repeated here.

[0111] The three-dimensional mass spectrometry imaging device based on sparse sampling and a depth generation model proposed in this application can acquire mass spectrometry data in full sampling mode by performing full sampling on a portion of tissue slices, and preprocess the mass spectrometry data to obtain processed mass spectrometry data; simulate the sparse sampling process of the data with masks of different scales at a preset sampling ratio to obtain ion images acquired by simulated sparse sampling; convert the ion images acquired by simulated sparse sampling from single-channel images to three-channel images to obtain processed simulated sparse sampling data; use the mass spectrometry data as real samples and the processed simulated sparse sampling data as input to a depth generation model for training. A deep generative model is obtained. Based on different sparse sampling ratios and the model's network structure parameters, the reconstruction effect of the deep generative model is further optimized to obtain the optimal model structure and hyperparameter settings, generating a trained deep generative model. According to actual needs, subsequent continuous tissue slices are sparsely sampled at a preset ratio to collect sparsely sampled mass spectrometry data. The trained deep generative model is then used to reconstruct the ion images from the sparsely sampled data, obtaining complete three-dimensional mass spectrometry imaging data. This ensures high-resolution imaging while reducing the number of required sampling pixels, thereby accelerating the mass spectrometry imaging process and achieving high-resolution, high-efficiency, and more accurate and reliable three-dimensional mass spectrometry imaging. This solves the problems in related technologies, such as the high time consumption of two-dimensional mass spectrometry imaging analysis of tissue slices, the sharp increase in imaging time as the imaging resolution increases, and the inability to reduce the number of sampling pixels while avoiding damage to the imaging resolution, thus reducing the efficiency and reliability of the mass spectrometry imaging process.

[0112] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0113] The memory 701, the processor 702, and the computer program stored on the memory 701 and capable of running on the processor 702.

[0114] When the processor 702 executes the program, it implements the three-dimensional mass spectrometry imaging method based on sparse sampling and depth generation model provided in the above embodiments.

[0115] Furthermore, electronic devices also include:

[0116] Communication interface 703 is used for communication between memory 701 and processor 702.

[0117] The memory 701 is used to store computer programs that can run on the processor 702.

[0118] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0119] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0120] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0121] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0122] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model.

[0123] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0124] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0125] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0126] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0127] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0128] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0130] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A three-dimensional mass spectrometry imaging method based on sparse sampling and a depth generation model, characterized in that, Includes the following steps: Full sampling was performed on a portion of tissue sections to acquire mass spectrometry data in full sampling mode, and the mass spectrometry data was preprocessed to obtain processed mass spectrometry data. Using a preset sampling ratio, the data is simulated by masks of different scales to obtain ion images obtained by simulated sparse sampling. The ion image obtained by the simulated sparse sampling is converted from a single-channel image to a three-channel image to obtain processed simulated sparse sampling data; The mass spectrometry data is used as the real sample, and the processed simulated sparse sampling data is used as the input to the deep generative model to train the deep generative model. Based on different sparse sampling ratios and network structure parameters of the model, the reconstruction effect of the deep generative model is further optimized to obtain the optimal model structure and hyperparameter settings, and to generate the trained deep generative model. as well as Based on actual needs, sparse sampling is performed on subsequent continuous tissue slices at a preset ratio to collect sparsely sampled mass spectrometry data. The trained depth generation model is then used to reconstruct the sparsely sampled ion images to obtain complete three-dimensional mass spectrometry imaging data.

2. The method according to claim 1, characterized in that, The conversion formula for the three-channel image is: in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

3. The method according to claim 1, characterized in that, The process of simulating sparse sampling of data using masks of different scales at a preset sampling ratio to obtain an ion image acquired through simulated sparse sampling includes: During the sparse sampling process, the 0 and 1 positions of the mask are not uniformly distributed, and the position of the sampled pixel is determined by the 0 and 1 positions in the mask.

4. The method according to claim 1, characterized in that, The loss function of the generator G in the trained deep generative model is: Furthermore, the loss function of the discriminator D is: Among them, L G Let G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate a three-channel ion image obtained through sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

5. A three-dimensional mass spectrometry imaging device based on sparse sampling and depth generation models, characterized in that, include: The sampling module is used to perform full sampling on a portion of tissue slices to acquire mass spectrometry data in full sampling mode, and to perform data preprocessing on the mass spectrometry data to obtain processed mass spectrometry data. The simulation module is used to simulate the sparse sampling process of data with masks of different scales at a preset sampling ratio, so as to obtain the ion image obtained by simulated sparse sampling. The conversion module is used to convert the ion image obtained by the simulated sparse sampling from a single-channel image to a three-channel image, so as to obtain the processed simulated sparse sampling data. The training module is used to train the deep generative model by taking the mass spectrometry data as real samples and the processed simulated sparse sampling data as input. The generation module is used to further optimize the reconstruction effect of the deep generative model based on different sparse sampling ratios and network structure parameters of the model, so as to obtain the optimal model structure and hyperparameter settings and generate the trained deep generative model. as well as The imaging module is used to perform sparse sampling on subsequent continuous tissue slices at a preset ratio according to actual needs, collect sparsely sampled mass spectrometry data, and reconstruct the ion image of the sparsely sampled data through the trained depth generation model to obtain complete three-dimensional mass spectrometry imaging data.

6. The apparatus according to claim 5, characterized in that, The conversion formula for the three-channel image is: in, To simulate the three-channel ion image obtained by sparse sampling, To simulate a single-channel ion image obtained from sparse sampling, i represents the corresponding mass-to-charge ratio, and M is the mask.

7. The apparatus according to claim 5, characterized in that, The simulation module includes: During the sparse sampling process, the 0 and 1 positions of the mask are not uniformly distributed, and the position of the sampled pixel is determined by the 0 and 1 positions in the mask.

8. The apparatus according to claim 5, characterized in that, The loss function of the generator in the trained deep generative model is: Furthermore, the loss function of the discriminator is: Among them, L G Let G be the loss function of the generator, N be the number of images during training, λ be the hyperparameters of the deep generative model, n be the mass-to-charge ratio, and G and D be the generator and discriminator, respectively. To simulate the three-channel ion image obtained by sparse sampling, I n For the three-channel ion image obtained from full sampling, L D Let be the loss function of the discriminator D.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the three-dimensional mass spectrometry imaging method based on sparse sampling and depth generation model as described in any one of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the three-dimensional mass spectrometry imaging method based on sparse sampling and depth generation model as described in any one of claims 1-4.

Images