A self-supervised representation learning method for heterogeneous mass spectrometry data

Abstract

The self-supervised representation learning method for heterogeneous mass spectrometry data disclosed by the application comprises the following steps: acquiring a mass spectrometry imaging data set, collecting meta information and encoding the meta information into a conditional embedding vector, performing peak detection and preprocessing on each pixel to obtain a peak pair list; a base model is constructed, which is used to map the peak embedding layer to obtain the peak embedding vector, the Perceiver encoder compresses the peak sequence of any length into a pixel latent representation through the latent token and cross attention, and the conditional embedding vector is injected; after superimposing the spatial position coding, the spatial context is learned through the spatial Transformer; the PerceiverIO decoder outputs a feature tensor; and the joint training is performed by using the masked pixel reconstruction, the masked peak reconstruction and the cross-sample contrast learning. The application can learn a general chemical-spatial representation from heterogeneous data without manual annotation, significantly reduces the dependence of downstream tasks on labeled samples, and supports applications such as cross-instrument biomarker verification.

Description

Technical Field

[0001] This invention belongs to the field of mass spectrometry imaging data analysis, specifically relating to a self-supervised representation learning method for heterogeneous mass spectrometry data. Background Technology

[0002] Mass spectrometry (MSI) technology can acquire spatial distribution information of hundreds to tens of thousands of metabolites, lipids, proteins, or elements in situ on tissue sections, achieving panoramic visualization of histochemical components without relying on exogenous markers. It is an important tool for disease pathology research, drug distribution analysis, and clinical tissue typing. Depending on the ionization method and applicable molecule types, current mainstream mass spectrometry technologies encompass various modes, including matrix-assisted laser desorption / ionization mass spectrometry (MALDI-MSI), desorption electrospray ionization mass spectrometry (DESI-MSI), secondary ion mass spectrometry (SIMS), and real-time direct analysis mass spectrometry (REIMS). Furthermore, different brands and models of the same type of instrument exhibit significant differences in parameter configurations, resulting in a dual heterogeneous data ecosystem of mode heterogeneity and instrument heterogeneity. However, intelligent analysis of MSI data faces four core technical bottlenecks, with the model generalization problem caused by instrument heterogeneity being the primary bottleneck: First, there is the model generalization problem caused by instrument heterogeneity. Data generated by mass spectrometry imaging instruments of different brands and models vary significantly in the number of channels (ranging from hundreds to tens of thousands), mass-to-charge ratio (m / z) coverage, and spatial resolution. Furthermore, the detection accuracy and noise levels of different instruments also differ under the same mode. Most existing deep learning methods assume that the input data has a fixed number of channels and a fixed m / z axis, treating data from different instruments as different data formats and modeling them separately. This means that a model trained for one instrument cannot be directly used for data from other instruments. Each time a new instrument is introduced, data needs to be reacquired, the network structure redesigned, and the model retrained, resulting in a large amount of repetitive work. This severely restricts the practicality and scalability of mass spectrometry imaging AI analysis methods in multi-instrument, multi-center scenarios, which is the most pressing pain point in the current field of MSI intelligent analysis.

[0003] Second, there is the problem of effective encoding of high-dimensional sparse data. The original intensity vector of a pixel in mass spectrometry imaging is a high-dimensional sparse vector. The effective signal peaks carrying reliable chemical information (channels with a signal-to-noise ratio exceeding the noise floor) typically account for only 5% to 15% of all channels, while the intensity values ​​of the remaining channels are close to or equal to the noise level. This problem is more prominent in cross-instrument scenarios: the noise distribution and effective peak ratios vary significantly between different instruments. Traditional fixed-bin dimensionality reduction methods not only disrupt the physical continuity of the m / z axes, but also the optimal binning parameters differ between different instruments, further exacerbating the encoding difficulties caused by instrument heterogeneity. In addition, for high-channel-count scenarios such as proteomics mass spectrometry imaging (the number of channels can reach over 50,000), the computational complexity of the self-attention mechanism is too high, the memory usage exceeds the current hardware limitations, directly processing the original vector is computationally infeasible, and a large number of noisy channels will interfere with the model's learning of real chemical signals.

[0004] Third, there is the issue of utilizing m / z physicochemical information. The m / z values ​​of each channel in mass spectrometry imaging data are not arbitrary labels, but rather continuous values ​​carrying clear physicochemical meanings: similar m / z values ​​often correspond to molecules with related chemical structures, mass differences correspond to a functional group, and mass differences correspond to isotope peaks. These physical laws constitute important prior knowledge for mass spectrometry data interpretation. Existing methods based on image convolution or general Transformers treat each spectral channel as an equivalent feature dimension, completely ignoring the chemical prior information contained in the m / z values. This prevents the model from utilizing physicochemical constraints such as molecular mass conservation, isotope distribution, and adduct ions to improve the chemical rationality and interpretability of feature learning. Especially when transferring data across instruments, this deficiency leads to a significant decrease in the universality of feature representations.

[0005] Fourth, there is a contradiction between the scarcity of labeled data and the diversity of downstream tasks. Professional labeling of mass spectrometry imaging data requires experts with a mass spectrometry background. This process is highly dependent on domain knowledge and is time-consuming and labor-intensive, severely limiting the scale of high-quality labeled data. Meanwhile, downstream analysis tasks for mass spectrometry imaging data are highly diverse, encompassing various task types such as cross-instrument biomarker validation, metabolic pathway enrichment analysis, and unsupervised microenvironment typing. All these tasks require models to have cross-instrument generalization capabilities. Existing supervised learning methods collect labeled data independently for each downstream task and train the model from scratch. This not only fails to reuse existing data accumulation and model knowledge across tasks and instruments but also faces the dilemma of insufficient labeled data for each new task and instrument, creating a vicious cycle of "data scarcity—task-specific—repetitive construction." How to fully utilize the large amount of unlabeled heterogeneous data accumulated by mass spectrometry imaging laboratories worldwide, and learn a universal mass spectrometry imaging feature representation in one go through self-supervised pre-training, enabling all downstream tasks to efficiently transfer across instruments with a small amount of labeled data or even zero labeling, is a key problem that urgently needs to be solved in the field of intelligent mass spectrometry imaging analysis.

[0006] To address the first and second issues, an input representation method completely decoupled from the number of instrument channels and instrument model is designed, along with a customized encoding architecture capable of handling input sequences of arbitrary length and outputting fixed-dimensional representations, thus overcoming the technical barriers to cross-instrument data encoding. For the third issue, a physicochemical prior of the mass-to-charge ratio is explicitly introduced during the encoding process, and a positional encoding scheme adapted to variable-length peak sequences is designed, enabling the model to perceive the correlation between mass differences and chemical structures between peaks, improving the rationality of cross-instrument feature representations. For the fourth issue, a basic model pre-training paradigm for cross-instrument heterogeneous data is constructed, learning task-independent and instrument-independent general feature representations from large-scale heterogeneous data without any manual annotation, thereby reducing the dependence of all downstream tasks on labeled data and achieving cross-instrument and cross-task knowledge reuse.

[0007] In existing research, the Perceiver architecture (Perceiver: General Perception with IterativeAttention[J], Computer Vision and Pattern Recognition (2021) introduced a fixed number of learnable latent processing units (tokens) as query endpoints, using a cross-attention mechanism to query input sequences of arbitrary length, thus achieving fixed-length compression encoding of variable-length inputs and providing an architectural foundation for processing mass spectrometry imaging data with variable channel numbers. Masked autoencoders (MAEs) have demonstrated powerful self-supervised representation learning capabilities in fields such as natural images and point clouds. However, directly applying the above architecture to the pre-training of general basic models for heterogeneous mass spectrometry imaging data faces the following inherent limitations, which cannot be solved by existing multimodal fusion schemes: First, the existing Perceiver architecture does not design a dedicated position encoding scheme for m / z physicochemical priors, and cannot perceive the chemical identity and physical correlation of each element in the peak sequence, making it difficult to adapt to the differences in mass-to-charge ratio coverage across instrument scenarios; Second, the mask reconstruction target of the existing MAE framework is a dense vector with a fixed format as the prediction object, which is not suitable for mass spectrometry imaging data with variable-length peak sequences as the basic representation unit, and cannot solve the encoding adaptation problem of cross-instrument data; Third, existing self-supervised pre-training methods have not yet established a unified encoding paradigm for heterogeneous data in the field of mass spectrometry imaging, and most of them focus on data processing of a single mode or a single instrument, and cannot achieve cross-instrument feature transfer.

[0008] In view of this, this invention proposes a self-supervised representation learning method for heterogeneous mass spectrometry data. It is specifically designed to address the instrument heterogeneity, high-dimensional sparsity, and physicochemical characteristics of mass spectrometry imaging data, in order to solve the technical problems of existing methods that cannot generalize across instruments, have low encoding efficiency, and insufficient utilization of chemical priors. This method enables unified encoding and universal representation learning for mass spectrometry imaging data of any instrument and any number of channels, providing technical support for intelligent MSI analysis in multi-instrument and multi-center scenarios. Summary of the Invention

[0009] Addressing the shortcomings of existing technologies in the field of intelligent analysis of heterogeneous mass spectrometry imaging data—namely, the inability to reuse models across instruments due to instrument heterogeneity, the lack of effective encoding methods for high-dimensional sparse data, the underutilization of mass-to-charge ratio physicochemical information, and the contradiction between scarce labeled data and the diversity of downstream tasks—this invention aims to provide a self-supervised representation learning method for heterogeneous mass spectrometry data, specifically achieving the following technical objectives: Firstly, a peak sequence-based instrument-independent input representation method is provided. By converting the original mass spectrometry imaging intensity vector of any instrument and any number of channels into a variable-length peak pair list, the dependence on a fixed number of channels is eliminated. This enables a unified input format for different detection objects such as metabolomics, lipidomics, and proteomics, as well as different instrument types such as MALDI, DESI, and SIMS. This overcomes the fundamental defect of existing methods that cannot be reused across instruments due to the fixed number of channels. Secondly, a variable-length sequence unified encoding method based on Perceiver cross-attention is provided. By using a fixed number of learnable latent words to query peak sequences of arbitrary length through a cross-attention mechanism, the variable peak sequences are encoded into latent representations with fixed dimensions. While reducing the computational complexity from 100% to 100%, it ensures that mass spectrometry imaging data with different channel numbers obtain comparable representation quality in the same feature space. Third, a peak embedding method with mass-to-charge ratio perception is provided. By applying Fourier position encoding to the mass-to-charge ratio value of each peak in the peak sequence, the physicochemical meaning of the mass-to-charge ratio value is explicitly encoded into a continuously differentiable high-dimensional vector, enabling the model to perceive the correlation between the mass difference and chemical structure between peaks. This overcomes the shortcomings of existing methods that treat each channel as an equivalent feature dimension and ignore the physical prior of mass-to-charge ratio. Fourth, a self-supervised pre-training method for basic models for mass spectrometry imaging data is provided. Through three complementary self-supervised pre-training tasks: masked pixel reconstruction, masked peak reconstruction, and cross-sample contrastive learning, a general chemical-spatial pixel-level feature representation that is independent of instrument type and downstream tasks can be learned from large-scale heterogeneous mass spectrometry imaging data without any manual annotation. This enables all downstream analysis tasks to be efficiently transferred by connecting a lightweight task header with a small amount of labeled data or zero annotation. Fifth, a method for dynamically specifying the output dimension during inference is provided. Through the PerceiverIO decoding architecture, the number of output query vectors is dynamically adjusted according to the needs of downstream tasks during the inference stage. The dimension of the output feature tensor can be flexibly controlled without retraining the model, so as to adapt to the diverse downstream tasks with different requirements for feature dimensions.

[0010] To achieve the above-mentioned objectives, this invention provides a self-supervised representation learning method for heterogeneous mass spectrometry data, comprising the following steps: Step 1: Obtain at least one mass spectrometry imaging dataset, collect the metadata in the mass spectrometry imaging dataset and encode it into a conditional embedding vector, and perform peak detection and preprocessing on the original intensity vector of each pixel to obtain a variable-length peak pair list for each pixel. Step 2: Construct a base model to obtain peak embedding vectors by mapping a variable-length list of peak pairs as input through a peak embedding layer. All peak embedding vectors of the same pixel form a peak lexical matrix. Using the latent lexical matrix as the query and the peak lexical matrix as the key, the pixel latent representation is obtained by encoding through a Perceiver encoder, and the conditional embedding vector is injected into the Perceiver encoder. After superimposing spatial position encoding on the pixel latent representation of each pixel, it is input into the spatial Transformer. The spatial context between pixels is learned through self-attention, and the spatially enhanced pixel representation is output. The spatially enhanced pixels are decoded by the PerceiverIO decoder to obtain the feature vector of each pixel. The feature vectors of all pixels constitute a feature tensor. Step 3: Jointly train the pedestal model using the mask pixel reconstruction task, mask peak reconstruction task, and cross-sample contrast learning task, update the parameters of the pedestal model, and obtain the trained pedestal model. Step 4: For the newly acquired mass spectrometry imaging data, after processing in Step 1, the corresponding variable-length peak pair list and conditional embedding vector are obtained. The trained base model parameters are loaded, and forward inference is performed to obtain the feature tensor for downstream analysis tasks.

[0011] Preferably, the metadata in step 1 includes at least the instrument type, ion mode, mass-to-charge ratio detection range, and spatial resolution.

[0012] Preferably, the encoding in step 1 into a conditional embedding vector includes: Categorical variables are encoded using one-hot encoding followed by linear projection; Continuous numerical values ​​are transformed logarithmically and then linearly projected. Natural language text is encoded into text embeddings using a pre-trained language model; The conditional embedding vector is obtained by concatenating the embedding vectors of all fields and then performing linear projection and layer normalization. Missing fields are replaced with all-zero vectors plus learnable missing labels.

[0013] Preferably, the peak detection and preprocessing in step 1 include baseline correction, signal-to-noise ratio filtering, normalization, peak list construction, and quality filtering in sequence.

[0014] Optionally, the baseline correction method includes one or more of the following: rolling minimum filtering method, local weighted regression smoothing, and convex bottom method; Signal-to-noise ratio (SNR) filtering includes: after baseline correction, calculating the SNR for each pixel at each channel position, and retaining only the channel positions where the SNR exceeds a preset threshold as valid peaks, wherein the SNR threshold is set to 3; Normalization methods include one or more of the following: total ion current normalization, median normalization, and reference peak normalization; Peak list construction includes: extracting the mass-to-charge ratio and normalized intensity values ​​of all valid peaks for each pixel, and constructing a peak list.

[0015] The rules for quality filtration include: Number of valid peaks in mass spectrometry imaging dataset The proportion of pixels accounting for no less than 60% of all pixels; The average number of valid peaks for all valid pixels in the mass spectrometry imaging dataset is not less than 20. Within the instrument's nominal mass-to-charge ratio range, at least 30% of the mass-to-charge ratio intervals contain valid signals; It is not allowed for all required fields to be missing.

[0016] Preferably, in step 2, the peak embedding layer applies Fourier position coding to the mass-to-charge ratio of each peak, and after applying a logarithmic transformation to the normalized intensity value, it obtains the intensity coding through linear projection: the Fourier position coding and intensity coding are concatenated, and then the peak embedding vector is output through linear projection and layer normalization. The temperature parameters of the Fourier position encoding are matched with the mass-to-charge ratio range to ensure that the Fourier position encoding has sufficient frequency resolution within the target range.

[0017] Preferably, the Perceiver encoder in step 2 consists of multiple layers of alternately stacked cross-attention sublayers and self-attention sublayers. After each layer of cross-attention calculation, self-attention calculation and feedforward network processing are performed. In the layer normalization of each sublayer, the conditional embedding vector is injected using a conditional layer normalization mechanism to obtain the latent representation of the pixel.

[0018] Preferably, in step 2, the batch processing method for the variable-length peak word sequence by the Perceiver encoder during training is as follows: The peak word sequences of all pixels in the same batch are concatenated along the sequence dimension into a single long sequence, and the cumulative sequence length vector is recorded to identify the boundaries of each original sequence in the concatenated sequence; during attention calculation, it is strictly restricted to be performed within the boundaries of each sequence, and attention interaction across samples is prohibited.

[0019] Preferably, the spatial location encoding in step 2 is a two-dimensional sine and cosine location encoding. After the output of the Perceiver encoder and before the input of the spatial Transformer, the spatial location encoding is superimposed on the pixel latent representation of each pixel to achieve hierarchical decoupling of chemical information and spatial information. The spatial location encoding is obtained by concatenating the row coordinates and column coordinates of the pixel after mapping them with sine and cosine functions of different frequencies.

[0020] More preferably, the processing method does not require zero padding for sequences of different lengths.

[0021] Preferably, in step 2, the spatial Transformer employs a spatial block training strategy: During the training phase, a spatial block containing multiple pixels is randomly selected from each mass spectrometry imaging slice as the basic unit for a single training iteration. The potential representations of all pixels within the spatial block are arranged in spatial order to form a sequence input space Transformer. During the inference phase, a sliding window approach is used to process the pixels of the entire slice block by block, with a step size equal to or half the window size. The features output by each window are then concatenated to form the feature tensor of the complete slice.

[0022] Preferably, the output dimension of the PerceiverIO decoder is fixed to a preset value during the training phase; During the inference phase, the user dynamically specifies the output dimension, constructs the output query vector corresponding to the number of output dimensions, interpolates and initializes the output query vector from the training phase, and then obtains the feature vector through cross-attention decoding. The feature vectors of all pixels constitute the feature tensor, eliminating the need to retrain the base model.

[0023] Preferably, the mask pixel reconstruction task in step 3 includes: Randomly mask a portion of pixels in the spatial block. Do not input the peak list of the masked pixels into the Perceiver encoder. Reconstruct the normalized intensity value of the masked pixels using the encoding features of the visible pixels through the cross-attention decoder. Establish a masked pixel reconstruction loss that includes mean square error and spectral angle similarity loss. The masked peak reconstruction task includes: randomly masking the intensity values ​​of some peaks in the peak list of each pixel, preserving the mass-to-charge ratio position encoding of the masked peaks, predicting the intensity values ​​of the masked peaks, and establishing the mean square error loss as the masked peak reconstruction loss. The cross-sample contrastive learning task includes: using pixel pairs from the same tissue type or the same mass spectrometry imaging dataset as positive sample pairs and pixel pairs from different tissue types or different mass spectrometry imaging datasets as negative sample pairs, and establishing InfoNCE loss. A total loss is established based on the mask pixel reconstruction loss, mask peak reconstruction loss, and InfoNCE loss. This loss is used to jointly train the pedestal model, update the parameters of the pedestal model, and obtain the trained pedestal model.

[0024] Preferably, after the pedestal model is trained, only the parameters of the peak embedding layer, the Perceiver encoder, the spatial Transformer, and the PerceiverIO decoder are saved as the pedestal model weights, while the reconstruction decoder parameters used specifically for the pre-training task are discarded.

[0025] Preferably, the following principles are followed when performing inference for downstream analysis tasks: The parameter size of the downstream analysis task head is smaller than that of the base model parameter size; The base model parameters are frozen by default. When the amount of labeled data reaches a preset threshold, it is allowed to make minor adjustments to the last few layers of the base model at a learning rate of less than one-tenth of the base model's learning rate. When no labeled data exists, unsupervised analysis is performed directly on the feature tensor, and zero-shot inference is performed using the semantic structure already learned by the base model.

[0026] Preferably, the downstream analysis tasks described in step 4 include cross-instrument biomarker validation, metabolic pathway enrichment analysis, and unsupervised typing of the tissue microenvironment.

[0027] Compared with the prior art, the beneficial effects of the present invention include at least the following: First, it achieves instrument-independent unified encoding of mass spectrometry imaging data. The peak sequence input representation method proposed in this invention transforms the original intensity vector of any instrument and any number of channels into a unified variable-length peak pair list format. Combined with the Perceiver cross-attention encoding architecture, it enables mass spectrometry imaging data of different instrument types, different numbers of channels, and different mass-to-charge ratio coverage ranges to be encoded using the same set of model parameters, without the need to design separate network structures or retrain models for each instrument. This instrument-independent characteristic allows this invention to cover the vast majority of instrument types currently used in mass spectrometry imaging laboratories worldwide and maintains natural compatibility with new instruments that will emerge in the future, fundamentally solving the core defect of existing methods that cannot be reused across instruments due to the fixed number of channels.

[0028] Second, it achieves efficient fixed-length encoding of variable-length peak sequences. The Perceiver cross-attention encoding architecture employed in this invention queries arbitrary lengths using a fixed number of learnable latent terms. The peak sequence reduces the computational complexity from Reduce to ,exist Fixed (Example value) Under the premise of any The peak sequences of values ​​can all be encoded within an acceptable computational cost. During training, the FlashAttention variable-length sequence computation interface is used, which eliminates the need for zero-padding of sequences of different lengths, fundamentally eliminating the interference of padding terms on gradient calculation and maximizing the effective utilization of GPU memory.

[0029] Third, it enables the explicit utilization of physicochemical information related to the mass-to-charge ratio. The peak embedding method proposed in this invention encodes the physical dimensional information of the mass-to-charge ratio value into a continuously differentiable high-dimensional vector by applying Fourier position encoding to the mass-to-charge ratio value of each peak. This allows peaks with similar mass-to-charge ratios to naturally acquire high similarity in the feature space, enabling the Perceiver cross-attention mechanism to perceive the correlation between the mass difference and chemical structure between peaks. This helps the model learn peak correlation patterns with clear chemical significance, such as isotope peak clusters and adduct ion pairs. This mass-to-charge ratio perception capability is not possessed by existing general deep learning methods, which helps improve the chemical rationality and interpretability of the feature representation of the base model.

[0030] Fourth, a fundamental model pre-training paradigm for the field of mass spectrometry imaging has been established. The three complementary self-supervised pre-training tasks proposed in this invention simultaneously apply learning pressure from three dimensions: spatial context (masked pixel reconstruction), channel chemical regularity (masked peak reconstruction), and cross-sample semantic consistency (cross-sample contrastive learning). This allows for the learning of universal chemical-spatial pixel-level feature representations from large-scale heterogeneous mass spectrometry imaging data that are independent of instrument type and downstream tasks, without any manual annotation. This fundamental model pre-training paradigm is the first of its kind in the field of mass spectrometry imaging, breaking the fragmented pattern of existing methods that train models independently for each task, and systematically releasing the value of the massive amounts of unlabeled data accumulated by mass spectrometry imaging laboratories worldwide.

[0031] Fifth, it supports dynamic specification of the output dimension during inference. The PerceiverIO decoding architecture used in this invention can dynamically specify the dimension of the output feature tensor without retraining the model by adjusting the number of output query vectors during the inference phase. This allows the same set of pre-trained base model weights to be adapted to diverse downstream tasks with different requirements for representation dimensions using different feature dimensions, thus balancing the model's versatility with the flexibility of adapting to downstream tasks.

[0032] Sixth, it significantly reduces the dependence of downstream tasks on labeled data. The general feature representations obtained by the base model pre-training on large-scale unlabeled heterogeneous data can be directly transferred to various downstream analysis tasks as a powerful feature backbone. For supervised downstream tasks, effective transfer can be achieved with no fewer than five labeled samples, which is significantly lower than the labeling scale required to train similar task models from scratch. For unsupervised downstream tasks (such as tissue microenvironment typing), no labeled data is required to directly perform cluster analysis on the base model features and obtain meaningful typing results, effectively alleviating the core bottleneck of scarce labeled data in the field of mass spectrometry imaging. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0034] Figure 1 This is a flowchart illustrating the overall process of the self-supervised representation learning method for heterogeneous mass spectrometry data proposed in this invention.

[0035] Figure 2 This is a flowchart of the mass spectrometry imaging data preprocessing process.

[0036] Figure 3 A schematic diagram of the internal structure of the base model provided by the present invention.

[0037] Figure 4 A diagram illustrating a downstream application embodiment provided by the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and given in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0039] The technical concept of this invention is as follows: Existing technologies in the field of intelligent analysis of heterogeneous mass spectrometry imaging data suffer from the following shortcomings: instrument heterogeneity prevents model reuse across instruments; high-dimensional sparse data lacks effective encoding methods; mass-to-charge ratio physicochemical information is not utilized; and there is a contradiction between the scarcity of labeled data and the diversity of downstream tasks. This invention provides a self-supervised representation learning method for heterogeneous mass spectrometry data, such as... Figure 1 As shown, it includes the following steps: S1. Obtain at least one mass spectrometry imaging dataset, collect the metadata in the mass spectrometry imaging dataset and encode it into a conditional embedding vector, and perform peak detection and preprocessing on the original intensity vector of each pixel to obtain a variable-length peak pair list for each pixel.

[0040] The purpose of this step is to acquire raw data from heterogeneous mass spectrometry imaging data sources and, through a standardized preprocessing workflow, transform mass spectrometry imaging data from any instrument and with any number of channels into a peak sequence representation in a unified format. Simultaneously, it collects and encodes instrument metadata, providing high-quality standardized input for subsequent pre-training of the pedestal model. This step outputs two types of information: first, a standardized peak sequence dataset in the format of a variable-length list of peak pairs; second, metadata condition vectors for each dataset, used by the conditional signal module of S2. For the complete processing flow of this step, please refer to [link to full workflow]. Figure 2 .

[0041] S1.1 Data Sources and Scale Requirements S1.1.1 Data Source The mass spectrometry imaging data used in this invention are sourced from the following three levels, each of which can be used individually or in combination: Tier 1: Public Mass Spectrometry Imaging Databases. Public mass spectrometry imaging databases should be prioritized as the primary source of pre-training data. The METASPACE database is recommended, as it contains over 10,000 public mass spectrometry imaging datasets, covering various instrument types such as MALDI-MSI and DESI-MSI, and multiple species including humans, mice, and plants. It provides various detection targets such as metabolomics and lipidomics, and the raw data files can be directly downloaded for pre-training without additional registration or annotation. In addition, the MetaboLights and PRIDE databases also provide usable raw mass spectrometry imaging data and can be used as supplementary sources.

[0042] Second Tier: Collaborative Institutional Experimental Data. Self-collected mass spectrometry imaging data from research or medical institutions can supplement public databases in areas where coverage for specific instrument brands, tissue types, or disease scenarios is insufficient. Collaborative institutional data must meet the quality control standards described in step S1.4 to be included in the training set. Data at this level does not require labeling; raw intensity data can be used for pre-training.

[0043] The third level: Simulated data generation (optional). Using computational tools such as CFM-ID, simulated mass spectrometry imaging spectra of known molecules are generated by overlaying instrument noise models onto a batch of simulated mass spectrometry imaging spectra. This is used to expand the sample size of low-frequency molecule types or rare instrument types in the training set. Simulated data should be labeled as originating from a simulation source and can be distinguished from real data during model training using sample weights.

[0044] S1.1.2 Data Scale Requirements The pre-training phase requires at least one thousand mass spectrometry imaging datasets, preferably at least five thousand, to cover as many instrument types, detection objects, and tissue sources as possible, thereby improving the base model's generalization ability to heterogeneous data. The downstream task fine-tuning phase requires at least five labeled samples, preferably ten to twenty, with the specific size depending on the complexity of the downstream task. The above data sizes are exemplary values ​​and do not limit the scope of protection of this invention.

[0045] S1.2 Meta-information Collection and Coding S1.2.1 Definition of Meta-information Fields For each mass spectrometry imaging dataset, the following metadata fields are collected synchronously during the data acquisition or data download phase. These metadata fields are divided into three levels: required fields, suggested fields, and optional fields.

[0046] Required fields: Required fields are necessary for the normal operation of the model. If they are missing, they should be filled with default values ​​and marked as unknown.

[0047] First, the instrument type, with values ​​including but not limited to MALDI, DESI, SIMS, REIMS, LAESI and other types, recorded as categorical variables.

[0048] Secondly, the ion mode, with values ​​ranging from positive ion mode to negative ion mode, is recorded as a binary variable.

[0049] Third, the mass-to-charge ratio detection range is recorded as an interval. The unit is Dalton (Da), and it is recorded as two consecutive numerical values.

[0050] Fourth, spatial resolution, recorded as the physical size corresponding to each pixel, in micrometers per pixel (µm / pixel), recorded in continuous numerical form.

[0051] Suggested fields: Suggested fields should be collected as much as possible when conditions permit. If they are missing, they should be filled with unknown categories. This will not affect the model's operation but will help improve the quality of feature representation.

[0052] Firstly, the types of objects to be detected include metabolomics, lipidomics, proteomics, elemental imaging, and other types.

[0053] Secondly, the species origin, the range of values ​​includes humans, rodents, plants and other species.

[0054] Third, the tissue type, the range of values ​​includes brain, liver, lung, tumor and other tissue types, which can be filled in according to the annotation information of the specific database.

[0055] Optional fields: Optional fields can further enhance the model's ability to perceive instrument characteristics when the corresponding conditions are met, and their absence will not affect the model's operation.

[0056] First, the instrument's technical specifications text, including natural language descriptions of technical parameters such as instrument model, mass resolution, and mass accuracy, are sourced from the instrument manual or official technical documents.

[0057] Secondly, the experimental conditions are described in natural language, including sample preparation methods, matrix types (for MALDI), solvent systems (for DESI), and other experimental parameters.

[0058] S1.2.2 Meta-information Encoding Method The aforementioned metadata fields, after being independently encoded, are fused into a unified conditional embedding vector. This is for use by the condition signal module in S2. The encoding method for each field is as follows: Categorical variable fields (instrument type, ion mode, detection object type, species origin, tissue type) are encoded using one-hot encoding and then mapped to a dense embedding vector of fixed dimensions via a linear projection layer.

[0059] Continuous numerical fields (minimum and maximum values ​​of the mass-to-charge ratio range, spatial resolution) are transformed logarithmically and then mapped to a dense embedding vector of fixed dimensions via a linear projection layer. The logarithmic transformation is used to compress the numerical range and improve numerical stability.

[0060] Natural language text fields (instrument technical specifications text, experimental condition descriptions) are encoded into high-dimensional text embedding vectors by a pre-trained language model, and then mapped to a fixed dimension through a linear dimensionality reduction layer. The pre-trained language model can be BGE-M3, SciBERT, or other applicable text encoding models. The text embeddings are calculated offline and cached during the preprocessing stage, and are not calculated in real time during the training and inference stages to control computational costs.

[0061] The embedding vectors of all fields are concatenated along the feature dimension, and then fused through a linear projection layer and a layer normalization operation. When a field is missing, the corresponding embedding vector is replaced with an all-zero vector and a learnable missing label embedding is appended, enabling the model to perceive the field's missing state. The above dimensions are exemplary values ​​and do not limit the scope of protection of this invention.

[0062] S1.3 Peak Detection and Preprocessing This step performs normalization preprocessing on the original intensity vector of each pixel, transforming the sparse intensity vector with any number of channels into a variable-length list of peak pairs. ,in For the first The mass-to-charge ratio values ​​corresponding to each effective peak. The normalized intensity value. The number of effective peaks is automatically determined by the data and requires no manual preset. This step corresponds to... Figure 2 The complete preprocessing flow shown below executes the following four sub-steps sequentially.

[0063] S1.3.1 Baseline Correction In the raw mass spectrometry imaging data, the intensity vector of each pixel generally has a slowly changing background baseline caused by factors such as instrument electronic noise, chemical background noise and matrix interference. This background baseline is superimposed on the real molecular signal, and if it is not eliminated, it will interfere with the accuracy of subsequent peak detection.

[0064] The mass-to-charge ratio dimension is used to calculate local minima using a rolling window size of 5% of the number of channels (an exemplary value) to estimate the continuous baseline curve. The original intensity vector is then subtracted from the estimated baseline and truncated to a non-negative value to obtain the baseline-corrected intensity vector. The above method is an exemplary implementation; locally weighted regression smoothing (LOESS), ConvexHull, or other equivalent baseline correction methods can also be used, without limiting the scope of protection of this invention.

[0065] S1.3.2 Signal-to-noise ratio filtering After baseline correction, the signal-to-noise ratio (SNR) is calculated for each pixel at each channel position, and only the channel positions with SNR exceeding a preset threshold are retained as valid peaks.

[0066] Background noise level Defined as the root mean square (RMS) of the channel intensity values ​​in the zeroth to twentieth percentile interval of the intensity vector after baseline correction for that pixel: , in This is the set of channel indices sorted by intensity value and located in the 0th to 20th percentile range. The root mean square (RMS) is used instead of the standard deviation as the noise estimator because the intensity values ​​of the noise channels are all non-negative after baseline correction. The RMS can stably reflect the absolute intensity level of the noise, avoiding the numerical instability caused by the standard deviation tending to zero when the noise distribution is uniform.

[0067] The signal-to-noise ratio of each channel is defined as follows: , in For the first The intensity values ​​after baseline correction for each channel. The SNR threshold is set to 3 (example value), retaining only the baseline-corrected intensity values ​​for each channel. The channel position is determined by noise, and the remaining channels are discarded as noise. The noise estimation range and SNR threshold mentioned above are exemplary values ​​and do not limit the scope of protection of this invention.

[0068] By using SNR filtering, the number of effective peaks in a typical metabolomics mass spectrometry imaging pixel Approximately the total number of channels The computational load for subsequent processing is significantly reduced by 5% to 15%. The intensity values ​​of the discarded channels are all below the noise floor and do not carry reliable chemical signals; therefore, their discarding does not constitute a loss of effective chemical information.

[0069] S1.3.3 Total ion current normalization Differences in ionization efficiency between different pixels, non-uniformity in sample preparation, and fluctuations in instrument response can lead to different absolute intensity levels at tissue locations with the same chemical composition. These differences do not reflect the true differences in chemical concentration, and if not eliminated, they will interfere with the model's learning of spatial metabolic distribution patterns.

[0070] For each pixel, the total ion current (TIC) is calculated by summing the intensities of all valid peaks in that pixel. The intensity values ​​of each valid peak are then normalized by dividing the TIC by the sum of the intensities. .

[0071] After normalization, the sum of the intensity values ​​of all valid peaks in each pixel is one, eliminating the systematic differences in absolute intensity between pixels and preserving the relative intensity relationships between peaks. The above method is an exemplary implementation; median normalization, reference peak normalization, or other equivalent methods may also be used, without limiting the scope of protection of this invention.

[0072] S1.3.4 Peak List Construction and Format Standardization After the above three sub-steps, the mass-to-charge ratio and normalized intensity values ​​of all valid peaks are extracted for each pixel, and a peak pair list is constructed: , in For the first The actual physical value of each effective peak on the mass-to-charge ratio axis, in Da, retained to the instrument's rated precision (example value is taken to two decimal places). The relative intensity value after TIC normalization, with a value range of (0,1]; The number of effective peaks is automatically determined from the raw data of that pixel through the preprocessing steps described above. This applies to data from different pixels, different instruments, and different numbers of channels. The values ​​vary and do not need to be standardized to a fixed length.

[0073] The true mass-to-charge ratio, serving as the peak's position coordinates in chemical space, will be transformed into a continuously differentiable chemical position feature vector via Fourier position encoding in the peak embedding layer of S2. This vector will be used to perceive the correlation between mass differences and chemical structures between peaks. The normalized intensity values ​​will be linearly projected into intensity feature vectors after logarithmic transformation. The peak pair list format is applicable to mass spectrometry imaging data of any instrument type and any number of channels, and all are of variable length. For sequences, they can be directly input into the S2 Perceiver encoder for unified processing.

[0074] Number of effective peaks Pixels exceeding 500 are sorted in descending order of intensity value and then truncated to the first 500 peaks (example values); Pixels with a count less than ten are considered low-quality pixels and are marked during the quality control stage in S1.4. The peak count thresholds mentioned above are exemplary values ​​and do not limit the scope of protection of this invention.

[0075] S1.4 Quality Control and Dataset Construction S1.4.1 Data Quality Filtering Standard For the mass spectrometry imaging dataset processed by S1.3, quality filtering is performed according to the following criteria; datasets that do not meet the criteria are removed from the training set: First, the effective pixel ratio is no less than 60%, which is the number of effective peaks in the dataset. The proportion of pixels in a given area to all pixels is no less than 60%.

[0076] Secondly, the average number of valid peaks across all valid pixels in the dataset. The number should be no less than twenty to ensure that each pixel carries enough chemical information for model learning.

[0077] Third, within the instrument's nominal mass-to-charge ratio range, at least 30% of the mass-to-charge ratio intervals contain valid signals, avoiding the inclusion of low-quality datasets with extremely sparse signals or failed acquisitions.

[0078] Fourth, all required fields (instrument type, ion mode, mass-to-charge ratio range, spatial resolution) must be validly recorded; it is not permissible for all required fields to be missing.

[0079] S1.4.2 Dataset Partitioning The quality-filtered datasets are divided into a pre-training set and a reserve set according to the following principles: Five percent of the dataset is randomly selected from different instrument types, species origins, and object categories to form the reserve set, which is used for validation and monitoring during pre-training; the remaining ninety-five percent of the dataset constitutes the pre-training set for S2 self-supervised pre-training. The reserve set should be constructed to ensure coverage of at least three instrument types and at least two object categories so that validation metrics can reflect the model's generalization ability to heterogeneous data.

[0080] S1.4.3 Step Output Summary This step ultimately outputs the following three types of information for use in subsequent steps: First, a standardized peak sequence dataset, comprising a pre-training set and a reserved set, wherein each sample consists of a spatial size of It is composed of pixels, and each pixel corresponds to a format of The list of peak pairs, It is determined automatically by the data and does not require standardization.

[0081] Secondly, the meta-information conditional vector, one for each dataset. In S2, it is injected as a conditional signal into each coding layer of the base model.

[0082] Third, the dataset metadata records include the quality control results of each dataset, statistics on instrument type distribution, statistics on channel number distribution, and statistics on effective peak number distribution, which are used to monitor the overall quality and diversity distribution of the pre-training dataset.

[0083] S2. Construct a base model that takes a variable-length list of peak pairs as input and maps it through a peak embedding layer to obtain peak embedding vectors. All peak embedding vectors of the same pixel form a peak lexical matrix. Using the latent lexical matrix as the query and the peak lexical matrix as the key, the pixel latent representation is encoded by a Perceiver encoder, and the conditional embedding vector is injected into the Perceiver encoder. After superimposing spatial location encoding on the pixel latent representation of each pixel, it is input into the spatial Transformer. The spatial context between pixels is learned through self-attention, and the spatially enhanced pixel representation is output. The spatially enhanced pixels are decoded by the PerceiverIO decoder to obtain the feature vector of each pixel. The feature vectors of all pixels constitute a feature tensor.

[0084] This step takes the standardized peak sequence dataset and meta-information conditional vector output from step S1 as input. Through a dedicated hierarchical coding architecture and three self-supervised pre-training tasks, it learns a general pixel-level feature representation from large-scale heterogeneous mass spectrometry imaging data without any manual annotation. After pre-training, the encoder weights are saved to form a base model for use in downstream applications of S3. For the complete architecture of this step, please refer to [link to full architecture]. Figure 3 .

[0085] The core technical design of this step is reflected in the following three aspects: First, by using peak sequence representation and Perceiver cross-attention encoding, the sparse intensity vector of any number of channels is compressed into a potential representation of fixed dimensions, fundamentally solving the heterogeneity problem that data from different instruments cannot be input into the same model; Second, by introducing spatial coordinates in the form of positional encoding after pixel aggregation, chemical information (characterized by mass-to-charge ratio and intensity value) and spatial information are decoupled at different levels; Third, through three complementary self-supervised pre-training tasks, learning pressure is applied to spatial context relationships, inter-channel chemical correlations, and cross-sample semantic consistency to learn a complete chemical-spatial unified representation.

[0086] S2.1 Peak Embedding Layer S2.1.1 Input Format The input for this step is the normalized peak sequence output from S1.3. For a spatial size of... Mass spectrometry imaging data, each pixel location A corresponding list of peak pairs ,in The number of valid peaks for a given pixel can vary across different pixels. The peak embedding layer uses a list of peaks for each individual pixel as the basic processing unit, performing embedding calculations independently on each peak in the list and outputting a dense feature vector representation of that peak. It should be noted that this layer only processes the chemical position information (mass-to-charge ratio) and intensity information of each peak; the spatial coordinates of the pixel are not considered. Instead of being introduced at this layer, it is injected separately at the pixel level in the form of position encoding in step S2.2.6, thereby achieving hierarchical decoupling of chemical information and spatial information.

[0087] S2.1.2 Mass-to-charge ratio Fourier position coding Mass-to-charge ratio These are the physical coordinates of each peak in chemical space, carrying specific molecular mass information. Similar mass-to-charge ratios often correspond to molecules with related chemical structures (e.g., a difference of 14 Da corresponds to a CH2 group). To enable the model to perceive the differences in mass-to-charge ratios between peaks and the chemical structural correlations they imply, this invention... Apply Fourier position encoding: , in This is a temperature hyperparameter (example value is 10000). The encoding dimension is 64 (exemplary value). This encoding method ensures that two peaks with similar mass-to-charge ratios have high cosine similarity in their encoding vectors, allowing the model to naturally perceive the correlation between chemically related peaks during attention calculations; simultaneously, the encoding is continuously differentiable and not limited by discrete bin boundaries. Note that the temperature parameter... The value should match the actual mass-to-charge ratio range to ensure sufficient frequency resolution of the encoding within the target mass-to-charge ratio range; when the mass-to-charge ratio range deviates significantly from the typical range of 100 to 50000 Da, it is recommended to adjust accordingly. The values ​​of these parameters are either normalized before encoding, or the mass-to-charge ratio is preprocessed. These parameters are all exemplary values ​​and do not limit the scope of protection of this invention.

[0088] S2.1.3 Intensity Coding Normalized intensity value After logarithmic transformation, the input to the linear projection layer yields the intensity feature vector: , in and For learnable parameters, This is the numerical stability constant (example value is 10). -6 ), The encoding dimension is the same as in S2.1.2 (an example value of 64). The logarithmic transformation is used to compress the dynamic range of the intensity values, enabling effective gradient updates even for weak peak signals during training.

[0089] S2.1.4 Peak Embedding Vector Calculation The mass-to-charge ratio Fourier position encoding vector and the intensity encoding vector are concatenated along the feature dimension, and then subjected to a linear projection layer and layer normalization operation to obtain the embedding vector of a single peak: , in For peak embedding dimension (example value is 128). Pixels of After the above embedding calculation, the input word matrix of the pixel is obtained: , The number of rows in the lexical matrix The number of columns can vary between different pixels. This is fixed to provide a consistent sequence input for the subsequent Perceiver encoder.

[0090] S2.2 Perceiver encoder S2.2.1 Design Motivation Number of effective peaks in different pixels Different, different instruments Significant distributional differences make it impossible to pad all peak sequences in a batch of data to a uniform length for batch parallel computation without introducing a large amount of invalid padding. To address this issue, this invention employs a Perceiver cross-attention architecture: it introduces a fixed number of learnable latent lexical sequences as the query end and variable-length peak lexical sequences as the key end. Through a cross-attention mechanism, peak sequences of arbitrary length are compressed into a fixed-dimensional latent representation, fundamentally eliminating the requirement for uniform input sequence length.

[0091] S2.2.2 Learnable latent lexical units definition A matrix composed of learnable potential lexical units ,in This represents the number of potential lexical units (an example value is 64). These are the learnable parameters of the model, which are randomly initialized with a truncated normal distribution before training begins and updated with gradients during training. The value of is related to the length of the input peak sequence. The irrelevance determines the representation capacity of the Perceiver encoder output.

[0092] S2.2.3 Cross-attention layer In the In layer perceiver encoding, the current latent lexical matrix is ​​used. As a query endpoint, using the peak keyword matrix As the key-value side, perform multi-head cross-attention calculation: , , in , , For the first The learnable projection matrix of the layer, For the attention head dimension, the computational complexity of the cross-attention is... ,in fixed, Variable, significantly superior to self-attention mechanisms. .

[0093] S2.2.4 Self-attention layer and conditional signal injection After each layer of cross-attention calculation, the updated latent word sequence is processed sequentially using self-attention and feedforward networks: , In the layer normalization operation of each sub-layer mentioned above, the meta-information conditional embedding vector calculated in step S1.2.2 is injected. This is achieved using a Conditional LayerNorm mechanism. , in , , and These are the characteristic mean and standard deviation, respectively. This is an element-wise multiplication. The conditional injection mechanism enables the model to dynamically adjust the feature normalization behavior of each layer based on meta-information such as instrument type and ion mode, which helps the model adapt to heterogeneous data distributions from different instruments under unified parameters.

[0094] The above-mentioned cross-attention layers and self-attention layers are stacked alternately. The layer (with an example value of 6) ultimately outputs pixels. The latent representation matrix: , The number of rows in the latent representation matrix Fixed, with respect to the length of the input peak sequence Completely unrelated, it achieves unified encoding for mass spectrometry imaging data with any number of channels and any instrument type.

[0095] S2.2.5 Batch processing of variable-length sequences During training, different samples in the same batch The values ​​are usually different. This invention uses the FlashAttention variable-length sequence calculation interface: it concatenates the peak word sequence of all pixels in the same batch along the sequence dimension into a single long sequence, and accumulates the sequence length vector. The sequence boundaries of each sample are recorded, and attention calculations are strictly limited to these boundaries to ensure cross-sample attention isolation. This implementation eliminates the need for zero-padding of sequences of different lengths, fundamentally removing the interference of padding terms on gradient calculations while maximizing GPU memory utilization.

[0096] S2.2.6 Spatial Location Encoding Injection The Perceiver encoder completes a single pixel. After the latent representation is computed, the latent representation matrix of each pixel is injected with the two-dimensional spatial location encoding of that pixel, so that subsequent spatial Transformers can perceive the spatial topological relationships between pixels: , in For pixels The corresponding two-dimensional sine and cosine position encoding matrix is ​​obtained by concatenating the row and column coordinates after mapping them with sine and cosine functions of different frequencies. The spatial position encoding is injected after the output of the Perceiver encoder and before the input of the spatial Transformer, ensuring that the peak embedding layer and the Perceiver encoder only process chemical information. Spatial coordinates are introduced separately at the pixel level, realizing hierarchical decoupling of chemical information and spatial information, and avoiding semantic confusion between the two types of information in the same vector space.

[0097] S2.3 Spatial Transformer S2.3.1 Spatial Block Training Strategy Since the number of pixels in a complete mass spectrometry imaging slice is typically tens of thousands to hundreds of thousands, performing spatial self-attention calculations on all pixels of the entire slice simultaneously would exceed the actual limitations of current hardware. Therefore, this invention employs a spatial patch training strategy: during the training phase, a patch is randomly selected from each slice. A spatial block composed of 10 pixels is used as the basic unit for a single training iteration (example value) =32, each piece contains =1024 pixels). The spatial block covers a physical area of ​​1600 μm × 1600 μm at typical mass spectrometry imaging resolution (e.g., 50 μm / pixel), which is sufficient to cover multiple cells and their microenvironment units, and has sufficient spatial context information.

[0098] S2.3.2 Spatial Self-Attention Calculation Will The latent representation matrix of pixels Arranged into a sequence based on spatial location, then... Layer (example value is 6) spatial Transformer encoder processing: , in The set of pixels within the current spatial block. This is the latent pixel representation enhanced with spatial context. Each layer of the spatial Transformer also employs conditional layer normalization injection. This enables spatial attention calculations to also perceive meta-information such as instrument type. The core information learned by the Spatial Transformer is the co-localization pattern between pixels in the spatial dimension, which corresponds to biological characteristics such as the physiological structure, metabolic partitions, and cellular microenvironment of tissues.

[0099] S2.3.3 Inference Stage Processing Backpropagation is not performed during the inference phase, resulting in significantly lower memory requirements compared to the training phase. When performing inference on the entire slice, a sliding window strategy is employed: The window slides sequentially across the entire slice's spatial dimensions, with a step size set to [value missing]. (No overlap) or (Semi-overlapping, smoother features at the boundaries), extract features block by block and stitch them together to form a feature map of a complete slice.

[0100] S2.4 PerceiverIO Decoding Output S2.4.1 Training Phase Output During the training phase, a fixed number of output query vectors are used to decode the pixel latent representation into a fixed-dimensional feature vector. A learnable output query matrix is ​​defined. The feature vector of a pixel is calculated as follows: ; Training phase The value is fixed at 256 (example value). The feature vectors of all pixels together constitute the feature tensor of the entire slice. .

[0101] S2.4.2 Dynamically Specifying Output Dimensions During the Inference Phase Inference phase, output dimension It can be dynamically specified by the user according to the needs of downstream tasks, without the need to retrain the model. The specific implementation method is as follows: Construct Each output query vector is initialized using the values ​​from the training phase. Interpolation is used to ensure semantic continuity under different query volumes; the output query vector is processed through cross-attention query pixel latent representation. Output The feature vector is dynamically specified during inference, enabling the base model to adapt to diverse downstream tasks with varying requirements for feature dimensions without retraining. During inference... When the parameters are reused directly from the training phase, the output query vector yields the most stable results; when... Unlike during training, it is recommended to perform adaptation fine-tuning on the output query vector with a small number of samples to obtain the best feature quality.

[0102] S3. Jointly train the pedestal model using mask pixel reconstruction task, mask peak reconstruction task and cross-sample contrast learning task, update the parameters of the pedestal model, and obtain the trained pedestal model.

[0103] This invention designs three complementary self-supervised pre-training tasks to exert learning pressure on the model from three dimensions: spatial context, channel chemistry regularity, and cross-sample semantic consistency. Figure 3The pre-training task module shown in column ⑤ consists of a weighted sum of the loss functions of the three tasks to form the total pre-training loss.

[0104] Task 1: Mask Pixel Reconstruction Masking strategy: For each training spatial block, randomly sample the mask index in the spatial pixel dimension, and set the mask ratio to . =75% (example value). The complete list of peak locations for masked pixels is not input into the Perceiver encoder; only the visible pixels that are not masked (approximately 25%) are included. (Number) input encoders.

[0105] Reconstruction Target: For the masked pixel location, the normalized intensity values ​​of each effective peak at that location are reconstructed from the encoded features of the visible pixels using a cross-attention decoder. The reconstruction target is the original normalized peak intensity vector of the masked pixel, not a sparse intensity vector with a fixed number of channels, which corresponds perfectly to the peak sequence input format of this invention. It should be noted that the mass-to-charge ratio position of the masked pixel during reconstruction is determined by the instrument's nominal parameters and is a known quantity; only the normalized intensity value is used as the reconstruction prediction target.

[0106] Loss function:

[0107] in and The SAM loss is defined as the reconstructed peak intensity vector and the original peak intensity vector, respectively: ; The mean squared error loss constrains the absolute numerical error of the peak intensity, while the SAM loss constrains the similarity of the normalized peak shape structure; the two are complementary. This is the SAM loss weighting coefficient (an example value of 0.1).

[0108] Task 2: Mask Peak Reconstruction Masking strategy: For each pixel's peak list, randomly sample the mask index along the peak sequence dimension, and set the mask ratio to [value missing]. =30% (example value). Intensity value of the masked peak. Set to zero, but its mass-to-charge ratio position is encoded. The intensity value is retained and participates normally in attention calculation. The design is based on the fact that the mass-to-charge ratio is a physical identifier of a molecule and is a known quantity under constant experimental conditions; while the intensity value reflects the concentration of that molecule in the current pixel and is an unknown quantity that needs to be inferred from the context. By retaining the mass-to-charge ratio positional encoding while only masking the intensity value, the model can perceive the chemical identity of the masked peak and infer its intensity value based on the intensity distribution of other peaks, thereby learning the chemical co-occurrence patterns between channels.

[0109] Loss function: , in The set of indices for the masked peaks is used. The loss is calculated only on the masked peaks, and the visible peaks are not included in the loss calculation.

[0110] Task 3: Cross-sample comparative learning Sample pair construction: Positive sample pairs are defined as pixel pairs randomly sampled from different mass spectrometry imaging samples of the same tissue type; negative sample pairs are defined as pixel pairs from different tissue types. When the tissue type field is missing in the metadata, the dataset source is used as a surrogate label. Pixel pairs within the same dataset are considered positive sample pairs, and pixel pairs from different datasets are considered negative sample pairs. The surrogate label based on the dataset source is a weakly supervised approximation. Multiple tissue types may exist within the same dataset, therefore the surrogate label contains a certain proportion of noise labels. Under large-scale training data conditions, contrastive learning has a certain robustness to a small number of noise labels, and this approximation does not affect the overall learning performance of the model.

[0111] Comparative loss: ,

[0112] in The feature vector of the anchor sample. The feature vector of the positive sample. For the set of negative samples within the batch, This is the temperature hyperparameter (example value is 0.07).

[0113] S3.4 Total Loss and Training Configuration ,

[0114] in =0.5, =0.3 (all are exemplary values). Task 1 is the main task, directly optimizing the reconstruction capability of pixel-level features; Tasks 2 and 3 are auxiliary tasks, with lower weights than the main task.

[0115] The training configuration is as follows (all are exemplary values): the optimizer uses AdamW, the initial learning rate is cosine annealing learning rate scheduling, the training is 200 epochs, the batch size is 32 spatial blocks, the weight decay coefficient is 0.05, the gradient clipping threshold is 1.0, mixed precision training (FP16) is used, and the loss scaling factor is adaptively adjusted.

[0116] S3.5 Pre-training Result Processing After pre-training is complete, save the model parameters according to the following principles: The retained parts include all parameters of the peak embedding layer, the Perceiver encoder (including learnable potential tokens and all attention layer parameters), the spatial Transformer encoder, and the PerceiverIO decoder. These modules together constitute the weights of the base model encoder.

[0117] Discarded parts: The reconstruction decoder and reconstruction head designed specifically for the pre-training task are discarded. This module is only used for the reconstruction loss calculation in the self-supervised training stage and does not participate in the feature extraction and inference process of the downstream task.

[0118] Base model validation: Feature quality assessment using the retained set data: Extract feature vectors of known tissue type samples from the retained set, train a lightweight linear classifier using linear probing, evaluate the linear separability on the tissue type classification task, and use this as a quantitative indicator of the feature representation quality of the base model. After confirming that the pre-training is sufficient, save the final model weights.

[0119] S4. For newly acquired mass spectrometry imaging data, after processing in step 1, the corresponding variable-length peak pair list and conditional embedding vector are obtained. The trained base model parameters are loaded, and forward inference is performed to obtain the feature tensor, which is used for downstream analysis tasks.

[0120] This step uses the encoder weights of the pedestal model output from step S2 as the feature extraction backbone to perform feature extraction on any newly acquired mass spectrometry imaging data. It supports various analysis tasks by connecting to a lightweight downstream task head. The parameters of the pedestal model are frozen by default in this step, and the learned general mass spectrometry imaging feature representations are directly transferred to downstream tasks, significantly reducing the dependence of downstream tasks on labeled data. For the overall framework of this step, please refer to [link to relevant documentation]. Figure 4 .

[0121] The core difference between this step and existing technologies is that the feature extraction of the base model is completely decoupled from the design of the downstream task head, and the same set of pre-trained encoder weights can be directly reused on data with different tasks, different instruments, and different numbers of channels; the dimension of the feature tensor output by the base model can be dynamically specified by the user during the inference stage without retraining the model.

[0122] S4.1 General Reasoning Flow S4.1.1 New Data Preprocessing For the newly acquired mass spectrometry imaging data, data processing is performed according to the complete preprocessing procedure described in step S1, including metadata collection and encoding (S1.2), peak detection and preprocessing (S1.3), and quality control (S1.4), outputting standardized peak sequence data and corresponding metadata condition vectors. The preprocessing procedure has no restrictions on the type of instrument used for new data. Even if a new type of instrument that has not appeared in the pre-training stage is used, as long as it can provide effective metadata fields and output a standardized peak sequence after peak detection, it can be directly input into the base model for feature extraction.

[0123] S4.1.2 Feature Extraction of Base Model Load the base model encoder weights saved in step S2, and perform forward inference on the preprocessed new data. No masking is applied, and gradient calculation is not performed throughout the process. The inference process executes sequentially: the peak list is transformed into a word matrix via the peak embedding layer. Each pixel-level word matrix is ​​compressed into a latent representation by the Perceiver encoder. It also adds spatial location encoding; the spatial Transformer processes the entire slice pixels block by block in a sliding window manner, outputting a spatial context-enhanced feature representation. The PerceiverIO decoder will Decoded as 1D feature vector, outputting the feature tensor of the complete slice: ; Feature tensor Each spatial location One The 3D feature vector, which integrates the chemical composition information, spatial context information, and instrument prior information at that location, serves as the unified input for subsequent downstream task heads.

[0124] S4.1.3 General Design Principles for Downstream Task Headers Based on feature tensor When designing a lightweight downstream task head, the following principles should be followed: First, the size of the task head parameters should be significantly smaller than the size of the base model parameters; Second, when there is a small amount of labeled data, only the task head parameters should be updated, and the base model encoder parameters should be frozen by default. When the labeled data is sufficient (preferably no less than fifty labeled samples), a very small learning rate of no more than one-tenth of the base model's learning rate can be used to perform lightweight fine-tuning on the last few layers of the encoder; Third, when there is no labeled data, the feature tensor can be directly modified. Perform unsupervised analysis and use the semantic structure learned by the base model for zero-shot inference.

[0125] S4.2 Application Scenario 1: Cross-Instrument Biomarker Validation (Main Implementation Example) S4.2.1 Problem Definition and Limitations of Existing Technology Independent cross-institutional validation of biomarkers is a necessary prerequisite for their clinical application. Different research institutions typically use different brands or models of instruments for data acquisition, resulting in systematic shifts in the mass-to-charge ratio axis and resolution differences in the data from different institutions. The same biomarker corresponds to different channel indices in the data from different instruments, making direct comparison in the raw data space impossible. Existing methods attempt to eliminate instrument differences through preprocessing operations such as mass-to-charge ratio axis alignment, but these rely on manual parameter adjustments, leading to unstable results and complete failure when the mass-to-charge ratio range varies significantly. This application scenario leverages the instrument independence of the pedestal model feature space to achieve cross-institutional biomarker validation without performing any mass-to-charge ratio axis alignment operations.

[0126] S3.2.2 Implementation Process Feature extraction: Institution A and Institution B respectively perform preprocessing and feature extraction on their respective mass spectrometry imaging data according to the process described in S3.1, and each obtains a feature tensor. and Both institutions use the same base model weights. Set to the same dimension.

[0127] Candidate biomarker region specificity score calculation: For candidate biomarkers, their region specificity score is defined as the cosine distance between the mean pixel feature vector within the region of interest (e.g., tumor region) and the mean pixel feature vector within the background region. A larger cosine distance indicates a more significant difference in chemical characteristics between the region of interest and the background region. , in and These are the spatial mean values ​​of the pixel feature vectors of the region of interest and the background region, respectively. The region boundaries are determined by simple region labeling or existing tissue segmentation results. The same method is used to calculate these values ​​on the organization B data. .

[0128] Cross-agency conformity assessment: if If the score is below a preset consistency threshold (0.05 for example), the biomarker is considered to have reproducible regional specificity in the data from both institutions, and the validation is successful. For multiple candidate biomarkers, a cross-institutional consistency score list is output, sorted from highest to lowest consistency, to provide researchers with a reference for biomarker priority.

[0129] This method does not require the mass-to-charge ratio axes of the two institutions to be aligned, does not require the same number of channels or mass-to-charge ratio coverage, and does not require new labeled data. It is applicable to the multicenter biomarker concordance assessment between any two or more institutions.

[0130] S4.3 Application Scenario 2: Metabolic Pathway Enrichment Analysis S4.3.1 Problem Definition and Limitations of Existing Technology Metabolic pathway analysis is one of the core requirements for interpreting mass spectrometry imaging data. Existing methods require molecular annotation of channels by mass-to-charge ratio before performing pathway enrichment analysis. However, only 10% to 30% of channels in typical mass spectrometry imaging data can be reliably annotated, forcing the discarding of chemical information carried by a large number of unannotated channels, which severely limits the coverage of pathway analysis. This application utilizes the inter-channel chemical co-occurrence relationships implicitly learned during the pre-training phase of the base model (metabolites of the same pathway tend to co-locate in tissues, thus their features are naturally correlated during pre-training) to directly predict metabolic pathway activity from pixel feature vectors, skipping the time-consuming step of mass-to-charge ratio annotation.

[0131] S4.3.2 Training of the prediction head for weakly supervised pathways Weakly supervised signal source: Using the known metabolite-pathway associations in the KEGG metabolic pathway database, combined with a small number of mass-to-charge ratio channels already annotated in the mass spectrometry imaging data, a weakly supervised training signal is constructed: For each annotated pixel, the pathway label vector of the pixel is determined according to the pathway to which its annotated metabolite belongs. Unannotated pixels do not participate in supervised training.

[0132] Task Header Design: In Feature Tensors The lightweight path prediction head is connected to the upper layer and consists of two fully connected layers. The input dimension is... The output dimension is (Example value is 50), the activity probability value of each pathway is output using the Sigmoid activation function, supporting multi-label prediction. The pathway labels of annotated pixels are used as supervision signals, and the pathway prediction head is trained using binary cross-entropy loss, while the base model encoder parameters are kept frozen.

[0133] S4.3.3 Inference Output The pathway prediction head outputs a probability map of pathway activity for the entire slice. The spatial distribution of activity for each pathway is taken to obtain a heatmap of activity for each metabolic pathway on the tissue slice. This visualizes the differences in activity of different pathways in different tissue regions, providing quantitative evidence at the spatial resolution level for metabolic mechanism research.

[0134] S4.4 Application Scenario 3: Unsupervised Fostering of Organizational Microenvironments S4.4.1 Problem Definition and Limitations of Existing Technology Precise typing of the tumor immune microenvironment is of significant clinical value for guiding immunotherapy regimens and predicting treatment response. Existing microenvironment analysis methods based on mass spectrometry imaging typically rely on supervised classification frameworks, requiring pre-defined typing categories and extensive labeled data. The typing results are limited by manually defined categories, making it difficult to discover novel data-driven metabolic subtypes. This application scenario performs unsupervised metabolic microenvironment typing without relying on any labeled data and uses immunohistochemical staining data for post-hoc semantic mapping to achieve a biological interpretation of the typing results.

[0135] S4.4.2 Unsupervised Clustering Classification Process Feature extraction and dimensionality reduction: Feature tensors were extracted from tumor tissue mass spectrometry imaging data according to the process described in S4.1. flattened pixel feature matrix ( ), perform principal component analysis (PCA) to reduce the dimension to 50 (example value).

[0136] Clustering and stability assessment: K-means clustering algorithm or hierarchical clustering algorithm is used to... Each pixel is divided into The clustering results are evaluated using the silhouette score (example values ​​range from 4 to 8) to assess the stability of the clustering results under different values. The value that maximizes the silhouette score is selected as the final number of clusters. The clustering results are then reorganized into... The spatial fractal diagram, where each pixel corresponds to an integer category label.

[0137] Biological Interpretation of S4.4.3 Subtyping Results Post-immunohistochemical correlation: Immunohistochemical staining with CD8 (a cytotoxic T cell marker) and CD68 (a macrophage marker) is performed on the same tissue section to obtain spatial distribution maps of various immune cell types. The Pearson correlation coefficient between the spatial distribution of each cluster category and the spatial distribution of each immune cell marker is calculated. If the correlation coefficient between a certain category and a certain marker exceeds a preset threshold (0.6 for example), the category is labeled as the microenvironment subtype enriched by the corresponding cell type.

[0138] Feature metabolic pattern extraction: For each cluster category, calculate the difference between the mean of all pixel feature vectors in that category and the mean vectors of other categories to obtain the difference feature vectors of each category; backproject the difference feature vectors to the mass-to-charge ratio space through a pre-trained decoder to extract candidate mass-to-charge ratio channels with high specific expression, identify the corresponding molecules by comparison with the database, and output a list of feature metabolites of each microenvironment subtype.

[0139] S4.4.4 Output The final output of this application scenario includes: First, The system includes: first, pixel-level metabolic microenvironment typing maps; second, biological semantic annotations for each typing category, including immune cell enrichment types and a list of characteristic metabolites; and third, feature difference heatmaps between each typing category, which researchers can use for subsequent target validation and mechanism studies.

[0140] S4.5 Cross-Scenario General Instructions All three application scenarios mentioned above inherit the instrument-independent characteristics and few-sample transfer capabilities of the base model. When downstream task data comes from novel instruments not used in the pre-training stage, as long as the instrument data can output a standardized peak sequence and provide a valid meta-information condition vector through the preprocessing flow in step S1, the base model can directly perform feature extraction without retraining or fine-tuning the encoder for the new instrument. For supervised downstream tasks, at least five labeled samples are required for effective transfer; for unsupervised downstream tasks, meaningful analysis results can be output without any labeled data. Inference Stage Selection recommendations: When computational resources are limited or labeled data is scarce, it is recommended to choose a smaller value (e.g., 128); when high feature semantic richness is required, it is recommended to choose a larger value (e.g., 512). Default recommendation. =256, achieving a good balance between feature representation capability and computational cost.

[0141] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A self-supervised representation learning method for heterogeneous mass spectrometry data, characterized in that, Includes the following steps: Step 1: Obtain at least one mass spectrometry imaging dataset, collect the metadata in the mass spectrometry imaging dataset and encode it into a conditional embedding vector, and perform peak detection and preprocessing on the original intensity vector of each pixel to obtain a variable-length peak pair list for each pixel. Step 2: Construct a base model to obtain peak embedding vectors by mapping a variable-length list of peak pairs as input through a peak embedding layer. All peak embedding vectors of the same pixel form a peak term matrix. Using the latent term matrix as the query and the peak term matrix as the key, the pixel latent representation is obtained by encoding through a Perceiver encoder, and the conditional embedding vector is injected into the Perceiver encoder. After superimposing spatial position encoding on the pixel latent representation of each pixel, it is input into a spatial Transformer, and spatial context between pixels is learned through self-attention, outputting a spatially enhanced pixel representation. The spatially enhanced pixels are decoded by a PerceiverIO decoder to obtain the feature vector of each pixel, and the feature vectors of all pixels constitute a feature tensor. Step 3: Jointly train the base model using the masked pixel reconstruction task, the masked peak reconstruction task, and the cross-sample contrastive learning task. The masked pixel reconstruction task includes: randomly masking a portion of pixels in a spatial block; not inputting the peak list of the masked pixels into the Perceiver encoder; reconstructing the normalized intensity values ​​of the masked pixels using the encoded features of visible pixels through a cross-attention decoder; and establishing a masked pixel reconstruction loss that includes mean square error and spectral angle similarity loss. The masked peak reconstruction task includes: randomly masking the intensity values ​​of a portion of the peaks in the peak list of each pixel, retaining... The mass-to-charge ratio position of the masked peak is encoded to predict the intensity value of the masked peak, and a mean squared error loss is established as the masked peak reconstruction loss. The cross-sample contrastive learning task includes: using pixel pairs from the same tissue type or the same mass spectrometry imaging dataset as positive sample pairs and pixel pairs from different tissue types or different mass spectrometry imaging datasets as negative sample pairs, and establishing an InfoNCE loss; a total loss is established based on the masked pixel reconstruction loss, the masked peak reconstruction loss, and the InfoNCE loss, which is used to jointly train the pedestal model, update the parameters of the pedestal model, and obtain the trained pedestal model. Step 4: For the newly acquired mass spectrometry imaging data, after processing in Step 1, the corresponding variable-length peak pair list and conditional embedding vector are obtained. The trained base model parameters are loaded, and forward inference is performed to obtain the feature tensor for downstream analysis tasks.

2. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, The metadata in step 1 includes at least the instrument type, ion mode, mass-to-charge ratio detection range, and spatial resolution.

3. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, In step 2, the peak embedding layer applies Fourier position coding to the mass-to-charge ratio of each peak, and after applying a logarithmic transformation to the normalized intensity value, it obtains the intensity coding through linear projection. The Fourier position coding and intensity coding are concatenated and then output as peak embedding vectors through linear projection and layer normalization. The temperature parameters of the Fourier position encoding are matched with the mass-to-charge ratio range to ensure that the Fourier position encoding has sufficient frequency resolution within the target range.

4. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, The Perceiver encoder in step 2 consists of multiple layers of alternating stacked cross-attention sublayers and self-attention sublayers. After each layer of cross-attention calculation, self-attention calculation and feedforward network processing are performed. In the layer normalization of each sublayer, the conditional embedding vector is injected using a conditional layer normalization mechanism to obtain the latent representation of the pixel.

5. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 4, characterized in that, In step 2, the Perceiver encoder processes the variable-length peak word sequences in batches during training as follows: The peak word sequences of all pixels in the same batch are concatenated along the sequence dimension into a single long sequence, and the cumulative sequence length vector is recorded to identify the boundaries of each original sequence in the concatenated sequence; during attention calculation, it is strictly restricted to be performed within the boundaries of each sequence, and attention interaction across samples is prohibited.

6. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, The spatial location encoding in step 2 is a two-dimensional sine and cosine location encoding. After the output of the Perceiver encoder and before the input of the spatial Transformer, the spatial location encoding is superimposed on the pixel latent representation of each pixel to achieve hierarchical decoupling of chemical information and spatial information. The spatial location encoding is obtained by concatenating the row coordinates and column coordinates of the pixel after mapping them with sine and cosine functions of different frequencies.

7. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 6, characterized in that, In step 2, the spatial Transformer employs a spatial block training strategy: During the training phase, a spatial block containing multiple pixels is randomly selected from each mass spectrometry imaging slice as the basic unit for a single training iteration. The potential representations of all pixels within the spatial block are arranged in spatial order to form a sequence input space Transformer. During the inference phase, a sliding window approach is used to process the pixels of the entire slice block by block, with a step size equal to or half the window size. The features output by each window are then concatenated to form the feature tensor of the complete slice.

8. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, The output dimension of the PerceiverIO decoder is fixed to a preset value during the training phase; During the inference phase, the user dynamically specifies the output dimension, constructs the output query vector corresponding to the number of output dimensions, interpolates and initializes the output query vector from the training phase, and then obtains the feature vector through cross-attention decoding. The feature vectors of all pixels constitute the feature tensor, eliminating the need to retrain the base model.

9. The self-supervised representation learning method for heterogeneous mass spectrometry data according to claim 1, characterized in that, The downstream analysis tasks described in step 4 include cross-instrument biomarker validation, metabolic pathway enrichment analysis, and unsupervised typing of the tissue microenvironment.

Images