Mass spectrum data high-throughput alignment and parallel qualitative method based on deep characterization learning

Abstract

The invention discloses a mass spectrum data high-throughput alignment and parallel qualitative method based on deep characterization learning, and relates to the field of chromatography-mass spectrometry data processing and calculation mass spectrography. The method comprises the following steps: performing streaming, micro-batch and single traversal analysis on original mass spectrum data files of a plurality of samples to be detected; extracting metadata and main data by taking the fragmented spectrogram as a unit, inputting the metadata and the main data into a pre-trained depth representation learning model to generate a high-dimensional embedded vector, and incrementally writing the metadata and the main data into a column type storage database; constructing a sparse approximate nearest neighbor graph; executing a clustering algorithm on the graph to generate a consensus spectrogram set, and generating a consensus feature vector for each clustering cluster and a single example cluster; determining the nature of the consensus feature vector; and constructing a sample * feature matrix based on the cluster affiliation relationship and the qualitative result. According to the method, the technical problems of I / O bottleneck, memory overflow, combinatorial explosion, computing power waste and the like in large-scale mass spectrum data analysis are effectively solved.

Description

A High-Throughput Alignment and Parallel Qualitative Method for Mass Spectrometry Data Based on Deep Representation Learning Technical Field

[0001] This application relates to the field of chromatography-mass spectrometry coupled data processing and computational mass spectrometry, and in particular to a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep characterization learning. Background Technology

[0002] With the widespread application of mass spectrometry technology in life sciences, environmental monitoring, food safety and other fields, the scale of raw data generated by chromatography-mass spectrometry (including LC-MS / MS and GC-MS) experiments has grown exponentially. In industrial-scale analyses involving tens of thousands of samples, the amount of data generated often reaches the TB to PB level, and contains hundreds of millions of spectra with structural fingerprint information (such as MS / MS secondary spectra or EI source fragment spectra).

[0003] Common raw mass spectrometry data files (such as .mzML, .raw, .d, etc.) typically feature high compression ratios, variable-length peak lists, and hierarchical storage. Current technologies for processing such large-scale, high-throughput data generally employ a two-stage processing paradigm: first, converting all raw files to an intermediate format or extracting peak lists, loading them into memory, and then performing cross-sample peak alignment, feature extraction, and qualitative analysis.

[0004] However, faced with the ever-increasing sample size and complex analytical matrices, the aforementioned traditional processing paradigms reveal the following significant shortcomings in computational architecture and algorithm efficiency: "I / O wall" bottlenecks and enormous overhead from repeated parsing: In the traditional two-stage architecture, the raw data needs to undergo a process of "reading-decoding-reassembly-writing to an intermediate file," and subsequent analysis stages require rereading the intermediate file or even retracing the original data. This multi-pass mode leads to huge disk I / O overhead. Furthermore, since mass spectrometry data parsing (especially XML parsing and Base64 decoding) is a CPU-intensive task, repeated parsing severely limits data throughput.

[0005] Memory resource contention and scalability limitations: Most existing algorithms assume that all feature tables or vector matrices to be aligned can be fully loaded into memory. When the sample size reaches tens of thousands, the matrix constructed from the full dataset can cause unpredictable spikes in memory / GPU memory usage, easily leading to OutOfMemory (OOM) errors. Furthermore, traditional alignment algorithms based on dynamic programming or geometric matching (such as RANSAC and LOESS) struggle to effectively utilize modern distributed computing clusters or GPU acceleration when processing samples across batches, often falling into the performance bottleneck of single-machine serial processing.

[0006] Alignment computational complexity exhibits a combinatorial explosion trend: chromatographic retention times (RTs) can drift systematically or nonlinearly across samples from different batches and with varying matrices. Furthermore, GC-MS calibration calculations based on retention indices (RIs) face efficiency challenges with massive sample volumes. Without effective semantic representation and index pruning mechanisms, traditional alignment algorithms require full pairwise comparisons between samples, resulting in computational complexity that quadratically increases with the number of samples (O(N)). 2 The growth rate, and even higher, makes joint analysis of large-scale cohorts unacceptable in terms of time cost.

[0007] Redundant computation and limitations of the qualitative framework lead to inefficiency: In cross-sample scenarios, the same chemical substance generates a large number of highly similar redundant spectra in different samples. Existing processes typically initiate a spectral library search request independently for each spectrum in each sample, a "brute-force search" strategy that wastes enormous computational resources. Furthermore, different ionization sources (such as soft ionization in ESI and hard ionization in EI) produce spectral features that differ significantly. Existing methods are usually optimized for specific sources, lacking a unified, general qualitative framework based on deep semantic representation of spectra to simultaneously address the qualitative challenges of LC-M / MS and GC-MS data.

[0008] In summary, there is an urgent need for a high-throughput alignment and parallel qualitative analysis technology for large-scale mass spectrometry data that breaks down physical file boundaries, enables one-time streaming parsing, has low memory footprint, linear scalability, and is universally applicable to LC-MS / MS and GC-MS platforms. Summary of the Invention

[0009] The purpose of this application is to provide a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning. It aims to achieve parallel processing of large-scale samples under resource-constrained conditions by performing a single traversal and synchronous columnarization of the original mass spectrometry data files / spectral data streams to disk. Furthermore, it constructs a sparse nearest neighbor graph for cluster alignment through a triple constraint of "precursor / master fragment - retention time / exponent - embedding vector". It further significantly reduces the number of spectral library retrieval requests by using a cluster-level two-stage spectral library matching strategy, thereby improving the end-to-end analysis throughput and system stability.

[0010] To achieve the above objectives, this application provides the following solution: This application provides a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning, comprising: acquiring raw mass spectrometry data of multiple samples to be tested; performing stream cytometry, micro-batch, and single-pass parsing on the raw mass spectrometry data; using fragmented spectra as processing units, extracting metadata and master data, including sample identifiers, retention times or retention indices, precursor ion characteristics, and a list of spectral peaks, from the parsed raw mass spectrometry data; inputting the list of spectral peaks into a pre-trained deep learning representation model to generate a high-dimensional embedding vector corresponding to the list of spectral peaks in real time; and inputting the metadata... The high-dimensional embedding vector is used as an associated field and incrementally written to the columnar storage database. A mapping relationship between sample file identifiers and global row indexes is established to form a global data view across samples. Based on the columnar storage database, the persistent high-dimensional embedding vectors and metadata are read. Based on the persistent high-dimensional embedding vectors and metadata, each spectrum is used as a query node. Preset precursor ion feature tolerance and retention time or retention index window are applied globally as hard constraints to filter and obtain a candidate neighbor set. The similarity of high-dimensional embedding vectors is calculated within the candidate neighbor set, and only the top-N nearest neighbors with the highest similarity are retained, thus constructing... A sparse approximate nearest neighbor graph is constructed; a clustering algorithm is performed on the sparse approximate nearest neighbor graph to divide the closely connected spectrum sets that satisfy retention time or retention exponential drift constraints into the same consensus spectrum set; the differential substances are defined as singleton clusters, and each consensus spectrum set and singleton cluster is collectively referred to as a feature cluster; each feature cluster is an independent chemical feature and is regarded as a potential chemical substance for subsequent qualitative analysis; the mean or weighted center of all spectrum embedding vectors in each feature cluster is calculated to generate the consensus feature vector corresponding to that feature cluster; the consensus feature vector is searched against a pre-vectorized reference spectral library to obtain the top K candidates with the highest similarity. Chemical substances; obtain representative original fragmented spectra of the feature clusters, and perform correlation calculation based on signal intensity with the original reference spectra of the top K candidate chemical substances to obtain correlation scores; reorder or filter based on the correlation scores to output the final qualitative results of the chemical substances; perform independent spectral library search matching or signal-to-noise ratio filtering thresholds on the single clusters for qualitative analysis to obtain discrete qualitative results; integrate the final qualitative results of the chemical substances and the discrete qualitative results to obtain integrated qualitative results; based on the integrated qualitative results, construct a large-scale sample-metabolite feature matrix and fill it with the corresponding ion intensity or peak area data.

[0011] Optionally, the high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning further includes, between the steps "performing independent spectral library search matching or signal-to-noise ratio filtering thresholds for the single clusters to obtain discrete qualitative results" and "integrating the final chemical substance qualitative results and discrete qualitative results to obtain integrated qualitative results": assigning a unique unknown identifier to feature clusters that do not match a reference spectral library; and after the step "constructing a large-scale sample-metabolite feature matrix based on the integrated qualitative results and filling in the corresponding ion intensity or peak area data", the high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning further includes: extracting the MS1 peak area from the original data based on consensus features for missing value imputation.

[0012] Optionally, the streaming, micro-batch, and single-pass parsing adopts a single-scan strategy. During the single pass of the original mass spectrometry data, file reading, metadata extraction, embedded vector inference, and columnar incremental writing are completed simultaneously, without performing repeated disk I / O read operations.

[0013] Optionally, the columnar storage database adopts a binary columnar file format that supports append-only writing and accesses data through a memory mapping mechanism; write operations are performed in configurable batches of records.

[0014] Optionally, the streaming, micro-batch, and single-pass parsing of the raw mass spectrometry data is specifically performed using a multi-process parallel approach. The multi-process parallel approach employs a coarse-grained strategy: multiple raw mass spectrometry data files are processed in parallel at the file level, and embedding vectors are generated in parallel at the spectral level using a micro-batch approach. A producer-consumer model-based processing pipeline is constructed between parsing, vector generation, and writing, and cascaded backpressure control is achieved using a bounded buffer queue.

[0015] Optionally, when generating high-dimensional embedding vectors, the processing pipeline of the producer-consumer model monitors the GPU memory or RAM usage of computing nodes in real time and dynamically and adaptively adjusts the size of the micro-batch between a preset upper and lower limit.

[0016] Optionally, inputting the spectral peak list into a pre-trained deep learning representation model to generate a high-dimensional embedding vector corresponding to the spectral peak list in real time specifically includes: capturing the dependencies between fragment ions in the spectrum through a multi-head self-attention mechanism to output a high-dimensional embedding vector; the deep representation learning model is a neural network model based on the Transformer architecture, such as Dreams.

[0017] Optionally, the specific application of the precursor ion characteristic tolerance and retention time or retention index window is as follows: for liquid chromatography-mass spectrometry (LC-MS) data, the precursor mass-to-charge ratio tolerance and retention time window are jointly constrained; for gas chromatography-mass spectrometry (GC-MS) data, the retention index window is jointly constrained; the sparse approximate nearest neighbor graph is constructed through an inverted index, Locally Sensitive Hash (LSH), or Hierarchical Navigation Small World Graph (HNSW) index structure.

[0018] Optionally, the clustering algorithm uses the density-based clustering algorithm HDBSCAN, which is run on the sparse approximate nearest neighbor graph. The consensus feature vector is calculated by: weighting the embedding vectors of all spectra within the cluster, with the weights being positively correlated with the total ion current intensity or signal-to-noise ratio of the spectra, or by calculating the geometric median of the embedding vectors within the cluster.

[0019] Optionally, after performing independent spectral library search matching or signal-to-noise ratio filtering threshold on the single cluster, qualitative analysis is performed to obtain discrete qualitative results. Specifically, this includes: determining whether the quality score or precursor ion intensity of the original fragmented spectrum corresponding to the single cluster is higher than a preset noise filtering threshold; if it is higher than the threshold, the single cluster is retained as a sample-specific metabolite feature to participate in matrix construction; if it is lower than the threshold, it is marked as background noise and removed.

[0020] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning, which has the following significant effects: Breaking through the "I / O wall" bottleneck and significantly improving data preprocessing efficiency: Existing technologies generally adopt a two-stage process of "file conversion first, then loading and analysis," resulting in repeated reading and writing of raw data, leading to huge I / O overhead. This application, through a single-pass strategy, simultaneously completes streaming, micro-batch, single-pass parsing, embedding generation, and incremental columnar disk writing during a single traversal of the raw file. Combined with memory mapping and zero-copy technology, this application eliminates the redundant intermediate file reading and writing process in the traditional process, improving the data preprocessing speed by several times. Especially when processing TB-level massive data, the I / O efficiency advantage is particularly obvious.

[0021] To address the combinatorial explosion problem and achieve fast alignment with linear expansion, traditional alignment algorithms (such as full pairwise comparison) suffer from computational complexity that increases quadratically (O(N²)) with the sample size, making them unsuitable for queues of tens of thousands or more samples. This application utilizes a triple constraint of "precursor / master fragment - retention time - embedding vector" to construct a sparse approximate nearest neighbor graph, transforming the complex global alignment problem into a local neighborhood search on the sparse graph. This method successfully avoids full computation, reducing the growth trend of alignment complexity from quadratic to near linear, making it possible to complete cross-sample alignment of tens of millions of spectral graphs within a finite time.

[0022] Resource consumption is controllable, avoiding OutOfMemory (OOM) errors in large-scale analysis: Existing tools typically require loading the entire feature table into memory, which can easily lead to system crashes due to a surge in data volume. This application employs streaming, micro-batch, single-pass parsing, and incremental columnar writing, with data processing and disk writing occurring simultaneously. Only the currently processed micro-batch data resides in memory. Combined with backpressure control based on the producer-consumer model and an adaptive micro-batch strategy for GPU memory, it ensures that the system's memory / GPU memory peak values ​​remain within safe thresholds. This means that users can stably complete ultra-large-scale queue analysis tasks that previously required large clusters on conventional servers or even high-performance workstations.

[0023] This significantly reduces computational redundancy in spectral library retrieval and improves qualitative throughput: Addressing the problem of numerous redundant spectra for the same substance across samples, this application innovatively proposes a "clustering-then-qualification" strategy. By generating consensus feature representations, the request unit for spectral library retrieval is reduced from "each original spectrum" to "each feature cluster," leveraging data redundancy to reduce the computational load by several orders of magnitude. Simultaneously, a two-stage cascaded strategy of "vector initial screening + original spectrum verification" ensures a significant leap in qualitative speed while maintaining high accuracy of the qualitative results through verification using the original signal intensity.

[0024] Improving the quality and completeness of feature matrix construction: Addressing common issues like noise interference and missing values ​​in large-scale datasets, this application employs rigorous signal-to-noise ratio (SNR) filtering on single-instance clusters to effectively eliminate background noise and low-quality fragments, ensuring the purity of the feature matrix. Furthermore, leveraging the efficient indexing capabilities of columnar storage, missing values ​​are imputed by extracting the MS1 peak area from the original data based on consensus features, significantly reducing the missing rate of the final metabolite feature matrix and providing a more complete and reliable data foundation for subsequent statistical analysis. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 is a flowchart illustrating a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning, according to an embodiment of this application; Figure 2 is an overall framework diagram illustrating a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning, according to an embodiment of this application; Figure 3 is a schematic diagram illustrating the principle of single scan and parallel processing, according to an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] In an exemplary embodiment, as shown in Figures 1 and 2, a high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning is provided. This method is executed by a computer device, specifically by a computer device such as a terminal or server alone, or by a terminal and server together. In this embodiment, the method includes the following steps: Step 101: Obtain the raw mass spectrometry data of multiple samples to be tested.

[0030] The system receives a set of raw mass spectrometry data files from multiple biological samples as input. These files can be non-targeted metabolomics data from an LC-MS system or volatile organic compound analysis data from a GC-MS system. In this embodiment, the sample to be tested consists of small metabolic molecules.

[0031] Step 102: Perform flow cytometry, micro-batch, and single-pass analysis on the raw mass spectrometry data.

[0032] Step 103: Using fragmented spectra as processing units, extract metadata and master data, including sample identifiers, retention times or retention indices, precursor ion characteristics, and spectral peak lists, from the raw mass spectrometry data after flow cytometry, micro-batch, and single-pass analysis.

[0033] The system employs a file-level coarse-grained parallel strategy, assigning different raw data files to different processing processes or threads.

[0034] Streaming Parsing involves each processing unit performing a streaming, micro-batch, single-pass parsing scan on the assigned raw data file. See Figure 3 for details. Figure 3 is a schematic diagram of the principle of single scan and parallel processing.

[0035] The parser uses "fragment spectra" as the basic processing unit.

[0036] For LC-MS data: the basic processing unit is usually a second-order mass spectrometer (MS2 Spectrum) in data-dependent acquisition (DDA) mode; for GC-MS data: the basic processing unit can be a full-scan mass spectrum generated by electron bombardment (EI) ionization, which is characteristically equivalent to a fragmentation spectrum.

[0037] Metadata is extracted spectrally by spectrally during the analysis process, including: sample ID, chromatographic retention information (retention time, RT or retention index, RI), and precursor information (precursor m / z and charge state in LC-MS; or characteristic ion / molecular ion mass in GC-MS). Simultaneously, a list of mass-to-charge ratios and intensities for all fragment ions in the spectrum is extracted.

[0038] Step 104: Input the spectral peak list into the pre-trained deep learning representation model to generate a high-dimensional embedding vector corresponding to the spectral peak list in real time.

[0039] To avoid excessive CPU-GPU scheduling overhead caused by calling the deep learning model for inference on each spectrum individually, this implementation introduces a micro-batch processing mechanism. The system sets up a micro-batch buffer to package the peak information of multiple spectra obtained from streaming parsing.

[0040] Deep representation learning models are neural network models based on the Transformer architecture, such as DreaMS (BUSHUIEVR, BUSHUIEV A, SAMUSEVICH R, et al. Self-supervised learning of molecular representations from millions of tandem mass spectra using DreaMS[J]. NatureBiotechnology, 2025. DOI: 10.1038 / s41587-025-02663-3.), which includes a self-attention encoder.

[0041] Pre-trained deep representation learning models (such as Transformer-based encoders) are designed to be compatible with both MS2 spectra generated by ESI and GC-MS spectra generated by EI. For GC-MS data, the model can learn fingerprint spectral features specific to EI sources; for LC-MS / MS data, the model can capture fragmentation patterns generated by collision-induced dissociation (CID / HCD).

[0042] Deep representation learning models encode the input peak sequence and output the corresponding high-dimensional embedding vector.

[0043] This embodiment also includes an adaptive adjustment mechanism, whereby the system monitors the GPU memory or RAM usage of the current computing node in real time. When resources are sufficient, the batch size is appropriately increased to improve throughput; when resources are scarce, the batch size is automatically reduced to prevent memory overflow (OOM), thereby achieving a dynamic balance between computing throughput and resource usage.

[0044] Step 105: The metadata and high-dimensional embedding vector are incrementally written into the columnar storage database as associated fields, and a mapping relationship between sample file identifiers and global row indexes is established to form a global data view across samples.

[0045] The system uses columnar storage formats (such as Lance, Parquet, Arrow, etc.) as intermediate storage media. For each processed spectrum, its metadata (sample ID, RT / RI, m / z), peak information (serialized binary), and the generated high-dimensional embedding vector are added incrementally to the columnar storage dataset as different columns of the same record in an append manner.

[0046] At the same time, the system maintains a global mapping table to establish a mapping relationship of "sample file identifier - global row index" and form a global virtual data table across samples.

[0047] Steps 101-105 above construct a producer-consumer pipeline that includes a parsing phase, an embedding generation phase, and a columnar writing phase. It is a double-buffered queue, wherein: the first queue connects the parsing thread (producer) and the model inference thread (consumer) and is used to pass the spectral data to be vectorized.

[0048] The second queue connects the model inference thread (producer) and the disk write thread (consumer) to transmit vectorized result data.

[0049] Backpressure control mechanism: Both queues are configured as bounded blocking queues with a capacity limit. Through a cascading blocking mechanism, the system automatically adjusts the upstream parsing rate based on the slowest step (usually model inference or disk I / O) to ensure memory safety.

[0050] Step 106: Based on the columnar storage database, read the persistent high-dimensional embedding vectors and metadata.

[0051] Step 107: Based on the persistent high-dimensional embedding vector and metadata, using each spectrum as a query node, apply the preset precursor ion feature tolerance and retention time or retention index window in the global scope to filter and obtain the candidate neighbor set.

[0052] Step 108: Calculate the similarity of high-dimensional embedding vectors within the candidate neighbor set, and retain only the top-N nearest neighbor relationships with the highest similarity, thereby constructing a sparse approximate nearest neighbor graph.

[0053] Steps 106-108 above describe how to construct a sparse approximate nearest neighbor graph for alignment, the core of which is to greatly prune the search space according to different chromatographic types.

[0054] Binning and initial screening: Based on the global virtual data table that has been placed in the tray, the system first performs coarse screening.

[0055] LC-MS mode: mainly based on the precursor mass-to-charge ratio (Precursor m / z) for binning, and set a tolerance window (e.g. ±10 ppm).

[0056] GC-MS mode: Since EI sources may lack molecular ion peaks, the system can segment based on retention time (RT) or nominal mass; or directly rely on subsequent vector indexes for retrieval.

[0057] Vector index retrieval: Within each bucket (or globally), an efficient retrieval structure (such as HNSW or IVF) is constructed using the generated embedding vectors. For each query spectrum, this structure is used to quickly obtain its approximate nearest neighbor candidate set in the vector space. This is the core constraint based on the similarity of material structures (i.e., spectral similarity).

[0058] Chromatographic retention constraint pruning: Apply a chromatographic retention window as a hard constraint while or after obtaining the candidate set.

[0059] For LC-MS data: retention time (RT) is typically used to directly remove spectral pairs whose absolute difference in retention time exceeds a threshold (e.g., ±30 seconds).

[0060] For GC-MS data: the Retention Index (RI) is preferred. The system reads the internal standards (such as n-alkane mixtures) from the samples, calculates the RI value for each spectrum, and removes candidate neighbors whose RI difference exceeds a threshold (e.g., ±20 units). This effectively solves the retention time drift problem caused by column efficiency differences in GC-MS.

[0061] Sparse graph construction: After the above constraints and filtering, the system retains only the top-N neighbors with the highest similarity for each spectral graph, and constructs an extremely sparse approximate nearest neighbor graph across the entire graph.

[0062] Step 109: Perform a clustering algorithm on the sparse approximate nearest neighbor graph to divide the set of closely connected spectra that satisfy retention time or retention exponential drift constraints into the same consensus spectra set.

[0063] Step 110: Define the differential substances as singleton clusters, and refer to each consensus spectrum set and singleton cluster as a feature cluster; each feature cluster is an independent chemical substance.

[0064] Steps 109-110 above aim to describe how to achieve cross-sample alignment on sparse graphs through clustering.

[0065] Density clustering execution: The cluster alignment module directly runs a density-based clustering algorithm, preferably HDBSCAN, on the sparse nearest neighbor graph constructed in step 108. This algorithm does not rely on cluster shape assumptions and is applicable to metabolite features with various complex distributions.

[0066] Composite distance metric: Defines a distance metric between spectral nodes during clustering. This metric combines structural similarity extracted by deep learning with similarity in chromatographic behavior. ;in: The distance between the embedding vectors (e.g., cosine distance); To account for chromatographic differences, in LC-MS, Take the retention time difference ( In GC-MS, Take the difference in retention index ( ), These are the weighting coefficients.

[0067] This mechanism can effectively distinguish chemical substances that have highly similar fragmentation patterns (i.e., similar vectors) but different chromatographic behaviors (like isomers).

[0068] Step 111: Calculate the mean or weighted center of all spectral embedding vectors within each feature cluster to generate the consensus feature vector corresponding to that feature cluster.

[0069] Consensus Representation Generation: After clustering, each cluster is considered a potential chemical substance. The system generates the consensus feature vector (Consensus Embedding) and consensus retention information (Consensus RT / RI) for the cluster by weighted averaging or calculating the geometric median.

[0070] Step 112: Search the consensus feature vector with the pre-vectorized reference spectrum library to obtain the top K candidate chemical substances with the highest similarity.

[0071] Step 113: Obtain the representative original fragmented spectrum of the feature cluster and perform correlation calculation based on signal intensity with the original reference spectrum of the top K candidate chemical substances to obtain the correlation score.

[0072] Step 114: Reorder or filter based on the correlation scores to output the final qualitative results of the chemical substances.

[0073] Step 115: Perform independent spectral library search matching or signal-to-noise ratio filtering threshold for the single-instance cluster and then perform qualitative analysis to obtain discrete qualitative results.

[0074] Step 116: Integrate the final qualitative results of the chemical substances and the discrete qualitative results to obtain the integrated qualitative results.

[0075] Steps 112-116 above mainly explain how to use clustering results to accelerate the qualitative process and be compatible with different types of standard spectral libraries.

[0076] Reference Spectrum Preprocessing: The system preloads the standard reference spectrum library and uses a deep representation learning model to convert it into a reference vector library.

[0077] LC-MS scenario: Load libraries such as MassBank, MoNA, and Metlin that contain MS2 spectra.

[0078] GC-MS scenario: Load libraries such as NIST and Wiley that mainly contain EI source spectra.

[0079] Phase 1: Vector Top-K Initial Screening: For the consensus vector of each feature cluster, perform a fast search in the reference vector library to obtain the top K candidate chemical substances with the highest similarity.

[0080] Second stage: Original signal verification: Select representative original spectra from the feature clusters and perform correlation calculations based on signal intensity with the original reference spectra of the Top-K candidates.

[0081] Calculation methods include Pearson correlation coefficient, cosine similarity, or the dot product algorithm specifically optimized for GC-MS.

[0082] Candidates are reordered based on the review score. For GC-MS, the retention index (RI) matching score can also be introduced as an additional filtering condition.

[0083] Step 117: Based on the integrated qualitative results, construct a large-scale sample-metabolite feature matrix and fill it with the corresponding ion intensity or peak area data.

[0084] A sample-metabolite feature matrix is ​​constructed based on clustering and qualitative results.

[0085] A row corresponds to a feature cluster (i.e., a metabolite or chemical component).

[0086] Column: Corresponds to a sample identifier.

[0087] Value: Fills the ion intensity or peak area of ​​the sample in the corresponding cluster.

[0088] Backtracking-based Gap Filling: For zero values ​​in the feature matrix, the system performs backtracking filling: 1. Construct a retention time correction function: Using the successfully aligned features in the sample as anchors, fit a non-linear correction function (such as LOESS regression) from "consensus retention time" to "local retention time of the sample" to eliminate retention time drift between samples.

[0089] 2. Mapping Prediction Window: Read the consensus retention information (RT or RI) and feature quality (Precursor m / z or Quantifier Ion m / z) of the feature cluster, and use the above correction function to map them to the predicted local retention time of the sample.

[0090] 3. Signal backtracking extraction: LC-MS filling: Backtracking extraction of the peak area of ​​the first-order mass spectrometer (MS1) within the corresponding RT window and m / z range.

[0091] GC-MS filling: Within the corresponding RI window, backtrack the peak area of ​​the extracted ion chromatogram (EIC) of the quantifier ion.

[0092] This step effectively solves the problem of missed triggering (LC-MS) or missed detection caused by deconvolution failure in DDA mode, and significantly improves data integrity.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning, characterized in that, The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning includes: acquiring raw mass spectrometry data of multiple samples to be tested; performing flow cytometry, micro-batch, and single-pass parsing on the raw mass spectrometry data; using fragmented spectra as processing units, extracting metadata and master data, including sample identifiers, retention times or retention indices, precursor ion characteristics, and a spectrum peak list, from the parsed raw mass spectrometry data; inputting the spectrum peak list into a pre-trained deep learning representation model to generate a high-dimensional embedding vector corresponding to the spectrum peak list in real time; and incrementally writing the metadata and high-dimensional embedding vector as association fields into a columnar expression. The database is stored, and a mapping relationship between sample file identifiers and global row indexes is established to form a global data view across samples. Based on the columnar storage database, the persistent high-dimensional embedding vectors and metadata are read. Based on the persistent high-dimensional embedding vectors and metadata, each spectrum is used as a query node, and a preset precursor ion feature tolerance and retention time or retention index window are applied globally as hard constraints to filter and obtain a candidate neighbor set. The similarity of high-dimensional embedding vectors is calculated within the candidate neighbor set, and only the top-N nearest neighbors with the highest similarity are retained, thereby constructing a sparse approximate nearest neighbor graph. Clustering algorithms are performed on sparse approximate nearest neighbor graphs to divide sets of closely connected spectra that satisfy retention time or retention exponential drift constraints into the same consensus spectrum set. The differential substances are defined as singleton clusters, and each consensus spectrum set and singleton cluster is collectively referred to as a feature cluster. Each feature cluster is an independent chemical feature and is considered a potential chemical substance for subsequent qualitative analysis. The mean or weighted center of all spectral embedding vectors within each feature cluster is calculated to generate the consensus feature vector corresponding to that feature cluster. The consensus feature vector is then searched against a pre-vectorized reference spectral library to obtain the top K candidate chemical substances with the highest similarity ranking. A representative original fragmented spectrum of the feature cluster is taken and correlated with the original reference spectra of the top K candidate chemical substances based on signal intensity to obtain a correlation score; the correlation scores are then reordered or filtered to output the final qualitative result of the chemical substance; for each individual cluster, an independent spectral library search matching or signal-to-noise ratio filtering threshold is performed for qualitative analysis to obtain a discrete qualitative result; the final qualitative result of the chemical substance and the discrete qualitative result are integrated to obtain an integrated qualitative result; based on the integrated qualitative result, a large-scale sample-metabolite feature matrix is ​​constructed and filled with the corresponding ion intensity or peak area data.

2. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning further includes, between the steps "performing independent spectral library search matching or signal-to-noise ratio filtering thresholds for the single clusters to obtain discrete qualitative results" and "integrating the final chemical substance qualitative results and discrete qualitative results to obtain integrated qualitative results": assigning a unique unknown identifier to feature clusters that do not match a reference spectral library; and after the step "constructing a large-scale sample-metabolite feature matrix based on the integrated qualitative results and filling in the corresponding ion intensity or peak area data", the high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning further includes: extracting the MS1 peak area from the original data based on consensus features for missing value imputation.

3. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The streaming, micro-batch, and single-pass parsing methods employ a single-scan strategy. During the single pass of the raw mass spectrometry data, file reading, metadata extraction, embedded vector inference, and columnar incremental writing are completed simultaneously, without performing repeated disk I / O read operations.

4. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The columnar storage database uses a binary columnar file format that supports append-only writes and accesses data through a memory mapping mechanism; write operations are performed in configurable batches of records.

5. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The raw mass spectrometry data is parsed using streaming, micro-batch, and single-pass methods. Specifically, a multi-process parallel approach is used to parse the raw mass spectrometry data files using streaming, micro-batch, and single-pass methods. The multi-process parallel approach adopts a coarse-fine granularity strategy: multiple raw mass spectrometry data files are processed in parallel at the file level, and embedding vectors are generated in parallel at the spectrum level using a micro-batch-based approach. A producer-consumer model-based processing pipeline is built between parsing, vector generation, and writing, and cascaded backpressure control is implemented using bounded buffer queues.

6. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 5, characterized in that, The producer-consumer model's processing pipeline monitors the GPU or RAM usage of computing nodes in real time when generating high-dimensional embedding vectors, and dynamically and adaptively adjusts the size of the micro-batch between preset upper and lower limits.

7. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The process of inputting the spectrum peak list into a pre-trained deep learning representation model to generate a high-dimensional embedding vector corresponding to the spectrum peak list in real time specifically includes: capturing the dependencies between fragment ions in the spectrum through a multi-head self-attention mechanism to output a high-dimensional embedding vector; the deep representation learning model is a neural network model based on the Transformer architecture.

8. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The specific application of the precursor ion characteristic tolerance and retention time or retention index window is as follows: for liquid chromatography-mass spectrometry (LC-MS) data, the precursor mass-to-charge ratio tolerance and retention time window are jointly constrained; for gas chromatography-mass spectrometry (GC-MS) data, the retention index window is jointly constrained; the sparse approximate nearest neighbor graph is constructed through an inverted index, Locally Sensitive Hash (LSH), or Hierarchical Navigation Small World Graph (HNSW) index structure.

9. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, The clustering algorithm is a density-based clustering algorithm, which is run on the sparse approximate nearest neighbor graph. The consensus feature vector is calculated by: weighting the embedding vectors of all spectra within the cluster, with the weights being positively correlated with the total ion current intensity or signal-to-noise ratio of the spectra, or by calculating the geometric median of the embedding vectors within the cluster.

10. The high-throughput alignment and parallel qualitative method for mass spectrometry data based on deep representation learning according to claim 1, characterized in that, After performing independent spectral library search matching or signal-to-noise ratio filtering threshold on the single cluster, qualitative analysis is performed to obtain discrete qualitative results. Specifically, this includes: determining whether the quality score or precursor ion intensity of the original fragmented spectrum corresponding to the single cluster is higher than a preset noise filtering threshold; if it is higher than the threshold, the single cluster is retained as a sample-specific metabolite feature to participate in matrix construction; if it is lower than the threshold, it is marked as background noise and removed.

Images