Protein mass spectrum coding method based on self-supervised learning

Abstract

The invention discloses a protein mass spectrum coding method based on self-supervised learning, and the method comprises the steps: guaranteeing the consistency of model input through the preprocessing steps of data enhancement, normalization, sequence length standardization and the like; by constructing a self-supervised learning task and utilizing a multi-head attention mechanism of a Transform architecture, efficient extraction of global and local features of mass spectrum data is realized; through an autoregressive encoder training framework, the model can automatically learn the internal structure of mass spectrum data without a large amount of labeled data, and high-dimensional coding representation with robustness and generalization ability is generated. According to the method, the workload of manual feature design is remarkably reduced, and the generated coding representation can be applied to downstream tasks such as mass spectrum data quality evaluation, protein identification and quantification, post-translational modification identification and biomarker discovery, and has wide applicability and high efficiency; and an innovative intelligent solution is provided for proteomics research.

Description

Technical Field

[0001] The present invention relates to a method for extracting and encoding sequence features of mass spectrometry data by combining deep learning and proteomics, specifically a method for encoding protein mass spectrometry based on self-supervised learning, which can efficiently extract data features without labels and can be further used for quality assessment, protein identification and quantification, or other downstream tasks of protein mass spectrometry data. Background Art

[0002] As an important tool in proteomics research, mass spectrometry technology can provide data such as the molecular mass and fragment ion information of proteins, and can be used to analyze protein expression, protein composition, and protein interaction networks in biological samples. Existing mass spectrometry data analysis methods face the following challenges:

[0003] 1. High data complexity: Mass spectrometry data has the characteristics of high dimensionality and strong noise. Traditional methods rely on manual feature extraction, which is inefficient and vulnerable to subjective factors.

[0004] 2. Scarce labeled data: The acquisition cost of labeled data for protein mass spectrometry is high, and it is difficult for supervised learning methods to fully train the model.

[0005] 3. Poor generalization: Existing model structures are difficult to capture the detailed features of mass spectrometry data, resulting in poor generalization test results of the model. <�

[0006] At present, deep learning methods have made remarkable progress in fields such as images, text, and speech. Among them, self-supervised learning methods can automatically capture the internal structure of data without labels, greatly reducing the dependence on manual feature design. Moreover, the Transformer model has excellent sequence modeling capabilities and shows significant advantages in capturing long-distance dependencies and complex feature relationships. Combining self-supervised learning with the Transformer can fully extract the features in mass spectrometry data and obtain better performance in subsequent quality assessment tasks.

[0007] However, encoding methods for mass spectrometry data mostly focus on traditional signal processing or shallow neural network methods, and encoding methods based on the Transformer architecture have not been fully applied. It is necessary to propose a new self-supervised based mass spectrometry data encoding method to achieve efficient encoding of mass spectrometry data. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method for encoding mass spectrometry data based on self-supervised learning, which can achieve efficient encoding of mass spectrometry data, extract key features, and improve the robustness to noise.

[0009] The technical solution adopted by the present invention to solve the above technical problems is as follows: A self-supervised protein mass spectrometry encoding method, comprising the following steps:

[0010] S1: Data preprocessing, converting the original mass spectrometry data into a unified format, including data augmentation, normalization processing, and sequence length standardization, to ensure the consistency of the model input;

[0011] S2: Construct a self-supervised learning task, determine the loss function of the model, so that the model automatically learns the inherent hidden structure and features in the mass spectrometry data in a self-supervised state;

[0012] S3: Build a Transformer architecture encoder, construct a self-supervised model, and use the multi-head attention mechanism to extract the global and local features of the mass spectrometry data;

[0013] S4: Model training, optimize the model parameters through self-supervised training, obtain a high-dimensional encoding representation with strong robustness and excellent generalization performance, and save the obtained model weights to provide an accurate and stable feature representation for subsequent mass spectrometry data encoding.

[0014] Compared with the prior art, the advantages of the present invention are that the proposed protein mass spectrometry encoding method based on self-supervised learning makes full use of the multi-head attention mechanism of the Transformer structure, can automatically mine the inherent hidden structure and key information from the mass spectrometry data without a large amount of manual annotation, and reduces the workload of manual feature design; this method preprocesses the data by normalizing and standardizing the sequence length, realizes the standardized input of different mass spectrometry samples, and then constructs a self-supervised task of data reconstruction, so that the model captures the overall and local features of the data in the reconstruction task, and obtains a high-dimensional encoding representation with strong robustness and generalization ability; the autoregressive encoder training mode enables each processing module to be optimized collaboratively, reducing the information loss and error accumulation that may be brought by traditional staged processing; the generated encoding representation has wide applicability and can be directly used for downstream tasks such as the quality assessment of mass spectrometry data.

[0015] Furthermore, the specific method for data preprocessing in step S1 is as follows:

[0016] S1.1 Data reading: Read the mass spectrometry information item by item from the original MGF file, parse the mass-to-charge ratio (m / z value), mass spectrometry intensity data, and precursor ion mass in the mass spectrometry information, and store this information in a list;

[0017] S1.2 Data augmentation: Randomly augment the read mass spectrometry data to improve the generalization ability of the model. Randomly scale the intensity value with a probability of 50%, and the scaling factor is sampled from a uniform distribution [0.8, 1.2]. The formula is:

[0018] intensity = intensity × random(0.8, 1.2)

[0019] And add Gaussian noise to the m / z value with a probability of 30% to simulate the instrument calibration error. The noise follows a normal distribution N(0, 0.05), and the formula is

[0020] mz = mz + random(N(0, 0.05));

[0021] S1.3 Normalization: Use Z-score to standardize the m / z value to eliminate the dimension difference. The formula is:

[0022]

[0023] where ∈ = 10 -8 , μ mz and σ mz are the mean and variance of the m / z values of the current spectrum respectively; perform a logarithmic transformation on the intensity value to compress the intensity value range. The formula is:

[0024] norm intensity = log(1 + intensity);

[0025] S1.4 Unify the sequence length: Set the maximum sequence length to 100 to eliminate the difference in the number of data points of different mass spectra, and construct a unified input tensor format to adapt to the model requirements. If the number of data points of the mass spectrum exceeds 100, sort them in descending order of intensity and retain the first 100 high-intensity data points to ensure that key information is not lost; if the number of data points is less than 100, use edge padding to copy the value of the last data point to 100 to avoid boundary noise introduced by zero padding. After processing, each mass spectrum sample will be converted into a two-dimensional tensor with a shape of (100, 2), where 100 represents the number of data points, and 2 corresponds to the two feature dimensions of m / z and intensity values respectively.

[0026] Step S2 Construct a self-supervised learning task, and the specific method is as follows:

[0027] S2.1 Design a pre-training task: Use the preprocessed mass spectrometry data as the input and the target output to construct the data pair required for self-supervised learning. Through model training, the input data is reconstructed into an output as similar as possible to the original input, prompting the model to learn the internal structure and features of the mass spectrometry data;

[0028] S2.2 Construct a loss function: Define a combined loss function to comprehensively measure the numerical accuracy and spectral shape consistency of the reconstructed data. Use the mean squared error loss to calculate the mean squared error between the predicted value and the true value, and constrain the reconstruction accuracy at the numerical level:

[0029]

[0030] Among them, is the real spectrum data, is the predicted spectrum data reconstructed by the model, and N is the total number of data points of a single sample. Calculate the spectral similarity loss, and measure the overall shape consistency between the predicted spectrum and the real spectrum through cosine similarity. The formula is:

[0031]

[0032] Among them, B is the batch size, is the cosine similarity of the b-th sample. The final loss is the weighted sum of the two, and the weight is α, and the value range is from 0 to 1:

[0033] L total = α·L MSE +(1 - α)·L spectral

[0034] Step S3 builds a self-supervised learning model, and the specific method is as follows:

[0035] S3.1 Embedding layer: Map the input mass spectrometry data with a shape of (100, 2) to a 256-dimensional embedding space through a fully connected layer, so that it can accommodate more feature information, so that the model can learn richer and more expressive feature representations. The fully connected layer can be represented as a linear transformation:

[0036] y = x·W + b

[0037] Among them, x is the input two-dimensional mass spectrometry data, the weight matrix W has a shape of (2, 256), b is a 256-dimensional bias vector, and the output shape after being processed by the embedding layer is (100, 256).

[0038] S3.2 Stacking of Transformer encoder layers: After the embedding layer, stack more than 2 Transformer encoder layers to further extract the features of the data. Each Transformer encoder layer contains the following key components:

[0039] (1) Multi-head attention module: Use the multi-head attention mechanism to model the long-range dependencies between data points in the input sequence. The multi-head attention mechanism allows the model to parallelly focus on different parts of the input sequence in different representation subspaces, capturing richer semantic information. The specific process is that the multi-head attention mechanism projects the query, key, and value of the input sequence into multiple low-dimensional subspaces respectively, then calculates the attention scores in each subspace, and finally weights and sums and concatenates these scores to obtain the final output.

[0040] (2) Feed-Forward Neural Network (FFN): It consists of two fully connected layers, with a Gaussian Error Linear Unit activation function in the middle. The FNN can perform further feature transformation and combination on the output of the multi-head attention module, increasing the non-linear expression ability of the model.

[0041] (3) Residual Connection and Layer Normalization: A residual connection and a normalization layer are added after the multi-head attention module and the FFN respectively to improve the training stability and the model convergence speed. The residual connection allows the model to more easily learn the identity mapping during training, avoiding the vanishing gradient problem. Layer normalization normalizes the input, making the distribution of the input more stable and accelerating the convergence of the model.

[0042] S3.3 Reconstructing the Output Layer: After the feature extraction by multiple Transformer encoder layers, the model uses a two-layer decoder to map the 256-dimensional hidden state back to the original data dimension. The specific process is as follows: The first-layer decoding non-linearly expands the 256-dimensional features to 512 dimensions through a fully connected layer combined with a Gaussian Error Linear Unit activation function. The second-layer decoding uses a fully connected layer without an activation function to compress the 512-dimensional features to the original dimension, and the output shape is (100, 2), corresponding to the reconstructed m / z value and intensity value respectively. The output of this layer is the mass spectrometry data reconstructed by the model, which is used to calculate the self-supervised reconstruction error. By mapping the high-dimensional features back to the original data dimension, the model can learn how to reconstruct the original mass spectrometry data from the feature representation and capture the internal structure and features of the data.

[0043] Step S4 Model Training, the specific method is as follows:

[0044] S4.1 Training Process: The Adaptive Moment Estimation optimizer is used for parameter update. The initial learning rate is set to be less than 1×10 -4 , ensuring that the model can converge stably during training. A dynamic learning rate scheduler is introduced to monitor the training loss. If the loss does not decrease in three consecutive training epochs, the learning rate decays by a factor of 0.5. The data is batched in a way that the batch size is greater than 10, and the model is iteratively optimized for 30 or more training epochs;

[0045] S4.2 Weight Saving: When the training loss tends to converge or reaches the preset number of training rounds, save the trained model weight file for subsequent data encoding applications and fine-tuning of downstream tasks. Description of the Drawings

[0046] Figure 1 is a schematic flowchart of the self-supervised protein mass spectrometry encoding method of the present invention;

[0047] Figure 2 is a schematic structural diagram of the Transformer encoder of the present invention;

[0048] Figure 3 It is a schematic diagram of the original spectrum in the example of Embodiment 1 of the present invention;

[0049] Figure 4 It is a schematic diagram of the preprocessed spectrum in the example of Embodiment 1 of the present invention;

[0050] Figure 5 It is a schematic diagram of the reconstructed spectrum in the example of Embodiment 1 of the present invention;

[0051] Figure 6 It is a schematic diagram of the output of the embedding layer in the example of Embodiment 1 of the present invention. Detailed implementation manners

[0052] To further understand the content, features and effects of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and the following examples.

[0053] Embodiment 1: A self-supervised protein mass spectrometry encoding method includes the following steps:

[0054] S1: Input the MGF file and read the mass spectrometry information item by item, parse the m / z value, intensity data, and precursor ion mass in the mass spectrometry, and store them in a list. Perform data preprocessing on the original mass spectrometry data, including data augmentation, normalization processing, and unifying the sequence length, to adapt to the model input;

[0055] S2: Construct a self-supervised learning task, use the processed mass spectrometry data as the input and the corresponding target values, and output the reconstructed data to be as similar as possible to the original input, so that the model learns the internal structure and features of the mass spectrometry data.

[0056] S3: Input the two-dimensional mass spectrometry data into the embedding layer for high-dimensional mapping, stack more than 2 Transformer encoder layers after the embedding layer to further extract the features of the data, and after extraction, map the high-dimensional hidden state back to the dimension of the original data through a fully connected layer, output the model reconstructed data, calculate the loss value and use it as the optimization target for model training, and guide the model to continuously adjust the parameters to improve the quality of the reconstructed data.

[0057] S4: Set the batch size to be greater than 10 to batch process the data, perform iterative optimization on the model for 30 or more training cycles, and when the training loss tends to converge or reaches the preset number of training rounds, save the trained model weights as a file.

[0058] The following is a further illustration through a specific example:

[0059] In this example, 4 MGF mass spectrometry data files were used for experiments. Among them, three data files were from a Q Exactive mass spectrometer, and one data file was from a 6550A iFunnel Q-TOF LC / MS mass spectrometer. The species of all data was mouse.

[0060] A custom parsing function was used to read the mass spectrometry information item by item from the MGF file, parsing out the mass-to-charge ratio, signal intensity, and precursor ion mass of each mass spectrometry graph, and performing preprocessing such as enhancement processing, normalization, and unifying the sequence length on the read mass spectrometry data. As Figure 3 shown is the schematic diagram of the original spectrum, Figure 4 and is the schematic diagram of the preprocessed spectrum.

[0061] A self-supervised learning model based on the Transformer structure was built, including an embedding layer, 4 Transformer encoder layers, and a reconstruction output layer.

[0062] In the model training stage, the adaptive moment estimation optimizer was used for parameter update, setting the learning rate to 0.00003. During the training process, the batch size was set to 32 to batch process the data, and the model was iteratively optimized for 50 training epochs.

[0063] In each training epoch, the preprocessed mass spectrometry data was input into the Transformer encoder. The model gradually extracted the features of the data through the embedding layer and 4 layers of Transformer encoder layers, and generated prediction results through the reconstruction output layer. A custom combined loss function was used to measure the difference between the prediction results and the real data, where the weight α was set to 0.7. This loss function could effectively predict the overall shape consistency between the spectrum and the real spectrum. As Figure 5 shown is the schematic diagram of the reconstructed spectrum, Figure 6 and is the schematic diagram of the output of the embedding layer.

[0064] Through the backpropagation mechanism, the model continuously adjusted its internal parameters according to the feedback of the loss function, gradually reducing the reconstruction error. During the training process, the loss value was relatively high in the initial stage, but as the training progressed, the loss value gradually decreased and tended to be stable, indicating that the model gradually learned the internal structure and features of the mass spectrometry data. When the training loss tended to converge or reached the preset number of training rounds, the trained model weights were saved as the file pretrained_weights.h5.

Claims

1. A protein mass spectrometry encoding method based on self-supervised learning, characterized in that The following steps are involved: S1: Data preprocessing, converting raw mass spectrometry data into a unified format, including data augmentation, normalization, and sequence length standardization to ensure consistency of model input; S2: Construct a self-supervised learning task and determine the model's loss function, so that the model can automatically learn the implicit structure and features inherent in the mass spectrometry data under self-supervision; S3: Build a Transformer architecture encoder, construct a self-supervised model, and use a multi-head attention mechanism to extract global and local features of mass spectrometry data; S4: Model training: Optimize model parameters through self-supervised training to obtain a high-dimensional coding representation with strong robustness and excellent generalization performance, and save the obtained model weights to provide accurate and stable feature representation for subsequent mass spectrometry data encoding.

2. The method for encoding protein mass spectrometry based on self-supervised learning according to claim 1, wherein The specific method of data preprocessing in step S1 is as follows: S1.1 Data reading: Read the mass spectrum information from the original MGF file one by one, parse the mass-to-charge ratio (m / z value), mass spectrum intensity data, and parent ion mass in the mass spectrum information, and store this information in a list; S1.2 Data augmentation: Randomly augment the read mass spectrometry data to improve the generalization ability of the model. The intensity values are randomly scaled with a probability of 50%, and the scaling factor is sampled from a uniform distribution [0.8, 1.2], and the formula is: intensity=intensity×random(0.8,1.2) Gaussian noise is added to the m / z value with a probability of 30% to simulate the instrument calibration error. The noise follows the normal distribution N(0,0.05) and the formula is: mz=mz+random(N(0,0.05)); S1.3 Normalization: Use Z-score to normalize the m / z value and eliminate the dimension difference. The formula is: where ∈ = 10 -8 , μ mz and σ mz are the mean and variance of the m / z values of the current spectrum, respectively; perform a logarithmic transformation on the intensity values to compress the intensity value range. The formula is: normi ntensity = log(1 + intensity); S1.4 Unified sequence length: Set the maximum sequence length to 100 to eliminate the difference in the number of data points in different mass spectra and build a unified input tensor format to adapt to the model requirements. If the number of data points in the mass spectrum exceeds 100, sort them from high to low by intensity and retain the first 100 high-intensity data points to ensure that key information is not lost; if the number of data points is less than 100, use edge padding to copy the value of the last data point to 100 to avoid boundary noise introduced by zero padding. After processing, each mass spectrometry sample will be converted into a two-dimensional tensor with a shape of (100, 2), where 100 represents the number of data points and 2 corresponds to the two feature dimensions of m / z and intensity value.

3. A method for encoding protein mass spectrometry based on self-supervised learning according to claim 1, characterized in that Step S2 constructs a self-supervised learning task. The specific method is as follows: S2.1 Design a pre-training task: Use pre-processed mass spectrometry data as input and target output to construct the data pairs required for self-supervised learning. Through model training, the input data is reconstructed into an output that is as similar as possible to the original input, prompting the model to learn the intrinsic structure and characteristics of the mass spectrometry data. S2.2 Construct loss function: Define a combined loss function to comprehensively measure the numerical accuracy of the reconstructed data and the consistency of the spectral shape. Use the mean square error loss to calculate the mean square error between the predicted value and the true value, and constrain the reconstruction accuracy at the numerical level: Among them, is the real spectral data, is the predicted spectral data reconstructed by the model, and N is the total number of data points of a single sample. Calculate the spectral similarity loss, and measure the overall shape consistency between the predicted spectrum and the real spectrum through cosine similarity. The formula is: where B is the batch size, is the cosine similarity of the b-th sample. The final loss is the weighted sum of the two, with the weight being α, and the value range is from 0 to 1: L total = α·L MSE +(1 - α)·L spectral 4. A method for encoding protein mass spectrometry based on self-supervised learning according to claim 1, characterized in that Step S3 builds a self-supervised learning model. The specific method is as follows: S3.1 Embedding layer: The mass spectrometry data with the input shape of (100, 2) is mapped to a 256-dimensional embedding space through a fully connected layer, enabling it to accommodate more feature information, so that the model can learn richer and more expressive feature representations. The fully connected layer can be represented as a linear transformation: y = x·W + b where x is the input two-dimensional mass spectrometry data, the weight matrix W has the shape of (2, 256), and b is a 256-dimensional bias vector. After being processed by the embedding layer, the output shape is (100, 256). S3.2 Stacking of Transformer encoder layers: After the embedding layer, more than 2 Transformer encoder layers are stacked to further extract the features of the data. Each Transformer encoder layer contains the following key components: (1) Multi-head attention module: The long-range dependencies between data points in the input sequence are modeled using the multi-head attention mechanism. The multi-head attention mechanism allows the model to concurrently focus on different parts of the input sequence in different representation subspaces, capturing richer semantic information. The specific process is that the multi-head attention mechanism projects the query, key, and value of the input sequence into multiple low-dimensional subspaces respectively, then calculates the attention scores in each subspace, and finally weights and sums up these scores and concatenates them to obtain the final output. (2) Feed-forward neural network (FFN): It consists of two fully connected layers, with the Gaussian error linear unit activation function in the middle. The FNN can further perform feature transformation and combination on the output of the multi-head attention module, increasing the non-linear expression ability of the model. (3) Residual connection and layer normalization: A residual connection and a normalization layer are added after the multi-head attention module and the FFN respectively to improve the training stability and the model convergence speed. The residual connection allows the model to more easily learn the identity mapping during training, avoiding the problem of gradient disappearance. Layer normalization normalizes the input, making the distribution of the input more stable and accelerating the convergence of the model. S3.3 Reconstruction output layer: After the feature extraction through multiple Transformer encoder layers, the model uses a two-layer decoder to map the 256-dimensional hidden state back to the original data dimension. The specific process is as follows: The first layer of decoding non-linearly expands the 256-dimensional features to 512 dimensions through a fully connected layer combined with the Gaussian error linear unit activation function. The second layer of decoding uses a fully connected layer without an activation function to compress the 512-dimensional features to the original dimension, and the output shape is (100, 2), corresponding to the reconstructed m / z value and intensity value respectively. The output of this layer is the mass spectrometry data reconstructed by the model, which is used to calculate the self-supervised reconstruction error. By mapping the high-dimensional features back to the original data dimension, the model can learn how to reconstruct the original mass spectrometry data from the feature representation and capture the internal structure and features of the data.

5. A method for encoding protein mass spectrometry based on self-supervised learning according to claim 1, characterized in that Step S4 Model training, the specific method is as follows: S4.1 Training process: The adaptive moment estimation optimizer is used for parameter update, and the initial learning rate is set to be less than 1×10 -4 , to ensure that the model can converge stably during training. A dynamic learning rate scheduler is introduced to monitor the training loss. If the loss does not decrease within 3 consecutive training epochs, the learning rate is decayed by a factor of 0.

5. The data is batched in a way that the batch size is greater than 10, and the model is iteratively optimized for 30 or more training epochs; S4.2 Weight saving: When the training loss tends to converge or reaches the preset number of training epochs, save the trained model weight file for subsequent data encoding applications and fine-tuning of downstream tasks.

Images