A NeoBert model and its application in identifying tumor neoantigens

Abstract

This invention provides a NeoBert model and its application in identifying tumor neoantigens, including a sequence input device, an embedding block, an encoder block, and a prediction block. The NeoBert model trained by this invention achieves an accuracy of 93% in predicting peptide-HLA binding, which is significantly better than NetMHCpan's accuracy of 75%, providing a novel predictive model for tumor neoantigens.

Description

Technical Field

[0001] This invention belongs to the field of biomedical technology, specifically relating to a NeoBert model and its application in identifying tumor neoantigens. Background Technology

[0002] HLA, or Human Leukocyte Antigen, is an expression product of the human major histocompatibility complex (MHC), with over 200 coding genes, all located on chromosome 6. Tumor-specific antigens (TSA), also known as tumor neoantigens, are short peptide antigens displayed by the MHC on the surface of tumor cells. These unique TSAs are absent in normal tissues but can bind to HLA in abnormal tissues, becoming recognized as foreign antigens by T cells and capable of evoking tumor-specific T cell responses, making them promising candidates for cancer vaccines. Current traditional methods for identifying these classic neoantigens heavily rely on whole-genome / whole-exome sequencing data. Several extensively studied classic neoantigen databases, including CEDAR, CAPED (https: / / caped.icp.ucl.ac.be), TANTIGEN, NEPdb, dbPepNeo, NEOdb, and TSNAdb, have been extensively explored for their applicability in cancer immunotherapy.

[0003] Although coding regions comprise only 40% of the genome, with the remaining 60% being non-coding regions, mounting evidence suggests that non-canonical mechanisms, including transcriptional variants (such as alternative splicing, intron retention, recessive introns, and chimeric transcripts derived from transposons) and proteasome cryptic peptides (cis / trans proteasome processing), contribute to the formation of a larger non-canonical neoantigen library. However, the collection and management of non-canonical neoantigen data remain limited. Currently, only one database exists for non-coding upstream open reading frame (uORF)-derived neoantigens identified via RNA-seq: SPENCER, which contains 2806 mass spectrometry data from 55 studies. However, this dataset is not derived from immunoproteomics; it predicts non-coding RNA (ncRNA) translation and uses a sliding window algorithm to segment each non-coding peptide into 8–14 polymer fragments. Regarding proteasome-processed neoantigens, the Proteomedb database contains such entities. Nevertheless, it requires the in vitro synthesis of 80 peptides, followed by proteasome digestion and detection using various mass spectrometry methods. The peptides were detected using a method called invitroSPI, so these neoantigens were not present in the tumor sample.

[0004] Detecting and sequencing HLA-bound peptides using liquid chromatography-tandem mass spectrometry (LC-MS / MS) offers a unique advantage: it allows for direct learning of information about endogenous cellular processing and presentation of peptides. We collected and compiled a large dataset of non-redundant, non-classical neoantigen peptides, containing 35,574 paired neoantigens-HLA from different RNA and protein-based sources, derived from 14 publications. This comprehensive compilation enabled us to compare their antigenic properties with those of peptides derived from somatic mutations (represented by CEDAR), tumor-associated peptides (represented by HLAthena), and neoantigens from infectious diseases (represented by IEDB), including aspects such as length, motif, entropy, and hydrophobicity.

[0005] These differences prompted us to train methods for predicting peptide-HLA (pHLA) binding. Currently, pHLA binding prediction methods are broadly categorized into four types: structure-based methods, scoring function-based methods, machine learning-based methods, and comprehensive methods combining multiple factors. Structure-based methods dissect the binding structure of HLA and peptides, while scoring function-based techniques evaluate peptides based on sequence-based properties. Machine learning-based methods extract features from peptides or HLA and train models for binding prediction. Deep learning methods, including CNNs, RNNs, and attention mechanisms, exhibit good performance due to their accuracy and efficiency. Recently, a converter-based model was developed to predict peptide-HLA class I binding and to mutant peptides for vaccine design. However, it is only trained on classical peptides and cannot fully explain the complex interactions between HLA and peptide sequences. Therefore, we integrated classical and non-classical neoantigen data to train and derive the NeoBert model of this invention. Summary of the Invention

[0006] After collecting data, this invention uses 70% of the data to train the NeoBert model, and the remaining 30% is used for validation and comparison of the NeoBert model. The model determines whether a peptide is a tumor neoantigen by judging whether it can bind to HLA. Based on this, this invention was completed.

[0007] In a first aspect, the present invention provides a NeoBert model, which consists of four consecutive components, including:

[0008] Sequence input device: The masked HLA and peptide sequence are packaged into a sentence (i.e.,<masked HLA,maskedpeptide> () is used for input;

[0009] Embedding block: Add position embeddings to amino acid (i.e., token) embeddings and fragment embeddings to generate sequence embeddings;

[0010] Encoder block: Contains 12 BertLayers, each BertLayer contains a masked multi-head self-attention mechanism and a feature optimization block for learning representations;

[0011] Prediction block: The aggregation sequence representation is used to predict the binding probability, while the masked amino acid model predicts the masked amino acid based on the representation of each corresponding position.

[0012] Furthermore, in the sequence input device, each pair of HLA and peptide sequences is represented as a sentence, for example, X = ([CLS], HLA sequence, [SEP], peptide sequence).

[0013] Furthermore, in the sequence input device, the final hidden feature of the first label ([CLS]) is used as the sequence representation for classification, while the ([SEP]) label distinguishes between the two sequences, and the random masked amino acids of the two sequences are used to train the deep bidirectional representation.

[0014] Furthermore, in the sequence input device, the input peptide and HLA sequence are padded to a length of 128 to accommodate variable input lengths.

[0015] Furthermore, in the embedding block, for each token, the learned embedding (E) is added. A or E B ∈R H The sequence is represented by H as either an HLA sequence or a peptide sequence, facilitating segmentation between the two. H represents the embedding dimension, and WordPiece embedding is used to encode the amino acid embedding (E). Ai or E Bj ∈R H ).

[0016] Furthermore, the embedding block also incorporates position embedding (E) i ∈R H ) is used to encode the position (X) of amino acids in a sentence.

[0017] Furthermore, in the embedding block, the input representation in the token (or the token in the HLA) is created by adding its token, segment, and position embeddings together, as follows:

[0018] E i =E A +E Arn +E i

[0019] Furthermore, in the encoder block, multi-head self-attention technology is used to combine bidirectional cross-attention between two sequences through a cascaded sequence.

[0020] Furthermore, in the encoder block, the mechanism includes mapping a query Q to a set of key-value (KV) pairs and obtaining an output, wherein the KV pairs store sequence elements in memory.

[0021] Furthermore, in the encoder block, the attention score is based on the correlation or similarity between Q and K, representing the importance of the information. The higher the attention score (i.e., V), the stronger the attention to the corresponding information.

[0022] Furthermore, in the encoder block, the feature optimization module combines a fully connected layer with LayerNorm and Dropout to learn an enhanced representation (H). i ) is used for the i-th tag.

[0023] Furthermore, in the encoder block, HLA and peptide linking sequences with a length less than 128 are filled with 0 to 128.

[0024] Furthermore, the prediction block employs two tasks to fine-tune the NeoBert model: HLA peptide binding prediction and masked amino acid prediction.

[0025] Furthermore, the HLA peptide binding prediction involves inputting the polymerase sequence characterization of each HLA-peptide pair into the S-type layer to predict the tag:

[0026] Y′=Sigmoid(W (2) (Tanh(W (1) H 0 +b (1) )))

[0027] Among them, W (1) W (2) , and b (1) H is a learnable parameter. 0 The representation of the special label [CLS] and the corresponding loss function are summarized below:

[0028]

[0029] Here, N represents the set of HLA peptide pairs containing both positive and negative binding data, and the generated data is similar to that of previous studies.

[34] .

[0030] Furthermore, a masked amino acid model predicts the probability of 20 different amino acids being located in a sequence based on their sequence representation:

[0031] P i =Softmax(WH i +b)

[0032] Where W and b are learnable parameters, H iThis represents the token. Its loss function is summarized as follows:

[0033]

[0034] Here, Mask represents the set of all masked amino acid positions, Pos represents the set of positive binding pairs, and the total loss function of the NeoBERT model is summarized as follows:

[0035]

[0036] Secondly, the present invention provides the application of the NeoBert model as described in the first aspect in the identification of tumor neoantigens, wherein the model determines whether a peptide is a tumor neoantigen by determining whether the peptide can bind to HLA.

[0037] Furthermore, if the peptide binds to HLA, it is predicted to be a neotumor antigen.

[0038] Thirdly, a method for identifying tumor neoantigens, the method comprising the following steps:

[0039] 1) Input the peptide and HLA sequences into the sequence input device. The masked HLA and peptide sequences are packaged into a sentence (i.e., <masked HLA, masked peptide>) for input.

[0040] 2) The embedding block adds position embeddings to amino acid (i.e., token) embeddings and fragment embeddings to generate sequence embeddings;

[0041] 3) The encoder block utilizes multi-head self-attention technology to combine bidirectional cross-attention between two sequences through cascading sequences;

[0042] 4) The prediction block uses aggregated sequence representation to predict the binding probability, while the masked amino acid model predicts the masked amino acid based on the representation of each corresponding position;

[0043] 5) Determine whether the peptide can bind to HLA.

[0044] Furthermore, in the sequence input device, each pair of HLA and peptide sequences is represented as a sentence, for example, X = ([CLS], HLA sequence, [SEP], peptide sequence).

[0045] Furthermore, in the sequence input device, the final hidden feature of the first label ([CLS]) is used as the sequence representation for classification, while the ([SEP]) label distinguishes between the two sequences, and the random masked amino acids of the two sequences are used to train the deep bidirectional representation.

[0046] Furthermore, in the sequence input device, the input peptide and HLA sequence are padded to a length of 128 to accommodate variable input lengths.

[0047] Furthermore, in the embedding block, for each token, the learned embedding (E) is added. A or E B ∈R H The sequence is represented by H as either an HLA sequence or a peptide sequence, facilitating segmentation between the two. H represents the embedding dimension, and WordPiece embedding is used to encode the amino acid embedding (E). Ai or E Bj ∈R H ).

[0048] Furthermore, the embedding block also incorporates position embedding (E) i ∈R H ) is used to encode the position (X) of amino acids in a sentence.

[0049] Furthermore, in the embedding block, the input representation in the token (or the token in the HLA) is created by adding its token, segment, and position embeddings together, as follows:

[0050] E i =E A +E Am +E i

[0051] Furthermore, in the encoder block, multi-head self-attention technology is used to combine bidirectional cross-attention between two sequences through a cascaded sequence.

[0052] Furthermore, in the encoder block, the mechanism includes mapping a query Q to a set of key-value (KV) pairs and obtaining an output, wherein the KV pairs store sequence elements in memory.

[0053] Furthermore, in the encoder block, the attention score is based on the correlation or similarity between Q and K, representing the importance of the information. The higher the attention score (i.e., V), the stronger the attention to the corresponding information.

[0054] Furthermore, in the encoder block, the feature optimization module combines a fully connected layer with LayerNorm and Dropout to learn an enhanced representation (H). i ) is used for the i-th tag.

[0055] Furthermore, in the encoder block, HLA and peptide linking sequences with a length less than 128 are filled with 0 to 128.

[0056] Furthermore, the prediction block employs two tasks to fine-tune the NeoBert model: HLA peptide binding prediction and masked amino acid prediction.

[0057] Furthermore, the HLA peptide binding prediction involves inputting the polymerase sequence characterization of each HLA-peptide pair into the S-type layer to predict the tag:

[0058] Y′=Sigmoid(W (2) (Tanh(W (1) H 0 +b (1) )))

[0059] Among them, W (1) W( 2) , and b (1) H is a learnable parameter. 0 The representation of the special label [CLS] and the corresponding loss function are summarized below:

[0060]

[0061] Here, N represents the set of HLA peptide pairs containing both positive and negative binding data, and the generated data is similar to that of previous studies.

[34] .

[0062] Furthermore, a masked amino acid model predicts the probability of 20 different amino acids being located in a sequence based on their sequence representation:

[0063] P i =Softmax(WH i +b)

[0064] Where W and b are learnable parameters, H i This represents the token. Its loss function is summarized as follows:

[0065]

[0066] Here, Mask represents the set of all masked amino acid positions, Pos represents the set of positive binding pairs, and the total loss function of the NeoBERT model is summarized as follows:

[0067]

[0068] Furthermore, if the peptide binds to HLA, it is predicted that the peptide is a tumor neoantigen; if the peptide does not bind to HLA, it is predicted that the peptide is not a tumor neoantigen.

[0069] Fourthly, the present invention provides a system for identifying tumor neoantigens, the system comprising:

[0070] The sequence input module is used to input peptide and HLA sequences. The masked HLA and peptide sequences are packaged into a sentence.

[0071] The sequence processing module is used to process the input sequence;

[0072] The probability prediction module uses aggregated sequence representation to predict binding probabilities, while the masked amino acid model predicts masked amino acids based on the representation of each corresponding position.

[0073] The results output module determines whether a peptide is a neotumor antigen by judging whether it can bind to HLA.

[0074] Furthermore, in the sequence processing module, the embedding block adds position embeddings to amino acid (i.e., token) embeddings and fragment embeddings to generate sequence embeddings, and the encoder block utilizes multi-head self-attention technology to effectively combine bidirectional cross-attention between two sequences through tandem sequences.

[0075] Furthermore, in the probability prediction module, the linear binary classifier uses aggregated sequence representation to predict the binding probability, while the masked amino acid model predicts the masked amino acid based on the representation of each corresponding position.

[0076] Furthermore, if the peptide binds to HLA, it is predicted to be a neotumor antigen.

[0077] Fifthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in the third aspect.

[0078] Beneficial effects

[0079] The NeoBert model trained in this invention achieves an accuracy of 93% in predicting peptide-HLA binding, which is significantly better than NetMHCpan's accuracy of 75%, providing a novel predictive model for tumor neoantigens. Attached Figure Description

[0080] Figure 1 This is a comparison of the prediction performance of the NeoBert model and NetMHCpan in this invention. Detailed Implementation

[0081] The specific embodiments of the present invention will be further described below. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the embodiments described below can be combined with each other as long as they do not conflict with each other.

[0082] Unless otherwise specified, the experimental methods used in the following embodiments are conventional methods, and the experimental materials used in the following embodiments are all available through conventional commercial channels.

[0083] Example 1: Training the NeoBERT Model

[0084] 1. Data Collection

[0085] 1.1 Collecting RNA-derived neoantigen data

[0086] From existing literature (Ouspenskaia, T. et al. Unannotated proteins expand the MHC-I-restricted immunopeptidome in cancer. Nat Biotechnol 40, 209-217, doi:10.1038 / s41587-021-01021-3 (2022); Chong, C. et al. Integrated proteogenomic deepsequencing and analytics accurately identify non-canonical peptides in tumorimmunopeptidomes. Nat Commun 11, 1293, 692doi:10.1038 / s41467-020-14968-9 (2020); Smart, A.C. et al. Intron retention is a source of neoepitopes in cancer. NatBiotechnol 36, 1056-1058, doi:10.1038 / nbt.4239 (2018); Kong, Y. et al. Transposableelement expression in tumors is associated with immune infiltration andincreased antigenicity. Nat Commun 10, 5228, doi:10.1038 / s41467-019-13035-2 (2019); Schwarz, S. et al. T cells of colorectal cancer patients'stimulated byneoantigenic and cryptic peptides better recognize autologous tumor cells. JImmunother Cancer 10, doi:10.1136 / jitc-2022-005651 (2022); Zhao, Q.etal. Proteogenomics Uncovers a Vast Repertoire of Shared Tumor-SpecificAntigens in Ovarian Cancer. Cancer Immunol Res 8, 544-555, doi:10.1158 / 2326-6066.CIR-19-0541(2020); Laumont,CMet al.Noncoding regions are the main source of targetable tumor-specific antigens.Sci Transl Med 10, doi:10.1126 / scitranslmed.aau5516(2018); Erhard, F., Dolken, L., Schilling, B. & Schlosser, A. Identification of the Cryptic HLA-I Immunopeptidome. Cancer Immunol Res 8,1018-1026,doi:10.1158 / 2326-6066.CIR-19-0886(2020); Laumont,CMet al.Globalproteogenomic analysis of human MHC class I-associated peptides derived from non-canonical reading frames.Nat Commun Data was collected from 7,10238, doi:10.1038 / ncomms10238 (2016). Each entry was annotated with the following information: "PMID", "Cell.line", "Peptide", "Length", "Genome", "Chromosome", "Start", "Stop", "Loci", "Transcript_strand", "Overlapping genomic feature", "HLA_locus", "HLA_allele", "Ensembl_gene_id", and "Protein_name".

[0087] 1.2 Collecting data on protein-derived neoantigens

[0088] Data from 4 previous studies were collected (Faridi, P. et al. A subset of HLA-I peptides are not genomically templated: Evidence for cis-and trans-spliced ​​peptideligands. Sci Immunol 3, doi: 10.1126 / sciimmunol.aar3947 (2018); Liepe, J. et al. A large fraction of HLA class I ligands are Proteome Res, 349-358, doi: 10.1021 / acs.jproteome.8b00651(2019); Faridi, P. et al. Spliced ​​Peptides and Cytokine-Driven Changes in the Immunopeptidome of Melanoma. Cancer Immunol Res8, 1322-1334, doi: 10.1158 / 2326-6066.CIR-19-0894(2020).).

[0089] Annotation data was downloaded from UniProtKB / Swiss-Prot (https: / / www.uniprot.org / ) to annotate proteins and obtain their corresponding genomic coordinates. For the two CIS datasets, “PMID: 27846572” and “PMID: 30315122”, the Parentprotein(s) format was normalized to “EntryEntry.namelocation”, and protein annotations were added under “GeneUniprot”.

[0090] For "PMID: 32938616", the peptide and protein names were extracted. For "PMID 30346791", the "Full Sequence" and "Protein Name" columns contained multiple peptides and proteins linked by "OR". These entries were split to list each sequence and protein name relationship separately, and then matched against protein, gene name, and ensemble from UniProt annotation files. For entries that did not match, the FASTA file for the corresponding protein was downloaded from UniProtKB.

[0091] 1.3 IEDAR, HLAthena data

[0092] IEDB data can be downloaded from https: / / www.iedb.org. HLAthena data can be downloaded from https: / / hlathena.tools. CEDAR data can be downloaded from https: / / cedar.iedb.org.

[0093] 2. NeoBERT training was performed on 70% of the HLA-peptides from five different datasets.

[0094] NeoBert consists of four consecutive components: (i) a masked HLA and peptide sequence are packaged into a sentence (i.e., <masked HLA, masked peptide>) for input; (ii) an embedding block adds position embeddings to amino acid (i.e., token) embeddings and fragment embeddings to generate sequence embeddings; (iii) an encoder block contains 12 BertLayers, each BertLayer containing a masked multi-head self-attention mechanism and a feature optimization block for learning representations; (iv) a linear binary classifier uses aggregated sequence representations to predict binding probabilities, while the masked amino acid model predicts masked amino acids based on the representation at each corresponding position. Specifically:

[0095] (i) Input sequence

[0096] Each pair of HLA and peptide sequences is represented as a sentence, e.g., X = ([CLS], HLA sequence, [SEP], peptide sequence). The final hidden feature of the first tag ([CLS]) is used as the sequence representation for classification, while the ([SEP]) tag distinguishes between the two sequences. Random masked amino acids of both sequences are used to train the deep bidirectional representation. The input peptide and HLA sequences are padded to a length of 128 to accommodate variable input lengths.

[0097] (ii) Sequence embedding

[0098] For each token, add the learned embedding (E). Aor E B ∈R H The sequence is represented by H as either an HLA sequence or a peptide sequence, facilitating segmentation between the two sequences. H represents the embedding dimension. WordPiece embedding is used to encode amino acid embeddings (E...). Ai or E Bj ∈R H In addition, location embedding (E) was added. i ∈R H The position (X) of an amino acid in a sentence is encoded using a token. Therefore, the input representation in the token (or the token in HLA) is created by adding its token, segment, and position embeddings, as follows:

[0099] E i =E A +E Arn +E i

[0100] (ii) Encoder block

[0101] Multi-head self-attention is utilized to effectively combine bidirectional cross-attention between two sequences through concatenated sequences. The mechanism involves mapping a query Q to a set of key-value (KV) pairs and obtaining the output, where the KV pairs store sequence elements in memory. Attention scores are based on the relevance or similarity between Q and K, representing the importance of the information. Higher attention scores (i.e., V) indicate stronger attention to the corresponding information. Furthermore, a feature optimization module combines fully connected layers with LayerNorm and Dropout to learn an enhanced representation (H). i ) is used for the i-th label. For HLA and peptide linker sequences less than 128 in length, we fill with 0–128. Therefore, non-amino acid features do not play a role in the computational attention.

[0102] (iv) Prediction Block

[0103] Two tasks were employed to fine-tune the NeoBert model: HLA peptide binding prediction and masked amino acid prediction. Specifically, we input the polymerase sequence characterization of each HLA-peptide pair into the S-layer to predict the tag:

[0104] Y′=Sigmoid(W (2) (Tanh(W (1) H 0 +b (1) )))

[0105] Among them, W (1) W (2) , and b (1) H is a learnable parameter. 0This represents the special label [CLS]. The corresponding loss function is summarized below:

[0106]

[0107] Here, N represents the set of HLA peptide pairs containing both positive and negative binding data, and the generated data is similar to that of previous studies.

[34] Meanwhile, a masked amino acid model predicts the probability of 20 different amino acids at their positions in the sequence based on their sequence representation:

[0108] P i =Softmax(WH i +b)

[0109] Where W and b are learnable parameters, H i This represents the token. Its loss function is summarized as follows:

[0110]

[0111] Here, Mask represents the set of all masked amino acid positions, and Pos represents the set of positive binding pairs. Therefore, the total loss function of the NeoBERT model can be summarized as follows:

[0112]

[0113] Example 2: Validation of the NeoBERT model and comparison of its prediction performance with NetMHCpan

[0114] After training the NeoBert model on 70% of the HLA-peptide pairs from five different datasets containing both classical and non-classical neoantigens, the predictive performance of the NeoBert model and NetMHCpan was compared using the remaining 30% of the data from the five different datasets.

[0115] The performance of each predictive model was evaluated based on the following metrics: accuracy, Matthews correlation coefficient (MCC), F1 score, and area under the receiver operating characteristic curve (AUC). Specifically,

[0116]

[0117] TP, FP, FN, and TN represent true positive, false positive, false negative, and true negative, respectively.

[0118] like Figure 1As shown, the benchmark results indicate that: 1) NeoBert has an overall accuracy of 93%, while NetMHCpan has an accuracy of 75%; 2) NeoBert outperforms NetMHCpan in predicting both classical and non-classical neoantigens, only performing slightly worse in predicting RNA-derived neoantigens; 3) NeoBert provides more consistent predictions than NetMHCpan across different databases, demonstrating its ability to capture diverse nonlinear patterns from different sources.

Claims

1. An application of a NeoBert model in identifying tumor neoantigens, said application using the NeoBert model, said NeoBert model consisting of four consecutive components, including: Sequence inputter: The masked HLA and peptide sequence are packaged into a sentence, i.e.<masked HLA, maskedpeptide> , used for input; Embedding block: Add position embeddings to amino acid embeddings and fragment embeddings to generate sequence embeddings, where amino acids are tokens; Encoder block: Contains 12 BertLayers, each BertLayer contains a masked multi-head self-attention mechanism and a feature optimization block for learning representations; Prediction block: The aggregation sequence representation is used to predict the binding probability, while the masked amino acid model predicts the masked amino acid based on the representation of each corresponding position; In the sequence input device, each pair of HLA and peptide sequences represents a sentence, i.e. The final hidden feature of the first tag [CLS] is used as the sequence representation for classification, while the [SEP] tag distinguishes between the two sequences. Random masked amino acids of the two sequences are used to train the deep bidirectional representation. The input peptide and HLA sequence are padded to a length of 128 to accommodate variable input lengths. In the embedding block, for each token, the learned embedding is added. or This indicates whether it belongs to HLA or a peptide, thus facilitating the segmentation of these two sequences. or Where H is the embedding dimension, using WordPiece embedding. or To encode the amino acid embedding, the or The embedded block also incorporates position embedding. To encode the position of amino acids in a sentence The For each token in the peptide or HLA sequence, its input representation is created by adding its token embedding, segment embedding, and position embedding, as follows: Among them, the This involves segment embedding that distinguishes HLA from peptides; the prediction block employs two tasks to fine-tune the NeoBert model: HLA peptide binding prediction and masked amino acid prediction. The HLA peptide binding prediction involves inputting the polymer sequence characterization of each HLA-peptide pair into the S-layer to predict the tag. in, , , and These are learnable parameters. The representation of the special label [CLS] and the corresponding loss function are summarized below: in, This represents a set of HLA peptide pairs containing both positive and negative binding data; a masked amino acid model predicts the probability of 20 different amino acids at their positions in the sequence based on their sequence representation. in, and These are learnable parameters. The token is represented; its loss function is summarized as follows: Here, Mask represents the set of all masked amino acid positions, Pos represents the set of positive binding pairs, and the total loss function of the NeoBERT model is summarized as follows: 。 2. In the application as described in claim 1, the encoder block utilizes multi-head self-attention technology to combine bidirectional cross-attention between two sequences through concatenated sequences. The mechanism includes mapping a query Q to a set of key-value pairs K-V and obtaining the output, wherein the K-V pairs store sequence elements in memory; the attention score is based on the relevance or similarity between Q and K, representing the importance of information, with a higher attention score, i.e., V, indicating stronger attention to the corresponding information; the feature optimization module combines fully connected layers with LayerNorm and Dropout to learn enhanced representations. Used for the i-th marker; for HLA and peptide linker sequences less than 128 in length, fill with 0~128.

3. The application of the NeoBert model as described in claim 1 in identifying tumor neoantigens, wherein the model determines whether a peptide is a tumor neoantigen by judging whether it can bind to HLA, and if the peptide binds to HLA, it is predicted that the peptide is a tumor neoantigen.

4. A method for identifying tumor neoantigens, the method comprising the following steps: Enter the peptide and HLA sequence into the sequence input device. The masked HLA and peptide sequence are packaged into a sentence, namely: masked HLA, masked peptide, for input. Embedding block: Adds position embeddings to amino acid embeddings and fragment embeddings to generate sequence embeddings, where, Amino acids are tokens; The encoder block utilizes multi-head self-attention technology to combine bidirectional cross-attention between two sequences through concatenated sequences; The prediction block uses aggregated sequence representations to predict binding probabilities, while the masked amino acid model predicts masked amino acids based on the representation of each corresponding position. Determine whether a peptide can bind to HLA; In the sequence input device, each pair of HLA and peptide sequences represents a sentence, i.e. The final hidden feature of the first tag [CLS] is used as the sequence representation for classification, while the [SEP] tag distinguishes between the two sequences. Random masked amino acids of the two sequences are used to train the deep bidirectional representation. The input peptide and HLA sequence are padded to a length of 128 to accommodate variable input lengths. In the embedding block, for each token, the learned embedding is added. or This indicates whether it belongs to HLA or a peptide, thus facilitating the segmentation of these two sequences. or Where H is the embedding dimension, using WordPiece embedding. or To encode the amino acid embedding, the or The embedded block also incorporates position embedding. To encode the position of amino acids in a sentence The For each token in the peptide or HLA sequence, its input representation is created by adding its token embedding, segment embedding, and position embedding, as follows: Among them, the This involves segment embedding that distinguishes HLA from peptides; the prediction block employs two tasks to fine-tune the NeoBert model: HLA peptide binding prediction and masked amino acid prediction. The HLA peptide binding prediction involves inputting the polymer sequence characterization of each HLA-peptide pair into the S-layer to predict the tag. in, , , and These are learnable parameters. The representation of the special label [CLS] and the corresponding loss function are summarized below: in, This represents a set of HLA peptide pairs that includes both positive and negative binding data. A masked amino acid model predicts the probability of 20 different amino acids at their positions in a sequence based on their sequence representation: in, and These are learnable parameters. The token is represented; its loss function is summarized as follows: Here, Mask represents the set of all masked amino acid positions, Pos represents the set of positive binding pairs, and the total loss function of the NeoBERT model is summarized as follows: If a peptide binds to HLA, it is predicted to be a neotumor antigen.

5. The method of claim 4, wherein the encoder block utilizes multi-head self-attention technology to combine bidirectional cross-attention between two sequences through concatenated sequences. The mechanism includes mapping a query Q to a set of key-value pairs K-V and obtaining an output, wherein the K-V pairs store sequence elements in memory, and the attention score is based on the relevance or similarity between Q and K, representing the importance of the information. A higher attention score, i.e., V, indicates stronger attention to the corresponding information. The feature optimization module combines a fully connected layer with LayerNorm and Dropout to learn enhanced representations. For the i-th marker, for HLA and peptide linker sequences less than 128 in length, fill with 0~128.

6. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method as claimed in claim 4.

Images