T cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation
By employing a multi-level knowledge distillation method, utilizing convolutional neural networks and bilinear attention networks, and combining internal multi-feature and inter-domain knowledge distillation, the problem of insufficient generalization of deep learning models on unseen epitopes is solved, achieving more efficient TCR-epitope combination prediction and result interpretation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2024-12-18
- Publication Date
- 2026-05-29
AI Technical Summary
Existing deep learning models perform poorly when generalizing to unseen epitopes and lack interpretability, limiting their application in the healthcare field.
We employ a multi-level knowledge distillation approach, extracting TCR-epitope pairs from public databases for model training and testing. We utilize convolutional neural networks and bilinear attention networks to capture sequence features, and combine internal multi-feature and inter-domain knowledge distillation to improve the model's generalization ability and interpretability.
It significantly improves the model's predictive performance on unseen epitopes, enhances the model's robustness and generalization ability, and provides interpretability of the model results, supporting a wider range of applications.
Smart Images
Figure CN119920319B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for predicting the specificity of T cell receptor-epitope binding, belonging to the field of bioinformatics. Background Technology
[0002] Understanding the interaction mechanisms between T cell receptors (TCRs) and epitopes is crucial for studying the adaptive immune system and advancing immunotherapy. Traditional experimental methods for identifying TCR-epitope binding pairs are costly and time-consuming. However, with the development of high-throughput sequencing technologies, an increasing amount of TCR-epitope binding pair data is archived in public databases such as VDJdb, IEDB, and McPAS-TCR, providing opportunities for computational methods. Therefore, developing computational methods to predict TCR-epitope binding specificity is particularly important.
[0003] Machine learning and deep learning methods can quickly capture potential patterns of TCR-epitope binding in large amounts of experimental data. Therefore, an increasing number of related methods have been developed to predict TCR-epitope binding. Some studies attempt to predict TCR specificity by extracting biological information from TCR sequences individually. For example, DeepTCR is an unsupervised and supervised deep learning method that models highly complex TCR sequencing data by learning a joint representation of TCRs using CDR3 sequences and V / D / J genes. Other methods aim to develop binding prediction models for specific peptides, such as clustering methods, decision trees, variational autoencoders, Gaussian processes, and deep walks. These methods typically train a specific model for each epitope and require a large amount of relevant training data. However, many real-world scenarios require prediction of novel epitopes, which often lack relevant binding data, thus limiting the application of these methods. Recently, an increasing number of researchers have been dedicated to developing general models that can predict the binding of any TCR-epitope pair. For example, NetTCR-2.0 is a convolutional neural network for predicting TCR-peptide binding; ERGO is a natural language processing (NLP) method that combines a large-scale TCR-peptide dictionary with deep learning methods.
[0004] While current research has made significant progress in predicting TCR-epitope bindings, these methods are limited by insufficient interaction data. Due to the high cost and limitations of experimental methods, currently available databases are limited to a small fraction of the potential sample space for TCR-epitope binding pairs. Recent studies have shown that existing deep learning models perform poorly when generalizing to unseen epitopes, and their performance is highly dependent on the balance and scale of the data. Therefore, further improvements in generalization and robustness are needed for model performance. Furthermore, model interpretability is equally important in the healthcare field. Interpreting model results can help researchers guide their research and is crucial for developing reliable systems. Summary of the Invention
[0005] To address the problem that existing deep learning models perform poorly when generalizing to unseen epitopes, this invention proposes a T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation.
[0006] The technical solution adopted by the present invention to solve the above problems is as follows: The steps of the present invention include:
[0007] Step 1: Extract TCR-tablet binding pairs from public databases, generate datasets, and divide them into training and test sets for model training and testing.
[0008] Step 2: Collect multiple sequence feature information of TCR and epitope for internal multi-feature knowledge distillation;
[0009] Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters, which are used for inter-domain knowledge distillation;
[0010] Step 4: Construct a feature extraction module, use a convolutional neural network to extract features from TCR and epitope sequences, and use a learnable amino acid embedding matrix to encode the sequence and capture the biological information in the sequence;
[0011] Step 5: Construct a bilinear attention network and explore the interaction pattern between TCR and epitopes through the bilinear attention mechanism to capture the binding features;
[0012] Step 6, Internal Multi-Feature Knowledge Distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features, and this knowledge is transferred to the student model.
[0013] Step 7, Inter-domain knowledge distillation: For different epitopes, train domain-specific models and distill their knowledge into a general model to improve the model's generalization ability on unseen epitopes;
[0014] Step 8: Input the TCR-epitope pairs from the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.
[0015] Furthermore, in step 1, TCR-epitope binding pairs are extracted from a public database to generate a dataset. The dataset is then randomly divided into a training set and a test set in a 4:1 ratio, with 80% being the training set and 20% being the test set.
[0016] Both the training and test sets contain each type of action, which is encoded as 0 and 1, where 0 represents no binding and 1 represents binding. The dataset is then subjected to 5-fold cross-validation.
[0017] Furthermore, in step 2, three different protein sequence features are introduced: embeddings extracted from the protein language model ESM, Atchley factor, and BLOSUM62.
[0018] Furthermore, in step 3, the cosine similarity index is used to calculate the similarity between epitope A and epitope B, as shown in the following formula:
[0019]
[0020] Furthermore, in step 4, two independent convolutional neural network layers are used to extract features from the TCR and epitope sequences. Specifically, the embedding matrices of the TCR and epitopes are input into three consecutive 1D convolutional layers. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer is used to capture residue-level features, while the second and third convolutional layers increase the receptive field and extract more abstract features from the TCR and epitope sequences. The structure of the convolutional layers is as follows:
[0021] F (l+1) =σ(f c (W l ,b l ,F l )) (2),
[0022] In formula (2), W l and b l Let F represent the learnable weight matrix and bias vector of the l-th convolutional layer, respectively. l It is the learned representation of the l-th convolutional layer; F 0 It is the encoded embedding matrix or σ(.) represents a non-linear activation function, using the Corrected Linear Unit (ReLU) as the activation function. The output of the feature extraction module is... and in It is the extracted TCR feature, D t It is the potential spatial dimension of TCR. It is the extracted epitope feature, D e It is the potential spatial dimension of the epitope.
[0023] Furthermore, in step 5, a bilinear attention network is used to capture pairwise interactions between TCRs and epitopes. After feature extraction, the bilinear attention network has two inputs, namely... and Two learnable weight matrices are introduced, denoted as follows: and And a bilinear attention map, represented as To obtain the joint intermediate representation Where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows:
[0024]
[0025] In formula (3), U k and V k These represent the k-th column of the weight matrix, and They are The i-th column and The j-th column is obtained by using the Hadamard product and matrix multiplication to form a bilinear attention graph. The definition is as follows:
[0026]
[0027] In formula (4), It is a vector consisting entirely of 1s. It is a learnable weight vector. The Hadamard product is represented by the elements of I, which refer to the interaction strength between each TCR-epitope substructure pair.
[0028] By learning learnable weight matrices, named U and V respectively, and a learnable weight vector p, the bilinear attention network can learn pairwise interactions between residues of TCR-epitopes and provide interpretability regarding the contributions of substructure pairs. Summation pooling is applied to the joint intermediate representation to generate the learned features; the summation pooling formula is as follows:
[0029] f = SumPool(f′, h) (5)
[0030] Multiple bilinear interaction graphs are computed by summing and pooling f′ using a one-dimensional non-overlapping window of size h.
[0031] Furthermore, in step 6, three different protein sequence features are introduced: embeddings extracted from the protein language model ESM, Atchley factor, and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode TCR and epitope sequences. The architecture of the teacher model and the student model is the same, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. To distill the knowledge learned from the teacher, two losses are introduced: teacher prediction loss and intermediate teacher feature loss.
[0032] The teacher prediction loss is calculated using the KL divergence between the teacher's predicted probability and the student's predicted probability. Different teacher weights are assigned by calculating the cross-entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows:
[0033]
[0034] In formulas (6) and (7), This represents the probability that the k-th teacher's predicted binding pair is 0. It is the cross-entropy between teacher predictions and actual labels, w k This is the confidence weight of the k-th teacher, where K is the number of teachers, which is 3 here;
[0035] The teachers predicted the following losses:
[0036]
[0037] In formula (8), It is the probability that the student predicts the binding pair to be 0;
[0038] Knowledge is transferred between the intermediate layers of the model. The intermediate teacher feature loss is calculated by matching intermediate features between teachers and students, while also considering the confidence levels of different teachers. The formula for the intermediate teacher feature loss is as follows:
[0039]
[0040] In formula (9), w k It is the confidence weight of the k-th teacher. Represents the L2 loss function;
[0041] The cross-entropy loss between the true label and the prediction in the student model is calculated as follows:
[0042]
[0043] The final overall loss formula for internal multi-feature knowledge distillation is as follows:
[0044]
[0045] In formula (11), α and β represent hyperparameters.
[0046] Furthermore, in step 7, interdomain knowledge distillation captures the features and assemblage patterns of different epitopes by training multiple domain-specific teacher models, thereby enabling student models to better adapt to new and unseen epitopes. Specific steps include:
[0047] Step 701, Domain-Specific Model Training: For each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the features of its corresponding domain, thus learning more detailed and accurate binding patterns.
[0048] Step 702: After the teacher model is trained, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as that of the teacher model. In this way, the student model can integrate knowledge from different domains, thereby improving its prediction ability on unseen epitopes. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross-entropy loss between the true label and the prediction.
[0049] The beneficial effects of this invention are as follows: This invention adopts a random tabletop negative sampling strategy. For each positive TCR-tabletop pair, a tabletop is randomly selected from all tabletops in the current dataset to pair with the TCR, thereby constructing a negative TCR-tabletop pair and ensuring that the pair does not appear in the positive pair. Using this strategy, the same number of negative sampling pairs matching the number of positive pairings are generated. Attached Figure Description
[0050] Figure 1 This is a flowchart of the present invention;
[0051] Figure 2 This is a framework diagram of the present invention;
[0052] Figure 3 This is a performance diagram of MKDBAN-TEI and the baseline model; (a) shows the performance comparison of four evaluation metrics (AUC, precision, recall, F1), and (b) indicates the AUC performance of each of the top 40 most abundant epitopes, with darker colors indicating the frequency of an epitope in the training dataset.
[0053] Figure 4 This is a visualization of the binding between TCR and epitope, where the PDB ID is 4G9F. (a) is the 3D structure of the 4G9F entry in the PDB database. The crystal structure of the complex shows that TCRβG111 and epitope G7 form a direct hydrogen bond. (b) is a heatmap used to visualize the pairwise amino acid distances (in angstroms) between CDR3β and epitope. The smaller the distance, the stronger the interaction. The row below the heatmap shows the minimum distance between each amino acid in CDR3β and the epitope amino acid. (c) is a heatmap visualizing the bilinear attention map. The X and Y axes represent the position and corresponding amino acid in the CDR3β sequence, respectively. Red indicates higher attention in binding prediction, while blue indicates lower attention. Detailed Implementation
[0054] Specific implementation method one: as follows Figure 1 and Figure 2 As shown, the T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation includes the following steps:
[0055] Step 1: Extract TCR-epitope binding pairs from public databases and generate a dataset, then divide it into training and test sets for model training and testing. Extract TCR-epitope binding pairs from public databases to generate a dataset, and randomly divide the dataset into training and test sets in a 4:1 ratio, with 80% as the training set and 20% as the test set.
[0056] Both the training and test sets contain each type of action. The types of actions are encoded with 0 and 1, where 0 represents no binding and 1 represents binding. The above datasets are then subjected to 5-fold cross-validation.
[0057] Step 2: Collect multiple sequence feature information of TCR and epitope for internal multi-feature knowledge distillation; introduce three different protein sequence features: embeddings extracted from the protein language model ESM, Atchley factor and BLOSUM62;
[0058] Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters for inter-domain knowledge distillation; calculate the similarity between epitope A and epitope B using the cosine similarity index, as shown in the formula below:
[0059]
[0060] Step 4: Construct a feature extraction module. A convolutional neural network (CNN) is used to extract features from the TCR and epitope sequences. A learnable amino acid embedding matrix is used to encode the sequences, capturing biological information within them. Two independent CNN layers are used to extract features from the TCR and epitope sequences. Specifically, the embedding matrices of the TCR and epitopes are input into three consecutive 1D convolutional layers. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer captures residue-level features, while the second and third convolutional layers increase the receptive field and extract more abstract features from the TCR and epitope sequences. The structure of the convolutional layers is as follows:
[0061] F (l+1) =σ(f c (W l ,b l ,F l ))(2),
[0062] In formula (2), W l and b l Let F represent the learnable weight matrix and bias vector of the l-th convolutional layer, respectively. l It is the learned representation of the l-th convolutional layer; F 0 It is the encoded embedding matrix or σ(.) represents a non-linear activation function, using the Corrected Linear Unit (ReLU) as the activation function. The output of the feature extraction module is... and in It is the extracted TCR feature, D t It is the potential spatial dimension of TCR. It is the extracted epitope feature, D e It is the potential spatial dimension of the epitope;
[0063] Step 5: Construct a bilinear attention network to explore the interaction patterns between the TCR and epitopes and capture combined features through the bilinear attention mechanism. The bilinear attention network captures pairwise interactions between the TCR and epitopes. After feature extraction, the bilinear attention network has two inputs: and Two learnable weight matrices are introduced, denoted as follows: and And a bilinear attention map, represented as To obtain the joint intermediate representation Where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows:
[0064]
[0065] In formula (3), U k and V k These represent the k-th column of the weight matrix, and They are The i-th column and The j-th column is obtained by using the Hadamard product and matrix multiplication to form a bilinear attention graph. The definition is as follows:
[0066]
[0067] In formula (4), It is a vector consisting entirely of 1s. It is a learnable weight vector. The Hadamard product is represented by the elements of I, which refer to the interaction strength between each TCR-epitope substructure pair.
[0068] By learning learnable weight matrices, named U and V respectively, and a learnable weight vector p, the bilinear attention network can learn pairwise interactions between residues of TCR-epitopes and provide interpretability regarding the contributions of substructure pairs. Summation pooling is applied to the joint intermediate representation to generate the learned features; the summation pooling formula is as follows:
[0069] f = SumPool(f′, h) (5)
[0070] Multiple bilinear interaction graphs are computed by summing and pooling f′ using a one-dimensional non-overlapping window of size h.
[0071] Step 6, Internal Multi-Feature Knowledge Distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features, and this knowledge is transferred to the student model. Three different protein sequence features are introduced: embeddings extracted from the protein language model ESM, Atchley factor, and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode TCR and epitope sequences. The architecture of the teacher model and the student model is the same, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. To distill the knowledge learned from the teachers, two losses are introduced: teacher prediction loss and intermediate teacher feature loss.
[0072] The teacher prediction loss is calculated using the KL divergence between the teacher's predicted probability and the student's predicted probability. Different teacher weights are assigned by calculating the cross-entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows:
[0073]
[0074] In formulas (6) and (7), This represents the probability that the k-th teacher's predicted binding pair is 0. It is the cross-entropy between teacher predictions and actual labels, w k This is the confidence weight of the k-th teacher, where K is the number of teachers, which is 3 here;
[0075] The teachers predicted the following losses:
[0076]
[0077] In formula (8), It is the probability that the student predicts the binding pair to be 0;
[0078] Knowledge is transferred between the intermediate layers of the model. The intermediate teacher feature loss is calculated by matching intermediate features between teachers and students, while also considering the confidence levels of different teachers. The formula for the intermediate teacher feature loss is as follows:
[0079]
[0080] In formula (9), w k It is the confidence weight of the k-th teacher. Represents the L2 loss function;
[0081] The cross-entropy loss between the true label and the prediction in the student model is calculated as follows:
[0082]
[0083] The final overall loss formula for internal multi-feature knowledge distillation is as follows:
[0084]
[0085] In formula (11), α and β represent hyperparameters;
[0086] Step 7, Interdomain Knowledge Distillation: For different epitopes, train domain-specific models and distill their knowledge into a general model to improve the model's generalization ability on unseen epitopes. Interdomain knowledge distillation captures the features and combination patterns of different epitopes by training multiple domain-specific teacher models, enabling student models to better adapt to new and unseen epitopes. Specific steps include:
[0087] Step 701, Domain-Specific Model Training: For each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the features of its corresponding domain, thus learning more detailed and accurate binding patterns.
[0088] Step 702: After the teacher model is trained, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as that of the teacher model. In this way, the student model can integrate knowledge from different domains, thereby improving its prediction ability on unseen epitopes. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross-entropy loss between the true label and the prediction.
[0089] Step 8: Input the TCR-epitope pairs from the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.
[0090] Among them, embedding refers to protein language embedding, Atchley factor refers to physicochemical information, and BLOSUM62 refers to evolutionary information.
[0091] To improve the generalization performance of the model, it is necessary to cluster all epitopes in the training set, thereby training the model for different epitope clusters and performing knowledge distillation. By comparing the similarity of evolutionary information such as the amino acid sequence and BLOSUM62 matrix of the epitopes and calculating the similarity score between epitopes, hierarchical clustering is used to group similar epitopes together to form different epitope clusters.
[0092] The individual graphs are then summed to create the final joint representation. In this implementation, a single pairwise interaction is extended to a two-headed form. The combined representation is then fed into a multilayer perceptron (MLP) layer to predict the binding probability and determine whether TCR-epitope pairs will bind.
[0093] Working principle
[0094] To better illustrate the effects of the method described in this invention, the following embodiments are provided:
[0095] This embodiment collected the training dataset VDJdb+McPAS from VDJdb and McPAS-TCR. Since VDJdb and McPAS-TCR have fewer MHC class II data points, only MHC class I data points were retained. Data points whose hosts are not human were filtered out. Then, the CDR3 sequences of all TCRs were processed to bring them into canonical form by adding missing anchor residues, i.e., C and F / W. Data that could not be processed was discarded. Ultimately, the dataset contained 54,111 known positive TCR-epitope interactions, involving 40,235 TCRs and 1,025 epitopes. Of these, 14.1% of the data came from the McPAS-TCR database and 85.9% from the VDJdb database. The CDR3 length ranged from 4 to 38, with 97.8% of the lengths between 10 and 18. The epitope length ranged from 7 to 20, with 97.5% of the lengths between 8 and 10.
[0096] The collected dataset contains only positive associative pairs. To train a more generalizable model, it is necessary to generate negative samples. Considering that negative sampling strategies may affect model performance, this invention employs a random epitope negative sampling strategy. For each positive TCR-episode pair, a bitope is randomly selected from all epitopes in the current dataset to pair with that TCR, thereby constructing a negative TCR-episode pair, ensuring that this pair does not appear in the positive pairs. Using this strategy, the same number of negative sampled pairs as the number of positive associative pairs are generated.
[0097] Evaluation indicators
[0098] The TCR-epitope binding specificity prediction method designed in this invention is a binary classification problem. This invention uses AUC, precision, recall, and F1 score as evaluation metrics, and the calculation formulas for these metrics are as follows:
[0099] AUC
[0100] AUC stands for Area Under Curve, which reflects the classification ability. Its horizontal axis represents the false positive rate (FPR), and its vertical axis represents the true positive rate (TPR). The closer the AUC is to 1, the better the model results.
[0101] accuracy
[0102] Accuracy measures the proportion of correctly classified samples out of all positive samples. The formula is as follows:
[0103]
[0104] Recall rate
[0105] Recall measures the proportion of correctly classified positive samples out of the actual number of positive samples. The formula is as follows:
[0106]
[0107] The definitions of TP, TN, FP, and FN are shown in Table 1.
[0108] Table 1
[0109]
[0110] F1 Score
[0111] The F1 Score is a metric used to evaluate the performance of classification models, especially when the dataset is imbalanced. It is the harmonic mean of precision and recall, comprehensively considering both the accuracy and completeness of the model. The calculation formula is as follows:
[0112]
[0113] Evaluation of experimental results
[0114] The present invention combines this method (MKDBAN-TEI) with NetTCR-2.0 (A.Montemurro et al.NetTCR-2.0enables accurate prediction of TCR-peptide binding by using paired TCRαandβsequence data), ERGO (Y.Louzoun et al.Prediction of Specific TCR-PeptideBinding From Large Dictionaries of TCR-Peptide Pairs), MIX-TPI (M.Yang et al. alMIX-TPI: a flexible prediction framework for TCR–pMHC interactions based on multimodal representations) for comparison.
[0115] Performance evaluations of MKDBAN-TEI and baseline models on the VDJdb+McPAS dataset are as follows: Figure 3 As shown, MKDBAN-TEI achieved the best results, with an AUROC of 0.779 and an AUPR of 0.794. The second-best baseline model, MIX-TPI, had an AUROC of 0.738 and an AUPR of 0.753. MKDBAN-TEI also achieved the best results in precision (0.729), recall (0.689), and F1 score (0.708), representing improvements of 6.6%, 5.2%, and 2.6% respectively compared to the second-best baseline model. The AUROC performance per epitope for both MKDBAN-TEI and MIX-TPI is also reported. Figure 3 (b) MKDBAN-TEI outperformed MIX-TPI in 34 of the 40 most abundant epitopes.
[0116] MKDBAN-TEI utilizes bilinear attention mapping to determine the contribution of each residue in the TCR and epitope to the final prediction. Visualizing bilinear attention mapping enables molecular-level interpretation and can provide explanations for the model's decision-making process. To demonstrate that MKDBAN-TEI can capture important residues in the TCR and epitope sequences, a resolved pHLA-TCR complex, PDB ID 4G9F, was selected. This complex shows the interaction between the TCR CDR3βCSAREGLGGTEAF and the epitope KRWIIMGLNK. Pairwise amino acid distances (in angstroms) between CDR3β and the epitope were calculated using the PDB 3D structure. Figure 4 (b) The actual TCR-epitope distance matrix was visualized using a heatmap to highlight closely paired amino acids. Figure 4 (c) The learned bilinear attention map was visualized to highlight important residues in the CDR3β sequence. MKDBAN-TEI emphasized the importance of glycine (G) at positions 8 and 9, threonine (T) at position 10, glutamic acid (E) at position 11, and alanine (A) at position 12. By comparing the bilinear attention map and the distance matrix, the distances between the learned important positions in the TCRβ sequence and the epitopes were also very close. Figure 4 As shown in (a), G111 (glycine at position 8 in CDR3β) in the TCRβ chain interacts with epitope G7 via hydrogen bonds, demonstrating that MKDBAN-TEI can effectively capture important residues in the interaction between TCR and epitope.
[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent substitutions, and improvements made to the above embodiments without departing from the scope of the present invention, based on the technical essence of the present invention and within the spirit and principles of the present invention, shall still fall within the protection scope of the present invention.
Claims
1. A method for predicting T-cell receptor-epitope binding specificity based on multi-level knowledge distillation, characterized in that, The specific steps include: Step 1: Extract TCR-tablet binding pairs from public databases, generate datasets, and divide them into training and test sets for model training and testing. Step 2: Collect multiple sequence feature information of TCR and epitope for internal multi-feature knowledge distillation; Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters, which are used for inter-domain knowledge distillation; Step 4: Construct a feature extraction module, use a convolutional neural network to extract features from TCR and epitope sequences, and use a learnable amino acid embedding matrix to encode the sequence and capture the biological information in the sequence; Step 5: Construct a bilinear attention network and explore the interaction pattern between TCR and epitopes through the bilinear attention mechanism to capture the binding features; Step 6, Internal Multi-Feature Knowledge Distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features, and this knowledge is transferred to the student model. A bilinear attention network is used to capture pairwise interactions between TCRs and epitopes. After feature extraction, the bilinear attention network has two inputs, namely... and Two learnable weight matrices are introduced, denoted as follows: and And a bilinear attention map, represented as To obtain the joint intermediate representation , where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows: （3）， In formula (3), and These represent the k-th column of the weight matrix, and They are The i-th column and The j-th column is obtained by using the Hadamard product and matrix multiplication to form a bilinear attention graph. The definition is as follows: （4）， In formula (4), It is a vector consisting entirely of 1s. It is a learnable weight vector. It represents the Hadamah accumulation. The element refers to the interaction strength between each TCR-epitope substructure pair; By learning the learnable weight matrices, they are named respectively. and and a learnable weight vector. Bilinear attention networks can learn pairwise interactions between residues of TCR-epitopes and provide interpretability regarding the contributions of substructures. Summation pooling is applied to the joint intermediate representation to generate the learned features. The summation pooling formula is as follows: （5） Using a one-dimensional non-overlapping window of size h Perform summation pooling to compute multiple bilinear interaction graphs; Step 7, Inter-domain knowledge distillation: For different epitopes, train domain-specific models and distill their knowledge into a general model to improve the model's generalization ability on unseen epitopes; Step 8: Input the TCR-epitope pairs from the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.
2. The T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation according to claim 1, characterized in that, In step 1, TCR-epitope binding pairs are extracted from public databases to generate a dataset. The dataset is then randomly divided into a training set and a test set in a 4:1 ratio, with 80% being the training set and 20% being the test set. Both the training and test sets contain each type of action, which is encoded as 0 and 1, where 0 represents no binding and 1 represents binding. The dataset is then subjected to 5-fold cross-validation.
3. The T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation according to claim 1, characterized in that, In step 2, three different protein sequence features are introduced: embeddings extracted from the protein language model ESM, Atchley factor, and BLOSUM62.
4. The T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation according to claim 1, characterized in that, In step 3, the cosine similarity index is used to calculate the similarity between epitope A and epitope B, as shown in the following formula: （1）。 5. The T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation according to claim 1, characterized in that, In step 4, two independent convolutional neural network layers are used to extract features from the TCR and epitope sequences. Specifically, the embedding matrices of the TCR and epitopes are input into three consecutive 1D convolutional layers. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer is used to capture residue-level features, while the second and third convolutional layers increase the receptive field and extract more abstract features from the TCR and epitope sequences. The structure of the convolutional layers is as follows: （2）， In formula (2), and Let these represent the learnable weight matrix and bias vector of the l-th convolutional layer, respectively. It is the learned representation of the l-th convolutional layer; It is the encoded embedding matrix or , This represents a non-linear activation function, using the Corrected Linear Unit (ReLU) as the activation function. The output of the feature extraction module is... and ,in These are the extracted TCR features. It is the potential spatial dimension of TCR. These are the extracted epitope features. It is the potential spatial dimension of the epitope.
6. The method for predicting T-cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that, In step 6, three different protein sequence features are introduced: embeddings extracted from the protein language model ESM, Atchley factor, and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode TCR and epitope sequences. The teacher model and the student model have the same architecture, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. To distill the knowledge learned from the teacher, two losses are introduced: teacher prediction loss and intermediate teacher feature loss. The teacher prediction loss is calculated using the KL divergence between the teacher's predicted probability and the student's predicted probability. Different teacher weights are assigned by calculating the cross-entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows: （6）， （7）， In formulas (6) and (7), This represents the probability that the k-th teacher's predicted binding pair is 0. It is the cross-entropy between teacher predictions and actual labels. This is the confidence weight of the k-th teacher, where K is the number of teachers, which is 3 here; The teachers predicted the following losses: （8）， In formula (8), It is the probability that the student predicts the binding pair to be 0; Knowledge is transferred between the intermediate layers of the model. The intermediate teacher feature loss is calculated by matching intermediate features between teachers and students, while also considering the confidence levels of different teachers. The formula for the intermediate teacher feature loss is as follows: （9）， In formula (9), It is the confidence weight of the k-th teacher. Represents the L2 loss function; The cross-entropy loss between the true label and the prediction in the student model is calculated as follows: （10）， The final overall loss formula for internal multi-feature knowledge distillation is as follows: （11）， In formula (11), and This represents hyperparameters.
7. The T-cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation according to claim 1, characterized in that, In step 7, interdomain knowledge distillation captures the features and assemblage patterns of different epitopes by training multiple domain-specific teacher models, thereby enabling student models to better adapt to new and unseen epitopes. Specific steps include: Step 701, Domain-Specific Model Training: For each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the features of its corresponding domain, thus learning more detailed and accurate binding patterns. Step 702: After the teacher model is trained, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as that of the teacher model. In this way, the student model can integrate knowledge from different domains, thereby improving its prediction ability on unseen epitopes. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross-entropy loss between the true label and the prediction.