T cell receptor-epitope binding specificity prediction method based on multilevel knowledge distillation

Abstract

The invention discloses a T cell receptor-epitope binding specificity prediction method based on multilevel knowledge distillation, and relates to a T cell receptor-epitope binding specificity prediction method. The method aims to solve the problem that an existing deep learning model is poor in performance when generalized to be free of epitopes. The feature extraction module is constructed by using the convolutional neural network, and the interaction mode between the TCR and the epitope is explored through a bilinear attention mechanism. Thirdly, internal multi-feature knowledge distillation and inter-domain knowledge distillation are carried out, knowledge is extracted from multiple teacher models, and the predictive ability of student models is enhanced; tCR-epitope pairs in a training set are input into the model, and the model is trained; after multiple iterations, a final prediction model is obtained; and inputting the TCR-epitope pair in the test set into the final prediction model to obtain a prediction result, and performing result analysis. The invention belongs to the technical field of biological information.

Description

Technical Field

[0001] The invention relates to a method for predicting T cell receptor-epitope binding specificity, and belongs to the field of bioinformatics technology. Background Art

[0002] Understanding the interaction mechanism between T cell receptors (TCRs) and epitopes is essential for studying the adaptive immune system and promoting the development of immunotherapy. Traditional experimental methods for identifying TCR-epitope binding pairs are costly and time-consuming. With the development of high-throughput sequencing technology, more and more TCR-epitope binding pair data are archived in public databases such as VDJdb, IEDB, and McPAS-TCR, providing opportunities for the development of computational methods. Therefore, it is particularly important to develop computational methods to predict TCR-epitope binding specificity.

[0003] Machine learning and deep learning methods can quickly capture the potential patterns of TCR-epitope binding in large amounts of experimental data, so more and more related methods have been developed to predict TCR binding to epitopes. Some studies have attempted to predict TCR specificity by extracting biological information from TCR sequences alone. For example, DeepTCR is an unsupervised and supervised deep learning method that models highly complex TCR sequencing data by learning a joint representation of TCR using CDR3 sequences and V / D / J genes. Other methods aim to develop binding prediction models for specific peptides, such as clustering methods, decision trees, variational autoencoders, Gaussian processes, and deep walks. These methods typically train a specific model for each epitope and require a large amount of relevant training data. However, many practical scenarios require prediction of new epitopes, which often lack relevant binding data, limiting the application of these methods. Recently, more and more researchers have devoted themselves to developing universal models that can predict the binding of any TCR-epitope pair, such as NetTCR-2.0, a convolutional neural network for predicting TCR-peptide binding; ERGO, a natural language processing (NLP) method that combines a large-scale TCR-peptide dictionary with deep learning methods.

[0004] Although current research has made significant progress in predicting TCR-epitope binding, these methods are limited by insufficient interaction data. Due to the high cost and limitations of experimental methods, currently available databases are limited to a small part of the potential sample space of TCR-epitope binding pairs. Recent studies have shown that existing deep learning models perform poorly when generalizing to unseen epitopes, and their performance depends heavily on the balance and scale of the data. Therefore, the performance of the model still needs further improvement in terms of generalization and robustness. In addition, the interpretability of the model is equally important in the field of healthcare. The interpretation of the model results can help researchers guide their research and is crucial to the development of trustworthy systems. Summary of the invention

[0005] In order to solve the problem that existing deep learning models perform poorly when generalizing to unseen epitopes, the present invention proposes a T cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation.

[0006] The technical solution adopted by the present invention to solve the above-mentioned problem is: the steps of the present invention include:

[0007] Step 1: Extract TCR-epitope binding pair data from the public database, generate a data set, and divide it into training and test sets to train and test the model;

[0008] Step 2: Collect multiple sequence feature information of TCR and epitopes for internal multi-feature knowledge distillation;

[0009] Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters for inter-domain knowledge distillation;

[0010] Step 4: Construct a feature extraction module, use a convolutional neural network to extract features from TCR and epitope sequences, and use a learnable amino acid embedding matrix to encode the sequences to capture the biological information in the sequences;

[0011] Step 5: Build a bilinear attention network to explore the interaction pattern between TCR and epitope and capture the binding features through the bilinear attention mechanism;

[0012] Step 6: Internal multi-feature knowledge distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features and this knowledge is transferred to the student model.

[0013] Step 7: Distillation of inter-domain knowledge: For different epitopes, domain-specific models are trained and their knowledge is distilled into a general model to improve the generalization ability of the model on unseen epitopes.

[0014] Step 8: Input the TCR-epitope pairs in the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.

[0015] Furthermore, in step 1, data of TCR-epitope binding pairs are extracted from a public database to generate a data set, and the data set is randomly divided into a training set and a test set in a ratio of 4:1, wherein 80% is a training set and 20% is a test set;

[0016] Both the training set and the test set contain each action type, and the action type is encoded as 0 and 1, where 0 indicates no binding and 1 indicates binding. The above data set is subjected to 5-fold cross-validation.

[0017] Furthermore, in step 2, three different protein sequence features are introduced: embedding extracted from protein language model ESM, Atchley factor and BLOSUM62.

[0018] Furthermore, in step 3, the cosine similarity index is used to calculate the similarity between epitope A and epitope B, and the formula is as follows:

[0019]

[0020] Furthermore, in step 4, two independent convolutional neural network layers are used to extract features from TCR and epitope sequences. Specifically, the embedding matrices of TCR and epitope are input into three consecutive 1D convolutional layers. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer is used to capture residue-level features. The second and third convolutional layers increase the receptive field and extract more abstract features from TCR and epitope sequences. The structure of the convolutional layer is as follows:

[0021] F (l+1) =σ(f c (W l ,b l ,F l )) (2),

[0022] In formula (2), W l and b l Represent the learnable weight matrix and bias vector of the lth convolutional layer, F l is the learned representation of the lth convolutional layer; F 0 is the encoded embedding matrix or σ(.) represents a nonlinear activation function. Rectified linear unit (ReLU) is used as the activation function. The output of the feature extraction module is and in is the extracted TCR feature, D t is the latent space dimension of TCR, is the extracted epitope feature, D e is the latent spatial dimension of the epitope.

[0023] Furthermore, in step 5, a bilinear attention network is used to capture the pairwise interactions between TCR and epitopes. After feature extraction, the bilinear attention network has two inputs, namely and Two learnable weight matrices are introduced, represented as and and a bilinear attention graph, represented as To obtain the joint intermediate representation Where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows:

[0024]

[0025] In formula (3), U k and V k represent the kth column of the weight matrix, and They are The i-th column and By using the Hadamard product and matrix multiplication, the bilinear attention map The definition is as follows:

[0026]

[0027] In formula (4), is a vector of all 1s, is a learnable weight vector, represents the Hadamard product, and the elements of I refer to the interaction strength between each TCR-epitope substructure pair;

[0028] By learning learnable weight matrices, named U and V, and a learnable weight vector p, the bilinear attention network can learn pairwise interactions between residues of TCR-epitopes and provide interpretability about the contributions of substructure pairs; sum pooling is applied to the joint intermediate representation to generate the learned features. The sum pooling formula is as follows:

[0029] f = SumPool(f′, h) (5)

[0030] f′ is summed and pooled using one-dimensional non-overlapping windows of size h to compute multiple bilinear interaction graphs.

[0031] Furthermore, in step 6, three different protein sequence features are introduced: embedding extracted from the protein language model ESM, Atchley factor and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode the TCR and epitope sequences. The architecture of the teacher model and the student model is the same, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. In order to distill the knowledge learned from the teacher, two losses are introduced, the teacher prediction loss and the intermediate teacher feature loss.

[0032] The teacher prediction loss calculates the KL divergence between the teacher's prediction probability and the student's prediction probability, and assigns different teacher weights by calculating the cross entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows:

[0033]

[0034] In formulas (6) and (7), represents the probability that the kth teacher predicts the binding pair to be 0, is the cross entropy between the teacher prediction and the true label, w k is the confidence weight of the kth teacher, K is the number of teachers, which is 3 here;

[0035] The teacher prediction loss is as follows:

[0036]

[0037] In formula (8), is the probability that the student predicts that the binding pair is 0;

[0038] Transferring knowledge between the intermediate layers of the model, the intermediate teacher feature loss calculates the intermediate feature matching between the teacher and the student while taking into account the confidence of different teachers. The formula for the intermediate teacher feature loss is as follows:

[0039]

[0040] In formula (9), w k is the confidence weight of the kth teacher, represents the L2 loss function;

[0041] The cross entropy loss between the true labels and predictions of the student model is calculated as follows:

[0042]

[0043] The final overall loss formula for internal multi-feature knowledge distillation is as follows:

[0044]

[0045] In formula (11), α and β represent hyperparameters.

[0046] Furthermore, in step 7, inter-domain knowledge distillation captures the characteristics and binding patterns of different epitopes by training multiple domain-specific teacher models, so that the student model can better adapt to new and unseen epitopes. The specific steps include:

[0047] Step 701, domain-specific model training: for each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the characteristics of its corresponding domain, and can learn more detailed and accurate binding patterns;

[0048] Step 702: After the teacher model training is completed, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as the teacher model. In this way, the student model can integrate knowledge from different fields, thereby improving its prediction ability on unseen surfaces. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross-entropy loss between true labels and predictions.

[0049] The beneficial effects of the present invention are as follows: the present invention adopts a random epitope negative sampling strategy. For each TCR in a positive TCR-epitope pair, an epitope is randomly selected from all epitopes in the current data set to pair with the TCR, thereby constructing a negative TCR-epitope pair and ensuring that the pair does not appear in the positive pair. The strategy is used to generate the same number of negative sampling pairs that match the number of positive binding pairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the present invention;

[0051] Figure 2 It is a framework diagram of the present invention;

[0052] Figure 3 Schematic diagram of the performance of MKDBAN-TEI and the baseline model; (a) shows the performance comparison of four evaluation indicators (AUC, precision, recall, F1), (b) indicates the AUC performance of each epitope of the top 40 most abundant epitopes, the more frequently the epitope appears in the training dataset, the darker the color;

[0053] Figure 4 is a visualization diagram of the binding between TCR and epitope, where the PDB ID is 4G9F, (a) is the 3D structure of the 4G9F entry in the PDB database, and the crystal structure of the complex shows that TCRβG111 and epitope G7 form a direct hydrogen bond, (b) is a heat map for visualizing the pairwise amino acid distance (in angstroms) between CDR3β and the epitope, the smaller the distance, the stronger the interaction, and the row below the heat map shows the minimum distance between each amino acid in CDR3β and the epitope amino acid, (c) is a heat map for visualizing the bilinear attention map, the X-axis and Y-axis represent the position in the CDR3β sequence and the corresponding amino acid, respectively, red indicates higher attention in binding prediction, while blue indicates lower attention. DETAILED DESCRIPTION

[0054] Specific implementation method 1: Figure 1 and Figure 2 As shown in FIG. 1 , a T cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation includes the following specific steps:

[0055] Step 1: Extract TCR-epitope binding pair data from a public database, generate a data set, and divide it into training and test sets to train and test the model; extract TCR-epitope binding pair data from a public database to generate a data set, and randomly divide the data set into a training set and a test set in a ratio of 4:1, of which 80% is a training set and 20% is a test set;

[0056] Both the training set and the test set contain each action type, and the action type is coded as 0 and 1, where 0 means no binding and 1 means binding. The above data sets are subjected to 5-fold cross-validation.

[0057] Step 2: Collect multiple sequence feature information of TCR and epitopes for internal multi-feature knowledge distillation; introduce three different protein sequence features: embedding extracted from protein language model ESM, Atchley factor and BLOSUM62;

[0058] Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters for inter-domain knowledge distillation; use the cosine similarity index to calculate the similarity between epitope A and epitope B. The formula is as follows:

[0059]

[0060] Step 4: Construct a feature extraction module, use a convolutional neural network to extract features from TCR and epitope sequences, use a learnable amino acid embedding matrix to encode the sequences and capture the biological information in the sequences; use two independent convolutional neural network layers to extract features from TCR and epitope sequences. Specifically, the embedding matrices of TCR and epitope are input into three consecutive 1D convolutional layers, respectively. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer is used to capture residue-level features. The second and third convolutional layers increase the receptive field and extract more abstract features from the TCR and epitope sequences. The structure of the convolutional layer is as follows:

[0061] F (l+1) =σ(f c (W l ,b l ,F l ))(2),

[0062] In formula (2), W l and b l Represent the learnable weight matrix and bias vector of the lth convolutional layer, F l is the learned representation of the lth convolutional layer; F 0 is the encoded embedding matrix or σ(.) represents a nonlinear activation function. Rectified linear unit (ReLU) is used as the activation function. The output of the feature extraction module is and in is the extracted TCR feature, D t is the latent space dimension of TCR, is the extracted epitope feature, D e is the potential spatial dimension of the epitope;

[0063] Step 5: Build a bilinear attention network. Through the bilinear attention mechanism, explore the interaction pattern between TCR and epitope and capture the binding features. Use the bilinear attention network to capture the pairwise interaction between TCR and epitope. After feature extraction, the bilinear attention network has two inputs, namely and Two learnable weight matrices are introduced, represented as and and a bilinear attention graph, represented as To obtain the joint intermediate representation Where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows:

[0064]

[0065] In formula (3), U k and V k represent the kth column of the weight matrix, and They are The i-th column and By using the Hadamard product and matrix multiplication, the bilinear attention map The definition is as follows:

[0066]

[0067] In formula (4), is a vector of all 1s, is a learnable weight vector, represents the Hadamard product, and the elements of I refer to the interaction strength between each TCR-epitope substructure pair;

[0068] By learning learnable weight matrices, named U and V, and a learnable weight vector p, the bilinear attention network can learn pairwise interactions between residues of TCR-epitopes and provide interpretability about the contributions of substructure pairs; sum pooling is applied to the joint intermediate representation to generate the learned features. The sum pooling formula is as follows:

[0069] f = SumPool(f′, h) (5)

[0070] Use a one-dimensional non-overlapping window of size h to sum and pool f′ and calculate multiple bilinear interaction graphs;

[0071] Step 6: Internal multi-feature knowledge distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features, and this knowledge is transferred to the student model; three different protein sequence features are introduced: embedding extracted from the protein language model ESM, Atchley factor and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode TCR and epitope sequences. The architecture of the teacher model and the student model is the same, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. In order to distill the knowledge learned from the teacher, two losses are introduced: teacher prediction loss and intermediate teacher feature loss;

[0072] The teacher prediction loss calculates the KL divergence between the teacher's prediction probability and the student's prediction probability, and assigns different teacher weights by calculating the cross entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows:

[0073]

[0074] In formulas (6) and (7), represents the probability that the kth teacher predicts the binding pair to be 0, is the cross entropy between the teacher prediction and the true label, w k is the confidence weight of the kth teacher, K is the number of teachers, which is 3 here;

[0075] The teacher prediction loss is as follows:

[0076]

[0077] In formula (8), is the probability that the student predicts that the binding pair is 0;

[0078] Transferring knowledge between the intermediate layers of the model, the intermediate teacher feature loss calculates the intermediate feature matching between the teacher and the student while taking into account the confidence of different teachers. The formula for the intermediate teacher feature loss is as follows:

[0079]

[0080] In formula (9), w k is the confidence weight of the kth teacher, represents the L2 loss function;

[0081] The cross entropy loss between the true labels and predictions of the student model is calculated as follows:

[0082]

[0083] The final overall loss formula for internal multi-feature knowledge distillation is as follows:

[0084]

[0085] In formula (11), α and β represent hyperparameters;

[0086] Step 7: Inter-domain knowledge distillation: For different epitopes, domain-specific models are trained and their knowledge is distilled into a general model to improve the generalization ability of the model on unseen epitopes. Inter-domain knowledge distillation captures the characteristics and binding patterns of different epitopes by training multiple domain-specific teacher models, so that the student model can better adapt to new and unseen epitopes. The specific steps include:

[0087] Step 701, domain-specific model training: for each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the characteristics of its corresponding domain, and can learn more detailed and accurate binding patterns;

[0088] Step 702: After the teacher model training is completed, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as that of the teacher model. In this way, the student model can integrate knowledge from different fields, thereby improving its prediction ability on unseen epitopes. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross entropy loss between true labels and predictions.

[0089] Step 8: Input the TCR-epitope pairs in the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.

[0090] Among them, embedding refers to protein language embedding, Atchley factor refers to physicochemical information, and BLOSUM62 refers to evolutionary information.

[0091] In order to improve the generalization performance of the model, it is necessary to cluster all epitopes in the training set, so as to train the model for different epitope clusters and perform knowledge distillation. By comparing the similarity of the amino acid sequence of the epitope and the evolutionary information such as the BLOSUM62 matrix, and calculating the similarity scores between epitopes, hierarchical clustering is used to cluster similar epitopes together to form different epitope clusters.

[0092] The individual graphs are then summed to create the final joint representation. In this embodiment, the single pairwise interaction is expanded into a two-headed form. The combined representation is then input to a multi-layer perceptron (MLP) layer to predict the binding probability and determine whether the TCR-epitope pair will bind.

[0093] How it works

[0094] In order to better illustrate the effect of the method of the present invention, the following examples are described:

[0095] This embodiment collects the training data set VDJdb+McPAS from VDJdb and McPAS-TCR. Since VDJdb and McPAS-TCR have fewer MHC class II data points, only MHC class I data points are retained. Data points where the host does not belong to humans are filtered out. Then, the CDR3 sequences of all TCRs are processed to be in a canonical form by adding missing anchor residues, i.e., C and F / W. Data that cannot be processed will be discarded. Ultimately, the data set contains 54,111 known TCR-epitope positive interactions, involving 40,235 TCRs and 1,025 epitopes. Among them, 14.1% of the data come from the McPAS-TCR database and 85.9% from the VDJdb database. The length of CDR3 ranges from 4 to 38, of which 97.8% are between 10 and 18. The length of the epitope ranges from 7 to 20, of which 97.5% are between 8 and 10.

[0096] The collected dataset contains only positive binding pairs. In order to train a more generalized model, it is necessary to generate negative samples. Considering that the negative sampling strategy may affect the performance of the model, the present invention adopts a random epitope negative sampling strategy. For each TCR in a positive TCR-epitope pair, an epitope is randomly selected from all epitopes in the current dataset to pair with the TCR, thereby constructing a negative TCR-epitope pair and ensuring that the pair does not appear in the positive pair. Using this strategy, the same number of negative sampling pairs matching the number of positive binding pairs are generated.

[0097] Evaluation indicators

[0098] The TCR-epitope binding specificity prediction method designed by the present invention is a binary classification problem. In the present invention, AUC, accuracy, recall rate and F1 Score are used as evaluation indicators. The calculation formulas of these indicators are as follows:

[0099] AUC

[0100] AUC stands for Area under curve, which is the area under the ROC curve. ROC can reflect the classification ability. Its horizontal axis is the false positive rate (FPR) and the vertical axis is the true positive rate (TPR). The closer the AUC is to 1, the better the model result.

[0101] Accuracy

[0102] The accuracy is used to measure the proportion of correctly classified samples among all positive samples. The calculation formula is as follows:

[0103]

[0104] Recall

[0105] The recall rate is used to measure the proportion of correctly classified positive samples to actual positive samples. The calculation formula is as follows:

[0106]

[0107] The definitions of TP, TN, FP, and FN are shown in Table 1.

[0108] Table 1

[0109]

[0110] F1 Score

[0111] F1 Score is an indicator used to evaluate the performance of classification models, especially when the data set is imbalanced. It is the harmonic mean of precision and recall, which can comprehensively consider the accuracy and completeness of the model. The calculation formula is as follows:

[0112]

[0113] Experimental results evaluation

[0114] The present invention combines this method (MKDBAN-TEI) with NetTCR-2.0 (A.Montemurro et al.NetTCR-2.0enables accurate prediction of TCR-peptide binding by using paired TCRαandβsequence data), ERGO (Y.Louzoun et al.Prediction of Specific TCR-PeptideBinding From Large Dictionaries of TCR-Peptide Pairs), MIX-TPI (M.Yang et al. alMIX-TPI: a flexible prediction framework for TCR–pMHC interactions based on multimodal representations) for comparison.

[0115] The performance evaluation of MKDBAN-TEI and baseline models on the VDJdb+McPAS dataset is shown in Figure 3 As shown. MKDBAN-TEI achieved the best result with an AUROC of 0.779 and an AUPR of 0.794. The second best baseline model MIX-TPI achieved an AUROC of 0.738 and an AUPR of 0.753. MKDBAN-TEI also achieved the best result in precision (Precision) of 0.729, recall (Recall) of 0.689, and F1 value of 0.708, which are 6.6%, 5.2%, and 2.6% higher than the second best baseline model, respectively. The AUROC performance of each epitope of MKDBAN-TEI and MIX-TPI is also reported. According to Figure 3 (b), MKDBAN-TEI outperformed MIX-TPI in 34 of the 40 most abundant epitopes.

[0116] MKDBAN-TEI uses a bilinear attention graph to determine the contribution of each residue in the TCR and epitope to the final prediction result. Visualizing the bilinear attention graph enables molecular-level interpretation and can provide explanations for the model's decision-making process. To demonstrate that MKDBAN-TEI can capture important residues in TCR and epitope sequences, a solved pHLA-TCR complex with PDB ID 4G9F was selected. The complex shows the interaction between TCR CDR3βCSAREGLGGTEAF and epitope KRWIIMGLNK. The pairwise amino acid distances (in angstroms) between CDR3β and the epitope were calculated using the PDB 3D structure. Figure 4 (b) The true TCR-epitope inter-residue distance matrix is ​​visualized using a heat map to highlight close pairs of amino acids. Figure 4 (c) Visualization of the learned bilinear attention graph to highlight the important residues in the CDR3β sequence. MKDBAN-TEI emphasizes the importance of glycine (G) at positions 8 and 9, threonine (T) at position 10, glutamate (E) at position 11, and alanine (A) at position 12. By comparing the bilinear attention graph and the distance matrix, the learned important positions in the TCRβ sequence are also close to the epitope. Figure 4 As shown in (a), G111 in the TCRβ chain (glycine at position 8 in CDR3β) interacts with epitope G7 through hydrogen bonds, demonstrating that MKDBAN-TEI can effectively capture important residues in the interaction between TCR and epitope.

[0117] The above is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement made to the above embodiments without departing from the content of the technical solution of the present invention, based on the technical essence of the present invention, within the spirit and principles of the present invention, still fall within the protection scope of the technical solution of the present invention.

Claims

1. A T cell receptor-epitope binding specificity prediction method based on multi-level knowledge distillation, characterized in that: The specific steps include: Step 1: Extract TCR-epitope binding pair data from the public database, generate a data set, and divide it into training and test sets to train and test the model; Step 2: Collect multiple sequence feature information of TCR and epitopes for internal multi-feature knowledge distillation; Step 3: Cluster all epitopes in the training set to obtain multiple epitope clusters for inter-domain knowledge distillation; Step 4: Construct a feature extraction module, use a convolutional neural network to extract features from TCR and epitope sequences, and use a learnable amino acid embedding matrix to encode the sequences to capture the biological information in the sequences; Step 5: Build a bilinear attention network to explore the interaction pattern between TCR and epitope and capture the binding features through the bilinear attention mechanism; Step 6: Internal multi-feature knowledge distillation: By training multiple teacher models, biologically meaningful binding patterns are extracted from different features and this knowledge is transferred to the student model. Step 7: Distillation of inter-domain knowledge: For different epitopes, domain-specific models are trained and their knowledge is distilled into a general model to improve the generalization ability of the model on unseen epitopes. Step 8: Input the TCR-epitope pairs in the test set into the prediction model, and use five-fold cross-validation to evaluate the model performance to ensure the robustness and generalization ability of the model.

2. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 1, data of TCR-epitope binding pairs are extracted from a public database to generate a data set, and the data set is randomly divided into a training set and a test set in a ratio of 4:1, with 80% for the training set and 20% for the test set; Both the training set and the test set contain each action type, and the action type is encoded as 0 and 1, where 0 indicates no binding and 1 indicates binding. The above data set is subjected to 5-fold cross-validation.

3. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 2, three different protein sequence features are introduced: embedding extracted from the protein language model ESM, Atchley factor and BLOSUM62.

4. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 3, the cosine similarity index is used to calculate the similarity between epitope A and epitope B. The formula is as follows:

5. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 4, two independent convolutional neural network layers are used to extract features from TCR and epitope sequences. Specifically, the embedding matrices of TCR and epitope are input into three consecutive 1D convolutional layers. The convolutional layers extract local residue patterns from the embedding matrices. The first convolutional layer is used to capture residue-level features. The second and third convolutional layers increase the receptive field and extract more abstract features from TCR and epitope sequences. The structure of the convolutional layer is as follows: F (l+1) =σ(f c (W l ,b l ,F l )) (2), In formula (2), W l and b l Represent the learnable weight matrix and bias vector of the lth convolutional layer, F l is the learned representation of the lth convolutional layer; F 0 is the encoded embedding matrix or σ(.) represents a nonlinear activation function. Rectified linear unit (ReLU) is used as the activation function. The output of the feature extraction module is and in is the extracted TCR feature, D t is the latent space dimension of TCR, is the extracted epitope feature, D e is the latent spatial dimension of the epitope.

6. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 5, a bilinear attention network is used to capture the pairwise interactions between TCR and epitopes. After feature extraction, the bilinear attention network has two inputs, namely and Two learnable weight matrices are introduced, represented as and and a bilinear attention graph, represented as To obtain the joint intermediate representation Where K is the dimension of the intermediate representation, and the Kth element of the intermediate representation is calculated as follows: In formula (3), U k and V k represent the kth column of the weight matrix, and They are The i-th column and By using the Hadamard product and matrix multiplication, the bilinear attention map The definition is as follows: In formula (4), is a vector of all 1s, is a learnable weight vector, ο represents the Hadamard product, and the elements of I refer to the interaction strength between each TCR-epitope substructure pair; By learning learnable weight matrices, named U and V, and a learnable weight vector p, the bilinear attention network can learn pairwise interactions between residues of TCR-epitopes and provide interpretability about the contributions of substructure pairs; sum pooling is applied to the joint intermediate representation to generate the learned features. The sum pooling formula is as follows: f = SumPool(f′, h) (5) f′ is summed and pooled using one-dimensional non-overlapping windows of size h to compute multiple bilinear interaction graphs.

7. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 6, three different protein sequence features are introduced: embedding extracted from the protein language model ESM, Atchley factor and BLOSUM62. The initial model is called the student model, which only uses the learnable residue embedding matrix to encode the TCR and epitope sequences. The architecture of the teacher model and the student model is the same, but the difference lies in the input features. Different teacher models use different protein sequence features to learn TCR-epitope interactions from different perspectives. To distill the knowledge learned from the teacher, two losses are introduced: teacher prediction loss and intermediate teacher feature loss. The teacher prediction loss calculates the KL divergence between the teacher's prediction probability and the student's prediction probability, and assigns different teacher weights by calculating the cross entropy loss between the teacher's prediction and the true label. The specific calculation formula is as follows: In formulas (6) and (7), represents the probability that the kth teacher predicts the binding pair to be 0, is the cross entropy between the teacher prediction and the true label, w k is the confidence weight of the kth teacher, K is the number of teachers, which is 3 here; The teacher predicts the loss as follows: In formula (8), is the probability that the student predicts that the binding pair is 0; Transferring knowledge between the intermediate layers of the model, the intermediate teacher feature loss calculates the intermediate feature matching between the teacher and the student while taking into account the confidence of different teachers. The formula for the intermediate teacher feature loss is as follows: In formula (9), w k is the confidence weight of the kth teacher, represents the L2 loss function; The cross entropy loss between the true labels and predictions of the student model is calculated as follows: The final overall loss formula for internal multi-feature knowledge distillation is as follows: In formula (11), α and β represent hyperparameters.

8. The method for predicting T cell receptor-epitope binding specificity based on multi-level knowledge distillation according to claim 1, characterized in that: In step 7, inter-domain knowledge distillation captures the characteristics and binding patterns of different epitopes by training multiple domain-specific teacher models, so that the student model can better adapt to new and unseen epitopes. The specific steps include: Step 701, domain-specific model training: for each epitope cluster, multiple domain-specific teacher models are trained using internal multi-feature knowledge distillation to capture the biological characteristics and binding patterns of different epitopes; each teacher model focuses on the characteristics of its corresponding domain, and can learn more detailed and accurate binding patterns; Step 702: After the teacher model training is completed, the knowledge of these domain-specific teacher models is transferred to a general student model. The architecture of the student model is the same as the teacher model. In this way, the student model can integrate knowledge from different fields, thereby improving its prediction ability on unseen surfaces. The loss of inter-domain knowledge distillation also includes teacher prediction loss, intermediate teacher feature loss, and cross-entropy loss between true labels and predictions.

Images