B cell epitope prediction method, device, equipment and computer program product based on protein language model

Abstract

The present application discloses a method, apparatus, device and computer program product for predicting B cell epitopes based on a protein language model. The method comprises: inputting the protein sequence to be predicted into a pre-trained ESM-IF1 model and a pre-trained ESM2 model respectively to capture the multi-dimensional structural embedding vector and sequence feature vector, and then performing splicing; based on the Transformer architecture, training the multi-dimensional data set by integrating a convolutional neural network and calculating the TIM loss function; using a deep convolutional neural network model to output the predicted probability of each amino acid position in the protein sequence; if the predicted probability is greater than a set threshold, the corresponding amino acid position is determined to be an epitope, otherwise it is a non-epitope. The present application achieves deep mining and efficient learning of the nonlinear characteristics of B cell epitopes by effectively integrating the vectors output by the protein language model and the organic fusion of the convolutional neural network and the Transformer, with the advantages of high prediction accuracy, strong generalization ability and good robustness.

Description

Technical Field

[0001] The present application relates to the field of biomedical technology, and in particular to a method, apparatus, device, and computer program product for predicting B cell epitopes based on a protein language model. Background Art

[0002] With the rapid development of artificial intelligence (AI), particularly breakthroughs in deep learning and large-scale models, the biomedical field is undergoing profound changes. B cell epitopes, defined as regions on antigen molecules that are specifically recognized by B cell receptors (BCRs) or antibodies, play a key role in immune responses. Generally speaking, B cell epitopes are categorized as linear (continuous) and conformational (discontinuous). Linear epitopes consist of continuous sequences of residues, while conformational epitopes are composed of residues that are discontinuous in the primary protein sequence but clustered together due to protein folding. It is estimated that the vast majority of B cell epitopes are conformational, with linear epitopes accounting for only a small fraction. Currently, epitope identification relies primarily on experimental methods, a process that is often time-consuming and costly. Therefore, the accurate prediction of B cell epitopes using computational methods is of great significance for vaccine design, antibody drug development, and immunodiagnostics.

[0003] Traditional epitope prediction methods have exposed many limitations when processing biomedical data. On the one hand, many tools rely only on sequence features or simple artificially designed features, which makes it difficult to fully capture the complex characteristics of epitopes, especially when identifying conformational epitopes. Moreover, these methods often have high false positive and false negative rates when predicting across pathogens or novel antigens, and their accuracy and robustness need to be improved. On the other hand, epitope recognition is not only affected by sequence information, but is also closely related to three-dimensional structure and functional properties. However, existing tools have limited ability to identify complex epitopes. More importantly, the prediction results of existing tools often lack biological interpretability, and it is difficult to clearly reveal which features or regions play a key role in the prediction process, limiting their application value in vaccine design and experimental verification.

[0004] The emergence of deep learning and large models offers new solutions to these challenges. Deep learning can automatically learn feature representations from massive amounts of data, reducing the workload of manual feature engineering and effectively improving the efficiency and accuracy of feature extraction. Large models, with their powerful representational and learning capabilities, demonstrate unique advantages in processing complex biomedical data. For example, AlphaFold 2 has demonstrated the ability to extract rich information from protein sequences alone, which is of great significance for protein feature engineering.

[0005] However, B cell epitope prediction based on deep learning and large models still faces some urgent problems. Current B cell epitope prediction methods are mostly based on sequence features, structural features or machine learning algorithms, but they still have shortcomings in prediction accuracy and generalization ability. First, the complexity and diversity of biomedical data increase the difficulty of model training, and more effective data preprocessing and model optimization methods are urgently needed. Second, the interpretability problem of large models hinders their widespread application in the biomedical field. Therefore, developing more transparent and interpretable model architectures and algorithms to achieve a more comprehensive and in-depth understanding of antigen-antibody interactions has become one of the key challenges of current research. Summary of the Invention

[0006] The present application aims to provide a B cell epitope prediction method, apparatus, device, and computer program product based on a protein language model to address the technical issues in the prior art, where existing B cell epitope prediction methods still lack accuracy and generalization capabilities. The various technical effects of the preferred technical solutions provided in this application are detailed below.

[0007] To achieve the above objectives, this application provides the following technical solutions:

[0008] In a first aspect, the present application provides a B cell epitope prediction method based on a protein language model, comprising:

[0009] The protein sequence to be predicted is input into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively to capture the multi-dimensional structure embedding vector and sequence feature vector;

[0010] splicing the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set;

[0011] Based on the Transformer architecture, the multidimensional data set is trained by integrating a convolutional neural network, and a TIM loss function is calculated. When the value of the TIM loss function is within a preset range, a deep convolutional neural network model is obtained;

[0012] Outputting the predicted probability of each amino acid position in the protein sequence using the deep convolutional neural network model;

[0013] If the predicted probability is greater than a set threshold, the corresponding amino acid position is determined to be an epitope, otherwise it is determined to be a non-epitope.

[0014] In some embodiments, the protein sequence to be predicted is input into a pre-trained ESM-IF1 model to obtain a multi-dimensional structure embedding vector, including:

[0015] Inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model, so that the pre-trained ESM-IF1 model learns the geometric features in the atomic coordinates of the protein main chain and characterizes each residue in the protein sequence;

[0016] A multi-dimensional structure embedding vector is obtained based on the geometric features and the feature vector of each residue.

[0017] In some embodiments, the protein sequence to be predicted is input into the pre-trained ESM2 model for prediction to obtain a multi-dimensional sequence feature vector, including:

[0018] Inputting the protein sequence to be predicted into the pre-trained ESM2 model, so that the ESM2 model can capture the local sequence information and global sequence information of the protein sequence and generate a corresponding embedding vector for each amino acid;

[0019] A multi-dimensional sequence feature vector is obtained based on the local sequence information, the global sequence information and the embedding vector corresponding to each amino acid.

[0020] In some embodiments, the concatenating the sequence feature vector and the structure embedding vector to obtain a multi-dimensional dataset comprises:

[0021] The sequence feature vector and the structure embedding vector are concatenated along the last dimension to generate the multidimensional dataset, where the number of dimensions of the multidimensional dataset is the sum of the number of dimensions of the sequence feature vector and the structure embedding vector, and the multidimensional dataset is a three-dimensional tensor whose shape is based on the batch size, sequence length, and feature dimension.

[0022] In some embodiments, the training of the multi-dimensional dataset by integrating a convolutional neural network based on a Transformer architecture includes:

[0023] Inputting the multidimensional data set into the Transformer architecture, and using scaled dot product attention through the Transformer architecture to obtain the final output result of the multi-head attention as the global feature;

[0024] Inputting the multidimensional data set into the convolutional neural network, and outputting the convolution result as a convolution feature through the residual block of the convolutional neural network;

[0025] Training is performed based on the global features and the convolutional features.

[0026] In some embodiments, calculating the TIM loss function includes:

[0027] Based on the multidimensional data set and the labels corresponding to the multidimensional data set, determining the empirical conditional entropy and the empirical marginal entropy of the labels, and calculating the cross entropy;

[0028] The TIM loss function is calculated based on the empirical conditional entropy, the empirical marginal entropy and the cross entropy.

[0029] In some embodiments, outputting the predicted probability of each amino acid position using the deep convolutional neural network model comprises:

[0030] The output of the fully connected layer of the deep convolutional neural network model is calculated using the following formula:

[0031] ;

[0032] in, represents the output of the fully connected layer of the deep convolutional neural network model, W represents the weight matrix, b represents the bias vector, A feature vector representing the i-th amino acid position in the protein sequence to be predicted;

[0033] The output of the fully connected layer is mapped to a probability interval to obtain the predicted probability.

[0034] In a second aspect, the present application provides a B cell epitope prediction device based on a protein language model, comprising:

[0035] The feature extraction module is used to input the protein sequence to be predicted into the pre-trained ESM-IF1 model and the pre-trained ESM2 model for prediction, and obtain multi-dimensional sequence feature vectors and structure embedding vectors;

[0036] a splicing module, configured to splice the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set;

[0037] A training module, configured to train the multidimensional dataset using an integrated convolutional neural network based on a Transformer architecture, and calculate a TIM loss function, and obtain a deep convolutional neural network model when the value of the TIM loss function is within a preset range;

[0038] An output module, configured to output the predicted probability of each amino acid position of the protein sequence using the deep convolutional neural network model;

[0039] The prediction module is used to determine that the corresponding amino acid position is an epitope if the predicted probability is greater than a set threshold, otherwise it is a non-epitope.

[0040] In a third aspect, the present application provides a prediction device, comprising:

[0041] one or more processors;

[0042] A memory for storing one or more computer programs, and one or more processors for executing the one or more computer programs stored in the memory, so that the one or more processors perform the B cell epitope prediction method based on the protein language model as described in any one of the first aspects.

[0043] In a fourth aspect, the present application provides a computer program product, which is stored on a data carrier and is designed to execute the B cell epitope prediction method based on the protein language model as described above.

[0044] Implementing one of the above-mentioned technical solutions of the present application has the following advantages or beneficial effects: the B cell epitope prediction method, device, equipment and computer program product based on the protein language model of the present application, first, the protein sequence to be predicted is input into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively, thereby capturing multi-dimensional sequence feature vectors and structure embedding vectors, which can effectively capture the structural characteristics and evolutionary characteristics of B cell epitopes and provide basic data for subsequent analysis; splicing the sequence feature vectors and the structure embedding vectors to achieve an effective combination of multimodal information to construct a unified epitope feature representation, thereby improving the efficiency and reliability of prediction; then, model training is carried out based on the Transformer architecture. During the training stage, a convolutional neural network is integrated and a TIM loss function is introduced. The convolutional neural network can effectively extract local features in a multidimensional data set. The TIM loss function helps to optimize the model training process, making the model more targeted and effective during the training process; finally, the trained model is used to perform probability prediction on each amino acid position in the protein sequence to achieve accurate prediction of different types of B cell epitopes.

[0045] This application effectively integrates the sequence feature vectors and structural embedding vectors output by the pre-trained ESM-IF1 model and the pre-trained ESM2 model, and combines them with a deep learning algorithm to construct an efficient and accurate deep convolutional neural network model. In addition, the organic fusion of the convolutional neural network and the Transformer in the deep learning algorithm enables in-depth mining and efficient learning of the nonlinear characteristics of B cell epitopes. With the advantages of high prediction accuracy, strong generalization ability, and good robustness, it can be widely used in multiple biomedical fields such as target screening for new vaccines, epitope identification of therapeutic antibodies, and development of immunodiagnostic reagents, providing key technical support for precision medicine and personalized treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work. In the drawings:

[0047] Figure 1 1 is a flow chart of a B cell epitope prediction method based on a protein language model according to an embodiment of the present application;

[0048] Figure 2 is a schematic block diagram of a B cell epitope prediction method based on a protein language model according to an embodiment of the present application;

[0049] Figure 3 Schematic diagram of the structure of a B cell epitope prediction device based on a protein language model according to an embodiment of the present application;

[0050] Figure 4 It is a structural diagram of the prediction device of an embodiment of the present application.

[0051] In the figure: 1. Prediction device; 10. Memory; 11. Processor. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the various exemplary embodiments to be described below will refer to the corresponding drawings, which constitute a part of the exemplary embodiments, in which various exemplary embodiments that may be used to implement the present application are described. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present disclosure. It should be understood that they are only examples of processes, methods and devices that are consistent with some aspects disclosed in the present application as detailed in the appended claims, and other embodiments may also be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and essence of the present application.

[0053] In the description of this application, it should be understood that the terms "center", "longitudinal", "transverse" and the like indicate the orientation or positional relationship based on the figures, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the elements referred to must have a specific orientation, be constructed and operated in a specific orientation. The terms "first", "second" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "multiple" means two or more. The terms "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, an integral connection, a mechanical connection, an electrical connection, a communication connection, a direct connection, an indirect connection through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0054] In order to illustrate the technical solution described in this application, a specific embodiment is provided below, and only the parts related to the embodiment of this application are shown.

[0055] like Figures 1 to 2 As shown, the present application provides a B cell epitope prediction method based on a protein language model, comprising the following steps S10 to S50.

[0056] S10. Input the protein sequence to be predicted into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively to capture the multi-dimensional sequence feature vector and structure embedding vector.

[0057] Specifically, ESM-IF1 (Evolutionary Scale Modeling - Inverse Folding 1) is a deep learning model developed by MetaAI that focuses on protein inverse folding tasks. The pre-trained ESM-IF1 model is designed to solve the inverse folding problem and can recover the native protein sequence from backbone atomic coordinates. The model has excellent generalization capabilities, not only for single-chain proteins but also for multi-chain protein complexes.

[0058] In some embodiments, inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model to obtain a multi-dimensional structure embedding vector may include:

[0059] Inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model, so that the pre-trained ESM-IF1 model learns the geometric features in the atomic coordinates of the protein main chain and characterizes each residue in the protein sequence;

[0060] A multi-dimensional structure embedding vector is obtained based on the geometric features and the feature vector of each residue.

[0061] Specifically, the protein sequence to be predicted is input into the pre-trained ESM-IF1 model. The pre-trained ESM-IF1 model is used for protein feature characterization. The model can learn the geometric features in the atomic coordinates of the protein main chain and has rotation and translation invariance. These features are crucial for understanding the three-dimensional structure and function of proteins. This feature can be used for a variety of protein-related tasks, including predicting protein binding affinity, stability, etc. The ESM-IF1 model can characterize each residue and generate a feature vector with rich information. The dimension of the feature vector for each residue is 512. Therefore, inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model can capture a multi-dimensional structural embedding vector, and the dimension of the multi-dimensional structural embedding vector is 512.

[0062] ESM2 (Evolutionary Scale Model 2) is a large-scale protein language model developed by Meta AI and the latest version of the "Evolutionary Scale Modeling" series. Its core goal is to capture evolutionary information and structural features from massive amounts of protein sequence data through self-supervised learning. The ESM2 model is capable of better predicting protein three-dimensional structures, including atomic-level structural information, and is extremely fast.

[0063] In some embodiments, inputting a protein sequence to be predicted into a pre-trained ESM2 model for prediction to obtain a multi-dimensional sequence feature vector may include:

[0064] Inputting the protein sequence to be predicted into the pre-trained ESM2 model, so that the ESM2 model can capture the local sequence information and global sequence information of the protein sequence and generate a corresponding embedding vector for each amino acid;

[0065] A multi-dimensional sequence feature vector is obtained based on the local sequence information, the global sequence information and the embedding vector corresponding to each amino acid.

[0066] Specifically, the protein sequence to be predicted is fed into the pre-trained ESM2 model, which then outputs an embedding representation of the protein sequence. These embeddings capture the characteristics of the protein sequence, including both local and global sequence information. The ESM2 model generates an embedding vector for each amino acid. These embedding vectors can also be used for a variety of downstream tasks, such as protein structure prediction and interaction prediction.

[0067] By inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model and the pre-trained ESM2 model, respectively, the multi-dimensional structural embedding vector and sequence feature vector captured represent the universal characteristics of protein sequences. They also contain spatial structural features and sequence characteristics, which can be used in a variety of downstream tasks and play a key role in the subsequent training and prediction of linear and conformational epitopes. The dimensionality of this multi-dimensional sequence feature vector is 1280.

[0068] S20: Concatenate the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set.

[0069] In order to achieve effective combination of multimodal information, the sequence feature vector and the structure embedding vector can be spliced.

[0070] In some embodiments, step S20 may include:

[0071] The sequence feature vector and the structure embedding vector are concatenated along the last dimension to generate the multidimensional dataset, where the number of dimensions of the multidimensional dataset is the sum of the number of dimensions of the sequence feature vector and the structure embedding vector, and the multidimensional dataset is a three-dimensional tensor whose shape is based on the batch size, sequence length, and feature dimension.

[0072] Specifically, the splicing can be expressed by the following formula:

[0073] ;

[0074] Among them, X represents a multidimensional dataset, and Concat represents splicing along the last dimension. represents the sequence feature vector, Represents the structure embedding vector.

[0075] Since the sequence eigenvector There are 1280 dimensions, the structure embedding vector There are 512 dimensions. After concatenating the two, we get a dataset X with 1792 dimensions.

[0076] Multidimensional datasets , R represents a real number set, and the multidimensional dataset X is a batch size , seqlen, dim) three-dimensional tensor, batch size Represents the batch size, seqlen represents the sequence length, and dim represents the feature dimension.

[0077] S30. Based on the Transformer architecture, the multidimensional data set is trained by integrating a convolutional neural network, and a TIM loss function is calculated. When the value of the TIM loss function is within a preset range, a deep convolutional neural network model is obtained.

[0078] In some embodiments, training the multi-dimensional dataset by integrating a convolutional neural network based on a Transformer architecture may include:

[0079] Inputting the multidimensional data set into the Transformer architecture, and using scaled dot product attention through the Transformer architecture to obtain the final output result of the multi-head attention as the global feature;

[0080] Inputting the multidimensional data set into the convolutional neural network, and outputting the convolution result as a convolution feature through the residual block of the convolutional neural network;

[0081] Training is performed based on the global features and the convolutional features.

[0082] Specifically, the core of the Transformer architecture is the multi-head attention mechanism, which can comprehensively extract features. Through the Transformer architecture, the scaled dot product attention is used to obtain the final output of the multi-head attention as a global feature, which can include:

[0083] Based on the multidimensional dataset X, initialize the query weight matrix , key weight matrix , value weight matrix , expressed by the following formula:

[0084] ;

[0085] ;

[0086] ;

[0087] Among them, Q (Query) represents the query vector, which represents the vector for asking questions and finding relevant content in the information; K (Key) represents the key vector, which represents the information matched and compared with the query vector Q; V (Value) represents the value vector, which is the vector that actually contains useful information. Based on the query weight matrix Corresponding query vector Q, key weight matrix The corresponding key vector K and the value vector V corresponding to the value weight matrix V are used as input vectors to calculate the scaled dot product attention, which is expressed by the following formula:

[0088] ;

[0089] in, Represents the dimension of the keyword vector, according to Scaling acts as a normalization factor to ensure that the increase in dimensionality does not lead to a significant increase in the dot product.

[0090] The query vector, the key vector, and the value vector are linearly transformed h times, and scaled dot product attention is applied to each linearly transformed vector to obtain the output results of each attention head; where h represents the number of attention heads.

[0091] The output results of each attention head are concatenated in order, and the final output result of the multi-head attention is obtained through linear transformation, which is expressed by the following formula:

[0092] ;

[0093] in, , Represents the output weight matrix, for attention head i, 、 、 are the weight matrices of the Q, K, and V vectors, allowing calculation , , , represents the query of the i-th attention head, represents the key of the i-th attention head, Represents the value of the i-th attention head.

[0094] In parallel with the Transformer architecture, convolutional neural networks (CNNs) are also integrated into the training framework. The core component of the convolutional neural network CNN is a residual block consisting of two one-dimensional convolutional layers. , the residual block is directly applied to the multi-dimensional dataset X, and the calculation formula is as follows:

[0095] ;

[0096] ;

[0097] ;

[0098] in, represents the input of the (l+1)th residual block, Represents the output of the residual block as a convolution feature; Conv1D represents a 1D convolution operation. 、 are the weights and biases of the i-th layer in the residual block, ReLU represents the rectified linear unit activation function, and BN represents the batch normalization operation. , represents two consecutive convolutional layers. Specifically, Is a one-dimensional convolution operation that accepts input data and process it, is the second one-dimensional convolution operation, accepting The output of is taken as input and further processed.

[0099] The obtained global features and convolutional features are used as the input of the fully connected layer of the convolutional neural network. The fully connected layer makes classification decisions based on the features to output the predicted probability.

[0100] In some embodiments, calculating the TIM loss function may include:

[0101] Based on the multidimensional data set and the labels corresponding to the multidimensional data set, determining the empirical conditional entropy and the empirical marginal entropy of the labels, and calculating the cross entropy;

[0102] The TIM loss function is calculated based on the empirical conditional entropy, the empirical marginal entropy and the cross entropy.

[0103] Specifically, the TIM loss function is used to calculate all losses on the training set. The label of the multidimensional dataset X is used to indicate whether the amino acid position is an epitope. The empirical mutual information between the multidimensional dataset X and its corresponding label Y is divided into two main components. The first component is the empirical conditional entropy of the label, denoted as ; The second component is the empirical marginal entropy of the label, denoted as .

[0104] In order to optimize the binary classification problem, it is also necessary to consider the cross entropy loss between the label and the data, and the cross entropy is denoted as CE.

[0105] Empirical conditional entropy , empirical marginal entropy And cross entropy CE are calculated using the following formulas:

[0106] ;

[0107] ;

[0108] ;

[0109] in, represents the probability of belonging to the kth category, Indicates the size of the dataset X, i is used to index the dataset X, and k is used to index the label category. represents the probability that the i-th sequence belongs to the k-th category, An indicator function that indicates whether the sequence indexed by i belongs to the kth category. K=2 can be set to implement binary classification.

[0110] Based on empirical conditional entropy , empirical marginal entropy And the cross entropy CE, calculate the TIM loss function, which can be calculated using the following formula:

[0111] ;

[0112] in, and Both represent hyperparameters, which are used to determine the convergence speed of each item in the loss function.

[0113] In some embodiments, it is possible to set , the TIM loss function at this time is the standard cross entropy theory and standard mutual information.

[0114] The model is considered to have converged when the value of the TIM loss function does not change significantly for multiple consecutive cycles. For example, if the value of the TIM loss function does not change significantly after 100 cycles, the model is considered to have converged and a deep convolutional neural network model is obtained.

[0115] S40. Outputting the predicted probability of each amino acid position in the protein sequence using the deep convolutional neural network model.

[0116] S50. If the predicted probability is greater than a set threshold, the corresponding amino acid position is determined to be an epitope, otherwise it is a non-epitope.

[0117] After completing feature extraction and model training in the previous steps, the model enters the prediction layer. The prediction layer of the deep convolutional neural network model uses a fully connected layer architecture, whose function is to map the feature vectors output by the previous layer into the final prediction result.

[0118] Specifically, for each amino acid position in the protein sequence to be predicted, the prediction layer outputs a probability value, which is used to characterize the possibility that the position is an epitope.

[0119] In some embodiments, step S40 may include:

[0120] The output of the fully connected layer of the deep convolutional neural network model is calculated using the following formula:

[0121] ;

[0122] in, represents the output of the fully connected layer of the deep convolutional neural network model, W represents the weight matrix, b represents the bias vector, A feature vector representing the i-th amino acid position in the protein sequence to be predicted;

[0123] The output of the fully connected layer is mapped to a probability interval to obtain the predicted probability.

[0124] Specifically, when calculating the output of the fully connected layer After that, the output of the fully connected layer Mapped to a probability interval, which can be (0,1), to obtain the predicted probability of each amino acid position .

[0125] The threshold can be set to 0.618, when the predicted probability When the predicted probability is ≥0.618, the amino acid position is predicted to be an epitope; when the predicted probability is <0.618, the amino acid position is predicted to be a non-epitope, thereby realizing epitope prediction of protein sequences.

[0126] In this application, first, the protein sequence to be predicted is input into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively, so as to capture multi-dimensional sequence feature vectors and structure embedding vectors, which can effectively capture the structural and evolutionary characteristics of B cell epitopes and provide basic data for subsequent analysis; the sequence feature vectors and the structure embedding vectors are spliced ​​to achieve an effective combination of multimodal information to construct a unified epitope feature representation, thereby improving the efficiency and reliability of prediction; then, the model training is carried out based on the Transformer architecture. During the training stage, the convolutional neural network is integrated and the TIM loss function is introduced. The convolutional neural network can effectively extract local features in the multi-dimensional data set. The TIM loss function helps to optimize the model training process, making the model more targeted and effective during the training process; finally, the trained model is used to perform probability prediction on each amino acid position in the protein sequence to achieve accurate prediction of different types of B cell epitopes.

[0127] This application effectively integrates the sequence feature vectors and structural embedding vectors output by the pre-trained ESM-IF1 model and the pre-trained ESM2 model, and combines them with a deep learning algorithm to construct an efficient and accurate deep convolutional neural network model. In addition, the organic fusion of the convolutional neural network and the Transformer in the deep learning algorithm enables in-depth mining and efficient learning of the nonlinear characteristics of B cell epitopes. With the advantages of high prediction accuracy, strong generalization ability, and good robustness, it can be widely used in multiple biomedical fields such as target screening for new vaccines, epitope identification of therapeutic antibodies, and development of immunodiagnostic reagents, providing key technical support for precision medicine and personalized treatment.

[0128] It should be noted that there is not necessarily a certain order between the above steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, they may be executed in parallel, or may be executed interchangeably, etc.

[0129] As another aspect of the present invention, an apparatus for predicting B cell epitopes based on a protein language model is provided. The apparatus for predicting B cell epitopes based on a protein language model in the present invention can serve as a software functional unit. The apparatus for predicting B cell epitopes based on a protein language model includes several instructions stored in a memory. A processor can access the memory and execute the instructions to complete the above-described method for predicting B cell epitopes based on a protein language model.

[0130] See also Figure 3 The B cell epitope prediction device 300 based on the protein language model includes:

[0131] Feature extraction module 301, used to input the protein sequence to be predicted into the pre-trained ESM-IF1 model and the pre-trained ESM2 model for prediction, and obtain multi-dimensional sequence feature vectors and structure embedding vectors;

[0132] A concatenation module 302 is configured to concatenate the sequence feature vector and the structure embedding vector to obtain a multi-dimensional dataset;

[0133] A training module 303 is configured to train the multidimensional data set by integrating a convolutional neural network based on a Transformer architecture, and calculate a TIM loss function. When the value of the TIM loss function is within a preset range, a deep convolutional neural network model is obtained;

[0134] An output module 304 is configured to output the predicted probability of each amino acid position in the protein sequence using the deep convolutional neural network model;

[0135] The prediction module 305 is configured to determine that the corresponding amino acid position is an epitope if the predicted probability is greater than a set threshold, otherwise it is a non-epitope.

[0136] In some embodiments, the feature extraction module 301 is further configured to:

[0137] Inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model, so that the pre-trained ESM-IF1 model learns the geometric features in the atomic coordinates of the protein main chain and characterizes each residue in the protein sequence;

[0138] A multi-dimensional structure embedding vector is obtained based on the geometric features and the feature vector of each residue.

[0139] In some embodiments, the feature extraction module 301 is further configured to:

[0140] Inputting the protein sequence to be predicted into the pre-trained ESM2 model, so that the ESM2 model can capture the local sequence information and global sequence information of the protein sequence and generate a corresponding embedding vector for each amino acid;

[0141] A multi-dimensional sequence feature vector is obtained based on the local sequence information, the global sequence information and the embedding vector corresponding to each amino acid.

[0142] In some embodiments, the splicing module 302 is further configured to:

[0143] The sequence feature vector and the structure embedding vector are concatenated along the last dimension to generate the multidimensional dataset, where the number of dimensions of the multidimensional dataset is the sum of the number of dimensions of the sequence feature vector and the structure embedding vector, and the multidimensional dataset is a three-dimensional tensor whose shape is based on the batch size, sequence length, and feature dimension.

[0144] In some embodiments, the training module 303 is further configured to:

[0145] Inputting the multidimensional data set into the Transformer architecture, and using scaled dot product attention through the Transformer architecture to obtain the final output result of the multi-head attention as the global feature;

[0146] Inputting the multidimensional data set into the convolutional neural network, and outputting the convolution result as a convolution feature through the residual block of the convolutional neural network;

[0147] Training is performed based on the global features and the convolutional features.

[0148] In some embodiments, the training module 303 is further configured to:

[0149] Based on the multidimensional data set and the labels corresponding to the multidimensional data set, determining the empirical conditional entropy and the empirical marginal entropy of the labels, and calculating the cross entropy;

[0150] The TIM loss function is calculated based on the empirical conditional entropy, the empirical marginal entropy and the cross entropy.

[0151] In some embodiments, the output module 304 is further configured to:

[0152] The output of the fully connected layer of the deep convolutional neural network model is calculated using the following formula:

[0153] ;

[0154] in, represents the output of the fully connected layer of the deep convolutional neural network model, W represents the weight matrix, b represents the bias vector, A feature vector representing the i-th amino acid position in the protein sequence to be predicted;

[0155] The output of the fully connected layer is mapped to a probability interval to obtain the predicted probability.

[0156] It should be noted that the aforementioned protein language model-based B cell epitope prediction device can implement the protein language model-based B cell epitope prediction method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects of implementing the method. For technical details not fully described in the embodiments of the protein language model-based B cell epitope prediction device, please refer to the protein language model-based B cell epitope prediction method provided in the embodiments of the present invention.

[0157] Those skilled in the art will appreciate that all or part of the features / steps of the aforementioned method embodiments can be implemented via methods, data processing systems, or computer programs. These features can be implemented entirely in software, without hardware, or through a combination of hardware and software. The aforementioned computer programs can be stored in one or more computer-readable storage media. The storage media stores the computer programs, and when executed by a system (e.g., a processor), the computer programs perform the steps of the aforementioned protein language model-based B cell epitope prediction method embodiments.

[0158] The aforementioned storage media that can store program codes include: static hard disks, solid-state hard disks, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage device or a combination thereof.

[0159] like Figure 4 As shown, the present application also provides an embodiment of a prediction device 1, comprising one or more processors 11 and a memory 10; wherein the memory 10 is used to store one or more computer programs, and the one or more processors 11 are used to execute the one or more computer programs stored in the memory 10, so that the processor 11 performs the features / steps of the above-mentioned embodiment of the B cell epitope prediction method based on the protein language model.

[0160] The present application also provides a computer program product stored on a data carrier and designed to implement the protein language model-based B cell epitope prediction method described above. Therefore, the computer program product according to the present application produces the same advantages as those described in detail with reference to the apparatus according to the present application. The computer program product can be executed as computer-readable instruction code in any suitable programming language, such as JAVA, C++, etc. Furthermore, the computer program product can be provided on a network, such as the Internet, or can be downloaded from a network, such as the Internet, by a user of the network, such as the Internet, when needed. The computer program product can be implemented using a computer program, i.e., software, or using one or more dedicated electronic circuits, i.e., hardware, or in any hybrid form, i.e., using software and hardware components, or in a hybrid form of software, hardware, or software and hardware.

[0161] The foregoing is merely a preferred embodiment of the present application. Those skilled in the art will appreciate that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present application. Furthermore, under the guidance of this application, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be included within the scope of protection of this application.

Claims

1. A B cell epitope prediction method based on a protein language model, characterized in that: include: The protein sequence to be predicted is input into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively to capture the multi-dimensional structure embedding vector and sequence feature vector; splicing the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set; Based on the Transformer architecture, the multidimensional data set is trained by integrating a convolutional neural network, and a TIM loss function is calculated. When the value of the TIM loss function is within a preset range, a deep convolutional neural network model is obtained; Outputting the predicted probability of each amino acid position in the protein sequence using the deep convolutional neural network model; If the predicted probability is greater than a set threshold, the corresponding amino acid position is determined to be an epitope, otherwise it is determined to be a non-epitope; The protein sequence to be predicted is input into the pre-trained ESM-IF1 model to obtain a multi-dimensional structure embedding vector, including: Inputting the protein sequence to be predicted into the pre-trained ESM-IF1 model, so that the pre-trained ESM-IF1 model learns the geometric features in the atomic coordinates of the protein main chain and characterizes each residue in the protein sequence; Obtaining a multi-dimensional structural embedding vector based on the geometric features and the feature vector of each residue; The multi-dimensional dataset is trained by integrating a convolutional neural network based on a Transformer architecture, including: Inputting the multidimensional data set into the Transformer architecture, and using scaled dot product attention through the Transformer architecture to obtain the final output result of the multi-head attention as the global feature; Inputting the multidimensional data set into the convolutional neural network, and outputting the convolution result as a convolution feature through the residual block of the convolutional neural network; Training is performed based on the global features and the convolutional features.

2. The B cell epitope prediction method based on protein language model according to claim 1, characterized in that: The protein sequence to be predicted is input into the pre-trained ESM2 model for prediction to obtain a multi-dimensional sequence feature vector, including: Inputting the protein sequence to be predicted into the pre-trained ESM2 model, so that the ESM2 model can capture the local sequence information and global sequence information of the protein sequence and generate a corresponding embedding vector for each amino acid; A multi-dimensional sequence feature vector is obtained based on the local sequence information, the global sequence information and the embedding vector corresponding to each amino acid.

3. The B cell epitope prediction method based on protein language model according to claim 1, characterized in that: The step of concatenating the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set includes: The sequence feature vector and the structure embedding vector are concatenated along the last dimension to generate the multidimensional dataset, where the number of dimensions of the multidimensional dataset is the sum of the number of dimensions of the sequence feature vector and the structure embedding vector, and the multidimensional dataset is a three-dimensional tensor whose shape is based on the batch size, sequence length, and feature dimension.

4. The B cell epitope prediction method based on protein language model according to claim 1, characterized in that: The calculating of the TIM loss function includes: Based on the multidimensional data set and the labels corresponding to the multidimensional data set, determining the empirical conditional entropy and the empirical marginal entropy of the labels, and calculating the cross entropy; The TIM loss function is calculated based on the empirical conditional entropy, the empirical marginal entropy and the cross entropy.

5. The B cell epitope prediction method based on protein language model according to claim 1, characterized in that: Outputting the predicted probability of each amino acid position using the deep convolutional neural network model includes: The output of the fully connected layer of the deep convolutional neural network model is calculated using the following formula: ； in, represents the output of the fully connected layer of the deep convolutional neural network model, W represents the weight matrix, b represents the bias vector, A feature vector representing the i-th amino acid position in the protein sequence to be predicted; The output of the fully connected layer is mapped to a probability interval to obtain the predicted probability.

6. A B cell epitope prediction device based on a protein language model, characterized in that: include: A feature extraction module is used to input the protein sequence to be predicted into the pre-trained ESM-IF1 model and the pre-trained ESM2 model respectively for prediction, thereby obtaining a multi-dimensional sequence feature vector and a structure embedding vector. The feature extraction module inputs the protein sequence to be predicted into the pre-trained ESM-IF1 model so that the pre-trained ESM-IF1 model learns the geometric features in the atomic coordinates of the protein main chain, and characterizes each residue in the protein sequence, and obtains a multi-dimensional structure embedding vector based on the geometric features and the feature vector of each residue. a splicing module, configured to splice the sequence feature vector and the structure embedding vector to obtain a multi-dimensional data set; A training module is used to train the multidimensional data set through an integrated convolutional neural network based on a Transformer architecture, and calculate a TIM loss function. When the value of the TIM loss function is within a preset range, a deep convolutional neural network model is obtained; the training module inputs the multidimensional data set into the Transformer architecture, uses scaled dot product attention through the Transformer architecture, obtains the final output result of the multi-head attention as a global feature, inputs the multidimensional data set into the convolutional neural network, outputs the convolution result as a convolution feature through the residual block of the convolutional neural network, and performs training based on the global feature and the convolution feature; An output module, configured to output the predicted probability of each amino acid position of the protein sequence using the deep convolutional neural network model; The prediction module is used to determine that the corresponding amino acid position is an epitope if the predicted probability is greater than a set threshold, otherwise it is a non-epitope.

7. A prediction device, characterized in that include: one or more processors; A memory for storing one or more computer programs, and one or more processors for executing the one or more computer programs stored in the memory, so that the one or more processors perform the B cell epitope prediction method based on the protein language model as described in any one of claims 1 to 5.

8. A computer program product, characterized in that The computer program product is stored on a data carrier and is designed to execute the B cell epitope prediction method based on a protein language model according to any one of claims 1 to 5.

Images