Peptide-mhc prediction method, system and device based on dual-path protein language model

Abstract

The application discloses a peptide-MHC prediction method, system and device based on a two-path protein language model, relates to a peptide-MHC prediction method, system and device. In order to solve the problem that the existing peptide-MHC prediction method has an unsatisfactory ability to aggregate condition information, the antigen peptide sequence and the MHC pseudo sequence are subjected to tokenization processing respectively, and antigen peptide token sequences and MHC token sequences are constructed, and then antigen peptide adaptation representations and MHC adaptation representations are obtained; the antigen peptide adaptation representation or the peptide side state output by the last layer is taken as a query feature, the MHC adaptation representation is taken as a generation source of key features and value features of each layer, directional multi-head cross attention operation is performed, and the MHC filling mask is used to prevent the MHC filling token from being used as an effective key and value in the multi-head cross attention; based on the antigen peptide token sequence, mask mean pooling is performed on the output of the last layer of directional cross attention, a peptide-MHC joint vector is obtained, and then prediction is realized.

Description

Technical Field

[0001] This invention belongs to the fields of bioinformatics, immunoinformatics and artificial intelligence-assisted biological sequence analysis technology, and specifically relates to a peptide-MHC prediction method, system and device. Background Technology

[0002] The interaction between peptides and MHC (major histocompatibility complex) molecules is a crucial step in antigen processing and presentation. Computational prediction of candidate peptide sequences and MHC sequences can be used for candidate antigen peptide screening, immunopeptidomics data analysis, and subsequent experimental subject sequencing. Depending on the source of the training labels, the prediction task can be defined as peptide-MHC binding prediction, presentation prediction, or immunogenicity prediction.

[0003] Existing sequence prediction methods typically use one-hot encoding of amino acids, physicochemical property encoding, substitution matrix encoding, or trainable embeddings to represent antigenic peptides and MHC sequences. These encoding methods can convert character sequences into numerical features, but fixed encodings often struggle to represent the conditional features of the same amino acid in different sequence contexts. With the development of protein language models, pre-trained models on large-scale protein sequences can be used to obtain label-by-label representations containing sequence context information. However, when directly applying protein language models to peptide-MHC prediction, issues such as encoding adaptation for different sequence types, label-level interaction modeling, and pooling of variable-length sequences still need to be addressed.

[0004] One approach is to concatenate the peptide and MHC sequences and then feed them into the same coding network. This approach allows both types of sequences to share all coding parameters, but it doesn't provide interfaces for updating parameters separately for short peptides and MHC sequences. Furthermore, the demarcation method, padding position, and length differences between the two types of sequences in the concatenated sequence can all affect subsequent characterization. Another approach is to obtain global vectors for the peptide and MHC sequences separately, then concatenate and classify them. This approach compresses residue-by-residue features before interaction modeling, making it difficult to explicitly describe the conditional dependencies of different peptide residues on different positions in the MHC sequence.

[0005] Some methods incorporate attention operations between peptide features and MHC features, but if the sources of queries, bonds, and values ​​are not clearly defined, different attention directions will result in different intermediate representations. When a joint representation indexed by peptide position is ultimately required, although it's possible to aggregate corresponding conditional information from the MHC sequence for each peptide tag, the ability and effectiveness of this aggregation are not ideal. Furthermore, batch training often requires padding sequences of varying lengths; if valid tags, padding tags, and sequence boundary tags are not distinguished during attention and aggregation operations, padding values ​​may participate in attention weight calculations or final averaging calculations, leading to feature bias and consequently affecting feature-based prediction performance. Summary of the Invention

[0006] This invention addresses the problem that existing peptide-MHC prediction methods have insufficient ability to provide polymerization condition information, and that the batch calculation of features affects the prediction results.

[0007] A peptide-MHC prediction method based on a dual-path protein language model includes:

[0008] Step S1: Obtain the antigen peptide sequence to be predicted and the MHC pseudo-sequence that forms a candidate pair with the antigen peptide sequence. Perform labeling processing on the antigen peptide sequence and the MHC pseudo-sequence respectively, and construct the antigen peptide label sequence and the MHC label sequence. When labeling, use PAD labels to pad the label sequence that is not long enough, and record it as MHC padding label.

[0009] Step S2: Input the antigen peptide label sequence into the peptide-side protein language model to obtain the peptide-side label-by-label context representation; input the MHC label sequence into the MHC-side protein language model to obtain the MHC-side label-by-label context representation. The peptide-side protein language model and the MHC-side protein language model constitute two encoding paths.

[0010] Step S3: Project the peptide-side label-by-label context representation and the MHC-side label-by-label context representation to the common feature dimension, and then perform feature transformation through a label-by-label residual fully connected network with non-shared parameters to obtain the antigen peptide adaptation representation and the MHC adaptation representation.

[0011] Step S4: Using the antigen peptide adaptor representation or the peptide-side state output from the previous layer as the query feature, and the MHC adaptor representation as the source of generation for the bond features and value features of each layer, perform at least two layers of directed multi-head cross-attention operation; the directed multi-head cross-attention operation uses the MHC padding mask to prevent the MHC padding marker from being used as a valid bond and value in the multi-head cross-attention.

[0012] The process of performing at least two layers of directed multi-head cross-attention computation includes:

[0013] The antigenic peptide is expressed as an adaptation. As the zeroth layer peptide side input ; will be represented by MHC adaptation As the source of key features and value features at each layer, for the first layer... For layer cross-attention, first perform layer normalization separately:

[0014] ,

[0015] For the Each attention head, the query matrix, key matrix, and value matrix are:

[0016] ,

[0017] in, For the first The parameter matrix of each attention head;

[0018] The number of attention heads is recorded as The dimensions of each head are ;

[0019] MHC padding mask In the middle, for the sample The The mask elements for each MHC marker position are:

[0020] ,

[0021] in, Number the MHC padding markers for the MHC side segmenter; For the sample The corresponding MHC marker sequence in the first The marking results for each MHC marker location;

[0022] Broadcast the mask along the peptide query positions, the first The output of each attention head is:

[0023] ,

[0024] Where Softmax represents the Softmax layer. This indicates a random deactivation applied to the attention weights;

[0025] Step S5: Based on the antigen peptide labeling sequence, exclude peptide-side filling markers, sequence start markers and sequence end markers, and perform mask mean pooling on the output of the last layer of directed cross attention to obtain the peptide-MHC joint vector;

[0026] Step S6: Input the peptide-MHC joint vector into the prediction network to obtain the peptide-MHC prediction result.

[0027] Furthermore, the peptide-side protein language model and the MHC-side protein language model are two independent models with the same architecture but without shared parameters.

[0028] Furthermore, the peptide-side protein language model and the MHC-side protein language model each include 36 Transformer layers, with each layer containing 18 attention heads.

[0029] Furthermore, the label-by-label residual fully connected network adopts a three-layer label-by-label residual fully connected network, including a layer normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a nonlinear activation layer, and a random deactivation layer.

[0030] Furthermore, in each of the directed multi-head cross-attention layers in step S4, in addition to the multi-head cross-attention sub-layer, attention residual connections, layer normalization, feedforward network, and feedforward residual connections are also included.

[0031] Furthermore, all network models used in the prediction process are pre-trained. During training, positive class weights are configured according to the ratio of negative to positive samples in the training set, and the model parameters are updated using a binary cross-entropy objective function with positive class weights.

[0032] A peptide-MHC prediction system based on a dual-path protein language model includes:

[0033] Sequence labeling module: Obtains the antigen peptide sequence to be predicted and the MHC pseudo-sequence that forms a candidate pair with the antigen peptide sequence, performs labeling processing on the antigen peptide sequence and the MHC pseudo-sequence respectively, and constructs antigen peptide labeling sequence and MHC labeling sequence; during labeling, PAD labels are used to pad the labeling sequence that is not long enough, which is recorded as MHC padding label;

[0034] Dual-path protein language model encoding module: The antigen peptide label sequence is input into the peptide-side protein language model to obtain the peptide-side label-by-label context representation; the MHC label sequence is input into the MHC-side protein language model to obtain the MHC-side label-by-label context representation. The peptide-side protein language model and the MHC-side protein language model constitute two encoding paths.

[0035] Dual-path residual adaptation module: The peptide-side label-by-label context representation and the MHC-side label-by-label context representation are projected onto a common feature dimension, and then feature transformation is performed through a label-by-label residual fully connected network with non-shared parameters to obtain the antigen peptide adaptation representation and the MHC adaptation representation.

[0036] Directed cross-attention module: Using the antigen peptide adaptor representation or the peptide-side state output from the previous layer as the query feature, and the MHC adaptor representation as the source of generation of bond features and value features at each layer, at least two layers of directed multi-head cross-attention operation are performed; the directed multi-head cross-attention operation uses the MHC padding mask to prevent the MHC padding marker from being used as a valid bond and value in the multi-head cross-attention.

[0037] Antigen peptide-side masking pooling module: Based on the antigen peptide labeling sequence, it excludes peptide-side filling markers, sequence start markers and sequence end markers, and performs masking mean pooling on the output of the last layer of directional cross attention to obtain the peptide-MHC joint vector;

[0038] Prediction output module: Input the peptide-MHC joint vector into the prediction network to obtain the peptide-MHC prediction result.

[0039] Furthermore, the peptide-side protein language model and the MHC-side protein language model are two independent models with the same architecture but without shared parameters.

[0040] Furthermore, in each directed multi-head cross-attention layer, in addition to the multi-head cross-attention sub-layer, attention residual connections, layer normalization, feedforward network, and feedforward residual connections are also included.

[0041] A peptide-MHC prediction device based on a dual-path protein language model, the device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the peptide-MHC prediction method based on a dual-path protein language model as described in any one of claims 1 to 6.

[0042] Beneficial effects:

[0043] Compared to processing structures that only use spliced ​​sequences or pre-obtain global vectors, this invention uses two protein language models to form peptide-side and MHC-side marker-by-marker features respectively, so that both types of sequences retain their respective context and position features before performing interactions, and provides implementation interfaces for freezing, fine-tuning or projecting the two features respectively.

[0044] This invention employs peptide-side lookup, MHC side-chain and value-oriented cross-attention, so that the output after interaction is still arranged according to the peptide label position, which facilitates the formation of a joint representation based on peptide residues and MHC sequence in subsequent steps.

[0045] This invention restricts the participation of padding positions in bond and value attention operations by using an MHC padding mask, and excludes padding markers and sequence boundary markers by using a peptide-side effective marker mask before performing mean pooling, which can reduce the impact of batch padding length and special markers on the joint vector.

[0046] This invention reuses the same MHC side source representation in multi-layer cross-attention, and each layer forms its own bonds and values. Only peptide-side joint features are updated layer by layer, thus forming a clear data transmission direction. The features of the previous layer are preserved through residual connections, so that the interaction results and the original label-by-label representation can be jointly transmitted to the next layer.

[0047] This invention provides a training implementation method suitable for fine-tuning pre-trained models and training on class-imbalanced samples by using common dimension projection to be compatible with two-way protein language models with the same or different output dimensions; and by using the group learning rate of the backbone network and the prediction head and the loss function with positive class weights. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a peptide-MHC prediction method based on a dual-path protein language model.

[0049] Figure 2 This is a schematic diagram of the module composition of the peptide-MHC prediction system;

[0050] Figure 3 This is a schematic diagram of the connection structure of the dual-path protein language model, the label-by-label residual fully connected network, the directional cross-attention and the prediction network;

[0051] Figure 4 This is a schematic diagram of a multi-headed directional cross-attention structure that uses peptide features as queries and MHC features as bonds and values.

[0052] Figure 5 This is a schematic diagram of the model training and model inference processes. Detailed Implementation

[0053] Based on the problems existing in the background technology, there is a need to provide a prediction method that can preserve the label-by-label context representation of each of the two types of sequences, perform peptide-MHC label-level information interaction according to a defined direction, and process the filling position and boundary labels accordingly. Furthermore, the method should be compatible with protein language models of the same or different types, and should be able to control the parameter updates of the pre-trained backbone network and the newly added prediction network separately during the training phase.

[0054] This invention provides a peptide-MHC prediction method, system, and device that preserves the label-by-label contextual features of the antigen peptide sequence and the MHC sequence, enabling each effective amino acid position on the antigen peptide side to obtain condition-related information according to the MHC sequence content, and reducing the interference of filler labels and sequence boundary labels on joint characterization pooling.

[0055] This invention can also solve the following technical problems: when the peptide-side protein language model and the MHC-side protein language model have different output dimensions, how to transform the two features into a common feature space that can perform cross-attention; how to reuse the same MHC source representation in multiple cross-attention layers without updating the source representation between layers; and how to configure the training process for imbalanced binary classification samples. Specific implementation method one:

[0057] The peptide-MHC prediction method based on a dual-path protein language model described in this embodiment includes the following steps:

[0058] Step S1: Obtain the antigen peptide sequence to be predicted and the MHC pseudo-sequence provided by an external data source, constructed according to the same rules as in the training phase, and paired with the antigen peptide sequence to form a candidate pair. Perform labeling processing on the antigen peptide sequence and the MHC pseudo-sequence respectively, and construct the antigen peptide label sequence and the MHC label sequence.

[0059] Step S2: Input the antigen peptide label sequence into the peptide-side protein language model to obtain the peptide-side label-by-label context representation; input the MHC label sequence into the MHC-side protein language model to obtain the MHC-side label-by-label context representation, wherein the peptide-side protein language model and the MHC-side protein language model constitute two encoding paths with non-shared parameters.

[0060] The peptide-side protein language model and the MHC-side protein language model can be two independent models with the same architecture, or they can be heterogeneous protein language models with different output dimensions. When the two output dimensions are different from the common feature dimension, the feature dimensions are subsequently transformed through the peptide-side linear projection layer and the MHC-side linear projection layer, respectively; when the output dimension is equal to the common feature dimension, the linear projection layer can be configured as an identity mapping.

[0061] "Common feature dimension" is the vector dimension used in conjunction with the two features when performing multi-head cross attention.

[0062] Step S3: Project the peptide-side label-by-label context representation and the MHC-side label-by-label context representation to the common feature dimension, and then perform feature transformation through a label-by-label residual fully connected network with non-shared parameters to obtain the antigen peptide adaptation representation and the MHC adaptation representation.

[0063] Optionally, the label-by-label residual fully connected network includes a layer normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a nonlinear activation layer, and a random deactivation layer; its input dimension and output dimension are the same, and the label-by-label features of the input are preserved through residual connections.

[0064] Step S4: Using the antigen peptide adaptor representation or the peptide-side state output from the previous layer as the query feature, and using the MHC adaptor representation as the source for generating the bond features and value features of each layer, perform at least two layers of directed multi-head cross-attention operations; in each layer, use the MHC padding mask to prevent the MHC padding marker from being used as a valid bond and value, and each layer generates its own bond and value based on the MHC adaptor representation, and the MHC adaptor representation is not updated between layers;

[0065] Each cross-attention layer includes, in addition to the multi-head cross-attention sublayer, attention residual connections, layer normalization, feedforward network, and feedforward residual connections.

[0066] Step S5: Construct an effective label mask for the peptide side based on the antigen peptide label sequence, exclude peptide side padding labels, sequence start labels and sequence end labels, and perform mask mean pooling on the output of the last layer of cross attention to obtain the peptide-MHC joint vector;

[0067] Step S6: Input the peptide-MHC joint vector into the prediction network to obtain the predicted logit, and convert the predicted logit into a predicted value between 0 and 1 using the Sigmoid function as needed, so as to perform computer screening or sorting of candidate peptide-MHC pairs.

[0068] When training the model used in the prediction (the entire model), the protein language model parameters and the newly added prediction network parameters are divided into different optimizer parameter groups, and learning rates are set for each group. Positive class weights are configured according to the ratio of negative samples to positive samples in the training set, and the model parameters are updated using a binary cross-entropy objective function with positive class weights.

[0069] Compared to processing structures that only use spliced ​​sequences or pre-obtain global vectors, this invention uses two protein language models to form peptide-side and MHC-side marker-by-marker features respectively, so that both types of sequences retain their respective context and position features before performing interactions, and provides implementation interfaces for freezing, fine-tuning or projecting the two features respectively.

[0070] This invention employs peptide-side lookup, MHC side-chain and value-oriented cross-attention, so that the output after interaction is still arranged according to the peptide label position, which facilitates the formation of a joint representation based on peptide residues and MHC sequence in subsequent steps.

[0071] This invention restricts the participation of padding positions in bond and value attention operations by using an MHC padding mask, and excludes padding markers and sequence boundary markers by using a peptide-side effective marker mask before performing mean pooling, which can reduce the impact of batch padding length and special markers on the joint vector.

[0072] This invention reuses the same MHC side source representation in multi-layer cross-attention, and each layer forms its own bonds and values. Only peptide-side joint features are updated layer by layer, thus forming a clear data transmission direction. The features of the previous layer are preserved through residual connections, so that the interaction results and the original label-by-label representation can be jointly transmitted to the next layer.

[0073] This invention provides a training implementation method suitable for fine-tuning pre-trained models and training on class-imbalanced samples by using common dimension projection to be compatible with two-way protein language models with the same or different output dimensions; and by using the group learning rate of the backbone network and the prediction head and the loss function with positive class weights. Specific Implementation Method Two:

[0075] This embodiment is a peptide-MHC prediction system based on a dual-path protein language model. The peptide-MHC prediction system 100 includes: a data acquisition module 101, a sequence labeling module 102, a dual-path protein language model encoding module 103 (including an antigen peptide-side encoding module and an MHC-side encoding module), a dual-path residual adaptation module 104 (including a common dimension projection module and a label-by-label residual transformation module), a directed cross-attention module 105, an antigen peptide-side mask pooling module 106, a prediction output module 107, and an optional training module 108. Each module is configured to perform the corresponding steps in the above method. Specific implementation method three:

[0077] This embodiment is a peptide-MHC prediction device based on a dual-path protein language model. The device includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the peptide-MHC prediction method based on a dual-path protein language model.

[0078] Those skilled in the art will understand that at least one instruction constitutes a computer program product corresponding to a method or system. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0079] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application, and can also be used with corresponding devices. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0083] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0084] Example

[0085] In this embodiment, "antigen peptide sequence" refers to a sequence composed of amino acid characters that serves as the input model for MHC candidate ligands, and is simply referred to as "peptide sequence" when there is no ambiguity; "MHC pseudo sequence" refers to an MHC amino acid fragment sequence generated by an external data source according to a preset positional order and provided according to rules consistent with the training phase, and is simply referred to as "MHC sequence" when there is no ambiguity. In this embodiment, the source record is a 34-residue MHC pseudo sequence; "tag" is a discrete number obtained by the word segmenter of the protein language model; "tag-by-tag feature" is a continuous vector that corresponds one-to-one with each input tag position of BOS (start of sequence tag), amino acid, EOS (end of sequence tag), and PAD (filler tag), and only residues corresponding to valid amino acid positions are represented; "common feature dimension" is the vector dimension commonly used when performing multi-head cross-attention between the two feature paths. Therefore, this specification uses "tag-by-tag" when describing the complete tensor containing special tags, and uses "residue-by-residue" when describing only valid amino acid positions.

[0086] The semantics of the prediction results are defined by a single predefined task label used in a single model training iteration. This predefined task can be one of a peptide-MHC binding, presentation, or immunogenicity binary classification task. Each model training iteration and the inference performed using the parameters obtained from that training iteration correspond to only one of these tasks, and the output semantics are not automatically converted between different tasks. When a positive label represents a peptide-MHC binding event, a presentation event, or a T-cell immune response confirmed under a given decision rule, the Sigmoid mapping result represents the binding propensity value, presentation propensity value, or immunogenicity propensity value corresponding to that positive label, respectively. The mapping result takes values ​​greater than 0 and less than 1; without further probability calibration, the mapping result is not limited to a statistically significant calibration probability.

[0087] For a single model training iteration, a single, pre-defined label specification is used. Positive samples in the binding label are binding pairs determined according to the same experimental type and a preset binding threshold, while negative samples are non-binding pairs determined under the same specification. Positive samples in the presentation label are presentation pairs confirmed according to a preset mass spectrometry or elution ligand determination specification, while negative samples are obtained according to the same data construction rules. Positive and negative samples in the immunogenicity label are reaction-positive and reaction-negative pairs confirmed according to the same T-cell response experiment and a preset determination threshold, respectively. Training records save label specification identifiers, data versions, and sample source identifiers, and labels defined by different experiments are not directly merged when there is no unified mapping. The predicted values ​​are only used for computer screening or ranking of candidate pairs and do not directly constitute a diagnostic conclusion for natural persons.

[0088] Example 1: Combining Figure 1 This embodiment is described below.

[0089] I. Construction of the dual-path model:

[0090] Reference Figures 3 to 4 This embodiment constructs a peptide-MHC prediction model. The model includes an antigen peptide-side coding path, an MHC-side coding path, a directed cross-attention path, and a prediction path.

[0091] S1. Input sequence and tokenization:

[0092] In step S1, the data acquisition module 101 receives a batch of antigen peptide sequences and MHC pseudo-sequences provided by an external data source. The sequence labeling module 102 performs labeling processing on the two types of sequences received. For the first sequence in the batch... For each sample, its antigenic peptide sequence is represented as:

[0093] (1)

[0094] The corresponding MHC sequence is represented as:

[0095] (2)

[0096] in, and These represent the number of amino acid residues in the peptide sequence and the MHC sequence, respectively.

[0097] In step S1, the sequence marker module 102 calls the word segmenters corresponding to the peptide-side protein language model and the MHC-side protein language model, respectively. The word segmenter adds a sequence start marker (BOS) before the amino acid marker and a sequence end marker (EOS) after the sequence; marker sequences in the same batch that are insufficient in length are padded with PAD (padding markers), and the PAD marker numbers are denoted as follows. Let the peptide labeling matrix and MHC labeling matrix be denoted as follows:

[0098] (3)

[0099] in, For batch size, and These represent the peptide label length and MHC label length after the current batch is completed, respectively. , Each sample contains a peptide marker sequence and an MHC marker sequence.

[0100] In this embodiment, the total length limit of the peptide marker sequence after adding BOS and EOS to the word segmenter is set to 14, and the total length limit of the MHC marker sequence is set to 36. When the marker sequence exceeds the corresponding limit, it is truncated from the right to the limit, and padding is used when the length is insufficient. Therefore, with BOS and EOS each occupying one marker, the effective number of amino acid markers sent to the two protein language models is no more than 12 and 34, respectively. The MHC source record can be a 34-residue MHC pseudo-sequence, and the current preprocessing configuration will truncate it to a total of 34 markers after adding the special marker.

[0101] S2. Two-way protein language model encoding:

[0102] In step S2, the peptide-side encoding path and the MHC-side encoding path in the dual-path protein language model encoding module 103 perform label-by-label context encoding, respectively. Let the peptide-side protein language model be... The MHC-side protein language model is ,but:

[0103] (4)

[0104] in . and These are the output dimensions of the two protein language models. and These are the parameters for the two-way model.

[0105] In this embodiment, both the first protein language model 203 and the second protein language model 204 adopt the ESMC-600M architecture. Each model has a label-by-label hidden dimension of 1152, including 36 Transformer layers, with each layer containing 18 attention heads. The two models can be initialized with the same pre-trained parameters, but are instantiated in the model as two separate parameter sets. and The data is saved and updated separately during the fine-tuning process. As a result, the peptide side and the MHC side can form different parameter update paths based on their respective input data.

[0106] It should be noted that the first protein language model 203 and the second protein language model 204 in this embodiment both adopt the ESMC-600M architecture. Other model architectures, such as general protein language models like ESM3 and ESM2, can be used in other embodiments.

[0107] This embodiment obtains the label-by-label embeddings of the model's final output without performing global averaging or maximum aggregation before interaction. If the output object of a specific protein language model contains multiple data orbitals, then sequence embedding orbitals are extracted from the output object; if the output has an additional single-element orbital dimension, then that single-element dimension is removed, unifying the features into a three-dimensional tensor.

[0108] S3. Projection of common feature dimensions:

[0109] In step S3, the dual-path residual adaptation module 104 first transforms the two labeled features into a common dimension. Then, modality-specific residual adaptation is performed. The dimensionality transformation process is as follows:

[0110] (5)

[0111] in, This is the peptide-side projection matrix. The MHC side projection matrix; , .

[0112] when In this case, the peptide-side projection layer 305 and the MHC-side projection layer 306 can use an identity mapping, without introducing additional projection parameters. In this embodiment... Therefore, identity mapping is used. When using two-way protein language models with different architectures or scales, the different output dimensions are unified by linear projection in equation (5). .

[0113] Construct a three-layer, label-by-label, fully connected residual network:

[0114] Peptide-side label-by-label features and MHC-side label-by-label features are respectively processed through a first label-by-label residual fully connected network 205 and a second label-by-label residual fully connected network 206 with non-shared parameters. (Input from either path is...) For example, its transformation can be expressed as:

[0115] (6)

[0116] in, Representation layer normalization, Represents the Gaussian error linear unit activation function. Indicates random inactivation. and These are trainable parameters. The first layer reduces the dimension from... Expand to The second layer remains The third layer restores the dimension to Therefore, the residual term has the same dimension as the output of the third layer.

[0117] The residual transformation results for the peptide side and MHC side are as follows:

[0118] (7)

[0119] in, and They have the same network structure but use different parameters.

[0120] In this embodiment, the dimensions of the label-by-label residual fully connected network are 1152, 4608, 4608, and 1152, respectively, with a random inactivation rate of 10%. The residual connections preserve the direct transmission path of the original label-by-label features of the protein language model in the transformed features.

[0121] S4. Targeted multi-head cross-attention processing:

[0122] In step S4, the directional cross-attention module 105 uses peptide-side features as the query and MHC-side features as the key and value. Let the number of cross-attention layers be... The zeroth layer peptide-side input is defined as:

[0123] (8)

[0124] MHC side-source representation remains constant across all cross-attention layers. Furthermore, updates do not occur between layers; each layer uses its own layer normalization and key-value projection parameters, derived from the same... Generate the keys and values ​​for this layer respectively.

[0125] For the For layer cross-attention, first perform layer normalization separately:

[0126] (9)

[0127] For the Each attention head, the query matrix, key matrix, and value matrix are:

[0128] (10)

[0129] in, For the first The parameter matrix of each attention head.

[0130] The number of attention heads is recorded as The dimensions of each head are .

[0131] Construct the MHC padding mask based on the MHC tag matrix. For the sample The Each MHC marker position, the mask element is defined as:

[0132] (11)

[0133] in, Number the PAD tags (MHC padding tags) for the MHC side segmenter; For the sample The corresponding MHC marker sequence in the first The marking results of each MHC marker location.

[0134] Broadcast the mask along the peptide query positions, the first The output of each attention head is:

[0135] (12)

[0136] In equation (12), Softmax is calculated along the MHC label dimension. This indicates random deactivation of the attention weights. Since the PAD position is assigned a negative infinity mask value before Softmax, the attention weight corresponding to that position is zero. BOS and EOS can be preserved in the MHC-side context; in another embodiment, the MHC-side BOS and EOS can also be added to the key-value mask.

[0137] In this embodiment, a mask matrix M corresponding to the MHC marker sequence is introduced into the relevance score of the directed multi-head cross-attention, and the score of the padding position is set to negative infinity before Softmax normalization, so that its attention weight is zero. Compared with cross-attention that does not distinguish between effective residues and padding markers, this design can block biologically meaningless PAD information from entering peptide-side feature aggregation, avoid weight dilution and characterization shift caused by different MHC sequence lengths and batch padding methods, and enable each peptide residue to extract conditional information only from the effective MHC context, thereby improving the stability and prediction reliability of variable-length sequence modeling.

[0138] Connect the outputs of multiple attention heads and project them onto the output:

[0139] (13)

[0140] Add the attention output to the current layer's query input (which is also the query output of the previous layer) using the first residual:

[0141] (14)

[0142] Subsequently, layer normalization and feedforward network were used. Perform the second residual transformation:

[0143] (15)

[0144] (16)

[0145] in Represents feedforward network Input, , As weight, , For bias, This is the activation function.

[0146] The first feedforward layer converts the feature dimension from... Expand to The second feedforward layer then recovers to .

[0147] This embodiment is set Each layer includes four attention heads with a common dimension of 1152, resulting in each attention head having a dimension of 288. The attention weights and the random inactivation rate of the feedforward network are both 0.1, and the intermediate dimension of the feedforward network is 4608. The first cross-attention layer uses... As input to the query, the second cross-attention layer uses the output of the first layer. This is the input for the query. Both levels use the same... As the source representation of keys and values, but generating local keys and values ​​respectively through local parameters; It is not covered by peptide-side outputs between layers. Ultimately, peptide-side joint features are obtained:

[0148] (17)

[0149] The second dimension of equation (17) still corresponds to the peptide label position, indicating that each peptide label has been bound to the condition information of the MHC sequence.

[0150] S5. Peptide-side mask mean pooling:

[0151] In step S5, the peptide-side mask pooling module 106 constructs an effective label mask based on the peptide label sequence. For the first... The first sample Peptide markers:

[0152] (18)

[0153] in, , and These are the numbers of the special markers corresponding to the peptide-side word segmenters; For the sample The corresponding peptide label sequence in the first Labeling results for each peptide labeling location.

[0154] Peptide-MHC Joint Vector Calculate as follows:

[0155] (19)

[0156] in, For the sample The feature value of the j-th peptide label position in the corresponding peptide-side joint features; the lower limit of the denominator is set to 1 to prevent division by zero due to abnormal empty input. A normal peptide sequence contains at least one valid amino acid label.

[0157] Equation (19) only converges along the peptide labeling dimension and does not form a separate global vector on the MHC side.

[0158] S6. Three-layer prediction network and predicted output:

[0159] In step S6, the prediction output module 107 inputs the joint vector into the three-layer prediction network 210. The calculation process is as follows:

[0160] (20)

[0161] in, , , As weight, , , For bias.

[0162] In this embodiment, the prediction network dimensions are 1152, 2304, 1152 and 1 respectively, and the random inactivation rate after the first two fully connected layers is 0.2.

[0163] Output This is the unnormalized predicted value, also known as the logit. Its corresponding Sigmoid predicted value is:

[0164] (twenty one)

[0165] When used for binary classification decision-making, it can be The value is compared with a preset threshold; the threshold can be 0.5, or it can be determined according to the screening rules preset by the application. The Sigmoid prediction value can also be directly used to rank candidate peptide-MHC pairings without performing discrete threshold determination; when probability calibration is not performed, the Sigmoid prediction value is not interpreted as a calibration probability.

[0166] II. Model Training and Inference:

[0167] Reference Figure 5 This embodiment illustrates how to train the model of Embodiment 1 and perform inference using the trained parameters.

[0168] 1. Training data construction:

[0169] Training sample 401 is based on records, with each record including at least a peptide sequence field, an MHC sequence field, and a tag field. First, records with any required fields empty are deleted. Then, both types of sequences are converted to a uniform uppercase amino acid character format, and the tag is converted to 0 or 1.

[0170] In the immunogenicity prediction implementation, positive labels should be obtained from pre-determined immune response criteria, and negative labels should be obtained according to the same data specifications; in the binding prediction or presentation prediction implementation, binding labels or presentation labels should be used accordingly. Data with different semantic labels can be used to train models separately, and labels from different sources and with inconsistent definitions should not be directly regarded as the same target in the absence of unified rules.

[0171] The training preprocessing unit 402 divides the data into training, validation, and test sets according to a label-stratified approach. In one specific implementation, an 8:1:1 ratio is used, with a random seed of 42. This stratified partitioning ensures that each data subset maintains the basic composition of positive and negative labels. The model parameters are updated only based on the objective function of the training set, the validation set is used to select training checkpoints, and the test set is used to independently generate prediction results after the model parameters are determined.

[0172] Each batch consists of 32 records. During the data loading phase, the original character sequence is preserved. Before the batch enters the computing device, the encoding interfaces of the two backbone networks are called to form a tag sequence, and then PAD padding is performed according to the longest tag sequence of the current batch.

[0173] 2. Objective function with positive class weights:

[0174] Let the number of positive samples in the training set be... The number of negative samples is The positive class weight is set as follows:

[0175] (twenty two)

[0176] For a batch of the first For each sample, the true label is: The model output logit is The binary cross-entropy loss with positive class weights is:

[0177] (twenty three)

[0178] In actual calculations, the logit can be directly implemented using numerically stable binary cross-entropy with logit, without having to explicitly calculate the Sigmoid prediction value of Equation (21) first. Equation (22) configures the weight of the positive sample item in the loss according to the composition of the training set categories; this weight is calculated only from the training set, avoiding the use of validation set or test set labels to set training parameters.

[0179] In practice, positive and negative samples are determined according to the predetermined task and labeling specifications used in a single model training. For the peptide-MHC binding prediction task, positive samples are peptide-MHC pairs labeled as bound based on predetermined binding experiments, database records, or binding determination thresholds, while negative samples are pairs labeled as non-bound based on the same labeling specifications. For the presentation prediction task, positive samples are presentation pairs confirmed based on mass spectrometry identification, eluted ligand records, or other predetermined presentation determination rules, while negative samples are non-presentation pairs obtained according to the same data source and negative sample construction rules. For the immunogenicity prediction task, positive and negative samples are positive and negative pairs, respectively, determined based on predetermined T cell response experiments and determination thresholds. If a negative sample is constructed using preset sampling or pairing rules, its label indicates the training negative class under that data specification, and does not necessarily mean that it has been experimentally proven to have no biological effect. and These represent the number of samples with labels y=1 and y=0 in the training set, respectively.

[0180] 3. Parameter grouping optimization:

[0181] The parameter optimization unit 405 divides the trainable parameters into three parameter groups: the peptide-side protein language model parameter group, the MHC-side protein language model parameter group, and the prediction head parameter group. The prediction head parameter group includes a common dimension projection layer, a two-way label-by-label residual fully connected network, various cross-attention layers, and a prediction network.

[0182] This embodiment uses the AdamW optimizer. The learning rates for both the peptide-side and MHC-side protein language models are set to... The learning rate for the prediction head parameters is set to The weight decay factor is set to 0.01. Therefore, the pre-trained backbone network is fine-tuned with a relatively small update step size, while the new network is trained with a different update step size.

[0183] Each training batch sequentially performs gradient clearing, forward computation, loss calculation, backpropagation, gradient pruning, and parameter update. Let the L2 norm of the concatenated gradients of all trainable parameters be . The gradient after gradient clipping is:

[0184] (twenty four)

[0185] Among them, the cropping threshold Set to 1, To prevent zero-norm division from using a pre-defined positive number, the processing limits the L2 norm of all trainable parameter gradients used in a single parameter update.

[0186] This embodiment sets the maximum number of training epochs to 30. After each training epoch, the Sigmoid prediction value is calculated on the validation set, and the Average Precision (AP) is obtained by calling the Average Precision calculation function. This evaluation metric is used to select parameter checkpoints. If the validation evaluation metric is higher than the previously saved value, the current model parameters are saved and the unimproved count is cleared to zero; otherwise, the unimproved count is incremented by one. Training stops when there is no improvement after 5 consecutive epochs. After training, the saved validation checkpoints are loaded to generate predictions for the validation and test sets. The above evaluation metrics are only used to select model parameters and do not limit any performance values ​​in this specification.

[0187] 4. Reasoning process:

[0188] In the inference flow, which consists of inference input 408 and prediction and ranking results 413, one or more peptide-MHC pairs to be predicted are received. The input is constructed according to the same character specification, word segmenter, BOS / EOS settings, residue upper limit, and PAD rules as in the training phase. The model is then switched to inference state, random inactivation is disabled, and gradient recording is stopped.

[0189] Antigen peptide marker sequences are used to form query features through an antigen peptide-side protein language model and an antigen peptide-side residual fully connected network; MHC marker sequences are used to form key-value source representations through an MHC-side protein language model and an MHC-side residual fully connected network. After two layers of cross-attention, antigen peptide-side mask mean pooling, and a prediction network, the logit is obtained, and then converted into a predicted value between 0 and 1 by a sigmoid function.

[0190] For scenarios where the same MHC sequence corresponds to multiple candidate peptides, the labeling results of the MHC sequence can be pre-cached, or its MHC side label-by-label features can be cached when the model parameters are fixed. Then, cross-attention is performed with the peptide-side features of different candidate peptides. The cached content needs to be associated with the current model parameter version, word segmenter version, and input sequence identifier; the cache is recalculated when the MHC-side model parameters change.

[0191] It should be noted that the prediction result of this invention is a predicted value between 0 and 1 obtained by mapping the unnormalized predicted logit value using the Sigmoid function. This is a scalar result output by the prediction model for a candidate peptide-MHC pair. The model output, as computational prediction information, can be used for ranking candidate pairings and selecting subsequent experimental subjects. More specifically, in this embodiment, each set of trained model parameters corresponds to only one predetermined binary classification task: when the training label is a binding label, the predicted value represents the peptide-MHC binding tendency; when the training label is a presentation label, it represents the presentation tendency; when the training label is an immunogenicity label, it represents the tendency to elicit a predetermined T-cell immune response. The antigen peptide sequence identifier, MHC sequence or allele identifier, and model version identifier are only input identifiers or traceability information associated with the prediction result and are not part of the target quantity predicted by the network model. For the entire scheme, the prediction output may include the antigen peptide sequence identifier, MHC sequence or allele identifier, and model version identifier. Example 2: This embodiment is an alternative to the backbone network:

[0192] 1. Implementation of heterogeneous backbone networks:

[0193] In another embodiment, the peptide-side protein language model and the MHC-side protein language model employ different architectures or scales, and their output dimensions are respectively... and Choose public dimensions The dimensions are obtained by using two independent unbiased linear projections as shown in equation (5). The label-by-label features are determined. Before projection, each label-by-label output is converted to the computing device and numerical type corresponding to the projection weight, ensuring that the data type of the projection input is consistent with that of the weight. As long as the protein language model can output a vector sequence corresponding to the input label position, it can be connected to the common dimension projection module.

[0194] The two models can have different word segmenters and different PAD, BOS, and EOS numbers. The system stores the unique tag numbers for the peptide side and the MHC side separately, and constructs a mask based on the corresponding tag matrix, without assuming that the two word segmenters use the same number. For backbone networks whose output dimensions cannot be directly read from the model configuration, a test sequence containing at least one amino acid residue can be input, and the final dimension size of its tag-by-tag output can be read, thereby establishing a common dimension projection layer.

[0195] In heterogeneous backbone network implementations, common dimensions The number of heads that should be able to receive cross-attention Divisible. The single-head dimension of the attention head is... The intermediate dimension of the label-by-label residual fully connected network and the cross-attention feedforward network can be set to... ,in It is an integer greater than or equal to 1; Example 1 takes .

[0196] 2. Implementation method for freezing two independent backbones:

[0197] In another embodiment, the peptide side and the MHC side still use two independent backbone instances, but one or both backbone parameters can be frozen. When the backbone is frozen, it still performs forward label-by-label encoding, but its parameters are not added to the optimizer; common dimension projection, label-by-label residual fully connected networks, cross-attention, and prediction networks can still be trained. Alternatively, the backbone can be frozen first to train the prediction head, and then the frozen state of some backbone layers can be unfrozen and training can continue using a smaller learning rate.

[0198] Regardless of whether the two backbones are frozen or fine-tuned, the two protein language models maintain independent parameter sets and retain the following data flow: obtaining label-by-label features on the peptide side and MHC side respectively; transforming the two features to a common dimension; generating bonds and values ​​at each layer using peptide-side features as queries and the same MHC source representation; performing effective label mask mean pooling on the peptide-side joint features; and generating predicted values ​​from the prediction network.

[0199] 3. Multi-layered interaction implementation method:

[0200] The number of cross-attention layers should be set to at least two. Layer output as the first Layer peptide side query input, MHC side source representation It is reused within each layer and not updated between layers; each layer utilizes its own parameters. Generate the key and value for this layer. In this embodiment, the number of cross-attention layers is 2; it can be set to an integer number greater than 2 when computing resources allow.

[0201] The MHC-side padding mask is reused in each cross-attention layer. Although the peptide-side PAD position can be used as a query position in the tensor for intermediate operations, it is excluded in the final convergence of Equation (19); in another embodiment, the peptide-side PAD position can also be set to zero after the output of each cross-attention layer. Regardless of the method used, the final joint vector is calculated based on the effective amino acid label on the peptide side.

[0202] The pooling method adopts the mask mean pooling of Equation (19) and ensures that PAD, BOS and EOS do not participate in the final pooling as effective amino acid residues.

[0203] Example 3: System and Electronic Equipment Implementation:

[0204] Reference Figure 2The peptide-MHC prediction system 100 includes a data acquisition module 101, a sequence labeling module 102, a dual-path protein language model encoding module 103, a dual-path residual adaptation module 104, a directional cross-attention module 105, an antigen peptide side mask pooling module 106, a prediction output module 107, and an optional training module 108.

[0205] The data acquisition module 101 is configured to receive peptide sequences and MHC sequences from local files, databases, or communication interfaces; the sequence labeling module 102 is configured to perform character normalization, labeling, truncation, and batch completion; the dual-path protein language model encoding module 103 loads the first protein language model 203 and the second protein language model 204 respectively and outputs label-by-label features; the dual-path residual adaptation module 104 performs transformations of equations (5) to (7); the directional cross-attention module 105 performs processing of equations (8) to (17); the peptide-side mask pooling module 106 performs processing of equations (18) and (19); and the prediction output module 107 performs processing of equations (20) and (21). The training module 108 is used to perform objective function calculation and parameter update corresponding to equations (22) to (24).

[0206] The system can be implemented by an electronic device. The electronic device includes at least one processor 212, a memory 211, and a communication interface 213. The memory 211 stores model structure information, model parameters, word segmenter configuration, special tag numbers, and computer programs; the processor 212 executes the computer program to call the aforementioned functional modules. The processor 212 may include a central processing unit, a graphics processing unit, or other processing units capable of performing tensor operations. The two protein language models can be deployed on the same processor or on multiple processors that communicate with each other.

[0207] In a multiprocessor implementation, peptide-side label-by-label features and MHC-side label-by-label features are computed on their respective processors and then transmitted to the processor performing cross-attention. For batch candidate peptide screening, different batches can be assigned to multiple processors; each batch is still processed according to the same MHC padding mask and peptide-side valid label mask rules.

[0208] The computer program may be stored in a non-transitory computer-readable storage medium. The storage medium may include a disk, solid-state memory, read-only memory, random access memory, or other storage media readable by an electronic device. When the program is executed by a processor, it completes the steps of input acquisition, two-way tag-by-tag encoding, common dimension projection, residual feature transformation, directed cross-attention, mask pooling, and output prediction.

[0209] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A peptide-MHC prediction method based on a dual-path protein language model, characterized in that, include: Step S1: Obtain the antigen peptide sequence to be predicted and the MHC pseudo-sequence that forms a candidate pair with the antigen peptide sequence. Perform labeling processing on the antigen peptide sequence and the MHC pseudo-sequence respectively, and construct the antigen peptide label sequence and the MHC label sequence. When labeling, use PAD labels to pad the label sequence that is not long enough, and record it as MHC padding label. Step S2: Input the antigen peptide label sequence into the peptide-side protein language model to obtain the peptide-side label-by-label context representation; input the MHC label sequence into the MHC-side protein language model to obtain the MHC-side label-by-label context representation. The peptide-side protein language model and the MHC-side protein language model constitute two encoding paths. Step S3: Project the peptide-side label-by-label context representation and the MHC-side label-by-label context representation to the common feature dimension, and then perform feature transformation through a label-by-label residual fully connected network with non-shared parameters to obtain the antigen peptide adaptation representation and the MHC adaptation representation. Step S4: Using the antigen peptide adaptor representation or the peptide-side state output from the previous layer as the query feature, and using the MHC adaptor representation as the source of generation for bond features and value features of each layer, perform at least two layers of directional multi-head cross-attention operation. Directed multi-head cross attention operation utilizes MHC padding masks to prevent MHC padding tags from being used as valid keys and values ​​in multi-head cross attention; Step S5: Based on the antigen peptide labeling sequence, exclude peptide-side filling markers, sequence start markers and sequence end markers, and perform mask mean pooling on the output of the last layer of directed cross attention to obtain the peptide-MHC joint vector; Step S6: Input the peptide-MHC joint vector into the prediction network to obtain the peptide-MHC prediction result.

2. The peptide-MHC prediction method based on a dual-path protein language model according to claim 1, characterized in that, The peptide-side protein language model and the MHC-side protein language model are two independent models with the same architecture but no shared parameters.

3. The peptide-MHC prediction method based on a dual-path protein language model according to claim 2, characterized in that, The peptide-side protein language model and the MHC-side protein language model each include 36 Transformer layers, with each layer containing 18 attention heads.

4. The peptide-MHC prediction method based on a dual-path protein language model according to claim 1, characterized in that, The label-by-label residual fully connected network is a three-layer label-by-label residual fully connected network, including a layer normalization layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a nonlinear activation layer, and a random deactivation layer.

5. A peptide-MHC prediction method based on a dual-path protein language model according to any one of claims 1 to 4, characterized in that, Each directed multi-head cross-attention layer in step S4 includes, in addition to the multi-head cross-attention sub-layer, attention residual connections, layer normalization, feedforward network, and feedforward residual connections.

6. The peptide-MHC prediction method based on a dual-path protein language model according to claim 5, characterized in that, All network models used in the prediction process are pre-trained. During training, positive class weights are configured according to the ratio of negative to positive samples in the training set, and the model parameters are updated using a binary cross-entropy objective function with positive class weights.

7. A peptide-MHC prediction system based on a dual-path protein language model, characterized in that, include: Sequence labeling module: Obtains the antigen peptide sequence to be predicted and the MHC pseudo-sequence that forms a candidate pair with the antigen peptide sequence, performs labeling processing on the antigen peptide sequence and the MHC pseudo-sequence respectively, and constructs antigen peptide labeling sequence and MHC labeling sequence; during labeling, PAD labels are used to pad the labeling sequence that is not long enough, which is recorded as MHC padding label; Dual-path protein language model encoding module: The antigen peptide label sequence is input into the peptide-side protein language model to obtain the peptide-side label-by-label context representation; the MHC label sequence is input into the MHC-side protein language model to obtain the MHC-side label-by-label context representation. The peptide-side protein language model and the MHC-side protein language model constitute two encoding paths. Dual-path residual adaptation module: The peptide-side label-by-label context representation and the MHC-side label-by-label context representation are projected onto a common feature dimension, and then feature transformation is performed through a label-by-label residual fully connected network with non-shared parameters to obtain the antigen peptide adaptation representation and the MHC adaptation representation. Directed cross-attention module: Using the antigen peptide adaptor representation or the peptide-side state output from the previous layer as the query feature, and the MHC adaptor representation as the source of generation of bond features and value features of each layer, perform at least two layers of directed multi-head cross-attention operations. Directed multi-head cross attention operation utilizes MHC padding masks to prevent MHC padding tags from being used as valid keys and values ​​in multi-head cross attention; Antigen peptide-side masking pooling module: Based on the antigen peptide labeling sequence, it excludes peptide-side filling markers, sequence start markers and sequence end markers, and performs masking mean pooling on the output of the last layer of directional cross attention to obtain the peptide-MHC joint vector; Prediction output module: Input the peptide-MHC joint vector into the prediction network to obtain the peptide-MHC prediction result.

8. The peptide-MHC prediction system based on a dual-path protein language model according to claim 7, characterized in that, The peptide-side protein language model and the MHC-side protein language model are two independent models with the same architecture but no shared parameters.

9. The peptide-MHC prediction method based on a dual-path protein language model according to claim 7, characterized in that, In each directed multi-head cross-attention layer, in addition to the multi-head cross-attention sub-layer, attention residual connections, layer normalization, feedforward network, and feedforward residual connections are also included.

10. A peptide-MHC prediction device based on a dual-path protein language model, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the peptide-MHC prediction method based on a dual-path protein language model as described in any one of claims 1 to 6.

Images