A protein-protein interaction prediction method based on cross-modal enhanced representation learning

Abstract

The present invention relates to the technical field of protein prediction, and discloses a protein interaction prediction method based on cross-modal enhanced representation learning. The present invention designs two modules, Convformer and Graphformer, and breaks through the limitation that the existing methods only focus on local features or global features by introducing a global attention mechanism in each module. This design enables the model to capture the local patterns and global dependencies of proteins at the same time, providing a more comprehensive protein representation. The present invention proposes a cross-modal enhancement method based on contrastive learning, which solves the difference problem between sequence and structural modes through an explicit modal alignment process. This method can not only maintain the unique information of each mode, but also give full play to the complementarity between modes, thereby obtaining a richer protein representation. The cross-protein fusion module proposed in the present invention can more accurately capture the essential characteristics of protein interactions and significantly improve the accuracy of PPI prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of protein prediction, and in particular to a protein interaction prediction method based on cross-modal enhanced representation learning. Background Art

[0002] Proteins are one of the most important biological molecules in cells. They are not only involved in the construction of various cellular components, but also participate in key biological processes such as signal transduction, gene transcription, and immune response through complex interaction networks. Building a comprehensive protein-protein interaction (PPI) dictionary can help us understand the complex biological pathways in organisms and provide a theoretical basis for disease treatment and drug development. With the continuous development of high-throughput sequencing technology, expensive and time-consuming wet experimental methods can no longer meet the large demand for determining unknown PPIs. In recent years, in order to break this cost limitation, more and more studies have focused on computer-based PPI prediction methods, which are mainly divided into: machine learning-based methods and data-driven deep learning methods.

[0003] Early PPI prediction methods mainly focused on machine learning (ML) methods. After training on PPI datasets, ML models can be used to effectively identify unknown PPIs, and compared with traditional wet experimental methods, these methods show better adaptability in high-throughput interaction prediction tasks. However, although these well-designed ML-based methods have achieved good results in PPI prediction, their feature engineering still requires expert experience or complex domain knowledge.

[0004] Data-driven deep learning methods learn high-quality protein representations directly from data by designing some learnable deep neural networks, such as convolutional neural networks (CNN), recurrent neural networks (RNN), Transformer, and graph neural networks (GNN). Data-driven deep learning methods have gradually become the mainstream method for predicting PPI due to their low cost and high efficiency.

[0005] In the application of deep learning to achieve PPI prediction, exploring an efficient protein representation method and improving the abundance of target protein representation play an important role in improving prediction performance. At present, according to the protein representation form, the existing deep learning-based PPI prediction methods are roughly divided into three categories: based on 1D sequence, based on 2D topological structure, and based on multimodal information.

[0006] 1) 1D sequence-based method: This method regards the amino acid sequence of a protein as a sentence, and each word in the sentence is the corresponding amino acid residue. By mapping each amino acid residue to a corresponding word vector, the residue representation of the entire protein sequence can be obtained, and then some neural networks for NLP processing can be applied to learn sequence text representation. With this form of text processing, the text semantic information of the protein sequence can be effectively captured.

[0007] 2) 2D structure-based method: This method regards amino acid residues as nodes. By constructing a protein residue contact map and using it as the edge between residues, the protein structure can be mapped into a 2D topological map. The protein structure data is used to analyze the surface features, secondary structure features and other geometric properties of the protein, and integrate them into the node features in the topological map. By applying a graph neural network to the protein topological map, the topological semantic information of the protein can be captured, improving the prediction performance of PPI.

[0008] 3) Methods based on multimodal information: In order to make full use of the sequence and structural information of proteins, some PPI prediction methods based on protein multimodal data have emerged in recent years, aiming to utilize the complementarity of different data modalities to enrich protein feature representation and improve the accuracy of prediction models.

[0009] Although existing deep learning-based PPI prediction methods show good performance when processing single modal information, they lack comprehensive consideration of local patterns and global dependencies of proteins when processing protein representation. The interaction between residues in proteins is not only reflected in the local coupling between adjacent residues, but also affected by the overall protein structure. Current methods often focus on only one aspect, resulting in incomplete and inaccurate protein representation. In addition, in multimodal methods, there are inherent differences between modalities, and different modalities have different data structures and feature extraction methods. If effective modal alignment is not performed before feature fusion, the complementary effect between modalities may be affected, reducing the effect of representation learning. Finally, when modeling the interaction between protein pairs, existing methods usually adopt a simple feature splicing method, ignoring the interaction modeling between specific residues on the protein surface, which may lead to the loss of important interaction information. Therefore, the present invention designs a new cross-modal enhanced protein representation learning framework, which realizes effective alignment between modalities through reasonable feature extraction and fusion strategies, while considering local and global dependencies, and introduces cross-protein fusion to capture the interaction information at the residue level between protein pairs, thereby improving the accuracy and reliability of PPI prediction. Summary of the invention

[0010] In view of the shortcomings of the existing PPI prediction model based on deep learning, the technical problems solved by the present invention are as follows:

[0011] (1) Existing unimodal representation methods fail to fully consider local patterns and global dependencies in proteins. In proteins, the interactions between residues not only originate from the local coupling between adjacent residues, but are also affected by the overall protein structure. Therefore, unimodal representations that only consider local or global dependencies are insufficient to fully express the characteristics of proteins.

[0012] (2) Existing multimodal joint representation methods have shortcomings in modeling modal complementarity. The sequence and structural modalities of proteins have different data structures and feature distribution characteristics. If effective modal alignment is not performed before feature fusion, the complementary effect between modalities may be affected, reducing the effectiveness of representation learning.

[0013] (3) Existing methods need to be improved in modeling residue interactions between protein pairs. The formation of PPIs usually involves interactions between specific residues on the protein surface. Simply connecting the representation of protein pairs is difficult to capture such residue-level interaction patterns, which may lead to the loss of important interaction information.

[0014] In order to make up for the deficiencies of the prior art, the present invention provides a protein interaction prediction method based on cross-modal enhanced representation learning, comprising the following steps:

[0015] 1. Sequence modal representation

[0016] Design a sequence feature encoder Convformer that combines CNN and multi-head self-attention mechanism to achieve global interaction between segments while preserving local features within each segment, thereby obtaining higher quality sequence embedding;

[0017] 2. Structural modal representation

[0018] A graph neural network model Graphformer is proposed to extract 2D protein topology graphs The key structural information of proteins in Graphformer consists of two stages: Representation diagram The nodes in Represent the edges between nodes: global attention stage and graph attention stage. In the global attention stage, the global dependencies between all nodes in the protein graph are captured through the self-attention mechanism; the graph attention stage learns local structural features through the attention aggregation of edges; the global attention mechanism uses the learned global features as context information to guide the graph attention to effectively capture the structural information of the protein graph; after these two stages, the final structural embedding of the protein graph is obtained ,in Represents a matrix with n rows and d columns. The values ​​of the matrix are real numbers. n represents the number of residue nodes in a protein graph, and d represents the dimension of the feature.

[0019] 3. Cross-modal enhancement

[0020] Due to the existence of modality differences, directly fusion sequence embedding and structural embedding This will lead to information loss or mutual interference, thus affecting the complementary effect between modalities. In this step, a cross-modal enhancement module is designed to achieve feature alignment and fusion. The module consists of two parts: cross-modal alignment and cross-modal fusion. Cross-modal alignment maximizes the similarity between the sequence representation and the structural representation of the same protein through contrastive learning to achieve alignment of feature spaces of different modalities; cross-modal fusion uses a multi-head cross-attention mechanism to fuse the aligned sequence and structural features to obtain a more comprehensive protein representation.

[0021] 4. Transprotein fusion module

[0022] To capture protein The local interaction between them is represented by the joint representation of their modes. and The cross-protein fusion module is applied on the protein pairs; this module consists of two layers: one layer is used to model the interaction of protein pairs, called the interaction mapping layer, which captures the interaction pattern at the residue level by calculating the multi-head interaction graph between protein pairs; the other layer is used to obtain the joint representation of protein pairs, called the attention pooling layer, which calculates the attention score based on the interaction graph and then aggregates to obtain the joint representation of protein pairs;

[0023] 5. Interaction prediction

[0024] Final protein union representation obtained from the cross-protein fusion module Is input into the classifier for PPI prediction:

[0025] ;

[0026] in represents the predicted label, is a classifier consisting of three fully connected linear layers.

[0027] Preferably, in step 1, the encoder extracts features by the following method:

[0028] First, at the local feature extraction level, a zero-filled TextCNN layer is used to extract local features of protein sequences, so that the model can recognize short sequence patterns that appear continuously in the sequence. lThe input sequence representation , local features Extracted by the following equation:

[0029] ；

[0030] in is a weight-sharing kernel with d channels, c is the kernel size, b is the bias vector, and i is the current convolution position, ranging from [0, l -c+1], indicating the starting position of the convolution window sliding on the sequence, l Represents the total length of the input protein sequence; this application is used as a standard classifier to map the protein joint representation obtained by the cross-protein fusion module to the final PPI prediction result;

[0031] At the global feature extraction level, a fragment-level self-attention mechanism is used to enhance the modeling ability of long-range dependencies while retaining the local features extracted by CNN. Specifically, a fragment-level self-attention mechanism is designed to balance local and global information. First, the sequence representation at the residue level is Divide into a series of residue blocks and obtain a fragment-level sequence representation , based on the following equation:

[0032] ;

[0033] Where m is the number of segments and p is the length of the segment, then the sinusoidal position is encoded Join To obtain position-aware sequence representation ;

[0034] ;

[0035] in is the frequency parameter, ω represents the frequency parameter, Represents the index of the feature dimension, ranging from 1 to d / 2, The total dimension of the feature is obtained by using sine and cosine functions of different frequencies to uniquely encode each position, making it easier for the model to learn the relative position relationship in the sequence;

[0036] Finally, a residual multi-head self-attention layer and a feedforward network are used to extract the global features of the fragment, where the residual multi-head self-attention layer captures the long-range dependencies of the sequence at the patch level, and the feedforward network further transforms and refines the feature representation to obtain the final sequence embedding. ;

[0037] ;

[0038] ;

[0039] in LN The representation layer is the normalization layer, and FFN stands for feed-forward neural network. With this method, global interactions between fragments can be achieved while retaining local features within each fragment, thereby obtaining higher quality sequence embedding; It is a multi-head self-attention layer, using a multi-head attention mechanism ,in Indicates splicing, is the linear transformation parameter, represents the kth attention head.

[0040] Preferably, step 2 is as follows:

[0041] First, in the global attention stage, two strategies are introduced based on the self-attention mechanism: random walk position encoding (RWPE) and graph structure encoding to enhance its perception of the overall graph structure. Among them, random walk position encoding captures the structural position information of nodes in the graph through a random walk process, and graph structure encoding encodes the overall topological structure of the graph through a high-order adjacency matrix. Specifically, given a protein graph , first based on its adjacency matrix Perform random graph diffusion to generate the RWPE matrix :

[0042] ;

[0043] in, represents the number of amino acid residues in the protein sequence, represents the dimension of the random walk position encoding (RWPE), that is, the size of the diffusion steps s, is a random walk operator, represents the degree matrix of A, s is the number of diffusion steps; then, the initial feature matrix of the protein graph is and Linear fusion to obtain structure-aware feature matrix:

[0044] ;

[0045] in, and is a learnable matrix, and is the bias vector, is the initial feature matrix of the protein graph; finally, the structure-aware feature matrix Input into the global attention mechanism for global feature extraction as follows:

[0046] ;

[0047] ;

[0048] in, represents the global attention score of nodes i and j, is a learnable scalar, represents the graph structure encoding calculated by the following formula, where n is the number of residue nodes in a protein graph, represents the structural encoding between node i and node r in the graph, ALL represents the set of all nodes in the protein graph, h is the number of attention heads, represents a nonlinear activation function, and denote the query vector and key vector respectively, and denote the learnable parameter matrices for query and key respectively, represents the structural relationship score between nodes i and j in the graph structure encoding, is the value matrix of the h-th attention head, represents the updated feature representation of node i, Indicates concatenating the outputs of H attention heads;

[0049] ;

[0050] in Represents the protein graph G The adjacency matrix of order, is the attenuation factor, satisfying , ；

[0051] Graph attention stage: New feature matrix of protein graph after global attention Encodes comprehensive contextual information; then a residual GAT is used to further capture the structural information of the protein through edge attention aggregation; this layer is defined as:

[0052] ;

[0053] ;

[0054] Where exp represents the exponential function, LeakyReLU is the activation function, , and Represent the feature representation of nodes i, j, and r respectively, represents the graph attention score of nodes i and j, is the attention vector, express is the transpose of , W is the shared parameter matrix used to apply linear transformations to node features, NBR represents the set of all neighbor nodes of node i, K is the number of attention heads, Indicates the aggregation of node i itself and its neighboring nodes.

[0055] Preferably, in step three,

[0056] (1) Cross-modal alignment stage: Contrastive learning is used to maximize the similarity between the global sequence representation and the global structure representation of the same protein, thereby achieving feature alignment between the two modalities. Specifically, the sequence embedding and structural embedding Perform global average pooling to obtain global sequence representations and global structure representation :

[0057] ;

[0058] Then, they are mapped to a new feature space using two modality-specific mapping blocks:

[0059] ;

[0060] ;

[0061] Where W is the learnable parameter matrix, b is the bias vector, and represent the structural and sequence characteristics of the ith residue, respectively. and They represent the global structural features and global sequence features obtained by average pooling, respectively. and is the transformation matrix of the first and second layers of the structural mode, and The bias vectors of the first and second layers of the structural mode, and is the transformation matrix of the first and second layers of the sequence modality, and Bias vectors of the first and second layers of the sequence modality;

[0062] Finally, contrastive learning is introduced on sequence-structure pairs to make the feature spaces of the two modalities closer; the sequence representation of protein i As an anchor point, we obtain a positive sample, i.e., the structural representation of the same protein. , and a set of negative samples, which are structural representations of other proteins in the same batch, and vice versa; in order to maximize the similarity between the anchor and the positive samples while minimizing the similarity with the negative samples, the InfoNCE loss function is introduced as follows:

[0063] ;

[0064] ;

[0065] ;

[0066] in is the vector similarity measure function, is the temperature coefficient, and Respectively represent the structure and sequence representation of the sample, the sequence representation of the current protein (anchor), is the corresponding positive sample (the structural representation of the same protein), It is a contrastive learning loss with sequence modality as anchor point; The structural representation of the current protein, is the corresponding positive sample, i.e., the sequence representation of the same protein, is the sequence representation of other proteins, ;

[0067] (2) Cross-modal fusion stage: After modality alignment, the present invention introduces a multimodal fusion block to further integrate the modalities and obtain the final cross-modal joint representation. :

[0068] ;

[0069] in is a feed-forward network, is the layer normalization layer, is a multi-head cross attention layer, It is a multi-head self-attention layer.

[0070] Preferably, in step 4,

[0071] (1) First, calculate the multi-head interaction graph between protein pairs:

[0072] ;

[0073] in represents the interaction score, k represents the number of attention heads, is the parameter matrix, is the attention vector, express The transpose of represents the Hadamard product, is the hyperbolic tangent activation function, is the parameter matrix used by the k-th head to transform the first protein feature, is the feature matrix of the first protein;

[0074] (2) Then, Apply column-wise and row-wise averaging operations separately:

[0075] ;

[0076] ;

[0077] in represents the jth column of the interaction matrix of the kth head, is the i-th row of the interaction matrix of the k-th head, is a normalized exponential function used to convert the score into a probability distribution. This step can obtain two attention score vectors and , among which represents the attention score of a protein to a single residue i in another protein, and k represents the number of attention heads;

[0078] (3) Finally, based on the corresponding attention score vector and ,right and Perform attention pooling to obtain the final joint representation of protein pairs :

[0079] ;

[0080] ;

[0081] ;

[0082] Where || represents the concatenation operation, k is the number of attention heads used in the present invention, and Represent the feature matrices of the first and second proteins, respectively, and are the parameter matrices for the first and second protein feature transformations, respectively.

[0083] Preferably, in step 5, the PPI prediction loss used is the cross entropy loss:

[0084] ;

[0085] in and Represent the number of samples and the number of PPI types, respectively. represents the true label that the i-th sample belongs to the c-th category, The model predicts the probability that the i-th sample belongs to the c-th class. Ultimately, the optimization objective of the framework consists of two parts: contrast loss and prediction loss:

[0086] ;

[0087] Where λ is an adjustable hyperparameter, represents the combined loss of prediction loss and contrast loss, represents the prediction loss, represents contrast loss.

[0088] Compared with the prior art, the present invention has the following advantages:

[0089] (1) The present invention designs two modules, Convformer and Graphformer, and introduces a global attention mechanism in each module, breaking through the limitation of existing methods that only focus on local features or global features. This design enables the model to capture both the local patterns and global dependencies of proteins, providing a more comprehensive protein representation.

[0090] (2) This paper proposes a cross-modal enhancement method based on contrastive learning, which solves the difference between sequence and structural modalities through an explicit modality alignment process. This method can not only maintain the unique information of each modality, but also give full play to the complementarity between modalities, thereby obtaining a richer protein representation.

[0091] (3) The cross-protein fusion module proposed in the present invention realizes the interaction modeling between protein pairs at the residue level for the first time by designing interaction mapping and attention pooling mechanism. This fine-grained modeling method can more accurately capture the essential characteristics of protein interactions and significantly improve the accuracy of PPI prediction.

[0092] (4) The present invention adopts a joint optimization strategy, and through the reasonable design of the loss function, the model can simultaneously optimize the two objectives of modal alignment and interaction prediction during the training process. This multi-task learning paradigm not only improves the performance of the model, but also enhances the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] The present invention will be further described below in conjunction with the accompanying drawings.

[0094] Figure 1 It is the overall technical roadmap of the present invention;

[0095] Figure 2It is a schematic diagram of the sequence mode representation of the present invention;

[0096] Figure 3 It is a schematic diagram showing the structural modal of the present invention;

[0097] Figure 4 This is a schematic diagram of cross-modal enhancement of the present invention;

[0098] Figure 5 This is a schematic diagram of the cross-protein fusion of the present invention;

[0099] Figure 6 Visualization results of the mutual attention of protein structures;

[0100] Figure 7 This is the prediction result of the single-nucleus network of the Wnt-related pathway;

[0101] Figure 8 This is the prediction result of the Wnt-related pathway cross-network. DETAILED DESCRIPTION

[0102] The present invention will be further described below in conjunction with the accompanying drawings.

[0103] Example 1

[0104] The overall technical roadmap of the present invention is as follows: Figure 1 As shown in the figure, deep learning-based PPI prediction usually adopts an encoder-decoder architecture, that is, a suitable neural network architecture is selected to encode the features of the two protein sequences and structures respectively: for the sequence part, the Convformer module is used to extract the sequence features; for the structure part, the Graphformer module is used to extract the structural features. This two-stream architecture design can make full use of the multimodal information of proteins, and then perform joint representation learning on the obtained protein representations, and finally realize the prediction of PPI.

[0105] 1. Sequence modal representation

[0106] The schematic diagram of sequence mode representation is as follows Figure 2 As shown in Figure 2, protein sequences contain rich sequence information, including key features such as motifs, conserved sites and domains, which are closely related to protein functions. In order to more fully mine the protein information in the sequence, the present invention designs a sequence feature encoder Convformer that combines CNN and multi-head self-attention mechanism; the encoder extracts features through the following methods:

[0107] 1. First, at the local feature extraction level, a zero-filled TextCNN layer is used to extract local features of protein sequences, enabling the model to recognize short sequence patterns (such as motifs) that appear continuously in the sequence. Given a length of l The input sequence representation , local features It can be extracted by the following equation:

[0108] ；

[0109] in is a weight-sharing kernel with d channels, c is the kernel size, b is the bias vector, and i is the current convolution position, ranging from [0, l -c+1], indicating the starting position of the convolution window sliding on the sequence. In this embodiment, in the convolution layer, the convolution kernel slides on the sequence, and the value range is [0, lc], where c is the convolution kernel size. By performing convolution operations at different positions i, the features of each local area in the sequence can be extracted.

[0110] 2. At the global feature extraction level, a fragment-level self-attention mechanism is used to enhance the modeling ability of long-range dependencies while retaining the local features extracted by CNN. Specifically, the present invention designs a fragment-level self-attention mechanism to balance local and global information. First, the sequence representation at the residue level is Divide into a series of residue blocks (fragments) and obtain a fragment-level sequence representation , based on the following equation:

[0111] ;

[0112] Where m is the number of segments and p is the length of the segment. Then the sinusoidal position encoding Join To obtain position-aware sequence representation ;

[0113] ;

[0114] in is the frequency parameter, ω represents the frequency parameter, represents the index of the feature dimension (ranging from 1 to d / 2), To increase the total dimension of the feature, each position can be uniquely encoded by using sine and cosine functions of different frequencies, making it easier for the model to learn the relative position relationship in the sequence.

[0115] 3. Finally, a residual multi-head self-attention layer and a feed-forward network are used to extract the global features of the fragment, where the residual multi-head self-attention layer captures the long-range dependencies of the sequence at the patch level, and the feed-forward network further transforms and refines the feature representation to obtain the final sequence embedding ;

[0116] ;

[0117] in LN The representation layer is the normalization layer, and FFN stands for feed-forward neural network. With this approach, global interactions between segments can be achieved while preserving local features within each segment, resulting in higher quality sequence embeddings.

[0118] 2. Structural modal representation

[0119] The schematic diagram of structural mode representation is as follows Figure 3 As shown, the present invention proposes a graph neural network model Graphformer for extracting 2D protein topology graphs The key structural information of proteins in the protein graph. The Graphformer proposed in the present invention mainly includes two stages: the global attention stage and the graph attention stage. Among them, the global attention stage captures the global dependencies between all nodes in the protein graph through the self-attention mechanism; the graph attention stage learns local structural features through edge attention aggregation; the global attention mechanism uses the learned global features as context information to guide the graph attention to effectively capture the structural information of the protein graph; through this design, the global features learned by the global attention mechanism can be used as context information to guide the graph attention to effectively capture the structural information of the protein graph.

[0120] 1. First, the global attention stage; in the global attention stage, in order to ensure that the model does not only extract global features from graph nodes as linear sequences, the present invention introduces two strategies based on the self-attention mechanism: random walk position encoding (RWPE) and graph structure encoding, so as to enhance its perception of the overall graph structure; among them, random walk position encoding captures the structural position information of nodes in the graph through a random walk process, and graph structure encoding encodes the overall topological structure of the graph through a high-order adjacency matrix; specifically, given a protein graph , first based on its adjacency matrix Perform random graph diffusion to generate the RWPE matrix :

[0121] ;

[0122] in, represents the number of amino acid residues in the protein sequence, represents the dimension of random walk position encoding (RWPE), that is, the size of the diffusion step number k, is a random walk operator, represents the degree matrix of A, and s is the number of diffusion steps. Next, the initial feature matrix of the protein graph is and Linear fusion to obtain structure-aware feature matrix:

[0123] ;

[0124] in and is a learnable matrix, and is the bias vector, is the initial feature matrix of the protein graph; finally, the structure-aware feature matrix Input into the global attention mechanism for global feature extraction as follows:

[0125] ;

[0126] ;

[0127] in represents the global attention score of nodes i and j, is a learnable scalar, represents the graph structure encoding calculated by the following formula, where n is the number of residue nodes in a protein graph, represents the structural encoding between node i and node r in the graph, ALL represents the set of all nodes in the protein graph, h is the number of attention heads, represents a nonlinear activation function, and denote the query vector and key vector respectively, and denote the learnable parameter matrices for query and key respectively, represents the structural relationship score between nodes i and j in the graph structure encoding, is the value matrix of the h-th attention head, represents the updated feature representation of node i, Indicates concatenating the outputs of H attention heads;

[0128] ;

[0129] in Represents the protein graph G The adjacency matrix of order, is the attenuation factor, satisfying , .

[0130] 2. Graph attention stage; after global attention, the new feature matrix of the protein graph Encodes comprehensive contextual information; then a residual GAT is used to further capture the structural information of the protein through side attention aggregation; this layer can be defined as:

[0131] ;

[0132] ;

[0133] Where exp represents the exponential function, LeakyReLU is the activation function, and denote the feature representation of nodes i and j respectively, represents the graph attention score of nodes i and j, is the attention vector, express is the transpose of , W is the shared parameter matrix used to apply linear transformations to node features, NBR represents the set of all neighbor nodes of node i, and K is the number of attention heads. After these two stages, we obtain the final structural embedding of the protein graph , Indicates the aggregation of node i itself and its neighboring nodes.

[0134] 3. Cross-modal enhancement

[0135] The schematic diagram of cross-modal enhancement is as follows Figure 4 As shown, due to the existence of modality differences, directly fusion sequence embedding and structural embedding This may lead to information loss or mutual interference, thus affecting the complementary effect between modalities. Therefore, the present invention designs a cross-modal enhancement module to achieve feature alignment and fusion. The module consists of two parts: cross-modal alignment and cross-modal fusion. Cross-modal alignment maximizes the similarity between the sequence representation and the structural representation of the same protein through contrastive learning to achieve alignment of feature spaces of different modalities; cross-modal fusion uses a multi-head cross-attention mechanism to fuse the aligned sequence and structural features to obtain a more comprehensive protein representation.

[0136] Cross-modal alignment stage: This paper uses contrastive learning to maximize the similarity between the global sequence representation and the global structure representation of the same protein, thereby achieving feature alignment between the two modalities. Specifically, the sequence embedding and structural embedding Perform global average pooling to obtain global sequence representations and global structure representation :

[0137] ;

[0138] Then, they are mapped to a new feature space using two modality-specific mapping blocks:

[0139] ;

[0140] ;

[0141] Where W is the learnable parameter matrix, b is the bias vector, and represent the structural and sequence characteristics of the ith residue, respectively. and They represent the global structural features and global sequence features obtained by average pooling, respectively. and is the transformation matrix of the first and second layers of the structural mode, and The bias vectors of the first and second layers of the structural mode, and is the transformation matrix of the first and second layers of the sequence modality, and Bias vectors for the first and second layers of the sequence modality.

[0142] Finally, the present invention introduces contrastive learning on sequence-structure pairs to make the feature spaces of the two modalities closer. As an anchor point, we obtain a positive sample, i.e., the structural representation of the same protein. , and a set of negative samples, which are structural representations of other proteins in the same batch, and vice versa; in order to maximize the similarity between the anchor and the positive samples while minimizing the similarity with the negative samples, the present invention introduces the InfoNCE loss function as follows:

[0143] ;

[0144] ;

[0145] ;

[0146] in is the vector similarity measure function, is the temperature coefficient, and Respectively represent the structure and sequence representation of the sample, the sequence representation of the current protein (anchor), is the corresponding positive sample (the structural representation of the same protein), It is a contrastive learning loss with sequence modality as anchor point; is the structural representation of the current protein, is the corresponding positive sample, i.e., the sequence representation of the same protein, is the sequence representation of other proteins, It is a contrastive learning loss with structural modes as anchors.

[0147] Cross-modal fusion stage: After modality alignment, the present invention introduces a multimodal fusion block to further integrate the modalities and obtain the final joint representation across modalities. :

[0148] ;

[0149] in is a feed-forward network, is the layer normalization layer, is a multi-head cross attention layer, It is a multi-head self-attention layer.

[0150] 4. Transprotein fusion module

[0151] Schematic diagram of cross-protein fusion Figure 5 As shown, in order to capture protein pairs The local interaction between them is represented by the joint representation of their modes. and The cross-protein fusion module is applied on the CNN. This module mainly consists of two layers: one layer is used to model the interaction of protein pairs, called the interaction mapping layer; the other layer is used to obtain the joint representation of protein pairs, called the attention pooling layer.

[0152] 1. First, the present invention calculates the multi-head interaction graph between protein pairs:

[0153] ;

[0154] in represents the interaction score, k represents the number of attention heads, is the parameter matrix, is the attention vector, express The transpose of represents the Hadamard product, is the hyperbolic tangent activation function, is the parameter matrix used by the k-th head to transform the first protein feature, is the feature matrix of the first protein.

[0155] 2. Then, Apply column-wise and row-wise averaging operations separately:

[0156] ;

[0157] ;

[0158] in represents the jth column of the interaction matrix of the kth head, is the i-th row of the interaction matrix of the k-th head, is a normalized exponential function used to convert the score into a probability distribution. This step can obtain two attention score vectors and , among which represents the attention score of one protein to a single residue i in another protein, and k represents the number of attention heads.

[0159] 3. Finally, based on the corresponding attention score vector and ,right and Perform attention pooling to obtain the final joint representation of protein pairs :

[0160] ;

[0161] ;

[0162] ;

[0163] Where || represents the concatenation operation, k is the number of attention heads used in the present invention, and Represent the feature matrices of the first and second proteins, respectively, and are the parameter matrices for the first and second protein feature transformations, respectively.

[0164] 5. Interaction prediction

[0165] Final protein union representation obtained from the cross-protein fusion module Is input into the classifier for PPI prediction:

[0166] ;

[0167] in represents the predicted label, is a classifier, which contains three fully connected linear layers. The PPI prediction loss used in this invention is the cross entropy loss:

[0168] ;

[0169] in and Represent the number of samples and the number of PPI types, respectively. represents the true label that the i-th sample belongs to the c-th category, The model predicts the probability that the i-th sample belongs to the c-th class. Finally, the optimization goal of the framework of the present invention consists of two parts: contrast loss and prediction loss:

[0170] ;

[0171] Where λ is an adjustable hyperparameter, represents the combined loss of prediction loss and contrast loss, represents the prediction loss, represents contrast loss.

[0172] Example 2 Verification and Evaluation

[0173] In order to fully verify the effectiveness and practicality of the method of the present invention, this embodiment has carried out a systematic evaluation on three representative benchmark datasets, namely, Yeast, Multi-species and Multi-class datasets. These three datasets represent PPI prediction tasks in different scenarios, among which Yeast and Multi-species are used to evaluate the binary classification prediction performance, and Multi-class is used to verify the multi-classification prediction capability.

[0174] 1) Performance evaluation based on the Yeast dataset

[0175] This example first verifies the prediction performance of the method of the present invention on the Yeast dataset. The dataset contains 11,188 PPI samples involving 2,497 non-redundant proteins. To ensure the reliability of the data, the positive samples in the dataset are all from the core subset of the experimentally verified protein interaction database (DIP), which has a high degree of credibility. On this dataset, the method of the present invention achieved the following performance indicators:

[0176] Accuracy: 98.93±0.11%;

[0177] Precision: 98.98±0.09%;

[0178] Recall: 98.88±0.13%;

[0179] F1 score: 98.93±0.11%.

[0180] Compared with the existing suboptimal method TAGPPI (accuracy 97.75±0.23%, precision 97.97±0.46%, recall 97.52±0.44%, F1 score 97.74±0.23%), the proposed method has achieved significant improvements in key indicators: accuracy increased by 1.2 percentage points, precision increased by 1 percentage point, recall increased by 1.4 percentage points, and F1 score increased by 1.2 percentage points. These experimental results fully confirm the superiority of the proposed method in the binary PPI prediction task.

[0181] 2) Generalization performance evaluation based on Multi-species dataset

[0182] In order to evaluate the prediction ability of the method of the present invention under different sequence similarity conditions, this example conducted a systematic test on multiple sequence similarity variants of the Multi-species dataset. The experimental results show that even under the harsh condition of sequence identity less than 1%, the method of the present invention can still maintain excellent performance:

[0183] Accuracy: 99.19±0.03%;

[0184] Accuracy: 99.87±0.08%;

[0185] Recall rate: 98.72±0.11%;

[0186] F1 score: 99.29±0.02%;

[0187] These experimental results show that the method of the present invention has a strong generalization ability and can effectively handle the interaction prediction task of protein pairs with large sequence differences. In particular, the stable performance under low sequence similarity conditions fully verifies the effectiveness of the cross-modal enhancement strategy proposed in the present invention.

[0188] 3) Multi-classification performance evaluation based on Multi-class dataset

[0189] In the more challenging multi-classification prediction task, the method of the present invention also shows excellent performance. The Multi-class dataset contains 75,875 samples involving 4,577 human proteins, and these interactions are divided into seven types: activation, binding, catalysis, expression, inhibition, post-translational modification, and reaction. On this dataset, the method of the present invention achieved a performance index of 58.07±0.68% in F1 score, which is a significant improvement of 1.8 percentage points in F1 score compared to the existing suboptimal method PIPR (F1 score 56.23±0.65%). This result fully demonstrates the superiority of the method of the present invention in dealing with complex multi-type protein interaction prediction tasks.

[0190] The above three groups of system experimental results jointly verify the following technical advantages of the method of the present invention:

[0191] 1. Achieved significant performance improvement on the standard binary classification prediction task;

[0192] 2. Shows excellent generalization ability under low sequence similarity conditions;

[0193] 3. Demonstrated strong classification capabilities in complex multi-classification prediction tasks.

[0194] Example 3 Analysis of protein complex binding surface using the method of the present invention

[0195] 1. Technical issues

[0196] In protein-protein interactions, determining the binding sites on the protein surface is crucial to understanding the interaction mechanism. This problem is based on an experimental observation: the interaction sites between proteins are usually located on the surface of their structures. However, existing methods often require a large amount of binding site annotation data for training and cannot directly identify potential binding regions from protein sequence and structural information. This reliance on annotation data severely limits the value of PPI prediction methods in practical applications.

[0197] 2. Solution

[0198] This example verifies the recognition ability of the method of the present invention on the protein binding surface based on the following steps:

[0199] (1) Data preparation

[0200] In this example, dimer complexes were first collected from the protein database (PDB) as test samples. In the selection process, the selected complexes should show significant physical complementarity in structure, and their spatial conformation should conform to the characteristics of the classic lock-key model. At the same time, these complexes should have complete three-dimensional structural information, and their binding interfaces should be clearly identifiable for subsequent analysis and verification.

[0201] (2) Model training

[0202] The present invention uses the Yeast dataset to train the model. During the entire training process, the model only receives the sequence and structure information of the protein as input, and does not introduce any binding site annotation data. The training process sequentially executes the four core modules of the present invention, including sequence modality representation, structural modality representation, cross-modality enhancement, and cross-protein fusion, thereby achieving comprehensive learning of protein features.

[0203] (3) Binding site analysis

[0204] In this example, the two protein chains of the dimer complex are first used as interacting protein pairs. Based on the mutual attention calculation mechanism of the present invention, the mutual attention scores between the two protein chains are calculated by the formula and :

[0205] ;

[0206] ;

[0207] in represents the j-th column of the interaction matrix of the k-th head, is the i-th row of the interaction matrix of the k-th head, is the normalized exponential function ， Used to convert scores into probability distributions.

[0208] On this basis, the top-2 residue segments with the highest attention scores were selected, and these high-attention regions were mapped back to the original complex structure for visual analysis.

[0209] 3. Technical Effects

[0210] Through the systematic analysis of multiple dimer complexes, the method of the present invention achieves the following technical effects:

[0211] (1) Prediction accuracy

[0212] The high attention areas identified by the model of the present invention show a high degree of overlap with the actual protein binding interface. This accurate prediction directly verifies that the method of the present invention can effectively capture potential protein binding sites. It is worth noting that this accuracy was obtained under the training condition without any binding site annotation data, which fully confirms that the method of the present invention has the ability to learn the physical and chemical laws of protein interactions.

[0213] (2) Results analysis

[0214] The prediction results of the present invention are displayed in two visual ways, such as Figure 6 As shown. The complex structure is presented using the Mol* visualization tool, with the cartoon mode on the left and the surface mode on the right. In the visualization results, yellow and blue are used to identify the two protein chains of the dimer complex, respectively, while the red area highlights the key interaction positions predicted by the model. These predicted interaction regions show significant consistency with the actual binding interface.

[0215] Example 4: Application of the method of the present invention to predict signal pathway networks

[0216] 1. Technical issues

[0217] In vivo, the diversity of protein interaction networks is not only reflected in direct physical contacts, but also includes functional associations. This complexity is closer to the characteristics of PPI networks in real biological scenarios. However, existing methods often only focus on a single type of interaction and find it difficult to effectively handle complex association patterns in signaling pathway networks. Therefore, it is of great significance to develop a prediction method that can handle multiple types of interactions at the same time.

[0218] 2. Solution

[0219] This example verifies the practical application effect of the method of the present invention by predicting the Wnt-related pathway network. The specific implementation scheme is as follows:

[0220] (1) Dataset construction

[0221] This example collects two Wnt-related pathway networks with different complexities for verification. The first is a single-core network, which consists of the interaction relationship between the CD9 core gene and 16 other genes, and these interactions play an important role in cell survival and tumor suppression. The second is a cross network, which has a more complex structure and contains 4 major core genes, involving 94 pairs of interaction relationships between a total of 75 genes, which play a key role in tumor growth and formation.

[0222] (2) Implementation of forecasting plan

[0223] First, this embodiment pre-trains the model on the Multi-species dataset. Then, the protein pair data in the above two pathway networks are input into the trained model for prediction. During the prediction process, the model extracts the sequence and structural features of the protein through the sequence modality representation module and the structural modality representation module, respectively, and then performs feature fusion through the cross-modality enhancement module, and finally obtains the final prediction result through the cross-protein fusion module.

[0224] 3. Technical Effects

[0225] The method of the present invention has achieved remarkable technical effects in predicting pathway networks of two different complexities:

[0226] (1) Single-core network prediction results

[0227] In this relatively simple network prediction task, the method of the present invention achieves 100% prediction accuracy. Figure 7As shown, the red circle marks the position of the core gene CD9, and the black solid line represents the interaction relationship correctly predicted by the model. This result shows that the method of the present invention can fully and accurately identify and predict protein interactions in such a large-scale network.

[0228] (2) Cross-network prediction results

[0229] In this more complex network prediction task, the method of the present invention still maintains a high accuracy of 97.9%. Figure 8 As shown in the figure, the red circles mark the positions of the four core genes, the black lines represent the interactions correctly predicted by the model, and the red lines represent a small amount of prediction deviation. In the prediction of a total of 94 pairs of interactions, the model only made misjudgments in very few cases, which fully demonstrated the stability and reliability of the method of the present invention in processing complex networks.

[0230] (3) Comprehensive analysis

[0231] Through the prediction experiments on these two networks of different complexity, the following technical advantages of the method of the present invention are fully verified: first, the method can simultaneously handle direct physical interactions and functional associations, reflecting the versatility of the method; second, it maintains high accuracy in networks of different scales and complexities, showing good adaptability and generalization capabilities; finally, the reliability and accuracy of the prediction results enable it to provide valuable reference information for biological research. These results together prove the important value of the method of the present invention in practical biological applications.

Claims

1. A protein interaction prediction method based on cross-modal enhanced representation learning, characterized in that: The following steps are involved:

1. Sequence modal representation The sequence feature encoder Convformer, which combines CNN and multi-head self-attention mechanism, realizes global interaction between segments while retaining local features within each segment, thus obtaining higher quality sequence embedding.

2. Structural modal representation A graph neural network model Graphformer is proposed to extract 2D protein topology graphs Key structural information of proteins in Representation diagram The nodes in Represents the edges between nodes; Graphformer consists of two stages: global attention stage and graph attention stage; the global features learned by the global attention mechanism are used as context information to guide graph attention to capture the structural information of the protein graph; After these two stages, the final structural embedding of the protein graph is obtained ,in Represents a matrix with n rows and d columns. The values ​​of the matrix are real numbers. n represents the number of residue nodes in a protein graph, and d represents the dimension of the feature.

3. Cross-modal enhancement The cross-modal enhancement module is used to achieve feature alignment and fusion. The module consists of two parts: cross-modal alignment and cross-modal fusion.

4. Transprotein fusion module To capture protein The local interaction between them is represented by their modal joint and The cross-protein fusion module is applied on the protein pairs; this module consists of two layers: one layer is used to model the interaction of protein pairs, called the interaction mapping layer, which captures the interaction pattern at the residue level by calculating the multi-head interaction graph between protein pairs; the other layer is used to obtain the joint representation of protein pairs, called the attention pooling layer, which calculates the attention score based on the interaction graph and then aggregates to obtain the joint representation of protein pairs; 5. Interaction prediction Final protein union representation obtained from the cross-protein fusion module Is input into the classifier for PPI prediction: ； in represents the predicted label, is a classifier consisting of three fully connected linear layers.

2. The method according to claim 1, characterized in that In step 1, the encoder extracts features by: First, at the local feature extraction level, a zero-filled TextCNN layer is used to extract local features of protein sequences, so that the model can recognize short sequence patterns that appear continuously in the sequence. l The input sequence representation , local features Extracted by the following equation: ； in is a weight-sharing kernel with d channels, c is the kernel size, b is the bias vector, and i is the current convolution position, ranging from [0, l -c+1], which indicates the starting position of the convolution window sliding on the sequence; At the global feature extraction level, the fragment-level self-attention mechanism is used to balance local and global information. First, the residue-level sequence representation Divide into a series of residue blocks and obtain a fragment-level sequence representation , based on the following equation: ； Where m is the number of segments and p is the length of the segment, then the sinusoidal position is encoded Join To obtain position-aware sequence representation ; ； in is the frequency parameter, ω represents the frequency parameter, Represents the index of the feature dimension, ranging from 1 to d / 2, To increase the total feature dimension, each position is uniquely encoded by using sine and cosine functions of different frequencies, making it easier for the model to learn the relative position relationship in the sequence; Finally, a residual multi-head self-attention layer and a feedforward network are used to extract the global features of the fragment, where the residual multi-head self-attention layer captures the long-range dependencies of the sequence at the patch level, and the feedforward network further transforms and refines the feature representation to obtain the final sequence embedding. ; ； in LN Represents the normalization layer, FFN represents the feedforward neural network, and uses a multi-head attention mechanism ,in Indicates splicing, is the linear transformation parameter, represents the kth attention head.

3. The method according to claim 1, characterized in that Step 2 is as follows: First, in the global attention stage, two strategies are introduced based on the self-attention mechanism: random walk position encoding (RWPE) and graph structure encoding to enhance its perception of the overall graph structure. , first based on its adjacency matrix Perform random graph diffusion to generate the RWPE matrix : ； in, represents the number of amino acid residues in the protein sequence, represents the dimension of the random walk position encoding RWPE, that is, the size of the diffusion step s, is a random walk operator, represents the degree matrix of A, s is the number of diffusion steps; then, the initial feature matrix of the protein graph is and Linear fusion to obtain structure-aware feature matrix: ； in, and is a learnable matrix, and is the bias vector, is the initial feature matrix of the protein graph; finally, the structure-aware feature matrix Input into the global attention mechanism for global feature extraction as follows: ； ； in represents the global attention score of nodes i and j, is a learnable scalar, represents the graph structure encoding calculated by the following formula, where n is the number of residue nodes in a protein graph, represents the structural encoding between node i and node r in the graph, ALL represents the set of all nodes in the protein graph, h is the number of attention heads, represents a nonlinear activation function, and denote the query vector and key vector respectively, and denote the learnable parameter matrices for query and key respectively, represents the structural relationship score between nodes i and j in the graph structure encoding, is the value matrix of the h-th attention head, represents the updated feature representation of node i, Indicates concatenating the outputs of H attention heads; ； in Represents the protein graph G The adjacency matrix of order, is the attenuation factor, satisfying , ； Graph attention stage: New feature matrix of protein graph after global attention Encodes comprehensive contextual information; then a residual GAT is used to further capture the structural information of the protein through edge attention aggregation; this layer is defined as: ； ； Where exp represents the exponential function, LeakyReLU is the activation function, and denote the feature representation of nodes i and j respectively, represents the graph attention score of nodes i and j, is the attention vector, express is the transpose of , W is the shared parameter matrix used to apply linear transformations to node features, NBR represents the set of all neighbor nodes of node i, K is the number of attention heads, Indicates the aggregation of node i itself and its neighboring nodes.

4. The method according to claim 1, characterized in that: In step three, (1) Cross-modal alignment stage: Contrastive learning is used to maximize the similarity between the global sequence representation and the global structure representation of the same protein, thereby achieving feature alignment between the two modalities. Specifically, the sequence embedding and structural embedding Perform global average pooling to obtain global sequence representations and global structure representation : ； Then, they are mapped to a new feature space using two modality-specific mapping blocks: ； ； Where W is the learnable parameter matrix, b is the bias vector, and represent the structural and sequence characteristics of the ith residue, respectively. and They represent the global structural features and global sequence features obtained by average pooling, respectively. and is the transformation matrix of the first and second layers of the structural mode, and The bias vectors of the first and second layers of the structural mode, and is the transformation matrix of the first and second layers of the sequence modality, and Bias vectors of the first and second layers of the sequence modality; Finally, contrastive learning is introduced on sequence-structure pairs to make the feature spaces of the two modalities closer; the sequence representation of protein i As an anchor point, we obtain a positive sample, i.e., the structural representation of the same protein. , and a set of negative samples, which are structural representations of other proteins in the same batch, and vice versa; in order to maximize the similarity between the anchor and the positive samples while minimizing the similarity with the negative samples, the InfoNCE loss function is introduced as follows: ； ； ； in is the vector similarity measure function, is the temperature coefficient, and Represent the structure and sequence representation of the sample respectively, The sequence representation of the current protein, is the corresponding positive sample, i.e., the structural representation of the same protein, It is a contrastive learning loss with sequence modality as anchor point; The structural representation of the current protein, is the corresponding positive sample, i.e., the sequence representation of the same protein, is the sequence representation of other proteins, It is a contrastive learning loss with structural modality as anchor; (2) Cross-modal fusion stage: After modal alignment, a multimodal fusion block is introduced to further integrate the modalities and obtain the final cross-modal joint representation. : ； in is a feed-forward network, is the layer normalization layer, is a multi-head cross attention layer, It is a multi-head self-attention layer.

5. The method according to claim 1, characterized in that In step four, (1) First, calculate the multi-head interaction graph between protein pairs: ； in represents the interaction score, k represents the number of attention heads, is the parameter matrix, is the attention vector, express The transpose of represents the Hadamard product, is the hyperbolic tangent activation function, is the parameter matrix used by the k-th head to transform the first protein feature, is the feature matrix of the first protein, is the parameter matrix used by the k-th head to transform the second protein feature, is the feature matrix of the second protein; (2) Then, Apply column-wise and row-wise averaging operations separately: ； ； in represents the j-th column of the interaction matrix of the k-th head, is the i-th row of the interaction matrix of the k-th head, is a normalized exponential function used to convert the score into a probability distribution. This step can obtain two attention score vectors and , whose elements represents the attention score of a single residue i of a protein in another protein, and k represents the number of attention heads; (3) Finally, based on the corresponding attention score vector and ,right and Perform attention pooling to obtain the final joint representation of protein pairs : ； ； ； Where || represents the concatenation operation, k is the number of attention heads used in the present invention, and Represent the feature matrices of the first and second proteins, respectively, and are the parameter matrices for the first and second protein feature transformations, respectively, Represents the final joint feature representation of protein pairs.

6. The method according to claim 1, characterized in that In step 5, the PPI prediction loss used is the cross entropy loss: ； in and Represent the number of samples and the number of PPI types, respectively. represents the true label that the i-th sample belongs to the c-th category, The model predicts the probability that the i-th sample belongs to the c-th class. Ultimately, the optimization objective of the framework consists of two parts: contrast loss and prediction loss: ； Where λ is an adjustable hyperparameter, represents the combined loss of prediction loss and contrast loss, represents the prediction loss, represents contrast loss.

Images