A method of an interpretable multi-modal molecular language model for drug-target interaction prediction

Abstract

The application discloses a method for drug-target interaction prediction by an interpretable multi-modal molecular language model, and relates to the technical field of artificial intelligence, and comprises the following steps: S1, acquiring drug molecule data and target protein sequence data, and constructing a drug-target heterogeneous graph network; S2, performing feature extraction on the heterogeneous graph network by using a graph neural network, and calculating attention coefficients between each node and its neighbor nodes in the heterogeneous graph network through an attention mechanism, and then updating the feature representation of the node, the application effectively alleviates the problem of scarce labeled data by constructing a drug-target heterogeneous graph, fusing multi-modal features through multi-core mapping, and using a contrast learning mechanism, significantly improves the generalization performance of the model in a few-shot scenario, fully excavates the complementarity and consistency of different perspective data, captures the complex structure and potential mode of the data, and then improves the prediction accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for predicting drug-target interactions using an interpretable multimodal molecular language model. Background Technology

[0002] Drug target interaction prediction (DTI) plays a central role in new drug development and drug reuse. While traditional in vitro experimental methods offer high accuracy, they are often costly and time-consuming, making them unsuitable for large-scale drug screening and analysis. In contrast, deep learning provides an efficient computational alternative, significantly narrowing the range of candidate compounds while improving the efficiency of drug and target identification. Among these methods, graph neural network-based modeling has become a research hotspot in recent years, demonstrating unique advantages in DTI prediction. For example, a method based on gene ontology information and expression profile data to construct graph neural networks has been proposed to improve the accuracy of target prediction; other studies have applied graph convolutional neural networks to drug molecule graphs and protein sequence characterization, achieving more generalized DTI prediction. These methods can capture long-range dependencies in graph structures, effectively characterizing the topological features of biological networks and the complex interaction patterns between drug molecules and target proteins.

[0003] Despite the progress made by existing methods, three challenges remain: (1) While multimodal fusion can improve prediction accuracy and generalization ability by utilizing the complementarity between modes, how to effectively deal with the inherent noise introduced in multimodal fusion is still a major challenge. Such noise may not only weaken the stability of model prediction, but may also lead to an increase in false positive results; (2) Another important challenge is that the number of known DTI relationships is limited, and the cost of obtaining high-quality experimental annotations is extremely high, which directly limits the generalization ability of the model; (3) The prediction results lack interpretability. Therefore, an interpretable multimodal molecular language model is proposed for the prediction of drug-target interactions. Summary of the Invention

[0004] The purpose of this invention is to address the problems in the prior art by proposing an interpretable multimodal molecular language model for predicting drug-target interactions.

[0005] A method for predicting drug-target interactions using an interpretable multimodal molecular language model includes the following steps: S1. Obtain drug molecule data and target protein sequence data, and construct a drug-target heterogeneous graph network; in the heterogeneous graph network, drugs and targets represent nodes in the graph, and the interaction relationship between drugs and targets represents the edges between nodes; S2. Use a graph neural network to extract features from the heterogeneous graph network, and use an attention mechanism to calculate the attention coefficient between each node and its neighboring nodes in the heterogeneous graph network, thereby updating the feature representation of the node. S3. Based on the heterogeneous graph network, a multi-kernel mapping method is used to process the data under multiple views in order to learn the differential distribution of multimodal data and map the feature representations under different views to a unified Hilbert space; wherein, the multiple views include node views, edge views and network views; S4. Construct a contrastive learning loss function and use the feature representation obtained after the multi-kernel mapping to perform contrastive learning in order to optimize the feature representation capability of the model. S5. Optimize the features of drug-target pairs using a pre-trained language model, and input the optimized features into a multilayer perceptron. Output the binary classification prediction results of drug-target interactions through the softmax function. S6. Output the attention weights calculated by the attention mechanism for visual interpretation of the prediction results.

[0006] Preferably, step S2 includes the following specific steps: A graph convolutional network is used to initially encode the node features; The attention coefficient between each node and its neighboring nodes is calculated using an attention mechanism and then normalized. We use the normalized attention coefficients to perform a weighted summation of the information from neighboring nodes; A multi-head attention mechanism is introduced, and the outputs of multiple attention heads are averaged to obtain the final comprehensive node feature representation.

[0007] Preferably, step S3 specifically includes the following steps: By randomly discarding nodes and boards in heterogeneous graph data, node views and edge views are obtained. In the graph neural network, a random deactivation mechanism is used to obtain the network view. Data from different views is encoded into a unified latent space using a mapping function; Multiple kernel functions are used to map the representations under different views to a unified Hilbert space, resulting in kernel matrices for each view.

[0008] Preferably, the plurality of kernel functions includes a cosine similarity kernel function and a Laplace kernel function.

[0009] Preferably, in step S4, the contrastive learning loss function is expressed as:

[0010] in, Indicates the number of kernel functions. Represents a set of nodes. This indicates the total number of samples. Represents the adjacency matrix. Indicates the first The first view The normalized feature representation of each sample Indicates the relationship with the first The first view corresponding to the One kernel function, This represents the loss normalization coefficient. This represents an exponential function.

[0011] Preferably, in step S5, optimizing the features of the drug-target pair using a pre-trained language model specifically includes: Graph convolutional neural networks and convolutional neural networks are used to encode drug molecule graphs and drug SMILES sequences to extract drug feature vectors; A convolutional neural network is used to encode the target protein sequence and extract the target feature vector; The drug feature vector and the target feature vector are fused and then input into a pre-trained language model based on LLaMA for feature optimization.

[0012] Preferably, in step S5, the overall loss function of the model is a weighted sum of the supervised cross-entropy loss function and the unsupervised contrastive learning loss function, used to train the model simultaneously using labeled and unlabeled data. The overall loss function is expressed as:

[0013] in, To compare the learning loss function, The supervised cross-entropy loss function is... and This is a hyperparameter.

[0014] Preferably, in step S6, key molecular interaction sites are identified by visualizing the top N target protein residues and drug atoms with the highest attention weights in the drug-target interaction pair.

[0015] Preferably, the optimization process of the model employs mini-batch sampling and stochastic gradient descent methods, and the model parameters and kernel mapping weights are iteratively updated according to the following formulas:

[0016]

[0017] in, Indicates model parameters, Indicates the first View kernel mapping weights, and This represents the learning rate.

[0018] Compared with existing technologies, the advantages of this invention are: 1. This invention effectively alleviates the problem of scarce labeled data by constructing a drug-target heterogeneous graph, fusing multi-core mapping with multimodal features and a contrastive learning mechanism, and significantly improves the generalization performance of the model in scenarios with few samples. This fully explores the complementarity and consistency of data from different perspectives, captures their complex structure and potential patterns, and thus improves the prediction accuracy of the model.

[0019] 2. This invention utilizes the attention mechanism to output the attention weights of drug atoms and target protein residues, enabling the model to focus on the most valuable parts of the input features, thereby revealing the molecular interaction logic behind the prediction and the molecular mechanism of drug-target interaction, making up for the lack of interpretability of prediction results in the prior art. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall structure of the M3LNet model in this invention.

[0021] Figure 2 This is a comparison chart of the DTI prediction results based on the clustering partitioning strategy in this invention.

[0022] Figure 3 This is a visualization of the attention weights of DTI in the 4N6H structure of this invention.

[0023] Figure 4 This is a visualization of the attention weights of DTI in the 5K31 structure of this invention. Detailed Implementation

[0024] To facilitate understanding of this application and to make the aforementioned objectives, features, and advantages of this application more apparent, a detailed description of specific embodiments of this application is provided below in conjunction with the accompanying drawings. Numerous specific details are set forth in the following description to provide a thorough understanding of this application.

[0025] Reference Figure 1-4 As shown, an interpretable multimodal molecular language model for predicting drug-target interactions includes the following steps: S1. Obtain drug molecule data and target protein sequence data, and construct a drug-target heterogeneity graph network; For the input drug-target dataset, a heterogeneous graph network was constructed, denoted as:

[0026] in, and Let A represent the set of nodes and the set of edges, respectively. In this graph, drugs and targets represent nodes, and the interaction between drugs and targets represents edges between nodes.

[0027] S2. Use a graph neural network to extract features from the heterogeneous graph network, and use an attention mechanism to calculate the attention coefficient between each node and its neighboring nodes in the heterogeneous graph network, thereby updating the feature representation of the node. For each node in the graph Its eigenvector is denoted as The normalized adjacency matrix of a graph is defined as:

[0028] in, Represents the identity matrix. Represents the adjacency matrix. Representation degree matrix.

[0029] Therefore, for each node Its feature vector is represented by a graph neural network as follows: (1) in, This represents the activation function. Let be the weight parameter matrix, and let the initial node features be... .

[0030] Subsequently, for each node and adjacent nodes Calculate the attention coefficient: (2) in, For trainable vectors, This represents a vector concatenation operation. This is the activation function. Next, the attention coefficients are normalized using the softmax function: (3) in, Represents a node The set of neighbors. The normalized attention weights are used to perform a weighted summation of the information from neighboring nodes, thereby generating an updated node feature representation: (4) Finally, to enhance the model's expressive power, a multi-head attention mechanism is introduced. For each attention head... First, calculate the representation obtained through the above steps, and then average the outputs of all heads according to equation (3) to obtain the final comprehensive representation: (5) in, This indicates the number of attention heads in a multi-head attention mechanism.

[0031] S3. Based on the heterogeneous graph network, a multi-kernel mapping method is used to process the data under multiple views in order to learn the differential distribution of multimodal data and map the feature representations under different views to a unified Hilbert space. Specifically, a multi-kernel mapping method is used to learn the differential distribution of multimodal data. For the data of each view, positive sample pairs are constructed based on the node view and the edge view, respectively. That is, by randomly discarding nodes and edges in the heterogeneous graph data, positive sample pairs are obtained. and .

[0032] Furthermore, to further enhance the model's expressive power, a network view is introduced, specifically by employing a random deactivation mechanism within the graph convolutional neural network to obtain... Therefore, graph data from diverse perspectives can be uniformly represented as:

[0033] in, Indicates the first The first view under the first view One sample.

[0034] Next, to address the dimensional differences between different views, they are first encoded into a unified latent space using a mapping function. Specifically, define the mapping function. Then the first The first view The representation of a sample can be written as:

[0035] in, Indicates the first The kernel mapping weights of the view are then applied. Subsequently, the resulting representation vector is subjected to... Norm normalization, to ensure the consistency of the representation space, takes the form of: Finally, multiple kernel functions are employed. Such as the cosine similarity kernel function and the Laplace kernel function, representing different views. Mapped to a unified Hilbert space In the middle. Therefore, the first The first view under the first view A kernel mapping can be represented as: (6) in, Indicates the relationship with the first The first view corresponding to the The kernel function can be further obtained at the nth kernel function. Kernel matrix from multiple perspectives: (7) S4. Construct a contrastive learning loss function and use the feature representation obtained after the multi-kernel mapping to perform contrastive learning in order to optimize the feature representation capability of the model. The loss function for multimodal contrastive kernel learning is defined as: (8) in, Indicates the number of kernel functions. Represents a set of nodes. This indicates the total number of samples. Represents the adjacency matrix. Indicates the first The first view The normalized feature representation of each sample Indicates the relationship with the first The first view corresponding to the One kernel function, This represents the loss normalization coefficient. This represents an exponential function that maps kernel similarity to similarity weights required for contrastive learning, amplifying the differences.

[0036] S5. Optimize the features of the drug-target pair using a pre-trained language model, and input the optimized features into a multilayer perceptron. Output the binary classification prediction results of the drug-target interaction through the softmax function.

[0037] To enhance the normalization effect and accelerate the model's convergence speed, labeled data was introduced into the training dataset for each drug-target combination. Graph convolutional neural networks and convolutional neural networks are used to encode drug molecule graphs and drug smiles, thereby extracting features and fusing them into feature vectors. Simultaneously, a convolutional neural network is used to encode the target sequence to obtain the target's feature vector. ; then, and Features of fusion into drug and target pairs This feature is further optimized using a large-scale language model based on LLaMA, and is represented as... The optimized features are then input into a multilayer perceptron, and the drug-target pair binary classification prediction results are obtained through a softmax function. On labeled data, the model is trained using the cross-entropy loss function, whose loss function is... Represented as: (9) in, This represents the original drug-target pair sample set. Ultimately, the goal of model training and parameter optimization is to minimize the overall loss. This method effectively utilizes both labeled and unlabeled data, thereby improving model performance. The overall loss is expressed as: (10) in, and This is a hyperparameter.

[0038] S6. Output the attention weights calculated by the attention mechanism for visual interpretation of the prediction results.

[0039] The objective function of the M3LNet proposed in this invention aims to minimize the total loss value in order to obtain the optimal model parameters. , is represented as:

[0040] (11) in, This represents the optimal kernel mapping weights. This represents the optimal model parameters. To optimize the above problems, mini-batch sampling is performed on both the original training samples and the constructed cross-view samples, i.e.:

[0041]

[0042] in, This indicates the number of samples in a mini-batch. Represents the sample's first Each view is then used. Subsequently, stochastic gradient descent is employed to minimize the loss value, expressed as: (12) in, This represents the learning rate.

[0043] During the training phase, the loss is iteratively reduced using stochastic gradient descent. Based on the above formula, the model parameters... The update formula is expressed as: (13) in, This represents the learning rate. Similarly, kernel-mapped weights. The optimization formula is expressed as: (14) in, This represents the learning rate. The complete pseudocode for M3LNet is as follows.

[0044]

[0045] The working process and principle of this invention are as follows: To further validate the robust performance of M3LNet in DTI prediction, the experiment compared M3LNet with four baseline methods: DeepConv-DTI, GraphDTA, MolTrans, and DrugBAN. This experiment used two datasets, BioSNAP and BindingDB, and performed clustering on each dataset. During data partitioning, single-chain clustering was first used to cluster drugs and targets based on drug fingerprints and pseudo-amino acid composition, respectively. Next, based on the clustered data samples, 60% of the drugs and targets were randomly selected as labeled data, and the remaining samples were treated as unlabeled data. For each dataset, this study used all labeled data and 80% of the unlabeled data for model training, and then used the 20% of unlabeled data for testing.

[0046] In this experiment, the results are presented as follows: Figure 2 These results show that, compared to the supervised learning paradigm, all methods exhibit a significant decline in predictive performance on both AUROC and AUPRC metrics. This decline may be due to the presence of unlabeled data, which reduces the model's ability to learn features. Figure 2 As shown, the M3LNet model proposed in this chapter consistently outperforms other baseline methods, further demonstrating that the M3LNet model helps improve the predictive performance of DTI. On the BindingDB dataset, M3LNet outperforms the second-best baseline method by 6.3% and 4.9% in AUROC and AUPRC metrics, respectively. On the BioSNAP dataset, M3LNet outperforms the second-best baseline method by 4.1% and 2.8% in AUROC and AUPRC metrics, respectively. Furthermore, M3LNet has the largest upper and lower quartile values ​​compared to other baseline methods. The performance improvement of M3LNet can be attributed to its ability to capture features from multiple perspectives, thereby enhancing its learning ability. Through the above analysis, M3LNet demonstrates strong capabilities in predicting drug-target interactions, providing an important method for drug discovery.

[0047] like Figure 3 and 4As shown, for sample pairs of drug-target protein interactions, this experiment uses 3D and 2D formats to display the weights of the top 30 target protein residues and drug atoms that received the highest attention, marked in blue. The 2D format primarily shows the binding sites identified by M3LNet, while the 3D format shows the binding sites identified using molecular docking technology. Taking the protein crystal structure 4N6H as an example, the attention weight analysis results show that the model pays significant attention to key amino acid residues such as LYS-81 and THR-78 during the prediction process. LYS-81 can form stable hydrogen bond interactions with the drug molecule Naltrindole, playing a crucial role in maintaining the overall stability of the drug-receptor complex; while THR-78 mainly participates in the binding process through polar interactions, contributing to improving the binding specificity between the drug and the target protein. These highly attention-grabbing residues are highly consistent with existing structural biology research findings, meaning that the model can effectively identify biologically significant key sites in the molecular recognition process. Similarly, in the analysis of the 5K31 structure, the randomly selected drug Triptonide was confirmed to interact with LYS-52 and TYR-56 in the target protein using molecular docking technology. These key residues also showed high importance in the attention weight distribution of M3LNet, consistent with the model's predictions. These results demonstrate that M3LNet possesses high accuracy in numerical prediction and exhibits good reliability and biological interpretability in identifying key sites of drug-target interactions, validating the model's effectiveness and practical application potential in DTI prediction tasks.

[0048] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A method for predicting drug-target interactions using an interpretable multimodal molecular language model, characterized in that, Includes the following steps: S1. Obtain drug molecule data and target protein sequence data, and construct a drug-target heterogeneous graph network; in the heterogeneous graph network, drugs and targets represent nodes in the graph, and the interaction relationship between drugs and targets represents the edges between nodes; S2. Use a graph neural network to extract features from the heterogeneous graph network, and use an attention mechanism to calculate the attention coefficient between each node and its neighboring nodes in the heterogeneous graph network, thereby updating the feature representation of the node. S3. Based on the heterogeneous graph network, a multi-kernel mapping method is used to process the data under multiple views in order to learn the differential distribution of multimodal data and map the feature representations under different views to a unified Hilbert space; wherein, the multiple views include node views, edge views and network views; S4. Construct a contrastive learning loss function and use the feature representation obtained after the multi-kernel mapping to perform contrastive learning in order to optimize the feature representation capability of the model. S5. Optimize the features of drug-target pairs using a pre-trained language model, and input the optimized features into a multilayer perceptron. Output the binary classification prediction results of drug-target interactions through the softmax function. S6. Output the attention weights calculated by the attention mechanism for visual interpretation of the prediction results.

2. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: Step S2 specifically includes the following steps: A graph convolutional network is used to initially encode the node features; The attention coefficient between each node and its neighboring nodes is calculated using an attention mechanism and then normalized. We use the normalized attention coefficients to perform a weighted summation of the information from neighboring nodes; A multi-head attention mechanism is introduced, and the outputs of multiple attention heads are averaged to obtain the final comprehensive node feature representation.

3. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: The specific steps in step S3 include: By randomly discarding nodes and boards in heterogeneous graph data, node views and edge views are obtained. In the graph neural network, a random deactivation mechanism is used to obtain the network view. Data from different views is encoded into a unified latent space using a mapping function; Multiple kernel functions are used to map the representations under different views to a unified Hilbert space, resulting in kernel matrices for each view.

4. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 3, characterized in that: The various kernel functions include the cosine similarity kernel function and the Laplace kernel function.

5. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: In step S4, the contrastive learning loss function is expressed as: in, Indicates the number of kernel functions. Represents a set of nodes. This indicates the total number of samples. Represents the adjacency matrix. Indicates the first The first view The normalized feature representation of each sample Indicates the relationship with the first The first view corresponding to the One kernel function, This represents the loss normalization coefficient. This represents an exponential function.

6. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: In step S5, optimizing the features of the drug-target pair using a pre-trained language model specifically includes: Graph convolutional neural networks and convolutional neural networks are used to encode drug molecule graphs and drug SMILES sequences to extract drug feature vectors; A convolutional neural network is used to encode the target protein sequence and extract the target feature vector; The drug feature vector and the target feature vector are fused and then input into a pre-trained language model based on LLaMA for feature optimization.

7. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 6, characterized in that: In step S5, the overall loss function of the model is a weighted sum of the supervised cross-entropy loss function and the unsupervised contrastive learning loss function, used to train the model simultaneously using labeled and unlabeled data. The overall loss function is expressed as: in, To compare the learning loss function, The supervised cross-entropy loss function is... and This is a hyperparameter.

8. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: In step S6, key molecular interaction sites are identified by visualizing the top N target protein residues and drug atoms with the highest attention weights in the drug-target interaction pairs.

9. The method for predicting drug-target interactions using an interpretable multimodal molecular language model according to claim 1, characterized in that: The optimization process of the model employs mini-batch sampling and stochastic gradient descent methods. The model parameters and kernel mapping weights are iteratively updated according to the following formulas: in, Indicates model parameters, Indicates the first View kernel mapping weights, and This represents the learning rate.

Images