Similarity relation network drug target interaction prediction method and system based on pre-training language model

Abstract

The invention relates to the field of drug and target interaction prediction, and discloses a similarity relation network drug and target interaction prediction method and system based on a pre-training language model, and the method comprises the steps: extracting an initial drug embedding feature and an initial protein embedding feature through the pre-training language model; constructing a drug similar network and a protein similar network; applying the graph neural network to a drug similar network and a protein similar network; extracting drug structure features by using a directed message passing neural network, and extracting protein structure features by using a convolutional neural network; performing cross fusion on the drug similarity relation characteristics and the protein similarity relation characteristics, and splicing cross fusion results with drug structure characteristics and protein structure characteristics; and inputting the drug-protein pair features into a classifier to carry out drug-target interaction prediction. According to the method, the generated embedded representation is combined with the similarity relation network, so that complementarity between semantic representation and relation representation is effectively mined.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of drug-target interaction prediction, in particular to a method and system for predicting drug-target interaction based on a pre-trained language model. BACKGROUND

[0002] New drug development is an important way to treat human diseases and improve human health. However, the traditional drug development process is often long, costly, and has a low success rate. Therefore, how to efficiently and accurately identify potential drug-target interactions (DTI) in the early stage of development has become a key to improving the efficiency and reducing the cost of new drug development. In recent years, with the rapid development of computational biology and artificial intelligence technology, DTI prediction methods based on traditional machine learning and deep learning have been widely used in drug development.

[0003] Traditional machine learning-based DTI prediction methods usually need to use prior knowledge or domain experience to perform feature engineering on drugs and proteins, and then use methods such as support vector machine (SVM), random forest (RF), and logistic regression (LR) to predict the interaction between drugs and targets. Although traditional machine learning methods have achieved some success in DTI prediction, they usually rely on manually designed molecular fingerprints, physicochemical properties, or amino acid features. These hand-designed features are difficult to effectively capture the complex nonlinear relationships between drugs and proteins, resulting in limited model prediction performance.

[0004] With the rapid development of deep learning technology, in the task of DTI prediction, people mainly focus on the prediction method based on structural features and the prediction method based on relational features. The prediction method based on structural features usually uses models such as convolutional neural networks (CNNs), graph neural networks (GNNs), and Transformers to mine structural information from the sequence representation or graph representation of drugs and proteins for prediction. DeepConv-DTI extracts drug molecule fingerprint features and protein sequence features through one-dimensional convolutional neural networks, and uses a fully connected layer to calculate the interaction score. DeepDTA uses CNN to process drug SMILES strings and protein sequences respectively to extract structural features for DTI prediction. AttentionDTA introduces an attention mechanism to fuse the structural features of drugs and proteins based on DeepDTA, thereby improving the expression ability of sequence features. MT-DTI combines Transformer and CNN to extract features from drug SMILES strings and protein sequences, and uses a multi-layer predictor for DTI prediction. GraphDTA, GraphomerDTI, and MolTrans represent feature learning methods based on drug molecule graphs and protein sequences. GraphDTA takes drug molecule graphs and protein sequences as input, uses graph convolution networks (GCN), graph attention networks (GAT), graph isomorphism networks (GIN), and graph attention-graph convolution networks (GAT-GCN) to learn drug molecule graph features, and uses CNN to learn protein sequence features, then concatenates the two structural features for DTI prediction. GraphomerDTI replaces the CNN-based drug sequence features with drug molecule graph features learned by the graph Transformer model based on AttentionDTA. MolTrans extracts structural features from drug molecule subgraphs and protein sub-sequences through a frequent continuous sub-sequence mining algorithm, and enhances embedding representation with Transformer to improve prediction accuracy. Although these methods have improved the performance of DTI prediction to some extent, they only focus on the structural information of a single modality, ignoring the potential relationship between drugs and targets. To overcome the above limitations, researchers have begun to focus on prediction models based on relational features, trying to enhance the reasoning ability of the model by constructing and fusing multiple relationship networks.DTINet is a network-based integration method for drug target interaction prediction. It uses drug-protein interaction relationships, drug-disease association relationships, and other relationships to construct heterogeneous networks. It achieves DTI prediction through network diffusion algorithms, feature compression, and matrix completion strategies. DTI-CNN is a drug target interaction prediction method based on feature representation learning and deep neural networks. It introduces a feature selector based on denoising autoencoders and an interaction predictor based on convolutional neural networks to DTINet for DTI prediction. MHGNN-DTI and AMGDTI use meta-path GNN and adaptive meta-graph methods to aggregate relationship information in heterogeneous networks for DTI prediction on multi-source heterogeneous biological information networks. MULGA further proposes a multi-view fusion-based graph auto-encoding framework and combines a reliable negative sample screening strategy to effectively integrate the relationship features between drugs and proteins. MIDTI uses multi-view similar network fusion and interaction attention mechanisms to more accurately capture the complex relationships between different entities. Although relationship-based models have significant advantages in revealing the potential semantics in biological networks, they can effectively extract drug and protein features with rich semantics and improve DTI prediction accuracy. However, their core limitation is the over-reliance on high-quality prior network information. These prior information, such as drug-protein, drug-drug interaction, or drug-disease association networks, is often difficult to obtain for newly discovered or rare proteins, greatly limiting the practical application of these methods. In addition, since these models usually lack deep modeling capabilities for molecular structure features, their generalization performance is severely limited in cold start scenarios or incomplete relationship networks.

[0005] In recent years, the successful application of pre-trained language models (PLMs) in natural language processing has driven their rapid development in bioinformatics, providing a new breakthrough for DTI representation learning. The universal embedding representation constructed by PLM has certain transferability, and compared with sequence-based features, it has significant advantages in cold start scenarios, i.e., the model still has strong discriminative ability when predicting unseen drugs or unknown targets. However, most current work only uses PLM as an initial encoder, ignoring the structural dependency between drug molecules and proteins, and lacking cross-modal deep fusion mechanisms, which limits the comprehensive representation ability of the model.

[0006] Currently, there is no effective solution to the problems in related technologies. SUMMARY

[0007] (I) Technical problems solved

[0008] In view of the deficiencies in the prior art, the present application provides a similar relationship network drug target interaction prediction method and system based on a pre-trained language model, which has the advantage of enhancing relationship feature expression capability, thereby solving the problems in the prior art of neglecting the structural dependency between drug molecules and proteins, and lacking a cross-modal deep fusion mechanism, which limits the comprehensive expression capability of the model.

[0009] (II) Technical solutions

[0010] To achieve the above-mentioned advantage of enhancing relationship feature expression capability, the present application adopts the following specific technical solutions:

[0011] According to one aspect of the present application, a similar relationship network drug target interaction prediction method based on a pre-trained language model is provided, which comprises:

[0012] S1, inputting a drug SMILES string and a protein sequence into a pre-trained language model, and extracting initial drug embedding features and initial protein embedding features through the pre-trained language model;

[0013] S2, taking the initial drug embedding features and the initial protein embedding features as an initial feature matrix, and constructing a drug similarity network and a protein similarity network based on a multi-view low-rank sparse representation technology;

[0014] S3, applying a graph neural network to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; using a directed message passing neural network to extract drug structure features, and using a convolutional neural network to extract protein structure features;

[0015] S4, cross-fusing the drug similarity relationship features and the protein similarity relationship features based on a cross-fusion attention mechanism, concatenating the cross-fusion results with the drug structure features and the protein structure features respectively to obtain drug-protein pair features;

[0016] S5, inputting the drug-protein pair features into a classifier for drug-target interaction prediction to obtain a prediction result of the drug-target interaction.

[0017] Preferably, the step of inputting the drug SMILES string and the protein sequence into the pre-trained language model and extracting the initial drug embedding features and the initial protein embedding features through the pre-trained language model comprises:

[0018] S11, input the drug SMILES string into the drug pre-training language model, and use the drug pre-training language model to extract initial drug embedding features of the drug SMILES string;

[0019] S12, input the protein sequence into the protein pre-training language model, and use the multi-layer Transformer architecture of the protein pre-training language model to encode the protein sequence to generate initial protein embedding features.

[0020] Preferably, the initial drug embedding features and the initial protein embedding features are taken as an initial feature matrix, and a drug similarity network and a protein similarity network are constructed based on a multi-view low-rank sparse representation technology.

[0021] S21, a light-weight convolutional neural network module is introduced to adjust the initial drug embedding features and the initial protein embedding features respectively to obtain adjusted drug embedding features and protein embedding features;

[0022] S22, the adjusted drug embedding features are taken as initial drug feature matrix of the drug similarity network, and the adjusted protein embedding features are taken as initial protein feature matrix of the protein similarity network;

[0023] S23, based on the initial drug feature matrix and the initial protein feature matrix, and combined with a multi-view low-rank sparse representation technology, a drug similarity network and a protein similarity network are constructed respectively.

[0024] Preferably, the drug similarity network and the protein similarity network are constructed based on the initial drug feature matrix and the initial protein feature matrix, and combined with a multi-view low-rank sparse representation technology.

[0025] S221, select the drug molecular descriptors of the molecular access system fingerprint, the Morgan fingerprint and the topological fingerprint, and encode each drug molecular descriptor to convert the drug molecular descriptors into corresponding feature matrices;

[0026] S222, solve the affinity matrix corresponding to the feature matrix based on the multi-view low-rank sparse representation technology, and obtain the drug similarity matrix based on the affinity matrix;

[0027] S223, select the protein representation mode of amino acid composition, combination triplet descriptor, autocorrelation descriptor and pseudo amino acid composition, convert the protein representation mode into a corresponding feature matrix, and obtain the protein similarity matrix in the manner of step S222;

[0028] S224, normalize the initial drug feature matrix and the drug similarity matrix respectively to obtain a normalized drug feature matrix and a drug adjacency matrix, and construct a drug similarity network based on the normalized drug feature matrix and the drug adjacency matrix;

[0029] S225, normalize the initial protein feature matrix and the protein similarity matrix respectively to obtain a normalized protein feature matrix and a protein adjacency matrix, and construct a protein similarity network based on the normalized protein feature matrix and the protein adjacency matrix.

[0030] Preferably, the graph neural network is applied to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; the directed message passing neural network is used to extract drug structure features, and the convolutional neural network is used to extract protein structure features, comprising:

[0031] S31, based on the preset similarity relationship network feature embedding learning module, the graph convolutional neural network is applied to the drug similarity network and the protein similarity network respectively to obtain drug similarity relationship features and protein similarity relationship features;

[0032] S32, the drug SMILES string is converted into a molecular graph, the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit to obtain drug molecular graph features, and the drug molecular graph features are taken as drug structure features;

[0033] S33, the protein amino acid sequence is encoded into a digital vector, the digital vector is mapped to an embedding space through an embedding layer, the protein sequence features are obtained in the embedding space through a one-dimensional convolutional layer, and the protein sequence features are taken as protein structure features.

[0034] Preferably, the drug SMILES string is converted into a molecular graph, the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit to obtain drug molecular graph features, comprising:

[0035] S321, the drug SMILES string is converted into a molecular graph by using a chemical information toolkit, wherein the node set in the molecular graph is the atom set in the drug molecule, and the edge set in the molecular graph is the chemical bond set between atoms;

[0036] S322, the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit, and the hidden state of the node, the initial hidden state and the hidden state of the edge, the initial hidden state are combined;

[0037] S323, in the message passing update, the node message is obtained by aggregating the edge hidden state at the node.

[0038] S324, obtaining the atomic hidden states according to the node message, and aggregating all the atomic hidden states in the molecular graph to obtain the drug molecule level representation.

[0039] Preferably, the cross-fusion attention mechanism is used to cross-fuse the drug similarity relationship features and the protein similarity relationship features, and the cross-fusion results are spliced with the drug structure features and the protein structure features, respectively, to obtain the drug-protein pair features, which include:

[0040] S41, based on the preset similarity relationship feature fusion module, cross-fusing the drug similarity relationship features and the protein similarity relationship features to obtain drug similarity relationship fusion features and protein similarity relationship fusion features;

[0041] S42, splicing the drug similarity relationship fusion features with the drug structure features to obtain drug combination features, and splicing the protein similarity relationship fusion features with the protein structure features to obtain protein combination features;

[0042] S43, splicing the drug combination features and the protein combination features to obtain the drug-protein pair features.

[0043] Preferably, the cross-fusion attention mechanism is used to cross-fuse the drug similarity relationship features and the protein similarity relationship features to obtain drug similarity relationship fusion features and protein similarity relationship fusion features, which include:

[0044] S411, constructing a similarity relationship feature fusion module composed of a plurality of cross-fusion attention layers, and each cross-fusion attention layer is composed of a self-attention module, a drug-protein cross-attention module, and a protein-drug cross-attention module;

[0045] S412, inputting the drug similarity relationship features and the protein similarity relationship features into the cross-fusion attention module, and obtaining self-attention drug similarity relationship features and self-attention protein similarity relationship features through the self-attention module;

[0046] S413, based on the self-attention drug similarity relationship features, calculating a first query matrix through the drug-protein cross-attention module, and based on the self-attention protein similarity relationship features, calculating a first key matrix and a first value matrix through the drug-protein cross-attention module;

[0047] S414, based on the first query matrix, the first key matrix, and the first value matrix, calculating drug fusion features.

[0048] Preferably, the calculating the drug fusion feature based on the first query matrix, the first key matrix and the first value matrix further comprises:

[0049] S415, calculating a second query matrix based on the self-attention protein similarity relationship feature through a protein-drug cross-attention module; calculating a second key matrix and a second value matrix based on the self-attention drug similarity relationship feature through the protein-drug cross-attention module;

[0050] S416, calculating a protein fusion feature based on the second query matrix, the second key matrix and the second value matrix;

[0051] S417, taking the drug fusion feature and the protein fusion feature as inputs of a next cross-fusion attention layer, and performing a new round of feature fusion; performing a splicing and mapping operation on the drug fusion features and the protein fusion features output by all cross-fusion attention layers in sequence to obtain a drug similarity relationship fusion feature and a protein similarity relationship fusion feature.

[0052] According to another aspect of the present application, a similarity relationship network drug target interaction prediction system based on a pre-trained language model is also provided, which comprises:

[0053] An initial embedding feature extraction module is configured to input a drug SMILES string and a protein sequence into a pre-trained language model respectively, and extract initial drug embedding features and initial protein embedding features through the pre-trained language model;

[0054] A similarity network construction model is configured to take the initial drug embedding features and the initial protein embedding features as initial feature matrices, and construct a drug similarity network and a protein similarity network based on a multi-view low-rank sparse representation technology;

[0055] A similarity relationship and structure feature extraction module is configured to apply a graph neural network to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; and extract drug structure features by using a directed message passing neural network, and extract protein structure features by using a convolutional neural network;

[0056] A feature fusion module is configured to cross-fuse the drug similarity relationship features and the protein similarity relationship features based on a cross-fusion attention mechanism, splice the cross-fusion results with the drug structure features and the protein structure features respectively, and obtain drug-protein pair features;

[0057] A prediction result generation module is configured to input the drug-protein pair features into a classifier to perform drug-target interaction prediction, and obtain a prediction result of the drug-target interaction.

[0058] (Three) beneficial effects

[0059] Compared with the prior art, the present application provides a pre-trained language model-based similar relationship network drug target interaction prediction method and system, which has the following beneficial effects:

[0060] (1) The present application extracts similarity relationship features using protein similarity networks and drug similarity networks and applies them to drug-target interaction prediction. The introduction of such similarity features can significantly improve the prediction accuracy of DTI. Importantly, the present application ingeniously uses intrinsic similarity measures to construct relationship networks, making them highly adaptable and successfully breaking the limitations of traditional relationship-based DTI models that rely heavily on external prior knowledge.

[0061] (2) The present application combines the embedding representation generated by the biological large language model with the similarity relationship network, thereby effectively exploiting the complementarity between semantic representation and relationship representation. Unlike traditional DTI models that rely solely on molecular or sequence features, the present application uses the context semantics captured by the large language model to further enhance the model's ability to use similarity networks to mine potential inter-molecular and inter-protein dependencies. This approach generates more discriminative and generalizable representation features for DTI prediction.

[0062] (3) The present application organically combines structural features with similarity relationship features, enabling the integration of global similarity information between molecules and targets while preserving the structural semantic information of molecules and proteins. This results in a more discriminative comprehensive representation in the feature space, further improving the accuracy and generalization performance of DTI prediction.

[0063] (4) The drug-target interaction prediction method provided by the present application designs a feature fusion module that integrates drug and protein similarity relationship features through a relationship cross-fusion attention mechanism, and combines them with drug and protein structure features to enhance the overall feature representation capability, thereby also demonstrating stronger generalization capability in cold-start prediction tasks. BRIEF DESCRIPTION OF DRAWINGS

[0064] To more clearly illustrate the technical solutions in the embodiments or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0065] Figure 1 is a flowchart of the pre-trained language model-based similar relationship network drug target interaction prediction method according to the embodiments of the present application;

[0066] Figure 2is a principle block diagram of a pre-training language model-based similar relationship network drug target interaction prediction system according to an embodiment of the present application;

[0067] Figure 3 is a whole architecture diagram of a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0068] Figure 4 is a feature fusion module structure diagram in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0069] Figure 5 is a structure diagram of self-attention (SA), drug-protein cross-attention (DPCA) and protein-drug cross-attention (PDCA) in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0070] Figure 6 is an influence diagram of embedding-dims and CA-layers on evaluation indexes in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0071] Figure 7 is an influence diagram of Att-heads and Neighbor-num on evaluation indexes in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0072] Figure 8 is a ROC curve diagram on a Human dataset in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0073] Figure 9 is a PR curve diagram on a Human dataset in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0074] Figure 10 is a ROC curve diagram on a C.elegans dataset in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0075] Figure 11 is a PR curve diagram on a C.elegans dataset in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the present application;

[0076] Figure 12 is a ROC curve diagram of a pre-training language model-based similar relationship network drug target interaction prediction method on a BioSNAP dataset according to an embodiment of the application;

[0077] Figure 13 is a PR curve diagram of a pre-training language model-based similar relationship network drug target interaction prediction method on a BioSNAP dataset according to an embodiment of the application;

[0078] Figure 14 is a ROC curve diagram of a pre-training language model-based similar relationship network drug target interaction prediction method on a DrugBank dataset according to an embodiment of the application;

[0079] Figure 15 is a PR curve diagram of a pre-training language model-based similar relationship network drug target interaction prediction method on a DrugBank dataset according to an embodiment of the application;

[0080] Figure 16 is a comparison diagram of experimental results on an imbalanced dataset in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the application;

[0081] Figure 17 is a comparison diagram of cold start prediction performance in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the application;

[0082] Figure 18 is a comparison diagram of cold start prediction performance in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the application;

[0083] Figure 19 is a drug-target embedding visualization diagram in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the application;

[0084] Figure 20 is a drug-target embedding visualization diagram in a pre-training language model-based similar relationship network drug target interaction prediction method according to an embodiment of the application.

[0085] In the figure:

[0086] 1, initial embedding feature extraction module; 2, similar network construction model; 3, similar relationship and structure feature extraction module; 4, feature fusion module; 5, prediction result generation module. DETAILED DESCRIPTION

[0087] To further illustrate the embodiments, the present application provides accompanying drawings which form part of the disclosure, which mainly serve to illustrate the embodiments, and can be used to explain the operating principles of the embodiments in conjunction with the relevant description. Those of ordinary skill in the art can understand other possible implementations and advantages of the present application by referring to these contents. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0088] According to an embodiment of the present application, a pre-trained language model-based similar relationship network drug target interaction prediction method and system are provided.

[0089] The present application will be further described in conjunction with the drawings and specific embodiments, as shown in Figure 1 According to an embodiment of the present application, a pre-trained language model-based similar relationship network drug target interaction prediction method is provided, which comprises:

[0090] S1, inputting drug SMILES string and protein sequence into pre-trained language model, extracting initial drug embedding features and initial protein embedding features by pre-trained language model;

[0091] S2, taking the initial drug embedding features and the initial protein embedding features as an initial feature matrix, and constructing a drug similarity network and a protein similarity network based on a multi-view low-rank sparse representation technology;

[0092] S3, applying a graph neural network to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; using a directed message passing neural network to extract drug structure features, and using a convolutional neural network to extract protein structure features;

[0093] S4, cross-fusing the drug similarity relationship features and the protein similarity relationship features based on a cross-fusion attention mechanism, splicing the cross-fusion results with the drug structure features and the protein structure features respectively to obtain drug-protein pair features;

[0094] S5, inputting the drug-protein pair features into a classifier to predict drug-target interactions, and obtaining a prediction result of the drug-target interactions.

[0095] In one embodiment, the inputting of the drug SMILES string and the protein sequence into the pre-trained language model to extract the initial drug embedding features and the initial protein embedding features by the pre-trained language model comprises:

[0096] S11, inputting the drug SMILES string into a drug pre-trained language model, and using the drug pre-trained language model to extract initial drug embedding features of the drug SMILES string;

[0097] S12, input the protein sequence into the protein pre-training language model, encode the protein sequence by using the multi-layer Transformer architecture of the protein pre-training language model to generate initial protein embedding features.

[0098] In one embodiment, the initial drug embedding features and the initial protein embedding features are taken as an initial feature matrix, and a drug similarity network and a protein similarity network are constructed based on a multi-view low-rank sparse representation technology.

[0099] S21, a light-weight convolutional neural network module is introduced to adjust the initial drug embedding features and the initial protein embedding features respectively to obtain adjusted drug embedding features and protein embedding features;

[0100] S22, the adjusted drug embedding features are taken as initial drug feature matrix of the drug similarity network, and the adjusted protein embedding features are taken as initial protein feature matrix of the protein similarity network;

[0101] S23, based on the initial drug feature matrix and the initial protein feature matrix, a drug similarity network and a protein similarity network are respectively constructed by combining a multi-view low-rank sparse representation technology.

[0102] In one embodiment, the drug similarity network and the protein similarity network are respectively constructed based on the initial drug feature matrix and the initial protein feature matrix by combining a multi-view low-rank sparse representation technology.

[0103] S221, select drug molecule descriptors of molecular access system fingerprints, Morgan fingerprints and topological fingerprints, and encode and embed each drug molecule descriptor to convert the drug molecule descriptors into corresponding feature matrices;

[0104] S222, solve the affinity matrix corresponding to the feature matrix based on the multi-view low-rank sparse representation technology, and obtain the drug similarity matrix based on the affinity matrix;

[0105] S223, select protein representation methods of amino acid composition, combination triplet descriptor, autocorrelation descriptor and pseudo-amino acid composition, convert the protein representation methods into corresponding feature matrices, and obtain the protein similarity matrix in the manner of step S222;

[0106] S224, normalize the initial drug feature matrix and the drug similarity matrix respectively to obtain normalized drug feature matrix and drug adjacency matrix, and construct the drug similarity network based on the normalized drug feature matrix and the drug adjacency matrix;

[0107] S225, normalize the initial protein feature matrix and the protein similarity matrix respectively to obtain a normalized protein feature matrix and a protein adjacency matrix, and construct a protein similarity network based on the normalized protein feature matrix and the protein adjacency matrix.

[0108] In one embodiment, the graph neural network is applied to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; the directed message passing neural network is used to extract drug structure features, and the convolutional neural network is used to extract protein structure features, including:

[0109] S31, based on the preset similarity relationship network feature embedding learning module, the graph convolutional neural network is applied to the drug similarity network and the protein similarity network respectively to obtain drug similarity relationship features and protein similarity relationship features;

[0110] S32, the drug SMILES string is converted into a molecular graph, the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit to obtain drug molecular graph features, and the drug molecular graph features are taken as drug structure features;

[0111] S33, the protein amino acid sequence is encoded into a digital vector, the digital vector is mapped to an embedding space through an embedding layer, the protein sequence features are obtained in the embedding space through a one-dimensional convolutional layer, and the protein sequence features are taken as protein structure features.

[0112] In one embodiment, the drug SMILES string is converted into a molecular graph, and the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit to obtain drug molecular graph features, including:

[0113] S321, the drug SMILES string is converted into a molecular graph by using a chemical information toolkit, wherein the node set in the molecular graph is the atom set in the drug molecule, and the edge set in the molecular graph is the chemical bond set between atoms;

[0114] S322, the directed message passing neural network is used to perform message passing update with the directed bond in the molecular graph as the basic unit, and the hidden state of the node, the initial hidden state and the hidden state of the edge, the initial hidden state are combined;

[0115] S323, in the message passing update, the node message is obtained by aggregating the edge hidden state at the node time;

[0116] S324, the atom hidden state is obtained according to the node message, and all atom hidden states in the molecular graph are aggregated to obtain a drug molecule level representation.

[0117] In one embodiment, the cross-fusion attention mechanism is based on the cross-fusion of drug similarity relationship features and protein similarity relationship features, the cross-fusion results are spliced with drug structure features and protein structure features respectively, and drug-protein pair features are obtained, including:

[0118] S41, based on the preset similarity relationship feature fusion module, cross-fusion of drug similarity relationship features and protein similarity relationship features is performed respectively to obtain drug similarity relationship fusion features and protein similarity relationship fusion features;

[0119] S42, the drug similarity relationship fusion features are spliced with the drug structure features to obtain drug combination features, and the protein similarity relationship fusion features are spliced with the protein structure features to obtain protein combination features;

[0120] S43, the drug combination features and the protein combination features are spliced to obtain drug-protein pair features.

[0121] In one embodiment, the cross-fusion attention mechanism is based on the cross-fusion of drug similarity relationship features and protein similarity relationship features, the cross-fusion results are spliced with drug structure features and protein structure features respectively, and drug-protein pair features are obtained, including:

[0122] S411, a similarity relationship feature fusion module composed of a plurality of cross-fusion attention layers is constructed, and each cross-fusion attention layer is composed of a self-attention module, a drug-protein cross-attention module and a protein-drug cross-attention module;

[0123] S412, the drug similarity relationship features and the protein similarity relationship features are input into the cross-fusion attention module, and the drug similarity relationship features and the protein similarity relationship features pass through the self-attention module to obtain self-attention drug similarity relationship features and self-attention protein similarity relationship features;

[0124] S413, based on the self-attention drug similarity relationship features, a first query matrix is calculated through the drug-protein cross-attention module; based on the self-attention protein similarity relationship features, a first key matrix and a first value matrix are calculated through the drug-protein cross-attention module;

[0125] S414, based on the first query matrix, the first key matrix and the first value matrix, drug fusion features are calculated.

[0126] S415, based on the self-attention protein similarity relationship features, a second query matrix is calculated through the protein-drug cross-attention module; based on the self-attention drug similarity relationship features, a second key matrix and a second value matrix are calculated through the protein-drug cross-attention module;

[0127] S416, based on the second query matrix, the second key matrix and the second value matrix, calculating protein fusion features;

[0128] S417, taking the drug fusion features and the protein fusion features as inputs of a next cross fusion attention layer, and performing a new round of feature fusion; performing splicing and mapping operations on the drug fusion features and the protein fusion features output by all cross fusion attention layers in sequence to obtain drug similarity relationship fusion features and protein similarity relationship fusion features.

[0129] According to another embodiment of the present application, as Figure 2 shown, a similar relationship network drug target interaction prediction system based on a pre-trained language model is also provided, and the system comprises:

[0130] An initial embedding feature extraction module 1 is configured to input a drug SMILES string and a protein sequence into a pre-trained language model respectively, and extract initial drug embedding features and initial protein embedding features through the pre-trained language model;

[0131] A similar network construction model 2 is configured to take the initial drug embedding features and the initial protein embedding features as initial feature matrices, and construct a drug similarity network and a protein similarity network based on a multi-view low-rank sparse representation technology;

[0132] A similar relationship and structure feature extraction module 3 is configured to apply a graph neural network to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; and extract drug structure features by using a directed message passing neural network, and extract protein structure features by using a convolutional neural network;

[0133] A feature fusion module 4 is configured to perform cross fusion on the drug similarity relationship features and the protein similarity relationship features based on a cross fusion attention mechanism, splice the cross fusion results with the drug structure features and the protein structure features respectively, and obtain drug-protein pair features;

[0134] A prediction result generation module 5 is configured to input the drug-protein pair features into a classifier to perform drug-target interaction prediction, and obtain a prediction result of the drug-target interaction.

[0135] In order to facilitate understanding of the above technical solutions of the present application, the above technical solutions of the present application are further described from the aspects of architecture and principle as follows:

[0136] The overall architecture diagram of the drug-target interaction prediction method proposed by the present application is as shown in Figure 3As shown, it is composed of seven modules: (A) feature extraction module based on pre-trained language model, (B) similar relationship network construction module, (C) feature extraction module based on graph neural network, (D) drug molecule graph feature extraction module, (E) protein sequence feature extraction module, (F) multi-feature fusion module, (G) classifier module.

[0137] In Figure 3 , Human, C.elegans, BioSNAP, DrugBank represent four data sets, Drug SMILES and Protein Seuqences represent drug SMILES string and protein sequence respectively, represent convolutional neural network, D-MPNN represents directed message passing neural network, GCN represents graph convolutional neural network, SA represents self-attention module, DPCA represents drug-protein cross-attention module, PDCA represents protein-drug cross-attention module, stack represents stacking operation, concatenate represents concatenation operation, Score represents DTI prediction score.

[0138] Specifically, the drug-target interaction prediction method provided by the present application firstly extracts pre-training features of drugs and proteins by using drug language model BPE and protein language model ESM-2 respectively; the features are fine-tuned by using CNN, and the pre-training features are used to initialize the drug similarity network and the protein similarity network, and then the graph neural network is applied to the similarity network to obtain drug relationship features and protein relationship features; the cross fusion attention module is used to cross fuse the two kinds of relationship features to obtain drug relationship fusion features and protein relationship fusion features; D-MPNN is used to extract drug structure features, and CNN is used to extract protein structure features; finally, the relationship features and the structure features are spliced and input into the classifier module for DTI prediction, which specifically includes:

[0139] I. Relationship network feature coding

[0140] ① Drug feature coding:

[0141] For drug feature coding, the SMILES string (Simplified Molecular Input Line Entry System) of the drug molecule is input into the drug pre-training language model, and the initial embedding features are extracted as the initialization features of the drug similarity network:

[0142] (1)

[0143] In the formula, represents the drug SMILES sequence, represents the number of drugs, representing a drug pre-training model, representing a set of numbers.

[0144] The present application maps the SMILES string of a drug molecule into a low-dimensional embedding vector with rich semantic information through a drug pre-training language model, which not only can capture the chemical structure information of the drug molecule, but also can extract potential semantic features through the deep learning ability of the pre-training language model, and enhance the expression ability of the drug features.

[0145] ② Protein feature encoding:

[0146] Since the features extracted by the protein pre-training language model can effectively capture complex features such as protein structure and function information, the present application encodes the protein using a protein pre-training language model to generate a low-dimensional vector representation of the protein as an initialization feature of the protein similarity network. Specifically, the present application inputs the protein sequence into the protein pre-training language model, encodes the sequence through the multi-layer Transformer architecture of the protein pre-training language model, and generates a low-dimensional vector representation:

[0147] (2)

[0148] In the formula, represents the amino acid sequence of the protein, represents the number of proteins, represents the protein pre-training language model ESM2.

[0149] II. Relationship network construction:

[0150] The relationship network provides crucial relationship information for drug-protein interaction prediction. The present application introduces drug similarity networks and protein similarity networks to capture the hidden associations between drugs and proteins. Drugs with similar chemical structures often exhibit similar physicochemical properties and biological activities, and proteins with similar sequences or structures often have similar functions and binding sites. Drug similarity networks can depict the potential functional connections between drugs with similar chemical structures, and protein similarity networks can reveal the functional consistency between proteins with similar sequences or structures. By aggregating neighbor features on these similarity networks, the model can enhance the representation ability of drugs and proteins, and thus improve the accuracy and robustness of interaction prediction. For example, Figure 3As shown in the (B) module in the figure, by using the initial drug embedding and the initial protein embedding obtained by the (A) module, after adjustment by the CNN, as the initial feature matrix of the drug similarity network and the protein similarity network, the two similarity networks contain rich drug and protein semantic features, and meanwhile, the drug similarity network and the protein similarity network are obtained by the Multi-View Low-Rank Sparse Representation (MV-LRSR) method based on the low-rank sparse representation of the multi-view to obtain the adjacency matrix of the drug similarity network and the protein similarity network to construct the drug similarity network and the protein similarity network.

[0151] ① Constructing the similarity network feature matrix:

[0152] The present application first introduces a lightweight convolutional neural network module to fine-tune the embedding representation generated by the drug pre-training language model:

[0153] (3)

[0154] In the formula, represents the initial drug embedding feature of the pre-training language model, represents the adjusted drug embedding feature. For proteins, the present application uses the same way to obtain the fine-tuned features:

[0155] (4)

[0156] In the formula, represents the initial protein embedding feature of the pre-training language model, the adjusted protein embedding feature. In order to fully excavate the semantic features between drugs and proteins, the embedding features obtained after fine-tuning are used as the initialization feature matrix for constructing the drug similarity network and the protein similarity network.

[0157] ② Constructing the similarity network adjacency matrix:

[0158] The adjacency matrix of the similar network is used to represent the similarity relationship between drugs or between proteins. To this end, a proper similarity calculation method needs to be used to measure the similarity between drug-drug or protein-protein, and on this basis, the adjacency matrix is constructed. Commonly used similarity calculation methods mainly include the Jaccard coefficient method and the cosine similarity method. However, such methods can only depict the shallow similarity in a single feature space, and it is difficult to effectively handle the redundancy and noise between features, and it is also impossible to fully mine the complementary information between multi-view features. In contrast, the multi-view based low-rank sparse representation method can integrate different feature views in a unified framework, effectively alleviating the influence of single fingerprint representation bias on similarity calculation. Therefore, the multi-view based low-rank sparse representation method is used to obtain the adjacency matrix of the drug similarity network and the protein similarity network. For drugs, the molecular access system (MACCS) fingerprint, Morgan fingerprint and topological fingerprint are used to represent the drug molecules, and each descriptor can be regarded as an independent view to depict the drug molecule features from different angles. Then each drug molecule descriptor is encoded and embedded to convert it into a feature matrix , wherein represents the feature dimension of the vth view, represents the number of drugs. Finally, the feature matrix corresponding to the affinity matrix is obtained by solving the multi-view based low-rank sparse representation method

[0159] (5)

[0160] , wherein represents the nuclear norm, represents the L1 norm, , and represent constraint coefficients, respectively used to control the low-rank constraint strength, the sparsity constraint strength and the multi-view consistency of the affinity matrix Figure 1 , Frobenius norm, matrix transpose, represents taking the absolute value of the matrix element by element. The affinity matrix is obtained by optimization, and then the affinity matrix is further processed to obtain the drug similarity matrix :

[0161] (6)

[0162] For proteins, the present application uses four representations: Amino Acid Composition (AAC), Composition / Transition / Distribution (CTD), Moran AutoCorrelation, and Pseudo-Amino Acid Composition (PAAC). where represents the feature dimension of the vth view, represents the number of proteins. Finally, the protein similarity matrix is obtained in the same way as the drug .

[0163] obtained drug similarity matrix and protein similarity matrix The drug feature matrix and similarity matrix are then normalized to obtain the feature matrix and adjacency matrix The protein feature matrix and adjacency matrix are obtained in the same way, and then the drug similarity relationship network and protein similarity relationship network are constructed by the feature matrix and the adjacency matrix. In the construction of the similarity relationship network, if the adjacency matrix is directly used, there are a large number of weak connections in the matrix, which will introduce noise and weaken the feature learning ability of the graph neural network. Therefore, the present application adopts a threshold screening strategy, which only retains the top 50 neighbors with the highest similarity for each node, thereby ensuring the connectivity of the network structure while reducing noise interference and enhancing the reliability of representation learning.

[0164] III. Feature embedding learning:

[0165] ① Similar network feature embedding learning: Figure 3 The (C) module in the (B) module shows a similarity relationship network feature embedding learning module. The module applies a graph convolutional neural network to the drug similarity relationship network and the protein similarity relationship network to extract drug embedding representation and protein embedding representation:

[0166] (7)

[0167] (8)

[0168]

[0169] (9)

[0170] (10)

[0171]

[0172] ② Drug molecule graph feature embedding learning:

[0173] Drug similarity relationship features represent drugs from the relationship level, while drug molecular structure features can directly reflect the inherent chemical properties and atomic level composition characteristics of drugs. Therefore, the application introduces drug molecular structure features on the basis of similarity relationship features to realize the complementarity of relationship information and structure information, thereby significantly improving the expression ability of drug features. To obtain drug molecular structure features, the application first converts drug SMILES strings into molecular graphs using the RDKit tool (RDKit is a widely used open source chemical informatics and molecular modeling toolkit, which is specially used for processing chemical information, molecular structure and related data calculation) wherein,​​​​​​​​​​​​​​​​​​​​​​​​​​​​ may contain element species, degree, valence, etc. information; edges characteristic vectors may contain structural information such as its bond type, whether conjugated, etc. Then, the present application learns drug structure features using D-MPNN. Unlike traditional atom-centered message passing, D-MPNN updates information with directed bonds as the basic unit, which can effectively avoid the repeated propagation of information in ring structures. In D-MPNN, denotes the hidden state of node , denotes the message of edge at the t-th step, denotes the message of edge at the (t+1)-th step, denotes the hidden state of edge at the t-th step, denotes the hidden state of edge at the (t+1)-th step, the initial hidden state of node is , the initial hidden state of edge is , and the message passing update process is:

[0174] (11)

[0175] (12)

[0176] In the formula, denotes the concatenation operation, denotes the activation function, denotes the neighborhood of node , . The node message is obtained by aggregating the node hidden state of edge at the t-th step :

[0177] (13)

[0178] Then, the atomic hidden state is obtained from the node message and its features:

[0179] (14)

[0180] Finally, the drug molecule-level representation is obtained by aggregating the hidden states of all nodes in the graph:

[0181] ​​ (15)

[0182] ③ Protein sequence feature embedding learning:

[0183] The sequence characteristics of a protein not only determine its spatial structure but also directly reflect its inherent biological functions and properties. Therefore, this invention introduces sequence features based on protein similarity characteristics to construct a protein characterization that combines similarity information with its own attributes. Specifically, the protein amino acid sequence is first encoded into a digital vector. The protein sequence features are then mapped to an embedding space using an embedding layer. Subsequently, protein sequence features are obtained through one-dimensional convolution.

[0184] (16)

[0185] In the formula, Indicates a convolutional layer. Indicates the embedding layer. Indicates protein sequence characteristics, Indicates the sequence length. Represents the dimension of sequence features.

[0186] IV. Multi-feature fusion:

[0187] By fusing drug similarity features, drug structural features, and protein similarity features with protein structural features, drugs and proteins can be characterized from multiple perspectives. The relationship features and structural features complement each other. Combining them improves the robustness of the features and mitigates biases introduced by a single data source. Therefore, this invention proposes a multi-feature fusion module, as shown in Figure 4. This module cross-fuses drug relationship features and protein relationship features through a similarity feature fusion module to obtain drug relationship fusion features and protein relationship fusion features respectively. These features are then concatenated with their respective structural features to obtain the final combined features.

[0188] exist Figure 4 middle, Indicates the characteristics of drug molecular diagrams. Indicates protein sequence characteristics, and These represent drug similarity features and protein similarity features, respectively. and These represent drug similarity fusion features and protein similarity fusion features, respectively. and These represent drug combinatorial features and protein combinatorial features, respectively, representing fusion structural features and relational features. Drug-protein pair features, SA is a self-attention module, DPCA is a drug-protein cross-attention module, PDCA is a protein-drug cross-attention module, Cat is a concatenation operation, and Linear is a linear layer.

[0189] To fully integrate the relationship features of drugs and proteins, the relationship feature fusion module (RFFM) uses a cross-fusion attention module (CFAM). This module consists of two self-attention modules (SA), a drug-protein cross-attention module (DPCA), and a protein-drug cross-attention module (PDCA). The structure of SA is shown in Figure 5 (A), and the structure of DPCA and PDCA is shown in Figure 5 (B).

[0190] In Figure 5 , and represent drug similarity relationship features and protein similarity relationship features, respectively, and represent self-attention features obtained by the SA module, and represent drug similarity relationship fusion features and protein similarity relationship fusion features, respectively, Linear and Linear_merge represent linear layers, Multi-Attention represents a multi-head attention module, Dropout represents a Dropout layer, Layer Norm represents a normalization layer, and Scaled Dot-Product represents a scaled dot-product attention mechanism.

[0191] For each cross-fusion attention module, the drug similarity relationship features and the protein similarity relationship features are passed through the SA module to obtain self-attention features and . The DPCA module uses the self-attention drug similarity relationship features to calculate the query matrix (Q), and then uses the self-attention protein similarity relationship features to calculate the key matrix (K) and the value matrix (V):

[0192] (17)

[0193] Finally, the drug fusion feature is calculated :

[0194] (18)

[0195] PDCA uses self-attention protein similarity relationship features The query matrix Q is calculated, and then self-attention drug similarity relationship features are used The key matrix K and the value matrix V are calculated, and protein fusion features are calculated The obtained drug fusion features and protein fusion features are used as the input of the next cross-fusion attention layer for further feature fusion. The relationship feature fusion module is composed of L layers of cross-fusion attention, and the drug similarity relationship fusion features and the protein similarity relationship fusion features are obtained by splicing and mapping the outputs of these layers:

[0196] (19)

[0197] (20)

[0198] Then, these features are spliced with the structure features to obtain the drug combination features and the protein combination features :

[0199] (21)

[0200] (22)

[0201] Finally, the drug combination features and the protein combination features are spliced to obtain the drug-protein pair features :

[0202] (23)

[0203] Five, DTI prediction:

[0204] The drug-protein pair features are input into a multi-layer perceptron (MLP) for prediction:

[0205] (24)

[0206] In the formula, is composed of four linear layers and Dropout layers connected alternately. The model loss function uses a binary cross-entropy loss function:

[0207] (25)

[0208] wherein, represents a true value, represents a predicted value, is a sample number.

[0209] In order to facilitate the understanding of the above technical solutions of the present application, the above technical solutions of the present application are further described from the aspects of architecture and principle as follows:

[0210] The present application selects four data sets of Human, C.elegans, BioSNAP and DrugBank to evaluate the performance of the model PLMSRNDTI (i.e. a pre-trained language model-based similar relationship network drug target interaction prediction model). Among them, the data scales of Human and C.elegans are small, while the data scales of BioSNAP and DrugBank are large, and the specific details of the data sets are shown in Table 1. In order to comprehensively evaluate the prediction ability of the model PLMSRNDTI, seven evaluation indexes of ACC, AUC, AUPR, Precision, Recall, F1-score and MCC are used to evaluate the model.

[0211] Table 1: Specific details of the data sets

[0212] Parameter analysis:

[0213] On the C.elegans data set, the present application carries out experiments on embedding dimension, number of attention heads, number of cross-fusion attention layers and number of neighbors of relationship similarity network nodes to determine the optimal settings. For embedding dimension, four dimension values of 128, 256, 512 and 1024 are selected for experiment, and the results are shown in Figure 6 (a), when the embedding dimension is 512, the values of ACC, AUC, AUPR, F1, MCC reach the maximum. For the number of attention heads, 1, 4, 8, 16 are selected for experiment, and when the number of attention heads is set to 8, all evaluation indexes except Precision reach the maximum value, and the results are shown in Figure 7 (b). For the number of cross-fusion attention layers, 1, 2, 3, 4 layers are selected for experiment, and the change trend of each index is shown in Figure 6 (c), which presents a trend of first increasing and then decreasing, and when the number of layers is 3, each index reaches the maximum. For the number of neighbors of relationship similarity network nodes, Top10, Top50, Top100, and no neighbor screening are selected for experiment, and when the number of node neighbors is 50, each index reaches the optimal value, as shown in Figure 7(d) as shown. According to the experimental analysis, the finally determined optimal hyperparameter configuration of the model is: the embedding dimension is 512, the number of attention heads is 8, the number of cross-fusion attention layers is 3, and the number of node neighbors of the relationship similarity network is 50.

[0214] In Figures 6-7 , ACC, AUC, AUPR, Precision, Recall, F1 and MCC are seven evaluation indexes, ACC is accuracy, AUC and AUPR are the areas under the ROC curve and PR curve, Precision is precision, Recall is recall, F1 is F1 score, and MCC is Matthew correlation coefficient.

[0215] Ablation experiment:

[0216] To evaluate the contribution of key parts of the model to the prediction performance, the present application constructs three ablation experiments on the C.elegans dataset. Among them, “PLMSRNDTI-Random” means that in the relationship similarity network, the node features are generated by random initialization to replace the initialization method based on the pre-trained language model; “Without Relation” means removing the drug relationship similarity network and protein relationship similarity network features; “Without Structure” means removing the drug molecular graph features and protein sequence features. The experimental results are shown in Table 7, and the results show that the absence of relationship features or structure features will cause the performance of the model to decrease, among which the MCC of the model decreases by 1.23% and 1.37% respectively, which shows that multi-modal feature fusion plays an important role in improving the performance of the model. At the same time, when the relationship similarity network features are replaced by random initialization instead of pre-trained language model features, all indicators decrease significantly, among which the MCC decreases by 1.26%, which further highlights the advantage of pre-trained language features in feature representation.

[0217] Table 7: Ablation comparison results of PLMSRNDTI on C.elegans

[0218] Comparison experiment:

[0219] To comprehensively evaluate the prediction performance of the model PLMSRNDTI, the present application selects CoaDTI, MHSADTI, GIFDTI, FMCADTI and BINDTI as baseline models, and uses 5-fold cross-validation strategy to evaluate on Human, C.elegans, BioSNAP and DrugBank datasets.

[0220] The model PLMSRNDTI of the application is compared with the baseline model on the Human dataset, and the experimental results are shown in Table 2. The highest index value in the table is marked in bold, and the second highest value is marked with an underline. On the Human dataset, the model PLMSRNDTI ranks first in ACC, AUC, AUPR, F1 and MCC, which are improved by 0.63%, 0.11%, 0.03%, 0.80% and 1.11% compared with the second model FMCADTI. Among them, the improvements of MCC and F1 are higher, indicating that the model has a good advantage in the balance of precision and recall. The Precision and Recall indexes of the model PLMSRNDTI of the application rank second in the baseline model, indicating that the model has good competitiveness in these two indexes. In order to intuitively show the performance difference of each model, the application gives the ROC curve and PR curve diagram as shown in Figures 8-9

[0221] Table 2: Comparison of experimental results on Human dataset

[0222] In Figures 8-9 , PLMSRNDTI represents the model of the application, CoaDTI, MHSADTI, GIFDTI, FMCADTI and BINDTI represent five comparison models, True Positive Rate and False Positive Rate represent true positive rate and false positive rate, and Precision and Recall represent precision and recall.

[0223] The experimental results on the C.elegans dataset further verify the superior performance of the application on small datasets. Compared with the baseline model, the model PLMSRNDTI of the application leads in all indexes, and compared with the best baseline model FMCADTI, it is improved by 0.76%, 0.07%, 0.12%, 0.02%, 1.54%, 0.96% and 1.38% in ACC, AUC, AUPR, Precision, Recall, F1 and MCC respectively. Among them, the improvements of Recall and MCC are higher, and the corresponding ROC curve and PR curve are shown in Figures 10-11

[0224] Table 3: Comparison of experimental results on C.elegans dataset

[0225] In Figures 10-11 ​​In Table 1, PLMSRNDTI represents the model of the present application, CoaDTI, MHSADTI, GIFDTI, FMCADTI and BINDTI represent five comparative models, True Positive Rate and False Positive Rate represent true positive rate and false positive rate, and Precision and Recall represent precision and recall.

[0226] The experimental results on the Human dataset and the C.elegans dataset show that the model PLMSRNDTI of the present application can achieve excellent prediction performance on small-scale datasets. Meanwhile, the present application further verifies the performance of the model on two larger-scale BioSNAP datasets and DrugBank datasets. The experimental results on the BioSNAP dataset are shown in Table 4, and the model of the present application leads the baseline model in all indicators, and the indicators ACC, AUC, AUPR, Recall, F1 and MCC are improved by 4.02%, 2.79%, 2.66%, 2.90%, 3.77% and 7.91% compared with the second model GIFDTI; at the same time, the Precision of the model of the present application is 0.8717, which is 3.04% higher than that of the FMCADTI model. These indicators are significantly improved compared with the baseline model, indicating that the model of the present application has more advantages on larger datasets. In order to more intuitively show the performance of the model, Figures 12-13 The ROC curve and the PR curve on the BioSNAP dataset are given.

[0227] Table 4: Comparison of experimental results on the BioSNAP dataset

[0228] In Figures 12-13 In Table 1, PLMSRNDTI represents the model of the present application, CoaDTI, MHSADTI, GIFDTI, FMCADTI and BINDTI represent five comparative models, True Positive Rate and False Positive Rate represent true positive rate and false positive rate, and Precision and Recall represent precision and recall.

[0229] The application is further experimented on the DrugBank dataset, and the experimental results are shown in Table 5. The model PLMSRNDTI of the application is superior to the baseline model in various indicators. The indicators ACC, AUC, AUPR, Recall, F1, and MCC are improved by 4.51%, 2.92%, 2.94%, 0.73%, 3.53%, and 8.91% compared with the excellent baseline model GIFDTI, respectively. At the same time, the Precision of the model of the application is improved by 0.64% compared with the baseline model FMCADTI. In addition, Figures 14-15 The ROC curve and the PR curve on the DrugBank dataset are given. The experimental results further prove that the model PLMSRNDTI has excellent prediction ability on a larger and more difficult dataset.

[0230] Table 5: Comparison of experimental results on the DrugBank dataset

[0231] In the Figures 14-15 , PLMSRNDTI represents the model of the application, CoaDTI, MHSADTI, GIFDTI, FMCADTI, and BINDTI represent five comparison models, True Positive Rate and False Positive Rate represent true positive rate and false positive rate, and Precision and Recall represent precision and recall.

[0232] To evaluate the prediction ability of the model PLMSRNDTI of the application on the unbalanced dataset, the application constructs an unbalanced dataset with a positive-to-negative sample ratio of 1:5 based on the BioSNAP and DrugBank datasets, and uses AUC and AUPR two indicators for evaluation, and the experimental results are shown in Figure 16 On the BioSNAP dataset, the model PLMSRNDTI is superior to all baseline models in two indicators. The AUC and AUPR are improved by 1.43% and 3.89% compared with the best baseline model GIFDTI, respectively. On the DrugBank dataset, the AUC value of the model is 0.9466, which is improved by 2.07% compared with the baseline model GIFDTI. The AUPR value is 0.8626, which is improved by 2.92% compared with the baseline model BINDTI. These results show that the model PLMSRNDTI can still maintain superior prediction performance under unbalanced data conditions.

[0233] In the Figure 16In the specific implementation, PLMSRNDTI represents the model of the application, CoaDTI, MHSADTI, GIFDTI, FMCADTI, and BINDTI represent five comparative models, and AUC and AUPR are the areas under the ROC curve and the PR curve, respectively.

[0234] Cold start comparative analysis:

[0235] In actual scenarios, the information of drugs and proteins is not completely known. With the development of new drugs and the discovery of new proteins, the model needs to have the ability to handle unknown objects. The pre-training model can obtain semantic-rich feature embedding through large-scale data learning, which can alleviate the challenge brought by part of unknown objects. To verify the performance of the model PLMSRNDTI in the cold start situation, the present application adopts the cold start division strategy on the Human dataset, and compares the prediction performance of the model PLMSRNDTI, the PLMSRNDTI-Random model and the optimal baseline model FMCADTI, wherein PLMSRNDTI-Random represents that the random initialization method is used to replace the pre-training language model feature of the relationship similarity network feature. The cold start experiment also uses five-fold cross-validation for evaluation. In the data set division method, 10% of the drugs are randomly selected as new drugs to join the test set in the drug cold start, and it is ensured that they are not visible in the training set. In the protein cold start, 10% of the proteins are taken as new proteins to join the test set, and these proteins are also not visible in the training set. The experimental results are shown in Table 6. In the drug cold start situation, compared with the model PLMSRNDTI-Random, the model PLMSRNDTI improves the ACC, AUC, AUPR, Precision, Recall, F1, and MCC by 24.99%, 38.57%, 19.56%, 2.63%, 41.63%, 27.24%, and 41.41%, respectively, indicating that the pre-training language model feature plays a key role in the model performance. Compared with the excellent baseline model FMCADTI, it is improved by 3.09%, 3.08%, 2.94%, 2.41%, 6.75%, 4.43%, and 6.74%, respectively. In the protein cold start situation, the model PLMSRNDTI is improved by 5.61%, 1.00%, 0.69%, 17.71%, 8.97%, and 8.60% in the ACC, AUC, AUPR, Recall, F1, and MCC, respectively, compared with the model PLMSRNDTI-Random, which further indicates the importance of the pre-training language model feature. Compared with the model FMCADTI, the ACC, AUC, AUPR, Recall, F1, and MCC are improved by 25.45%, 3.55%, 0.06%, 35.82%, 23.96%, and 32.77%, respectively. Figures 17-18The experimental comparison results under the conditions of drug cold start and protein cold start are shown, and the model PLMSRNDTI has excellent performance under the conditions of drug cold start and protein cold start.

[0236] Table 6: Cold start comparison results

[0237] wherein Difference1 = value (OURs) - value (OURs-Random), and Difference2 = value (OURs) - value (FMCADTI).

[0238] In Figures 17-18 , ACC, AUC, AUPR, Precision, Recall, F1 and MCC are seven evaluation indexes, ACC is accuracy, AUC and AUPR are the areas under the ROC curve and the PR curve, Precision is precision, Recall is recall, F1 is F1 score, MCC is Matthew correlation coefficient, PLMSRNDTI is the model of the application, PLMSRNDTI-Random indicates that in the relationship similarity network, the node features are generated by random initialization to replace the self-contrast model based on the pre-training language model initialization method, and FMCADTI is a comparison model.

[0239] Embedding visualization:

[0240] To prove the feature representation ability of the model for drug-target pairs, the application uses the t-SNE method to map the features of drug-target pairs in the test set to a two-dimensional space on the C. elegans dataset to show the distribution difference between positive and negative samples, and the visualization result is as shown in Figures 19-20 From the figure, it can be seen that the original drug-target pair features obtained by using the pre-training language model are difficult to distinguish in the low-dimensional space, and as the number of training rounds increases, the model gradually shows the discrimination ability, and at the 50th epoch, the features begin to show clustering trend, and the positive and negative samples are preliminarily separated; at the 100th epoch, the positive and negative samples have formed a relatively clear boundary in the feature space; when trained to the 150th epoch, the distribution boundary between the positive and negative samples is more significant, and the discrimination ability is further enhanced. The visualization result shows that the model can gradually learn effective semantic representation in the training process, and in Figures 19-20 , Positive Pairs and Negative Pairs represent positive sample pairs and negative sample pairs.

[0241] The application proposes a similar relationship network drug target interaction prediction method based on a pre-trained language model, which effectively improves the DTI prediction accuracy by fully fusing similar relationship features and structure features, the main innovation of the application is to use a pre-trained language model to extract drug and protein embedding, which enhances the expression ability of similar relationship network features, and adopts a cross-fusion attention mechanism to fuse two kinds of similar relationship features, and combines with the structure features, so as to realize the organic combination of structure features and similar relationship features. The experimental results on four data sets fully prove that the model is better than other baseline models as a whole, and has strong competitiveness. Through the ablation experiment, the effectiveness of the key part of the model is verified, at the same time, it is further verified that the model also has good performance in the drug cold start and protein cold start scene. In summary, the application shows excellent prediction performance in the field of DTI prediction, and has good application potential.

[0242] The above only describes the preferred embodiments of the application and is not intended to limit the application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for predicting drug target interaction based on a pre-trained language model, characterized in that, The method comprises: S1, inputting the drug SMILES string and the protein sequence into a pre-trained language model respectively, and extracting initial drug embedding features and initial protein embedding features through the pre-trained language model; S2, taking the initial drug embedding features and the initial protein embedding features as an initial feature matrix, and constructing a drug similarity network and a protein similarity network based on a multi-view low-rank sparse representation technology; S3, applying a graph neural network to the drug similarity network and the protein similarity network to extract drug similarity relationship features and protein similarity relationship features; simultaneously extracting drug structure features by using a directed message passing neural network, and extracting protein structure features by using a convolutional neural network; S4, cross-fusing the drug similarity relationship features and the protein similarity relationship features based on a cross-fusion attention mechanism, splicing the cross-fusion results with the drug structure features and the protein structure features respectively to obtain drug-protein pair features; S5, inputting the drug-protein pair features into a classifier to predict drug-target interactions, and obtaining a prediction result of the drug-target interactions. 2.The pre-trained language model based similar relation network drug target interaction prediction method of claim 1, wherein, The method comprises: S11, inputting the drug SMILES string into a drug pre-trained language model, and extracting initial drug embedding features of the drug SMILES string by using the drug pre-trained language model; S12, inputting the protein sequence into a protein pre-trained language model, and encoding the protein sequence by using a multi-layer Transformer architecture of the protein pre-trained language model to generate initial protein embedding features. 3.The pre-trained language model based similar relation network drug target interaction prediction method of claim 1, wherein, The method comprises: S21, introducing a light-weight convolutional neural network module to adjust the initial drug embedding features and the initial protein embedding features respectively to obtain adjusted drug embedding features and protein embedding features; S22, taking the adjusted drug embedding features as an initial drug feature matrix of the drug similarity network, and taking the adjusted protein embedding features as an initial protein feature matrix of the protein similarity network; S23, constructing the drug similarity network and the protein similarity network based on the initial drug feature matrix and the initial protein feature matrix and combining a multi-view low-rank sparse representation technology. 4.The pre-trained language model based similarity relationship network drug target interaction prediction method of claim 3, wherein, The method comprises: S221, selecting molecular access system fingerprints, Morgan fingerprints and topological fingerprints of drug molecular descriptors, and encoding and embedding each drug molecular descriptor to convert the drug molecular descriptors into corresponding feature matrices; S222, solving a proximity matrix corresponding to the feature matrix based on a multi-view low-rank sparse representation technology, and obtaining a drug similarity matrix based on the proximity matrix. S223, select the protein representation mode of amino acid composition, combination triplet descriptor, autocorrelation descriptor and pseudo-amino acid composition, convert the protein representation mode into a corresponding feature matrix, and obtain a protein similarity matrix in the manner of step S222; S224, normalize the initial drug feature matrix and the drug similarity matrix respectively to obtain a normalized drug feature matrix and a drug adjacency matrix, and construct a drug similarity network based on the normalized drug feature matrix and the drug adjacency matrix; S225, normalize the initial protein feature matrix and the protein similarity matrix respectively to obtain a normalized protein feature matrix and a protein adjacency matrix, and construct a protein similarity network based on the normalized protein feature matrix and the protein adjacency matrix. 5.The pre-trained language model based similar relation network drug target interaction prediction method of claim 1, wherein, The application of the graph neural network to the drug similarity network and the protein similarity network is used to extract drug similarity relationship features and protein similarity relationship features; extracting drug structure features by using a directed message passing neural network and extracting protein structure features by using a convolutional neural network includes: S31, based on the preset similarity relationship network feature embedding learning module, the graph convolutional neural network is applied to the drug similarity network and the protein similarity network respectively to obtain drug similarity relationship features and protein similarity relationship features; S32, converting the drug SMILES string into a molecular graph, using the directed message passing neural network to perform message passing update based on the directed bond in the molecular graph as a basic unit to obtain drug molecular graph features, and taking the drug molecular graph features as drug structure features; S33, encoding the protein amino acid sequence into a digital vector, mapping the digital vector to an embedding space through an embedding layer, obtaining protein sequence features in the embedding space through a one-dimensional convolutional layer, and taking the protein sequence features as protein structure features. 6.The pre-trained language model based similarity network drug target interaction prediction method of claim 5, wherein, The application of the graph neural network to the drug similarity network and the protein similarity network is used to extract drug similarity relationship features and protein similarity relationship features; S321, using a chemical information toolkit to convert the drug SMILES string into a molecular graph, wherein the node set in the molecular graph is the atom set in the drug molecule, and the edge set in the molecular graph is the chemical bond set between atoms; S322, using a directed message passing neural network to perform message passing update based on the directed bond in the molecular graph as a basic unit, and combining the node hidden state, the initial hidden state and the edge hidden state, the initial hidden state; S323, in the message passing update, the node message is obtained by aggregating the edge hidden state at the node time; S324, according to the node message, the atom hidden state is obtained, and all atom hidden states in the molecular graph are aggregated to obtain a drug molecule level representation. 7.The pre-trained language model based similar relation network drug target interaction prediction method of claim 1, wherein, The application of the graph neural network to the drug similarity network and the protein similarity network is used to extract drug similarity relationship features and protein similarity relationship features; obtaining drug-protein pair features includes: S41, cross-fuse the drug similarity relation feature and the protein similarity relation feature respectively based on a preset similarity relation feature fusion module to obtain drug similarity relation fusion features and protein similarity relation fusion features; S42, splice the drug similarity relation fusion features and the drug structure features to obtain drug combination features, and splice the protein similarity relation fusion features and the protein structure features to obtain protein combination features; S43, splice the drug combination features and the protein combination features to obtain drug-protein pair features. 8.The pre-trained language model based similarity network drug target interaction prediction method of claim 7, wherein, The similarity relation feature fusion module includes: S411, construct a similarity relation feature fusion module composed of a plurality of cross-fusion attention layers, and each cross-fusion attention layer is composed of a self-attention module, a drug-protein cross-attention module, and a protein-drug cross-attention module; S412, input the drug similarity relation feature and the protein similarity relation feature into the cross-fusion attention module, and obtain self-attention drug similarity relation features and self-attention protein similarity relation features through the self-attention module; S413, calculate a first query matrix based on the self-attention drug similarity relation features through the drug-protein cross-attention module, and calculate a first key matrix and a first value matrix based on the self-attention protein similarity relation features through the drug-protein cross-attention module; S414, calculate drug fusion features based on the first query matrix, the first key matrix, and the first value matrix. 9.The pre-trained language model based similarity network drug target interaction prediction method of claim 8, wherein, After the drug fusion features are calculated based on the first query matrix, the first key matrix, and the first value matrix, the method further includes: S415, calculate a second query matrix based on the self-attention protein similarity relation features through the protein-drug cross-attention module, and calculate a second key matrix and a second value matrix based on the self-attention drug similarity relation features through the protein-drug cross-attention module; S416, calculate protein fusion features based on the second query matrix, the second key matrix, and the second value matrix; S417, take the drug fusion features and the protein fusion features as inputs of a next cross-fusion attention layer, and perform a new round of feature fusion; splice and map the drug fusion features and the protein fusion features output by all cross-fusion attention layers in sequence to obtain drug similarity relation fusion features and protein similarity relation fusion features.

10. A pre-trained language model based similar relation network drug target interaction prediction system for implementing the pre-trained language model based similar relation network drug target interaction prediction method of any one of claims 1-9, characterized in that, The system includes: An initial embedding feature extraction module is configured to input drug SMILES strings and protein sequences into a pre-trained language model respectively, and extract initial drug embedding features and initial protein embedding features through the pre-trained language model; A similarity network construction model is configured to take the initial drug embedding features and the initial protein embedding features as initial feature matrices, and construct drug similarity networks and protein similarity networks based on a multi-view low-rank sparse representation technology. The similarity relationship and structure feature extraction module is configured to apply a graph neural network to a drug similarity network and a protein similarity network to extract drug similarity relationship features and protein similarity relationship features; extract drug structure features by using a directed message passing neural network, and extract protein structure features by using a convolutional neural network; The feature fusion module is configured to cross-fuse the drug similarity relationship features and the protein similarity relationship features based on a cross-fusion attention mechanism, splice the cross-fusion results with the drug structure features and the protein structure features respectively, and obtain drug-protein pair features. The prediction result generation module is configured to input the drug-protein pair features into a classifier to perform drug-target interaction prediction, and obtain a prediction result of the drug-target interaction.

Images