Drug target correlation prediction method and system based on multi-modal fusion

Abstract

The invention discloses a multi-modal fusion-based drug target correlation prediction method and system, and the scheme proposes a modal retention and interactive fusion strategy, and retains the specific information of a drug and a target in modals such as sequence semantics and a three-dimensional geometric structure through a bilinear model. And meanwhile, cross-modal contrast learning is introduced as a regularization constraint, so that the consistency of representation of different modals of the same entity is ensured, and the problems of information loss and insufficient interaction caused by forced unified representation in a traditional method are fundamentally solved.

Description

Technical Field

[0001] This invention belongs to the field of computer bioinformatics, and in particular relates to a method and system for predicting drug target associations based on multimodal fusion. Background Technology

[0002] Drug-target interaction prediction in the field of bioinformatics is crucial for accelerating drug discovery and reducing R&D costs. Traditional biological experimental methods are time-consuming and expensive, making computational methods and artificial intelligence tools important supplementary approaches.

[0003] Existing computational methods can be mainly divided into several categories: methods based on medicinal chemistry characteristics transform prediction into a classification problem, utilizing molecular fingerprints and protein sequence features, combined with support vector machines, random forests, and even deep learning models for prediction; structure-based methods rely on the three-dimensional information of the target and are limited when the structure is unknown; ligand-based methods have limited performance when known activity data is scarce.

[0004] In recent years, network methods that integrate multi-omics data to construct heterogeneous networks have gradually developed, enhancing predictive capabilities through graph embedding and topology learning. Furthermore, knowledge graph-based methods have developed rapidly, constructing multimodal biomedical knowledge graphs and transforming them into link prediction problems to infer potential interactions. However, existing methods still face significant challenges: knowledge graphs often suffer from incomplete information, and traditional link prediction models are susceptible to triplet structure bias. More importantly, in terms of multimodal fusion, existing methods typically forcibly project different modal information such as drug and target sequences and structures into a single shared space for fusion. While this can uncover commonalities between modalities, it inevitably loses the unique specific information of each modality and struggles to fully characterize the complex high-order interactions between modalities, thus limiting the accuracy and generalization ability of the prediction model. Summary of the Invention

[0005] The purpose of this invention is to propose a drug target association prediction method and system based on multimodal fusion to address the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: A drug target association prediction method based on multimodal fusion, the method comprising: S1. Obtain the SMILES sequence and three-dimensional structure information of drug molecules, the amino acid sequence and three-dimensional structure information of target proteins, and a knowledge graph constructed based on biomedical heterogeneous information; S2. Extract multimodal feature representations of drug and target protein respectively, including: Extracting sequence semantic features of drugs based on pre-trained molecular language models; Extracting the three-dimensional geometric structure features of drugs based on geometric graph neural networks; Extracting sequence semantic features of target proteins based on pre-trained protein language models; A three-dimensional spatial structural feature representation of target proteins is extracted based on a structural graph neural network. Based on the knowledge graph, a graph attention network is used to extract the knowledge graph structural features of drugs and target proteins. S3. The sequence semantic feature representation, structural feature representation, and knowledge graph structural feature representation are interactively fused across modalities using a bilinear fusion model. Contrastive learning regularization constraints are introduced to strengthen the correlation between modalities, and the model training is completed by combining the joint loss function. S4. Input the multimodal feature representation of the drug to be predicted and the target protein into the trained model, and output the prediction results of the drug-target interaction.

[0007] In the aforementioned drug target association prediction method based on multimodal fusion, the three-dimensional geometric structure feature representation is extracted in the following way: Obtain the three-dimensional coordinate information of drug molecules, and construct atom-bond diagrams based on this information. Bond-Angle Diagram The node set V corresponds to all atoms in the drug molecule, the edge set ℇ corresponds to the chemical bonds between atoms, and the edge set A corresponds to the bond angles formed between chemical bonds. The chemical characteristics of atoms and the type characteristics of chemical bonds are used as the node features and edge features of the atom-bond graph, respectively. The geometric characteristics of bond angles are used as the edge features of the bond-angle graph. The complete features of the atom-bond graph and the bond-angle graph are input into a pre-trained geometric graph neural network. The representations of atoms, bonds and bond angles are updated iteratively through message passing, and the three-dimensional geometric structure feature representation is obtained through pooling operation.

[0008] In the above-mentioned drug target association prediction method based on multimodal fusion, the three-dimensional spatial structure feature representation is extracted in the following way: Each amino acid residue is treated as a graph node, and three types of directed edges are constructed: Sequence edge: Connects residues in a linear sequence that are no more than 3 positions apart; Radius edge: connects residues in a three-dimensional structure where the distance between Cα atoms is less than 10 Å; K-nearest neighbor edges: connect the 10 nearest neighbor residues of each residue in space; The three types of edges are merged and input into the structural graph neural network. Neighbor information is aggregated through relation-specific learnable transformations and fused with the node's own information. After multiple iterations, residue-level features are obtained. The residue-level features are aggregated to generate a three-dimensional spatial structural feature representation of the target protein.

[0009] In the above-mentioned drug target association prediction method based on multimodal fusion, the knowledge graph structural feature representation is extracted in the following way: Graph attention networks perform weighted aggregation of the neighbor features of entities in a knowledge graph, capturing the local topological and semantic information of entities; Introducing the TransE energy function As a constraint, among which These are the embedding vectors of the head entity, relation, and tail entity, respectively. Training is performed using the Hinge loss function, which is: Where T is the set of positive triples and T' is the set of negative triples. This is a marginal parameter.

[0010] In the above-mentioned drug target association prediction method based on multimodal fusion, the molecular language model is a ChemBERTa model based on the Transformer architecture and pre-trained on a large-scale SMILES sequence. The protein language model is a ProtBERT model based on the Transformer architecture and pre-trained on a large number of amino acid sequences. The geometric graph neural network is the GeoGNN model; The structural graph neural network is the GearNet model; The graph attention network is a GAT model with TransE loss.

[0011] In the above-described drug target association prediction method based on multimodal fusion, in step S3, the bilinear fusion model achieves multimodal interaction based on Tucker decomposition using the following formula: in For entity embedding after multimodal fusion, It is the latent representation obtained through transformation. ; s corresponds to the knowledge graph mode, v corresponds to the structural mode of the two molecules, t corresponds to the sequence mode of the two molecules, and m corresponds to the result of the fusion of multiple modes; These are the transformation matrices for each mode. It is the core tensor.

[0012] In the above-mentioned drug target association prediction method based on multimodal fusion, step S3, the method of introducing contrastive learning regularization constraints, includes: Using representations of the same entity in different modalities as positive samples and representations of different entities in different modalities as negative samples, the difference between samples is calculated using a distance metric function, and modality alignment is optimized using the contrastive learning loss shown in formula (6): in L CLi Let M be the contrast loss for the i-th entity, and M be the modality set. Let N be the embedding of the i-th entity in the p and q modes, and N be the number of mini-batch samples.

[0013] In the above-described drug target association prediction method based on multimodal fusion, the joint loss function in step S3 is as follows: in, This represents the binary cross-entropy loss of the k-th mode. These are learnable weight parameters. s corresponds to the knowledge graph mode, v corresponds to the structural mode of the two molecules, t corresponds to the sequence mode of the two molecules, and m corresponds to the result of the fusion of multiple modes. .

[0014] In the above-mentioned drug target association prediction method based on multimodal fusion, in step S1, the biomedical heterogeneous information comes from at least one database among KEGG, DrugBank, InterPro, and UniProt, and the knowledge graph includes at least three node types: drug, protein, pathway, BRITE, and biological process, as well as at least two edge types among drug-protein, protein-protein, and drug-pathway.

[0015] A drug target association prediction system based on multimodal fusion, the system being used to implement the drug target association prediction method based on multimodal fusion, comprising: Data acquisition module: used to acquire SMILES sequences and three-dimensional structural information of drug molecules, amino acid sequences and three-dimensional structural information of target proteins, and knowledge graphs constructed based on biomedical heterogeneous information; Feature extraction module: includes molecular sequence feature extraction unit, drug structure feature extraction unit, protein sequence feature extraction unit, protein structure feature extraction unit, and knowledge graph feature extraction unit, which are used to extract the sequence semantic feature representation and three-dimensional geometric structure feature representation of drugs, the sequence semantic feature representation and three-dimensional spatial structure feature representation of target proteins, and the knowledge graph structure feature representation, respectively. Multimodal fusion module: Used to achieve cross-modal interactive fusion through a bilinear fusion model, and introduces contrastive learning regularization constraints, combined with a joint loss function to complete model training; Prediction output module: It receives the multimodal feature representation of the drug and target protein to be predicted, and outputs the prediction results of drug-target interaction through the trained model.

[0016] The advantages of this invention are as follows: This solution proposes a modality preservation and interactive fusion strategy. It preserves the specific information of drugs and targets in modalities such as sequence semantics and three-dimensional geometric structure through a bilinear model. Moreover, it explicitly models the complex high-order interaction relationships between these modalities. At the same time, it introduces cross-modal contrastive learning as a regularization constraint to ensure the consistency of the representation of the same entity in different modalities. Thus, it fundamentally solves the problem of information loss and insufficient interaction caused by the forced uniform representation in traditional methods. This approach integrates the embedding learning of knowledge graphs containing rich biomedical relationships with multimodal feature extraction within the same training loop. This allows the structured logic in the knowledge graph to dynamically guide and constrain the learning direction of multimodal representations. Conversely, the fine-grained information from multimodal representations can also enrich entity representations. This bidirectional collaborative approach effectively alleviates the data sparsity and bias problems inherent in knowledge graphs, enabling the model to possess stronger biological interpretability and the ability to reason from network relationships. Attached Figure Description

[0017] Figure 1 This is a flowchart of the drug target association prediction method based on multimodal fusion of the present invention; Figure 2 This is a flowchart of the three-dimensional geometric structure feature extraction process of drugs in the drug target association prediction method based on multimodal fusion of the present invention. Figure 3 This is a flowchart illustrating the extraction of three-dimensional spatial structural features of target proteins in the drug target association prediction method based on multimodal fusion of this invention. Figure 4 This is a block diagram of the drug target association prediction system based on multimodal fusion of the present invention. Detailed Implementation

[0018] To address the challenges of existing multimodal methods that forcibly project each modality into a shared space, resulting in the loss of modality-specific information and difficulty in characterizing high-order interactions, this proposal suggests a drug-target association prediction method based on multimodal fusion. This method organically integrates molecular language and three-dimensional geometric representation within a unified learning framework and co-optimizes with biomedical knowledge graphs to achieve interpretable predictions of complex mechanisms of action. Figure 1 As shown, the implementation method of this embodiment is as follows: 1) First, prepare the dataset. This example uses the Yamanishi_08 dataset. The Yamanishi_08 dataset is composed of four sub-datasets: enzymes, ion channels, G protein-coupled receptors, and nuclear receptors. Its data comes from multiple databases such as KEGG, DrugBank, InterPro, and UniProt, covering heterogeneous information including drug ATC codes, BRITE identifiers, associated diseases, and pathways. The knowledge graph constructed based on this dataset contains 25,487 nodes and 95,579 edges. The node types are diverse, including more than ten types such as drugs, proteins, pathways, BRITEs, and biological processes; the edge types involve various biological associations such as drug-protein, protein-protein, and drug-pathway.

[0019] 2) The above datasets are preprocessed, and heterogeneous information resources from multiple omics levels, including genomics, transcriptomics, and metabolomics, are systematically integrated. During the construction of the knowledge graph (KG), different categories of biomedical concepts, such as drugs, proteins, diseases, and biological pathways, are defined as node types. Simultaneously, various biological interactions or relationships, such as drug-protein interactions, drug-drug associations, and protein-pathway associations, are classified as edge types. Subsequently, the interaction or association information between each unique pair of entities is stored in the form of triples, thereby building the basic architecture of the knowledge graph, for example, "aspirin, drug-target interaction, cyclooxygenase 1 (COX1)".

[0020] The chemical semantic information of drug molecules plays a crucial role in prediction tasks. To effectively acquire the contextual dependencies and semantic features in SMILES molecular sequences, this approach introduces a BERT-based molecular language model, ChemBERTa, which has been pre-trained on over ten million SMILES molecular strings, enabling it to learn general chemical structural semantics. The specific process is as follows: For each drug, its corresponding SMILES expression is first obtained from the PubChem database. Then, a word segmenter constructed using the BytePairEncoding (BPE) algorithm is used to perform sub-word-level segmentation on the SMILES string, ultimately converting it into a token sequence with a maximum length of no more than 512. This token sequence is then input into the pre-trained ChemBERTa model to generate the corresponding contextual semantic embedding representation—the sequence semantic feature representation of the drug. .

[0021] To fully extract the three-dimensional structural information of drug molecules to aid in prediction tasks, this approach introduces the geometry-enhanced molecular representation learning framework (GEM) to extract the three-dimensional structural features of drugs. Specifically, such as... Figure 2As shown, the RDkit tool is first used to obtain the three-dimensional coordinate information of the drug molecule and construct its atomic-bond diagram. Bond-Angle Diagram ,in A collection of atoms It is a set of chemical bonds. This represents the set of bond angles. The chemical characteristics of atoms, the type of bonds, and the geometric characteristics of bond angles are used as inputs to the atom-bond diagram and the bond-angle diagram, respectively, and then fed into a pre-trained GeoGNN model. The GeoGNN model explicitly encodes the geometric relationships between atoms, bonds, and bond angles through multiple rounds of message passing, iteratively updating the atom representation vectors. The key represents a vector. The sum of the key angles represents the vector. Finally, by performing pooling operations on these representation vectors, a feature embedding capable of characterizing the three-dimensional spatial structure of drug molecules—a three-dimensional geometric structural feature representation of the drug—is generated. .

[0022] To capture long-range dependencies and core biological features in protein amino acid sequences, this approach introduces the ProtBERT protein language model based on the BERT architecture. This model is pre-trained on large-scale amino acid sequence databases such as UniRef and is capable of deep learning the general biological semantic rules of protein sequences. The specific process is as follows: First, the original amino acid sequence of the target protein is obtained and processed according to ProtBERT's labeling rules, transforming the sequence into a token sequence with a maximum length of no more than 4000 tags to suit the model's ability to handle long sequences. Then, this token sequence is input into the pre-trained ProtBERT model, which uses its self-attention mechanism to capture the positional relationships and feature dependencies of amino acids in the sequence, ultimately generating a contextual semantic embedding representation that characterizes the protein's biological function—a sequence semantic feature representation of the target protein. .

[0023] In the second stage, the GearNet structural graph neural network is used to fuse protein sequence and structural information to extract node / overall representations, such as... Figure 3 As shown, each amino acid residue is treated as a graph node, and three types of directed edges are added to characterize the relationships between residues: (1) Sequence edge: Connects residues on a linear sequence that are no more than 3 positions apart. The directions i→j and j→i are considered to be different types. (2) Radius edge: In a three-dimensional structure, any two residues... Atoms with a distance of less than 10 Å are considered connected. (3) K-nearest neighbor edge: Select the 10 nearest neighbors of each residue according to spatial distance and connect them.

[0024] After merging the three types of edges, the data is input into GearNet. At each layer, the GearNet model employs a learnable transformation matching the relation type to aggregate information from the corresponding neighbors, then fuses this information with the node's own information and updates the representation using ReLU activation. After multiple iterations, residue-level features incorporating sequence proximity, spatial contact, and local geometry are obtained, along with their converged protein-level features—a three-dimensional spatial structure representation of the target protein. .

[0025] 3) Based on the knowledge graph constructed in step 2), a graph attention network (GAT) with TransE loss is further employed to extract knowledge graph features. Specifically, the GAT encoder uses an attention mechanism to weighted aggregate the features of an entity's neighbors to capture the entity's local topological and semantic information. Simultaneously, an energy function of TransE is introduced to constrain the semantic consistency of the embedding space; this energy function is defined as... ,in , and These are the embedding vectors of the head entity, relation, and tail entity, respectively. During training, Hinge loss is used to distinguish between positive and negative triples, and its formula is: h,r,t These represent the head entity, relation, and tail entity in a positive triple of a knowledge graph, respectively. T Denotes the set of all positive sample triples; h ’ ,r,t ’ These represent the head entity, relation, and tail entity in a negative triple of a knowledge graph, respectively. T ’ Represents the set of all negative sample triples; It is a marginal parameter.

[0026] 4) Contrastive learning and multimodal fusion: The core objective of this stage is to effectively acquire multimodal representations, thereby fully capturing the complex interactions between different modalities. Many existing multimodal fusion methods have achieved promising results in tasks such as visual question answering (VQA). However, these methods use modal projection or cross-modal attention to mine the commonalities between modalities. Their limitation is that they cause the loss of unique information in different modalities and make it difficult to achieve full interaction between modalities. To address this problem, this scheme proposes a bilinear model with full parameter interaction capability as the core framework for multimodal information fusion. Specifically, the Tucker decomposition is extended to four modal factor forms, as shown in formula (2): s corresponds to the knowledge graph mode, v corresponds to the structural mode of the two molecules, t corresponds to the sequence mode of the two molecules, and m corresponds to the result of the fusion of multiple modes; in , , and Here are the transformation matrices for each mode. It is the core tensor.

[0027] Within this framework, entity embeddings are first projected into a low-dimensional space and then fused with the core tensor. Furthermore, the core tensor is decomposed, and the representations of all modalities are integrated into a unified space through element-wise multiplication. The specific calculation process is shown in formula (3): in This represents the entity embedding vector after multimodal fusion. It is the latent representation obtained through transformation. For the original embedding representation, It is for each modality. The decomposition transformation matrix.

[0028] Multimodal interaction is achieved through the bilinear fusion method described above. Furthermore, this embodiment introduces constraints to enhance the correlation between different modal representations of the same entity, giving the model clear boundary constraints and allowing the gradient of the final prediction result to more directly guide parameter learning. Specifically, the contrastive learning between different entities and modalities is used as an additional regularization learning objective. In the contrastive learning setting, "representation pairs of the same entity in different modalities" are treated as positive samples, and "representation pairs of different entities" are treated as negative samples. The goal is to make the distance between negative samples much greater than the distance between positive samples, thereby strengthening the multimodal fusion effect. in, For distance measurement, Let x be the embedding function, representing the anchor sample. x + Indicates a positive sample. x − This indicates a negative sample.

[0029] In practice, a mini-batch is formed by randomly sampling N entities from the entity set, and a contrastive learning loss is constructed based on this mini-batch. Positive sample pairs are naturally constructed from different modal representations of the same entity, while negative sample pairs are constructed through negative sharing among all other entities. Cosine similarity is used as the distance metric, calculated as follows: u and v represent two embedding vectors whose distance is to be calculated.

[0030] Finally, for each entity i, the contrastive loss function can be expressed as: L CLi Let N represent the contrastive learning loss for the i-th entity; N represents the number of mini-batch samples for contrastive learning; M represents the modality set; p and q represent modality indices, and j represents the negative sample index.

[0031] e i p This represents the embedding representation of the i-th entity in the p-th modality; e i q This represents the embedding representation of the i-th entity in the q-th modality; e j q This represents the embedding representation of the j-th entity in the q-th modality.

[0032] 5) Existing multimodal methods primarily project representations from different modalities into the same space and perform unified predictions based on commonalities among modalities. However, this often weakens the unique information of each modality. This approach alleviates this problem by performing joint learning during the decision-making stage and fusing the predictions from various modalities, thereby further leveraging the complementarity between modalities. In the multimodal setting, a different context modeler is selected for each modality, and its respective output is used for training. Combined with contrastive learning loss, the overall training objective is to minimize the following joint loss: in, This represents the binary cross-entropy loss of the k-th mode. These are learnable weight parameters. , During the inference phase, the predictions of both single-modal and multi-modal branches are considered simultaneously, and their average value is used as the final prediction result.

[0033] Thus, this solution integrates datasets, extracts multimodal features, mines structural features from knowledge graphs, employs a multimodal fusion strategy based on modality preservation and interactive fusion, and incorporates contrastive learning regularization constraints. Ultimately, it optimizes the overall model using a joint loss function, constructing a drug-target association prediction framework that combines comprehensive features with deep interaction. This framework preserves modality-specific information from drug molecule semantics / 3D structure, protein sequence / spatial features, and biomedical knowledge graphs. Furthermore, it characterizes complex cross-modal interactions through bilinear fusion and contrastive alignment, effectively addressing the problems of representation degradation and insufficient modeling of high-order interactions inherent in traditional multimodal methods.

[0034] Furthermore, this embodiment also provides a drug target association prediction system based on multimodal fusion to implement the above method, such as... Figure 4 As shown, it includes: Data acquisition module 1: used to acquire the SMILES sequence and three-dimensional structure information of drug molecules, the amino acid sequence and three-dimensional structure information of target proteins, and a knowledge graph constructed based on biomedical heterogeneous information; Feature extraction module 2 includes a molecular sequence feature extraction unit, a drug structure feature extraction unit, a protein sequence feature extraction unit, a protein structure feature extraction unit, and a knowledge graph feature extraction unit, which are used to extract the sequence semantic feature representation and three-dimensional geometric structure feature representation of drugs, the sequence semantic feature representation and three-dimensional spatial structure feature representation of target proteins, and the knowledge graph structure feature representation, respectively. Multimodal fusion module 3: It is used to achieve cross-modal interactive fusion through a bilinear fusion model, and introduces contrastive learning regularization constraints, combined with the joint loss function to complete model training; Prediction output module 4: It is used to receive the multimodal feature representation of the drug to be predicted and the target protein, and output the prediction results of the drug-target interaction through the trained model.

[0035] To verify the potential of this invention in the drug target association prediction task, this embodiment uses an independent test set to evaluate the performance of the trained model. The evaluation metrics include: Recall measures a model's ability to correctly identify true positive samples. Values ​​range from 0 to 1; a value closer to 1 indicates better prediction performance. The calculation expression is: .

[0036] Precision: Measures the proportion of samples where the model predicts a positive result. It is usually used in conjunction with the Recall metric. The value ranges from 0 to 1; a value closer to 1 indicates better prediction performance. The calculation expression is: .

[0037] Accuracy, Acc: Measures the proportion of samples correctly classified by the model. The value ranges from 0 to 1; a value closer to 1 indicates better prediction performance. The calculation expression is: .

[0038] F1 score: A balanced metric that considers both precision and recall. It is the harmonic mean of precision and recall, ranging from 0 to 1. A value closer to 1 indicates better predictive performance. The calculation formula is: .

[0039] The area under the ROC curve (AU-ROC) measures the overall discriminative power of a model across all possible classification thresholds, reflecting the separability of positive and negative samples. The closer the AU-ROC value is to 1, the stronger the model's ability to distinguish between positive and negative samples.

[0040] The Area Under the Recall Curve (AU-PRC) is particularly suitable for situations with a high imbalance between positive and negative classes, and can more accurately evaluate the model's ability to identify positive samples. Compared to AU-ROC, AU-PRC is generally more discriminative on imbalanced datasets; a higher value indicates a stronger ability of the model to maintain high precision while improving recall.

[0041] Where TP is the number of logs correctly predicted as interacting, TN is the number of logs correctly predicted as not interacting, FP is the number of logs incorrectly predicted as interacting, and FN is the number of logs incorrectly predicted as not interacting.

[0042] In addition, this embodiment also performed ablation analysis on the three modalities in the model, extracting a single modality for target prediction and performance evaluation. Specifically, the full model corresponds to the full-modal model evaluation of this scheme; the knowledge graph model retains only the knowledge graph modality; the structural model retains only the structural modality of drugs and proteins; and the sequence model retains only the sequence modality of drugs and proteins. Table 1 below shows the performance evaluation results of the model, illustrating the model performance under each modality combination.

[0043] Table 1 Model Performance Evaluation and Ablation Analysis The table above shows the performance evaluation results of the proposed model on the database of this embodiment, demonstrating that the model is feasible and reproducible. The bolded data in the table represents the best-performing results among several models, while the underlined data represents the second-best-performing results. Although the improvement in individual metrics is subtle, the complete model achieves a comprehensive leading advantage across all key metrics on the challenging prediction task and the challenging dataset of this embodiment, proving the success of the multimodal fusion strategy. Furthermore, the performance reaches a level close to 0.8, indicating that the overall model performance has reached an advanced level in this task domain.

[0044] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A drug target association prediction method based on multimodal fusion, characterized in that, The method includes: S1. Obtain the SMILES sequence and three-dimensional structure information of drug molecules, the amino acid sequence and three-dimensional structure information of target proteins, and a knowledge graph constructed based on biomedical heterogeneous information; S2. Extract multimodal feature representations of drug and target protein respectively, including: Extracting sequence semantic features of drugs based on pre-trained molecular language models; Extracting the three-dimensional geometric structure features of drugs based on geometric graph neural networks; Extracting sequence semantic features of target proteins based on pre-trained protein language models; A three-dimensional spatial structural feature representation of target proteins is extracted based on a structural graph neural network. Based on the knowledge graph, a graph attention network is used to extract the knowledge graph structural features of drugs and target proteins. S3. The sequence semantic feature representation, structural feature representation, and knowledge graph structural feature representation are interactively fused across modalities using a bilinear fusion model. Contrastive learning regularization constraints are introduced to strengthen the correlation between modalities, and the model training is completed by combining the joint loss function. S4. Input the multimodal feature representation of the drug to be predicted and the target protein into the trained model, and output the prediction results of the drug-target interaction.

2. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, The three-dimensional geometric structure features are extracted in the following way: Obtain the three-dimensional coordinate information of drug molecules, and construct atom-bond diagrams based on this information. Bond-Angle Diagram The node set V corresponds to all atoms in the drug molecule, the edge set ℇ corresponds to the chemical bonds between atoms, and the edge set A corresponds to the bond angles formed between chemical bonds. The chemical characteristics of atoms and the type characteristics of chemical bonds are used as the node features and edge features of the atom-bond graph, respectively. The geometric characteristics of bond angles are used as the edge features of the bond-angle graph. The complete features of the atom-bond graph and the bond-angle graph are input into a pre-trained geometric graph neural network. The representations of atoms, bonds and bond angles are updated iteratively through message passing, and the three-dimensional geometric structure feature representation is obtained through pooling operation.

3. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, The three-dimensional spatial structural features are extracted in the following way: Each amino acid residue is treated as a graph node, and three types of directed edges are constructed: Sequence edge: Connects residues in a linear sequence that are no more than 3 positions apart; Radius edge: connects residues in a three-dimensional structure where the distance between Cα atoms is less than 10 Å; K-nearest neighbor edges: connect the 10 nearest neighbor residues of each residue in space; The three types of edges are merged and input into the structural graph neural network. Neighbor information is aggregated through relation-specific learnable transformations and fused with the node's own information. After multiple iterations, residue-level features are obtained. The residue-level features are aggregated to generate a three-dimensional spatial structural feature representation of the target protein.

4. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, The structural features of the knowledge graph are extracted in the following manner: Graph attention networks perform weighted aggregation of the neighbor features of entities in a knowledge graph, capturing the local topological and semantic information of entities; Introducing the TransE energy function As a constraint, where , , These are the embedding vectors of the head entity, relation, and tail entity, respectively. Training is performed using the Hinge loss function, which is: Where T is the set of positive triples and T' is the set of negative triples. This is a marginal parameter.

5. The drug target association prediction method based on multimodal fusion according to any one of claims 1 to 4, characterized in that, The molecular language model is a ChemBERTa model based on the Transformer architecture and pre-trained on a large number of SMILES sequences. The protein language model is a ProtBERT model based on the Transformer architecture and pre-trained on a large number of amino acid sequences. The geometric graph neural network is the GeoGNN model; The structural graph neural network is the GearNet model; The graph attention network is a GAT model with TransE loss.

6. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, In step S3, the bilinear fusion model achieves multimodal interaction based on Tucker decomposition using the following formula: in For entity embedding after multimodal fusion, It is the latent representation obtained through transformation. ; s corresponds to the knowledge graph mode, v corresponds to the structural mode of the two molecules, t corresponds to the sequence mode of the two molecules, and m corresponds to the result of the fusion of multiple modes; These are the transformation matrices for each mode. It is the core tensor.

7. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, In step S3, the method for introducing contrastive learning regularization constraints includes: Using representations of the same entity in different modalities as positive samples and representations of different entities in different modalities as negative samples, the difference between samples is calculated using a distance metric function, and modality alignment is optimized using the contrastive learning loss shown in formula (6): in L CLi Let M be the contrast loss for the i-th entity, and M be the modality set. , Let N be the embedding of the i-th entity in the p and q modes, and N be the number of mini-batch samples.

8. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, In step S3, the joint loss function is as follows: in, This represents the binary cross-entropy loss of the k-th mode. These are learnable weight parameters. s corresponds to the knowledge graph mode, v corresponds to the structural mode of the two molecules, t corresponds to the sequence mode of the two molecules, and m corresponds to the result of the fusion of multiple modes. .

9. The drug target association prediction method based on multimodal fusion according to claim 1, characterized in that, In step S1, the biomedical heterogeneous information comes from at least one database among KEGG, DrugBank, InterPro, and UniProt. The knowledge graph includes at least three node types: drug, protein, pathway, BRITE, and biological process, as well as at least two edge types among drug-protein, protein-protein, and drug-pathway.

10. A drug target association prediction system based on multimodal fusion, characterized in that, The system is used to implement the method according to any one of claims 1 to 9, comprising: Data acquisition module: used to acquire SMILES sequences and three-dimensional structural information of drug molecules, amino acid sequences and three-dimensional structural information of target proteins, and knowledge graphs constructed based on biomedical heterogeneous information; Feature extraction module: includes molecular sequence feature extraction unit, drug structure feature extraction unit, protein sequence feature extraction unit, protein structure feature extraction unit, and knowledge graph feature extraction unit, which are used to extract the sequence semantic feature representation and three-dimensional geometric structure feature representation of drugs, the sequence semantic feature representation and three-dimensional spatial structure feature representation of target proteins, and the knowledge graph structure feature representation, respectively. Multimodal fusion module: Used to achieve cross-modal interactive fusion through a bilinear fusion model, and introduces contrastive learning regularization constraints, combined with a joint loss function to complete model training; Prediction output module: It is used to receive the multimodal feature representation of the drug to be predicted and the target protein, and output the prediction results of drug-target interaction through the trained model.

Images