Drug-target interaction prediction method and system based on multi-modal self-supervised learning

Abstract

The present application relates to the technical field of computational biology and artificial intelligence assisted drug discovery, and discloses a drug-target interaction prediction method and system based on multi-modal self-supervised learning, wherein the method comprises the steps of drug-side self-supervised pre-training, protein-side self-supervised pre-training, multi-feature fusion, drug-protein interaction modeling and multi-task joint prediction. The present application predicts drug-target interaction based on self-supervised pre-training and multi-modal information fusion. First, a self-supervised learning method is used to train a deep learning model using unlabeled data. Then, the deep learning model is used to perform multi-modal representation, multi-modal fusion and interaction modeling of drug molecules and protein targets. The present application can realize joint prediction of drug-target interaction, affinity and mechanism of action, effectively improving the prediction accuracy and robustness of the model.

Description

Technical Field

[0001] This invention relates to the fields of computational biology and artificial intelligence-assisted drug discovery, and in particular to a drug-target interaction prediction method and system based on multimodal self-supervised learning. Background Technology

[0002] With the development of computational chemistry and deep learning, using machine learning models to predict the interaction relationships, affinity, and mechanisms of action between drug molecules and protein targets has become an important direction in the field of drug discovery. Regarding drug-protein modeling, existing technologies have developed a series of representative methods in the following four main areas, but significant shortcomings still exist.

[0003] 1. Existing techniques and their limitations in drug and protein characterization and fusion.

[0004] In drug characterization, common practices in existing technologies include:

[0005] (1) Based on the representation of SMILES sequences, the drug's SMILES string is input into a model such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a Transformer to obtain a one-dimensional sequence embedding vector;

[0006] (2) Based on the representation of molecular fingerprints, high-dimensional sparse vectors constructed using rules such as Morgan fingerprints are directly used as input features of downstream models;

[0007] (3) Based on the molecular graph representation, the drug is represented as a graph structure (atoms as nodes and chemical bonds as edges), and encoded using graph convolutional networks (GCN), graph attention networks (GAT), etc., to obtain graph-level embedding.

[0008] In protein characterization, existing techniques mostly employ:

[0009] (1) Extract sequence features using CNN, BiLSTM or Transformer based solely on one-dimensional amino acid sequences;

[0010] (2) Enhance the sequence by introducing positional features and evolutionary features (such as PSSM);

[0011] Regarding the fusion of drug and protein characteristics, most existing methods employ the following approaches:

[0012] (1) Simple concatenation, that is, after obtaining the drug vector and protein vector respectively, they are directly concatenated on the feature dimension, and then input into the multilayer perceptron for classification or regression;

[0013] (2) Simple weighted or element-wise operations (e.g., element-wise addition, dot product) are used as interactive features;

[0014] (3) A few studies introduce unidirectional attention, that is, using drugs as queries and proteins as keys, or vice versa, weighting and summing the other sequence before splicing.

[0015] However, the aforementioned existing technologies have the following shortcomings in drug and protein characterization and fusion:

[0016] (1) Drug side usually only selects one or a few of SMILES, molecular fingerprints, and molecular graphs for modeling. There is a lack of systematic alignment and fusion of multi-perspective information of "SMILES semantics + substructure fingerprints + graph topology" in a unified framework, and the complementarity between different perspectives is not fully utilized.

[0017] (2) Proteins are often represented globally using the whole sequence, without specifically modeling the pocket regions that are most closely related to ligand binding. This results in the local structural information that is highly relevant to drug binding being diluted in the long sequence.

[0018] (3) In terms of the fusion method of different modal data, simple splicing or element-wise operation does not take into account the significant differences between drug features and protein features in terms of scale, distribution and information density, which can easily lead to one side of the features dominating the training while the other side of the features contributes only to a limited extent.

[0019] 2. Existing techniques and their limitations in label-free data and protein 3D structure modeling.

[0020] In terms of training paradigms, most existing drug-target interaction prediction methods employ a purely supervised learning framework. The typical approach is as follows: given a sample set labeled with interaction tags (binding status), affinity values, or mechanism of action categories, the drug and protein representations are input into a neural network, and the model parameters are directly trained using supervised loss functions (such as cross-entropy, mean squared error, etc.). These methods rely primarily on a limited amount of labeled data, ignoring the vast amount of unlabeled drug molecule and protein sequence / structure data in publicly available databases. This unlabeled data is usually only used for pre-training general language models or as data augmentation background, and is rarely specifically incorporated into drug-target modeling frameworks. Therefore, when the amount of labeled data is limited or the data distribution changes, these models, which rely entirely on supervised signals, are prone to overfitting and insufficient generalization ability.

[0021] To alleviate the problem of scarce labeled data, some works in recent years have attempted to introduce self-supervised learning on the drug or protein side. However, most existing self-supervised methods are limited to relatively simple pre-training tasks, such as: (1) randomly masking some characters or residues at the sequence level and requiring the model to recover the masked tokens (similar to MLM); (2) randomly hiding some nodes or edges at the molecular graph level and requiring the model to reconstruct the hidden substructures or adjacency relationships; (3) performing comparative learning on nodes / fragments and designing positive and negative sample pairs only at the local substructure level.

[0022] While these self-supervised tasks can improve the representation quality of single-modal features to some extent, they have two prominent problems:

[0023] (1) First, it mainly focuses on local structural reconstruction of a single modality (such as drug SMILES sequences or molecular graphs), making it difficult to align the semantic and topological relationships between different modalities or different perspectives (such as SMILES, fingerprints, molecular graphs) under a unified framework;

[0024] (2) Then, most of the work still only focuses on relatively simple pre-training objectives such as "local masking / restoration", and does not fully explore higher-level structural priors such as multi-view consistency and cross-modal mutual information, and fails to make full use of the statistical regularities contained in large-scale unlabeled data.

[0025] Furthermore, in utilizing protein 3D structures, existing technologies primarily introduce 3D information as auxiliary features rather than designing a self-supervised modeling framework around 3D geometric priors. One typical approach is to obtain the protein's 3D structure from AlphaFold or PDB, extract simple geometric features (such as inter-residue distance matrices and contact maps), and then concatenate them with sequence features before feeding them into downstream networks. Another approach involves constructing 3D meshes or graph structures for proteins or protein-ligand complexes, and then directly training them on supervised tasks using 3D-CNNs or 3D graph neural networks.

[0026] However, these existing methods have significant shortcomings in the following aspects:

[0027] (1) Insufficient explicit extraction and modeling of pocket information. Many methods still model from the perspective of full-length protein or full-atom / full-residue, without specifically extracting and modeling the "pocket region" that is most tightly bound to the ligand. As a result, the local geometric information that is highly related to drug binding is overwhelmed by a large amount of global background information, and the model is difficult to focus on the truly critical local three-dimensional structure.

[0028] (2) Lack of systematic consideration of SE(3) equivariance / invariance. Many 3D modeling methods directly process coordinates or voxels in Euclidean space without explicitly guaranteeing equivariance or invariance to rotation and translation in 3D space in the network structure. This means that the same protein conformation may have significantly different feature representations under different coordinate systems or different spatial orientations. The model's expression of 3D geometry is redundant and unstable, which is not conducive to generalization and transfer.

[0029] (3) Insufficient design of 3D self-supervised tasks. Most existing works only use 3D information in the supervision task and lack specialized self-supervised tasks based on 3D coordinates and geometric relationships (such as adjacent distance, angle, local conformation, etc.), such as coordinate denoising, local geometry restoration, 3D distance prediction, etc. As a result, it is difficult to fully mine and encode the higher-order priors in the 3D structure of proteins (such as local rigid body structure, spatial arrangement pattern) through large-scale unlabeled data.

[0030] In summary, in terms of both unlabeled data utilization and protein 3D structure modeling, existing technologies are still mainly based on small-scale supervised training and simple mask-based self-supervision. They do not make full use of large-scale unlabeled multimodal data, nor do they build a systematic 3D self-supervised modeling framework around protein pocket regions and variable geometric properties such as SE(3), thus limiting the model's generalization ability under new targets, new chemical spaces, and complex 3D conformations.

[0031] 3. Existing techniques and limitations in drug-protein interaction modeling.

[0032] At the interaction modeling level, existing methods can be broadly categorized as follows:

[0033] (1) Early fusion method: Drug and protein are encoded as global vectors and then directly concatenated (or simple calculations are performed), and then input into MLP for prediction. This type of method does not explicitly model "which part of the drug interacts with which part of the protein", and the interaction information mainly relies on the implicit learning of subsequent fully connected layers.

[0034] (2) Interaction methods based on unidirectional attention: Some works have introduced unidirectional attention mechanisms, such as using drug vectors as queries and protein sequence features as keys, and weighted summation of protein residues to obtain "drug-oriented protein representation", or vice versa. These methods have enhanced the interaction to some extent, but still have the problems of attention being only unidirectional and having a small number of layers.

[0035] (3) Matching methods based on scoring functions: After encoding the drug and protein into vectors respectively, the matching scores of the two are calculated using methods such as dot product similarity, cosine similarity, and bilinear scoring, which serve as the basis for predicting the interaction strength or affinity. At the sequence level, some methods calculate the pairwise scoring matrix between drug tokens and protein residues, and then summarize them into a global score through simple pooling or attention.

[0036] The existing interaction modeling methods described above have the following shortcomings:

[0037] Schemes based on simple concatenation simply cascade the global vectors of drugs and proteins directly and then process them through a fully connected network. They neither explicitly model fine-grained correspondences such as "substructure-residue" nor provide a unified alignment of the feature scale and distribution on both sides. The interaction information relies entirely on the implicit capture of downstream networks. Schemes based on unidirectional attention can allow one party (such as a drug) to perform weighted aggregation on another party (such as a protein sequence), but the interaction is only modeled in one direction. They cannot characterize the true bidirectional dependence and high-order coupling between drugs and proteins, and usually only a few attention layers are stacked, resulting in limited expressive power. Matching schemes based on simple cross-attention or scoring functions mostly perform one-time or coarse-grained pairwise scoring (such as global vector similarity or shallow cross-attention). They lack parameter sharing and symmetrical design, making it difficult to characterize complex interaction patterns at multiple levels in a unified and stable structure. They are also not conducive to expansion and reuse in multi-task and multi-modal scenarios.

[0038] 4. Current status and shortcomings of drug-target effect prediction.

[0039] In the prior art, most works design models for a single task, such as: (1) binary classification modeling only for DTI; (2) affinity regression only for DTA; (3) a few works share some encoders between DTI and DTA, but still mainly use task-specific network structures.

[0040] In a few multi-tasking tasks, a common practice is to build a shared encoder and then append multiple independent task headers (multi-header output) to it. This type of simple multi-tasking framework has the following problems:

[0041] (1) Insufficient utilization of the correlation between tasks. There is a natural correlation between DTI, DTA and MoA, but a simple shared encoder + multi-head output cannot explicitly utilize this structural relationship, and may even lead to gradient conflicts and negative transfer phenomena between tasks;

[0042] (2) Few works combine multi-task learning with automated machine learning (AutoML) to systematically search and integrate model families and hyperparameters. They usually use fixed structures or a small number of experiments, which cannot fully explore the model space. Summary of the Invention

[0043] To address the aforementioned issues, this invention proposes a drug-target interaction prediction method and system based on multimodal self-supervised learning. First, a deep learning model is trained using unlabeled data through self-supervised learning. Then, the deep learning model is used to perform multimodal characterization, multimodal fusion, and interaction modeling of drug molecules and protein targets, thereby achieving joint prediction of drug-target interactions, affinity, and mechanisms of action.

[0044] The technical solution adopted in this invention is as follows:

[0045] A drug-target interaction prediction method based on multimodal self-supervised learning includes:

[0046] Drug-side self-supervised pre-training: Construct representations of the same drug molecule from multiple perspectives, align the representations using comparative learning loss from multiple perspectives on large-scale unlabeled drug data, and train to obtain a drug pre-trained model;

[0047] Protein-side self-supervised pre-training: The predicted three-dimensional structure is obtained based on the amino acid sequence. Based on the three-dimensional structure, the protein binding pocket region is identified and the set of pocket residues and their three-dimensional coordinates are extracted to construct a graph representation of the pocket. The protein pre-training model is obtained by pre-training a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks.

[0048] Multi-feature fusion: Input the multi-perspective representations of the drug to be predicted into the drug pre-training model to obtain the drug multimodal representation, and fuse them to obtain the unified drug representation vector; input the pocket diagram representation of the protein to be predicted into the protein pre-training model to obtain the protein pocket representation vector; input the amino acid sequence into the existing protein language model to obtain the protein sequence representation vector, and fuse them to obtain the unified protein representation vector.

[0049] Drug-protein interaction modeling: Through a bidirectional cross-attention mechanism, the cross-attention between the drug and the protein is calculated in two directions respectively. Combined with residual connections and feedforward networks, the drug and protein representations with enhanced interaction are obtained.

[0050] Multi-task joint prediction: The interaction-enhanced drug representation and protein representation are aggregated to obtain an interaction feature vector. Multi-task prediction is performed based on the interaction feature vector, and at least two prediction results are output.

[0051] Furthermore, the multiple perspectives for characterizing the same drug molecule include: encoding representation based on simplified linear input canonical sequences of the molecule, molecular fingerprint vector representation based on rule generation, and graph neural network encoding representation based on molecular graph structure.

[0052] Further, identifying protein-binding pocket regions based on the three-dimensional structure and extracting the set of pocket residues and their three-dimensional coordinates includes: identifying a set of candidate pockets based on the predicted three-dimensional structure, filtering the candidate pockets according to their confidence level, and extracting the set of residues and their three-dimensional coordinates of the filtered pockets.

[0053] Furthermore, the self-supervised tasks include residue or atom feature mask recovery tasks, neighboring residue pair 3D distance regression tasks, and 3D coordinate denoising and recovery tasks.

[0054] Furthermore, the calculation method of the unified drug representation vector or the unified protein representation vector includes: constructing a gated discriminant input matrix for the multimodal drug representation or the multimodal protein representation based on the gated weight fusion method, calculating the position-wise and dimension-wise gated weights through a multilayer perceptron and a sigmoid function, and performing feature fusion on the corresponding representations according to the gated weights; the multimodal protein representation includes a protein pocket representation vector and a protein sequence representation vector.

[0055] Furthermore, the method of calculating cross-attention in both drug and protein directions through a bidirectional cross-attention mechanism includes: achieving structural symmetry modeling by sharing a weight matrix through a bidirectional cross-attention layer; obtaining interactive features from the corresponding perspective by using the unified representation vector of the drug as the query and the unified representation vector of the protein as the key and value; and obtaining interactive features from the corresponding perspective by using the unified representation vector of the protein as the key and value.

[0056] Furthermore, the prediction results include binary classification results of drug-target interactions, numerical regression results of drug-target affinity, and multi-classification results of drug-target action mechanisms, including inhibitory or activating effects.

[0057] A drug-target interaction prediction system, comprising:

[0058] The drug-side self-supervised pre-training module is configured to construct representations of the same drug molecule from multiple perspectives. The representations are then aligned using a multi-perspective comparative learning loss on large-scale unlabeled drug data to train a drug pre-training model.

[0059] The protein-side self-supervised pre-training module is configured to obtain the predicted three-dimensional structure based on the amino acid sequence, identify the protein binding pocket region based on the three-dimensional structure, extract the set of pocket residues and their three-dimensional coordinates, and construct a graph representation of the pocket; and pre-train the protein pre-training model through a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks.

[0060] The multi-feature fusion module is configured to input multiple perspective representations of the drug to be predicted into the drug pre-training model to obtain a multimodal representation of the drug, and fuse them to obtain a unified representation vector of the drug; input the pocket diagram representation of the protein to be predicted into the protein pre-training model to obtain a protein pocket representation vector; input the amino acid sequence into an existing protein language model to obtain a protein sequence representation vector, and fuse them to obtain a unified representation vector of the protein.

[0061] The drug-protein interaction modeling module is configured to calculate the cross-attention between the drug and the protein in two directions through a bidirectional cross-attention mechanism, and obtain the drug and protein representations with enhanced interaction by combining residual connections and feedforward networks.

[0062] The multi-task joint prediction module is configured to aggregate the interaction-enhanced drug representation and protein representation to obtain an interaction feature vector, perform multi-task prediction based on the interaction feature vector, and output at least two prediction results.

[0063] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the drug-target interaction prediction method based on multimodal self-supervised learning.

[0064] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the drug-target interaction prediction method based on multimodal self-supervised learning.

[0065] The beneficial effects of this invention are as follows:

[0066] This invention predicts drug-target interactions based on self-supervised pre-training and multimodal information fusion. First, a deep learning model is trained using unlabeled data using a self-supervised learning method. Then, the deep learning model is used to perform multimodal characterization, multimodal fusion, and interaction modeling of drug molecules and protein targets, thereby achieving joint prediction of drug-target interactions, affinity, and mechanisms of action. Attached Figure Description

[0067] Figure 1 This is a flowchart of the drug-target interaction prediction method based on multimodal self-supervised learning in Embodiment 1 of the present invention.

[0068] Figure 2 This is a flowchart of the drug multi-view self-supervised pre-training process in Embodiment 2 of the present invention.

[0069] Figure 3 This is a flowchart of the protein pocket three-task joint self-supervised pre-training process in Embodiment 2 of the present invention.

[0070] Figure 4This is a flowchart of bidirectional cross-attention interaction modeling based on shared weights in Embodiment 2 of the present invention. Detailed Implementation

[0071] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments are now described. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0072] Example 1

[0073] like Figure 1 As shown, this embodiment provides a drug-target interaction prediction method based on multimodal self-supervised learning, including:

[0074] Drug-side self-supervised pre-training: Construct representations of the same drug molecule from multiple perspectives, align the representations using comparative learning loss from multiple perspectives on large-scale unlabeled drug data, and train to obtain a drug pre-trained model;

[0075] Protein-side self-supervised pre-training: The predicted three-dimensional structure is obtained based on the amino acid sequence. The protein binding pocket region is identified based on the three-dimensional structure, and the set of pocket residues and their three-dimensional coordinates are extracted to construct a graph representation of the pocket. The protein pre-training model is obtained by pre-training a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks.

[0076] Multi-feature fusion: Input the multi-perspective representations of the drug to be predicted into the drug pre-training model to obtain the drug multimodal representation, and fuse them to obtain the unified drug representation vector; input the pocket diagram representation of the protein to be predicted into the protein pre-training model to obtain the protein pocket representation vector; input the amino acid sequence into the existing protein language model to obtain the protein sequence representation vector, and fuse them to obtain the unified protein representation vector.

[0077] Drug-protein interaction modeling: Through a bidirectional cross-attention mechanism, the cross-attention between the drug and the protein is calculated in two directions respectively. Combined with residual connections and feedforward networks, the drug and protein representations with enhanced interaction are obtained.

[0078] Multi-task joint prediction: Aggregate the drug representation and protein representation with enhanced interaction to obtain the interaction feature vector, perform multi-task prediction based on the interaction feature vector, and output at least two prediction results.

[0079] It should be noted that this method constructs a complete drug-target interaction prediction process through a step-by-step approach with each step closely connected. The steps work together, from data preprocessing to model training and prediction output, progressing layer by layer, effectively ensuring the comprehensiveness and systematic nature of the prediction, and laying a solid foundation for subsequent accurate prediction.

[0080] Preferably, the characterization of the same drug molecule from multiple perspectives includes: encoding characterization based on the simplified linear input canonical sequence of the molecule, molecular fingerprint vector characterization based on rule generation, and graph neural network encoding characterization based on molecular graph structure.

[0081] Specifically, for the same drug molecule, its simplified linear input canonical sequence is first obtained and converted into a coding representation through the corresponding encoding method; then, according to the preset rules, the drug molecule is processed to generate a molecular fingerprint vector representation; at the same time, a graph structure of the drug molecule is constructed, and the graph structure is encoded using a graph neural network to obtain a graph neural network coding representation.

[0082] It should be noted that constructing characterizations for drug molecules from different dimensions can comprehensively capture the characteristic information of drug molecules, avoid the information loss caused by a single characterization, and enable subsequent model training to be based on richer drug characteristics, thereby improving the model's understanding and grasp of drug properties.

[0083] Preferably, identifying protein-binding pocket regions based on three-dimensional structure and extracting the set of pocket residues and their three-dimensional coordinates includes: identifying a set of candidate pockets based on the predicted three-dimensional structure, filtering the candidate pockets based on their confidence level, and extracting the set of residues and their three-dimensional coordinates of the filtered pockets.

[0084] Specifically, based on the predicted three-dimensional structure of the protein, a suitable recognition algorithm is first used to preliminarily identify the protein binding pocket region to obtain a candidate pocket set; then, a credibility evaluation standard is set to evaluate the credibility of each pocket in the candidate pocket set; pockets that meet the credibility requirements are selected according to the evaluation results, and then residue sets and the three-dimensional coordinates of each residue are extracted from these selected pockets.

[0085] It should be noted that by first identifying candidate pockets and then filtering them based on their credibility, false or irrelevant pocket regions can be effectively eliminated, ensuring the accuracy and validity of the extracted pocket residue set and its three-dimensional coordinates, and providing high-quality data support for subsequent protein-related model training.

[0086] Preferably, the self-supervised tasks include residue or atom feature masking recovery, neighboring residue pair 3D distance regression, and 3D coordinate denoising recovery. Specifically, these three self-supervised tasks are carried out simultaneously during the protein-side self-supervised pre-training process. For the residue or atom feature masking recovery task, the features of some residues or atoms are masked, and then the model learns to recover the masked features. In the neighboring residue pair 3D distance regression task, the 3D distance between neighboring residue pairs is used as the target, and the model learns to predict this distance. In the 3D coordinate denoising recovery task, noise interference is first added to the protein's 3D coordinates, and then the model is prompted to learn to recover the original 3D coordinates.

[0087] It should be noted that joint training with multiple self-supervised tasks can train the protein model from different perspectives, enhance the model's ability to learn protein structure and features, and enable the pre-trained protein model to have better performance and capture key information about the protein more accurately.

[0088] Preferably, the calculation method for the unified drug representation vector or the unified protein representation vector includes: constructing a gated discriminant input matrix for the multimodal drug representation or the multimodal protein representation based on the gated weight fusion method; calculating position-wise and dimension-wise gated weights through a multilayer perceptron and a sigmoid function; and performing feature fusion on the corresponding representations according to the gated weights; the multimodal protein representation includes a protein pocket representation vector and a protein sequence representation vector.

[0089] Specifically, when calculating the unified drug representation vector, multiple modal representations of the drug are collected, and a gating discriminant input matrix is ​​constructed based on these representations. This matrix is ​​then input into a multilayer perceptron for processing, followed by sigmoid function calculation to obtain position-wise and dimension-wise gating weights. Finally, based on these gating weights, the multiple modal representations of the drug are weighted and fused to obtain the unified drug representation vector. When calculating the unified protein representation vector, the protein pocket representation vector and the protein sequence representation vector are used as the protein multimodal representations. The same gating weight fusion method is employed to construct the input matrix, calculate the gating weights, and perform feature fusion to obtain the unified protein representation vector.

[0090] It should be noted that the gated weight fusion method can adaptively assign appropriate weights to the representations of different modalities, highlight important features, suppress irrelevant or redundant features, and enable the unified representation vector obtained by fusion to more effectively reflect the core features of drugs or proteins, thereby improving the accuracy of subsequent interaction modeling and prediction.

[0091] Preferably, the cross-attention mechanism is used to calculate the cross-attention in both the drug and protein directions, including: achieving structural symmetry modeling by sharing a weight matrix in the bidirectional cross-attention layer; obtaining the interaction features from the corresponding perspective by using the unified representation vector of the drug as the query and the unified representation vector of the protein as the key and value; and obtaining the interaction features from the corresponding perspective by using the unified representation vector of the protein as the key and value.

[0092] Specifically, in the drug-protein interaction modeling process, a bidirectional cross-attention layer is constructed and a shared weight matrix is ​​set to achieve structurally symmetrical modeling. On one hand, the unified drug representation vector is used as the query, and the unified protein representation vector is used as the key and value, which are then input into the bidirectional cross-attention layer to calculate the interaction features between the drug and protein from this perspective. On the other hand, the unified protein representation vector is used as the query, and the unified drug representation vector is used as the key and value, which are also input into the same bidirectional cross-attention layer to obtain the interaction features from another perspective.

[0093] It should be noted that the bidirectional cross-attention mechanism can fully capture the characteristics of the interaction between drugs and proteins. The structural symmetry modeling achieved by the shared weight matrix ensures the consistency and fairness of the calculation of interaction features in both directions, making the obtained interaction features more comprehensive and accurate, and providing a more reliable basis for subsequent predictions.

[0094] Preferably, the prediction results include binary classification results of drug-target interactions, numerical regression results of drug-target affinity, and multi-classification results of drug-target action mechanisms, including inhibitory or activating effects.

[0095] Specifically, in the multi-task joint prediction stage, corresponding prediction models are constructed based on the interaction feature vectors. For the binary classification prediction of drug-target interaction results, the model learns to determine whether there is an interaction between the drug and the target; in the numerical regression prediction of drug-target affinity results, the model learns to predict the specific numerical value of the affinity between the drug and the target; and in the multi-classification prediction of drug-target action mechanism results, the model learns to distinguish whether the interaction between the drug and the target is inhibitory, activating, or other mechanisms of action.

[0096] It should be noted that the simultaneous output of multiple prediction results can meet the application needs of different scenarios, provide more comprehensive reference information for drug development, disease treatment and other related research, and help researchers to gain a deeper understanding of the interaction between drugs and targets.

[0097] Accordingly, this embodiment also provides a drug-target interaction prediction system based on multimodal self-supervised learning, including:

[0098] The drug-side self-supervised pre-training module is configured to construct representations of the same drug molecule from multiple perspectives. On large-scale unlabeled drug data, multiple perspectives are used to compare learning loss to align the representations and train the drug pre-training model.

[0099] The protein-side self-supervised pre-training module is configured to obtain the predicted three-dimensional structure based on the amino acid sequence, identify the protein binding pocket region based on the three-dimensional structure, extract the set of pocket residues and their three-dimensional coordinates, and construct a graph representation of the pocket; the protein pre-training model is obtained by pre-training a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks.

[0100] The multi-feature fusion module is configured to input multiple perspective representations of the drug to be predicted into the drug pre-training model to obtain a multimodal representation of the drug, and then fuse them to obtain a unified representation vector of the drug; input the pocket diagram representation of the protein to be predicted into the protein pre-training model to obtain a protein pocket representation vector; input the amino acid sequence into the existing protein language model to obtain a protein sequence representation vector, and then fuse them to obtain a unified representation vector of the protein.

[0101] The drug-protein interaction modeling module is configured to calculate the cross-attention between the drug and the protein in two directions through a bidirectional cross-attention mechanism, and obtain the drug and protein representations with enhanced interaction by combining residual connections and feedforward networks.

[0102] The multi-task joint prediction module is configured to aggregate the drug representation and protein representation with enhanced interaction to obtain an interaction feature vector, perform multi-task prediction based on the interaction feature vector, and output at least two prediction results.

[0103] It should be noted that each of the above modules performs its own function while cooperating with each other to form an organic whole. Through modular design, the system structure is clear, making it easy to maintain and upgrade. At the same time, the collaborative work of each module can efficiently complete the drug-target interaction prediction task, improving prediction efficiency and quality.

[0104] Example 2

[0105] This embodiment provides a drug-target interaction prediction method based on multimodal self-supervised learning, including the following steps:

[0106] 1. Multi-view comparative learning of drugs in pre-training. Existing methods generally use only a single modality (such as SMILES, fingerprints, or molecular graphs) to represent drug molecules, lacking unified alignment and robust fusion of multi-view information on unlabeled data. This method introduces a three-view comparative learning approach (Tri-view InfoNCE) based on SMILES-Transformer, Morgan fingerprints, and molecular graph GCNs on large-scale unlabeled drug data. This achieves self-supervised alignment and unified pre-trained representation of drug semantics, substructures, and topological information.

[0107] 2. Joint self-supervised learning of three protein pocket tasks during pre-training. To address the shortcomings of existing methods in utilizing the 3D structure of proteins, especially the binding pocket region, and the fact that self-supervised tasks often remain at the level of simple masking and lack 3D geometric prior injection, this method constructs an SE(3)-Transformer network around the protein pocket region. It designs three geometrically related self-supervised tasks: mask recovery, neighbor distance regression, and coordinate denoising. These tasks are jointly pre-trained on large-scale unlabeled structural data, explicitly injecting pocket-level 3D geometric priors.

[0108] 3. Multimodal Characterization and Fusion of Drugs and Proteins. Existing methods often employ simple splicing between multimodal / multi-view features of drugs and proteins, lacking mechanisms for scale calibration, distribution alignment, and information reweighting within a unified embedding space. This method maps pre-trained three-view representations of drugs and three-dimensional pocket representations of proteins into a unified embedding space. Through learnable normalization and reweighting, dimensional consistency and distribution calibration are achieved, constructing a unified multimodal feature base for interactive modeling.

[0109] 4. Shared-weight cross-attention modeling of drugs and proteins. Addressing the issue that existing interaction modeling methods often rely on simple concatenation, unidirectional attention, or one-time scoring functions, making it difficult to characterize the bidirectional dependencies and higher-order coupling relationships between drugs and proteins with fine-grained precision, this method employs a shared-weight bidirectional cross-attention structure. It simultaneously models the bidirectional attention between "drug→protein" and "protein→drug" within a unified embedding space. Furthermore, it obtains an enhanced interaction representation through residual fusion with monomeric representations, thereby improving the symmetry, expressiveness, and scalability of the interaction modeling.

[0110] 5. Multi-task Joint Training. Existing methods often model DTI (binary classification) or DTA (regression) tasks separately, paying less attention to MoA (multi-classification), and lacking a unified model for joint prediction and collaborative optimization of the three tasks. This method constructs a multi-head output structure for DTI (binary classification), DTA (regression), and MoA (multi-classification) tasks based on unified interaction features. It introduces a multi-task joint loss for collaborative training and combines AutoML / model ensemble techniques to automatically select and optimize downstream prediction structures, thereby simultaneously improving the overall performance and generalization ability of the three tasks.

[0111] Specifically, this drug-target interaction prediction method based on multimodal self-supervised learning can be implemented using the following steps:

[0112] 1) Drug multi-perspective self-supervised pre-training, such as Figure 2 As shown.

[0113] For the same drug molecule, three different characterization perspectives were constructed:

[0114] a. SMILES-Transformer encoding based on SMILES sequences;

[0115] b. Morgan's fingerprint vector generated based on rules;

[0116] c. GCN (or other graph neural network) encoding based on molecular graph structure.

[0117] On large-scale unlabeled drug data, a Tri-view InfoNCE contrastive learning loss is designed to align the representations from the three perspectives mentioned above, resulting in a consistent unified drug representation vector.

[0118] 2) Protein pocket three-task joint self-supervised pre-training, such as... Figure 3 As shown.

[0119] Protein language models (such as ESM-2) are used to encode protein amino acid sequences to obtain residue-level sequence semantic representations;

[0120] Based on the three-dimensional structure of proteins, the binding pocket region is identified, and the set of pocket residues and their three-dimensional coordinates are extracted.

[0121] Construct an SE(3)-Transformer pocket modeling network, taking pocket residue features and 3D coordinates as input, and pre-train it under at least two or three joint constraints from the following three self-supervised tasks:

[0122] a. Residue / atom feature mask recovery task;

[0123] b. Three-dimensional distance regression task for neighboring residues;

[0124] c. Three-dimensional coordinate denoising and restoration task;

[0125] Vector norm constraints and linear projections are applied to the pocket representation output by SE(3)-Transformer, and the residue sequence is downsampled or pooled to obtain a fixed-length pocket sequence representation.

[0126] 4) Bidirectional cross-attention interaction modeling based on shared weights, such as... Figure 4 As shown.

[0127] Construct a cross-attention layer with shared parameters in a unified embedding space:

[0128] a. Using drug representation as the query and protein pocket sequence as the key / value pair, calculate the cross-attention between "drug → protein" to obtain protein interaction features from a drug-oriented perspective;

[0129] b. Using protein pocket sequences as queries and drug representations as keys and values, calculate the cross-attention between "protein and drug" to obtain drug interaction features from a protein perspective;

[0130] The cross-attention layers in both directions share a weight matrix, achieving bidirectional modeling with structural symmetry;

[0131] The interaction outputs in their respective directions are fused with the original monomer representations through residual connections and feedforward networks to obtain interaction-enhanced drug representations and interaction-enhanced protein representations.

[0132] 5) Multi-task joint prediction

[0133] By aggregating the drug representation with enhanced interaction with the protein representation (e.g., by splicing or other learnable aggregation methods), a drug-target interaction feature vector is obtained.

[0134] A multi-task learning structure is constructed on top of this interactive feature vector, and the output is:

[0135] a. DTI binary classification results (whether there is an interaction);

[0136] b. DTA regression results (affinity values);

[0137] c. MoA multiclassification results (mechanisms of action such as inhibition / activation);

[0138] By introducing automated machine learning tools (such as AutoGluon), the model family and hyperparameters of the multi-task prediction head are searched and integrated to obtain the multi-task prediction configuration with the best overall performance.

[0139] The entire network is trained end-to-end using multi-task joint loss.

[0140] In this embodiment, at least one of the three perspectives on the drug side is used for self-supervised pre-training aligned through contrastive learning: the same drug molecule is simultaneously subjected to at least two of the following perspectives: SMILES sequence perspective, molecular fingerprint perspective, and molecular graph perspective; on unlabeled drug data, different perspective representations are aligned using Tri-view InfoNCE or equivalent multi-view contrastive learning loss to obtain a consistent drug characterization.

[0141] In this embodiment, three-dimensional geometric self-supervised modeling is performed around the protein pocket region based on SE(3)-Transformer: modeling is performed on the protein binding pocket as a unit, rather than using only the full-length protein global vector; a network with denaturation / invariance constraints such as SE(3) (such as SE(3)-Transformer) is used to encode the pocket residues and their three-dimensional coordinates; at least two types of joint self-supervised training are used in the three tasks of mask recovery, adjacent distance regression, and coordinate denoising to inject three-dimensional geometric priors.

[0142] In this embodiment, the bidirectional cross-attention interaction modeling structure with shared weights includes: simultaneously modeling cross-attention in both the drug→protein and protein→drug directions; the cross-attention layers in both directions share parameters to achieve structural symmetry; and the cross-attention output and the individual representation are fused through residuals to form an interactively enhanced representation.

[0143] Preferably, drug characterization and protein pocket characterization are scaled and distributed in a unified embedding space: the unified multi-view drug characterization and protein pocket characterization are mapped to a unified embedding space; scale normalization, distribution alignment and information reweighting are performed in this space to provide a unified feature base for interactive attention modeling.

[0144] Preferably, multi-task joint training is performed on the basis of unified interaction features: using the same interaction feature base, at least two of the three task results of DTI, DTA and MoA are output simultaneously; collaborative training is performed through multi-task joint loss, and the overall performance is improved by utilizing the correlation between tasks.

[0145] Preferably, the specific network backbone is selected as follows: the SMILES encoder can be any Transformer structure or other sequence model, not limited to a specific SMILES-Transformer implementation; the molecular graph encoder can be any graph neural network such as GCN, GAT, MPNN; the protein language model can be ESM-2 or a similar protein pre-trained model.

[0146] Preferably, the combination of self-supervised tasks includes: in addition to mask recovery, distance regression, and coordinate denoising, protein pocket-side self-supervised tasks can also be added / replaced with similar geometric tasks (such as angle prediction, contact map prediction, etc.); drug-side multi-view contrastive learning can introduce additional perspectives (such as 3D conformational perspectives) or use other forms of multi-view contrastive loss.

[0147] It should be noted that the drug-target interaction prediction method based on multimodal self-supervised learning in this embodiment has the following advantages:

[0148] 1. Improve the robustness of drug characterization and the efficiency of labeled data utilization.

[0149] By introducing SMILES-Transformer, Morgan fingerprint, and molecular graph GCN into unlabeled drug data, a three-view Tri-view InfoNCE was compared with pre-training:

[0150] This method unifies and aligns semantic (SMILES), substructure (fingerprint), and topological (molecular graph) information in the representation space, significantly reducing information conflict and redundancy between different modalities compared to existing methods that use only a single modality or simple splicing.

[0151] Even with a small number of labeled samples, it is still possible to obtain a stable unified characterization of drugs by relying on large-scale unlabeled data, thereby improving the prediction accuracy and generalization ability of the model in low-sample scenarios.

[0152] Reducing reliance on large-scale labeled DTI / DTA data can achieve higher performance at the same annotation scale, resulting in improved annotation efficiency and reduced risk of overfitting.

[0153] 2. Enhances protein three-dimensional geometry perception and improves transconformation generalization.

[0154] By constructing an SE(3)-Transformer around a protein pocket, and combining it with three self-supervised tasks: mask recovery, neighbor distance regression, and coordinate denoising:

[0155] It can explicitly learn the 3D geometric prior of pocket localities on large-scale unlabeled structured data, significantly enhancing the model's sensitivity to key factors such as pocket shape, depth, and local conformation compared to existing methods that rely solely on one-dimensional sequences or simple 3D auxiliary features.

[0156] The constraints of SE(3) and other variable / invariant properties enable the same pocket to obtain a consistent representation under different spatial orientations. Compared with the 3D modeling method that does not consider spatial symmetry, it reduces the sensitivity to coordinate system and conformational pose and improves the robustness under cross-conformation and cross-structure source (predicted structure / actual structure) conditions.

[0157] In subsequent DTI / DTA / MoA tasks, it demonstrates enhanced ability to distinguish "which residues / local regions are involved in binding", thereby improving the accuracy of structure-based predictions.

[0158] 3. The refined interaction characterization and parameter efficiency brought about by shared weight bidirectional cross attention.

[0159] A bidirectional cross-attention structure with shared weights is employed to simultaneously model the bidirectional dependencies between "drug → protein" and "protein → drug," and this is fused with the residual representation of the individual entity.

[0160] Compared to simple splicing and unidirectional attention, it can characterize "which part of the drug focuses on which part of the protein" and "how the local environment of the protein, in turn, affects the drug characterization" at the substructure-residue level, significantly improving the expressive power and interpretability of interaction features.

[0161] By sharing parameters through two-way attention, structural symmetry and parameter reuse are achieved. Compared with two completely independent cross-attention paths or multiple sets of scoring functions, this reduces the number of parameters, lowers the training difficulty, and helps the model learn common patterns in interaction patterns.

[0162] Residual fusion ensures that the enhanced interactive information is superimposed on the prior information of the original single structure rather than being overwritten, thus balancing expressive power and stability and reducing gradient decay and information loss problems when the interaction layer is deep.

[0163] 4. Improved overall decision-making quality and generalization brought about by multi-task joint training.

[0164] Based on unified interaction features, multi-task joint training of three tasks—DTI (binary classification), DTA (regression), and MoA (multi-class classification)—is performed simultaneously, and the structure and hyperparameters are optimized by combining AutoML / model ensemble.

[0165] By fully leveraging the inherent relationships among the three tasks in terms of "whether it works – the strength of its effect – the mode of its effect", compared to existing methods that model DTI, DTA and MoA separately, the prediction accuracy and stability of each task can be improved without increasing the annotation cost.

[0166] Multi-task constraints play a regularization role to some extent, suppressing overfitting to single-label noise or single-task distribution, enabling the model to exhibit better generalization ability in cross-dataset and cross-target scenarios.

[0167] AutoML enables automatic searching and integration of different prediction head structures and hyperparameters, reducing the cost of manual tuning and repeated trial and error, and improving model development efficiency and engineering usability.

[0168] In practical applications, the ability to provide multi-dimensional information such as "whether it is worth paying attention to", "the possible affinity", and "whether it is more likely to inhibit or activate" in the same model output is beneficial for more refined sorting and decision-making in the virtual screening stage, thereby improving the quality and efficiency of the overall drug discovery process.

[0169] In summary, this method, through the combination of key technical features such as multi-view self-supervised pre-training, pocket-level SE(3) 3D modeling, unified multimodal embedding, shared weight bidirectional cross attention, and multi-task joint learning, can achieve substantial improvements in prediction accuracy, robustness, efficiency of labeled data utilization, model interpretability, and development efficiency compared to existing technologies that rely solely on single-modal supervised training, simple feature splicing, and single-task modeling. It has significant technological advancements and application value.

[0170] Example 3

[0171] This embodiment is based on embodiment 2:

[0172] This embodiment provides a drug-target interaction prediction method based on multimodal self-supervised learning, including the following steps:

[0173] 1. Self-supervised multi-perspective drug discovery and protein pocket learning

[0174] In real-world drug discovery scenarios, high-quality labeled data is scarce, while unlabeled data is abundant. Relying solely on supervised learning can lead to overfitting and a lack of generalization ability in the model. This embodiment designs specific self-supervised pre-training methods for drugs and proteins, respectively. By constructing surrogate targets consistent with downstream tasks, transferable prior representations are extracted from molecular geometry, semantics, and cross-modal consistency without the need for manual annotation.

[0175] (1) Drugs: Self-supervised training of three-view alignment

[0176] The drug SMILES are input into a Transformer with absolute position encoding to obtain the drug sequence representation. The SMILES sequence is further expanded into a fingerprint representation of a pseudo-token sequence using the Morgan algorithm. Then, input the molecular graph converted by RDKit into GCN to obtain the molecular graph topological representation. Subsequently, for the three-view sequence representations, each independent projection head is used to unify the dimensions of each embedded representation to... And employ cross-attention layers to combine different lengths The sequence alignment is fixed length. The three-view isomorphic matrix is ​​obtained. Then, any two perspectives of the same drug molecule... The positive pair is represented by , and the negative pairs are represented by other samples in the same batch. Let the batch size be . , No. The sample at the th k The token is 1 bit Bidirectional, position-by-position comparison learning is employed.

[0177]

[0178] Where sim is the cosine similarity. This is the temperature coefficient.

[0179] (2) Proteins: Pocket-level geometric self-supervision

[0180] Drug-protein binding occurs within a three-dimensional pocket, and the binding and regulation methods are determined by the coupling of the pocket's local geometry and chemical semantics. Therefore, this embodiment adopts a geometrically consistent self-supervised scheme based on SE(3)-Transformer to learn a symmetry-consistent, robust, and transferable pocket representation for the pocket's atomic-level graph under unlabeled conditions. First, based on the predicted structure of AlphaFold2 and the detection results of DoGSite3, the optimal binding pocket is selected. Atoms are trimmed within a fixed radius with the pocket center as the sphere center, and an atomic-level radius graph is constructed accordingly. The node semantics and physicochemical properties are organized as scalar features, and the radial basis of distance is used as the edge features. After rigid body enhancement, a small amount of Gaussian noise is added to the coordinates to obtain the graph representation. ,in Add noise to the three-dimensional coordinates of the atom. Characterized by atomic scalars. For the radius graph edge set, Edge features. Represent the graph. After inputting SE(3)-Transformer, the scalar and vector channel features of each atom are obtained. and ， Based on this atomic feature, perform the following three tasks in a joint self-supervised training:

[0181] (a) Mask semantic recovery

[0182] exist A mask set is constructed by randomly selecting some unnoisy atoms. Record the semantic features of postfix zeros. However, the coordinates are preserved. The masked sequence is input into an MLP to predict the features of the masked atoms, and the prediction results are then used. The loss is modeled using cross-entropy by comparing the features with those of the original records.

[0183]

[0184]

[0185] (b) Neighbor distance regression

[0186] In the set of adjacent edges Above, for spatially nearest neighbor pairs of atoms Regressing its Euclidean distance; the objective value is derived from the unnoised atomic coordinates. Calculated The scalar channel features at both ends are concatenated and input into the regression head to predict. And model the loss using Smooth-L1:

[0187]

[0188]

[0189] (c) Coordinate prediction

[0190] With vector channels As input, the linear head directly predicts the noisy coordinates. and with the target In contrast, the loss is constructed using Smooth-L1:

[0191]

[0192]

[0193] The three self-monitoring tasks were jointly optimized using a weighted average, among which... , :

[0194]

[0195] In summary, the self-supervised methods described above significantly reduce reliance on labeled data and mitigate instability caused by single-modal bias and structural noise, improving cross-distribution generalization for novel scaffold drugs and low-homology proteins. Simultaneously, they produce fixed-length matrix inputs naturally compatible with downstream cross-attention, enhancing interpretability at the pocket / residue level. Overall, self-supervised characterization training leads to higher accuracy and stability for subsequent downstream tasks.

[0196] 2. Multimodal drug and protein characterization and fusion

[0197] Building upon the self-supervised backbone trained in the previous step, this step aims to robustly acquire and fuse multimodal sequence representations of drugs and proteins, constructing a feature matrix that is shape-consistent, semantically aligned, and geometrically interpretable. Standardized projection and shared query pooling are used to minimize cross-modal distribution differences. Subsequently, location-gated fusion units adaptively select and mix reliable information at the same location, providing standardized input for subsequent interaction modeling.

[0198] (1) Characterization of the drug side

[0199] A single drug modality cannot simultaneously cover sequence information, functional group semantics, and molecular topology. Furthermore, supervised learning is prone to overfitting a single distribution when annotations are scarce. Therefore, this embodiment constructs three complementary perspectives for each drug, namely:

[0200] Sequence Perspective: Drug SMILES format is segmented and processed by a Transformer with added absolute position encoding to obtain sequence embeddings. .

[0201] Fingerprint perspective: The drug Morgan fingerprint vector obtained by the algorithm is divided into equal parts. Each segment is a continuous block. A learnable linear mapping is performed on each segment, and block location information is added. The segments are then arranged in order to obtain the fingerprint embedding. .

[0202] Topological perspective: A molecular graph with atoms as nodes and chemical bonds as edges is constructed using RDKit, and then input into GCN for message passing and aggregation to obtain an embedded representation that reflects the molecular topology and bond connection patterns. .

[0203] Then the representation matrix for each viewpoint Each dimension is projected using an independent projection head. Unified to :

[0204]

[0205] in, For LayerNorm (by (Normalize the dimensions). For dimension-wise nonlinear activation; Used to convert the original dimensions Unified to the public dimension ; This is used to stabilize the mapping. Then, shared queries are introduced. Perform cross-attention pooling, combining pools of different lengths. The sequence alignment is fixed length. :

[0206]

[0207]

[0208] Obtain the three-view isomorphic matrix .

[0209] (2) Characterization of the protein side

[0210] After obtaining the protein representation using the same method as in the previous step, input it into SE(3)-Transformer to obtain scalar channels. With vector channels The output is taken and By concatenating the sequences, linear mapping, and downsampling, a pocket sequence is obtained. Simultaneously, the protein amino acid sequence is input into ESM-2 to obtain a sequence-level representation. The two representations described above are processed in the same way as the drug-side representation, using independent projection heads and shared queries respectively to align them to the same dimension and length, resulting in a homomorphic matrix. .

[0211] After obtaining the multimodal representations of drugs and proteins, this embodiment introduces a position-based adaptive gating fusion method: without changing the matrix shape, it selects and mixes the most reliable modal information at the same position using learned weights, thereby condensing the multi-source representations into a unified drug representation. With unified protein representation :

[0212] Multimodal data of the same category Construct gated input discrimination :

[0213]

[0214] in and Depicting conflict and consistent themes separately, then applying a light touch The gating weights are obtained by combining them with the Sigmoid function:

[0215]

[0216] And perform linear mixing based on position:

[0217]

[0218] For the three-view isomorphic matrix of the drug side First and The intermediate representation is obtained by fusion. Then use and The final drug representation is obtained by fusion. :

[0219]

[0220]

[0221] Sequence and pocket representation on the protein side The final protein representation can be obtained directly through single-step fusion. :

[0222]

[0223] This step involves multimodal data characterization and fusion of drugs and proteins to preserve complementary information and geometric priors across modalities while significantly reducing cross-modal distribution differences and suppressing unimodal noise and bias. This provides a more robust input foundation and a higher prediction ceiling for downstream tasks.

[0224] 3. Interaction relationship modeling based on cross-attention mechanism with shared weights

[0225] Based on the unified representation obtained in the previous step: Drug matrix With protein matrix This step builds upon this foundation by employing bidirectional multi-head cross-attention with shared weights to learn the position-to-position level drug-protein interaction characterization, and then residually fuses the interaction results with the original representation to obtain the final sequence representation. This approach accurately captures fine-grained contact patterns and intensities without compromising uniformity, and provides interpretability.

[0226] First, the same set of linear projections is used for both the drug and the protein:

[0227]

[0228]

[0229] in, By sharing in both directions, we ensure that "looking at proteins from a drug perspective" and "looking at drugs from a protein perspective" are in the same metric space, thus making attention scores comparable and stable.

[0230] Searching for drugs, protein is... K , V This involves finding the most relevant interaction sites for drugs within the protein representation, and vice versa, querying the proteins and performing the same operation; resulting in interpretable alignment and interaction matrices.

[0231]

[0232] in, The bit-to-bit alignment matrix is ​​processed in parallel by multiple heads and then concatenated to obtain the final interactive representation.

[0233] Without compromising the original semantics, the interaction information and the original representation are residually fused to obtain the final drug and protein representation matrix. and :

[0234]

[0235] 4. Prediction of drug-target interaction relationship

[0236] The final sequence representation obtained in the above step and Based on this, attention pooling is first used to refine sample-level representations, which are then concatenated to form fixed-dimensional features for downstream tasks. Learnable vectors are then used first. Each position is scored, and the position weights are normalized. reuse We obtain the drug vector representation by performing a weighted summation on the entire row vector. :

[0237]

[0238] Similarly, the protein vector representation is obtained. After splicing, it forms .by As input features, predictors for the three tasks DTI, DTA, and MoA are integrated from the model pool using AutoGluon. , , The objectives and losses for each task are as follows:

[0239] For drug-target interaction (DTI) prediction, With tags As a data table, the loss is characterized by binary cross-entropy:

[0240]

[0241] For drug-target binding affinity (DTA) prediction, With tags (Use the negative logarithm of the dissociation constant,) As a data table, the loss is characterized by mean squared error:

[0242]

[0243] Three tasks are modeled independently and trained in parallel, with the combined loss being:

[0244]

[0245] The parameters of the feature extraction step, the interactive modeling step, and each task predictor are jointly backpropagated and updated.

[0246] Example 4

[0247] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the drug-target interaction prediction method based on multimodal self-supervised learning as described in Embodiment 1, Embodiment 2, or Embodiment 3. The computer program can be in the form of source code, object code, executable file, or some intermediate form.

[0248] Specifically, the computer device's memory stores computer programs containing instructions related to a drug-target interaction prediction method based on multimodal self-supervised learning. When the prediction method needs to be executed, the processor retrieves the computer program from memory and begins execution. Following the instructions in the program, the processor sequentially executes steps such as drug-side self-supervised pre-training, protein-side self-supervised pre-training, multi-feature fusion, drug-protein interaction modeling, and multi-task joint prediction, gradually completing the drug-target interaction prediction process.

[0249] It should be noted that by utilizing the hardware resources of computer equipment, the drug-target effect prediction method based on multimodal self-supervised learning can be executed quickly and efficiently. The computing power of the processor and the storage capacity of the memory ensure the smooth progress of the prediction process, improve the prediction efficiency, and enable the prediction method to be applied to various scenarios.

[0250] Example 5

[0251] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the drug-target interaction prediction method based on multimodal self-supervised learning as described in Embodiments 1, 2, or 3. The computer program can be in the form of source code, object code, executable file, or some intermediate form. The storage medium includes any entity or device capable of carrying computer program code, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0252] Specifically, a computer program containing the logic related to a drug-target interaction prediction method based on multimodal self-supervised learning is stored in a computer-readable storage medium. When this storage medium is connected to a processor, the processor can read and execute the computer program in the storage medium. During execution, the processor performs self-supervised pre-training on the drug and protein sides, multi-feature fusion, interaction modeling, and multi-task joint prediction according to the steps set in the program, ultimately achieving drug-target interaction prediction.

[0253] It should be noted that computer-readable storage media facilitate the storage and transmission of computer programs, enabling drug-target effect prediction methods based on multimodal self-supervised learning to be shared and used across different computer devices. This improves the portability and practicality of the method and expands its application scope.

[0254] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

[0255] It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

Claims

1. A drug-target interaction prediction method based on multimodal self-supervised learning, characterized in that, include: Drug-side self-supervised pre-training: Construct representations of the same drug molecule from multiple perspectives, align the representations using comparative learning loss from multiple perspectives on large-scale unlabeled drug data, and train to obtain a drug pre-trained model; Protein-side self-supervised pre-training: The predicted three-dimensional structure is obtained based on the amino acid sequence. Based on the three-dimensional structure, the protein binding pocket region is identified and the set of pocket residues and their three-dimensional coordinates are extracted to construct a graph representation of the pocket. The protein pre-training model is obtained by pre-training a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks. Multi-feature fusion: Input the multi-perspective representations of the drug to be predicted into the drug pre-trained model to obtain a multi-modal representation of the drug, and fuse them to obtain a unified representation vector of the drug; The pocket diagram representation of the protein to be predicted is input into the protein pre-training model to obtain the protein pocket representation vector, the amino acid sequence is input into the existing protein language model to obtain the protein sequence representation vector, and the two are fused to obtain the unified protein representation vector. Drug-protein interaction modeling: Through a bidirectional cross-attention mechanism, the cross-attention between the drug and the protein is calculated in two directions respectively. Combined with residual connections and feedforward networks, the drug and protein representations with enhanced interaction are obtained. Multi-task joint prediction: The interaction-enhanced drug representation and protein representation are aggregated to obtain an interaction feature vector. Multi-task prediction is performed based on the interaction feature vector, and at least two prediction results are output. The multiple perspectives for characterizing the same drug molecule include: encoding representation based on simplified linear input canonical sequences of the molecule, molecular fingerprint vector representation based on rule generation, and graph neural network encoding representation based on molecular graph structure. Identifying protein-binding pocket regions based on the three-dimensional structure and extracting the set of pocket residues and their three-dimensional coordinates includes: identifying a set of candidate pockets based on the predicted three-dimensional structure, filtering the candidate pockets based on their confidence level, and extracting the set of residues and their three-dimensional coordinates of the filtered pockets. The self-supervised tasks include residue or atom feature mask recovery task, neighboring residue pair 3D distance regression task, and 3D coordinate denoising and recovery task. The calculation method for the unified drug representation vector or the unified protein representation vector includes: constructing a gated discriminant input matrix for the multimodal drug representation or the multimodal protein representation based on a gated weight fusion method; calculating position-wise and dimension-wise gated weights using a multilayer perceptron and a sigmoid function; and performing feature fusion on the corresponding representations based on the gated weights; the multimodal protein representation includes a protein pocket representation vector and a protein sequence representation vector. The method of calculating cross-attention in both drug and protein directions through a bidirectional cross-attention mechanism includes: achieving structural symmetry modeling by sharing a weight matrix through a bidirectional cross-attention layer; obtaining interactive features from the corresponding perspective by using the unified drug representation vector as the query and the unified protein representation vector as the key and value; and obtaining interactive features from the corresponding perspective by using the unified protein representation vector as the key and value.

2. The drug-target interaction prediction method based on multimodal self-supervised learning according to claim 1, characterized in that, The prediction results include binary classification results of drug-target interactions, numerical regression results of drug-target affinity, and multi-classification results of drug-target action mechanisms, including inhibitory or activating effects.

3. A drug-target interaction prediction system based on multimodal self-supervised learning, employing the drug-target interaction prediction method as described in claim 1, characterized in that, include: The drug-side self-supervised pre-training module is configured to construct representations of the same drug molecule from multiple perspectives. The representations are then aligned using a multi-perspective comparative learning loss on large-scale unlabeled drug data to train a drug pre-training model. The protein-side self-supervised pre-training module is configured to obtain the predicted three-dimensional structure based on the amino acid sequence, identify the protein binding pocket region based on the three-dimensional structure, extract the set of pocket residues and their three-dimensional coordinates, and construct a graph representation of the pocket; and pre-train the protein pre-training model through a network with three-dimensional isovariability or invariance constraints under the joint constraints of at least two self-supervised tasks. The multi-feature fusion module is configured to input multiple perspective representations of the drug to be predicted into the drug pre-training model to obtain a multimodal representation of the drug, and fuse them to obtain a unified representation vector of the drug; input the pocket diagram representation of the protein to be predicted into the protein pre-training model to obtain a protein pocket representation vector; input the amino acid sequence into an existing protein language model to obtain a protein sequence representation vector, and fuse them to obtain a unified representation vector of the protein. The drug-protein interaction modeling module is configured to calculate the cross-attention between the drug and the protein in two directions through a bidirectional cross-attention mechanism, and obtain the drug and protein representations with enhanced interaction by combining residual connections and feedforward networks. The multi-task joint prediction module is configured to aggregate the interaction-enhanced drug representation and protein representation to obtain an interaction feature vector, perform multi-task prediction based on the interaction feature vector, and output at least two prediction results.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the drug-target interaction prediction method based on multimodal self-supervised learning as described in claim 1 or 2.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the drug-target interaction prediction method based on multimodal self-supervised learning as described in claim 1 or 2.

Images