Application of multi-modal feature fusion model in drug target binding affinity prediction

Abstract

The invention relates to the field of drug target interaction, and discloses an application of a multi-modal feature fusion model in drug target binding affinity prediction, which comprises the following steps: S1, obtaining an original data set of drug target binding affinity, and taking a binding affinity index in the data set as a label of a training set and a test set; s2, acquiring representation information of other modals by using the drug sequence and target sequence representation provided in the original data set, preprocessing the acquired representation information of drugs and targets in different modals, and processing the representation information into a model acceptable feature vector form; and S3, respectively storing the multi-modal feature vectors of the drug and the target as well as the binding affinity tags in a form of pytorchsensor (. Pt). According to the method, shallow characterization of the drug and protein sequences is learned by applying the pre-training model, and the problem that a drug target binding affinity data set is insufficient is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of drug-target interaction, and in particular to the application of a multimodal feature fusion model in drug-target binding affinity prediction. Background Art

[0002] Drug-target binding affinity refers to the tightness of the binding between the drug molecule and the biological target, usually expressed as a binding constant (such as IC50, Kd or Ki). These constants reflect the effectiveness of the drug molecule binding to the target under certain conditions. The higher the affinity, the more effective the drug is at a lower concentration. It should be noted that when using Kd, the negative logarithm of Kd is usually taken for standardization (such as in the Davis data set), as shown in the formula:

[0003] p Kd =-log 10 (Kd×10 -9 )

[0004] After this processing, the affinity value range is more suitable for model processing and better reflects the binding affinity between the drug and the target. The larger the value, the higher the affinity. The prediction of drug-target binding affinity is of guiding significance for drug design, drug repositioning and other work.

[0005] Traditionally, affinity measurements rely on drug experimental methods, which are accurate but time-consuming and costly, limiting their application in large-scale drug screening. With the continuous improvement of computing power, computer-based methods have also been widely used in the prediction of drug target binding affinity.

[0006] Traditional machine learning methods rely on molecular descriptors and the physicochemical properties of proteins. By manually extracting features, some classic machine learning methods are used to predict drug target affinity, such as support vector machine (SVM), random forest (RF), etc.

[0007] Deep learning-based methods reduce the reliance on manual feature extraction by automatically learning features in molecular structures and protein sequences, and have become a hot topic in drug-target binding affinity prediction in recent years. DeepDTA uses convolutional neural networks to extract features from the molecular descriptors and sequences of drugs and targets to predict the binding affinity between drugs and targets; GraphDTA uses graph neural networks to learn the characteristic information of drugs from molecular graphs; TransDTI, a neural network framework based on Transformer, uses a more efficient Transformer to perform affinity prediction tasks. The patent name "Drug-target binding affinity prediction method based on three-branch CNN" (application number CN202211552644.2) uses the features of convolutional network learning input; the patent name "Drug-target binding affinity prediction method based on cross-scale and cross-modal attention mechanism" (application number CN202410505535.8) uses the cross-modal attention mechanism to fuse multiple features; the patent name "A drug-target binding affinity prediction method based on TransVAE" (application number CN202310840256.2) combines the advantages of Transformer and Variational Autoencoder (VAE) models to predict drug-target binding affinity. For traditional drug experiments, they are often costly, time-consuming and labor-intensive.

[0008] For traditional machine learning methods, they rely on manually extracted molecular descriptors or proteins and may not be able to fully capture the interactions between complex molecules and targets; most machine learning methods only use manual descriptors or one-dimensional sequence data, ignoring the three-dimensional structure of molecules and proteins; during training, traditional machine learning methods are prone to overfitting small-scale, high-dimensional data sets, resulting in poor generalization ability of the model on data sets that have never been seen before.

[0009] In the past, deep learning methods relied on large-scale, high-quality data, but drug-target binding affinity datasets were limited and biased. In the absence of sufficient data, deep learning models were prone to overfitting. Although deep learning models can automatically learn features, their internal mechanisms are complex and it is difficult to explain their decision-making process. This makes the model less interpretable and difficult to provide biological insights. Many past deep learning models were limited to a single representation when learning drug molecules and protein representations, and were unable to capture comprehensive structural features. Summary of the invention

[0010] In order to make up for the above shortcomings, the present invention provides an application of a multimodal feature fusion model in the prediction of drug-target binding affinity. The model processes the features of the drug and target branches respectively through a deep learning method, performs multimodal fusion and makes predictions. The model overcomes the problem of insufficient drug-target interaction data sets and achieves good test results even on small data sets.

[0011] In order to achieve the above object, the present invention adopts the following technical solution: Application of multimodal feature fusion model in drug target binding affinity prediction includes the following steps:

[0012] S1. Obtain the original data set of drug target binding affinity, and use the binding affinity index in the data set as the label of the training set and the test set;

[0013] S2. Use the drug sequence and target sequence provided in the original data set to obtain the representation information of other modalities, and preprocess the obtained representation information of drugs and targets of different modalities to process them into feature vector forms acceptable to the model;

[0014] S3. Save the multimodal feature vectors of drugs and targets and the binding affinity labels in the form of pytorchtensor (.pt);

[0015] S4. Load the .pt file and input the information of different modalities into the deep neural network for training and testing.

[0016] Preferably, the step S2 includes the following steps:

[0017] S2.1: Use RDKit in the python library to generate molecular objects with smiles of drug molecules as input, and then convert the molecular objects into the corresponding molecular fingerprint form (ECFP4);

[0018] S2.2: Use the BRICS segmentation method in RDKit to segment the smiles of the drug molecule. The segmented parts are used as new nodes. A new edge is used to connect the breakpoints. RDKit is used to generate the 167-bit MACCS fingerprint code and 27-bit pharmacophore fingerprint code of the node. The two codes are connected as the node information of each part. The chemical reaction code during BRICS segmentation is used as the edge information at the breakpoint. The above nodes, node information, edges, and edge information constitute the fragment graph of the drug molecule.

[0019] S2.3: Use the sequence of the protein target as an index to search in the PDB database, download the protein structure with the highest sequence similarity, and use the protein structure to calculate the distance between each residue. The C-α atom is a key atom used to represent the position of the protein main chain skeleton. In order to simplify the calculation and stabilize the protein structure representation, the C-α atom is used as the center of the protein residue, that is, the distance between C-α atoms is used as the residue distance. Finally, the names of each amino acid are mapped to a dictionary to generate an adjacency matrix of protein residue distance contacts;

[0020] S2.4: For drug smiles strings and protein sequences, since the lengths of each sequence are different, the overly long sequences are truncated, while the shorter sequences are padded with zeros. The sequence length is 512, and the shallow features of the sequence are extracted using a pre-trained model based on Bert.

[0021] Preferably, the step S4 includes the following steps:

[0022] S4.1: Load the pytorchtensor file and assign the read values ​​of the drug smiles and protein protein dictionaries to the corresponding variables, namely, the representation of the drug smiles sequence, the drug molecule fragment graph, the drug fingerprint, the protein sequence, and the protein residue contact graph;

[0023] S4.2: Input the representation information into the model to obtain the prediction results;

[0024] S4.3: The model is continuously trained and predicted. The learning rate optimizer uses the AdamW optimization algorithm and the weight decay strategy is used to prevent overfitting. The learning rate scheduler uses the cosine annealing algorithm. The loss function uses the root mean square error loss and the loss function in the weighted MoE. The formula is as follows:

[0025]

[0026] loss=loss mse +W moe .loss moe

[0027] Where N is the number of samples, output i is the predicted value of the model, label i is the corresponding true value, W moe is the weight of MoE loss, loss moe is the loss of MoE. After each epoch, the model will enter the verification mode, and the parameters will not be updated. Only the loss on the test set is calculated. The following indicators are used to measure the performance of the model. The consistency coefficient (CI) is expressed as follows:

[0028]

[0029] in, and represents the predicted value of samples i and j, N represents the total number of comparable samples, is an indicator function that takes the value 1 when any of the following is true: i >x j and or x i <x j and (The prediction order is correct), otherwise, the exponential function value is 0, MSE (mean square error), the expression formula is as follows:

[0030]

[0031] R 2 (Coefficient of determination): in is the average of the actual values.

[0032] Preferably, the step S4.2 includes the following steps:

[0033] S4.2.1: For the smiles sequence and protein sequence of the drug, the operations on the two are similar. First, the embedding operation is performed to convert the high-dimensional features of the sequence into low-dimensional features. At this time, the shape of the sequence feature is (B, L sequence , H), where B represents the batch size, L represents the sequence length, and H represents the feature dimension. Then, the shape of smiles is changed from (B, L sequence , H) changes to (B, H, L sequence ), use linear mapping to compress the sequence length (L), and continue to input the newly obtained smiles sequence representation into the convolutional network. The convolutional network is composed of stacked convolutional layers, and a normalization layer is added between blocks. The final output uses a residual connection. The convolutional layer further compresses the sequence length (L) while extracting sequence features. sequence ) was compressed;

[0034] S4.2.2: For the fingerprint representation of the drug, after the embedding operation, it is necessary to add a dimension L to it, changing it from (B, H) to (B, H, L fingeprint ), where L fingerprint The value of

[0035] Preferably, the step S4.2 also includes S4.2.3: for the molecular fragment graph representation of the drug, the representation information is a graph structure including node feature x, edge feature edge_attr and edge connection information edge_index, and the graph structure data is input into a graph neural network based on message passing. The internal operation of the graph network is as follows: first, the node features and edge features are mapped to the output channel using linear changes, which is expressed as:

[0036] x ′ =W node .x

[0037] e′=W edge .e

[0038] The edge connection information is stored in the form of an adjacency matrix, and the normalized term of the adjacency matrix is ​​calculated as follows:

[0039]

[0040] When passing messages, the feature x of the neighbor node is passed j And the edge feature e′ is spliced ​​and expressed as:

[0041] f fusion＝ concat(x j , e′)

[0042] Then the node and edge features are further fused using a linear layer and multiplied by the normalization coefficient norm, expressed as:

[0043] m=w fusion .f fusion .norm

[0044] Perform mean aggregation on the neighbor messages of the node and add the bias term, which is expressed as:

[0045] h i =mean(m)+bias

[0046] Finally, the node features in the same batch are globally pooled to form the final graph representation, and the output returns an output tensor with a shape of (batch_size, 1, out_channels);

[0047] S4.2.4: For the residue contact map representation of proteins, a two-dimensional convolutional network is used for feature learning. The convolutional network module first performs multi-layer convolution operations on the input protein matrix, where ordinary two-dimensional convolutions are two layers and deformable convolutions are four layers. The residue contact map representation processed by the two-dimensional convolution layer finally needs to compress one dimension and make a transformation from (B, H, 1, 1) to (B, 1, H) to meet the needs of subsequent fusion.

[0048] Preferably, the S4.5 further includes S4.2.5: for the feature vectors obtained above, the drug and protein branches are fused respectively, and the fusion strategy is to first connect the multi-modal feature vectors, and the shape after connection is (B, L drug , H) and (B, L protein , H), where L drug =L sequence +L graph +L fingerprint , L protein =L sequence +L fingerprint , and then input the concatenated feature vector into a multi-head self-attention layer based on a multi-expert architecture. The core operation of multi-head self-attention is as follows: linear transformation is performed on the input x to obtain the query vector Q, key vector K and value vector V, which are expressed as:

[0049] Q=XW Q , K = XW K , V = XW V

[0050] Among them, W Q， W K , W V are the weight matrices of linear transformations, respectively. The attention weights are calculated by the query vector Q and the key vector K, expressed as:

[0051]

[0052] Among them, d k is the dimension of the query vector or key vector, softmax is the activation function used to normalize the attention weight, and the multi-head attention mechanism divides the query, key, and value vectors into multiple heads, each of which performs an independent attention operation, expressed as:

[0053] MutiHead(Q,K,V)=Concat(head1,...,head h )W O

[0054] Each expert's multi-head self-attention processes the input, and the output of the i-th expert can be expressed as:

[0055] Y i =MSA i (X)

[0056] Among them, MSA iIt represents the multi-head self-attention mechanism in the i-th expert. The gating mechanism in the MoE module selects the weights of the two experts through Top2Gating, that is, the two with the highest scores are selected through the score matrix calculated by the gating. Assuming that expert i and expert j are selected, the model outputs a weighted combination of the two experts, which is expressed as:

[0057] Y=g i Y i +g j Y j

[0058] Among them, g i and g j The attention weights calculated by Top2Gating ensure that each input can be processed by the most suitable expert. Finally, the outputs of all experts are weighted and fused to obtain the final output of the model.

[0059] S4.2.6: After the two branches are fused by the expert module, the feature vectors of the new drugs and proteins are obtained respectively. The vectors are input into the interactive fusion module, which consists of two Transformer decoders. For the feature vector of the drug, the expression formula is as follows:

[0060]

[0061] For the characteristic vector of protein, the expression formula is as follows:

[0062]

[0063] Finally, the drug feature vector and protein feature vector are connected to obtain a shape of (B, L drug +L protein , H) of the fused feature vector.

[0064] Preferably, the S4.2 further includes S4.2.7: inputting the fused feature vector into a fully connected layer to obtain a final prediction result.

[0065] The present invention has the following beneficial effects:

[0066] 1. In the present invention, by using the pre-trained model to learn the shallow representation of drug and protein sequences, the problem of insufficient drug-target binding affinity data set is overcome. At the same time, multi-modal feature learning of drug and target branches is performed separately through deep learning methods, local features are fused using MoE (multi-expert architecture), and global fusion is performed using interactive decoders, which effectively improves the prediction accuracy. The concise model structure not only saves computing resources and time, but also enhances interpretability.

[0067] 2. In this invention, the accuracy of drug-target binding affinity prediction is improved by introducing multimodal feature fusion and deep learning model. It performs well in comparison with traditional methods (such as RF, SVM) and advanced deep learning methods (such as DeepDTA, AttentionDTA), and achieves good performance on both Davis and Kiba datasets, with CI (consistency index) and R 2 (Determination coefficient) indicators have reached leading levels and are creative. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 Shown is a flow chart of the application of a multimodal feature fusion model provided by an embodiment of the present invention in drug target binding affinity prediction;

[0069] Figure 2 Shown is a deep learning model diagram of the application of a multimodal feature fusion model in drug target binding affinity prediction provided by an embodiment of the present invention;

[0070] Figure 3 The figure shows a process diagram of constructing a molecular fragment graph using the BRICS algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Figure 1 FIG. 1 is a flow chart showing the application of a multimodal feature fusion model in drug target binding affinity prediction provided by an embodiment of the present invention. Figure 1 As shown, the MoE-Transformer model method based on multimodal feature fusion specifically includes the following steps:

[0073] Step S1, obtaining an original data set of drug-target binding affinity, and using the affinity index in the data set as a label for a training set and a test set;

[0074] In this embodiment, the data set selected two commonly used data sets Davis and Kiba from previous papers and experimental records, which respectively contain 30056 and 118254 interaction data. The specific contents of the two data sets are shown in Table 1.

[0075] Step S2, using the drug sequence and target sequence provided in the original data set to obtain representation information of other modalities, and preprocessing the obtained representation information of drugs and targets of different modalities to process them into feature vector forms acceptable to the model;

[0076] In this embodiment, it is necessary to generate a variety of new multimodal representation information. Specifically, for drug branches, RDKit in the python library is used to operate, and the smiles of the drug molecules are used as input to generate molecular objects, and the molecular objects are converted into the corresponding molecular fingerprint form (ECFP4), thereby obtaining the molecular fingerprint representation of the drug; the BRICS segmentation method in RDKit is used to segment the smiles of the drug molecules, and the segmented parts are used as new nodes. A new edge is used to connect the breakpoints, and RDKit is used to generate a 167-bit MACCS fingerprint code and a 27-bit pharmacophore fingerprint code of the node, and the two codes are connected as the node information of each part. The chemical reaction code during BRICS segmentation is used as the edge information at the breakpoint. The above nodes, node information, edges, and edge information constitute a fragment graph of the drug molecules, thereby obtaining the molecular fragment graph representation of the drug. The specific process is as follows: Figure 3 shown.

[0077] Furthermore, the sequence of the protein target is used as an index to search in the PDB database, and the protein structure with the highest sequence similarity is downloaded. The distance between each residue is calculated using the protein structure. The C-α atom is a key atom used to represent the position of the protein main chain skeleton. In order to simplify the calculation and stabilize the protein structure representation, the C-α atom is used as the center of the protein residue, that is, the distance between C-α atoms is used as the residue distance. Finally, the names of each amino acid are mapped to a dictionary to generate an adjacency matrix of protein residue distance contacts, thereby obtaining a residue contact graph representation of the protein.

[0078] In this embodiment, for drug and protein sequences, since the lengths of the sequences are different, the overly long sequences are truncated, while the shorter sequences are padded with zeros. The sequence length is 512, and then a pre-trained model is used for shallow feature extraction. The pre-trained model is selected from HuggingFace and fine-tuned to adapt to the drug-target binding affinity prediction task.

[0079] Step S3, save the multimodal feature vectors of drugs and targets and affinity labels in the form of pytorchtensor (.pt);

[0080] In this embodiment, three different dictionaries are constructed with drug, target, interaction affinity representation, and label, and stored and compressed in the form of pytorchtensor (.pt).

[0081] Step S4. Load the .pt file, use five-fold cross validation, and input information of different modalities into the deep neural network for training and testing.

[0082] In this example, the pytorchtensor file is loaded, and the values ​​of the drug and protein dictionaries read are assigned to the corresponding variables, namely, the representation of the drug smiles sequence, the drug molecule fragment graph representation, the drug fingerprint representation, the protein sequence representation, and the protein residue contact graph representation;

[0083] In this embodiment, the model is continuously trained and predicted. The learning rate optimizer uses the AdamW optimization algorithm and uses the weight decay strategy to prevent overfitting. The learning rate scheduler uses the cosine annealing algorithm. The loss function uses the root mean square error loss and the loss function in the weighted MoE is expressed as:

[0084]

[0085] loss=loss mse +W moe .loss moe

[0086] Where N is the number of samples, output i is the predicted value of the model, label i is the corresponding true value, W moe is the weight of MoE loss, loss moe is the loss of MoE. After each epoch, the model will enter the verification mode, the parameters will not be updated, and only the loss on the test set will be calculated. The following indicators are used to measure the performance of the model, the consistency coefficient (CI), expressed as:

[0087]

[0088] in, and represents the predicted value of samples i and j, N represents the total number of comparable samples, is an indicator function that takes the value 1 when any of the following is true: i >x j and or x i <x j and (The predicted order is correct); otherwise, the exponential function value is 0.

[0089] MSE (mean square error), expressed as:

[0090]

[0091] R 2 (Coefficient of determination): in is the average of the actual values.

[0092] Furthermore, in addition to the loss function, CI, consistency coefficient, mean square error (MSE) and determination coefficient (R 2 ), the mean absolute error method, Pearson correlation coefficient method and regularization loss can also be used to more comprehensively measure the performance and performance of the model. Specifically, the mean absolute error method is expressed as:

[0093]

[0094] in: is the predicted value of the model, y i is the true value, N is the number of samples;

[0095] MAE is an indicator that measures the mean absolute error between the predicted value and the true value. Compared with MSE, MAE will not amplify larger errors;

[0096] The Pearson correlation coefficient method is an indicator to measure the linear correlation between the true value and the predicted value. The range is [-1,1], 1 indicates a complete positive correlation, and -1 indicates a complete negative correlation. The formula is:

[0097]

[0098] in: and are the average values ​​of the true value and the predicted value respectively. The Pearson correlation coefficient method is simple to calculate, and only needs to calculate the mean, variance, and covariance, which is easy to understand and explain;

[0099] In order to prevent the model from overfitting, a regularization loss term can be added. The most common one is L2 regularization, which is expressed as:

[0100]

[0101] Where λ is the regularization coefficient, is the weight parameter of the model, and M is the sum of the parameters. It limits the complexity of the model by introducing additional penalty terms in the loss function, preventing the model from overfitting the training data, especially in high-dimensional data, and suppressing the model from over-learning the noise of the training data.

[0102] In this embodiment, according to Figure 2 The model structure diagram describes the structure of the model:

[0103] 1) For the smiles sequence and protein sequence of the drug, the operations on the two are similar. First, the embedding operation is performed to convert the high-dimensional features of the sequence into low-dimensional features. At this time, the shape of the sequence feature is (B, L sequence , H), where B represents the batch size, L represents the sequence length, and H represents the feature dimension. Then, the shape of smiles is changed from (B, L sequence , H) changes to (B, H, L sequence ), use linear mapping to compress the sequence length (L), and continue to input the newly obtained smiles sequence representation into the convolutional network. The convolutional network is composed of stacked convolutional layers, with a normalization layer added between blocks, and the final output uses a residual connection. The convolutional layer uses three one-dimensional convolutions, with the same convolution kernel size and padding. The convolution kernel size is 3, the padding is 1, and the step size is 1, 2, and 1 in the stacking order.

[0104] Furthermore, the activation function elu is added between the one-dimensional convolutions, and the final output uses a residual connection. The convolution layer further extracts the sequence features and further adjusts the sequence length (L sequence ) is compressed.

[0105] 2) For the fingerprint representation of the drug, after the embedding operation, it is necessary to add a dimension L to it, changing it from (B, H) to (B, H, L fingeprint ), where L fingerprint The value of is 1.

[0106] 3) For the molecular fragment graph representation of drugs, the representation information is a graph structure containing node features x, edge features edge_attr and edge connection information edge_index. The graph structure data is input into the graph neural network based on message passing. The internal operation of the graph network is as follows. First, the node features and edge features are mapped to the output channel using linear changes, which is expressed as:

[0107] x ′ =W node .x

[0108] e′=W edge .e

[0109] The edge connection information is stored in the form of an adjacency matrix, and the normalized term of the adjacency matrix is ​​calculated as follows:

[0110]

[0111] When passing messages, the feature x of the neighbor node is passed j And the edge feature e′ is spliced ​​and expressed as:

[0112] f fusion＝ concat(x j, e′)

[0113] Then the node and edge features are further fused using a linear layer and multiplied by the normalization coefficient norm, expressed as:

[0114] m=w fusion .f fusion .norm

[0115] Perform mean aggregation on the neighbor messages of the node and add the bias term, which is expressed as:

[0116] h i =mean(m)+bias

[0117] Finally, the node features in the same batch are globally pooled to form the final graph representation, and the output returns an output tensor with a shape of (batch_size, 1, out_channels).

[0118] 4) For the residue contact graph representation of proteins, a two-dimensional convolutional network is used to learn features. The convolutional network module first performs multi-layer convolution operations on the input protein matrix, of which ordinary two-dimensional convolution is two layers and deformable convolution is four layers. The first layer is an ordinary two-dimensional convolution layer, with the number of input channels and output channels both being 1. It uses a 3x3 convolution kernel, a stride of 2, and a padding of 1. The function of this layer is to extract local spatial features and downsample the input protein matrix to reduce its spatial dimension. Next, a 4x4 maximum pooling is performed through the maximum pooling layer to further reduce the size of the features while retaining the most important features. Then, the ELU activation function is used to introduce nonlinearity, giving the model a stronger feature learning ability. In the second layer, the network still uses ordinary two-dimensional convolution, expanding the number of channels from 1 to 64 to capture more abundant features. The convolution kernel size is still 3x3, the stride is 2, and the padding is 1. After convolution, the nonlinear expression of the model is also enhanced by the ELU activation function.

[0119] The next few layers introduce deformable convolution layers. The third layer converts the input 64 channels into 128 channels, and uses deformable convolution to capture the features of irregular areas, with a stride of 2, a convolution kernel size of 3x3, and a padding of 1, further reducing the size of the feature map. Then, maximum pooling is performed to reduce the spatial dimension of the feature map. The number of channels in the fourth layer remains unchanged at 128, with a stride of 1 and a padding of 1. The features of the 128 channels are then normalized, and the ELU activation function is used to improve the nonlinear processing capability.

[0120] The fifth layer increases the number of channels from 128 to 256, with a stride of 2 and a padding of 1, continues to compress the spatial dimension, and performs batch normalization, followed by an ELU activation function.

[0121] The last layer converts the number of channels from 256 to hidden_size (the feature dimension defined in the model), with a stride of 2 and a padding of 1, and continues to extract high-level features. Finally, adaptive average pooling is performed to unify the size of the feature map to 1x1 and output a feature vector of a fixed size of (B, H, 1, 1).

[0122] The residue contact graph representation after processing by the two-dimensional convolutional layer finally needs to be compressed by one dimension and transformed from (B, H, 1, 1) to (B, 1, H) to meet the needs of subsequent fusion.

[0123] 5) For the feature vectors obtained above, the drug and protein branches are fused separately. The fusion strategy is to first connect the multi-modal feature vectors, and the shape after connection is (B, L drug , H) and (B, L protein , H),

[0124] Among them, L drug =L sequence +L graph +L fingerprint , L protein =L sequence +L fingerprint The concatenated feature vector is then input into a multi-head self-attention layer based on a multi-expert architecture. The core operation of multi-head self-attention is as follows: the input x is linearly transformed to obtain the query vector Q, key vector K and value vector V, expressed as:

[0125] Q=XW Q , K = XW K , V = XW V

[0126] Among them, W Q， W K , W V are the weight matrices of linear transformations respectively. The attention weight is calculated by the query vector Q and the key vector K, expressed as:

[0127]

[0128] Among them, d k is the dimension of the query vector or key vector, and softmax is the activation function used to normalize the attention weight. The multi-head sub-attention mechanism divides the query, key, and value vectors into multiple heads, each of which performs an independent attention operation, expressed as:

[0129] MutiHead(Q,K,V)=Concat(head1,...,head h )W O

[0130] Each expert's multi-head self-attention processes the input, and the output of the i-th expert can be expressed as:

[0131] Y i =MSA i (X)

[0132] Among them, MSA i Represents the multi-head self-attention mechanism in the i-th expert. The gating mechanism in the MoE module selects the weights of the two experts through Top2Gating, that is, the two with the highest scores are selected through the score matrix calculated by gating. Assuming that expert i and expert j are selected, the model outputs a weighted combination of the two experts, expressed as:

[0133] Y=g i Y i +g j Y j

[0134] Among them, g i and g j The attention weights calculated by Top2Gating ensure that each input can be processed by the most suitable expert. Finally, the outputs of all experts are weighted and fused to obtain the final output of the model.

[0135] 6) After the two branches are fused by the expert module, the feature vectors of the new drugs and proteins are obtained respectively. The vectors are input into the interactive fusion module, which consists of two Transformer decoders. For the feature vector of the drug, it is expressed as:

[0136]

[0137] For the feature vector of protein, it is expressed as:

[0138]

[0139] Finally, the drug feature vector and protein feature vector are connected to obtain a shape of (B, L drug +L protein , H) of the fused feature vector.

[0140] 7) For the fused feature vector, it is input into the fully connected layer to obtain the final prediction result.

[0141] In this embodiment, the model is continuously trained and tested to obtain optimal model parameters.

[0142] Step 5. Use the trained model to predict drug target binding affinity.

[0143] The effect of the present invention is further described below in conjunction with simulation experiments:

[0144] 1. Simulation experimental conditions

[0145] The hardware platform of the simulation experiment of the present invention is: graphics card Nvidia GeForce RTX 3090, memory 24G. The software platform is the Ubuntu system based on Linux.

[0146] 2. Contents of simulation experiment

[0147] The simulation experiment of the present invention uses the present invention to make a preliminary comparison with the existing technology, and the comparison is made on the drug target binding affinity data sets Davis and Kiba respectively.

[0148] 3. Simulation test results

[0149] The method of the present invention will use the following indicators to measure the performance of the model, the consistency coefficient (CI):

[0150]

[0151] in, and represents the predicted value of samples i and j, N represents the total number of comparable samples, is an indicator function that takes the value 1 when any of the following is true: i >x j and or x i <x j and (The predicted order is correct); otherwise, the exponential function value is 0.

[0152] MSE(Mean Square Error):

[0153]

[0154] R 2 (Coefficient of determination): in is the average of the actual values.

[0155] Table 1 Davis and Kiba datasets

[0156] Dataset name Protein quantity Number of drugs The number of interactions Davis 442 68 30056 KIBA 229 2111 118254

[0157] Table 2 Comparison of indicators of different methods on Davis dataset

[0158] Model / Method Name CI (Consistency Index) MSE (mean square error) <![CDATA[R 2 (coefficient of determination)]]> RF 0.854 0.359 0.549 SVM 0.857 0.383 0.513 SimBoost 0.872 0.282 0.644 DeepDTA 0.878 0.261 0.630 AttetionDTA 0.887 0.245 0.657 AttenionMGT-DTA 0.891 0.193 0.699 DoubleSG-DTA 0.902 0.219 0.725 Method of the present invention 0.902 0.214 0.734

[0159] Table 3 Comparison of indicators obtained by different methods on the Kiba dataset

[0160] Model / Method Name CI (Consistency Index) MSE (mean square error) <![CDATA[R 2 (coefficient of determination)]]> RF 0.837 0.245 0.581 SVM 0.799 0.308 0.513 SimBoost 0.836 0.222 0.629 DeepDTA 0.863 0.194 0.673 AttentionDTA 0.88 0.158 0.742 AttentionMGT-DTA 0.893 0.140- 0.786 DoubleSG-DTA 0.896 0.138 0.787 Method of the present invention 0.899 0.147 0.789

[0161] It can be seen from Table 2 and Table 3 that compared with other methods, CI and R of the method of the present invention are better on both Davis and Kiba. 2 All are the best. For the MSE index, although the present invention is not the best, it is still not far behind the most advanced methods and is in the forefront.

[0162] It can be seen that the application of a multimodal feature fusion model disclosed in this embodiment in the prediction of drug target binding affinity includes: data set collection; data processing; processing sequence features using convolutional layers and Transformer encoders; processing graph features using a graph network based on message passing; processing residue contact matrices using deformable convolutional layers; applying the framework of mixed experts to a multi-head attention mechanism for local fusion of branch features; using a combination of two interacting Transformer decoders for global feature fusion; and using a fully connected layer as the final prediction layer. The prediction performance of the deep learning model constructed by the present invention has greatly surpassed traditional machine learning models, and has also performed well in the latest deep learning methods, providing a high-accuracy solution for drug target binding affinity prediction.

[0163] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. Application of multimodal feature fusion model in drug target binding affinity prediction, characterized by: The following steps are involved: S1. Obtain the original data set of drug target binding affinity, and use the binding affinity index in the data set as the label of the training set and the test set; S2. Use the drug sequence and target sequence provided in the original data set to obtain the representation information of other modalities, and preprocess the obtained representation information of drugs and targets of different modalities to process them into feature vector forms acceptable to the model; S3. Save the multimodal feature vectors of drugs and targets and the binding affinity labels in the form of pytorchtensor (.pt); S4. Load the .pt file and input the information of different modalities into the deep neural network for training and testing.

2. The application of the multimodal feature fusion model according to claim 1 in drug target binding affinity prediction is characterized by: The step S2 includes the following steps: S2.1: Use RDKit in the python library to generate molecular objects with smiles of drug molecules as input, and then convert the molecular objects into the corresponding molecular fingerprint form (ECFP4); S2.2: Use the BRICS segmentation method in RDKit to segment the smiles of the drug molecule. The segmented parts are used as new nodes. A new edge is used to connect the breakpoints. RDKit is used to generate the 167-bit MACCS fingerprint code and 27-bit pharmacophore fingerprint code of the node. The two codes are connected as the node information of each part. The chemical reaction code during BRICS segmentation is used as the edge information at the breakpoint. The above nodes, node information, edges, and edge information constitute the fragment graph of the drug molecule. S2.3: Use the sequence of the protein target as an index to search in the PDB database, download the protein structure with the highest sequence similarity, and use the protein structure to calculate the distance between each residue. The C-α atom is a key atom used to represent the position of the protein main chain skeleton. In order to simplify the calculation and stabilize the protein structure representation, the C-α atom is used as the center of the protein residue, that is, the distance between C-α atoms is used as the residue distance. Finally, the names of each amino acid are mapped to a dictionary to generate an adjacency matrix of protein residue distance contacts; S2.4: For drug smiles strings and protein sequences, since the lengths of each sequence are different, the overly long sequences are truncated, while the shorter sequences are padded with zeros. The sequence length is 512, and the shallow features of the sequence are extracted using a pre-trained model based on Bert.

3. The application of the multimodal feature fusion model according to claim 1 in drug target binding affinity prediction is characterized by: The step S4 includes the following steps: S4.1: Load the pytorchtensor file and assign the read values ​​of the drug smiles and protein protein dictionaries to the corresponding variables, namely, the representation of the drug smiles sequence, the drug molecule fragment graph, the drug fingerprint, the protein sequence, and the protein residue contact graph; S4.2: Input the representation information into the model to obtain the prediction results; S4.3: The model is continuously trained and predicted. The learning rate optimizer uses the AdamW optimization algorithm and the weight decay strategy is used to prevent overfitting. The learning rate scheduler uses the cosine annealing algorithm. The loss function uses the root mean square error loss and the loss function in the weighted MoE. The formula is as follows: loss=loss mse +W moe .loss moe Where N is the number of samples, output i is the predicted value of the model, label i is the corresponding true value, W moe is the weight of MoE loss, loss moe is the loss of MoE. After each epoch, the model will enter the verification mode, and the parameters will not be updated. Only the loss on the test set is calculated. The following indicators are used to measure the performance of the model. The consistency coefficient (CI) is expressed as follows: in, and represents the predicted value of samples i and j, N represents the total number of comparable samples, ∑I is an indicator function that takes the value 1 when any of the following is true: i >x j and or x i <x j and (The prediction order is correct), otherwise, the exponential function value is 0, MSE (mean square error), the expression formula is as follows: R 2 (Coefficient of determination): in is the average of the actual values.

4. The application of the multimodal feature fusion model in drug target binding affinity prediction according to claim 3, characterized in that: The step S4.2 includes the following steps: S4.2.1: For the smiles sequence and protein sequence of the drug, the operations on the two are similar. First, the embedding operation is performed to convert the high-dimensional features of the sequence into low-dimensional features. At this time, the shape of the sequence feature is (B, L sequence , H), where B represents the batch size, L represents the sequence length, and H represents the feature dimension. Then, the shape of smiles is changed from (B, L sequence , H) changes to (B, H, L sequence ), use linear mapping to compress the sequence length (L), and continue to input the newly obtained smiles sequence representation into the convolutional network. The convolutional network is composed of stacked convolutional layers, and a normalization layer is added between blocks. The final output uses a residual connection. The convolutional layer further compresses the sequence length (L) while extracting sequence features. sequence ) was compressed; S4.2.2: For the fingerprint representation of the drug, after the embedding operation, it is necessary to add a dimension L to it, changing it from (B, H) to (B, H, L fingeprint ), where L fingerprint The value of is 1.

5. The application of the multimodal feature fusion model in drug target binding affinity prediction according to claim 1, characterized in that: The step S4.2 also includes S4.2.3: for the molecular fragment graph representation of the drug, the representation information is a graph structure including node feature x, edge feature edge_attr and edge connection information edge_index, and the graph structure data is input into the graph neural network based on message passing. The internal operation of the graph network is as follows: first, the node features and edge features are mapped to the output channel using linear changes, which is expressed as: x′=W node .x e′=W edge .have been The edge connection information is stored in the form of an adjacency matrix, and the normalized term of the adjacency matrix is ​​calculated as follows: When passing messages, the feature x of the neighbor node is passed j And the edge feature e′ is spliced ​​and expressed as: f fusion＝ concat(x j ,e′) Then the node and edge features are further fused using a linear layer and multiplied by the normalization coefficient norm, expressed as: m=w fusion .f fusion .norm Perform mean aggregation on the neighbor messages of the node and add the bias term, which is expressed as: h i =mean(m)+bias Finally, the node features in the same batch are globally pooled to form the final graph representation, and the output returns an output tensor with a shape of (batch_size, 1, out_channels); S4.2.4: For the residue contact map representation of proteins, a two-dimensional convolutional network is used for feature learning. The convolutional network module first performs multi-layer convolution operations on the input protein matrix, where ordinary two-dimensional convolutions are two layers and deformable convolutions are four layers. The residue contact map representation processed by the two-dimensional convolution layer finally needs to compress one dimension and make a transformation from (B, H, 1, 1) to (B, 1, H) to meet the needs of subsequent fusion.

6. The application of the multimodal feature fusion model according to claim 1 in drug target binding affinity prediction is characterized by: The S4.5 also includes S4.2.5: For the feature vectors obtained above, the drug and protein branches are fused respectively. The fusion strategy is to first connect the multi-modal feature vectors. The shape after connection is (B, L drug , H) and (B, L protein , H), where L drug =L sequence +L graph +L fingerprint , L protein =L sequence +L fingerprint , and then input the concatenated feature vector into a multi-head self-attention layer based on a multi-expert architecture. The core operation of multi-head self-attention is as follows: linear transformation is performed on the input x to obtain the query vector Q, key vector K and value vector V, which are expressed as: Q=XW Q ,K=XW K ,V=XW V Among them, W Q， W K , W V are the weight matrices of linear transformations, respectively. The attention weights are calculated by the query vector Q and the key vector K, expressed as: Among them, d k is the dimension of the query vector or key vector, softmax is the activation function used to normalize the attention weight, and the multi-head sub-attention mechanism divides the query, key, and value vectors into multiple heads, each of which performs an independent attention operation, expressed as: MutiHead(Q,K,V)=Concat(head1,…,head h )W O Each expert's multi-head self-attention processes the input, and the output of the i-th expert can be expressed as: Y i =MSA i (X) Among them, MSA i It represents the multi-head self-attention mechanism in the i-th expert. The gating mechanism in the MoE module selects the weights of the two experts through Top2Gating, that is, the two with the highest scores are selected through the score matrix calculated by the gating. Assuming that expert i and expert j are selected, the model outputs a weighted combination of the two experts, which is expressed as: Y=g i AND i +g j AND j Among them, g i and g j The attention weights calculated by Top2Gating ensure that each input can be processed by the most suitable expert. Finally, the outputs of all experts are weighted and fused to obtain the final output of the model. S4.2.6: After the two branches are fused by the expert module, the feature vectors of the new drugs and proteins are obtained respectively. The vectors are input into the interactive fusion module, which consists of two Transformer decoders. For the feature vector of the drug, the expression formula is as follows: For the characteristic vector of protein, the expression formula is as follows: Finally, the drug feature vector and protein feature vector are connected to obtain a shape of (B, L drug +L protein , H) of the fused feature vector.

7. The application of the multimodal feature fusion model in drug target binding affinity prediction according to claim 6, characterized in that: The S4.2 also includes S4.2.7: for the fused feature vector, input it into the fully connected layer to obtain the final prediction result.

Images