A DTI prediction method, system, storage medium and device based on reinforcement learning

Abstract

The present invention relates to a DTI prediction method, system, storage medium and device based on reinforcement learning. It mainly includes S100, establishing a drug-target interaction matrix Y train , establishing a chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establishing a target similarity matrix: SIM strtar , SIM costar , SIM Jactar , S200, performing a fusion operation on SIM strdrug , SIM cosdrug and SIM Jacdrug through the Q-learning algorithm, and performing a fusion operation on SIM strtar , SIM costar and SIM Jactar to obtain a SIM drug matrix and a SIM tar matrix. S300, inputting the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tar matrix into n models for training respectively, where n is greater than or equal to 2, obtaining n prediction score matrices respectively, initializing the weight values of the n prediction score matrices, using the Q-learning algorithm to optimize each weight value, obtaining the optimal weight value, and performing linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F. The method of this application has a relatively high prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of DTI prediction, and in particular to a DTI prediction method, system, storage medium, and device based on reinforcement learning. Background Art

[0002] Drug-target interaction (DTI) refers to the binding of a drug to a target site, resulting in a change in its behavior or function. Currently, there are various DTI prediction methods, but the accuracy of different models for predicting different datasets is different. Traditional DTI predictions all use a single model for prediction, so the prediction accuracy is relatively low. Summary of the Invention

[0003] Based on this, in order to address the problem of relatively low prediction accuracy, a DTI prediction method based on reinforcement learning is provided.

[0004] A DTI prediction method based on reinforcement learning includes:

[0005] S100. Establish a drug-target interaction matrix Y train , establish a chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establish a target similarity matrix: SIM strtar , SIM costar , SIM Jactar ,

[0006] S200. Perform a fusion operation on SIM strdrug , SIM cosdrug and SIM Jacdrug through the Q-learning algorithm, and perform a fusion operation on SIM strtar , SIM costar and SIM Jactar to obtain a SIM drug matrix and a SIM tar matrix.

[0007] S300. Input the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tar matrix into n models for training respectively, where n is greater than or equal to 2, obtain n prediction score matrices respectively, initialize the weight values of the n prediction score matrices, optimize each weight value using the Q-learning algorithm, obtain the optimal weight value, and perform linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F.

[0008] In one embodiment,

[0009] In step S200, the fusion operation on SIM strdrug , SIM cosdrug , SIM Jacdrug and the fusion operation on SIM strtar , SIM costar , SIM Jactar are performed to obtain the SIM drug matrix and the SIM tar matrix, specifically including:

[0010] Input the Y train matrix, the chemical structure similarity matrix SIM of the drug compound strdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strtar , SIM costar and SIM Jactar into the NRLMF model, and initial weight values are respectively assigned to the chemical structure similarity matrix SIM of the drug compound strdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strtar , SIM costar and SIM Jactar . The weight values are continuously optimized through the Q-learning algorithm, and finally the optimal weight values are obtained. Based on the optimal weight values, linear weighted operations are performed on SIM strdrug , SIM cosdrug and SIM Jacdrug to obtain the finally fused SIM drug matrix. Based on the optimal weight values, linear weighted operations are performed on SIM strtar , SIM costar and SIM Jactar to obtain the SIM tar matrix.

[0011] In one embodiment,

[0012] In step S300, the n models include NRLMF, CMF, BLM-NII, NetLapRLS, and WNN-GIP.

[0013] In one embodiment,

[0014] In step S100, the Tanimoto coefficient, cosine similarity, and Jaccard similarity coefficient are respectively used to calculate two drug compounds d i and d jThe chemical structure similarity between them is used to establish a chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug .

[0015] In one embodiment,

[0016] In step S100, the normalized Smith-Waterman score, cosine similarity, and Jaccard similarity coefficient are respectively used to calculate the sequence similarity between proteins t i and t j , and then a target similarity matrix is established: SIM strtar , SIM costar , SIM Jactar .

[0017] A DTI prediction system based on reinforcement learning, comprising:

[0018] A data acquisition module for acquiring data;

[0019] A data processing module for performing the following operations based on the data acquired by the data acquisition module:

[0020] S100. Establish a drug-target interaction matrix Y train , establish a chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establish a target similarity matrix: SIM strtar , SIM costar , SIM Jactar ,

[0021] S200. Perform a fusion operation on SIM strdrug , SIM cosdrug and SIM Jacdrug , and perform a fusion operation on SIM strtar , SIM costar and SIM Jactar to obtain the SIM drug matrix and the SIM tar matrix,

[0022] S300. Combine the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tarThe matrix is respectively input into n models for training, where n is greater than or equal to 2, and n prediction score matrices are respectively obtained. The weight values of the n prediction score matrices are initialized, and the Q-learning algorithm is used to optimize each weight value to obtain the optimal weight value. Based on the optimal weight value, the n prediction score matrices are linearly weighted to obtain the final prediction score matrix F.

[0023] A computer storage medium stores at least one executable instruction, and the executable instruction causes a processor to perform operations corresponding to the DTI prediction method based on reinforcement learning.

[0024] A computer device includes: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the DTI prediction method based on reinforcement learning.

[0025] Advantages of the present application:

[0026] 1) This method does not need to rely on the structure of proteins and the number of ligands;

[0027] 2) This method integrates three types of heterogeneous data, ensuring the comprehensiveness of the data;

[0028] 3) This method combines different models together, solving the problem that different models have different prediction accuracies for different data sets;

[0029] 4) This method performs model fusion on the basis of data fusion, significantly improving the prediction accuracy. Description of the Drawings

[0030] Figure 1 It is a flowchart of an embodiment of the DTI prediction method based on reinforcement learning of the present application. Detailed Embodiments

[0031] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following describes the detailed embodiments of the present invention with reference to the accompanying drawings. Many specific details are set forth in the following description to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0032] It should be noted that when an element is referred to as "fixed to" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0034] As Figure 1 shown, the embodiments of the present application provide a DTI prediction method based on reinforcement learning, and the method includes:

[0035] S100. Establish a drug-target interaction matrix Y train , establish a chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establish a target similarity matrix: SIM strtar , SIM costar , SIM Ja c tar ,

[0036] S200. Perform a fusion operation on SIM strdrug , SIM cosdrug and SIM Jacdrug through the Q-learning algorithm and perform a fusion operation on SIM strtar , SIM costar and SIM Jactar to obtain a SIM drug matrix and a SIM tar matrix.

[0037] S300. Input the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tar matrix into n models for training respectively, where n is greater than or equal to 2, obtain n prediction score matrices respectively, initialize the weight values for the n prediction score matrices, use the Q-learning algorithm to optimize each weight value, obtain the optimal weight value, and perform linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F.

[0038] It should be noted that step S200 of the present application is to fuse data, and step S300 is to fuse models.

[0039] It should be noted that this application can use a variety of data sets for training. For example, protein-coupled receptor (GPCR) data can be used as a data set for training each module in the present invention. Define represents the drug set, represents the target set, where m and n represent the number of drugs and the number of targets respectively. Y ∈ R m×n represents the interaction matrix between drugs and targets, which is composed of drug d i (1 ≤ i ≤ m) and target t j (1 ≤ j ≤ n). The definition of the Y matrix is as follows:

[0040]

[0041] Specifically, each drug-target pair in the Y matrix can be used as a sample. All drug-target pairs in the Y matrix are evenly divided into five parts by random seed shuffling. Then, one sample is selected as the test set in turn, and the remaining four samples are used as the training set. It can also be other ratios. At the same time, all samples corresponding to the subscripts in the Y matrix in the test set are obtained, and all corresponding positions are assigned 0. In this way, a new drug-target interaction matrix Y train ∈ R m×n is obtained, and the purpose is to predict these unknown interaction pairs marked as 0 by the method of this application.

[0042] In one embodiment, in step S100, the chemical structure similarity between two drug compounds d i and d j is calculated respectively through the Tanimoto coefficient, cosine similarity, and Jaccard similarity coefficient, and then a chemical structure similarity matrix of drug compounds is established: SIM stdrug , SIM cosdrug , SIM Jacdrug .

[0043] In one embodiment, in step S100, the sequence similarity between proteins t i and t j is calculated respectively using the normalized Smith-Waterman score, cosine similarity, and Jaccard similarity coefficient, and then a target similarity matrix is established: SIM strtar , SIM costar , SIM Jactar .

[0044] In one embodiment, in step S200, by using the Q-learning algorithm for SIM strdrug , SIM cosdrug , SIM JacdrugPerform the fusion operation and for SIM strtarr ,SIM costar ,SIM Jactar Perform the fusion operation to obtain SIM drug matrix and SIM tar matrix, specifically including:

[0045] Input the Y train matrix, the chemical structure similarity matrix SIM strdrug of the drug compound, SIM cosdrug and SIM Jacdrug , as well as the target similarity matrix SIM strtar , SIM costar and SIM Jactar into the NRLMF model, and assign initial weight values to the chemical structure similarity matrix SIM strdrug of the drug compound, SIM cosdrug and SIM Jacdrug , as well as the target similarity matrix SIM strtar , SIM costar and SIM Jactar respectively. Continuously optimize the weight values through the Q - learning algorithm, and finally obtain the optimal weight values. Based on the optimal weight values, perform a linear weighted operation on SIM strdrug , SIM cosdrug and SIM Jacdrug to obtain the finally fused SIM drug matrix. Based on the optimal weight values, perform a linear weighted operation on SIM strtar , SIM costar and SIM Jactar to obtain the SIM tar matrix.

[0046] In one embodiment, in step S300, the n models include NRLMF, CMF, BLM - NII, NetLapRLS, and WNN - GIP, that is, the above - mentioned 5 models are used in step S300 of the present application. Using the above - mentioned 5 models in the present application can significantly improve the prediction accuracy. Specifically, see Tables 1, 2, and 3. It should be noted that after obtaining the final prediction score matrix F through the above - mentioned method in the present application, by comparing the prediction score matrix F with the initial Y matrix, the AUC value (Area Under Curve) or AUPR value (area under the precision - recall curve) is calculated. Among them, for example, the initial Y matrix is a 10 - by - 10 interaction matrix. Set all the values in the first row of the Y matrix to zero to obtain the Y train matrix, and then through this Y trainThe matrix and the above method of the present application obtain an F matrix. Compare all the predicted values in the first row of the F matrix with the initial Y matrix, and calculate the AUC value and the AUPR value.

[0047] NR, GPCR, IC, and E in Table 1 represent different data sets. Among them, NR - nuclear receptor (NR), GPCR - G protein coupled receptor (GPCR), IC - ion channel (IC), E - enzyme (E). QLDTI in Table 1 is the test result obtained by the present application using the Q-learning algorithm for data fusion and fusing five models using the Q-learning algorithm. Other test results are predictions made only through a single model.

[0048] Table 1

[0049]

[0050] QLDTI in Table 2 is the test result obtained by the present application using the Q-learning algorithm for data fusion and fusing five models using the Q-learning algorithm. Other test results are predictions made using a single model after data fusion using the Q-learning algorithm.

[0051] Table 2

[0052]

[0053]

[0054] In Table 3, QLDTI is the test result obtained by the present application using the Q-learning algorithm for data fusion and fusing five models using the Q-learning algorithm. Other test results are the results of testing after fusing multiple models using only the Q-learning algorithm without data fusion.

[0055] Table 3

[0056]

[0057]

[0058]

[0059]

[0060] The above method of the present application will be introduced in detail through specific embodiments below.

[0061] Step 1:

[0062] Calculate the chemical structure similarity between two drug compounds di and dj through the Tanimoto coefficient, cosine similarity, and Jaccard similarity coefficient to obtain the chemical structure similarity matrix SIM of the drug compounds strdrug , SIM cosdrug , SIM Jacdrug , where:

[0063]

[0064]

[0065]

[0066] Calculate the sequence similarity between two proteins ti and tj through the normalized Smith-Waterman score, cosine similarity, and Jaccard similarity coefficient to obtain the target similarity matrix SIM strtar , SIM costar , SIM Jactar , where:

[0067]

[0068]

[0069]

[0070] Step 2:

[0071] Model the DTI data fusion problem as an MDP model and construct the state, action, and immediate reward function:

[0072] 1) State, denoted by s. Take the weights in front of the drug and target similarity matrices as the state, that is, S(α, β, γ, x, y, z), where α, β, γ, x, y, z are the weights respectively;

[0073] 2) Action, denoted by a. Set the change value of the weights as the action, and the sum of the weights is 1;

[0074] 3) Reward function, denoted by r. Take the AUC value of the output result of each training as the reward value. If the weight value is not between 0 and 1, the reward value is 0.

[0075] Step 3:

[0076] Use the Q-learning algorithm for data fusion:

[0077] 1) Initialize the Q-table;

[0078] 2) Initialize the weights of the drug space and the target space;

[0079] 3) Q-learning uses the ε-greedy policy to select an action and outputs the action a;

[0080] 4) Take the action a and transfer to the next state;

[0081] 5) Calculate and obtain the fused matrix:

[0082] SIM drug = αSIM strdrug + βSIM cosdrug + γ SIM Jacdrug ,

[0083] SIM tar = xSIM strtar + ySIM costar + zSIM Jactar ,

[0084] 6) Obtain the reward r from the AUC value;

[0085] 7) Update the Q-table using the formula Q(s, a) ← Q(s, a) + α[r + γmax a′ Q(s′, a′) - Q(s, a)];

[0086] 8) Until the Q-table converges;

[0087] 9) Calculate the final fused drug-target similarity matrix.

[0088] Step Four:

[0089] Model the DTI model fusion problem as an MDP model and construct the state, action, and immediate reward functions:

[0090] 1) State, denoted by s. Take the weights in front of the model prediction score matrix as the state. For example, S(a, b, c, d, e), where a, b, c, d, e are weights;

[0091] 2) Action, denoted by a. Set the change value of the weights as the action, and the sum of the weights is 1;

[0092] 3) Reward function, denoted by r. Take the difference between the AUC value of each result and the previous one as the reward value. If the difference is greater than 0, give a reward with a value of 1, otherwise give a reward with a value of -1.

[0093] Step Five:

[0094] Model fusion using the Q-learning algorithm:

[0095] 1) Use the obtained drug-target similarity matrix as the input of the model;

[0096] 2) Train different models separately and extract their respective prediction score matrices;

[0097] 3) Initialize the Q-table;

[0098] 4) Initialize the weights of the prediction score matrix;

[0099] 5) Q-learning uses the ∈-greedy strategy to select an action and outputs the action a;

[0100] 6) Take the action a and transfer to the next state;

[0101] 7) Calculate and obtain the fused matrix, F = a*F1 + b*F2 + c*F3 + d*F4 + e*F5, where F1 represents the first model, F2 represents the second model, F3 represents the third model, F4 represents the fourth model, and F5 represents the fifth model. Here, 5 models are used;

[0102] 8) Obtain the reward r from the AUC difference;

[0103] 9) Update the Q-table using the formula Q(s, a) ← Q(s, a) + α[r + γmax a′ Q(s′, a′) - Q(s, a)];

[0104] 10) Until the Q-table converges;

[0105] Calculate the finally fused model.

[0106] This application also provides a DTI prediction system based on reinforcement learning, specifically including:

[0107] A data acquisition module, which is used to acquire data;

[0108] A data processing module, which is used to perform the following operations based on the data acquired by the data acquisition module:

[0109] S100. Establish a drug-target interaction matrix Y train , establish a chemical structure similarity matrix of drug compounds: SIM stdrug , SIM cosdrug , SIM Jacdrug , establish a target similarity matrix: SIM strtar , SIM costar , SIM Jactar ,

[0110] S200. Perform a fusion operation on SIM through the Q - learning algorithm strdrug SIM cosdrug and SIM Jacdrug and perform a fusion operation on SIM strtar SIM costar and SIM Jactar to obtain the SIM drug matrix and the SIM tar matrix.

[0111] S300. Input the SIM drug matrix, the drug - target interaction matrix Y train and the SI Mtar matrix into n models for training respectively, where n is greater than or equal to 2, obtain n prediction score matrices respectively, initialize the weight values for the n prediction score matrices, use the Q - learning algorithm to optimize each weight value, obtain the optimal weight value, and perform linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F.

[0112] This application also provides a computer storage medium, in which at least one executable instruction is stored, and the executable instruction enables the processor to perform the operations corresponding to the above - mentioned reinforcement - learning - based DTI prediction method.

[0113] This application also provides a computer device, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operations corresponding to the above - mentioned reinforcement - learning - based DTI prediction method.

[0114] The technical features of the above - mentioned embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above - mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0115] The above - mentioned embodiments only represent several implementation manners of the present invention. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent should be subject to the appended claims.

Claims

1. A DTI prediction method based on reinforcement learning, characterized in that, Including: S100. Establish the drug-target interaction matrix Y train , establish the chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establish the target similarity matrix: SIM strtar , SIM costar , SIM Jactar , Calculate the chemical structure similarity between two drug compounds d i and d j respectively through the Tanimoto coefficient, cosine similarity, and Jaccard similarity coefficient, and then establish a chemical structure similarity matrix for drug compounds: SIM strdrug ,SIM cosdrug ,SIM Jacdrug , Calculate the sequence similarity between proteins t i and t j using the normalized Smith-Waterman score, cosine similarity, and Jaccard similarity coefficient, respectively, and then establish a target similarity matrix: SIM strtar SIM costar SIM Jactar , S200. Perform fusion operations on SIM strdrug SIM cosdrug and SIM Jacdrug and perform fusion operations on SIM strtar SIM costar and SIM Jactar to obtain the SIM drug matrix and the SIM tar matrix, specifically including: Take Y train matrix, the chemical structure similarity matrix SIM of drug compounds strdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strtar , SIM costar and SIM Jactar are input into the NRLMF model. For the chemical structure similarity matrix SIM of drug compounds strdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strtar , SIM costar and SIM Jactar are respectively assigned initial weight values, and the weight values are continuously optimized through the Q-learning algorithm. Finally, the optimal weight values are obtained. Based on the optimal weight values, linear weighting operations are performed on SIM strdrug , SIM cosdrug and SIM Jacdrug to obtain the finally fused SIM drug matrix. Based on the optimal weight values, linear weighting operations are performed on SIM strtar , SIM costar and SIM Jactar to obtain the SIM tar matrix. S300. Input the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tar matrix into n models for training respectively, where n is greater than or equal to 2, obtain n prediction score matrices respectively, initialize the weight values for the n prediction score matrices, use the Q-learning algorithm to optimize each weight value, obtain the optimal weight value, and perform linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F.

2. The DTI prediction method based on reinforcement learning according to claim 1, wherein In step S300, the n models include NRLMF, CMF, BLM-NII, NetLapRLS, and WNN-GIP.

3. A DTI prediction system based on reinforcement learning, characterized in that, Including: A data acquisition module, which is used to acquire data; A data processing module, which is used to perform the following operations based on the data acquired by the data acquisition module: S100. Establish the drug-target interaction matrix Y train , establish the chemical structure similarity matrix of drug compounds: SIM strdrug , SIM cosdrug , SIM Jacdrug , establish the target similarity matrix: SIM strtar , SIM costar , SIM Jactar , Calculate the chemical structure similarity between two drug compounds d i and d j respectively through the Tanimoto coefficient, cosine similarity, and Jaccard similarity coefficient, and then establish a chemical structure similarity matrix of drug compounds: SIM strdrug ,SIM cosdrug ,SIM Jacdrug , Calculate the sequence similarity between proteins t i and t j using the normalized Smith-Waterman score, cosine similarity, and Jaccard similarity coefficient, respectively, and then establish a target similarity matrix: SIM strtar ,SIM costar ,SIM Jactar , S200. Perform fusion operations on SIM strdrug SIM cosdrug and SIM Jacdrug and perform fusion operations on SIM strtar SIM costar and SIM Jactar to obtain the SIM drug matrix and the SIM tar matrix, specifically including: Input Y train matrix, the chemical structure similarity matrix SIM of drug compounds strdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strtar , SIM costar and SIM Jactar are input into the NRLMF model. For the chemical structure similarity matrix SIM of drug compounds stcrdrug , SIM cosdrug and SIM Jacdrug , and the target similarity matrix SIM strrtar、 SIM costar and SIM Jactar are respectively assigned initial weight values, and the weight values are continuously optimized through the Q-learning algorithm. Finally, the optimal weight values are obtained. Based on the optimal weight values, linear weighting operations are performed on SIM strdrug , SIM cosdrug and SIM Jacdrug to obtain the finally fused SIM drug matrix. Based on the optimal weight values, linear weighting operations are performed on SIM strtar , SIM costar and SIM Jactar to obtain the SIM tar matrix. S300. Input the SIM drug matrix, the drug-target interaction matrix Y train and the SIM tar matrix into n models for training respectively, where n is greater than or equal to 2, and obtain n prediction score matrices respectively. Initialize the weight values for the n prediction score matrices, use the Q-learning algorithm to optimize each weight value, obtain the optimal weight value, and perform linear weighting on the n prediction score matrices based on the optimal weight value to obtain the final prediction score matrix F.

4. A computer storage medium, in which at least one executable instruction is stored, and the executable instruction causes the processor to perform the operations corresponding to the reinforcement learning-based DTI prediction method according to claim 1 or 2.

5. A computer device, comprising: A processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform the operations corresponding to the reinforcement learning-based DTI prediction method according to claim 1 or 2.

Images