Drug target combination degree regression prediction method based on MolrProtTrans model

Abstract

The invention discloses a drug target combination degree regression prediction method based on a MolrProtTrans model, and the method is based on a baseline model MolrProtTrans, and constructs a simplified model. In the regression prediction task of the drug target combination degree, the baseline model and the corresponding simplified model are also used. In order to enrich the model, on the basis of a baseline model transformer framework, a representation model of various drugs and proteins is provided, and a new representation model can be very conveniently added into the framework. According to the method, a network structure is established to solve the problem of regression prediction of the combination degree related to drug molecules and protein, a local module is improved to strengthen the performance, and a new theoretical basis is provided for designing high-quality drugs which can be effectively combined with target spots, are moderate in stability and are beneficial to human body absorption.

Description

Technical Field

[0001] The present invention belongs to the technical field of drug research and development, and in particular relates to a drug target binding degree regression prediction method based on a MolrProtTrans model. Background Art

[0002] After a new drug is successfully developed, it generally needs to undergo a large number of clinical trials before it can be put into use. The development of modern software technology and its increasingly mature application in biomedicine have made virtual binding experiments between new drugs and target proteins possible. The binding degree calculated by software simulation can preliminarily determine the effectiveness of new drugs and whether there are serious side effects, and guide developers to evaluate the properties of drugs. As the scale and accuracy of the model increase, its reliability and interpretability make it truly applicable to actual medical industrial environments.

[0003] The mechanism of drug target binding is that the drug binds to macromolecules in the body, including receptors, and promotes or inhibits them, ultimately enhancing the operation of immune cells and systems in the body. These macromolecules are called "targets" in drug research and development tasks. And drugs produce various molecular interactions with targets by designing corresponding structures. These include hydrogen bonds, ionic bonds, and van der Waals bonds, which enable drug molecules to be embedded in the pockets of target macromolecules. A series of biochemical reactions after binding to the pockets change their enzyme activity and signal transduction ability, promote immune cell proliferation, and cause apoptosis of lesion cells, thus achieving the purpose of treating diseases.

[0004] In the field of drug molecules, DeepDTA only uses 1D features to represent molecules and proteins without using 3D information, and uses convolutional layers to further extract feature information. GraphDTA uses graphs to represent molecular information and uses graph neural networks for processing. Researchers used DeepChem and Rdkit software as extractors to convert molecules into graph structures. An Jingjie used a bipartite graph model to model molecules and proteins, and solved the problem of negative sample sampling by balancing the data set. Meng Zhichang collected molecular fingerprints and used Tanimoto as a similarity indicator based on the drug similarity law and GBA principle. In the field of proteins, Liu, Senior, Yang and others first proposed the use of co-evolutionary protein classification to assist protein structure prediction, and starting from Adhikari's research, multiple sequence alignment MSA has become one of the main features of protein structure. The well-known alphafold2 then combined MSA with the Transformer model with an attention mechanism. The attention mechanism of the Transformer just corresponds to the correlation of a cluster of protein groups in MSA. The model achieved over 95% accuracy on the CASP test set, which fully demonstrates that the rational fusion of MSA and deep learning models is the best way to understand protein coding. Summary of the invention

[0005] 1. Pre-train the drug molecule and protein representation network of the MolrProtTrans model, connect the synthesized vectors to each machine learning model, and build a hybrid structure model;

[0006] 2. MolrProtTrans operation and simplification. Change the model training loss, cancel the activation layer of the last layer, and use MolrProtTrans to complete the regression task of drug target prediction;

[0007] Third, use various other molecular and protein representation models as extractors to build a comprehensive drug-target binding degree regression prediction model.

[0008] Beneficial effects of the present invention:

[0009] The present invention uses MolrProtTrans as a baseline model and designs a comprehensive model that can select multiple molecular and protein feature extraction models. The regression task requires the model to predict a clear numerical binding degree between the drug and the target, and the real data of this indicator can be measured experimentally.

[0010] 1. It provides a rich library of molecular and protein feature representations, including molecular representation models such as KCL and Grover, and protein models such as prose and ESM, from which a comprehensive model for classification prediction can be selected.

[0011] 2. Experiments have shown that the performance of the baseline model is strong and outperforms the simplified model in all aspects, indicating that the selected molecular and protein representation modules and the designed dual attention mechanism have important contributions to the overall performance.

[0012] 3. Promoted the virtual screening and structural optimization process of new drugs, and provided information support for the biomedical industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is the technical roadmap of the present invention.

[0014] Figure 2 It is the KCL model structure in the present invention.

[0015] Figure 3 This is the fitting result of MolrProtTrans and the basic model in the present invention.

[0016] Figure 4 It is the training loss of various combination methods in the present invention on Davis.

[0017] Figure 5This is the fitting effect of KCL and CNN on Kiba in the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] Step 1: Pre-train the drug molecule representation network and protein representation network of the MolrProtTrans model.

[0020] The vector representation obtained by combining the Transformer architecture is described, a dual attention mechanism is designed, and the MolrProtTrabs model is trained and evaluated.

[0021] The drug and target vectors obtained by MolrProtTrans are directly spliced ​​and connected to various machine learning models to construct a hybrid structure model.

[0022] Step 2: Use MolrProtTrans to perform regression tasks. The regression task converts labels from logical binary classifications of 0 or 1 to floating point numbers, using specific numerical values ​​to describe the degree of binding between the drug and the target. The datasets selected are GPCR, Kinase, human, BindingDB, Davis, and Kiba.

[0023] When the MolrProtTrans model is used for regression tasks, the MolR network needs to be trained separately, and the parameters of the protein models ProtTrans and prose need to be loaded. Then, the transformer structure is connected for feature fusion, and finally, a decimal is obtained through linear layer processing as the binding degree value predicted by the model.

[0024] The Brenda and Rhea datasets were used when training the Molr network.

[0025] In terms of model construction, the dgl library provides graph convolutional networks GraphConv, graph convolutional networks with attention GATConv, and other more complex variants SAGEConv, SGConv, TAGConv, etc. GraphConv is used as the model backbone network, and the structure of the model's overlay network layer is set to 2 layers.

[0026] The molecules in the data set are first converted into the mol molecular graph format using the rdkit function, and then converted back into the smiles format using rdkit to obtain the standard format.

[0027] Before using the Molr model to train data, it is necessary to obtain an overview of the attributes of all molecules in the entire data set. The attribute list is represented by a dictionary, and the dictionary is recorded as a feature encoder.

[0028] The regression model modifies the output interface based on the original classification model, modifies the probabilities of the two categories into a numerical output, and deletes the activation function of the last layer so that the numerical value is not limited to between 0 and 1.

[0029] The final output of the main model of the classification task is not [64,2], but [64,1], which means that the probability p of each prediction result is 1, and the probability of the remaining predictions being 0 can be expressed as 1-p. Therefore, it is only necessary to remove the softmax activation of the last layer of the classification MolrProtTrans model and cancel the range limit of the result value.

[0030] Step 3: Aggregate the binding prediction network of multiple models. The molecular representation models used include KCL, Grover, and Mpnn, and the protein representation models include ESM and CNN. The output vector shape of all feature extraction models is [64,1024].

[0031] The molecular information model provides MPNN, Molr, KCL, grover, etc.

[0032] The protein information model provides CNN, prose, ProtTrans, ESM, etc.

[0033] The respective information extractor models of the molecules and proteins are specified by parameters, and a corresponding set is selected from the pool of all models.

[0034] For the regression task, the evaluation indicators used in the present invention include Pearson coefficient, covariance and r2_score.

[0035] For the classification task, the present invention uses precision and AUC indicators for evaluation.

[0036] The Pearson index describes the strength of the linear correlation between two sets of variables. The value of the Pearson coefficient is between [-1,1]. If r_pearson is equal to 1, it means that y and x are completely linearly related in the same direction. If the value is -1, it means that they are completely linearly negatively correlated. If it is equal to 0, it means that there is no direct linear relationship between them. In short, the larger the absolute value of r_pearson, the stronger the correlation between the two sets of variables. The calculation formula is as follows, which requires the use of covariance and respective variance.

[0037] Formula 1:

[0038] Among them, σ(X) and σ(Y) represent the variance of the distribution of X and Y respectively, and is their covariance.

[0039] The integration and multi-layer perceptron part concatenates the two features extracted by the model into a complete vector, and passes through linear convolution layer by layer, outputs a fully connected layer with a smaller dimension and a one-dimensional pooling layer, and obtains a final classification result or model fitting result of size 1.

[0040] During the training and testing process, the model first obtains each piece of data and organizes it into a list, then obtains the processing methods of molecules and proteins and puts them into the global parameters.

[0041] The comprehensive prediction module is named predictor, and the expression models of the two branches are named molemodel and protein model respectively.

[0042] Considering that the vector dimensions expressed by the model may not match the moledim and protein dim specified by the predictor, an additional linear layer is added to meet the shape requirements of the interface.

[0043] The continuous linear connection layers are used to reduce the feature dimension layer by layer and extract the final information. The input dimension of the first layer does not need to be determined in advance, but is dynamically determined by calculating the sum of the dimensions of the two vectors.

[0044] According to the dimension array parameters of the stacked layers, the dimension of the first layer output is 1024, and the following layers are all transformed from 1024 to 1024 for simple processing. The output layer of the last layer is reduced to 512, and then greatly reduced to 1, which can be used as the result of the final classification or fitting task.

[0045] The above uses multiple local models as feature extraction modules for molecules and proteins, unifies their final output size, and concatenates the two features and sends them to the Transformer fusion network. Taking the drug molecule model as an example, it is shown as follows:

[0046] The KCL molecular representation model combines chemical molecular knowledge into contrastive learning. Its structure is as follows Figure 2 .

[0047] The properties of the atoms have an important influence on the properties of the entire molecule. In order to establish this relationship, a molecular graph related to chemical elements is first constructed.

[0048] The graph KG adds a chemical property node C to the original molecular structure graph. For any node set N that is in the same local area and meets the requirements, each atom will lead a one-way edge to point C. In this way, the new graph not only has the original molecular characteristic structure, but also includes information on atomic properties, and is divided by local area.

[0049] The KCL uses three structural modules to create knowledge-enhanced contrastive learning. The first part is the feature enhancement module mentioned above. The second part, the knowledge-aware graph, uses a conventional graph message passing network to diffuse the information of surrounding nodes and extract the feature space information of the original graph and the graph structure after knowledge augmentation. The third part is the part where contrastive learning calculates the loss.

[0050] The molecular graph is constructed as G=(V, E), which respectively represent the sets of nodes and edges in the graph.

[0051] For an atomic node v, let N(v) represent the set of neighboring nodes around it.

[0052] The x(v) is used as the initial node feature vector of point v.

[0053] The e(uv) is represented as the initial feature of the edge connecting points u and v.

[0054] The h(v) is the hidden layer vector propagated by point v in the graph message network.

[0055] The h(uv) is used as the hidden layer vector of edge uv during the transmission process.

[0056] The entire process of message processing is x(v), e(u,v)→h(v), h(u,v)→R[d]. Finally, a d-dimensional full-image feature vector is obtained. It can reflect the internal structural information and other knowledge of the molecule to a certain extent, and this vector can be used for downstream tasks such as various molecular property predictions.

[0057] The described KMPNN encoder uses a molecular graph structure for structural enhancement.

[0058] The MSG1 is used as a transfer mode when the surrounding neighbors are internal atoms, and MSG0 is used when the neighbors are inserted attribute nodes.

[0059] The KCL model redefines the message passing and self-attention calculation methods on the extended graph. Normalization and softmax activation are integrated into the calculation, so that different types of nodes can be passed without distinction. The formula is as follows:

[0060] Formula 2:

[0061] Among them, αuv is the attention size between nodes u and v, k is the neighbor node of node u, H is the function of calculating the node vector, Relu is the activation function, and || represents vector concatenation.

[0062] The meaning of formula 1 is to calculate the ratio of the closeness of the relationship between the pair of nodes to all neighboring node pairs. After obtaining the weight ratio, it is multiplied into the matrix update process in MSG0. The obtained α parameter is used in the KMPNN message passing module in the KCL model, and the formula is as follows:

[0063] Formula 3: MSG0 = α uv *W0*H k-1 (u,e uv );

[0064] Among them, the W0 matrix controls the respective weights of neighbor node information when aggregating, and the α parameter further adjusts the degree of influence.

[0065] H k-1 (u,e uv ) is the attention calculated by the model based on the previous round of nodes u and v. MSG0 is a branch of message passing in the graph neural network. The calculation method of the other branch MSG1 is similar to MSG0, but it needs to be multiplied by another different parameter β.

[0066] The Grover described is a deep model for molecular representation, and is a molecular representation model of the Transformer architecture trained with self-supervision.

[0067] The Grover described creates two types of self-supervised tasks.

[0068] Regarding the representation of nodes and edges, Grover does not use the structural graph of the entire molecule to make global predictions, but randomly deletes a part of the nodes and edges to form a subgraph, and predicts the structure of the shielded part through the embedded features between the nodes, inducing the model to observe more of the properties of the nodes and edges themselves and the context.

[0069] The Grover model includes two processing branch modules, including a node branch and an edge branch, and each branch uses a GNN as the main network.

[0070] The Grover combines the GNN graph structure and the attention mechanism of the Transformer core, and completes two mutual transformations between the graph structure and the vector space. In the above process, the structure of the molecular data is fully modeled.

[0071] Step 3.1: For numerical regression tasks, the Davis and Kiba datasets are commonly used in the field of integration prediction.

[0072] The MolrProtTrans performs better than the DeepDTA deep model.

[0073] The comparison between the predicted value and the actual label of the MolrProtTrans and its corresponding basic model in the regression task on the kiba dataset is as follows: Figure 3 As shown in Figure 2. On the Kiba dataset, the Pearson coefficients of the model reached 0.53 and 0.439, respectively, which are sufficient to support the conclusion that the predicted value of the model is linearly positively correlated with the actual value.

[0074] Step 3.2: After various methods are trained on various data sets, the changes in the loss value during training are recorded to determine whether the model is indeed learning the label value, and the speed of model iteration convergence is observed, and it is evaluated according to the task type on the corresponding divided test set.

[0075] The above mentioned combination model of various molecular and protein information extractors was used on the Davis dataset. A combination of basic graph message propagation network MPNN and CNN was also added as a control. The loss during training is as follows Figure 4 shown.

[0076] Step 3.3: Use the combined method of KCL and CNN to perform the fitting task on the Kiba dataset. The results are as follows: Figure 5 shown.

[0077] The experimental results of the above methods on the Davis dataset are shown in Table 1:

[0078] Table 1 R score of each method on Davis

[0079]

[0080]

[0081] From the above results, we can see that the models using Grover and CNN as molecular and protein feature extraction branches respectively have a MSE loss reduced to 1.0 after 800 rounds of training, while the models using Molr and ESM have a loss of up to 50 after the same rounds of training, which shows that Grover and CNN models are better on the Davis dataset. Overall, each model can significantly reduce the loss within the first 10 rounds, indicating that they all have basic learning capabilities on the Davis dataset. Regression verification is performed on the test set, and their performance is not much different, with regression indicators ranging from 0.49 to 0.57. The complete MolrProtTrans model can achieve an Rscore of 0.655, and after removing advanced modules such as attention, its basic model performance is reduced to 0.485, which is a significant decline. The most basic combination model, MPNN and CNN, achieves a result of 0.274, which is much lower than other combinations, proving that the effect of the combination model is related to its structural complexity. The gap between the indicators of regression tasks and classification tasks still comes from the distribution of data labels. In regression tasks, there are both a large number of unified labels and some scattered high-value labels, which increases the difficulty of training.

[0082] In summary, the excellent performance of the baseline model is reflected in the drug target regression task. The model proposed in this invention has significant improvements over previous models on a variety of public drug target datasets, and can effectively solve the problem of unbalanced distribution of drug and target data. The performance of the baseline model is better than the simplified model in all aspects, proving that the selected molecular and protein representation modules and the designed dual attention mechanism have important contributions to the overall performance.

[0083] The above are only preferred embodiments of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These should also be regarded as the protection scope of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A drug target binding degree regression prediction method based on the MolrProtTrans model. The MolrProtTrans model is trained with the drug molecule and protein binding degree dataset to complete the classification and regression tasks. It is characterized in that The following steps are involved: Step 1: Pre-train the drug molecule and protein representation network of the MolrProtTrans model, connect the synthesized vectors to each machine learning model, and construct a hybrid structure model; Step 2: MolrProtTrans operation and simplification. Change the model training loss, cancel the activation layer of the last layer, and use MolrProtTrans to complete the regression task of drug target prediction; Step 3: Use various other molecular and protein representation models as extractors to build a comprehensive drug-target binding degree regression prediction model.

2. The drug target binding degree regression prediction method based on the MolrProtTrans model according to claim 1, It is characterized in that In the step 1, the drug molecule representation network and protein representation network of the MolrProtTrans model are pre-trained. The vector representation obtained by combining the Transformer architecture is described, a dual attention mechanism is designed, and the MolrProtTrabs model is trained and evaluated. The drug and target vectors obtained by MolrProtTrans are directly spliced ​​and connected to various machine learning models to construct a hybrid structure model.

3. The drug target binding degree regression prediction method based on the MolrProtTrans model according to claim 1, It is characterized in that In step 2, MolrProtTrans is used for regression tasks, which converts labels from logical binary classifications of 0 or 1 into floating point numbers, and describes the degree of binding between drugs and targets with specific numerical values. The data sets selected are GPCR, Kinase, human, BindingDB, Davis, and Kiba. When the MolrProtTrans model is used for regression tasks, the MolR network needs to be trained separately, and the parameters of the protein models ProtTrans and prose need to be loaded. Then, the transformer structure is connected for feature fusion, and finally, a decimal is obtained through linear layer processing as the binding degree value predicted by the model. The Brenda and Rhea datasets were used when training the Molr network. In terms of model construction, the dgl library provides graph convolutional networks GraphConv, graph convolutional networks with attention GATConv, and other more complex variants SAGEConv, SGConv, TAGConv, etc. GraphConv is used as the model backbone network, and the structure of the model's overlay network layer is set to 2 layers. The molecules in the data set are first converted into the mol molecular graph format using the rdkit function, and then converted back into the smiles format using rdkit to obtain the standard format. Before using the Molr model to train data, it is necessary to obtain an overview of the attributes of all molecules in the entire data set. The attribute list is represented by a dictionary, and the dictionary is recorded as a feature encoder. The regression model modifies the output interface based on the original classification model, modifies the probabilities of the two categories into a numerical output, and deletes the activation function of the last layer so that the numerical value is not limited to between 0 and 1. The final output of the main model of the classification task is [64,1], which means the probability p of each prediction result being 1, and the probability of the remaining predictions being 0 can be expressed as 1-p. Simply remove the softmax activation of the last layer of the classification MolrProtTrans model to cancel the range limit of the result value.

4. The drug target binding degree regression prediction method based on the MolrProtTrans model according to claim 1, It is characterized in that In step 3, the combination degree prediction network of multiple models is aggregated, and the whole model consists of two parts. The output vector shape of all feature extraction models is [64,1024]. The molecular information model provides MPNN, Molr, KCL, grover, etc. The protein information model provides CNN, prose, ProtTrans, ESM, etc. The respective information extractor models of the molecules and proteins are specified by parameters, and a corresponding set is selected from the pool of all models. For the regression task, the evaluation indicators used in the present invention include Pearson coefficient, covariance and r2_score. For the classification task, the present invention uses precision and AUC indicators for evaluation. The Pearson coefficient value is between [-1,1]. If r_pearson is equal to 1, it means that y and x are completely linearly related in the same direction. If the value is -1, it means that they are completely linearly negatively correlated. If it is equal to 0, it means that there is no direct linear relationship between them. The larger the absolute value of r_pearson, the stronger the correlation between this group of variables. The calculation formula is as follows, which requires the use of covariance and respective variance. Formula 1: Among them, σ(X) and σ(Y) represent the variance of the distribution of X and Y respectively, and is their covariance. The integration and multi-layer perceptron part concatenates the two features extracted by the model into a complete vector, and passes through linear convolution layer by layer, outputs a fully connected layer with a smaller dimension and a one-dimensional pooling layer, and obtains a final classification result or model fitting result of size 1. During the training and testing process, the model first obtains each piece of data and organizes it into a list, then obtains the processing methods of molecules and proteins and puts them into the global parameters. The comprehensive prediction module is named predictor, and the expression models of the two branches are named molemodel and protein model respectively. Considering that the dimension of the vector expressed by the model does not necessarily match the moledim and proteindim specified by the predictor, an additional linear layer is added to meet the shape requirements of the interface. The continuous linear connection layers are used to reduce the feature dimension layer by layer and extract the final information. The input dimension of the first layer does not need to be determined in advance, but is dynamically determined by calculating the sum of the dimensions of the two vectors. According to the dimension array parameters of the stacked layers, the dimension of the first layer output is 1024, and the following layers are all transformed from 1024 to 1024 for simple processing. The output layer of the last layer is reduced to 512, and then greatly reduced to 1, which can be used as the result of the final classification or fitting task.

5. According to the content of claim 4, It is characterized in that Use multiple local models as feature extraction modules for molecules and proteins, unify their final output size, and concatenate the two features into the Transformer fusion network. Take the drug molecule model as an example to show the following: The core idea of ​​the KCL molecular representation model is to use chemical knowledge to amplify the graph and the comparative learning method. First, a molecular graph related to chemical elements is constructed. The graph KG adds a chemical attribute node C to the original molecular structure graph. For any node set N that is in the same local area and meets the requirements, each atom will lead a unidirectional edge to point C. The KCL uses three structural modules to create knowledge-enhanced contrastive learning. The first part is the feature enhancement module mentioned above. The second part, the knowledge-aware graph, uses a conventional graph message passing network to diffuse the information of surrounding nodes and extract the feature space information of the original graph and the graph structure after knowledge augmentation. The third part is the part where contrastive learning calculates the loss. The molecular graph is constructed as G=(V, E), which respectively represent the sets of nodes and edges in the graph. For an atomic node v, let N(v) represent the set of neighboring nodes around it. The x(v) is used as the initial node feature vector of point v. The e(uv) is represented as the initial feature of the edge connecting points u and v. The h(v) is the hidden layer vector propagated by point v in the graph message network. The h(uv) is used as the hidden layer vector of edge uv during the transmission process. The entire process of message processing is x(v), e(u,v)→h(v), h(u,v)→R[d]. Finally, a d-dimensional full-image feature vector is obtained. It can reflect the internal structural information and other knowledge of the molecule to a certain extent, and this vector can be used for downstream tasks such as various molecular property predictions. The described KMPNN encoder uses a molecular graph structure for structural enhancement. The MSG1 is used as a transfer mode when the surrounding neighbors are internal atoms, and MSG0 is used when the neighbors are inserted attribute nodes. The KCL model redefines the message passing and self-attention calculation methods on the extended graph. Normalization and softmax activation are integrated into the calculation, so that different types of nodes can be passed without distinction. The formula is as follows: Formula 2: Among them, αuv is the attention size between nodes u and v, k is the neighbor node of node u, H is the function of calculating the node vector, Relu is the activation function, and || represents vector concatenation. The meaning of formula 2 is to calculate the ratio of the closeness of the relationship between the pair of nodes to all neighboring node pairs. After obtaining the weight ratio, it is multiplied into the matrix update process in MSG0. The obtained α parameter is used in the KMPNN message passing module in the KCL model, and the formula is as follows: Equation 3: MSG 0 = α uv * W 0 * H k-1 (u, e uv ); Among them, the W0 matrix controls the respective weights of neighbor node information when aggregating, and the α parameter further adjusts the degree of influence. H k-1 (u,e uv ) is the attention calculated by the model based on the previous round of nodes u and v. MSG0 is a branch of message passing in the graph neural network. The calculation method of the other branch MSG1 is similar to MSG0, but it needs to be multiplied by another different parameter β. The Grover described is a deep model for molecular representation, and is a molecular representation model of the Transformer architecture trained with self-supervision. The Grover described creates two types of self-supervised tasks. For the representation of nodes and edges, Grover randomly deletes a part of the nodes and edges to form a subgraph, and predicts the structure of the shielded part through the embedded features between the nodes, inducing the model to observe more the properties of the nodes and edges themselves and the context environment. The Grover model includes two processing branch modules, including a node branch and an edge branch, and each branch uses a GNN as the main network. The Grover combines the GNN graph structure and the attention mechanism of the Transformer core, completes two mutual transformations between the graph structure and the vector space, and fully models the structure of molecular data. For the numerical regression task, the combined degree prediction uses the Davis and Kiba datasets. The MolrProtTrans performs better than the DeepDTA deep model. In the regression task of the MolrProtTrans and its corresponding basic model on the kiba dataset, the Pearson coefficients of the models reached 0.53 and 0.439 respectively. The above-mentioned combined model of molecular and protein information extractors is used on the Davis dataset. A combination of basic graph message propagation network MPNN and CNN is also added as a control. The method described above uses a combination of KCL and CNN to perform a fitting task on the Kiba dataset. The model using Grover and CNN as molecular and protein feature extraction branches, respectively, reduced the MSE loss to 1.0 after 800 rounds of training. The models using Molr and ESM have a loss of up to 50 after training for the same number of epochs, and the Grover and CNN models perform better on the Davis dataset. The models described above have basic learning capabilities on the Davis dataset. The complete MolrProtTrans model can achieve an Rscore of 0.

655. After removing advanced modules such as attention, the performance of its basic model is reduced to 0.485, which is a significant decline. The most basic combination model of MPNN and CNN achieved a result of 0.274, which is much lower than other combinations.

Images