Multimodal drug molecule prediction method based on knowledge prompt

Abstract

The invention discloses a multi-modal drug molecule prediction method based on knowledge prompt, belongs to the field of artificial intelligence assisted drug research and development, and particularly relates to a multi-modal drug molecule prediction method. The objective of the invention is to solve the problems that an existing multi-modal molecule prediction method is often difficult to capture a complex relationship between a molecular graph structure and a text, and the prediction accuracy on a downstream task is low. The multi-modal drug molecule prediction method based on knowledge prompt specifically comprises the following steps: constructing a MolPrompt model; the MolPrompt model is subjected to pre-training, and the pre-trained MolPrompt model is obtained; on the basis of the downstream task type, performing fine adjustment on the pre-trained MolPrompt model to obtain a fine-adjusted MolPrompt model; and predicting a downstream task based on the MolPrompt model after fine tuning.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence-assisted drug research and development, and specifically relates to a multimodal drug molecule prediction method based on knowledge prompts. Background Art

[0002] Understanding the properties and functions of molecules is crucial for drug development, providing a foundation for the development of targeted therapies, the elucidation of disease mechanisms, and the development of personalized medicine. Traditional drug development relies on laboratory experiments, which are time-consuming and require large and continuous investments. In recent years, artificial intelligence (AI) has played an increasingly important role in this field, significantly improving research efficiency and reducing costs. Inspired by large-scale pre-trained models such as BERT and GPT series, pre-training models of drug molecules have become an important direction for AI-assisted drug development. Existing research works apply self-supervised learning tasks for pre-training on large-scale molecular datasets, aiming to explore molecular representations from data such as molecular SMILES strings and molecular graph structures.

[0003] Text mining and knowledge discovery of molecule-related biomedical literature play an important role in molecular science, drug discovery, and pathology research. It is an important record of molecular science and a key modality in the field of multimodal molecular pre-training. In order to give artificial intelligence the ability to understand molecular structures and molecular description texts in a multimodal way, researchers have begun to explore multimodal molecular representation learning, such as molecular SMILES strings, molecular graph structures, molecular description texts, etc. Although molecular SMILES strings and texts are sequential in nature and can be seamlessly connected, some molecular structure information may be ignored using SMILES strings. Compared with molecular SMILES strings, two-dimensional molecular graphs can more intuitively reveal the functional structure of molecules and provide richer molecular structure information. However, existing multimodal molecular pre-training models often have difficulty in capturing the complex relationship between molecular graph structures and texts, and their performance on downstream tasks is somewhat insufficient. First, most methods rely entirely on data-driven methods to learn molecular representations from inherent molecular information without combining any external prior knowledge, especially the knowledge of chemical and physical properties that are closely related to molecular properties. Second, current molecular graph structure encoders are limited to capturing structural features and fail to grasp the rich semantic information related to molecular properties. Third, the cross-modal contrastive learning pre-training task has its inherent limitations. Considering the wide range of downstream tasks for multimodal molecular representation learning, the pre-trained model may not perform well on all tasks. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that existing multimodal molecular prediction methods often find it difficult to capture the complex relationship between molecular graph structure and text, and have low prediction accuracy in downstream tasks, and to propose a multimodal drug molecule prediction method based on knowledge hints.

[0005] The specific process of the multimodal drug molecule prediction method based on knowledge prompts is as follows:

[0006] Step 1: Build the MolPrompt model;

[0007] Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model;

[0008] Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model;

[0009] Step 4: Predict downstream tasks based on the fine-tuned MolPrompt model.

[0010] Preferably, in step 1, a MolPrompt model is constructed; the specific process is:

[0011] Step 1: Construct knowledge prompts;

[0012] Step 1 and 2: Based on the knowledge hint, input the molecular graph into the molecular graph structure encoder to obtain the molecular graph feature representation:

[0013] Step 13: Input the molecular description text corresponding to the molecular graph in step 12 into the text encoder to obtain the molecular description text feature representation.

[0014] Preferably, the knowledge prompt is constructed in the step one by one; the specific process is:

[0015] Step 1: Use the RDkit tool to extract 209 molecular descriptors of numerical type from the molecular graph, and select 10 molecular descriptors of numerical type from the 209 molecular descriptors of numerical type as prior knowledge of external chemical and physical properties of molecules;

[0016] The 10 molecular descriptors of numerical type are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3 and BalabanJ;

[0017] The MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings, FractionCSP3 is the proportion of carbon atoms with three bonds to hydrogen, and BalabanJ is Balaban's J index;

[0018] The extracted 10 molecular descriptors of numerical type are uniformly set to the format of Descriptor1:Value1,Descriptor2:Value2,Descriptor3:Value3,Descriptor4:Value4,Descriptor5:Value5,Descriptor6:Value6,Descripto7:Value7,Descripto8:Value8,Descripto9:Value9,Descripto10:Value10;

[0019] Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 retains one decimal place;

[0020] The 10 numerical molecular descriptors after the unified format are text-type knowledge prompts;

[0021] Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type;

[0022] Descriptor2:Value2 represents a molecule descriptor 2 of numeric type: value 2;

[0023] Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type;

[0024] Descriptor4:Value4 represents a molecule descriptor 4 of numeric type: value 4;

[0025] Descriptor5:Value5 represents a molecule descriptor 5:value 5 of numeric type;

[0026] Descriptor6:Value6 represents a molecule descriptor 6:value 6 of numeric type;

[0027] Descripto7:Value7 represents the molecule descriptor 7 of numeric type: value 7;

[0028] Descriptor8:Value8 represents a molecule descriptor of numeric type 8:value8;

[0029] Descripto9:Value9 represents a molecule descriptor of numeric type 9:value9;

[0030] Descripto10:Value10 represents a molecule descriptor 10 of numeric type: value 10;

[0031] Step 112:

[0032] The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints

[0033] in, represents the first word feature embedding in the knowledge prompt, represents the second word feature embedding in the knowledge prompt, Indicates the Xth in the knowledge prompt p word-unit feature embedding, N p Indicates the length of the text, d p Represents the dimension of BERT’s word embedding layer.

[0034] Preferably, in the steps 1 and 2, based on the knowledge prompt, the molecular graph feature representation is obtained through the molecular graph structure encoder: the specific process is:

[0035] Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 ,

[0036] The molecular graph node feature H 0 The last atomic feature in is replaced by the class label h p ,

[0037]

[0038] in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, and d gRepresents the feature dimension of the molecular graph structure encoder;

[0039] Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 .

[0040] Preferably, the processing process of each of the first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer and the twelfth Graphormer layer is:

[0041] H i-1 Input the i-th Graphormer layer, and the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows:

[0042]

[0043] In the formula, K g Indicates the total number of Graphormer layers, K g =12; H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

[0044] Preferably, in step 1-3, the molecular description text corresponding to the molecular graph in step 1-2 is input into a text encoder to obtain a molecular description text feature representation; the specific process is:

[0045] Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation

[0046] in, The molecular description text feature represents the first word feature embedding in W0, The molecular description text feature represents the second word feature embedding in W0. Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text, d t Represents the dimension of the text encoder;

[0047] Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

[0048] Preferably, the processing process of each of the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is:

[0049] The molecular description text feature representation W output by the i-1th Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows:

[0050]

[0051] In the above formula, K t Represents the total number of Transformer layers, K t =12;W i Indicates the molecular description text feature representation output by the i-th Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

[0052] Preferably, in step 2, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model; the specific process is:

[0053] Step 21: Initialize the weights of Graphormer and Bert;

[0054] Step 22: Obtain N molecular image-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )};

[0055] The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j;

[0056] N molecular graph-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )} is divided into M groups, each group contains a picture-text pair data;

[0057] Step 23: Compare the molecular graph structure g in each group of molecular graphs i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ;

[0058] The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecule description text feature representation w K ;

[0059] Step 24: Represent a molecular graph features as h K Input a multi-layer perceptron MLP, which outputs a projected molecular graph feature representations;

[0060] Represent a molecular description text features w K Input a multi-layer perceptron MLP, which outputs a projected molecular description text feature representation;

[0061] The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown below:

[0062]

[0063] Step 25: Define the loss function as

[0064]

[0065] Among them, sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, which is set to 0.1;

[0066] is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function;

[0067] Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16;

[0068] Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

[0069] Preferably, in step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain a fine-tuned MolPrompt model; the specific process is:

[0070] When the downstream task is molecular property prediction, 8 molecular property datasets are used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecular property prediction;

[0071] The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE;

[0072] When the downstream task is molecule-text cross-modal retrieval, the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

[0073] Preferably, in step 4, the downstream task is predicted based on the fine-tuned MolPrompt model; the specific process is:

[0074] 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows:

[0075] The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; the molecular graph feature representation is input into the prediction head, and the prediction head outputs the molecular attribute;

[0076] The prediction head is a linear layer;

[0077] 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used for molecular graph-text cross-modal retrieval; the specific process is:

[0078] Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M;

[0079] M2T means given a molecular graph, retrieving the molecular description text that matches the molecular graph;

[0080] T2M means given a molecular description text, retrieving a molecular graph that matches the molecular description text;

[0081] The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecular-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation;

[0082] In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation;

[0083] At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts;

[0084] Calculate the cosine similarity between the molecular graph feature representation and all molecular description text feature representations and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text.

[0085] In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation;

[0086] At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations;

[0087] The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

[0088] The beneficial effects of the present invention are:

[0089] The present invention proposes a multimodal molecular pre-training model based on knowledge prompts, MolPrompt. The framework adopts a contrastive learning pre-training method with two independent encoding branches to process the molecular graph structure and the molecular description text respectively. In the molecular graph structure encoding branch, text-type knowledge prompts constructed by molecular descriptors are introduced to enhance the capture of microscopic molecular structure information. The method of the present invention effectively utilizes the semantic information related to molecular chemistry and physical properties provided by knowledge prompts, thereby significantly improving the performance of the pre-training model on downstream tasks such as retrieval and property prediction. The present invention emphasizes the importance of integrating molecular chemistry and physical property knowledge. The introduction of knowledge prompts enables the molecular graph structure encoder to capture information containing molecular chemistry and physical properties while encoding molecular graph structural features, thereby obtaining molecular feature representations with more comprehensive semantic information. MolPrompt demonstrates excellent generalization ability in a wide range of downstream tasks. MolPrompt shows the best performance in the molecular-text cross-modal retrieval task, demonstrating its strong generalization ability and significant advantages in cross-dataset and cross-task generalization. MolPrompt performed well in the molecular property prediction task, highlighting its ability in classification tasks related to physiology and biophysics. It solves the problem that existing multimodal molecular prediction methods often have difficulty in capturing the complex relationship between molecular graph structure and text, and have low prediction accuracy in downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 It is a flow chart of the present invention;

[0091] Figure 2 This is a diagram of the feature coding structure of the present invention. DETAILED DESCRIPTION

[0092] Specific implementation method 1: The specific process of the multimodal drug molecule prediction method based on knowledge prompts in this implementation method is as follows:

[0093] The purpose of the present invention is to solve the problem that the multimodal molecular pre-training model is difficult to capture the complex relationship between the molecular graph structure and the text, and has certain deficiencies in performance on downstream tasks, and propose a multimodal drug molecule pre-training method based on knowledge prompts.

[0094] The knowledge of chemical and physical properties of molecules is combined as prompts in the pre-training and fine-tuning stages. This method aims to enhance fine-grained multimodal molecular representation learning and bridge the gap between contrastive learning pre-training and downstream tasks, thereby improving performance for a wider range of applications. The specific process of the multimodal drug molecule pre-training method based on knowledge prompts is:

[0095] Step 1: Build the MolPrompt model;

[0096] Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model;

[0097] Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model;

[0098] Step 4: Predict downstream tasks based on the fine-tuned MolPrompt model.

[0099] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that: in step 1, a MolPrompt model is constructed; the specific process is:

[0100] Step 1: Construct knowledge prompts;

[0101] Step 1 and 2: Based on the knowledge hint, input the molecular graph into the molecular graph structure encoder to obtain the molecular graph feature representation:

[0102] Step 13: Input the molecular description text corresponding to the molecular graph in step 12 (the molecular graph and molecular description text of the same molecule) into the text encoder to obtain the molecular description text feature representation.

[0103] The other steps and parameters are the same as those in the first embodiment.

[0104] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that: in the step one, a knowledge prompt is constructed;

[0105] The specific process is:

[0106] Step 1: Use the RDkit tool to extract 209 molecular descriptors of numerical type from the molecular graph, and select the 10 most commonly used molecular descriptors of numerical type from the 209 molecular descriptors of numerical type as the prior knowledge of the external chemical and physical properties of the molecules;

[0107] The 10 most commonly used molecular descriptors of numerical type are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3, and BalabanJ;

[0108] The MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings, FractionCSP3 is the proportion of carbon atoms with three bonds to hydrogen, and BalabanJ is Balaban's J index;

[0109] Molecular descriptors are able to quantitatively describe the physical and chemical properties of molecules. Molecular descriptors are zero-dimensional and represent the simplest form of molecular representation;

[0110] Zero-dimensional molecular descriptors include molecular weight, atomic number, atom type count, and other basic descriptors (e.g., number of heavy atoms);

[0111] In order to solve the limitation that the existing molecular graph structure encoder can only capture structural features but cannot obtain semantic information related to molecular properties, and at the same time enable the molecular graph structure encoder to have a deeper understanding of text modal information;

[0112] The present invention sets the extracted 10 most commonly used molecular descriptors of numerical type into a unified format Descriptor1: Value1, Descriptor2: Value2, Descriptor3: Value3, Descriptor4: Value4, Descriptor5: Value5, Descriptor6: Value6, Descripto7: Value7, Descripto8: Value8, Descripto9: Value9, Descripto10: Value10;

[0113] Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 retains one decimal place;

[0114] The 10 most commonly used numerical molecular descriptors after unified format are text-type knowledge prompts;

[0115] Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type;

[0116] Descriptor2:Value2 represents a molecule descriptor 2 of numeric type: value 2;

[0117] Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type;

[0118] Descriptor4:Value4 represents a molecule descriptor 4 of numeric type: value 4;

[0119] Descriptor5:Value5 represents a molecule descriptor 5:value 5 of numeric type;

[0120] Descriptor6:Value6 represents a molecule descriptor 6:value 6 of numeric type;

[0121] Descripto7:Value7 represents the molecule descriptor 7 of numeric type: value 7;

[0122] Descriptor8:Value8 represents a molecule descriptor of numeric type 8:value8;

[0123] Descripto9:Value9 represents a molecule descriptor of numeric type 9:value9;

[0124] Descripto10:Value10 represents a molecule descriptor 10 of numeric type: value 10;

[0125] Step 112:

[0126] The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints

[0127] in, represents the first word feature embedding in the knowledge prompt, represents the second word feature embedding in the knowledge prompt, Indicates the Nth pword-unit feature embedding, N p Indicates the length of the text (the text contains 10 molecular descriptors and their corresponding values), d p Represents the dimension of BERT's word embedding layer (dimension is 768).

[0128] The other steps and parameters are the same as those in the first or second embodiment.

[0129] Specific implementation method 4: This implementation method is different from one of the specific implementation methods 1 to 3 in that: in the steps 1 and 2, based on the knowledge prompt, the molecular graph feature representation is obtained through the molecular graph structure encoder:

[0130] The specific process is:

[0131] Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 ,

[0132] The molecular graph node feature H 0 The last atomic feature in is replaced by the class label h p ,

[0133]

[0134] in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, and d g Represents the feature dimension of the molecular graph structure encoder (the dimension is 768);

[0135] After replacement, H 0 The last node feature of contains the chemical and physical prior knowledge of the molecule;

[0136] Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 .

[0137] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0138] Specific embodiment 5: This embodiment is different from any one of specific embodiments 1 to 4 in that the processing process of each of the 1st Graphormer layer, the 2nd Graphormer layer, the 3rd Graphormer layer, the 4th Graphormer layer, the 5th Graphormer layer, the 6th Graphormer layer, the 7th Graphormer layer, the 8th Graphormer layer, the 9th Graphormer layer, the 10th Graphormer layer, the 11th Graphormer layer and the 12th Graphormer layer is:

[0139] H i-1 Input the i-th Graphormer layer, and the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows:

[0140]

[0141] In the formula, K g Indicates the total number of Graphormer layers, K g =12; H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

[0142] The other steps and parameters are the same as those in Specific Implementation 1 to 4-1.

[0143] Specific implementation method 6: This implementation method is different from one of specific implementation methods 1 to 5 in that: in step 13, the molecular description text corresponding to the molecular graph in step 12 is input into the text encoder to obtain the molecular description text feature representation;

[0144] The specific process is:

[0145] Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation

[0146] in, The molecular description text feature represents the first word feature embedding in W0, The molecular description text feature represents the second word feature embedding in W0. Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text (128), d t Represents the dimension of the text encoder (the dimension is 768);

[0147] Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

[0148] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5-1.

[0149] Specific implementation manner seven: This implementation manner is different from any one of specific implementation manners one to six in that: the processing process of each of the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is:

[0150] The molecular description text feature representation W output by the i-1th Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows:

[0151]

[0152] In the above formula, K t Represents the total number of Transformer layers, K t =12;W i Indicates the molecular description text feature representation output by the i-th Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

[0153] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0154] Specific implementation eight: This implementation differs from any one of specific implementations one to seven in that: in step two, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model;

[0155] The specific process is:

[0156] Step 21: Before pre-training, initialize the weights of Graphormer and Bert; the specific process is:

[0157] Initialize the weights of Graphormer using graphormer-base-pcqmv1 (pre-trained weights of Graphormer on the PCQM4M-LSC dataset);

[0158] Initialize Bert’s weights using KV-PLM’s pre-trained weights;

[0159] Step 22: In the pre-training stage, obtain N molecular image-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )};

[0160] The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j;

[0161] N molecular graph-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )} is divided into M groups (each group contains 16 molecular image-text pairs), each group contains a image-text pairs;

[0162] The original molecule-text data are all positive pairs, and during the pre-training process, negative pairs are automatically matched for pre-training;

[0163] Step 23: Compare the molecular graph structure g in each group of molecular graphs i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ;

[0164] The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecule description text feature representation w K ;

[0165] Step 24: Represent a molecular graph features as h K Input a multi-layer perceptron (MLP), which outputs a projected molecular graph feature representations;

[0166] Represent a molecular description text features w K Input a multi-layer perceptron (MLP), which outputs a projected molecular description text feature representation;

[0167] The molecular graph structure representation h is obtained by using the molecular graph structure encoder and the text encoder respectively. K and the molecular description text representation w K Afterwards, the present invention uses a multi-layer perceptron (MLP) as a projection network to project these two types of representations into the same dimensional feature space. The present invention adopts a structure-text contrastive learning pre-training strategy, the purpose of which is to bring different modality samples with the same semantic information closer in the feature space, and to separate samples with different semantic information.

[0168] The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown below:

[0169]

[0170] The structure-context contrast loss is used to bring different modality samples with the same semantic information closer in the feature space, and separate samples with different semantic information;

[0171] Step 25: Define the loss function as

[0172]

[0173] Among them, sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, which is set to 0.1;

[0174] is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function;

[0175] Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16;

[0176] Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

[0177] The other steps and parameters are the same as those in Specific Implementation 1 to 7-1.

[0178] Specific implementation method 9: This implementation method is different from any one of specific implementation methods 1 to 8 in that: in step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain the fine-tuned MolPrompt model; the specific process is:

[0179] When the downstream task is molecular attribute prediction (such as molecular liver toxicity, neurotoxicity, anti-HIV activity, etc.), the pre-trained MolPrompt model is fine-tuned using 8 molecular attribute data sets to obtain a fine-tuned MolPrompt model for the downstream task of molecular attribute prediction;

[0180] The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE;

[0181] When the downstream task is molecule-text cross-modal retrieval (i.e., in a data set containing a series of molecules and texts, given a molecule, retrieve the text description that matches it; given a text, retrieve the molecule that matches its description), the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

[0182] The other steps and parameters are the same as those in Specific Implementation 1 to 8-1.

[0183] Specific implementation method 10: This implementation method is different from any one of specific implementation methods 1 to 9 in that: in step 4, downstream tasks are predicted based on the fine-tuned MolPrompt model; the specific process is:

[0184] 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows:

[0185] The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs a molecular graph feature representation of a fixed length; the molecular graph feature representation is input into the prediction head, and the prediction head outputs the molecular attribute;

[0186] The prediction head is a linear layer;

[0187] Depending on the prediction task, different prediction heads can be used. Therefore, MolPrompt can complete binary or multi-classification tasks.

[0188] 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used for molecular graph-text cross-modal retrieval; the specific process is:

[0189] Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M;

[0190] M2T means given a molecular graph, retrieving the molecular description text that matches the molecular graph;

[0191] T2M means given a molecular description text, retrieving a molecular graph that matches the molecular description text;

[0192] The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecular-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation;

[0193] In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation;

[0194] At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts;

[0195] Calculate the cosine similarity between the molecular graph feature representation and all molecular description text feature representations and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text.

[0196] The task scenario of M2T is: suppose there is a molecular graph, but we don’t know which molecular description text the molecular graph corresponds to. At the same time, there are 100,000 molecular description texts, and we need to retrieve the molecular description text that best matches the molecular graph.

[0197] In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation;

[0198] At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations;

[0199] The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

[0200] The task scenario of T2M is: suppose there is a molecular description text, but we don’t know which molecular graph the molecular description text corresponds to. At the same time, there are 100,000 molecular graphs, and we need to retrieve the molecular graph that best matches the molecular description text.

[0201] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 9.

[0202] The following examples are used to verify the beneficial effects of the present invention:

[0203] Embodiment 1:

[0204] The first step is to build knowledge prompts:

[0205] Knowledge can be defined as any quantifiable information that characterizes a molecule. Molecular descriptors are able to quantitatively describe the physical and chemical properties of a molecule. Molecular descriptors are zero-dimensional and represent the simplest form of molecular representation. Zero-dimensional molecular descriptors include molecular weight, atomic number, atom type count, and other basic descriptors (e.g., number of heavy atoms).

[0206] First, the present invention uses RDkit to extract molecular descriptors from molecules. RDKit provides more than 200 molecular descriptors, from which the present invention selects 10 most commonly used descriptors as prior knowledge of the external chemical and physical properties of molecules, including MolWt (molecular weight), MolLogP (molecular logP), NumRotatableBonds (number of rotatable bonds), TPSA (topological polar surface area), NumHDonors (number of hydrogen bond donors), NumHAcceptors (number of hydrogen bond acceptors), RingCount (number of rings), NumAromaticRings (number of aromatic rings), FractionCSP3 (the proportion of carbon atoms with three bonds to hydrogen) and BalabanJ (Balaban's J index).

[0207] In order to solve the limitation that the existing molecular graph structure encoder can only capture structural features but cannot obtain semantic information related to molecular properties, and at the same time enable the molecular graph structure encoder to have a deeper understanding of text modal information, the present invention constructs the numerical type molecular descriptors extracted from molecules into text type knowledge prompts, and uniformly sets the format to "Descriptor1:Value1,Descriptor2:Value2,…,Descriptor10:Value10", where the value of each molecular descriptor retains one decimal place. Subsequently, the obtained text type knowledge prompt is input into the word embedding layer of BERT to generate the knowledge prompt embedding

[0208] The second step is to obtain feature representation through the molecular graph structure encoder and text encoder:

[0209] The present invention uses Graphormer as a molecular graph structure encoder, which adopts the Graph Transformer architecture and is built on the basic framework of the classic Transformer. Graphormer consists of multiple Graphormer layers. In Graphormer, three simple and effective structural encoding methods are adopted, namely centrality encoding, spatial encoding and edge encoding. These methods utilize the structural information reflected on the nodes and the relationship between the node pairs in the graph to enhance the modeling ability of Graphormer for graph structure data.

[0210] The present invention inputs the molecular graph into the graph feature encoding layer in the molecular graph structure encoder to obtain the molecular graph node feature H 0 , The present invention extracts class labels from knowledge hint embedding, denoted as h p , H 0 The last node feature of is replaced by h p .

[0211]

[0212] After replacement, H 0 The last node feature contains the chemical and physical prior knowledge of the molecule.

[0213] At each stage, H i The (i+1)th Graphormer layer is fed into the molecular graph structure encoder The formula is shown below.

[0214]

[0215] In the above formula, K g represents the number of layers of the molecular graph structure encoder, K g =12.

[0216] The present invention uses BERT as a text encoder, which is a widely used feature extractor in natural language processing. Compared with general image-text data, text data related to molecules, especially high-quality molecular description text, is relatively limited. This scarcity makes it insufficient to train a text encoder from scratch. Therefore, the present invention initializes BERT with KV-PLM. KV-PLM is a self-supervised molecular representation learning model based on BERT, which is pre-trained on a combination of molecular SMILES strings and molecular description texts, so it is more suitable for processing texts describing molecular features.

[0217] The text encoder works by tokenizing the words and projecting them into word embeddings At each stage, W i Input to the (i+1)th Transformer layer of the text description encoder The formula is shown below.

[0218]

[0219] In the above formula, K t Represents the total number of layers of the text encoder, K t =12.

[0220] The third step is to apply contrastive learning pre-training:

[0221] Let {(g1,t1),(g2,t2),…,(g N ,t N )} represents a batch of molecular graph-text pair data, where the molecular graph structure g b The corresponding molecular description text t b Constitute a positive pair (g b ,t b ), with different molecular description texts t j Negative pair (g b ,t j ) b≠j . N molecular graph-text pairs {(g1, t1), (g2, t2), …, (g N ,t N )} is divided into M groups (each group contains 16 molecular graph-text pairs), each group contains a graph-text pairs; the a molecular graph structures g in the molecular graph-text pairs of each group are i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ; each group of molecular image and text pairs in the data a molecular description text t i Input text encoder, the text encoder outputs a molecule description text feature representation w K ;

[0222] The molecular graph structure representation h is obtained by using the molecular graph structure encoder and the text encoder respectively. K and the molecular description text representation w K Afterwards, the present invention uses a multilayer perceptron (MLP) as a projection network to project these two types of representations into the same dimensional feature space, as shown in the following formula.

[0223]

[0224] The present invention expresses the feature after projection of the b-th molecular graph as The projected features of the molecular description text are expressed as This paper adopts structure-context contrast loss, the purpose of which is to bring different modality samples with the same semantic information closer in the feature space, and separate samples with different semantic information. The loss function is defined as

[0225]

[0226] Here, sim(·,·) is the cosine similarity and τ is the temperature hyperparameter, which is set to 0.1.

[0227] Before pre-training, the present invention uses graphormer-base-pcqmv1 (the pre-trained weights of Graphormer on the PCQM4M-LSC dataset) to initialize the weights of Graphormer, and uses the pre-trained weights of KV-PLM to initialize the weights of Bert.

[0228] In the pre-training stage, the present invention uses the AdamW optimizer with a learning rate of 1e-4 and a weight decay of 1e-5 to pre-train MolPrompt. The number of pre-training rounds is 30 and the batch size is set to 16.

[0229] The fourth step is to apply the pre-trained model to complete downstream tasks:

[0230] The pre-trained model can be applied to a wide range of downstream tasks, such as molecular property prediction and molecule-text cross-modal retrieval.

[0231] When the downstream task is molecular attribute prediction (such as molecular liver toxicity, neurotoxicity, anti-HIV activity, etc.), the present invention uses 8 molecular attribute data sets (BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV and BACE) to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for molecular attribute prediction for the downstream task. Specifically, the present invention inputs the molecular graph to be tested into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs a fixed-length molecular graph feature representation; the molecular graph feature representation is input into the prediction head (linear layer), and the prediction head outputs the molecular attribute. Different prediction heads can correspond to different prediction tasks, so MolPrompt can complete binary or multi-classification tasks.

[0232] When the downstream task is molecule-text cross-modal retrieval (i.e., in a data set containing a series of molecules and texts, given a molecule, retrieve the text description that matches it; given a text, retrieve the molecule that matches its description), the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval. Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M. M2T means that given a molecular graph, the molecular description text that matches the molecular graph is retrieved; T2M means that given a molecular description text, the molecular graph that matches the molecular description text is retrieved. In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; at the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs all molecular description text feature representations. The cosine similarity between the molecular graph feature representation and all molecular description text feature representations is calculated and sorted. The molecular description text with the largest cosine similarity is the best matching molecular description text. The T2M task is the opposite.

[0233] The present invention may also have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multimodal drug molecule prediction method based on knowledge hints, characterized by: The specific process of the method is: Step 1: Build the MolPrompt model; Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model; Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model; Step 4: Predict downstream tasks based on the fine-tuned MolPrompt model.

2. The multimodal drug molecule prediction method based on knowledge hint according to claim 1, characterized in that: In the step 1, the MolPrompt model is constructed; the specific process is: Step 1: Construct knowledge prompts; Step 1 and 2: Based on the knowledge hint, input the molecular graph into the molecular graph structure encoder to obtain the molecular graph feature representation: step Step 1-3: Input the molecular description text corresponding to the molecular graph in step 1-2 into the text encoder to obtain the molecular description text feature representation.

3. The multimodal drug molecule prediction method based on knowledge hint according to claim 2, characterized in that: The knowledge prompt is constructed in the steps one by one; the specific process is: Step 1: Use the RDkit tool to extract 209 molecular descriptors of numerical type from the molecular graph, and select 10 molecular descriptors of numerical type from the 209 molecular descriptors of numerical type as prior knowledge of external chemical and physical properties of molecules; The 10 molecular descriptors of numerical type are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3 and BalabanJ; The MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings, FractionCSP3 is the fraction of carbon atoms with three bonds to hydrogen, and BalabanJ is Balaban's J index; The extracted 10 numerical molecular descriptors are uniformly set to the format of Descriptor1:Value1,Descriptor2:Value2,Descriptor3:Value3,Descriptor4:Value4,Descriptor5:Value5,Descripto r6:Value6,Descripto7:Value7,Descripto8:Value8,Descripto9:Value9,Descripto10:Value10; Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 retains one decimal place; The 10 numerical molecular descriptors after the unified format are text-type knowledge prompts; Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type; Descriptor2:Value2 represents a molecule descriptor 2 of numeric type: value 2; Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type; Descriptor4:Value4 represents a molecule descriptor 4 of numeric type: value 4; Descriptor5:Value5 represents a molecule descriptor 5:value 5 of numeric type; Descriptor6:Value6 represents a molecule descriptor 6:value 6 of numeric type; Descripto7:Value7 represents the molecule descriptor 7 of numeric type: value 7; Descriptor8:Value8 represents a molecule descriptor of numeric type 8:value8; Descripto9:Value9 represents a molecule descriptor of numeric type 9:value9; Descripto10:Value10 represents a molecule descriptor 10 of numeric type: value 10; Step 112: The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints in, represents the first word feature embedding in the knowledge prompt, represents the second word feature embedding in the knowledge prompt, Indicates the Nth p word-unit feature embedding, N p Indicates the length of the text, d p Represents the dimension of BERT’s word embedding layer.

4. The multimodal drug molecule prediction method based on knowledge hint according to claim 3, characterized in that: In the steps 1 and 2, based on the knowledge prompt, the molecular graph feature representation is obtained through the molecular graph structure encoder: the specific process is: Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 , The molecular graph node feature H 0 The last atomic feature in is replaced by the class label h p , in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, and d g Represents the feature dimension of the molecular graph structure encoder; Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 .

5. The multimodal drug molecule prediction method based on knowledge hint according to claim 4, characterized in that: The processing process of each Graphormer layer in the first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer is: H i-1 Input the i-th Graphormer layer, and the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows: In the formula, K g Indicates the total number of Graphormer layers, K g =12; H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

6. The multimodal drug molecule prediction method based on knowledge hint according to claim 5, characterized in that: In step 13, the molecular description text corresponding to the molecular graph in step 12 is input into the text encoder to obtain the molecular description text feature representation; the specific process is: Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation in, The molecular description text feature represents the first word feature embedding in W0, The molecular description text feature represents the second word feature embedding in W0. Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text, d t Represents the dimension of the text encoder; Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

7. The multimodal drug molecule prediction method based on knowledge hint according to claim 6, characterized in that: The processing process of each of the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is: The molecular description text feature representation W output by the i-1th Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows: In the above formula, K t Represents the total number of Transformer layers, K t =12;W i Indicates the molecular description text feature representation output by the i-th Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

8. The multimodal drug molecule prediction method based on knowledge hint according to claim 7, characterized in that: In the step 2, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model; the specific process is: Step 21: Initialize the weights of Graphormer and Bert; Step 22: Obtain N molecular image-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )}; The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j; N molecular graph-text pair data {(g1, t1), (g2, t2), ..., (g N , t N )) is divided into M groups, each group contains a picture-text pair data; Step 23: Compare the molecular graph structure g in each group of molecular graphs i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ; The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecule description text feature representation W K ; Step 24: Represent a molecular graph features as h K Input a multi-layer perceptron MLP, which outputs a projected molecular graph feature representations; Represent a molecular description text features w K Input a multi-layer perceptron MLP, which outputs a projected molecular description text feature representation; The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown below: Step 25: Define the loss function as Among them, sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, which is set to 0.1; is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function; Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16; Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

9. The multimodal drug molecule prediction method based on knowledge hint according to claim 8, characterized in that: In step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain a fine-tuned MolPrompt model; the specific process is: When the downstream task is molecular property prediction, 8 molecular property datasets are used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecular property prediction; The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE; When the downstream task is molecule-text cross-modal retrieval, the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

10. The multimodal drug molecule prediction method based on knowledge hint according to claim 9, characterized in that: In step 4, the downstream task is predicted based on the fine-tuned MolPrompt model; the specific process is as follows: 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows: The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; the molecular graph feature representation is input into the prediction head, and the prediction head outputs the molecular attribute; The prediction head is a linear layer; 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used for molecular graph-text cross-modal retrieval; the specific process is: Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M; M2T means given a molecular graph, retrieving the molecular description text that matches the molecular graph; T2M means given a molecular description text, retrieving a molecular graph that matches the molecular description text; The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecular-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation; In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts; Calculate the cosine similarity between the molecular graph feature representation and all molecular description text feature representations and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text. In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation; At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations; The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

Images