Multimodal drug molecule prediction method based on knowledge hints

Abstract

A multimodal drug molecule prediction method based on knowledge prompts. The present invention belongs to the field of artificial intelligence-assisted drug research and development, and specifically relates to a multimodal drug molecule prediction method. The purpose of the present invention is to solve the problem that existing multimodal molecule prediction methods often have difficulty in capturing the complex relationship between molecular graph structure and text, and have low prediction accuracy in downstream tasks. The specific process of the multimodal drug molecule prediction method based on knowledge prompts is: constructing a MolPrompt model; pre-training the MolPrompt model to obtain a pre-trained MolPrompt model; based on the downstream task type, fine-tuning the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model; and predicting downstream tasks based on the fine-tuned MolPrompt model.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence-assisted drug research and development, and specifically relates to a multimodal drug molecule prediction method based on knowledge prompts. Background Art

[0002] Understanding the properties and functions of molecules is crucial for drug development, providing a foundation for the development of targeted therapies, the elucidation of disease mechanisms, and the advancement of personalized medicine. Traditional drug development relies on laboratory experiments, which are time-consuming and require large and ongoing investments. In recent years, artificial intelligence (AI) has played an increasingly important role in this field, significantly improving research efficiency and reducing costs. Inspired by large-scale pre-trained models such as BERT and the GPT series, pre-training models for drug molecules has become an important direction for AI-assisted drug development. Existing research works have applied self-supervised learning tasks for pre-training on large-scale molecular datasets, aiming to explore molecular representations from data such as molecular SMILES strings and molecular graph structures.

[0003] Text mining and knowledge discovery of molecular-related biomedical literature plays a vital role in molecular science, drug discovery, and pathology research. It represents a crucial record of molecular science and a key modality in the field of multimodal molecular pre-training. To empower AI with the ability to understand molecular structures and molecular description text in a multimodal manner, researchers have begun exploring multimodal molecular representation learning, such as molecular SMILES strings, molecular graph structures, and molecular description text. Although molecular SMILES strings and text are inherently sequential and can be seamlessly connected, using SMILES strings can overlook some molecular structural information. Compared to molecular SMILES strings, two-dimensional molecular graphs can more intuitively reveal the functional structure of molecules and provide richer molecular structural information. However, existing multimodal molecular pre-training models often struggle to capture the complex relationship between molecular graph structure and text, and their performance on downstream tasks is limited. First, most methods rely solely on data-driven approaches, learning molecular representations from inherent molecular information without incorporating any external prior knowledge, particularly knowledge of chemical and physical properties that are closely related to molecular attributes. Second, current molecular graph structure encoders are limited to capturing structural features and fail to capture the rich semantic information associated with molecular properties. Third, the cross-modal contrastive learning pre-training task has its inherent limitations. Considering the wide range of downstream tasks for multimodal molecular representation learning, the pre-trained model may not perform well on all tasks. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that existing multimodal molecular prediction methods often have difficulty in capturing the complex relationship between molecular graph structure and text, and have low prediction accuracy in downstream tasks, and to propose a multimodal drug molecule prediction method based on knowledge hints.

[0005] The specific process of the multimodal drug molecule prediction method based on knowledge hints is as follows:

[0006] Step 1: Build the MolPrompt model;

[0007] Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model;

[0008] Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model;

[0009] Step 4: Make predictions for downstream tasks based on the fine-tuned MolPrompt model.

[0010] Preferably, in step 1, a MolPrompt model is constructed; the specific process is:

[0011] Step 1: Construct knowledge prompts;

[0012] Step 1 and 2: Based on the knowledge hint, input the molecular graph into the molecular graph structure encoder to obtain the molecular graph feature representation:

[0013] Step 13: Input the molecular description text corresponding to the molecular graph in step 1 and 2 into the text encoder to obtain the molecular description text feature representation.

[0014] Preferably, the knowledge prompt is constructed in the steps one by one; the specific process is:

[0015] Step 1: Use the RDkit tool to extract 209 numerical molecular descriptors from the molecular graph, and select 10 numerical molecular descriptors from the 209 numerical molecular descriptors as prior knowledge of the external chemical and physical properties of the molecules;

[0016] The 10 numerical molecular descriptors are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3, and BalabanJ;

[0017] MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings; FractionCSP3 is the proportion of carbon atoms with three bonds to hydrogen; BalabanJ is Balaban's J index;

[0018] The extracted 10 numerical type molecular descriptors were unified into the format of Descriptor1:Value1,Descriptor2:Value2,Descriptor3:Value3,Descriptor4:Value4,Descriptor5:Value5,Descrip tor6:Value6,Descripto7:Value7,Descripto8:Value8,Descripto9:Value9,Descripto10:Value10;

[0019] Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 is rounded to one decimal place;

[0020] The 10 numerical molecular descriptors after unified format are text-type knowledge prompts;

[0021] Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type;

[0022] Descriptor2:Value2 represents the molecule descriptor 2:value 2 of the numeric type;

[0023] Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type;

[0024] Descriptor4:Value4 represents the molecule descriptor 4:value 4 of the numeric type;

[0025] Descriptor5:Value5 represents the molecule descriptor 5:value 5 of the numeric type;

[0026] Descriptor6:Value6 represents the molecule descriptor 6:value 6 of the numeric type;

[0027] Descripto7:Value7 represents the molecular descriptor 7 of the numeric type: value 7;

[0028] Descriptor8:Value8 represents a molecule descriptor of a numerical type 8:value8;

[0029] Descripto9:Value9 represents the molecular descriptor 9 of the numerical type: value 9;

[0030] Descripto10:Value10 represents the molecule descriptor 10:value10 of the numeric type;

[0031] Step 112:

[0032] The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints

[0033] in, represents the first word feature embedding in the knowledge prompt, Represents the second word feature embedding in the knowledge prompt, Indicates the Xth in the knowledge prompt p word-unit feature embedding, N p Indicates the length of the text, d p Represents the dimension of BERT's word embedding layer.

[0034] Preferably, in steps one and two, based on the knowledge hint, the molecular graph feature representation is obtained through the molecular graph structure encoder: the specific process is:

[0035] Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 ,

[0036] The molecular graph node feature H 0 The last atomic feature in is replaced by the class tag h p ,

[0037]

[0038] in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, d gRepresents the feature dimension of the molecular graph structure encoder;

[0039] Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 .

[0040] Preferably, the processing process of each of the first graphormer layer, the second graphormer layer, the third graphormer layer, the fourth graphormer layer, the fifth graphormer layer, the sixth graphormer layer, the seventh graphormer layer, the eighth graphormer layer, the ninth graphormer layer, the tenth graphormer layer, the eleventh graphormer layer, and the twelfth graphormer layer is:

[0041] H i-1 Input the i-th Graphormer layer, the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows:

[0042]

[0043] Where K g Indicates the total number of Graphormer layers, K g =12;H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

[0044] Preferably, in step 13, the molecular description text corresponding to the molecular graph in step 12 is input into a text encoder to obtain a feature representation of the molecular description text; the specific process is:

[0045] Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation

[0046] in, Indicates that the molecular description text feature represents the first word feature embedding in W0, Indicates that the molecular description text feature represents the second word feature embedding in W0, Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text, d t Represents the dimension of the text encoder;

[0047] Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

[0048] Preferably, the processing process of each of the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is as follows:

[0049] The molecular description text feature representation W output by the i-1th layer Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows:

[0050]

[0051] In the above formula, K t Indicates the total number of Transformer layers, K t =12;W i Indicates the output of the molecular description text feature representation of the i-th layer Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

[0052] Preferably, in step 2, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model; the specific process is:

[0053] Step 21: Initialize the weights of Graphormer and Bert;

[0054] Step 22: Get N molecular image-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )};

[0055] The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Constitute a negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j;

[0056] N molecular graph-text pair data {(g1,t1),(g2,t2),…,(g N ,t N )} is divided into M groups, each group contains a picture-text pair data;

[0057] Step 2 and 3: Compare the molecular graph structure g in each group of molecular graph data i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ;

[0058] The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecular description text feature representation w K ;

[0059] Step 24: Represent a molecular graph feature as h K Input a multi-layer perceptron MLP, which outputs a projected molecular graph feature representation;

[0060] Represent a molecule description text feature w K Input a multi-layer perceptron MLP, which outputs a projected molecular description text feature representation;

[0061] The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown in the following formula:

[0062]

[0063] Step 25: Define the loss function as

[0064]

[0065] Where sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, which is set to 0.1;

[0066] is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function;

[0067] Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16;

[0068] Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

[0069] Preferably, in step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain a fine-tuned MolPrompt model; the specific process is:

[0070] When the downstream task is molecular property prediction, the pre-trained MolPrompt model is fine-tuned using 8 molecular property datasets to obtain a fine-tuned MolPrompt model for the downstream task of molecular property prediction.

[0071] The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE;

[0072] When the downstream task is molecule-text cross-modal retrieval, the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

[0073] Preferably, in step 4, downstream tasks are predicted based on the fine-tuned MolPrompt model; the specific process is:

[0074] 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows:

[0075] The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction. The molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; the molecular graph feature representation is input into the prediction head, and the prediction head outputs the molecular attributes;

[0076] The prediction head is a linear layer;

[0077] 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used to perform molecular graph-text cross-modal retrieval; the specific process is as follows:

[0078] Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M;

[0079] M2T means that given a molecular graph, the molecular description text that matches the molecular graph is retrieved;

[0080] T2M means that given a molecular description text, a molecular graph that matches the molecular description text is retrieved;

[0081] The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation;

[0082] In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation;

[0083] At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts;

[0084] Calculate the cosine similarity between the molecular graph feature representation and the feature representation of all molecular description texts and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text;

[0085] In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation;

[0086] At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations;

[0087] The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

[0088] The beneficial effects of the present invention are:

[0089] The present invention proposes a multimodal molecular pre-training model based on knowledge prompts, MolPrompt. The framework adopts a contrastive learning pre-training method with two independent encoding branches to process the molecular graph structure and the molecular description text respectively. In the molecular graph structure encoding branch, text-type knowledge prompts constructed by molecular descriptors are introduced to enhance the capture of microscopic molecular structure information. The method of the present invention effectively utilizes the semantic information related to molecular chemistry and physical properties provided by knowledge prompts, thereby significantly improving the performance of the pre-training model in downstream tasks (such as retrieval and property prediction). The present invention emphasizes the importance of integrating molecular chemistry and physical property knowledge. The introduction of knowledge prompts enables the molecular graph structure encoder to capture information containing molecular chemistry and physical properties while encoding molecular graph structural features, thereby obtaining molecular feature representations with more comprehensive semantic information. MolPrompt demonstrates excellent generalization capabilities in a wide range of downstream tasks. MolPrompt shows the best performance in the molecule-text cross-modal retrieval task, demonstrating its strong generalization ability and significant advantages in cross-dataset and cross-task generalization. MolPrompt performed exceptionally well in molecular property prediction tasks, highlighting its capabilities in classification tasks related to physiology and biophysics. This approach addresses the challenges faced by existing multimodal molecular prediction methods, which often struggle to capture the complex relationships between molecular graph structures and text, and suffer from low prediction accuracy in downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 Flowchart of the present invention;

[0091] Figure 2 This is a diagram of the feature coding structure of the present invention. DETAILED DESCRIPTION

[0092] Specific embodiment 1: The specific process of the multimodal drug molecule prediction method based on knowledge hints in this embodiment is as follows:

[0093] The purpose of this invention is to solve the problem that multimodal molecular pre-training models are difficult to capture the complex relationship between molecular graph structure and text, and have certain shortcomings in performance on downstream tasks, and propose a multimodal drug molecule pre-training method based on knowledge prompts.

[0094] This approach incorporates knowledge of the chemical and physical properties of molecules as hints during pre-training and fine-tuning. This approach aims to enhance fine-grained multimodal molecular representation learning, bridging the gap between contrastive learning pre-training and downstream tasks, thereby improving performance for a wider range of applications. The specific process of the multimodal drug molecule pre-training method based on knowledge hints is as follows:

[0095] Step 1: Build the MolPrompt model;

[0096] Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model;

[0097] Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model;

[0098] Step 4: Make predictions for downstream tasks based on the fine-tuned MolPrompt model.

[0099] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that: in step 1, a MolPrompt model is constructed; the specific process is as follows:

[0100] Step 1: Construct knowledge prompts;

[0101] Step 1 and 2: Based on the knowledge hint, input the molecular graph into the molecular graph structure encoder to obtain the molecular graph feature representation:

[0102] Step 13: Input the molecular description text corresponding to the molecular graph in step 12 (the molecular graph and molecular description text of the same molecule) into the text encoder to obtain the molecular description text feature representation.

[0103] Other steps and parameters are the same as those in the first embodiment.

[0104] Specific embodiment three: This embodiment differs from specific embodiment one or two in that: in step one, a knowledge prompt is constructed;

[0105] The specific process is:

[0106] Step 1: Use the RDkit tool to extract 209 numerical molecular descriptors from the molecular graph, and select the 10 most commonly used numerical molecular descriptors from the 209 numerical molecular descriptors as the prior knowledge of the external chemical and physical properties of the molecules;

[0107] The 10 most commonly used numerical type molecular descriptors are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3 and BalabanJ;

[0108] MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings; FractionCSP3 is the proportion of carbon atoms with three bonds to hydrogen; BalabanJ is Balaban's J index;

[0109] Molecular descriptors can quantitatively describe the physical and chemical properties of molecules. Molecular descriptors are zero-dimensional and represent the simplest form of molecular representation;

[0110] Zero-dimensional molecular descriptors include molecular weight, atomic number, atom type count, and other basic descriptors (e.g., number of heavy atoms);

[0111] To address the limitation of existing molecular graph structure encoders that can only capture structural features but cannot obtain semantic information related to molecular properties, and at the same time enable molecular graph structure encoders to have a deeper understanding of textual modal information;

[0112] The present invention unifies the extracted 10 most commonly used molecular descriptors of numerical type into the format of Descriptor1:Value1, Descriptor2:Value2, Descriptor3:Value3, Descriptor4:Value4, Descriptor5:Value5, Descriptor6:Value6, Descripto7:Value7, Descripto8:Value8, Descripto9:Value9, Descripto10:Value10;

[0113] Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 is rounded to one decimal place;

[0114] The 10 most commonly used numerical molecular descriptors after unified format are text-type knowledge prompts;

[0115] Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type;

[0116] Descriptor2:Value2 represents the molecule descriptor 2:value 2 of the numeric type;

[0117] Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type;

[0118] Descriptor4:Value4 represents the molecule descriptor 4:value 4 of the numeric type;

[0119] Descriptor5:Value5 represents the molecule descriptor 5:value 5 of the numeric type;

[0120] Descriptor6:Value6 represents the molecule descriptor 6:value 6 of the numeric type;

[0121] Descripto7:Value7 represents the molecular descriptor 7 of the numeric type: value 7;

[0122] Descriptor8:Value8 represents a molecule descriptor of a numerical type 8:value8;

[0123] Descripto9:Value9 represents the molecular descriptor 9 of the numerical type: value 9;

[0124] Descripto10:Value10 represents the molecule descriptor 10:value10 of the numeric type;

[0125] Step 112:

[0126] The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints

[0127] in, represents the first word feature embedding in the knowledge prompt, Represents the second word feature embedding in the knowledge prompt, Indicates the Nth pword-unit feature embedding, N p Indicates the length of the text (the text contains 10 molecular descriptors and their corresponding values), d p Represents the dimension of BERT's word embedding layer (dimension is 768).

[0128] Other steps and parameters are the same as those in the first or second embodiment.

[0129] Specific embodiment 4: This embodiment differs from one of specific embodiments 1 to 3 in that: in steps 1 and 2, based on knowledge prompts, the molecular graph feature representation is obtained through the molecular graph structure encoder:

[0130] The specific process is:

[0131] Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 ,

[0132] The molecular graph node feature H 0 The last atomic feature in is replaced by the class tag h p ,

[0133]

[0134] in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, d g Represents the feature dimension of the molecular graph structure encoder (dimension is 768);

[0135] After replacement, H 0 The last node feature of contains the chemical and physical prior knowledge of the molecule;

[0136] Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 .

[0137] The other steps and parameters are the same as those in the first to third embodiments.

[0138] Specific embodiment 5: This embodiment is different from any one of specific embodiments 1 to 4 in that the processing process of each of the 1st Graphormer layer, the 2nd Graphormer layer, the 3rd Graphormer layer, the 4th Graphormer layer, the 5th Graphormer layer, the 6th Graphormer layer, the 7th Graphormer layer, the 8th Graphormer layer, the 9th Graphormer layer, the 10th Graphormer layer, the 11th Graphormer layer, and the 12th Graphormer layer is as follows:

[0139] H i-1 Input the i-th Graphormer layer, the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows:

[0140]

[0141] Where K g Indicates the total number of Graphormer layers, K g =12;H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

[0142] Other steps and parameters are the same as those in Specific Embodiments 1 to 4-1.

[0143] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that: in step 13, the molecular description text corresponding to the molecular graph in step 12 is input into a text encoder to obtain a feature representation of the molecular description text;

[0144] The specific process is:

[0145] Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation

[0146] in, Indicates that the molecular description text feature represents the first word feature embedding in W0, Indicates that the molecular description text feature represents the second word feature embedding in W0, Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text (128), d t Indicates the dimension of the text encoder (the dimension is 768);

[0147] Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

[0148] Other steps and parameters are the same as those in Specific Implementations 1 to 5-1.

[0149] Specific embodiment 7: This embodiment differs from any one of specific embodiments 1 to 6 in that the processing process of each Transformer layer among the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is as follows:

[0150] The molecular description text feature representation W output by the i-1th layer Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows:

[0151]

[0152] In the above formula, K t Indicates the total number of Transformer layers, K t =12;W i Indicates the output of the molecular description text feature representation of the i-th layer Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

[0153] The other steps and parameters are the same as those in the first to sixth embodiments.

[0154] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that: in step two, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model;

[0155] The specific process is:

[0156] Step 21: Before pre-training, initialize the weights of Graphormer and Bert; the specific process is:

[0157] Initialize the weights of Graphormer using graphormer-base-pcqmv1 (Graphormer pre-trained weights on the PCQM4M-LSC dataset);

[0158] Initialize Bert’s weights using KV-PLM’s pre-trained weights;

[0159] Step 2: In the pre-training stage, obtain N molecular image-text pair data {(g1, t1), (g2, t2), ..., (g N ,t N )};

[0160] The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Constitute a negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j;

[0161] N molecular graph-text pair data {(g1,t1),(g2,t2),…,(g N ,t N )} is divided into M groups (each group contains 16 molecular image-text pairs), each group contains a image-text pairs;

[0162] The original molecule-text data are all positive pairs. During the pre-training process, negative pairs are automatically matched for pre-training;

[0163] Step 2 and 3: Compare the molecular graph structure g in each group of molecular graph data i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ;

[0164] The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecular description text feature representation w K ;

[0165] Step 24: Represent a molecular graph feature as h K Input a multi-layer perceptron (MLP), which outputs a projected molecular graph feature representation;

[0166] Represent a molecule description text feature w K Input a multi-layer perceptron (MLP), which outputs a projected molecular description text feature representation;

[0167] The molecular graph structure representation h is obtained by using the molecular graph structure encoder and the text encoder respectively. K and the molecular description text representation w K The present invention then uses a multi-layer perceptron (MLP) as a projection network to project these two types of representations into the same dimensional feature space. This invention employs a structure-context contrastive learning pre-training strategy, aiming to bring different modality samples with the same semantic information closer together in the feature space, while separating samples with different semantic information.

[0168] The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown in the following formula:

[0169]

[0170] The structure-context contrast loss is used to bring different modality samples with the same semantic information closer together in the feature space, while separating samples with different semantic information.

[0171] Step 25: Define the loss function as

[0172]

[0173] Where sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, which is set to 0.1;

[0174] is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function;

[0175] Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16;

[0176] Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

[0177] Other steps and parameters are the same as those in Specific Embodiments 1 to 7-1.

[0178] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that: in step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain a fine-tuned MolPrompt model; the specific process is:

[0179] When the downstream task is molecular attribute prediction (such as hepatotoxicity, neurotoxicity, anti-HIV activity, etc.), the pre-trained MolPrompt model is fine-tuned using eight molecular attribute datasets to obtain a fine-tuned MolPrompt model for the downstream task of molecular attribute prediction.

[0180] The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE;

[0181] When the downstream task is molecule-text cross-modal retrieval (i.e., in a data set containing a series of molecules and texts, given a molecule, retrieve the text description that matches it; given a text, retrieve the molecule that matches its description), the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

[0182] The other steps and parameters are the same as those in the specific implementation modes 1 to 8-1.

[0183] Specific embodiment 10: This embodiment differs from any one of specific embodiments 1 to 9 in that: in step 4, downstream tasks are predicted based on the fine-tuned MolPrompt model; the specific process is as follows:

[0184] 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows:

[0185] The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction. The fine-tuned MolPrompt model molecular graph structure encoder outputs a fixed-length molecular graph feature representation. The molecular graph feature representation is input into the prediction head, which outputs the molecular attributes.

[0186] The prediction head is a linear layer;

[0187] Depending on the prediction task, different prediction heads can be used. Therefore, MolPrompt can complete binary or multi-classification tasks.

[0188] 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used to perform molecular graph-text cross-modal retrieval; the specific process is as follows:

[0189] Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M;

[0190] M2T means that given a molecular graph, the molecular description text that matches the molecular graph is retrieved;

[0191] T2M means that given a molecular description text, a molecular graph that matches the molecular description text is retrieved;

[0192] The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation;

[0193] In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation;

[0194] At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts;

[0195] Calculate the cosine similarity between the molecular graph feature representation and the feature representation of all molecular description texts and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text;

[0196] The task scenario of M2T is: suppose there is a molecular graph, but we don’t know which molecular description text the molecular graph corresponds to. At the same time, there are 100,000 molecular description texts, and we need to retrieve the molecular description text that best matches the molecular graph.

[0197] In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation;

[0198] At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations;

[0199] The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

[0200] The task scenario of T2M is: suppose there is a molecular description text, but it is unknown which molecular graph the molecular description text corresponds to. At the same time, there are 100,000 molecular graphs, and the molecular graph that best matches the molecular description text is retrieved.

[0201] The other steps and parameters are the same as those in the specific implementation modes 1 to 9-1.

[0202] The following examples are used to verify the beneficial effects of the present invention:

[0203] Example 1:

[0204] The first step is to build knowledge prompts:

[0205] Knowledge can be defined as any quantifiable information that characterizes a molecule. Molecular descriptors are able to quantitatively describe the physical and chemical properties of a molecule. Molecular descriptors are zero-dimensional and represent the simplest form of molecular representation. Zero-dimensional molecular descriptors include molecular weight, atomic number, atom type count, and other basic descriptors (e.g., the number of heavy atoms).

[0206] First, the present invention uses RDKit to extract molecular descriptors from molecules. RDKit provides more than 200 molecular descriptors, from which the present invention selects 10 of the most commonly used descriptors as prior knowledge of the external chemical and physical properties of molecules, including MolWt (molecular weight), MolLogP (molecular logP), NumRotatableBonds (number of rotatable bonds), TPSA (topological polar surface area), NumHDonors (number of hydrogen bond donors), NumHAcceptors (number of hydrogen bond acceptors), RingCount (number of rings), NumAromaticRings (number of aromatic rings), FractionCSP3 (the proportion of carbon atoms with three bonds to hydrogen), and BalabanJ (Balaban's J index).

[0207] In order to solve the limitation of existing molecular graph structure encoders that can only capture structural features but cannot obtain semantic information related to molecular properties, and at the same time enable molecular graph structure encoders to have a deeper understanding of text modal information, the present invention constructs the numerical type molecular descriptors extracted from molecules into text type knowledge prompts, and uniformly sets the format to "Descriptor1:Value1,Descriptor2:Value2,…,Descriptor10:Value10", where the value of each molecular descriptor retains one decimal place. Subsequently, the obtained text type knowledge prompts are input into the word embedding layer of BERT to generate knowledge prompt embedding

[0208] The second step is to obtain feature representation through the molecular graph structure encoder and the text encoder:

[0209] The present invention uses Graphormer as a molecular graph structure encoder, employing the Graph Transformer architecture and building upon the foundational framework of the classic Transformer. Graphormer consists of multiple Graphormer layers. Three simple and effective structural encoding methods are employed within Graphormer: centrality encoding, spatial encoding, and edge encoding. These methods leverage the structural information reflected on nodes and the relationships between node pairs in the graph to enhance Graphormer's ability to model graph-structured data.

[0210] The present invention inputs the molecular graph into the graph feature encoding layer in the molecular graph structure encoder to obtain the molecular graph node feature H 0 , The present invention extracts the class label from the knowledge hint embedding, denoted as h p , H 0 The last node feature is replaced by h p .

[0211]

[0212] After replacement, H 0 The last node feature contains the chemical and physical prior knowledge of the molecule.

[0213] At each stage, H i The (i+1)th Graphormer layer is sent to the molecular graph structure encoder The formula is as follows.

[0214]

[0215] In the above formula, K g represents the number of layers of the molecular graph structure encoder, K g =12.

[0216] This paper uses BERT, a widely used feature extractor in natural language processing, as a text encoder. Compared to general image-text data, molecular-related text data, particularly high-quality molecular description text, is relatively limited. This scarcity makes it insufficient for training a text encoder from scratch. Therefore, this paper initializes BERT with KV-PLM. KV-PLM is a self-supervised molecular representation learning model based on BERT. It is pre-trained on a combination of molecular SMILES strings and molecular description text, making it more suitable for processing text describing molecular features.

[0217] The text encoder works by tokenizing the words and projecting them into word embeddings At each stage, W i Input to the (i+1)th Transformer layer of the text description encoder The formula is as follows.

[0218]

[0219] In the above formula, K t represents the total number of layers of the text encoder, K t =12.

[0220] The third step is to apply contrastive learning pre-training:

[0221] Let {(g1,t1),(g2,t2),…,(g N ,t N )} represents a batch of molecular graph-text pair data, where the molecular graph structure g b The corresponding molecular description text t b Constitute a positive pair (g b ,t b ), with different molecular description text t j Constitute a negative pair (g b ,t j ) b≠j . N molecular graph-text data {(g1,t1),(g2,t2),…,(g N ,t N )} is divided into M groups (each group contains 16 molecular graph-text pairs), each group contains a graph-text pairs; the a molecular graph structures g in each group of molecular graph-text pairs are converted into i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ; each group of molecular image text pairs in the data a molecular description text t i Input text encoder, the text encoder outputs a molecular description text feature representation w K ;

[0222] The molecular graph structure representation h is obtained by using the molecular graph structure encoder and the text encoder respectively. K and the molecular description text representation w K Afterwards, the present invention uses a multilayer perceptron (MLP) as a projection network to project these two types of representations into the same dimensional feature space, as shown in the following formula.

[0223]

[0224] The present invention expresses the feature after projection of the b-th molecular graph as The projected features of the molecular description text are expressed as This paper adopts structure-context contrast loss, which aims to bring different modality samples with the same semantic information closer in the feature space and separate samples with different semantic information. The loss function is defined as

[0225]

[0226] Here, sim(·,·) is the cosine similarity and τ is the temperature hyperparameter, which is set to 0.1.

[0227] Before pre-training, the present invention uses graphormer-base-pcqmv1 (the pre-trained weights of Graphormer on the PCQM4M-LSC dataset) to initialize the weights of Graphormer, and uses the pre-trained weights of KV-PLM to initialize the weights of Bert.

[0228] In the pre-training stage, the present invention uses the AdamW optimizer with a learning rate of 1e-4 and a weight decay of 1e-5 to pre-train MolPrompt. The number of pre-training rounds is 30 and the batch size is set to 16.

[0229] The fourth step is to apply the pre-trained model to complete downstream tasks:

[0230] The pre-trained model can be applied to a wide range of downstream tasks, such as molecular property prediction and molecule-text cross-modal retrieval.

[0231] When the downstream task is molecular attribute prediction (such as molecular liver toxicity, neurotoxicity, anti-HIV activity, etc.), the present invention uses 8 molecular attribute data sets (BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV and BACE) to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecular attribute prediction. Specifically, the present invention inputs the molecular graph to be tested into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs a fixed-length molecular graph feature representation; the molecular graph feature representation is input into the prediction head (linear layer), and the prediction head outputs the molecular attribute. Depending on the prediction task, different prediction heads can be used. Therefore, MolPrompt can complete binary or multi-classification tasks.

[0232] When the downstream task is molecule-text cross-modal retrieval (i.e., given a molecule, retrieve its matching textual description; given a textual description, retrieve molecules matching its description) in a dataset containing a series of molecules and text, the pre-trained MolPrompt model is fine-tuned using the PCDes dataset to obtain a fine-tuned MolPrompt model for this downstream task. Molecular graph-text cross-modal retrieval consists of two subtasks: M2T and T2M. M2T refers to retrieving the matching textual description of a molecule given a molecular graph; T2M refers to retrieving the matching molecular graph given a molecular description text. In the M2T task, the molecular graph is fed into the fine-tuned MolPrompt model's molecular graph structure encoder, which then outputs a molecular graph feature representation. Simultaneously, all molecular description texts are fed into the fine-tuned MolPrompt model's text encoder, which then outputs feature representations of all molecular description texts. The cosine similarity between the molecular graph feature representation and all molecular description text feature representations is calculated and sorted. The molecular description text with the largest cosine similarity is the best matching molecular description text. The T2M task is the opposite.

[0233] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multimodal drug molecule prediction method based on knowledge hints, characterized by: The specific process of the method is: Step 1: Build the MolPrompt model; Step 2: Pre-train the MolPrompt model to obtain a pre-trained MolPrompt model; Step 3: Based on the downstream task type, fine-tune the pre-trained MolPrompt model to obtain the fine-tuned MolPrompt model; Step 4: Predict downstream tasks based on the fine-tuned MolPrompt model; In step 1, the MolPrompt model is constructed; the specific process is: Step 1: Construct knowledge prompts; the specific process is: Step 1: Use the RDkit tool to extract 209 numerical molecular descriptors from the molecular graph, and select 10 numerical molecular descriptors from the 209 numerical molecular descriptors as prior knowledge of the external chemical and physical properties of the molecules; The extracted 10 numerical type molecular descriptors are uniformly set to the format Descriptor1:Value1,Descriptor2:Value2,Descriptor3:Value3,Descriptor4:Value4,Descriptor5:Value5,Descriptor6:Value6,Descripto7:Value7,Descripto8:Value8,Descripto9:Value9,Descripto10:Value10; The 10 numerical molecular descriptors after unified format are text-type knowledge prompts; Step 112: The 10 numerical molecular descriptors in a unified format are input into the word embedding layer of BERT, and the word embedding layer of BERT outputs knowledge hints in, represents the first word feature embedding in the knowledge prompt, Represents the second word feature embedding in the knowledge prompt, Indicates the Nth p word-unit feature embedding, N p Indicates the length of the text, d p Represents the dimension of BERT's word embedding layer; Step 1 and 2: Based on the knowledge hint, the molecular graph is input into the molecular graph structure encoder to obtain the molecular graph feature representation: The specific process is as follows: Step 121: Input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs the molecular graph node feature H 0 , The molecular graph node feature H 0 The last atomic feature in is replaced by the class tag h p , in, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the n-1th atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, n represents the number of atoms in the molecular graph, d g Represents the feature dimension of the molecular graph structure encoder; Step 122: Replace the node features of the molecular graph The first Graphormer layer, the second Graphormer layer, the third Graphormer layer, the fourth Graphormer layer, the fifth Graphormer layer, the sixth Graphormer layer, the seventh Graphormer layer, the eighth Graphormer layer, the ninth Graphormer layer, the tenth Graphormer layer, the eleventh Graphormer layer, and the twelfth Graphormer layer in the molecular graph structure encoder are input in sequence, and the twelfth Graphormer layer outputs the molecular graph feature representation H 12 ; Step 13: Input the molecular description text corresponding to the molecular graph in step 1 and 2 into the text encoder to obtain the molecular description text feature representation.

2. The multimodal drug molecule prediction method based on knowledge hints according to claim 1, characterized in that: The 10 molecular descriptors of numerical type are MolWt, MolLogP, NumRotatableBonds, TPSA, NumHDonors, NumHAcceptors, RingCount, NumAromaticRings, FractionCSP3 and BalabanJ; The MolWt is the molecular weight; MolLogP is the molecular logP; NumRotatableBonds is the number of rotatable bonds; TPSA is the topological polar surface area; NumHDonors is the number of hydrogen bond donors; NumHAcceptors is the number of hydrogen bond acceptors; RingCount is the number of rings; NumAromaticRings is the number of aromatic rings, FractionCSP3 is the fraction of carbon atoms with three bonds to hydrogen, and BalabanJ is Balaban's J index; Each value in Value1, Value2, Value3, Value4, Value5, Value6, Value7, Value8, Value9, and Value10 is rounded to one decimal place; Descriptor1:Value1 represents the molecule descriptor 1:value1 of the numeric type; Descriptor2:Value2 represents the molecule descriptor 2:value 2 of the numeric type; Descriptor3:Value3 represents the molecule descriptor 3:value 3 of the numeric type; Descriptor4:Value4 represents the molecule descriptor 4:value 4 of the numeric type; Descriptor5:Value5 represents the molecule descriptor 5:value 5 of the numeric type; Descriptor6:Value6 represents the molecule descriptor 6:value 6 of the numeric type; Descripto7:Value7 represents the molecular descriptor 7 of the numeric type: value 7; Descriptor8:Value8 represents a molecule descriptor of a numerical type 8:value8; Descripto9:Value9 represents the molecular descriptor 9 of the numerical type: value 9; Descripto10:Value10 represents the molecule descriptor 10:value10 of the numeric type.

3. The multimodal drug molecule prediction method based on knowledge hints according to claim 2, characterized in that: The processing process of each of the 1st Graphormer layer, the 2nd Graphormer layer, the 3rd Graphormer layer, the 4th Graphormer layer, the 5th Graphormer layer, the 6th Graphormer layer, the 7th Graphormer layer, the 8th Graphormer layer, the 9th Graphormer layer, the 10th Graphormer layer, the 11th Graphormer layer and the 12th Graphormer layer is as follows: H i-1 Input the i-th Graphormer layer, the i-th Graphormer layer outputs the molecular graph feature representation H i ; The formula is as follows: Where K g Indicates the total number of Graphormer layers, K g =12;H i-1 Represents the molecular graph feature representation output by the i-1th Graphormer layer; represents the i-th Graphormer layer, H i Represents the molecular graph feature representation output by the i-th Graphormer layer.

4. The multimodal drug molecule prediction method based on knowledge hints according to claim 3, characterized in that: In step 13, the molecular description text corresponding to the molecular graph in step 12 is input into the text encoder to obtain the molecular description text feature representation; the specific process is: Step 131: Input the molecular description text into the BERT word embedding layer of the text encoder, and the BERT word embedding layer of the text encoder outputs the molecular description text feature representation in, Indicates that the molecular description text feature represents the first word feature embedding in W0, Indicates that the molecular description text feature represents the second word feature embedding in W0, Indicates the molecular description text feature representation of the Nth t word-unit feature embedding, N t Indicates the length of the molecular description text, d t Represents the dimension of the text encoder; Step 132: Input the molecular description text feature representation W0 output by the BERT word embedding layer of the text encoder into the 1st Transformer layer, 2nd Transformer layer, 3rd Transformer layer, 4th Transformer layer, 5th Transformer layer, 6th Transformer layer, 7th Transformer layer, 8th Transformer layer, 9th Transformer layer, 10th Transformer layer, 11th Transformer layer, and 12th Transformer layer of the text encoder in sequence. The 12th Transformer layer outputs the molecular description text feature representation W 12 .

5. The multimodal drug molecule prediction method based on knowledge hints according to claim 4, characterized in that: The processing process of each Transformer layer among the first Transformer layer, the second Transformer layer, the third Transformer layer, the fourth Transformer layer, the fifth Transformer layer, the sixth Transformer layer, the seventh Transformer layer, the eighth Transformer layer, the ninth Transformer layer, the tenth Transformer layer, the eleventh Transformer layer, and the twelfth Transformer layer is as follows: The molecular description text feature representation W output by the i-1th Transformer layer i-1 Input to the i-th Transformer layer, the i-th Transformer layer outputs the molecular description text feature representation W i ; The formula is as follows: In the above formula, K t Indicates the total number of Transformer layers, K t =12;W i Indicates the output of the molecular description text feature representation of the i-th layer Transformer layer; represents the i-th Transformer layer; W i-1 Represents the molecular description text feature representation output by the i-1th Transformer layer.

6. The multimodal drug molecule prediction method based on knowledge hints according to claim 5, characterized in that: In step 2, the MolPrompt model is pre-trained to obtain a pre-trained MolPrompt model. The specific process is as follows: Step 21: Initialize the weights of Graphormer and Bert; Step 22: Get N molecular image-text pair data {(g1, t1), (g2, t2), …, (g N ,t N )}; The molecular graph structure g b Same molecule description text as t b Constitute a positive pair (g b ,t b ), molecular graph structure g b With different molecular description text t j Constitute a negative pair (g b ,t j ) b≠j ; N represents the total number of molecular image-text pairs; b = 1, 2, ..., N, j = 1, 2, ..., N, b ≠ j; N molecular graph-text pair data {(g1,t1),(g2,t2),…,(g N ,t N )} is divided into M groups, each group contains a picture-text pair data; Step 2 and 3: Compare the molecular graph structure g in each group of molecular graph data i Input molecular graph structure encoder, molecular graph structure encoder outputs a molecular graph feature representation h K ; The molecular image and text of each group are paired with a molecular description text t in the data. i Input text encoder, the text encoder outputs a molecular description text feature representation w K ; Step 24: Represent a molecular graph feature as h K Input a multi-layer perceptron MLP, which outputs a projected molecular graph feature representation; Represent a molecule description text feature w K Input a multi-layer perceptron MLP, which outputs a projected molecular description text feature representation; The molecular graph feature after the b-th projection is expressed as The molecular description text feature after the b-th projection is expressed as As shown in the following formula: Step 25: Define the loss function as Where sim(·,·) is the cosine similarity; τ is the temperature hyperparameter, set to 0.1; is the molecular graph structure-text contrast loss; is the text-molecule graph structure contrast loss; L is the contrastive learning loss function; Set the learning rate to 1×10 -4 , the weight decay is 1×10 -5 , the optimizer is AdamW optimizer, and the batch size is set to 16; Repeat steps 22 to 25 for 30 rounds to obtain the pre-trained MolPrompt model.

7. The multimodal drug molecule prediction method based on knowledge hints according to claim 6, characterized in that: In step 3, the pre-trained MolPrompt model is fine-tuned based on the downstream task type to obtain a fine-tuned MolPrompt model. The specific process is as follows: When the downstream task is molecular property prediction, the pre-trained MolPrompt model is fine-tuned using 8 molecular property datasets to obtain a fine-tuned MolPrompt model for the downstream task of molecular property prediction. The eight molecular attribute datasets are BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE; When the downstream task is molecule-text cross-modal retrieval, the PCDes dataset is used to fine-tune the pre-trained MolPrompt model to obtain a fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval.

8. The multimodal drug molecule prediction method based on knowledge hints according to claim 7, characterized in that: In step 4, the downstream task is predicted based on the fine-tuned MolPrompt model; the specific process is as follows: 1) When the downstream task is molecular property prediction, the MolPrompt model fine-tuned for the downstream task of molecular property prediction is used to predict molecular properties; the specific process is as follows: The molecular graph to be tested is input into the molecular graph structure encoder of the fine-tuned MolPrompt model for the downstream task of molecular attribute prediction. The molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; the molecular graph feature representation is input into the prediction head, and the prediction head outputs the molecular attributes; The prediction head is a linear layer; 2) When the downstream task is molecular-text cross-modal retrieval, the MolPrompt model fine-tuned for the downstream task of molecular-text cross-modal retrieval is used to perform molecular graph-text cross-modal retrieval; the specific process is as follows: Molecular graph-text cross-modal retrieval includes two subtasks: M2T and T2M; M2T means that given a molecular graph, the molecular description text that matches the molecular graph is retrieved; T2M means that given a molecular description text, a molecular graph that matches the molecular description text is retrieved; The molecular graph-text to be retrieved is input into the fine-tuned MolPrompt model for the downstream task of molecule-text cross-modal retrieval. The fine-tuned MolPrompt model outputs the molecular graph feature representation and the molecular description text feature representation; In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs the molecular graph feature representation; At the same time, all molecular description texts are input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the feature representation of all molecular description texts; Calculate the cosine similarity between the molecular graph feature representation and the feature representation of all molecular description texts and sort them. The molecular description text with the largest cosine similarity is the best matching molecular description text; In the T2M task, the molecular description text is input into the text encoder of the fine-tuned MolPrompt model, and the text encoder of the fine-tuned MolPrompt model outputs the molecular description text feature representation; At the same time, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned MolPrompt model, and the molecular graph structure encoder of the fine-tuned MolPrompt model outputs all molecular graph feature representations; The cosine similarities between the feature representations of the molecular description text and the feature representations of all molecular graphs are calculated and sorted. The molecular graph with the largest cosine similarity is the best matching molecule.

Images