Multimodal drug molecule prediction method based on missing modality generation

Abstract

The application belongs to the field of artificial intelligence assisted drug research and development, and particularly relates to a multi-modal drug molecule prediction method. The application aims to solve the problems that the scale of a multi-modal molecule-text pre-training data set is limited by the missing of a molecule description text mode, a model is difficult to fully capture the complex relationship between a molecule graph structure and a text, and the existing multi-modal molecule pre-training model has certain deficiencies in the performance on a downstream task. The specific process is as follows: 1, constructing a multi-modal molecule pre-training model; 2, pre-training the multi-modal molecule pre-training model to obtain a pre-trained multi-modal molecule pre-training model; 3, based on the type of a downstream task, fine-tuning the pre-trained multi-modal molecule pre-training model to obtain a fine-tuned multi-modal molecule pre-training model; and 4, predicting the downstream task based on the fine-tuned multi-modal molecule pre-training model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence assisted drug development, and particularly relates to a multi-modal drug molecule prediction method. BACKGROUND

[0002] Understanding the properties and functions of molecules is crucial for drug development, providing a basis for the development of targeted therapies, the elucidation of disease mechanisms, and the development of personalized medicine. Traditional drug discovery relies on laboratory experiments, which require a large amount of time and economic cost. In recent years, artificial intelligence has played an increasingly important role in assisting drug development, with significant improvements in efficiency and cost-effectiveness. Researchers have applied self-supervised learning strategies to molecular representation learning, training models to understand various molecular representations, such as molecular SMILES strings and molecular graph structures. These models aim to learn pre-trained models from a large amount of unlabeled data to support various molecular tasks, such as molecular property prediction and molecule generation. According to the different input modalities, current molecular pre-training models can be divided into two categories, namely single-modal pre-training models and multi-modal pre-training models. Compared with single-modal molecular pre-training models, multi-modal molecular pre-training models can integrate information from multiple different modalities, have stronger generalization ability, and are suitable for a wider range of downstream tasks. In reality, humans have the ability to understand and learn knowledge from multiple perspectives, and can recognize different forms of molecules by combining molecular graph structures, molecular SMILES strings, and molecular description texts. Similarly, existing research work explores multi-modal molecular-text pre-training models to learn molecular representations by combining text knowledge.

[0003] However, the existing multi-modal molecular pre-training models still have certain deficiencies in the performance on downstream tasks. First, compared with single-modal molecular pre-training datasets, multi-modal molecular-text pre-training datasets are small in size. The main challenge is the difficulty in obtaining high-quality molecular description text data, which is due to the need for professional knowledge in molecular annotation process, making large-scale manual annotation both expensive and tedious. Therefore, there is a significant gap in data size between molecular description text data and molecular SMILES strings, molecular graph structures. Researchers have used large language models (LLM) to generate pseudo-text data to address the challenge of missing molecular description text data. This method has achieved certain results, but the generation efficiency of pseudo-text data is low, and it requires a certain amount of time and economic cost. Another notable problem is that existing multi-modal molecular graph structure-text pre-training models either ignore and waste different levels of semantic knowledge contained in different layers of single-modal encoders, or are difficult to achieve effective cross-modal alignment. SUMMARY

[0004] The present application aims to solve the problem that the scale of the multi-modal molecule-text pre-training dataset is limited by the lack of the molecular description text modality, the model is difficult to fully capture the complex relationship between the molecular graph structure and the text, and the existing multi-modal molecular pre-training model has certain deficiencies in the performance on the downstream task, and proposes a multi-modal drug molecule prediction method based on missing modality generation.

[0005] The specific process of the multi-modal drug molecule prediction method based on missing modality generation is as follows:

[0006] Step one: constructing a multi-modal molecular pre-training model MolBT;

[0007] Step two: pre-training the multi-modal molecular pre-training model MolBT to obtain the pre-trained multi-modal molecular pre-training model MolBT;

[0008] Step three: based on the type of downstream task, fine-tuning the pre-trained multi-modal molecular pre-training model MolBT to obtain the fine-tuned multi-modal molecular pre-training model MolBT;

[0009] Step four: predicting the downstream task based on the fine-tuned multi-modal molecular pre-training model MolBT.

[0010] The present application has the following beneficial effects:

[0011] The present application proposes a multi-modal molecular pre-training model based on missing modality generation, MolBT. MolBT explores a method to solve the problem of lack of molecular description text modality, that is, generating shallow feature representation of the missing text modality using existing modalities. This method significantly expands the scale of the multi-modal molecular pre-training dataset. At the same time, we emphasize the use of different levels of semantic knowledge contained in different layers of single modality encoders. Through BridgeLayer, we establish a connection between the top layers of single modality encoders and each layer of cross-modality encoders, thereby optimizing the use of multiple single modality features and promoting effective bottom-up cross-modality interaction and fusion between molecular graph structure representation, text representation, and molecular SMILES string modality. MolBT exhibits excellent generalization ability in a series of downstream tasks. MolBT performs outstandingly in molecular-text cross-modality retrieval. In the molecular property prediction task, MolBT performs outstandingly, highlighting its excellent performance in physiological and biophysical related classification tasks. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 It is a multi-modal drug molecule pre-training model architecture diagram based on missing modality generation. DETAILED DESCRIPTION

[0013] Specific implementation method one: the specific process of the multi-modal drug molecule prediction method based on the missing modal generation in the embodiment is:

[0014] The application constructs a new multi-modal molecule-text pre-training model, MolBT. MolBT is composed of a molecular graph structure encoder, a sequence encoder and a cross-modal encoder. Specifically, we use Graphormer as the molecular graph structure encoder. Since the molecular description text and the molecular SMILES string are both sequence type data, we use BERT as the molecular sequence encoder to process these two modal data. We establish a cross-modal encoder as a bridge connecting the molecular graph structure encoder and the molecular sequence encoder, so that the molecular graph structure feature representation, the text feature representation and the molecular SMILES feature representation can realize effective bottom-up cross-modal interaction and fusion. In addition, a missing modal generation module based on the normalized flow model is introduced in MolBT. This method generates the shallow feature representation of the missing text modal using the existing modal (i.e. the molecular graph structure and the molecular SMILES string). During the pre-training process, the generated feature representation of the missing text modal greatly expands the size of the multi-modal molecule-text pre-training dataset.

[0015] Step one: constructing a multi-modal molecule pre-training model MolBT;

[0016] Step two: pre-training the multi-modal molecule pre-training model MolBT to obtain the pre-trained multi-modal molecule pre-training model MolBT;

[0017] Step three: based on the type of downstream task, fine-tuning the pre-trained multi-modal molecule pre-training model MolBT to obtain the fine-tuned multi-modal molecule pre-training model MolBT;

[0018] Step four: predicting the downstream task based on the fine-tuned multi-modal molecule pre-training model MolBT.

[0019] Specific implementation method two: the difference between the embodiment and the specific implementation method one is that the multi-modal molecule pre-training model MolBT is constructed in step one: the specific process is:

[0020] Step one, input the molecular graph into the molecular graph structure encoder, and the molecular graph structure encoder outputs the molecular graph structure feature representation;

[0021] The molecular graph structure encoder is Graphormer;

[0022] Step two, input the molecular description text into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular description text feature representation;

[0023] The molecular sequence encoder is BERT.

[0024] Step III, input the molecular SMILES string into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular SMILES feature representation;

[0025] The molecular sequence encoder is BERT.

[0026] Step IV, establish a cross-modal encoder, input the molecular graph structure feature representation, the molecular description text feature representation and the molecular SMILES feature representation into the cross-modal encoder, and the cross-modal encoder outputs the molecular graph structure cross-modal feature representation and the molecular description text cross-modal feature representation.

[0027] Connect the molecular graph structure encoder and the molecular sequence encoder.

[0028] Step V, construct a missing modal generation module, input the molecular graph structure feature representation and the molecular SMILES feature representation of the missing molecular description text into the missing modal generation module, and the missing modal generation module outputs the corresponding molecular description text shallow feature representation.

[0029] The other steps and parameters are the same as those in the first embodiment.

[0030] The third embodiment is different from the first or second embodiment in that in step I-1, the molecular graph is input into the molecular graph structure encoder, and the molecular graph structure encoder outputs the molecular graph structure feature representation; the specific process is as follows:

[0031] Step III, input the molecular SMILES string into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular SMILES feature representation;

[0032] wherein, represents the first atomic feature embedding, represents the second atomic feature embedding, represents the nth atomic feature embedding, R represents a real number, and n g represents the number of atoms in the molecular graph, d g represents the feature dimension of the molecular graph structure encoder;

[0033] Step one two, input the molecular graph node feature G0 into the first layer Graphormer layer, the second layer Graphormer layer, the third layer Graphormer layer, the fourth layer Graphormer layer, the fifth layer Graphormer layer, the sixth layer Graphormer layer, the seventh layer Graphormer layer, the eighth layer Graphormer layer, the ninth layer Graphormer layer, the tenth layer Graphormer layer, the eleventh layer Graphormer layer, the twelfth layer Graphormer layer of the molecular graph structure encoder in turn, and the twelfth layer Graphormer layer outputs the molecular graph structure feature representation G 12 ;

[0034] The formula is as follows:

[0035]

[0036] In the formula, K g represents the total number of layers of the Graphormer layer in the molecular graph structure encoder, K g = 12.

[0037] represents the i-th layer Graphormer layer;

[0038] G i-1 represents the molecular graph structure feature output by the i-1-th layer Graphormer layer;

[0039] G i represents the molecular graph structure feature output by the i-th layer Graphormer layer.

[0040] The other steps and parameters are the same as those in embodiment one or two.

[0041] Embodiment four: different from one of embodiments one to three, in the step one two, the molecular description text is input into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular description text feature representation; the specific process is as follows:

[0042] Step one two one, input the molecular description text into the BERT word embedding layer of the molecular sequence encoder, and the BERT word embedding layer of the molecular sequence encoder outputs the molecular description text feature T0;

[0043] Step 122, the molecular description text feature T0is input into the first layer of the molecular sequence encoder, the second layer of the molecular sequence encoder, the third layer of the molecular sequence encoder, the fourth layer of the molecular sequence encoder, the fifth layer of the molecular sequence encoder, the sixth layer of the molecular sequence encoder, the seventh layer of the molecular sequence encoder, the eighth layer of the molecular sequence encoder, the ninth layer of the molecular sequence encoder, the tenth layer of the molecular sequence encoder, the eleventh layer of the molecular sequence encoder, and the twelfth layer of the molecular sequence encoder in sequence, and the twelfth layer of the molecular sequence encoder outputs the molecular description text feature representation T 12 ;

[0044] The formula is as follows:

[0045]

[0046] In the formula, K s represents the total number of layers of the Transformer layer in the molecular sequence encoder, K s = 12.

[0047] represents the i-th layer of the Transformer layer;

[0048] T i-1 represents the molecular description text feature output by the i-1-th layer of the Transformer layer;

[0049] T i represents the molecular description text feature output by the i-th layer of the Transformer layer.

[0050] The other steps and parameters are the same as one of the first to third embodiments.

[0051] Embodiment Five: The difference between this embodiment and one of the first to fourth embodiments is that in step 113, the molecular SMILES string is input into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular SMILES feature representation; the specific process is as follows:

[0052] Step 1131, the molecular SMILES string is input into the BERT word embedding layer of the molecular sequence encoder, and the BERT word embedding layer of the molecular sequence encoder outputs the molecular SMILES feature S0;

[0053] Step one three two, input the molecular SMILES feature S0 into the first layer of the molecular sequence encoder, the second layer of the molecular sequence encoder, the third layer of the molecular sequence encoder, the fourth layer of the molecular sequence encoder, the fifth layer of the molecular sequence encoder, the sixth layer of the molecular sequence encoder, the seventh layer of the molecular sequence encoder, the eighth layer of the molecular sequence encoder, the ninth layer of the molecular sequence encoder, the tenth layer of the molecular sequence encoder, the eleventh layer of the molecular sequence encoder, and the twelfth layer of the molecular sequence encoder in sequence, and the twelfth layer of the molecular sequence encoder outputs the molecular SMILES feature representation S 12 ;

[0054] The formula is as follows:

[0055]

[0056] In the formula,

[0057] K s represents the total number of layers of the Transformer layer in the molecular sequence encoder, K s = 12.

[0058] represents the i-th layer of the Transformer layer;

[0059] S i-1 represents the molecular SMILES feature output by the i-1-th layer of the Transformer layer;

[0060] S i represents the molecular SMILES feature output by the i-th layer of the Transformer layer.

[0061] Since the molecular SMILES string and the molecular description text both belong to sequence type data, the molecular SMILES string and the molecular description text can share an encoder.

[0062] The other steps and parameters are the same as one of the first to fourth embodiments.

[0063] Embodiment six: The difference between this embodiment and one of the first to fifth embodiments is that in step one four, the cross-modal encoder is established, the molecular graph structure feature representation, the molecular description text feature representation, and the molecular SMILES feature representation are input into the cross-modal encoder, and the cross-modal encoder outputs the molecular graph structure cross-modal feature representation and the molecular description text cross-modal feature representation; the specific process is as follows:

[0064] The G7 output by the 7th layer of the molecular graph structure encoder, the T7 and S7 output by the 7th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0065] The G8 output by the 8th layer of the molecular graph structure encoder, the T8 and S8 output by the 8th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0066] The G9 output by the 9th layer of the molecular graph structure encoder, the T9 and S9 output by the 9th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0067] The G 10 output by the 10th layer of the molecular graph structure encoder, the T 10 and S 10 output by the 10th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0068] The G 11 output by the 11th layer of the molecular graph structure encoder, the T 11 and S 11 output by the 11th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0069] The G 12 output by the 12th layer of the molecular graph structure encoder, the T 12 and S 12 output by the 12th layer of the molecular sequence encoder are input into the cross-modal encoder, and the cross-modal encoder outputs C G T

[0070] The number of layers of the cross-modal encoder is 6, and each layer comprises a self-attention layer, a common attention layer and a feed-forward layer (FFN) in sequence.

[0071] The operation process of each layer in the cross-modal encoder is as follows:

[0072]

[0073] In the formula, CME l represents the lth layer of the cross-modal encoder.

[0074] ​​​​​​​​​​​​Each layer of the cross-modal encoder comprises, in sequence, a self-attention layer, a co-attention layer, and a feed-forward layer FFN;

[0075] and respectively represent the molecular graph structure cross-modal feature representation and the molecular text cross-modal feature representation output by the lth layer of the cross-modal encoder;

[0076] L C is the number of layers of the cross-modal encoder, L C = 6;

[0077] represents the molecular graph structure feature representation output by the l-1th layer of the cross-modal encoder;

[0078] represents the text feature representation output by the l-1th layer of the cross-modal encoder;

[0079] represents the molecular SMILES feature representation input to the lth layer of the cross-modal encoder;

[0080] The cross-modal encoder based on the Transformer architecture adopts a co-attention mechanism;

[0081] The specific process is as follows:

[0082] 1) The molecular graph structure feature representation G is processed by the BridgeLayer to obtain the input to the lth layer of the cross-modal encoder G

[0083] The text feature representation T is processed by the BridgeLayer to obtain the input to the lth layer of the cross-modal encoder T

[0084] is represented as:

[0085]

[0086] wherein

[0087]

[0088] In the formula,

[0089] W G represents a molecular graph structure feature projection matrix (MLP); W T represents a text feature projection matrix (MLP);

[0090] G type represents the molecular graph structure feature representation G kT type T k T

[0091] G k T k T

[0092] LN

[0093] LN

[0094] 2) Feature representation based on molecular SMILES string Class label, denoted as

[0095]

[0096] wherein, T k T T k T T k T s T T k T s T s T

[0097] 3) Replace the last node feature in and with to obtain and as follows:

[0098]

[0099] wherein, T

[0100] express The first atomic feature embedding in; express The second atomic feature is embedded in; express The (n-1)th atom feature is embedded in the matrix;

[0101] This represents the cross-modal feature representation of the molecular description text after feature replacement;

[0102] express The first lexical feature embedding in; express The second lexical feature embedding in; express The embedding of the (n-1)th lexical feature;

[0103] n g Indicates the number of atoms in the molecular diagram; d g n represents the feature dimension of the molecular graph structure encoder; t Indicates the length of the molecular description text; d t This represents the feature dimension of the text encoder;

[0104] 4) The molecular graph structure portion and molecular text portion are input into the l-th layer of the cross-modal encoder to obtain... and

[0105]

[0106] In the formula, The molecular graph structure of the output of the l-th layer of the cross-modal encoder is represented by its cross-modal feature representation. Indicates

[0107] Transmodal feature representation of the processed molecular graph structure;

[0108] This represents the text cross-modal feature representation output by the l-th layer of the cross-modal encoder; Indicates Processed cross-modal feature representation of molecular description text;

[0109] Indicates the replacement Transmodal feature representation of the processed molecular graph structure; Indicates the replacement Processed cross-modal feature representation of molecular description text;

[0110] CME l denotes the l-th layer of the cross-modal encoder;

[0111] Each layer of the cross-modal encoder sequentially comprises a self-attention layer, a co-attention layer, and a feed-forward layer FFN.

[0112] The other steps and parameters are the same as one of the first to fifth embodiments.

[0113] Embodiment seven: different from one of the first to sixth embodiments, the step one five constructs a missing modality generation module, inputs the molecular graph and the molecular SMILES string of the missing molecular description text into the missing modality generation module, and the missing modality generation module outputs the corresponding molecular description text; the specific process is as follows:

[0114] In order to reduce the distribution gap between the recovered data and the real data, we construct a missing modality generation module to learn the latent distribution space of each modality, and perform cross-modal distribution transfer to estimate the distribution of the missing text modality, and finally recover the shallow feature representation of the missing text modality through a projection network. In order to convert the cross-modal distribution, we introduce a modality-related flow, and bridge different modalities within the embedded distribution space. In order to facilitate distribution transfer, we use a reversible modality-specific normalization flow for each modality, a classic generative probability model, to map the features of different modalities into a latent space with a Gaussian distribution, reducing the distribution gap between modalities. Our main idea is to transfer the distribution from the observed modality to the missing modality through a cross-flow, and produce more confident predictions with higher distribution consistency.

[0115] Step one five one, the molecular graph structure feature representation G6 obtained after the first 6 layers of the molecular graph structure encoder, the molecular description text feature representation T6 obtained after the first 6 layers of the molecular sequence encoder, and the molecular SMILES feature representation S6 are denoted as shallow feature representations, i.e. {G6, T6, S6};

[0116] Step one five two, the shallow feature representations G6, T6, and S6 are respectively projected into the same dimensional space through a one-dimensional convolution layer to obtain Z (m) ∈R T×d ,m∈{t,g,s};

[0117] In the formula, denotes Z (t) , Z (g) , or Z (s) ;

[0118] Z (t) denotes the feature output by the one-dimensional convolution layer after the shallow feature representation T6; Z (g)denotes the feature output by the shallow feature representation G6 after passing through a one-dimensional convolutional layer; Z (s) denotes the feature output by the shallow feature representation S6 after passing through a one-dimensional convolutional layer;

[0119] T denotes the length of the feature after passing through a one-dimensional convolutional layer; d denotes the dimension of the feature after passing through a one-dimensional convolutional layer;

[0120] Step one five three, the subsequent missing modal recovery task aims to utilize Z (g) ,Z (s) to generate the missing molecular description text modality shallow feature representation;

[0121] Let denote the normalized flow model corresponding to modality m, denote its inverse transformation;

[0122] Z (g) and Z (s) are input into the corresponding normalized flow model (Z (g) is input into the normalized flow model , and Z (s) is input into the normalized flow model ), to obtain latent feature representations with the same Gaussian distribution;

[0123] In the case of missing molecular description text modality, the molecular graph structure feature representation G6 obtained by the first 6 layers of the molecular graph structure encoder and the molecular SMILES feature representation S6 obtained by the first 6 layers of the molecular sequence encoder are utilized, Z (g) and Z (s) are input into the normalized flow models and to obtain the feature distributions Y (g) and Y (s) of the molecular graph structure and the molecular SMILES string, respectively;

[0124]

[0125] In the formula, Y (g) denotes the molecular graph structure feature distribution obtained by the normalized flow model;

[0126] Y (s) denotes the molecular SMILES string feature distribution obtained by the normalized flow model;

[0127] Step one five four, Y (g) and Y (s) are averaged to sample the latent text representation denoted as:

[0128]

[0129] Y (g) ~ N(μ c ,∑ c ),Y (s) ~ N(μ c ,∑ c )

[0130] In the formula, N(μ c ,∑ c ) represents the standard normal distribution after the molecular graph structure features and the molecular SMILES string features are projected to the shared latent space by the normalized flow model; μ c represents the distribution center;∑ c represents the covariance; and

[0131] Step one five five, input to generate the formula as follows:

[0132]

[0133] In the formula, represents inverse transformation, the normalized flow model; represents the missing text feature distribution generated by the inverse transformation of the normalized flow model;

[0134] Step one five six, input to the projection network MLP to obtain the shallow feature representation of the missing text modality;

[0135]

[0136] In the formula, Proj (t) () represents the feature projection network MLP.

[0137] The other steps and parameters are the same as one of the first to fifth embodiments.

[0138] Embodiment eight: different from one of the first to seventh embodiments, the step two is to pre-train the multi-modal molecular pre-training model MolBT to obtain the pre-trained multi-modal molecular pre-training model MolBT; the specific process is as follows:

[0139] Step three one, construct the missing modality reconstruction loss, defined as follows:

[0140]

[0141] In the formula, Lrec denotes the missing modality reconstruction loss; T k denotes the real text shallow feature representation (the molecular graph structure feature representation G6 obtained by the real molecular graph passing through the first 6 layers of the molecular graph structure encoder, the molecular description text feature representation T6 obtained by the real molecular description text passing through the first 6 layers of the molecular sequence encoder, and the molecular SMILES feature representation S6 obtained by the real molecular SMILES string passing through the first 6 layers of the molecular sequence encoder, denoted as shallow feature representation, i.e., {G6, T6, S6});

[0142] Step three two, construct the contrastive loss function; the specific process is as follows:

[0143] Step three two one, let {(g1, t1), (g2, t2), …, (g N , N )} be a batch of molecular-text pairs, N = 16;

[0144] Among them, the molecule g i and the corresponding molecular description text t i constitute the positive pair (g i , t i );

[0145] The molecule g i and the molecular description text t j of different molecules constitute the negative pair (g i , t j ) i≠j ;

[0146] The molecule g i denotes the molecular graph and the molecular SMILES string;

[0147] Step three two two, use the molecular graph structure encoder and the molecular sequence encoder to obtain the molecular graph structure representation g K , the molecular description text representation t K and the molecular SMILES string representation s K ;

[0148] g K , t K and s K are input into the cross-modal encoder to obtain the cross-modal feature representation and

[0149] Use a multi-layer perceptron (MLP) as a projection network to project the four representations g K , t K , into the same dimensional feature space; the formula is as follows:

[0150]

[0151]

[0152] wherein z G denotes the projected molecular graph structure feature representation; z T denotes the projected molecular description text feature representation; c G denotes the projected molecular graph structure cross-modal feature representation; c T denotes the projected molecular description text cross-modal feature representation;

[0153] GraphProj() denotes a molecular graph structure feature projection network; TextProj() denotes a molecular description text feature projection network;

[0154] GraphProj'() denotes a molecular graph structure cross-modal feature projection network; TextProj'() denotes a molecular description text cross-modal feature projection network;

[0155] d g denotes the dimension of the molecular graph structure feature; d t denotes the dimension of the molecular description text feature;

[0156] A multi-modal contrastive loss is adopted, aiming to bring different modal samples with the same semantic information close in the feature space, while pushing samples with different semantic information apart;

[0157] The contrastive loss adopted includes three contrastive losses between four kinds of feature representations, i.e., (z G ,z T ), (z G ,c G ) and (c G ,c T );

[0158] Step three two three, the contrastive loss function l1 between (z G ,z T ) is calculated by using the contrastive loss; the specific process is as follows:

[0159] Let the projected feature representation of the i-th molecular graph structure be the projected feature representation of the j-th molecular graph structure be the projected feature representation of the molecular description text of the same molecule as the i-th molecular graph structure g i be the projected feature representation of the molecular description text t i of the different molecule from the i-th molecular graph structure g j be The formula is as follows:

[0160]

[0161] wherein,

[0162] denotes the molecular graph structure-text contrastive loss; denotes the text-molecular graph structure contrastive loss;

[0163] sim(·,·) denotes the cosine similarity;

[0164] τ is a temperature hyper-parameter, set to 0.1;

[0165] Step three two four, the contrastive loss is calculated between (z G , z G ) by using a contrastive loss function l2;

[0166] Let the projected feature representation of the i-th molecular graph structure be the projected feature representation of the j-th molecular graph structure be the projected molecular graph structure cross-modal feature representation of the i-th molecular graph structure be the projected molecular graph structure cross-modal feature representation of the i-th molecular graph structure g i different molecular g j ; The formula is as follows:

[0167]

[0168] wherein,

[0169] denotes the molecular graph structure feature-molecular graph structure cross-modal feature contrastive loss; denotes the molecular graph structure cross-modal feature-molecular graph structure feature contrastive loss;

[0170] sim(·,·) denotes the cosine similarity;

[0171] τ is a temperature hyper-parameter, set to 0.1;

[0172] Step three two five, the contrastive loss is calculated between (z G , z T ) by using a contrastive loss function l3;

[0173] Let the projected cross-modal feature representation of the i-th molecular graph structure be the projected cross-modal feature representation of the j-th molecular graph structure be the projected molecular description text cross-modal feature representation of the molecular description text of the same molecule g i ; the i-th molecular graph structure g i the projected molecular description text of the different molecules is represented as The formula is as follows:

[0174]

[0175] wherein,

[0176] denotes the molecular graph structure cross-modal feature-text cross-modal feature contrast loss; denotes the text cross-modal feature-molecular graph structure cross-modal feature contrast loss;

[0177] sim(·,·) denotes the cosine similarity;

[0178] τ is a temperature hyperparameter, set to 0.1;

[0179] Step three two six, based on the contrast loss function l1, the contrast loss function l2, the contrast loss function l3, the total contrast loss function L is constructed CL , which is represented as:

[0180] L CL =(l1+l2+l3) / 3

[0181] Step three three, based on the missing modality reconstruction loss L rec and the total contrast loss function L CL , the loss function L of the missing modality generation module is constructed mmg , which is represented as:

[0182] L mmg =αL rec +L CL

[0183] wherein, α denotes a coefficient, α=5;

[0184] Step three four, the multi-modal molecular pre-training model MolBT is trained to obtain the pre-trained multi-modal molecular pre-training model MolBT; the specific process is:

[0185] Step three four one, pre-training phase one:

[0186] The molecular graph structure encoder, the molecular sequence encoder, the cross-modal encoder, and the missing modality generation module in the multi-modal molecular pre-training model MolBT are pre-trained using real molecule-text pair data, and the loss function is L mmg ;

[0187] The molecules in the real molecule-text pair data are molecular graphs and molecular SMILES strings;

[0188] The multi-modal molecular pre-training model MolBT is pre-trained for 20 rounds using the AdamW optimizer, the learning rate is set to 1e-4, the weight decay is set to 1x10 -5 , and the batch size is set to 16;

[0189] A pre-trained multi-modal molecular pre-training model MolBT is obtained in the pre-training phase one;

[0190] Step three four two, pre-training phase two:

[0191] The molecular data (molecular graph structure, molecular SMILES string) with missing molecular description text is used;

[0192] The molecular graph structure encoder, molecular sequence encoder, cross-modal encoder, and missing modal generation module in the multi-modal molecular pre-training model MolBT are pre-trained using the molecular graph and the molecular SMILES string with missing molecular description text, the model weights saved in the pre-training phase one are applied, and the loss function is L CL ;

[0193] The multi-modal molecular pre-training model MolBT is pre-trained for 20 rounds using the AdamW optimizer, the learning rate is set to 1e-4, the weight decay is set to 1x10 -5 , and the batch size is set to 16;

[0194] A pre-trained multi-modal molecular pre-training model MolBT is obtained in the pre-training phase two;

[0195] Step three four three, pre-training phase three:

[0196] The molecular graph structure encoder, molecular sequence encoder, and cross-modal encoder in the multi-modal molecular pre-training model MolBT are pre-trained using real molecular and text pair data, the model weights saved in the pre-training phase two are applied, and the loss function is L CL ;

[0197] The molecules in the real molecular and text pair data are molecular graphs and molecular SMILES strings;

[0198] The molecular graph structure encoder, molecular sequence encoder, cross-modal encoder, and missing modal generation module in the multi-modal molecular pre-training model MolBT are pre-trained for 20 rounds using the AdamW optimizer, the learning rate is set to 1e-4, the weight decay is set to 1x10 -5 , and the batch size is set to 16;

[0199] obtain a pre-trained multi-modal molecular pre-training model MolBT (the pre-trained multi-modal molecular pre-training model MolBT includes a molecular graph structure encoder, a molecular sequence encoder, a cross-modal encoder, and a missing modal generation module).

[0200] The other steps and parameters are the same as one of embodiments 1-7.

[0201] Embodiment 9: Different from one of embodiments 1-8, in the step three, the pre-trained multi-modal molecular pre-training model MolBT is fine-tuned based on the type of downstream task to obtain a fine-tuned multi-modal molecular pre-training model MolBT; the specific process is as follows:

[0202] 1) When the downstream task is molecular property prediction, input 8 molecular property datasets into the pre-trained multi-modal molecular pre-training model MolBT to obtain a fine-tuned multi-modal molecular pre-training model MolBT;

[0203] The 8 molecular property datasets include BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV, and BACE.

[0204] 2) When the downstream task is molecular and text cross-modal retrieval, input the PCDes dataset into the pre-trained multi-modal molecular pre-training model MolBT to obtain a fine-tuned multi-modal molecular pre-training model MolBT.

[0205] The pre-trained multi-modal molecular pre-training model MolBT is applied to a wide range of downstream tasks, such as molecular property prediction, molecular-text cross-modal retrieval, etc.

[0206] The other steps and parameters are the same as one of embodiments 1-8.

[0207] Embodiment 10: Different from one of embodiments 1-9, in the step four, the fine-tuned multi-modal molecular pre-training model MolBT is used to predict the downstream task; the specific process is as follows:

[0208] 1) When the downstream task is molecular property prediction, the fine-tuned multi-modal molecular pre-training model MolBT is used to predict the molecular property based on the downstream task being molecular property prediction; the specific process is as follows:

[0209] Input the to-be-tested molecular graph into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder outputs the molecular graph structure feature representation; input the molecular graph structure feature representation into the prediction head (linear layer), and the prediction head outputs the molecular property (used to predict whether the molecule has the target property);

[0210] According to different prediction task tasks, different prediction heads can be corresponded, and therefore, the MolBT can complete a binary classification or a multi-classification task.

[0211] 2) When the downstream task is cross-modal retrieval of molecular graphs and molecular description texts, the cross-modal retrieval of molecular graphs and molecular description texts is performed based on the fine-tuned multi-modal molecular pre-training model MolBT for cross-modal retrieval of molecules and texts; a specific process is as follows:

[0212] 21) The cross-modal retrieval of molecular graphs and molecular description texts includes two sub-tasks: M2T and T2M.

[0213] M2T means that given a molecular graph, a molecular description text matching the molecular graph is retrieved.

[0214] T2M means that given a molecular description text, a molecular graph matching the molecular description text is retrieved.

[0215] 22) In the M2T task, the molecular graph is input into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs a molecular graph feature representation.

[0216] Meanwhile, all molecular description texts are input into the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs all molecular description text feature representations; the molecular SMILES string is not required for the downstream task, and the molecular SMILES string is only used in the pre-training stage.

[0217] The cosine similarity between the molecular graph feature representation and all molecular description text feature representations is calculated and ranked, and the molecular description text with the maximum cosine similarity is the best matching molecular description text.

[0218] 23) In the T2M task, the molecular description text is input into the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs a molecular description text feature representation; the molecular SMILES string is not required for the downstream task, and the molecular SMILES string is only used in the pre-training stage.

[0219] Meanwhile, all molecular graphs are input into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs all molecular graph feature representations.

[0220] The cosine similarity between the molecular description text feature representation and all the molecular graph feature representations is calculated respectively and ranked, and the molecular graph with the largest cosine similarity is the best matching molecule.

[0221] Other steps and parameters are the same as one of the first to ninth embodiments.

[0222] The present application can also have other various embodiments, and those skilled in the art can make various corresponding changes and modifications according to the present application without departing from the spirit and essence of the present application, but these corresponding changes and modifications should all belong to the protection scope of the claims attached to the present application.

Claims

1. A method for multi-modal drug molecule prediction based on missing modality generation, characterized in that: The method specifically comprises the following steps: Step one: constructing a multi-modal molecular pre-training model MolBT; Step two: pre-training the multi-modal molecular pre-training model MolBT to obtain a pre-trained multi-modal molecular pre-training model MolBT; the specific process is as follows: Step three one, constructing a missing modality reconstruction loss, which is defined as follows: In the formula, L rec denotes the missing modal reconstruction loss; T k denotes the real text shallow feature representation; Step three two, constructing a contrastive loss function; the specific process is as follows: Step three two one, let {(g1, t1), (g2, t2), …, (g N , t N )} be a batch of molecule-text pairs, N = 16; Wherein, molecule g i and the corresponding molecule description text t i constitute a direct pair (g i , t i ); molecule g i molecular description text t of different molecules j constituting negative pair (g i ,t j ) i≠j ; Molecule g i Molecule graph and SMILES string representation of molecule Step three two, obtain molecular graph structure representation g K , molecular description text representation t K , and molecular SMILES string representation s K using molecular graph structure encoder and molecular sequence encoder respectively g K 、t K and s K input to a cross-modal encoder to obtain cross-modal feature representations and Using a multi-layer perceptron, MLP, as the projection network to project the four representations g K , t K , into the same dimensional feature space; the formula is shown as follows: In the formula, z G denotes the projected molecular graph structure feature representation; z T denotes the projected molecular description text feature representation; c G denotes the projected molecular graph structure cross-modal feature representation; c T denotes the projected molecular description text cross-modal feature representation; GraphProj() represents a molecular graph structure feature projection network; TextProj() represents a molecular description text feature projection network; GraphProj'() represents a molecular graph structure cross-modal feature projection network; TextProj'() represents a molecular description text cross-modal feature projection network; d g dimension representing molecular graph structure features; d t dimension representing molecular description text features; Step three two three, calculate the contrast loss function l1 between the contrast loss (z G ,z T ) between the contrast loss function l1; the specific process is: The feature representation of the i-th molecular graph structure after projection is denoted as The feature representation of the j-th molecular graph structure after projection is denoted as The feature representation of the i-th molecular graph structure g i The feature representation of the molecular description text of the same molecule after projection is denoted as The feature representation of the i-th molecular graph structure g i The feature representation of the molecular description text t j of the different molecule after projection is denoted as The formula is as follows: Wherein, represents a molecular graph structure-text contrastive loss; represents a text-molecular graph structure contrastive loss; sim(·,·) represents a cosine similarity; τ is a temperature hyperparameter, which is set to 0.1; Steps 3-4: Calculate the contrast loss (z) G ,c G The contrast loss function l2 between ) The feature representation of the projected i-th molecular graph structure is denoted as The feature representation of the projected j-th molecular graph structure is denoted as The projected molecular graph structure cross-modal feature representation of the i-th molecular graph structure is denoted as The projected molecular graph structure cross-modal feature representation of the i-th molecular graph structure is denoted as i The projected molecular graph structure cross-modal feature representation of the different molecular g j The projected molecular graph structure cross-modal feature representation of the different molecular g The formula is as follows: Wherein, molecular graph structure feature - molecular graph structure cross-modal feature contrast loss; molecular graph structure cross-modal feature - molecular graph structure feature contrast loss; sim(·,·) represents a cosine similarity; τ is a temperature hyperparameter, which is set to 0.1; Step 325: Calculate the comparative loss (c) G ,c T The contrast loss function l3 between ) The cross-modal feature representation of the i-th molecular graph structure after projection is denoted as The cross-modal feature representation of the j-th molecular graph structure after projection is denoted as The i-th molecular graph structure g i The projected molecular description text cross-modal feature representation of the molecular description text of the same molecule is denoted as The i-th molecular graph structure g i The projected molecular description text cross-modal feature representation of the molecular description text of the different molecule is denoted as The formula is as follows: Wherein, represents a molecular graph structure cross-modal feature-text cross-modal feature contrast loss; represents a text cross-modal feature-molecular graph structure cross-modal feature contrast loss; sim(·,·) represents a cosine similarity; τ is a temperature hyperparameter, which is set to 0.1; Step three two six, based on the contrast loss function l1, the contrast loss function l2, the contrast loss function l3, the total contrast loss function L is constructed CL , is expressed as: L CL ＝(l1+l2+l3) / 3 Step three, reconstruct the loss L based on the missing modal rec and the total contrast loss function L CL , the loss function L of the missing modal generation module is constructed mmg , which is represented as: L mmg = aL rec + L CL Wherein, α represents a coefficient, and α = 5; Step three four, training the multi-modal molecular pre-training model MolBT to obtain a pre-trained multi-modal molecular pre-training model MolBT; Step three: based on the type of a downstream task, fine-tuning the pre-trained multi-modal molecular pre-training model MolBT to obtain a fine-tuned multi-modal molecular pre-training model MolBT; Step four: predicting the downstream task based on the fine-tuned multi-modal molecular pre-training model MolBT.

2. The multi-modal drug molecule prediction method based on missing modality generation of claim 1, wherein: The multi-modal molecular pre-training model MolBT in step one is constructed as follows: Step one one, inputting a molecular graph into a molecular graph structure encoder, and the molecular graph structure encoder outputs a molecular graph structure feature representation; The molecular graph structure encoder is Graphormer; Step one two, inputting a molecular description text into a molecular sequence encoder, and the molecular sequence encoder outputs a molecular description text feature representation; The molecular sequence encoder is BERT; Step one three, inputting a molecular SMILES string into the molecular sequence encoder, and the molecular sequence encoder outputs a molecular SMILES feature representation; The molecular sequence encoder is BERT; Step one four, establishing a cross-modal encoder, and inputting the molecular graph structure feature representation, the molecular description text feature representation and the molecular SMILES feature representation into the cross-modal encoder, and the cross-modal encoder outputs a molecular graph structure cross-modal feature representation and a molecular description text cross-modal feature representation; Step one five, constructing a missing modality generation module, and inputting the molecular graph structure feature representation and the molecular SMILES feature representation of the missing molecular description text into the missing modality generation module, and the missing modality generation module outputs a corresponding molecular description text shallow feature representation.

3. The multi-modal drug molecule prediction method based on missing modality generation of claim 2, wherein: In step one one, the molecular graph is input into the molecular graph structure encoder, and the molecular graph structure encoder outputs the molecular graph structure feature representation; The specific process is as follows: Step III, input the molecular graph into the graph feature encoding layer in the molecular graph structure encoder, and the graph feature encoding layer in the molecular graph structure encoder outputs molecular graph node features G0, wherein, denotes a first atomic feature embedding, denotes a second atomic feature embedding, denotes an n-th atomic feature embedding, R denotes a real number, n g denotes the number of atoms in the molecular graph, d g denotes the feature dimension of the molecular graph structure encoder; Step one-two, input the molecular graph node features G0 into the first layer Graphormer layer, the second layer Graphormer layer, the third layer Graphormer layer, the fourth layer Graphormer layer, the fifth layer Graphormer layer, the sixth layer Graphormer layer, the seventh layer Graphormer layer, the eighth layer Graphormer layer, the ninth layer Graphormer layer, the tenth layer Graphormer layer, the eleventh layer Graphormer layer, the twelfth layer Graphormer layer of the molecular graph structure encoder in turn, and the twelfth layer Graphormer layer outputs the molecular graph structure feature representation G 12 ; The formula is as follows: In the formula, K g represents the total number of layers of the Graphormer layer in the molecular graph structure encoder, K g = 12; denotes the i-th layer Graphormer layer; G i-1 denotes the molecular graph structure features output by the i-1th Graphormer layer; G i denotes the molecular graph structure features output by the i-th layer Graphormer layer.

4. The multi-modal drug molecule prediction method based on missing modality generation of claim 3, wherein: In step one two, the molecular description text is input into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular description text feature representation; the specific process is as follows: Step one two one, input the molecular description text into the BERT word embedding layer of the molecular sequence encoder, and the BERT word embedding layer of the molecular sequence encoder outputs the molecular description text feature T0; Step 1222, input the molecular description text feature T0 into the first layer of the molecular sequence encoder, the second layer of the molecular sequence encoder, the third layer of the molecular sequence encoder, the fourth layer of the molecular sequence encoder, the fifth layer of the molecular sequence encoder, the sixth layer of the molecular sequence encoder, the seventh layer of the molecular sequence encoder, the eighth layer of the molecular sequence encoder, the ninth layer of the molecular sequence encoder, the tenth layer of the molecular sequence encoder, the eleventh layer of the molecular sequence encoder, and the twelfth layer of the molecular sequence encoder in sequence, and the twelfth layer of the molecular sequence encoder outputs the molecular description text feature representation T 12 ; The formula is as follows: In the formula, K s represents the total number of Transformer layers in the molecular sequence encoder, K s = 12; denotes the i-th layer of the Transformer layer; T i-1 represents the molecular description text feature output by the i-1th layer of the Transformer layer; T i denotes the molecular description text feature output by the i-th layer of the Transformer.

5. The multi-modal drug molecule prediction method based on missing modality generation of claim 4, wherein: In step one three, the molecular SMILES string is input into the molecular sequence encoder, and the molecular sequence encoder outputs the molecular SMILES feature representation; The specific process is as follows: Step one three one, input the molecular SMILES string into the BERT word embedding layer of the molecular sequence encoder, and the BERT word embedding layer of the molecular sequence encoder outputs the molecular SMILES feature S0; Step one three two, input the molecular SMILES feature S0 into the first layer of the molecular sequence encoder, the second layer of the molecular sequence encoder, the third layer of the molecular sequence encoder, the fourth layer of the molecular sequence encoder, the fifth layer of the molecular sequence encoder, the sixth layer of the molecular sequence encoder, the seventh layer of the molecular sequence encoder, the eighth layer of the molecular sequence encoder, the ninth layer of the molecular sequence encoder, the tenth layer of the molecular sequence encoder, the eleventh layer of the molecular sequence encoder, the twelfth layer of the molecular sequence encoder in turn, and the twelfth layer of the molecular sequence encoder outputs the molecular SMILES feature representation S 12 ; The formula is as follows: In the formula, K s represents the total number of Transformer layers in the molecular sequence encoder, K s = 12; denotes the i-th layer of the Transformer layer; S i-1 SMILES features output by the i-1th layer of the Transformer; S i denotes the molecular SMILES features output by the i-th layer of the Transformer.

6. The multi-modal drug molecule prediction method based on missing modality generation of claim 5, wherein: In step one four, the cross-modal encoder is established, and the molecular graph structure feature representation, the molecular description text feature representation, and the molecular SMILES feature representation are input into the cross-modal encoder, and the cross-modal encoder outputs the molecular graph structure cross-modal feature representation and the molecular description text cross-modal feature representation; The specific process is as follows: G7, T7 and S7 outputted by the 7th layer of the molecular graph structure encoder, the 7th layer of the molecular sequence encoder are inputted into the cross-modal encoder, and C is outputted by the cross-modal encoder G T ;​ G8 outputted by the 8th layer of the molecular graph structure encoder, T8 and S8 outputted by the 8th layer of the molecular sequence encoder are inputted into the cross-modal encoder, and the cross-modal encoder outputs C G ,C T ; G9 outputted by the 9th layer of the molecular graph structure encoder, T9 and S9 outputted by the 9th layer of the molecular sequence encoder are inputted into the cross-modal encoder, and the cross-modal encoder outputs C G T ;​ G 10 , T 10 , and S 10 inputs into a cross-modal encoder, and the cross-modal encoder outputs C G , C T ; G 11 , T 11 , and S 11 input into the cross-modal encoder, and the cross-modal encoder outputs C G , C T ; G 12 , T 12 , and S 12 input into a cross-modal encoder, and the cross-modal encoder outputs C G , C T ; The number of layers of the cross-modal encoder is 6, and each layer includes a self-attention layer, a common attention layer, and a feed-forward layer FFN in turn; The operation process of each layer in the cross-modal encoder is as follows: where CME l represents the l-th layer of the cross-modal encoder; Each layer of the cross-modal encoder includes a self-attention layer, a common attention layer, and a feed-forward layer FFN in turn. and respectively denote the molecular graph structure cross-modal feature representation and the molecular text cross-modal feature representation of the output of the l-th layer of the cross-modal encoder; L C is the number of layers of the cross-modal encoder, L C = 6; molecular graph structure feature representation representing the output of the cross-modal encoder at the l-1th layer; represents the text feature representation of the output of the cross-modal encoder layer l-1; SMILES feature representation of the molecular SMILES representation of the input to the cross-modal encoder layer i; The specific process is as follows: 1) Representing molecular graph structure features The input of the first layer of the cross-modal encoder is obtained through the BridgeLayer processing Representing text features BridgeLayer processing to obtain input to the first layer of the cross-modal encoder It is represented as: Wherein In the formula, W G denotes the molecular graph structure feature projection matrix; W T denotes the text feature projection matrix; G type representing molecular graph structure feature representation G k obtaining a modal type embedding through a type embedding layer T type representing the molecular description text feature representation T k obtaining a modality type embedding through a type embedding layer G k and T k denote the molecular graph structure feature representation and the molecular description text feature representation of the k-th layer output of the molecular graph structure encoder and the molecular sequence encoder, respectively, k = 7, 8, …, 12; representative layer normalization LN; representative layer normalization LN; 2) Feature representation based on molecular SMILES string Extract class labels, denoted as wherein denotes a molecular SMILES string feature representation S k the first token feature embedding in S denotes a molecular SMILES string feature representation S k the second token feature embedding in S denotes a molecular SMILES string feature representation S k the n s token feature embedding in S a molecular SMILES string feature representation S k class label; n s denotes the length of the molecular SMILES string; d s denotes the feature dimension of the molecular sequence encoder; 3) replace the last node feature in and with to get and as follows: In the formula, represents the molecular graph structure cross-modal feature representation after replacing the features. represents the 1st atomic feature embedding in ; represents the 2nd atomic feature embedding in ; represents the (n-1)th atomic feature embedding in ; the molecular description text after the replacement of the features represents a cross-modal feature representation; represents a first token feature embedding in ; represents a second token feature embedding in ; represents an (n-1)th token feature embedding in ; n g denotes the number of atoms in the molecular graph; d g denotes the feature dimension of the molecular graph structure encoder; n t denotes the length of the molecular description text; d t denotes the feature dimension of the text encoder; 4) to the molecular graph structure part and the molecular text part of the input to the first layer of the cross-modal encoder, to obtain and In the formula, denotes the molecular graph structure cross-modal feature representation of the output of the l-th layer of the cross-modal encoder; denotes the molecular graph structure cross-modal feature representation after processing; a text cross-modal feature representation representing the output of the l-th layer of the cross-modal encoder; a text cross-modal feature representation representing the output of the l-th layer of the cross-modal encoder; a text cross-modal feature representation representing the output of the l-th layer of the cross-modal encoder; processed molecular graph structure cross-modal feature representation; processed molecular graph structure cross-modal feature representation; processed molecular graph structure cross-modal feature representation; processed molecular graph structure cross-modal feature representation; CME l denotes the l-th layer of the cross-modal encoder; Each layer of the cross-modal encoder includes a self-attention layer, a common attention layer, and a feed-forward layer FFN in turn.

7. The multi-modal drug molecule prediction method based on missing modality generation of claim 6, wherein: In step one five, the missing modal generation module is constructed, and the molecular graph and the molecular SMILES string of the missing molecular description text are input into the missing modal generation module, and the missing modal generation module outputs the corresponding molecular description text; The specific process is as follows: Step one five one, the molecular graph structure feature representation G6 obtained by passing through the first 6 layers of the molecular graph structure encoder, the molecular description text feature representation T6 obtained by passing through the first 6 layers of the molecular sequence encoder, and the molecular SMILES feature representation S6 are denoted as shallow feature representation, that is, {G6, T6, S6}; Step one five two, the shallow feature representation G6, T6, S6 is projected into the same dimensional space through a one-dimensional convolution layer respectively, to obtain Z (m) ∈R T×d m∈{t,g,s}; wherein represents Z (t) , Z (g) or Z (s) ; Z (t) denotes the output of the shallow feature representation T6 after passing through one 1D convolution layer;Z (g) denotes the output of the shallow feature representation G6 after passing through one 1D convolution layer;Z (s) denotes the output of the shallow feature representation S6 after passing through one 1D convolution layer; T represents the feature length after passing through the one-dimensional convolution layer; d represents the feature dimension after passing through the one-dimensional convolution layer; Step one five three, in the case of missing molecular description text modal, using the molecular graph structure feature representation G6 obtained by the first 6 layers of the molecular graph structure encoder, the molecular SMILES feature representation S6 obtained by the first 6 layers of the molecular sequence encoder, and Z (g) and Z (s) are input into the normalized flow model and to obtain the feature distribution Y (g) and Y (s) of the molecular graph structure and the molecular SMILES string respectively; In the formula, Y (g) denotes the molecular graph structure feature distribution obtained by the normalized flow model; Y (s) represents the normalized flow model obtained molecular SMILES string feature distribution; Step one five four, to Y (g) and Y (s) Perform an averaging operation to sample potential text representations is represented as: Y (g) ~ N (μ c ,∑ c ), Y (s) ~ N (μ c ,∑ c ) In the formula, N(μ c ,∑ c ) represents the standard normal distribution after the molecular graph structure features and the molecular SMILES string features are projected to the shared latent space by the normalization flow model; μ c represents the distribution center; ∑ c represents the covariance; and represents the generation process of the normalization flow model. Step one fifty-five, the input to generate The formula is as follows: wherein denotes inverse transform, denotes a normalized flow model; denotes a missing text feature distribution generated by inverse transform of the normalized flow model; Step one five six, to Input into the projection network MLP to get the shallow feature representation of the missing text modality In the formula, Proj (t) ( ) denotes a characteristic projection network MLP.

8. The multi-modal drug molecule prediction method based on missing modality generation of claim 7, wherein: In step three four, the multi-modal molecular pre-training model MolBT is trained to obtain a pre-trained multi-modal molecular pre-training model MolBT; The specific process is as follows: Step three four one, pre-training phase one: The molecular graph structure encoder, the molecular sequence encoder, the cross-modal encoder, and the missing modality generation module in the multi-modal molecular pre-training model MolBT are pre-trained using real molecule-text pair data, and the loss function is L mmg ; The molecules in the real molecule-text pair data are molecular graphs and molecular SMILES strings; The multimodal molecular pre-training model MolBT is pre-trained for 20 rounds using the AdamW optimizer, with a learning rate set to 1e-4, a weight decay set to 1x10 -5 , and a batch size set to 16. A pre-trained multi-modal molecular pre-training model MolBT is obtained in the pre-training phase one; Step three four two, pre-training phase two: Use the molecular data with missing molecular description text; The molecular graph structure encoder, the molecular sequence encoder, the cross-modal encoder, and the missing modality generation module in the multi-modal molecular pre-training model MolBT are pre-trained using the molecular graph and the molecular SMILES string describing the text of the missing molecule, and the model weight saved in the pre-training stage is applied, and the loss function is L CL ; The multimodal molecular pre-training model MolBT is pre-trained for 20 rounds using the AdamW optimizer, with a learning rate set to 1e-4, a weight decay set to 1x10 -5 , and a batch size set to 16. A pre-trained multi-modal molecular pre-training model MolBT is obtained in the pre-training phase two; Step three four three, pre-training phase three: The molecular graph structure encoder, the molecular sequence encoder, and the cross-modal encoder in the multi-modal molecular pre-training model MolBT are pre-trained using real molecules and text, and the model weights saved in the pre-training phase two are applied, and the loss function is L CL ; The molecules in the real molecule-text pair data are molecular graphs and molecular SMILES strings; The AdamW optimizer is used to pre-train the molecular graph structure encoder, the molecular sequence encoder, the cross-modal encoder, and the missing modality generation module in the multi-modal molecular pre-training model MolBT for 20 rounds, the learning rate is set to 1e-4, the weight decay is set to 1x10 -5 , and the batch size is set to 16. A pre-trained multi-modal molecular pre-training model MolBT is obtained in the pre-training phase three.

9. The multi-modal drug molecule prediction method based on missing modality generation of claim 8, wherein: In step three, based on the type of downstream task, the pre-trained multi-modal molecular pre-training model MolBT is fine-tuned to obtain a fine-tuned multi-modal molecular pre-training model MolBT; The specific process is as follows: 1) when the downstream task is molecular property prediction, input 8 molecular property datasets into the pre-trained multi-modal molecular pre-training model MolBT to obtain a fine-tuned multi-modal molecular pre-training model MolBT; The 8 molecular property datasets include BBBP, Tox21, ToxCast, SIDER, ClinTox, MUV, HIV and BACE; 2) when the downstream task is molecular and text cross-modal retrieval, input the PCDes dataset into the pre-trained multi-modal molecular pre-training model MolBT to obtain a fine-tuned multi-modal molecular pre-training model MolBT.

10. The multi-modal drug molecule prediction method based on missing modality generation of claim 9, wherein: The step four is to predict the downstream task based on the fine-tuned multi-modal molecular pre-training model MolBT; the specific process is: 1) when the downstream task is molecular property prediction, predict the molecular property based on the fine-tuned multi-modal molecular pre-training model MolBT for molecular property prediction; the specific process is: input the to-be-tested molecular graph into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder outputs the molecular graph structure feature representation; input the molecular graph structure feature representation into the prediction head, and the prediction head outputs the molecular property; 2) when the downstream task is molecular graph and molecular description text cross-modal retrieval, perform molecular graph and molecular description text cross-modal retrieval based on the fine-tuned multi-modal molecular pre-training model MolBT for molecular and text cross-modal retrieval; the specific process is: 21) molecular graph and molecular description text cross-modal retrieval includes two subtasks: M2T and T2M; M2T means that given a molecular graph, the molecular description text matching the molecular graph is retrieved; T2M means that given a molecular description text, the molecular graph matching the molecular description text is retrieved; 22) in the M2T task, input the molecular graph into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs the molecular graph feature representation; At the same time, input all molecular description texts into the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs all molecular description text feature representations; Calculate the cosine similarity between the molecular graph feature representation and all molecular description text feature representations respectively and sort them, and the molecular description text with the maximum cosine similarity is the best matching molecular description text; 23) in the T2M task, input the molecular description text into the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular sequence encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs the molecular description text feature representation; At the same time, input all molecular graphs into the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT, and the molecular graph structure encoder of the fine-tuned multi-modal molecular pre-training model MolBT outputs all molecular graph feature representations; The cosine similarity between the molecular description text feature representation and all the molecular graph feature representations is calculated respectively and ranked, and the molecular graph with the largest cosine similarity is the best matching molecule.

Images