Molecular multimodal model training and application method, storage medium and chip

Abstract

This invention utilizes methods from the field of network security to realize a training and application method, storage medium, chip, and system for molecular multimodal models. First, the Encoder structure in a Transformer facilitates interaction between candidate document sets and between subtopics or queries. After obtaining formal representations of documents and subtopics, weights are combined using selected documents, all candidate documents, and subtopics. Explicit and implicit scores are obtained through interaction, and finally, the updated weights combine the explicit and implicit scores into a final diversified score range. The method provided in this invention designs an explicit-implicit feature combination model that dynamically adjusts weights at different steps in different queries to improve the diversification of search results. The model is trained using a listpairwise LambdaRank loss function, and experimental results demonstrate the model's effectiveness and interpretability.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to methods for training and applying molecular multimodal models, storage media, and chips. Background Technology

[0002] Understanding molecular knowledge and discovering molecular properties is crucial for scientific exploration in many fields such as biomedicine, chemistry, and materials science. Traditional exploration methods require professionals to conduct numerous wet biochemical experiments and repeated trials, which is not only expensive but also time-consuming. With the advancement of deep learning, it has become possible to use artificial intelligence to assist scientific exploration, such as predicting molecular properties and generating candidate molecules, and significant progress has been made.

[0003] However, unlike humans who understand molecules from multiple perspectives, most existing AI models are designed for a single cognitive ability (e.g., attribute prediction, molecule generation, literature understanding) and a single form of molecule (e.g., molecular diagrams, SMILES strings, text). These models fall into two main categories. Language-based models take natural language, containing knowledge about molecules and / or SMILES strings, as input. For example, in Sheng Wang et al.'s work "Smiles-bert: large-scale unsupervised pre-training for molecular property prediction" and Seyone Chithrananda et al.'s work "Chemberta: large-scale self-supervised pretraining formolecular property prediction," molecular property prediction models were designed for SMILES molecular strings. Works such as Iz Beltagy et al.'s "Scibert: A pretrained language model for scientific text," Diya Li et al.'s "Biomedical event extraction based on knowledge-driven tree-LSTM," and Jinhyuk Lee et al.'s "Biobert: a pre-trained biomedical language representation model for biomedical text mining" focus on learning literary understanding from biochemical texts. Zheni Zeng et al. developed a deep learning system in "A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals" to jointly learn molecularly related text and molecular SMILES strings to establish relationships between them.In molecular generative models such as Jike Wang et al.'s Multi-constraint molecular generation based on conditional transformer, knowledge-edge distillation and reinforcement learning, Jeff Guo et al.'s Improving de novomolecular design with curriculum learning, Daniel Flam-Shepherd et al.'s Languagemodels can learn complex molecular distributions, and Samuel C. Hoffman et al.'s Optimizing molecules using efficient queries from property evaluations, the generated molecules are represented as SMILES. Graph-based models can only handle molecular graphs. Currently, graph neural network (GNN)-based molecular property prediction models learn from molecular graphs, or generative models learn directly from graph data to generate molecular graphs. Training these models requires a large amount of manual annotation or a set of specific attributes, while other molecular properties and related conditions are often ignored. Therefore, these models can only handle one morphology of the molecule and cannot achieve a comprehensive understanding of the molecule. Summary of the Invention

[0004] To address this, the present invention first proposes a method, storage medium, and chip for training and applying molecular multimodal models. It jointly learns a graph encoder and a text encoder from multimodal molecular data to associate molecular graphs with their biomedical text descriptions, thus solving the technical problems of existing single-modal molecular data machine learning models mentioned above.

[0005] This invention first provides a method for training and applying molecular multimodal models, including the following steps:

[0006] S100: Establish a data collection unit to extract molecular diagrams and their semantically weakly related text data from published SCI papers and construct a pair of molecular diagram text datasets.

[0007] S200, establish a molecular multimodal model and its pre-training unit, construct a molecular multimodal model containing a graph encoder and a text encoder, and train the model using contrastive learning.

[0008] S300 establishes a text-based molecular graph generation unit, applying the model to different downstream tasks such as cross-modal retrieval and text-based molecular graph generation.

[0009] The method for constructing the dataset in the data collection unit includes:

[0010] S201, collect the names, synonyms and SMILES strings of the top 50K molecular compounds in PubChem;

[0011] S202, for each collected molecule, use the smiles2graph function provided by OGB to convert its SMILES string into a molecular graph;

[0012] S203: Using the name of a molecule as a query, search the abstracts, introductions, and conclusions of published scientific papers in the fields of medicine, biology, chemistry, and computer science in the S2orc database for sentences containing that name. Each retrieved sentence and its adjacent sentences are recorded as a paragraph in a document. If fewer than two paragraphs are retrieved by name, the search is repeated using the molecule's synonyms or aliases. The molecule search is terminated early when a specified number of paragraphs or a specified document size are retrieved.

[0013] S204, obtain molecular graph-document pairs to form a multimodal molecular dataset.

[0014] The molecular multimodal model consists of a graph encoder and a text encoder, which extract molecular graph representations and text representations respectively. A graph isomorphic network is used as the graph encoder, and the BERT language model is used as the text encoder. During the training phase, the model additionally uses a similarity calculation module, which uses two mapping heads to map the molecular graph and text representations to a joint representation space, and then calculates the cosine similarity of the mapped features.

[0015] The molecular multimodal model pre-training method specifically includes:

[0016] S401, initialize the graph encoder with the self-supervised training weights of the graph isomorphic network, and initialize the text encoder with the pre-trained weights of BERT in Sci-BERT or KV-PLM.

[0017] S402, for each training period, sample a batch of N molecular graph-text pairs from the training sequence in sequence;

[0018] S403, for each mini-batch data {G1,…,G…} NTwo different augmentations are generated from each graph using random node deletion and random subgraphs, resulting in a total of 2N augmented graphs. These graphs are then input into a graph encoder to obtain their representation vectors. in and Represents the i-th graph G i The representation of the two enhanced graphs;

[0019] S404, randomly select two different sentences from the document corresponding to each molecule. For the i-th graph, use a text encoder to extract the description of G. i The representations obtained from the two different sentences are as follows Each molecular graph in the mini-batch corresponds to two distinct sentences, generating 2N text representations.

[0020] S405, for the i-th graph G i The total multi-view loss includes four representation pairs from the multimodal view. and The four contrasting losses,

[0021] in, Corresponding contrast loss:

[0022]

[0023] Corresponding contrast loss:

[0024]

[0025] Corresponding contrast loss:

[0026]

[0027] Corresponding contrast loss:

[0028]

[0029] Where τ is the temperature parameter. This is the similarity calculation module, which first uses two independent mapping heads to... and Project onto the same dimension, and then calculate the cosine similarity between the projected vectors;

[0030] S406, Calculate the contrast loss of the graph modes: The contrast loss of the graph modes of the i-th graph is:

[0031]

[0032] S407, calculate the sum of the losses for all samples within a batch:

[0033]

[0034] Where λ is the balance factor between the cross-modal loss and the graph mode loss, and is a hyperparameter;

[0035] S408, for each batch, update the parameters of the graph encoder, text encoder and mapping head by backpropagating the overall loss L, until all batches in the current epoch have been processed;

[0036] S409, repeat S402 to S408 until the preset maximum number of epochs is reached.

[0037] The text-based molecular diagram generation method:

[0038] It consists of a molecular multimodal model trained by a pre-training method and a pre-trained molecular generator based on random seed sampling and allowing backpropagation of gradients. The parameters of the molecular multimodal model and the molecular generator are fixed.

[0039] S501, Input a text x describing a molecule. T ;

[0040] S502, Initialize and generate seed q; set q as a learnable parameter;

[0041] S503, using a trained molecular generator to generate molecular diagram x based on q. G ;

[0042] S504, respectively x G and x T The graph encoder and text encoder of the pre-trained molecular multimodal model are fed into the graph encoder and text encoder to extract the corresponding graph representation z. G and text representation z T Using the similarity calculation module of the molecular multimodal model, the negative similarity between the two is calculated as the loss function:

[0043] l q =-sim(z G ,z T ) / τ,

[0044] S505, regarding loss l q Perform backpropagation and update the seed q using gradient descent;

[0045] S506, repeat S503 to S506 until the preset maximum number of epochs is reached;

[0046] S507 feeds the optimized q into the molecular generator to generate the final molecular graph and outputs the molecular graph.

[0047] The present invention also provides a storage medium for an embedded molecular multimodal model training and application method, and a chip using the above storage medium.

[0048] The technical effects to be achieved by this invention are as follows:

[0049] This invention proposes a molecular multimodal model (MoMu) that implicitly establishes a link between molecular structure and linguistic description. Because it can handle molecules with multiple modalities, this model can be applied to a very wide range of downstream tasks.

[0050] This method generates molecules from molecular functional descriptions. It's a text-based molecular graph generation method that can directly generate new molecules from text descriptions with desired conditions, solving the problem of existing technologies failing to generate effective molecules from descriptions. For descriptions with multiple conditions, it can generate molecular structures by satisfying as many conditions as possible. Compared to existing AI-based molecular generation methods that can only generate molecules with specified attributes, this method can adaptively generate molecular candidates based on the input text, which can describe any one or more desired conditions. Due to its strong generalization ability and imaginative power, the pre-trained molecular multimodal model can advance scientific exploration in multiple molecular-related fields such as biology, chemistry, materials science, and medicine.

[0051] The molecular multimodal model MoMu provided by this invention is pre-trained based on pairwise multimodal data consisting of molecular graphs and their weakly correlated biochemical descriptions retrieved from publicly available SCI papers. Because it can handle molecules of various modalities, the pre-trained molecular multimodal model provided by this invention can be applied to a very wide range of downstream tasks. Therefore, this invention provides a zero-shot molecule generation method and a molecule description method based on the pre-trained model. Experimental results show that the pre-trained model exhibits strong generalization ability in a wide range of downstream tasks, including cross-modal molecule retrieval, molecule heading, zero-shot molecule generation, and molecular property prediction. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the collection of the image and text dataset for this invention.

[0053] Figure 2 This is a schematic diagram illustrating the architecture and pre-training principle of the molecular multimodal model of the present invention;

[0054] Figure 3 This is a schematic diagram illustrating the principle of the text-based molecular diagram generation method of the present invention.

[0055] Figure 4The present invention provides the results of generating text inputs containing functional descriptions using the text-based molecular diagram generation method.

[0056] Figure 5 The present invention provides the results of generating text inputs containing structural descriptions using the text-based molecular diagram generation method. Detailed Implementation

[0057] The following are preferred embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0058] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description and claims. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0059] This invention proposes a method for training and applying molecular multimodal models, as well as a storage medium and chip based on this method.

[0060] The training and application method of molecular multimodal models includes three components: data collection, molecular multimodal (MoMu) models and their pre-training, and text-based molecular graph generation.

[0061] Data collection unit:

[0062] The data collection unit is implemented by constructing a dataset of molecular image-text pairs for pre-training molecular multimodal models.

[0063] The dataset was constructed as follows: First, the names, synonyms, and SMILES strings of the top 50,000 molecular compounds in PubChem were collected. The PubChem database contains basic information on over 150 million chemical substances. To obtain molecular graphs of the collected compounds, the `smiles2graph` function provided by OGB was used to convert the SMILES strings into molecular graphs. Text related to the corresponding molecules in published scientific papers was retrieved from the S2orc database as weak semantic supervision. S2orc is a corpus containing over 136 million papers from various fields; only papers from the fields of medicine, biology, chemistry, and computer science were extracted, as they are more likely to contain descriptions related to molecules. To avoid special characters related to experimental data in the text as much as possible, the retrieval was performed only from the abstract, introduction, and conclusion sections of each extracted paper.

[0064] like Figure 1As shown, for each molecule, its name is first used as the query to retrieve sentences containing that name. Each retrieved sentence and its adjacent sentences are recorded as a paragraph in a document. If fewer than two paragraphs are retrieved by name, the search is repeated using synonyms or aliases of the molecule as queries. The molecule search is terminated early when 5,000 paragraphs are retrieved or the document size exceeds 500 Mb. Not all 50,000 molecules can be retrieved using their corresponding text descriptions. Finally, 15,613 molecular graph-document pairs were obtained to form a multimodal molecular dataset. There are approximately 37 million paragraphs in all collected documents. In each pair, the sentences in the document contain weakly related semantic information about the corresponding molecular graph.

[0065] Molecular Multimodal (MoMu) Model and its Pre-trained Units:

[0066] The overall architecture and pre-training process of the Molecular Multimodal (MoMu) model are as follows: Figure 2 As shown, MoMu consists of a graph encoder and a text encoder, which encode the molecular graph and text into a joint representation space, respectively. A graph isomorphic network (GIN) is used as the graph encoder, and the widely used language model BERT is used as the text encoder.

[0067] Unlike training basic graph-text multimodal models using general graph-text data, molecular-related graph-text data is relatively scarce, insufficient for training molecular graph-text encoders from scratch. Just as humans should possess the ability to recognize graphics and language when learning specialized knowledge, enabling artificial intelligence to learn specialized molecular knowledge also requires building upon trained general-purpose graph and text encoders. Therefore, the graph encoder is initialized with the self-supervised training weights of GIN provided in [the documentation], and the text encoder is initialized with the pre-trained weights of BERT provided by Sci-BERT and KV-PLM, respectively. The MoMu initialized with the weights of Sci-BERT and KV-PLM are denoted as MoMu-S and MoMu-K, respectively.

[0068] MoMu was then trained using the collected pairing dataset. For each pair of graph-document data in the mini-batch, two separate graphs were created from the molecular graph using two different types of graph augmentation. Graph augmentation was performed using data augmentation introduced by GraphCL. For graph augmentation, two types of graph augmentation were considered: node removal and subgraph. Node removal randomly discards a portion of the vertices of the original graph. For the molecular graph, the absence of certain atoms (e.g., some hydrogen atoms in a compound) does not change its semantic information. Subgraph refers to sampling a subgraph from the original graph using a random walk. The properties of the molecule and the properties of the molecule formed by its subgraph have certain similarities; for example, some molecules contain the same functional groups. Then, two different sentences were randomly sampled from the document. Therefore, each modality has two samples containing the same semantic information.

[0069] For graph modalities, the mini-batch of molecular graph-document pairs of size N {G1,…,G} is given by the following data: N Each graph generates two distinct augmentations, resulting in a total of 2N augmented graphs. These graphs are then fed into a graph encoder to obtain their representation vectors. in and Represents the i-th graph G i Two enhanced versions of the representation. Simultaneously, the description of G is encoded via a text encoder. i The representations obtained from the two different sentences are as follows Each molecular graph in the mini-batch corresponds to two distinct sentences, generating 2N text representations. Therefore, for the i-th graph G i The total multi-view loss comprises four contrast losses between four representation pairs from the multimodal view, namely... and For simplicity, only The contrast loss is expressed as:

[0070]

[0071] Where τ is the temperature parameter. First, through two independent mapping heads... and Projecting onto the same dimension, then calculating the cosine similarity between the projected vectors. The other three cross-modal contrastive losses have the same form.

[0072] To further enhance the representational power of the graph encoder, contrastive learning is utilized in the graph modality. Specifically, positive pairs are introduced by minimizing the normalized cross-entropy loss, while negative pairs are pushed aside. Positive pairs are two enhancements from the same molecular graph, while negative pairs come from different molecular graphs. Based on the preceding definition, the graph modality contrastive loss for the i-th graph is derived as follows:

[0073]

[0074] Where τ is the temperature parameter, and the final loss is calculated across all samples in a mini-batch.

[0075] Pre-trained MoMu can process molecular graphs and natural language text in a uniform way and extract general and transferable knowledge from these heterogeneous data, which can be easily generalized to different downstream tasks.

[0076] Implementation details:

[0077] A 5-layer GIN with a hidden layer dimension of 300 was used as the graph encoder. The text encoder was a BERT with a hidden layer size of 768. Two multilayer perceptrons were used to project graph and sentence features into the same feature space, with each perceptron having an output dimension of 256. For both graph augmentations, a 10% node dropout rate was applied, and the sampled subgraph size was 80% of the original graph. The input data for the text modality consisted of two sentences randomly selected from the corresponding graph data documents. The model was pre-trained using the AdamW optimizer with a learning rate of 0.0001, a weight decay of 1e-5, and a duration of 300 epochs. τ was set to 0.1, and the batch size was set to 256. The entire pre-training process was implemented using PyTorch and trained on eight NVIDIA Tesla V100 PCIe 32GB GPUs.

[0078] Application of Molecular Multimodal Models in Cross-Modal Retrieval Tasks. Since the MoMu model provided in this invention is pre-trained by matching weakly related text with corresponding molecular graphs, it can handle both graph and textual modalities of molecules.

[0079] We evaluated its performance in cross-modal retrieval. Given a molecular graph, graph-to-text (GT) retrieval aims to retrieve the most relevant textual description of that molecule. Conversely, given a text paragraph, text-to-graph (TG) retrieval aims to retrieve the most relevant molecular graph it describes. MoMu was evaluated on the PCdes dataset, which contains SMILES and paired textual descriptions of 15K molecules from PubChem. The dataset was split into a training set of 10,500 pairs, a validation set of 1,500 pairs, and a test set of 3,000 pairs (two SMILES pairs in the test set could not be converted to graphs using Rdkit, so the remaining 2,998 pairs were used for detection). We converted the SMILES strings in each pair to molecular graphs. In the GT / TG task, the query graph / text was represented using MoMu's graph / text encoder, and the key text / graphs to be retrieved were represented using MoMu's text / graph encoder. Cosine similarity was calculated between the query representation and all key representations, and the key text / graphs were sorted in descending order of similarity. Following the experimental setup in reference

[11] , searches were performed in randomly sampled small batches (64 pairs per batch) and all test pairs, and the mean precision and recall (mean ± standard deviation) of the top-1 search results were reported. In the reference, a sentence was randomly selected from the text corresponding to each molecule for retrieval; we use sentence-level retrieval to represent this setup. Further evaluation was conducted using a complete paragraph description of each molecule, referred to as paragraph-level retrieval.

[0080] The results of comparing the method of this invention with Sci-BERT[8] and KV-PLM (KV-PLM* differs from KV-PLM in that it handles SMILES tags differently) are shown in Table 1. These methods were all fine-tuned on the PCdes training set for fair comparison. For different settings of GT and TG tasks, MoMu-S and MoMu-K outperformed other methods that directly use SMILES for retrieval. Compared with KV-PLM, which jointly models molecular SMILES and language text, MoMu of this invention can better connect molecular structure and natural language description.

[0081] Table 1 shows the performance of different methods on the PCdes dataset in graph-to-text (GT) retrieval and text-to-graph (TG) retrieval, with the results of sentence-level retrieval by Sci-Bert, KV-PLM, and KV-PLM* being the results reported in the literature.

[0082]

[0083] Considering that some of the 15K molecular image-text pairs in PCdes may have been collected as pre-training data for MoMu, 5,562 image-text pairs with molecular IDs ranging from 50,000 to 100,000 were collected from PubChem and were not used for pre-training. The comparison with Sci-BERT and KV-PLM on this collected zero-shot retrieval test set is shown in Table 2. MoMu-S and MoMu-K significantly outperform Sci-BERT and KV-PLM, further demonstrating MoMu's generalization ability. On both datasets, MoMu-S and MoMu-K perform similarly, meaning that initializing MoMu's text encoder with KV-PLM does not result in better performance than initializing with Sci-BERT. This indicates that structural information learned from one-dimensional SMILES molecular strings is not easily transferred to structured molecular graphs, while MoMu directly utilizes graph neural networks to capture structural information under the supervision of language description.

[0084] Table 2 shows the performance of different methods in zero-shot graph-to-text (GT) retrieval and zero-shot text-to-graph (TG) retrieval on the datasets we collected.

[0085]

[0086] Text-based molecular diagram generation unit:

[0087] like Figure 3 As shown, the zero-sample text-to-graph molecular generation method consists of a MoMu-based similarity measurement module and a molecular generator based on random seed sampling that allows gradient backpropagation. This invention uses the flow-based molecular generator MoFlow as an example to illustrate the method. MoFlow defines a parameterized invertible mapping flow from a Gaussian distribution to a molecular distribution. The molecular graph G consists of an atomic matrix. Bond matrix Composition, where N is the number of atoms in the molecule, C a and C b This represents the number of atom types and bond types. If the nth atom belongs to the cth atom type, then V n,c =1; otherwise V n,c =0. If the bond between the nth atom and the n'th atom belongs to the c'th bond type, then E n,n',c' =1; otherwise E n,n',c' =0. MoFlow contains a graph conditional flow q v =f c (V|E) is used to encode the atom matrix V given the bond matrix E, thereby transforming it into the latent variable q. v gflow:q e =fg (E) is used to encode the key matrix E into latent variables q. e f c and f g It is implemented using a graph convolutional neural network based on graph coupling layers. v and q e The connection q = [q v ;q e The variable q follows a Gaussian distribution P(q). After training with MoFlow, a variable q can be sampled from P(q) and decomposed into two parts q. v and q e They are input into the reverse graph conditional flow. and reverse gflow To obtain the probability matrix:

[0088]

[0089]

[0090] in It is the predicted probability that the bond between the nth atom and the n'th atom is a c' bond. V is the probability that the nth atom belongs to the cth atom type. V and E can be determined by... and The last dimension is obtained by performing a maximum index operation. GN is the graph normalization layer. By sampling different q values ​​from P(q), MoFlow can generate different novel and efficient molecules.

[0091] Zero-sample text-to-graphic molecule generation method transforms text description x T As input. The only learnable parameter in the method is q, which is initialized by random sampling from P(q). All parameters of the pre-trained MoMu and MoFlow are frozen. Input x T The text is fed into MoMu's text encoder to obtain the text representation z. T q is input into MoFlow to obtain and To make all operations differentiable, thus allowing gradient backpropagation, Instead of inputting V into MoMu's graph encoder to obtain the graph representation z G The trained graph encoder GIN contains embeddings for all atom and bond types. The original V is used as an indicator sentence to select the corresponding embedding from the atom types in the first layer. When using... At that time, for each atom, the obtained representation is actually a weighted sum of all atom embeddings. The probabilities between the atomic types of each node are used as attention scores. The loss function is the projection z.T and z G Cosine similarity between them:

[0092] l q =-sim(z G ,z T ) / τ,

[0093] Where sim(z) G ,z T ) is used to calculate the similarity between projected representations in MoMu. q can be obtained by considering l q The gradient is backpropagated to update. The Adam optimizer is used for updates. After repeating the update for up to 500 iterations, the optimized q* is obtained, and then fed into MoFlow to obtain... and Finally, and The molecular graph G = (V, E) is obtained by performing a maximum value indexing operation on the last dimension.

[0094] exist Figure 4 The invention demonstrates molecules generated from molecular functional descriptions using the method of this invention. For descriptions of "fluorescent molecules," MolT5 failed to generate effective molecules, while the method of this invention generated molecules with conjugated double bonds or conjugated molecules. For the description "The molecule comprises hydroxyl and carboxyl groups, decomposes to produce ammonia, and has an oxygen content exceeding 20%" with four conditions, multiple molecules with hydroxyl groups, high oxygen content, and nitrogen atoms to produce ammonia were successfully generated, thus satisfying three-quarters of the conditions. Unlike existing AI-based molecule generation methods that can only generate specified properties, the method of this invention adaptively generates molecular candidates based on the input text, which can describe any desired one or more conditions. In the last description, three desired molecular properties were specified, including high water solubility, high barrier permeability, and low toxicity, which can be evaluated by fine-tuning the property prediction model. The method of this invention, based on MoMu-S and MoMu-K, can generate different molecules with high permeability, low toxicity, and high water solubility.

[0095] exist Figure 5 The diagram illustrates molecules generated from molecular structure descriptions. For descriptions containing nucleophilic groups, the method of the present invention generates different molecules having amino, hydroxyl, or double bonds. For descriptions containing electrophilic groups, the method of the present invention, while prohibiting formal charge, is still able to generate different molecules having carbonyl, alkyl-like, or halogen atoms. For descriptions containing hydrophilic groups, the method of the present invention can generate molecules containing hydroxyl, amino, or aldehydes with different structures. For descriptions containing lipophilic groups, the method of the present invention generates molecules containing alkyl-like groups, halogen atoms, or benzene rings with different structures.

[0096] The parts of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A method for training and applying a molecular multimodal model, characterized in that: Includes the following steps: S100: Establish a data collection unit to extract molecular diagrams and their semantically weakly related text data from published SCI papers and construct a pair of molecular diagram text datasets. S200, establish a molecular multimodal model and its pre-training unit, construct a molecular multimodal model containing a graph encoder and a text encoder, and train the model using contrastive learning. The specific methods for pre-training the molecular multimodal model include: S401, initialize the graph encoder with the self-supervised training weights of the graph isomorphic network, and initialize the text encoder with the pre-trained weights of BERT in Sci-BERT or KV-PLM. S402, for each training period, sample a batch of N molecular graph-text pairs from the training sequence in sequence; S403, for each small batch of data Two different augmentations are generated from each graph using random node deletion and random subgraphs respectively, resulting in a total of 2N augmented graphs. These graphs are then input into a graph encoder to obtain their representation vectors. ,in and Represents the i-th graph The representation of the two enhanced graphs; S404, randomly extract two different sentences from the document corresponding to each molecule. For the i-th graph, use a text encoder to extract the description. The representations obtained from the two different sentences are as follows , Each molecular graph in the mini-batch corresponds to two different sentences, producing 2N text representations. ; S405, for the i-th graph The total multi-view loss includes four representation pairs from the multimodal view. , , and The four contrasting losses, in, Corresponding contrast loss: Corresponding contrast loss: Corresponding contrast loss: Corresponding contrast loss: in It's a temperature parameter. This is the similarity calculation module, which first uses two independent mapping heads to... and Project onto the same dimension, and then calculate the cosine similarity between the projected vectors; S406, Calculate the graph modal contrast loss: The graph modal contrast loss of the i-th graph is: S407, calculate the sum of the losses for all samples within a batch: in It is a balance factor between cross-modal loss and graphical mode loss, and is a hyperparameter; S408, for each batch, update the parameters of the graph encoder, text encoder and mapping head by backpropagating the overall loss L, until all batches in the current epoch have been processed; S409, repeat S402~S408 until the preset maximum number of epochs is reached; S500 establishes a text-based molecular graph generation unit, applies the model to different downstream tasks such as cross-modal retrieval and text-based molecular graph generation, and finally outputs the generated molecular graph.

2. The method for training and applying a molecular multimodal model as described in claim 1, characterized in that: The method for constructing the dataset in the data collection unit includes: S201, collect the names, synonyms, and SMILES strings of the top 50K molecular compounds in PubChem; S202, for each collected molecule, use the smiles2graph function provided by OGB to convert its SMILES string into a molecular graph; S203: Using the name of a molecule as a query, search the abstracts, introductions, and conclusions of published scientific papers in the fields of medicine, biology, chemistry, and computer science in the S2orc database for sentences containing that name. Each retrieved sentence and its adjacent sentences are recorded as a paragraph in a document. If fewer than two paragraphs are retrieved by name, the search is repeated using the molecule's synonyms or aliases. The molecule search is terminated early when a specified number of paragraphs or a specified document size are retrieved. S204, obtain molecular graph-document pairs to form a multimodal molecular dataset.

3. The method for training and applying a molecular multimodal model as described in claim 2, characterized in that: The molecular multimodal model consists of a graph encoder and a text encoder, which extract molecular graph representations and text representations respectively; a graph isomorphic network is used as the graph encoder, and the BERT language model is used as the text encoder. During the training phase, the model additionally uses a similarity calculation module. This module uses two mapping heads to map the molecular graph and text representation to a joint representation space, and then calculates the cosine similarity of the mapped features. 。 4. The method for training and applying a molecular multimodal model as described in claim 3, characterized in that: The text-based molecular diagram generation method: It consists of a molecular multimodal model trained by a pre-training method and a pre-trained molecular generator based on random seed sampling and allowing backpropagation of gradients. The parameters of both the molecular multimodal model and the molecular generator are fixed. S501, Input a text describing a molecule. ; S502, Initialize and generate seed q; set q as a learnable parameter; S503 uses a trained molecular generator to generate a molecular diagram based on q. ; S504, respectively and The graph encoder and text encoder of the pre-trained molecular multimodal model are fed into the graph encoder and text encoder to extract the corresponding graph representations. and text representation Using the similarity calculation module of the molecular multimodal model, the negative similarity between the two is calculated as the loss function: S505, regarding losses Perform backpropagation and update the seed q using gradient descent; S506, repeat S503~S506 until the preset maximum number of epochs is reached; S507 feeds the optimized q into the molecular generator to generate the final molecular graph and outputs the molecular graph.

5. A computer storage medium, characterized in that: The method for training and applying a molecular multimodal model is embedded as described in any one of claims 1-4.

6. An electronic chip, characterized in that: The computer storage medium described in claim 5 is used.

Images