A molecular text joint understanding and editing method based on multi-modal contrast learning

Abstract

The application discloses a kind of molecular text joint understanding and editing method based on multi-modal contrast learning, to improve the semantic control ability and search efficiency of molecular design and optimization process.The method comprises: using molecular graph encoder and pre-training natural language text encoder, based on molecule-text pair, using symmetric InfoNCE loss to carry out contrast learning training, so as to obtain the model with good understanding to molecular structure and natural language text two modes.Through the design of adapter module, the pre-training molecular generator is aligned with the molecular text embedding space, and vector operation is carried out in the aligned latent space, realizing the molecular attribute editing and generation based on natural language;At the same time, vector similarity retrieval is used to realize the bidirectional retrieval task of molecular structure and text description.The method does not need to retrain the generation model for different tasks, has high flexibility and scalability, and is suitable for drug molecular design, property optimization and multi-modal chemical information processing and other scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of molecular representation learning and molecular generation, in particular to a molecular text joint understanding and editing method based on multi-modal contrast learning, which is suitable for drug molecule design, property optimization, structure retrieval and multi-modal chemical information processing scenarios. BACKGROUND

[0002] Drug molecule design is an important basis for chemical research such as drug discovery and material innovation. Traditional molecular design mainly relies on expert experience, SMILES-based sequence generation model or molecular graph neural network for prediction and generation, but these methods are difficult to directly control semantics through natural language text, lack flexible human-computer interaction ability, and limit the explainability and controllability of molecular generation.

[0003] In recent years, structure-text multi-modal learning methods have become a new direction for molecular design, which aligns molecular structure and natural language text description through contrast learning to achieve retrieval and semantic mapping. However, the existing methods generally have the following shortcomings: (1) molecular editing needs model fine-tuning on specific tasks, lacking universality; (2) it is difficult to directly use natural language text to modify molecular properties; (3) lacking a bidirectional retrieval and editing integrated framework, making it difficult to meet the needs of retrieval-design-verification closed loop in drug discovery process. Therefore, there is an urgent need for a unified framework that can edit and retrieve molecules through natural language. SUMMARY

[0004] To solve the above problems, the present application provides a molecular text joint understanding and editing method based on multi-modal contrast learning, which trains a structure-text shared embedding space through symmetric contrast learning, and introduces a latent space adapter to realize natural language driven molecular property editing, supports bidirectional retrieval and multi-task collaboration, and improves the semantic controllability, generation efficiency and interaction experience of the molecular design process.

[0005] The technical solution adopted by the present application to achieve the above purpose is:

[0006] A molecular text joint understanding and editing method based on multi-modal contrast learning, comprising the following steps:

[0007] 1) Obtain a molecule in the form of a SMILES string and corresponding molecular property description, and convert the molecule in the form of a SMILES string into a molecular graph structure;

[0008] 2) Encode the molecular graph structure and the molecular property description respectively to obtain molecular embedding vectors and text embedding vectors;

[0009] 3) Perform contrast learning on the molecular embedding vectors and the text embedding vectors through a contrast learning loss function to obtain a shared embedding space;

[0010] 4) By sharing the embedding space, respectively performing downstream text-driven molecular editing tasks and downstream molecule-text bidirectional retrieval tasks, realizing molecular structure editing and mutual retrieval between molecular structure and natural language description.

[0011] The step 2) comprises the following steps:

[0012] 2.1) using a pre-trained MolGT model as a molecular encoder to encode the molecular graph to obtain a molecular embedding vector;

[0013] 2.2) using a pre-trained PubMedBERT model as a text encoder to encode the natural language description corresponding to the molecular attribute to obtain a text embedding vector.

[0014] The step 3) comprises the following steps:

[0015] 3.1) constructing a molecule-text pair training data set, the data set containing N molecule-text pairs , wherein, is the molecular embedding, is the text embedding;

[0016] 3.2) for each matched sample, taking it as a positive example, and the remaining N-1 unmatched samples as negative examples, calculating the cosine similarity of the molecular embedding vector and the text embedding vector and normalizing, maximizing the similarity of the positive example pair and minimizing the similarity of the negative example pair using the symmetric InfoNCE loss function, and updating the parameters of the molecular encoder and the text encoder through back propagation, so that the molecular embedding and the text embedding converge to the same shared embedding space.

[0017] The symmetric InfoNCE loss function is used to calculate the similarity between the molecular vector and the text vector, that is:

[0018] ;

[0019] wherein, and represent the structure-text pair of each molecule, and are negative samples randomly sampled from a noise distribution, is an energy function with flexibility, , is a function composition, and represent two linear transformations, is the molecular structure corresponding vector, is the text corresponding vector.

[0020] The step 4) comprises the following steps:

[0021] 4.1) Align the molecular-text embedding space with the pre-trained molecule generator, and edit the molecular structure through latent space vector operations;

[0022] 4.2) Realize the mutual retrieval of the molecular structure and the natural language description based on the vector cosine similarity of the molecular embedding vector and the text embedding vector in the shared embedding space.

[0023] The step 4.1) comprises the following steps:

[0024] 4.1.1) Calculate the target molecular embedding vector :

[0025] ;

[0026] Wherein, represents the source molecular embedding vector, represents the source text embedding vector, is a weight coefficient in the range of [0, 1], used to control the degree of the source molecular embedding vector approaching the text embedding vector;

[0027] 4.1.2) Use the molecular generator to generate the corresponding molecular structure according to the target molecular embedding vector or obtain the structure candidate through vector similarity search.

[0028] The molecular generator is an autoregressive-based molecular decoder, or a VAE or diffusion model-based generator.

[0029] The step 4.2) comprises the following steps:

[0030] 4.2.1) Calculate the embedding vectors of the molecular structure and the N text descriptions as candidates in the shared space, and retrieve the text description in the N texts that is most likely to match the given molecular structure as the retrieval result;

[0031] 4.2.2) Calculate the embedding vectors of the text description and the N molecular structures as candidates in the shared space, and retrieve the molecular structure in the N molecular structures that is most likely to match the given text description as the retrieval result.

[0032] A molecular-text joint understanding and editing system based on multi-modal contrast learning, comprising:

[0033] A molecular structure data processing module for obtaining a molecule in the form of a SMILES string and a corresponding molecular attribute description, and converting the molecule in the form of a SMILES string into a molecular graph structure;

[0034] an encoder module for encoding the molecular graph structure and molecular property description respectively to obtain molecular embedding vectors and text embedding vectors;

[0035] a structure-text contrastive learning module for contrastive learning of the molecular embedding vectors and the text embedding vectors through a contrastive learning loss function to obtain a shared embedding space;

[0036] a molecule generation adaptation and bidirectional retrieval module for performing a downstream text-driven molecular editing task and a downstream molecule-text retrieval task respectively through the shared embedding space to realize molecular structure editing and mutual retrieval of molecular structure and natural language description.

[0037] The present application has the following advantages and benefits:

[0038] 1. Accelerated convergence: The pre-trained model has learned rich feature representations on large-scale data, which can make the model converge faster on new tasks as initialization.

[0039] 2. Improved performance: With existing knowledge, the model can more effectively extract useful information when processing structure and text data, thereby improving overall performance.

[0040] 3. Reduced training cost: Avoid training the entire model from scratch, saving computing resources and time.

[0041] 4. Natural language-driven molecular editing: Modify molecular properties directly through latent space vector operations without retraining the generative model.

[0042] 5. Support structure-text bidirectional retrieval: Improve the efficiency of molecular query, knowledge discovery and drug design.

[0043] 6. Strong model scalability: Compatible with different molecular generators, suitable for various molecular editing and generation tasks.

[0044] 7. Improve the interactive experience of molecular design: Complete the closed-loop process of retrieval, editing and design through a natural language interface, enhance explainability and ease of use. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 a schematic diagram of the present application based on a multi-modal contrastive learning molecular-natural language joint understanding framework;

[0046] Figure 2 a schematic diagram of the alignment of molecular representation and text representation in a shared space;

[0047] Figure 3 a schematic diagram of the present application text-driven molecular editing process overview;

[0048] Figure 4 N text description retrieval flowchart for a given molecule in the present application;

[0049] Figure 5 N molecular structure retrieval flowchart for a given text in the present application. DETAILED DESCRIPTION

[0050] The present application will be further described in detail below in conjunction with the accompanying drawings and examples.

[0051] A molecular-natural language joint understanding and editing method based on multi-modal contrast learning, comprising the following steps:

[0052] Step 1: molecular structure data processing module

[0053] Convert the input molecular SMILES representation into a 2D molecular graph structure. The 2D molecular graph is not in image format (such as PNG / JPG), but exists in the form of a "molecular graph structure", which is the graph structure format used in graph neural networks (GNN), not a visual image.

[0054] Use the MolGT molecular encoder to encode the molecular graph and output the molecular embedding vector.

[0055] Step 2: natural language encoder construction module

[0056] Use the PubMedBERT model to encode the natural language description and output the text embedding vector.

[0057] Step 3: structure-text contrast learning module

[0058] Use the symmetric InfoNCE loss to train the molecular-text pair for contrast learning and obtain the structure-text shared embedding space. Figure 2 Intuitively show how contrast learning makes molecular representation and text representation close in the shared space, from disordered distribution to paired alignment.

[0059] Step 4: molecular generation adaptation module

[0060] Through the latent space vector operation formula: Calculate the target molecular embedding vector, where represents the source molecular embedding vector, represents the source text embedding vector, is a weight coefficient in the range [0, 1], used to control the degree of the source molecular embedding vector approaching the text embedding vector, thereby adjusting the strength of the molecular structure modification.

[0061] The target molecule is embedded into a vector input into a pre-trained molecule generator or a vector similarity nearest neighbor search to obtain a target molecule structure that matches the natural language description.

[0062] Step 5: Molecule-Text Bidirectional Retrieval Module

[0063] Input a molecule structure and N text descriptions as candidates, calculate their embedding vectors in the shared space, and retrieve the text description in the N text that is most likely to match the given molecule structure as the retrieval result;

[0064] Input a text description and N molecule structures as candidates, calculate their embedding vectors in the shared space, and retrieve the molecule structure in the N molecule structures that is most likely to match the given text description as the retrieval result.

[0065] Embodiment

[0066] (1) Molecule-Text Contrastive Learning

[0067] In a cross-modal system (such as text molecule structure), the goal of contrastive learning is to project semantically related cross-modal data (such as a certain molecule structure and its descriptive text) into the same shared embedding space and make them as close as possible, while making unrelated pairs as far apart as possible. As shown in Figure 1 The molecule input is converted into a molecule embedding vector by the MolGT encoder, and the text input is converted into a text embedding vector by the PubMedBERT encoder. Both are contrastively learned by the symmetric InfoNCE loss to obtain a structure-text shared semantic space.

[0068] 1. Input and preprocessing

[0069] In an embodiment of the present application, the molecule input is in the form of a SMILES string, and is parsed into a two-dimensional molecule graph by a chemical toolkit (such as RDKit). The graph contains atomic nodes (node features include atomic number, charge state, hybridization mode, etc.) and chemical bond edges (edge features include bond type, whether aromatic, bond order, etc.).

[0070] The text input is a natural language description provided by the user, such as "make it more polar" "reduce toxicity" "increase solubility", etc. Before input, it is segmented by a text segmenter (such as WordPiece or BPE) and then mapped to a vector index.

[0071] 2. Encoder design

[0072] Molecule encoder MolGT is a graph neural network framework for molecule modeling tasks, aiming to enhance the modeling capability of traditional GNNs (such as GIN) on chemical tasks. With the MolGT model, the molecule graph is input into the graph neural network, and after multiple attention and graph convolution operations, the molecule embedding vector is output. The MolGT pre-training has captured the geometric, topological and chemical semantic features between atoms.

[0073] Text encoder The PubMedBERT model is used to perform Transformer encoding on the input text sequence, outputting a text embedding vector to capture semantic features in the chemical and biomedical context.

[0074] Although and are pre-trained models, they are pre-trained on other tasks or datasets (such as the molecule encoder may be pre-trained on the molecule property prediction task, and the text encoder is pre-trained on general text tasks), and our goal is to align molecules and texts in the same representation space, which may require fine-tuning the encoder to make them more suitable for this specific multi-modal alignment task.

[0075] 3. Embedding space alignment

[0076] The role of the projection layer is to map the features output by the encoder to the common space, but if the features of the encoder themselves are not good enough (such as the distribution difference is large), the projection layer may be difficult to learn well; updating the encoder and the projection layer together can make them "progress together", the encoder learns better feature representation, and the projection layer learns more accurate mapping.

[0077] To achieve cross-modal alignment, the invention uses a symmetric InfoNCE loss function. Specifically:

[0078] In the training phase, let a batch of training samples contain N molecule-text pairs , where is the molecule embedding, is the text embedding;

[0079] For each matched sample, the remaining N-1 unmatched samples are taken as negative examples; by calculating the cosine similarity and normalizing, the similarity of the positive example pair is maximized, and the similarity of the negative example pair is minimized;

[0080] The loss function is:

[0081] ;

[0082] where, and denotes the structure-text pair for each molecule, and is a negative sample randomly sampled from the noise distribution, using the empirical data distribution. is an energy function with flexibility, we use dot product on the co- learning space, i.e. , is the function composition, and two linear transformations (MLP, fully connected layers) or simpler affine mappings, to map the original molecule structure corresponding vector , text corresponding vector to a shared contrastive learning space.

[0083] By using contrastive learning to align molecules and texts, this learning approach is usually end-to-end; end-to-end optimization means that all parameters (including encoders and projection layers) need to participate in gradient calculation and update to maximize the overall performance of the model.

[0084] 4. Technical effects

[0085] Through this process, molecules and texts are mapped to a unified shared semantic space. In this space, molecules and texts with the same semantics are closer, and pairs with different semantics are farther apart, thereby laying the foundation for subsequent molecular editing and bidirectional retrieval.

[0086] (2) Natural language driven molecular editing

[0087] As shown in Figure 3 , given a molecule structure and editing instructions, first calculate the molecule and text embeddings, use latent vector operations to generate the target molecule embedding, and get the modified molecule structure through the generator or retrieval, to achieve attribute change and semantic alignment.

[0088] 1. Editing task definition

[0089] In another embodiment of the present application, users can input editing instructions through natural language to modify the attributes of existing molecules. For example: "reduce toxicity" → target: find a molecule with similar structure but lower toxicity; "increase solubility and decrease molecular weight" → target: find a molecule with higher solubility and lower molecular weight.

[0090] 2. Latent space operations

[0091] The application designs a lightweight adaptor for aligning the contrastive learning space and the latent space of the pre-trained molecule generator.

[0092] The specific steps are as follows: the input molecule is encoded into a vector , the text description is encoded into a vector , and the edited molecule vector is obtained through linear transformation:

[0093]

[0094] wherein, represents the source molecule embedding vector, represents the source text embedding vector, is a weight coefficient in the range of [0, 1], which is used to control the degree of the source molecule embedding vector approaching the text embedding vector, so as to adjust the strength of the molecular structure modification.

[0095] The input molecule generator or the nearest neighbor retrieval in the molecule database is performed to obtain the candidate molecule structure meeting the description.

[0096] 3. Molecule generator

[0097] The molecule generator can be an autoregressive-based molecule decoder combined with beam search to realize structure generation; or a VAE or diffusion model-based generator. In order to improve diversity, the application supports output of a candidate molecule set, which is then screened by a downstream evaluation function.

[0098] 4. Technical advantages

[0099] Without retraining the molecule generator, only vector operations in the shared latent space are needed;

[0100] Single-objective optimization (such as only increasing solubility) and multi-objective optimization (such as simultaneously reducing toxicity and molecular weight) can be supported at the same time;

[0101] Zero-shot editing is supported, that is, even if the target attribute does not appear in the training set, semantic embedding generalization can be achieved.

[0102] (3) Bidirectional retrieval task

[0103] As shown in Figure 4 and Figure 5 , the method supports:

[0104] Structure→text retrieval: input the molecular structure and N text descriptions as candidates, calculate the embedding vectors of the two in the shared space, and retrieve the text description in the N texts that is most likely to match the given molecular structure as the retrieval result;​

[0105] Text-to-Structure Retrieval: Given a text description and N molecular structures as candidates, compute the embedding vectors of both in the shared space, and retrieve the molecular structure that is most likely to match the given text description as the retrieval result.

[0106] 1. Structure-to-Text Retrieval

[0107] As shown in Figure 4 , the present application supports given a molecular structure and N text descriptions as candidates, returning the relevant ranking scores of natural language descriptions.

[0108] Input: molecular SMILES and N text descriptions as candidates;

[0109] Output: output the scores of the text describing the mechanism of action, use or properties of the molecule according to the matching degree;

[0110] Application scenario: drug discovery personnel input a new molecular structure, and the system returns the scores of semantic labels such as "likely to act as serotonin reuptake inhibitor"; "anti-inflammatory agent"; assist drug classification, drug efficacy prediction and literature retrieval.

[0111] 2. Text-to-Structure Retrieval

[0112] As shown in Figure 5 , the present application supports given a text description and N molecular structures as candidates, returning the ranking scores of the most matching molecular structures.

[0113] Input: natural language description (such as "an analgesic molecule with β-lactam ring") as N molecular structures as candidates;

[0114] Output: output the score ranking of the corresponding molecule according to the matching degree of the description and the given text description;

[0115] Application scenario: drug design personnel input ideal molecular characteristics, and the system retrieves potential candidate molecules; drug repurposing, find structurally diverse but functionally related molecules according to disease description or drug efficacy description.

[0116] 3. Retrieval Implementation

[0117] Input a molecule or text to get its embedding vector;

[0118] Calculate the cosine similarity with all vectors in the candidate set;

[0119] Select the top-N most similar results as output.

[0120] 4. Technical advantages

[0121] Support zero-shot retrieval: even if the description contains novel words, the model can achieve cross-modal matching through embedding semantics;

[0122] Support multi-level semantic retrieval: both high-level use description and low-level molecular attribute can be retrieved;

[0123] Implement structure-semantic closed loop: researchers can optimize molecular design through retrieval-editing-retrieval iteration.

[0124] Table 1 module description table

[0125] Use Role in molecular editing tasks Text encoder Encode user input descriptions (e.g. "make it more polar") into vectors Text tokenizer Convert text into token format for model input (e.g. WordPiece) Text embedding dimension Dimension of the vector output by the text_model Text mapping module Map the text_model output vector into the latent space of the contrastive learning pre-trained model Molecule encoder Encode raw molecules (SMILES or graphs) into vectors Molecule mapping module Map the output of the molecule encoder into the latent space of the contrastive learning pre-trained model Molecule embedding dimension Dimension of the molecule latent vector Molecule generator Generate new molecules (based on conditional latent) via decoder + beam search, etc. Mapper 1 Convert MolGT's latent -> into the latent space of the contrastive learning pre-trained model (for editing) Mapper 2 Convert edited latent -> back into the space required by MolGT (for generation)

[0126] Through the synergistic work of the above-mentioned table 1 modules, the present application realizes the deep integration of natural language and molecular structure in the process of molecular design, has high controllability, universality and interactivity, and can be widely applied to the fields of drug discovery, molecular retrieval, property optimization and intelligent molecular design.

[0127] The above is the preferred embodiment of the present application, and those skilled in the art can make various modifications and improvements to the present application without departing from the spirit and principles of the present application. These modifications and improvements should also be considered as the protection of the present application.

Claims

1. A multi-modal contrast learning based molecular text joint understanding and editing method, characterized in that, The method comprises the following steps: 1) obtaining a molecule in the form of a SMILES string and a corresponding molecular attribute description, and converting the molecule in the form of a SMILES string into a molecular graph structure; 2) encoding the molecular graph structure and the molecular attribute description respectively to obtain a molecular embedding vector and a text embedding vector; 3) performing contrastive learning on the molecular embedding vector and the text embedding vector through a contrastive learning loss function to obtain a shared embedding space; 4) performing a downstream text-driven molecular editing task and a downstream molecule-text bidirectional retrieval task through the shared embedding space to realize molecular structure editing and mutual retrieval between a molecular structure and a natural language description; The step 3) comprises the following steps: 3.1) constructing a molecule-text pair training dataset, said dataset comprising N molecule-text pairs wherein, is a molecule embedding, is a text embedding; 3.2) for each pair of matched samples, taking them as positive examples, and taking the remaining N-1 unmatched samples as negative examples, calculating the cosine similarity of the molecular embedding vector and the text embedding vector and normalizing, maximizing the similarity of the positive example pair and minimizing the similarity of the negative example pair by using a symmetric InfoNCE loss function, and updating the parameters of the molecular encoder and the text encoder through back propagation to make the molecular embedding and the text embedding converge to the same shared embedding space.

2. The method of claim 1, wherein, The step 2) comprises the following steps: 2.1) using a pre-trained MolGT model as a molecular encoder to encode the molecular graph to obtain a molecular embedding vector, wherein the MolGT is a graph neural network framework for molecular modeling tasks, and the target is to enhance the modeling capability of traditional GNN on chemical tasks, and the MolGT model is used to input the molecular graph into a graph neural network, and through multiple layers of attention and graph convolution operation, a molecular embedding vector is output, and the MolGT has captured the geometric, topological and chemical semantic features between atoms during pre-training; 2.2) using a pre-trained PubMedBERT model as a text encoder to encode the natural language description corresponding to the molecular attribute to obtain a text embedding vector.

3. The method of claim 1, wherein, Using a symmetric InfoNCE loss function Computing the similarity between the molecular vector and the text vector, i.e.: ； where, and denote the structure-text pair of each molecule, and are negative samples randomly sampled from the noise distribution, is an energy function with flexibility, , is the function composition, and denote two linear transformations, is the molecule structure corresponding vector, is the text corresponding vector, denotes the energy function with and as variables.

4. The method of claim 1, wherein, The step 4) comprises the following steps: 4.1) aligning the molecular-text embedding space with a pre-trained molecular generator in space, and editing the molecular structure through a latent space vector operation; 4.2) realizing mutual retrieval between a molecular structure and a natural language description based on the vector cosine similarity of the molecular embedding vector and the text embedding vector in the shared embedding space.

5. The method of claim 4, wherein, The step 4.1) comprises the following steps: 4.1.1) Calculating target molecule embedding vectors : ； wherein, denotes a source molecule embedding vector, denotes a source text embedding vector, is a weight coefficient located in the range [0, 1] for controlling the degree of the source molecule embedding vector approaching the text embedding vector; 4.1.2) using the molecular generator to generate a corresponding molecular structure according to the target molecular embedding vector or obtaining a structure candidate through vector similarity search.

6. The method of claim 5, wherein, The molecular generator is a molecular decoder based on autoregression, or a generator based on VAE or diffusion model.

7. The method of claim 4, wherein, The step 4.2) comprises the following steps: 4.2.1) calculating the embedding vectors of the molecular structure and N text descriptions as candidates in the shared space, and retrieving the text description that is most likely to match the given molecular structure from the N text descriptions as a retrieval result; 4.2.2) calculating the embedding vectors of the text description and N molecular structures as candidates in the shared space, and retrieving the molecular structure that is most likely to match the given text description from the N molecular structures as a retrieval result.

8. A multi-modal contrastive learning based molecular text joint understanding and editing system, characterized in that, comprises: A molecular structure data processing module is configured to obtain a molecule in a SMILES string form and a corresponding molecular attribute description, and convert the molecule in the SMILES string form into a molecular graph structure. An encoder module is configured to encode the molecular graph structure and the molecular attribute description respectively to obtain a molecular embedding vector and a text embedding vector. A structure-text contrastive learning module is configured to perform contrastive learning on the molecular embedding vector and the text embedding vector through a contrastive learning loss function to obtain a shared embedding space. A molecule generation adaptation and bidirectional retrieval module is configured to perform a downstream text-driven molecule editing task and a downstream molecule-text retrieval task through the shared embedding space, and implement molecule structure editing and mutual retrieval between the molecule structure and the natural language description. The structure-text contrastive learning module performs the following steps: 3.1) construct a molecule-text pair training dataset, said dataset containing N molecule-text pairs wherein, is a molecule embedding, is a text embedding; 3.2) For each pair of matched samples, the matched samples are taken as positive examples, and the remaining N-1 unmatched samples are taken as negative examples. The cosine similarity of the molecular embedding vector and the text embedding vector is calculated and normalized. The symmetric InfoNCE loss function is used to maximize the similarity of the positive example pairs and minimize the similarity of the negative example pairs. The parameters of the molecular encoder and the text encoder are updated through back propagation, so that the molecular embedding and the text embedding converge to the same shared embedding space.

Images