Molecular text joint understanding and editing method based on multi-modal comparative learning

Abstract

The invention discloses a molecular text joint understanding and editing method based on multi-modal comparative learning, and aims to improve the semantic control capability and retrieval efficiency of a molecular design and optimization process. The method comprises the steps that a molecular graph encoder and a pre-trained natural language text encoder are utilized, and symmetric InfoNCE loss is adopted based on a molecule-text pair to carry out comparative learning training, so that a model with good understanding on two modes of a molecular structure and a natural language text is obtained. An adapter module is designed to align a molecular text embedding space and a pre-training molecular generator, vector operation is carried out in an aligned potential space, and natural language-based molecular attribute editing and generation are realized; and meanwhile, vector similarity retrieval is adopted, so that a bidirectional retrieval task of a molecular structure and text description is realized. The method does not need to re-train a generation model for different tasks, has high flexibility and expandability, and is suitable for scenes of drug molecule design, property optimization, multi-modal chemical information processing and the like.

Description

Technical Field

[0001] This invention relates to the fields of molecular representation learning and molecular generation technology, and in particular to a method for joint understanding and editing of molecular text based on multimodal contrastive learning, applicable to drug molecule design, property optimization, structure retrieval and multimodal chemical information processing scenarios. Background Technology

[0002] Drug molecule design is a crucial foundation for chemical research such as drug discovery and materials innovation. Traditional molecular design mainly relies on expert experience, SMILES-based sequence generation models, or molecular graph neural networks for prediction and generation. However, these methods are difficult to control semantically directly through natural language text, lack flexible human-computer interaction capabilities, and limit the interpretability and controllability of molecular generation.

[0003] In recent years, structure-text multimodal learning methods have become a new direction in molecular design, aligning molecular structures with natural language text descriptions through contrastive learning to achieve retrieval and semantic mapping. However, existing methods generally suffer from the following shortcomings: (1) molecular editing requires model fine-tuning for specific tasks, lacking universality; (2) it is difficult to directly modify molecular properties using natural language text; and (3) there is a lack of a bidirectional retrieval and editing integrated framework, making it difficult to meet the needs of the retrieval-design-validation closed loop in the drug discovery process. Therefore, a unified framework for molecular editing and retrieval via natural language is urgently needed. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a molecular text joint understanding and editing method based on multimodal contrastive learning. It trains a structure-text shared embedding space through symmetric contrastive learning and introduces a latent space adapter to achieve natural language-driven molecular attribute editing. This method supports bidirectional retrieval and multi-task collaboration, improving the semantic controllability, generation efficiency, and interactive experience of the molecular design process.

[0005] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0006] A molecular text joint understanding and editing method based on multimodal contrastive learning includes the following steps:

[0007] 1) Obtain the molecules in SMILES string format and their corresponding molecular attribute descriptions, and convert the molecules in SMILES string format into molecular graph structures;

[0008] 2) Encode the molecular graph structure and molecular attribute descriptions separately to obtain molecular embedding vectors and text embedding vectors;

[0009] 3) By comparing and learning the molecular embedding vector and the text embedding vector through a contrastive learning loss function, a shared embedding space is obtained;

[0010] 4) By sharing the embedding space, downstream text-driven molecular editing tasks and downstream molecular-text bidirectional retrieval tasks are carried out respectively, realizing molecular structure editing and mutual retrieval of molecular structure and natural language description.

[0011] Step 2) includes the following steps:

[0012] 2.1) A pre-trained MolGT model is used as a molecular encoder to encode the molecular graph and obtain molecular embedding vectors;

[0013] 2.2) The pre-trained PubMedBERT model is used as a text encoder to encode the natural language descriptions corresponding to the molecular attributes, and the text embedding vectors are obtained.

[0014] Step 3) includes the following steps:

[0015] 3.1) Construct a molecular-text pair training dataset, which contains N molecular-text pairs. ,in, For molecular intercalation, Embedded in text;

[0016] 3.2) For each pair of matching samples, treat each pair of matching samples as a positive example and the remaining N-1 non-matching samples as negative examples. Calculate and normalize the cosine similarity between the molecular embedding vector and the text embedding vector. Use the symmetric InfoNCE loss function to maximize the similarity of positive pairs and minimize the similarity of negative pairs. Update the parameters of the molecular encoder and the text encoder through backpropagation so that the molecular embedding and the text embedding converge to the same shared embedding space.

[0017] Using the symmetric InfoNCE loss function Calculate the similarity between the molecular vector and the text vector, i.e.:

[0018] ;

[0019] in, and Structure-text pairs representing each molecule. and These are negative samples randomly sampled from the noise distribution. It is a flexible energy function. , It is composed of functions. and Represent two linear transformations, This is the vector corresponding to the molecular structure. This is the vector corresponding to the text.

[0020] Step 4) includes the following steps:

[0021] 4.1) Align the molecular-text embedding space with the pre-trained molecular generator and edit the molecular structure using latent space vector operations;

[0022] 4.2) Mutual retrieval of molecular structure and natural language description is achieved based on the vector cosine similarity between molecular embedding vectors and text embedding vectors in the shared embedding space.

[0023] Step 4.1) includes the following steps:

[0024] 4.1.1) Calculate the target molecule embedding vector :

[0025] ;

[0026] in, Represents the source molecule embedding vector. Represents the source text embedding vector. These are weighting coefficients located in the range [0,1], used to control how closely the source molecular embedding vector approaches the text embedding vector;

[0027] 4.1.2) Use a molecular generator to generate the corresponding molecular structure based on the target molecule embedding vector or obtain structural candidates through vector similarity search.

[0028] The molecular generator is an autoregressive molecular decoder, or a generator based on a VAE or diffusion model.

[0029] Step 4.2) includes the following steps:

[0030] 4.2.1) Calculate the embedding vectors of the molecular structure and the N candidate text descriptions in the shared space, and retrieve the text description that is most likely to match the given molecular structure from the N texts as the retrieval result;

[0031] 4.2.2) Calculate the embedding vectors of the text description and the N candidate molecular structures in the shared space, and retrieve the molecular structure among the N molecular structures that is most likely to match the given text description as the retrieval result.

[0032] A molecular text joint understanding and editing system based on multimodal contrastive learning, comprising:

[0033] The molecular structure data processing module is used to obtain molecules in SMILES string format and their corresponding molecular attribute descriptions, and to convert molecules in SMILES string format into molecular graph structures.

[0034] The encoder module is used to encode the molecular graph structure and molecular attribute description respectively to obtain molecular embedding vectors and text embedding vectors;

[0035] The structure-text contrast learning module is used to compare and learn the molecular embedding vector and the text embedding vector through a contrast learning loss function to obtain a shared embedding space;

[0036] The molecular generation, adaptation, and bidirectional retrieval module is used to perform downstream text-driven molecular editing tasks and downstream molecular-text retrieval tasks respectively by sharing an embedding space, thereby realizing molecular structure editing and mutual retrieval between molecular structures and natural language descriptions.

[0037] The present invention has the following beneficial effects and advantages:

[0038] 1. Accelerated convergence: Pre-trained models have already learned rich feature representations on large-scale data, which can be used as initialization to enable the model to converge faster on new tasks.

[0039] 2. Improve performance: By leveraging existing knowledge, the model can extract useful information more effectively when processing structured and textual data, thereby improving overall performance.

[0040] 3. Reduce training costs: Avoid training the entire model from scratch, saving computing resources and time.

[0041] 4. Enable natural language-driven molecular editing: Directly modify molecular properties through latent space vector operations without retraining the generative model.

[0042] 5. Supports bidirectional structure-text retrieval: improves the efficiency of molecular query, knowledge discovery, and drug design.

[0043] 6. Highly scalable model: Compatible with different molecular generators, suitable for a variety of molecular editing and generation tasks.

[0044] 7. Enhance the interactive experience of molecular design: Complete the closed-loop process of retrieval, editing and design through a natural language interface, enhancing interpretability and ease of use. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the molecular-natural language joint understanding framework based on multimodal contrastive learning of this invention;

[0046] Figure 2 This is a schematic diagram illustrating the alignment of molecular representation and text representation in a shared space.

[0047] Figure 3 This is an overview diagram of the text-driven molecular editing process in this invention;

[0048] Figure 4 This is a flowchart of the N-text description retrieval process under a given molecule in this invention;

[0049] Figure 5 This is a flowchart of the N-molecule structure retrieval process given the text in this invention. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0051] A molecular-natural language joint understanding and editing method based on multimodal contrastive learning includes the following steps:

[0052] Step 1: Molecular structure data processing module

[0053] The input SMILES representation is converted into a 2D molecular graph structure. The 2D molecular graph is not an image format (such as PNG / JPG), but exists in the form of a "molecular graph structure", which is the graph structure format used in graph neural networks (GNNs), rather than a visual image.

[0054] The molecular graph is encoded using the MolGT molecular encoder, which outputs molecular embedding vectors.

[0055] Step 2: Natural Language Encoder Construction Module

[0056] The PubMedBERT model is used to encode natural language descriptions and output text embedding vectors.

[0057] Step 3: Structure-Text Comparison Learning Module

[0058] The symmetric InfoNCE loss is used to train the molecular-text pairs through contrastive learning to obtain the structure-text shared embedding space. Figure 2 The visual demonstration shows how contrastive learning brings molecular representations and text representations closer together in a shared space, transforming them from a disordered distribution to a paired alignment.

[0059] Step 4: Molecular Generation Adaptation Module

[0060] Formulas using vector operations in latent space: Calculate the target molecule embedding vector, where, Represents the source molecule embedding vector. Represents the source text embedding vector. The weight coefficients are located in the range [0,1] and are used to control how close the source molecular embedding vector is to the text embedding vector, thereby adjusting the intensity of molecular structure modification.

[0061] The target molecule embedding vector is input into a pre-trained molecule generator or a vector similarity nearest neighbor search is performed to obtain the target molecule structure that conforms to the description in natural language.

[0062] Step 5: Molecular-Text Bidirectional Retrieval Module

[0063] Given a molecular structure and N candidate text descriptions, calculate their embedding vectors in the shared space, and retrieve the text description from the N texts that is most likely to match the given molecular structure as the retrieval result.

[0064] Given a text description and N candidate molecular structures, calculate their embedding vectors in a shared space, and retrieve the molecular structure among the N structures that most likely matches the given text description as the search result.

[0065] Example

[0066] (1) Molecular-text contrastive learning

[0067] In cross-modal systems (such as text) Molecular structures), and contrastive learning aims to project semantically related heteromodal data (such as a molecular structure and its descriptive text) into the same shared embedding space, making them as close as possible while keeping unrelated pairs as far apart as possible. For example... Figure 1 As shown, molecular input is converted into molecular embedding vectors by the MolGT encoder, and text input is converted into text embedding vectors by the PubMedBERT encoder. The two are compared and learned through symmetric InfoNCE loss to obtain a structure-text shared semantic space.

[0068] 1. Input and Preprocessing

[0069] In one embodiment of the present invention, the molecular input is in the form of a SMILES string and is parsed into a two-dimensional molecular graph using a chemical toolkit (such as RDKit). The graph contains atomic nodes (node ​​features include atomic number, charge state, hybridization, etc.) and chemical bond edges (edge ​​features include bond type, aromaticity, bond order, etc.).

[0070] The text input uses user-provided natural language descriptions, such as "make it more polar", "reduce toxicity", "increase solubility", etc. Before input, it is segmented by a text segmenter (such as WordPiece or BPE) and then mapped to vector indices.

[0071] 2. Encoder Design

[0072] Molecular encoder MolGT is a graph neural network framework (GraphTransformer) designed for molecular modeling tasks. Its goal is to enhance the modeling capabilities of traditional GNNs (such as GINs) in chemical tasks. Using the MolGT model, a molecular graph is input into the graph neural network, and after multiple layers of attention and graph convolution operations, the molecular embedding vector is output. MolGT is pre-trained to capture the geometric, topological, and chemical semantic features between atoms.

[0073] Text encoder The PubMedBERT model is used to encode the input text sequence using Transformer and output text embedding vectors to capture semantic features in chemical and biomedical contexts.

[0074] Although and These are pre-trained models, but they were pre-trained on other tasks or datasets (for example, a molecular encoder might be pre-trained on a molecular property prediction task, and a text encoder might be pre-trained on a general text task). Our goal is to align molecules and text in the same representation space, which may require fine-tuning the encoders to make them more suitable for this specific multimodal alignment task.

[0075] 3. Embedded Space Alignment

[0076] The role of the projection layer is to map the features output by the encoder to a common space. However, if the features of the encoder itself are not good enough (e.g., the distribution is very different), the projection layer may have difficulty learning well. Updating both the encoder and the projection layer at the same time allows them to "improve together", with the encoder learning better feature representations and the projection layer learning more accurate mappings.

[0077] To achieve cross-modal alignment, this invention uses a symmetric InfoNCE loss function. Specifically:

[0078] During the training phase, a batch of training samples is assumed to contain N molecule-text pairs. ,in, For molecular intercalation, Embedded in text;

[0079] For each pair of matching samples, the remaining N-1 non-matching samples are treated as negative examples; by calculating and normalizing the cosine similarity, the similarity of positive example pairs is maximized and the similarity of negative example pairs is minimized.

[0080] The Loss function takes the form of:

[0081] ;

[0082] in, and Structure-text pairs representing each molecule. and It consists of negative samples randomly sampled from a noise distribution, using an empirical data distribution. It is a flexible energy function, and we use the dot product in the common learning space, that is... , It is composed of functions. and Two linear transformations (MLP, fully connected layers) or simpler affine mappings are used to map the original molecular structure to a vector. Text-to-vector Mapped to a shared comparative learning space.

[0083] Aligning molecules and text using contrastive learning is typically an end-to-end approach; end-to-end optimization means that all parameters (including the encoder and projection layers) need to participate in gradient calculation and updates in order to maximize the overall performance of the model.

[0084] 4. Technical Effects

[0085] Through this process, molecules and texts are mapped to a unified, shared semantic space. In this space, molecules and texts with the same semantics are closer together, while pairs with different semantics are further apart, thus laying the foundation for subsequent molecule editing and bidirectional retrieval.

[0086] (2) Natural language driven molecular editing

[0087] like Figure 3 As shown, given the molecular structure and editing instructions, the molecular and text embeddings are first calculated using latent vector operations. The target molecule embedding is generated, and the modified molecular structure is obtained through generator or retrieval, realizing attribute change and semantic alignment.

[0088] 1. Edit task definition

[0089] In another embodiment of the invention, the user can modify the properties of existing molecules by inputting editing commands in natural language. For example: "reduce toxicity" → target: find molecules with similar structures but lower toxicity; "increase solubility and decrease molecular weight" → target: find molecules that are more soluble and have a lower molecular weight.

[0090] 2. Latent space operations

[0091] This invention designs a lightweight adapter for aligning the contrastive learning space with the latent space of a pre-trained molecule generator.

[0092] The specific steps are as follows: Encode the input molecule into a vector. The text description is encoded as a vector. The edited molecular vector is obtained through linear transformation:

[0093]

[0094] in, Represents the source molecule embedding vector. Represents the source text embedding vector. The weight coefficients are located in the range [0,1] and are used to control how close the source molecular embedding vector is to the text embedding vector, thereby adjusting the intensity of molecular structure modification.

[0095] Will Input the molecular generator or perform a nearest neighbor search in the molecular database to obtain candidate molecular structures that match the description.

[0096] 3. Molecular generator

[0097] The molecular generator can be an autoregressive molecular decoder combined with beam search to generate structures; or it can be a generator based on VAE or diffusion models. To improve diversity, this invention supports outputting a candidate molecule set, which is then filtered by a downstream evaluation function.

[0098] 4. Technological advantages

[0099] There is no need to retrain the molecular generator; vector operations can be performed simply in the shared latent space.

[0100] It can simultaneously support single-objective optimization (such as increasing solubility only) and multi-objective optimization (such as reducing toxicity and molecular weight at the same time).

[0101] It supports zero-shot editing, which can be achieved through semantic embedding generalization even if the target attribute has not appeared in the training set.

[0102] (3) Bidirectional retrieval task

[0103] like Figure 4 and Figure 5 As shown, this method supports:

[0104] Structure → Text Retrieval: Input a molecular structure and N candidate text descriptions, calculate their embedding vectors in the shared space, and retrieve the text description from the N texts that is most likely to match the given molecular structure as the retrieval result;

[0105] Text → Structure Retrieval: Input a text description and N candidate molecular structures, calculate their embedding vectors in the shared space, and retrieve the molecular structure among the N molecular structures that is most likely to match the given text description as the retrieval result.

[0106] 1. Structure → Text Retrieval

[0107] like Figure 4 As shown, this invention supports returning a ranking score of a natural language description given a molecular structure and N candidate text descriptions.

[0108] Input: SMILES molecules and N text descriptions as candidates;

[0109] Output: Output a score based on the matching degree of the text describing the molecule's mechanism of action, uses, or properties;

[0110] Application scenarios: Drug discoverers input new molecular structures, and the system returns scores for semantic tags such as "likely to act asserotonin reuptake inhibitor" and "anti-inflammatory agent"; it also assists in drug classification, efficacy prediction, and literature retrieval.

[0111] 2. Text → Structure Search

[0112] like Figure 5 As shown, this invention supports returning the ranking score of the best-matching molecular structure given a text description and N candidate molecular structures.

[0113] Input: Natural language descriptions (e.g., "an analgesic molecule with β-lactam ring") as N candidate molecular structures;

[0114] Output: Rank the corresponding molecules based on their matching scores with the given text description, according to their matching degree.

[0115] Application scenarios: Drug designers input ideal molecular characteristics, and the system searches for potential candidate molecules; drug repurposing: based on disease descriptions or efficacy descriptions, it finds structurally diverse but functionally related molecules.

[0116] 3. Search Implementation Method

[0117] Input a molecule or text to obtain its embedding vector;

[0118] Calculate the cosine similarity with all vectors in the candidate set;

[0119] Select the results with the highest Top-N similarity as the output.

[0120] 4. Technological Advantages

[0121] Supports zero-shot retrieval: Even if the description contains novel words, the model can achieve cross-modal matching by embedding semantics;

[0122] Supports multi-level semantic retrieval: it can retrieve both high-level usage descriptions and low-level molecular attributes;

[0123] Achieving a structure-semantic closed loop: Researchers can iteratively optimize molecular designs through retrieval-editing-re-retrieval.

[0124] Table 1 Module Description Table

[0125] use Its role in molecular editing tasks Text encoder Encode the user's input description (such as "make it more polar") into a vector. Text segmenter Convert the text into a token format (such as WordPiece) for the model input. Text embedding dimension Indicates the vector dimension of the text_model output. Text mapping module Map the text_model output vector to the latent space of the contrastive learning pre-trained model. Molecular encoder Encode the original molecules (SMILES or graphs) into vectors Molecular mapping module Map the output of the molecular encoder to the latent space of the contrastive learning pre-trained model. Molecular embedding dimension The dimension of the latent vector of the molecule Molecular generator New molecules are generated using decoder + beam search (based on conditional latency). Mapper 1 Transform the latent space of MolGT to the latent space of the contrastive learning pre-trained model (for editing). Mapper 2 Convert the edited latent to the required space for MolGT (for generation).

[0126] Through the collaborative work of the modules in Table 1 above, this invention achieves deep integration of natural language and molecular structure in the molecular design process, possessing high controllability, versatility, and interactivity, and can be widely applied in fields such as drug discovery, molecular retrieval, property optimization, and intelligent molecular design.

[0127] The above description represents the preferred embodiments of the present invention. Those skilled in the art can make various modifications and improvements to the present invention without departing from its spirit and principles, and such modifications and improvements should also be considered as protections of the present invention.

Claims

1. A method for joint understanding and editing of molecular text based on multimodal contrastive learning, characterized in that, Includes the following steps: 1) Obtain the molecules in SMILES string format and their corresponding molecular attribute descriptions, and convert the molecules in SMILES string format into molecular graph structures; 2) Encode the molecular graph structure and molecular attribute descriptions separately to obtain molecular embedding vectors and text embedding vectors; 3) By comparing and learning the molecular embedding vector and the text embedding vector through a contrastive learning loss function, a shared embedding space is obtained; 4) By sharing the embedding space, downstream text-driven molecular editing tasks and downstream molecular-text bidirectional retrieval tasks are carried out respectively, realizing molecular structure editing and mutual retrieval of molecular structure and natural language description; Step 3) includes the following steps: 3.1) Construct a molecular-text pair training dataset, which contains N molecular-text pairs. ,in, For molecular intercalation, Embedded in text; 3.2) For each pair of matching samples, it is taken as a positive example and the remaining N-1 non-matching samples are taken as negative examples. The cosine similarity between the molecular embedding vector and the text embedding vector is calculated and normalized. The symmetric InfoNCE loss function is used to maximize the similarity of positive pairs and minimize the similarity of negative pairs. The parameters of the molecular encoder and the text encoder are updated through backpropagation so that the molecular embedding and the text embedding converge to the same shared embedding space.

2. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 1, characterized in that, Step 2) includes the following steps: 2.1) A pre-trained MolGT model is used as a molecular encoder to encode the molecular graph and obtain molecular embedding vectors; 2.2) The pre-trained PubMedBERT model is used as a text encoder to encode the natural language descriptions corresponding to the molecular attributes, and the text embedding vectors are obtained.

3. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 1, characterized in that, Using the symmetric InfoNCE loss function Calculate the similarity between the molecular vector and the text vector, i.e.: ； in, and Structure-text pairs representing each molecule. and These are negative samples randomly sampled from the noise distribution. It is a flexible energy function. , It is composed of functions. and Represent two linear transformations, This is the vector corresponding to the molecular structure. This is the vector corresponding to the text. Indicated by and The energy function is a variable.

4. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 1, characterized in that, Step 4) includes the following steps: 4.1) Align the molecular-text embedding space with the pre-trained molecular generator and edit the molecular structure using latent space vector operations; 4.2) Mutual retrieval of molecular structure and natural language description is achieved based on the vector cosine similarity between molecular embedding vectors and text embedding vectors in the shared embedding space.

5. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 4, characterized in that, Step 4.1) includes the following steps: 4.1.1) Calculate the target molecule embedding vector : ； in, Represents the source molecule embedding vector. Represents the source text embedding vector. These are weighting coefficients located in the range [0,1], used to control how closely the source molecular embedding vector approaches the text embedding vector; 4.1.2) Use a molecular generator to generate the corresponding molecular structure based on the target molecule embedding vector or obtain structural candidates through vector similarity search.

6. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 5, characterized in that, The molecular generator is an autoregressive molecular decoder, or a generator based on a VAE or diffusion model.

7. The molecular text joint understanding and editing method based on multimodal contrastive learning according to claim 4, characterized in that, Step 4.2) Includes the following steps: 4.2.1) Calculate the embedding vectors of the molecular structure and the N candidate text descriptions in the shared space, and retrieve the text description that is most likely to match the given molecular structure from the N texts as the retrieval result; 4.2.2) Calculate the embedding vectors of the text description and the N candidate molecular structures in the shared space, and retrieve the molecular structure among the N molecular structures that is most likely to match the given text description as the retrieval result.

8. A molecular text joint understanding and editing system based on multimodal contrastive learning, characterized in that, include: The molecular structure data processing module is used to obtain molecules in SMILES string format and their corresponding molecular attribute descriptions, and to convert molecules in SMILES string format into molecular graph structures. The encoder module is used to encode the molecular graph structure and molecular attribute description respectively to obtain molecular embedding vectors and text embedding vectors; The structure-text contrast learning module is used to compare and learn the molecular embedding vector and the text embedding vector through a contrast learning loss function to obtain a shared embedding space; The molecular generation, adaptation, and bidirectional retrieval module is used to perform downstream text-driven molecular editing tasks and downstream molecular-text retrieval tasks respectively by sharing an embedding space, thereby realizing molecular structure editing and mutual retrieval between molecular structures and natural language descriptions.

Images