Monomolecular structure design and generation method based on large language model

Abstract

The invention discloses a monomolecular structure design and generation method based on a large language model, and relates to the crossing field of monomolecular electronics and artificial intelligence. In order to solve the problems of low single molecule design efficiency and insufficient data, SingMolT5 is obtained through field fine tuning on the basis of MolT5, and generation from a natural language to a molecular SMILES is realized. According to the method, data are collected from literatures in the single molecule field and an existing disclosed molecule library, a fine tuning data set containing 329 high-quality instructions is constructed, and the fine tuning data set comprises three basic data types including a molecular skeleton, an anchoring group and molecules. The model is initialized by a MolT5 check point, and a cross entropy loss function is used in the fine tuning process; reasoning, generating and using a beam search strategy; the follow-up evaluation indexes comprise BLEU, Levenshtein, MACCS, RDK, Morga and effectiveness. The method provides an innovative technical path for'low experience and data dependent type single molecule design 'in the field of single molecule electronics, and is particularly suitable for aided design of a single molecule device core function unit and a single molecule sensing probe molecule, namely, a target molecular structure can be quickly converted through a natural language, the design threshold of researchers is reduced, and the design efficiency is improved. And a traceable technical basis can be provided for screening and iteration of a monomolecular structure based on a high-quality data set and rigorous evaluation logic.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of single-molecule electronics and artificial intelligence, and in particular to a method for designing and generating single-molecule structures based on Large Language Model (LLM). Specifically, it belongs to the field of single-molecule electronics, which uses an LLM-driven molecular generation model as a technical support to achieve precise design and generation of single-molecule structures. Background Technology

[0002] Single-molecule electronics, an important branch of nanoelectronics, focuses on the electrical transport properties of individual molecules. It provides a unique research platform for developing molecular devices and ultra-sensitive molecular sensing and detection technologies, playing a crucial role in solving key scientific problems in physics, chemistry, and life sciences (Science Bulletin, 2023, 68, 2197−2212). Currently, researchers have designed and synthesized various types of molecules with specific electrical functions, and numerous single-molecule-scale prototype devices, such as molecular rectifiers and molecular memristors, have been reported, demonstrating their potential value in breaking through the limitations of traditional silicon-based devices in miniaturization.

[0003] In single-molecule electronics research, to obtain single molecules with specific electrical transport properties and corresponding device functions, it is usually necessary to first measure and characterize the electrical properties of individual molecules experimentally, and then select or iteratively design single molecules that meet functional requirements based on this. In actual electrical measurement experiments, to achieve reproducible coupling between molecules and electrodes, the single-molecule structure can often be abstracted as a combination of "molecular skeleton + anchoring group". The anchoring group (such as thiols, amino groups, pyridyl groups, etc.) is used to form stable bonds or coordination with metal electrodes (such as gold, silver, etc.), thereby constructing a molecular junction that can perform electrical measurements. The conductivity, rectification ratio, on / off ratio, noise spectrum, and other characterization results obtained from the analysis of the measured single-molecule electrical transport signal data are used to guide the structural modification of molecules and device-level design.

[0004] However, existing single-molecule design-validation processes generally face the following problems in practice:

[0005] (1) High reliance on experts. The design of candidate molecular structures, selection of substituents and anchor sites largely depend on empirical rules, making it difficult to systematically cover the vast chemical space;

[0006] (2) Long cycle and high cost. From molecular design and quantitative calculation screening to synthesis and single-molecule electrical testing, there are many steps, high trial and error costs, and limited sample throughput;

[0007] (3) Limited search efficiency. The combined effect of multiple objectives (such as electrode anchoring strength, molecular junction stability, specific electrotransport properties and synthesizability) makes it difficult for traditional heuristic or local optimization strategies to explore efficiently;

[0008] Artificial intelligence-assisted prediction and generation of molecular properties is a current research frontier and hot topic, and has achieved groundbreaking applications in areas such as protein structure prediction, as exemplified by AlphaFold (Nature, 2021, 596, 583−589) and RoseTTAFold (Science, 2021, 373, 871−876). Prior to the advent of LLMs, various deep learning models were already used for molecular space exploration, such as sequence-based recurrent neural networks (RNNs), variational autoencoders (VAEs), generative adversarial networks (GANs), and diffusion models. Segler et al. trained the SMILES molecule using an RNN model and then combined it with reinforcement learning or transfer learning to generate molecules with desired properties (ACS Cent. Sci., 2018, 4, 120−131). Gupta et al. also used an RNN architecture to design molecules from scratch (Mol. Inf., 2018, 37, 1700111). Regarding Auto-Encoder variants, researchers used VAEs, adversarial autoencoders, and the Junction Tree VAE proposed by Jin et al. to model molecular distributions, generating molecules by encoding molecules into latent vectors and then decoding them (PMLR ICML, 2018, pages 2323–2332). In the GANs field, Putin et al.'s RANC combined reinforcement learning and GAN loss functions to optimize molecular properties (Mol. Pharmaceutics, 2018, 15). (4386−4397), Prykhodko et al.'s LatentGAN utilizes a decoder of a pre-trained autoencoder to process latent vectors to achieve molecular generation (J. Cheminf. 2019, 11, 74). However, deep learning models at this stage have limitations such as weak ability to handle long-range dependencies, susceptibility to mode collapse, insufficient control over the coordinated properties of multiple molecules, the need for manually designed optimization objectives, and difficulty in autonomously exploring a vast chemical space, which prevent them from achieving truly autonomous exploration.

[0009] With the rise of text-molecule cross-modal learning and LLM, researchers have begun to explore semantic alignment between linear molecular representations such as SMILES and SELFIES and natural language descriptions, and have attempted to use LLMs for molecular generation. Bagal et al. proposed MolGPT (J. Chem. Inf. Model., 2022, 62, 2064–2076), based on a transformer decoder. MolGPT learns certain specific molecular properties (such as logP, TPSA, SAS, QED), maps these properties to feature vectors, combines them, and then trains the model as conditions for molecular generation. MolGPT outperforms other modern machine learning frameworks in generating effective, unique, and novel molecules. However, its training and generation rely on numerical property conditions and it still cannot directly understand instructions in natural language form. However, a landmark work by Edwards et al. in recent years, MolT5, directly established the bidirectional understanding ability of LLMs of the semantic relationship between natural language descriptions and molecular SMILES representations, further intelligentizing molecular design generation (ACL EMNLP, 2022, pages 375–413). LLMs can generate molecules that meet semantic requirements simply by receiving natural language from researchers, further eliminating feature engineering steps and greatly simplifying the entire design process. This type of method shows potential in metrics such as "effectiveness, uniqueness, and novelty," and to some extent reduces feature engineering and manual condition design.

[0010] Nevertheless, significant gaps and challenges remain regarding the specific needs of single-molecule electronics:

[0011] 1) There is a lack of specialized knowledge organization for "molecular backbone-anchoring group" structural units, resulting in insufficient adaptability to single-molecule design scenarios;

[0012] 2) Limited conditional expression. Most rely on numerical or manually designed property vectors as conditions, making it difficult to directly understand and execute complex natural language design intentions;

[0013] 3) The evaluation-feedback loop is weak. There is a lack of standards for feedback from text instructions to molecular generation, and then to quantitative calculation and experimental characterization, making it difficult to form efficient iterations.

[0014] Given the current situation, there is an urgent need for a method that can precisely align the design intent of natural language with the structural semantics of the "molecular skeleton-anchoring group" in the context of single-molecule electrical research, and explicitly introduce chemical and device constraints and support closed-loop optimization during the generation process. This would improve design efficiency, controllability, and feasibility, reduce trial-and-error costs, and expand the explorable molecular space. This invention proposes a solution to the above problems. Summary of the Invention

[0015] This invention proposes a method for single-molecule structure design and generation based on a large language model. We implemented natural language-guided single-molecule structure design by fine-tuning the MolT5 model, naming it SingMolT5. This invention is the first application of LLMs in the single-molecule domain. Figure 1 When generating molecules, we also use the same molecular representation as MolT5, namely SMILES. In the single-molecule field, molecules possess unique characteristics; therefore, we collected relevant molecules from literature and experimental reports in the single-molecule field, as well as existing publicly available molecular libraries, and constructed a single-molecule fine-tuning dataset consisting of "text description – SMILES" pairs. Through fine-tuning with this data, SingMolT5 has established a more accurate understanding of the relevant structures and anchoring groups in the single-molecule field, thereby effectively improving the quality of single-molecule generation. This invention has certain theoretical significance and forward-looking potential, bringing new possibilities to the exploration of the single-molecule field. The technical solution adopted includes the following steps:

[0016] 1) Design and Construction of a Single-Molecular Fine-Tuning Dataset. To enable the model to effectively understand single molecules, we constructed a fine-tuning dataset that conforms to the characteristics of single molecules. Each molecule consists of a backbone and anchoring groups, therefore the dataset contains three basic data types: molecular backbone, anchoring groups, and molecules. Corresponding textual descriptions (captions) are constructed for each type of data. The caption for the molecular backbone includes the compound name and structural information; the caption for the anchoring groups includes the group name and diverse semantic context descriptions; and the caption for the molecules is a combination of information from the molecular backbone and anchoring groups, such as... Figure 2 This caption construction method better aligns with the cues provided by researchers when designing single molecules for experiments. Simultaneously, to ensure robustness and data uniformity, each molecular skeleton and molecule has three equivalent descriptions. Since the number of anchoring groups that can be used to construct single molecules is limited, each anchoring group has five equivalent captions. Ultimately, the fine-tuning dataset consists of 329 high-quality data points, divided into training / validation / test sets in an 8:1:1 ratio. The molecules and captions in the validation and test sets were not used during the training phase.

[0017] 2) Initialization of the large language model. The model is initialized using open-source MolT5 model weight checkpoints (molt5-small-caption2smiles; molt5-base-caption2smiles; molt5-large-caption2smiles) for caption-based SMILES generation. During fine-tuning, the text descriptions of molecules are first tokenized, then input into MolT5 checkpoints of different sizes for text molecule generation. After obtaining the output SMILES molecules, the cross-entropy loss is calculated between these SMILES molecules and the SMILES molecules used as labels.

[0018] 3) Model Fine-tuning Method Flow. The entire fine-tuning process can be described as using a generative model for sequence generation. First, the caption text is processed by the MolT5 model into an embedding matrix, which is then input into the model's parameter space. Next, the generated sequence content is compared with the corresponding label SMILES, and the cross-entropy loss is calculated. Then, backpropagation is performed to update the model's parameter space, ultimately ensuring that the model's output conforms to the correct SMILES sequence, such as... Figure 3 An evaluation strategy is enabled during training to evaluate the performance of the model at the checkpoints of the current round after each training round, retaining the checkpoints with excellent performance.

[0019] 4) Iterative training. Input the data into the MolT5 model from step 2), and then perform multiple rounds of joint training until the loss reaches the early stopping condition;

[0020] 5) Result Evaluation. Molecules different from those in the training data were selected as generation targets, and corresponding natural language generation descriptions were designed. The generated SMILES results were compared with the results of the real-label SMILES using quantitative metrics, including general sequence-based metrics such as Bilingual Substitution Evaluation (BLEU) and Levenshtein distance; and molecular-specific metrics such as Molecular Access System fingerprint (MACCS), RDKit fingerprint (RDK), Morgan fingerprint, and Validity. A horizontal comparative experiment was conducted with similar models on a test set to verify its effectiveness. Figure 4 .

[0021] This invention proposes a single-molecule structure design generation framework that integrates the creation of a single-molecule structured design dataset, fine-tuning of a single-molecule generated large language model, and systematic evaluation of the generation effect. Attached Figure Description

[0022] Figure 1 A schematic diagram of the single-molecule large model concept.

[0023] Figure 2 A schematic diagram of the structure of a single-molecule fine-tuning dataset.

[0024] Figure 3 Schematic diagram of the fine-tuning process.

[0025] Figure 4 A diagram showing the comparison of model metrics.

[0026] Figure 5 Model capability performance chart Detailed Implementation

[0027] The following embodiments, in conjunction with the accompanying drawings, will further illustrate the technical solution of the present invention; however, the scope of protection of the present invention is not limited to these embodiments. All changes or equivalent substitutions that do not depart from the concept of the present invention are included within the scope of protection of the present invention.

[0028] The specific implementation steps of this invention are as follows:

[0029] 1. Dataset Construction and Semantic Annotation

[0030] Data source:

[0031] 1) Molecules that have been used for single-molecule measurements in publicly available academic literature and patents;

[0032] 2) Candidate molecules matching the unimolecular configuration from the publicly available molecular database PubChem;

[0033] Data cleaning process:

[0034] 1) Remove those that cannot be synthesized yet;

[0035] 2) Unanchorable molecular structures;

[0036] 3) Standardization and deduplication are consistent with the SMILES corresponding molecules;

[0037] 4) Constrain size and configuration, eliminating overly complex molecules;

[0038] Semantic annotation process (i.e., all captions and descriptions are in English):

[0039] 1) Textual descriptions of the molecular skeleton (including benzene, biphenyl, diphenylacetylene, etc.)

[0040] 2) Construct a text description of the anchoring group (including thiol, amino, carboxyl, etc.)

[0041] 3) Construct a text description of the molecule (in the form: This molecule contains / features / has… + “molecular skeleton” core / backbone / skeleton… + “anchor groups”).

[0042] 4) Finally, the training / validation / test sets are divided into 80% / 10% / 10% sets. The numerators and captions of the validation and test sets are not used during the training phase to prevent data leakage.

[0043] 2. Base Model and Initialization

[0044] This invention uses publicly available MolT5 "caption to SMILES" checkpoints as initial weights (including small / base / large scales), and retains SMILES as the numerator representation. The text segmentation uses the Sentence Piece tokenizer compatible with MolT5. Let the input caption be... The target SMILES is The model parameters are denoted as .

[0045] 3. Training Framework and Loss Function

[0046] The model maximizes the conditional likelihood in an autoregressive manner, and uses cross-entropy loss during training (e.g., Figure 1 ):

[0047]

[0048] The optimizer uses AdamW, with weight decay set to... Initial learning rate The system uses the default linear learning rate scheduling strategy; the batch size is adjusted based on GPU memory (and can be used in conjunction with gradient accumulation). Early stopping is triggered to prevent overfitting when the loss on the validation set stops decreasing or falls below the expected threshold within a certain number of consecutive evaluation periods.

[0049] 4. Reasoning Generation

[0050] This invention employs a beam search generation strategy based on the Transformer decoder during the inference phase. Given a natural language design instruction (Caption), the goal is to solve for the conditional maximum a posteriori (MAP) sequence under a length upper limit constraint.

[0051]

[0052] in These are the model parameters obtained after fine-tuning. The upper limit for the length of the generated sequence is set (considering that the actual SMILES means the length is set to 128).

[0053] The content of each basic atom in a real molecule is not uniform, so direct enumeration is not possible. This would cause the model to tend to output the mode element, making it infeasible. This solution uses a beam search approximation with a beam width of B=5. At time step t, the B highest-scoring prefix hypotheses are retained. And it expands in an autoregressive manner:

[0054]

[0055] Meanwhile, to avoid over-preference for short sequences, a length penalty is introduced. Standard form (default) ):

[0056]

[0057] This implementation sets the default length penalty to 1.0 (i.e., no additional length amplification / suppression) and enables early stop: when any beam is generated to the termination mark. <eos>Furthermore, when its score dominates within the current bundle, it immediately stops expanding to obtain a stable and controlled output length. To ensure determinism and reproducibility, the decoding process does not introduce a random sampling mechanism, i.e., it does not set temperature T, top-k, or top-p parameters. Under the same conditions... Given the same input x and the same word segmenter, the output sequence is unique. Formally, the decoding operator can be written as:

[0058]

[0059] 5. Calculation of evaluation indicators

[0060] This invention separates generation and evaluation: the generation stage outputs only a single SMILES; the evaluation stage calculates sequence class and molecular specificity indices in batches within an independent scoring program. This decoupling ensures the determinism of the decoding process, while the evaluation module can freely expand the indices without affecting the generation results, as shown in the figure.

[0061] 1) BLEU (Sequence Precision) measures the matching precision between the generated sequence and the reference sequence at the n-gram level, taking into account both local and global consistency:

[0062]

[0063] in, To correct the precision of n-grams, As weight, For brevity, in this scenario, n-grams are defined using the SMILES notation sequence.

[0064] 2) Levenshtein (edit distance) similarity measures the difference between two sequences using the minimum number of edit steps and normalizes it to a similarity score:

[0065]

[0066] in Minimum number of insertions / deletions / replacements, and These are for generating and referencing SMILES, respectively.

[0067] 3) MACCS Fingerprint Similarity (Fragment-Level Consistency): MACCS is a fixed-length (binary) substructure presence fingerprint. A MACCS fingerprint represents whether a molecule contains a specific substructure fragment as a fixed-length (binary) vector. For this type of molecule-specific index (based on fingerprint), let... For fingerprint mapping (bit vector / set), the similarity is uniformly calculated using the Tanimoto (Jaccard) coefficient:

[0068]

[0069] Therefore, given two binary fingerprints A and B, their MACCS similarity based on the Tanimoto coefficient is:

[0070]

[0071] Where a and b are the number of 1s in A and B respectively, and c is the number of overlapping 1s in both.

[0072] 4) The RDK fingerprint is a path-based binary fingerprint of the RDKit: each bit is encoded based on the existence of a certain "atom-bond path" (within a given length) in the molecular graph (parsed by SMILES), measuring the structural characterization of the topological path. Based on the Tanimoto coefficient, its RDK similarity is:

[0073]

[0074] in, and These represent the path-based fingerprint vectors of RDKit calculated from the generated SMILES and the reference SMILES, respectively. Both belong to... (The vector has a length of d). Each component of the vector indicates whether a certain hashed "atomic-bond" path exists.

[0075] 5) Morgan fingerprints (also known as ECFP) iteratively encode a ring-shaped local environment centered on atoms with a radius not exceeding a given threshold into a fixed-length bit vector (Extended-connectivity fingerprints. J.Chem. Inf. Model. 50(5), 742–754). This vector is used to measure the degree to which the generated SMILES preserve the atom-centered local environment of the reference molecule, complementing path-type fingerprints. Based on the Tanimoto coefficient, its RDK similarity is:

[0076]

[0077] in, and These represent the Morgan fingerprint vectors extracted from the generated SMILES and the label SMILES, respectively. Both belong to [the category of fingerprint vectors]. (The vector length is d).

[0078] 6) Validity measures whether the generated SMILES correspond to chemically valid molecules, i.e., whether they can be successfully parsed into molecular diagrams by cheminformatics tools and whether there are obvious valence state errors.

[0079]

[0080] Samples that fail to be resolved or exhibit significant chemical inconsistencies are considered invalid. This metric ensures the basic validity of the assessment from the perspectives of resolvability and chemical plausibility.< / eos>

Claims

1. A method for designing and generating single-molecule structures based on a large language model. Its design process is characterized by taking the natural language description of the experimental single molecule as input and outputting the corresponding SMILES molecular representation. The technical solution includes the following steps: Step S1, Design and Construction of Single-Molecular Fine-Tuning Dataset: Designing a corpus consisting of three types of data—molecular backbone, anchoring groups, and molecules—for single-molecule electrical measurement scenarios; Text descriptions (captions) were written for the three types of data. The caption for the molecular skeleton included the compound name and structural information, the caption for the anchoring group included the group name and a description of diverse semantic contexts, and the caption for the molecule was composed of a combination of information from the molecular skeleton and the anchoring group. Data cleaning and partitioning were completed to form a dataset for fine-tuning and testing. Step S2, Initialization of the large language model: Initialize using the open-source MolT5 "caption to SMILES" series of checkpoints, and use SMILES as the molecular representation; Step S3, Model Fine-tuning Method: Input the caption into the model after word segmentation to generate the SMILES sequence, calculate the cross-entropy loss with the label SMILES, and use backpropagation to update the parameters to obtain the mapping from natural language to SMILES; Step S4, Iterative Training: Perform multiple rounds of training under the monitoring of the validation set, and end the training when the early stopping condition is met, while retaining the checkpoints with better performance; Step S5, Result Evaluation: For data that did not participate in training, generate SMILES using deterministic decoding, and calculate evaluation metrics including BLEU, Levenshtein, MACCS fingerprint similarity, RDK fingerprint similarity, Morgan fingerprint similarity, and effectiveness to verify the generation effect.

2. The method according to claim 1, characterized in that, In step S1, the dataset includes three types of data: "molecular backbone + anchoring group" structural abstract organization; three equivalent captions are set for each molecular backbone and molecule, and five equivalent captions are set for anchoring groups; the data comes from reported literature in the field of molecular electronics, and is divided into training / verification / test in an 8:1:1 ratio, and the molecules and captions for verification and testing do not appear in the training set.

3. The method according to claim 1, characterized in that, In step S2, the MolT5 checkpoints are at least one of molt5-small-caption2smiles, molt5-base-caption2smiles, or molt5-large-caption2smiles, and the text end uses a word segmenter compatible with MolT5 to perform sub-word level encoding on the caption.

4. The method according to claim 1, characterized in that, In step S3, the loss function for fine-tuning training is: conditional autoregressive cross-entropy loss based on effective positions, with the position of the filler not included in the loss, and an early stopping strategy is adopted to suppress model overfitting.

5. The method according to claim 1, characterized in that, In step S5, the evaluation metrics for the single-molecule generation effect include: BLEU and Levenshtein based on the SMILES molecular text sequence; and Tanimoto similarity and efficiency based on MACCS, RDK and Morgan fingerprints, wherein the efficiency is the proportion of molecules that can be successfully resolved by cheminformatics tools and have reasonable structures.

Images