Molecular inverse synthesis method based on information enhanced multi-mode large model

Abstract

The invention relates to a molecular inverse synthesis method based on an information enhanced multi-modal large model, and belongs to the technical field of multi-modal artificial intelligence, and the method comprises the following steps: collecting inverse synthesis data, generating dialogue data for multi-modal large model training, and carrying out the cleaning and preprocessing of the data; a multi-modal information processing model based on a multi-stage representation enhancement and feedforward fusion architecture is constructed, and the model comprises a molecular specificity vocabulary system optimization module used for enhancing semantic precision of molecular linear representation; the graph multi-scale feature integration module provides comprehensive understanding of a complex molecular structure; the enhanced feedforward pre-fusion unit promotes cross-modal information flow and interaction and ensures seamless integration of graphic and text information; the large language model trunk module is used for receiving the input information subjected to preprocessing and feature fusion and carrying out multi-modal reasoning and task execution; optimizing model parameters by adopting a pre-training weight in combination with a LoRA fine tuning technology; and evaluating the performance of the model through post-processing.

Description

Technical Field

[0001] This invention belongs to the field of multimodal artificial intelligence technology and relates to a molecular retrosynthesis method based on an information-enhanced multimodal large model. Background Technology

[0002] With the rapid development of science and technology, cutting-edge fields such as materials science and drug discovery are placing higher demands on multimodal learning techniques. In these fields, effectively processing and integrating complex structured and unstructured data has become a core challenge for improving research efficiency and accuracy. In particular, the processing of molecular graph information, as a fundamental problem in chemical and biological research, urgently requires the establishment of effective methods to achieve deep integration with other modal information such as text, in order to significantly improve the effectiveness of molecular analysis and drug design. Especially in retrosynthetic tasks, the efficient integration of structured data such as molecular graphs with unstructured data such as text has become crucial for improving the efficiency of molecular analysis and drug design.

[0003] Molecular diagrams precisely depict molecular structures using nodes (atoms) and edges (chemical bonds), containing rich information such as atom types, chemical bonds, spatial relationships, and specific chemical environments. These structural features are crucial for understanding molecular properties, predicting reaction pathways, and analyzing intermolecular interaction mechanisms. However, the highly structured nature of molecular diagrams presents significant challenges to their deep integration with modalities such as text, especially in retrosynthetic scenarios driven by large language models. Efficiently processing and utilizing molecular diagram information has become a critical technical hurdle that needs to be overcome.

[0004] Existing research indicates that while large language models have made significant strides in text processing, they generally suffer from information dilution when dealing with multimodal inputs, especially highly structured non-textual data such as molecular graphs. Specifically, as the number of model layers increases, fine-grained structural information in molecular graphs is gradually lost or overgeneralized, leading to a significant decrease in the model's accuracy in understanding and utilizing this crucial information. This problem prevents existing large multimodal models from fully extracting the inherent key structural features of molecular graphs, severely limiting their performance on related tasks.

[0005] To address the aforementioned technical challenges, academia and industry have proposed various multimodal learning solutions, including representative methods such as InstructMol and MolTC. However, these methods still have significant limitations in practical applications, mainly in the insufficient capture of fine-grained features and the lack of effective interaction mechanisms during intermodal information transfer. Inverse synthesis tasks not only require models to accurately resolve complex molecular structures but also need to integrate molecular representations such as SELFIES and textual information such as user instructions to predict reaction pathways. Therefore, how to achieve efficient fusion of molecular diagrams and textual information, overcoming information dilution and interaction bottlenecks, is a core technical challenge for improving the performance of inverse synthesis tasks. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a molecular retrosynthesis method based on an information-enhanced multimodal large model.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A molecular retrosynthesis method based on an information-enhanced multimodal large model includes the following steps:

[0009] S1: Collect inverse synthetic data and generate dialogue data for training multimodal large models, and clean and preprocess the data;

[0010] S2: Construct a multimodal information processing model based on a multi-stage representation enhancement and feedforward fusion architecture, the model including:

[0011] A molecular-specific vocabulary system optimization module is used to enhance the semantic accuracy of linear molecular representations;

[0012] The multi-scale feature integration module provides a comprehensive understanding of complex molecular structures;

[0013] Enhanced feedforward pre-fusion unit promotes cross-modal information flow and interaction, ensuring seamless integration of text and image information;

[0014] The main module of the large language model receives input information after preprocessing and feature fusion, and performs multimodal reasoning and task execution;

[0015] S3: Pre-trained weights are combined with LoRA fine-tuning technology to optimize model parameters, and the cross-entropy loss function is used to maximize the matching degree between the prediction results and the true labels.

[0016] S4: Input the molecular diagram to be tested and the corresponding text information, output the prediction results of the retrosynthetic reaction, and evaluate the model performance through post-processing.

[0017] Furthermore, in step S1, the acquisition of retrosynthetic data and generation of dialogue data for training a multimodal large model includes: collecting chemical reaction data from a database, extracting information on target molecules and their reactants, generating diverse instructions through human-AI collaboration, and converting structured reaction data into a text instruction format containing inputs and outputs, wherein the input is a target molecule SELFIES representation, and the output is reactant SELFIES or a path description.

[0018] Furthermore, the data cleaning and preprocessing includes: using SELFIES as molecular descriptors to filter out noise and redundancy, and formatting the data into an instruction form containing task instructions, target molecules, and reactant SELFIES; then, the data is segmented into training, validation, and test sets.

[0019] Furthermore, the molecular-specific vocabulary system optimization module enhances chemical semantic understanding by expanding the LMM segmenter, introducing chemical concept vocabulary units, and fine-tuning the embedding and lm_head layers.

[0020] Furthermore, the graph multi-scale feature integration module extracts node features from different levels of the GNN and constructs multi-scale pyramid features using lateral connections, top-down paths, and attention mechanisms.

[0021] Furthermore, the representation of the stitched graph of the enhanced feedforward pre-fusion unit and the text representation are then fused early through a multi-layer Transformer decoder to generate a more aligned multimodal representation.

[0022] Furthermore, in step S3, a two-stage strategy is adopted for model training. First, the multi-scale feature integration module and projection layer of the pre-trained graph are trained, and then the LoRA fine-tuning technique is used to perform end-to-end joint fine-tuning, decomposing the original weight matrix into the product of two low-rank matrices.

[0023] Furthermore, during model testing, molecular diagrams and text information are input, and the model generates reactants in SELFIES format via autoregression. The generation process is controlled by the hyperparameters temperature, top_p, and num_beams.

[0024] Furthermore, the post-processing involves converting the SELFIES strings generated by the model prediction into SMILES strings, followed by normalization processing for use in standardized chemical structure evaluation.

[0025] Furthermore, the model loss calculation adopts the cross-entropy loss of causal language modeling, and the evaluation indicators include the exact match rate EXACT, the similarity between the generated sequence and the reference sequence BLEU, the minimum number of single-character editing operations required to convert one string to another LEVENSHTEIN, the topological fingerprint structural similarity between the molecule and the target molecule RDK FTS, the presence of chemical structural features corresponding to the MACCS fingerprint MACCS FTS, and the chemical validity of the molecular representation generated by the model VALIDITY.

[0026] The beneficial effects of this invention are as follows:

[0027] Compared with traditional methods, the method of this invention has significant advantages in efficiency, accuracy, and automation. Through a novel model architecture and processing flow, this method can overcome the problems of long processing time, high dependence on manual labor, and insufficient result stability in traditional retrosynthetic analysis, bringing technological innovation to chemical research and application.

[0028] The method of this invention improves the model's ability to capture molecular structural details and the depth of its understanding of chemical semantics from the input level by introducing a molecular-specific vocabulary system optimization module.

[0029] The method of this invention, through its unique multi-stage representation enhancement and feedforward fusion architecture, can effectively integrate the multi-scale structural information of molecular graphs with textual semantics, and significantly alleviate the information dilution effect of large language models when processing such multimodal data.

[0030] The method of this invention significantly improves the accuracy and reliability of retrosynthetic reaction pathway prediction, enhances the ability to comprehensively understand complex molecular structures and related textual information, provides strong technical support for key scientific fields such as drug discovery, helps accelerate the new drug development process, reduce R&D costs, and promotes continuous progress in fields such as chemistry.

[0031] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0033] Figure 1 The flowchart shows a molecular retrosynthesis method based on an information-enhanced multimodal large model.

[0034] Figure 2 This is a general model structure diagram of a molecular retrosynthesis method based on an information-enhanced multimodal large model;

[0035] Figure 3 Optimization module structure diagram for a molecular-specific vocabulary system;

[0036] Figure 4 This is a structural diagram of the molecular graph multi-scale feature integration module;

[0037] Figure 5 This is a structural diagram of the enhanced feedforward pre-fusion module;

[0038] Figure 6 The results are from the retrosynthetic prediction experiment. Detailed Implementation

[0039] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0040] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0041] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0042] Example 1:

[0043] like Figure 1 As shown, this invention provides a molecular retrosynthesis method based on an information-enhanced multimodal large model, which is implemented in the following steps:

[0044] Step 1: Prepare the dataset.

[0045] (1) Prepare inverse synthesis data.

[0046] Chemical reaction data was collected from databases such as PubChem, UniProtKB, BC4CHEMD, ChemProt, BC5CDR, PubMed, MedMCQA, and MMLU to extract information on target molecules and their reactants. The data was used to generate diverse instructions through human-AI collaboration. These instructions were generated using gpt-3.5-turbo based on clear task descriptions and were manually reviewed to ensure quality. Structured reaction data was converted into a text instruction format containing inputs (target molecule self-leanings) and outputs (reactant self-leanings or path descriptions) using templates.

[0047] (2) Data cleaning and preprocessing.

[0048] The cleaning process removes chemically invalid SMILES strings, using SELFIES as molecular descriptors and leveraging their strict rules to ensure the validity of molecular structures, avoiding syntax errors or chemical inconsistencies in the SMILES format. Data mining removes noise and redundancy through screening, retaining high-quality retrosynthetic reaction pairs, and ensures data integrity through manual review and data mining techniques. Preprocessing includes formatting the data into instruction form (task instructions, target molecule SELFIES input, reactant SELFIES), uniformly converting it to SELFIES format for LMM processing. The BPE algorithm based on the LLaMA model is used for word segmentation, and the longest padding strategy is applied to the sequences. The data is then divided into training, validation, and test sets, with the training set containing 124,000 data points, and the test and validation sets each containing 1,000 data points.

[0049] Step 2: Design a model for molecular retrosynthesis based on an information-enhanced multimodal large model.

[0050] This invention designs a molecular retrosynthesis method based on an information-enhanced multimodal large model, specifically optimized for use in molecular graph and textual multimodal large models in the field of drug discovery. The model structure is shown in the appendix. Figure 2 This network, through a multi-stage representation enhancement and feedforward fusion strategy, effectively integrates multi-scale structural information of molecular graphs with textual semantics, thereby significantly mitigating the information dilution effect in LMMs (Large Language Models). The core innovation of this architecture mainly consists of three parts: a molecular-specific vocabulary system optimization module, a molecular graph multi-scale feature integration module, and an enhanced feedforward pre-fusion unit. The structure diagrams of each module correspond to... Figure 3 , Figure 4 and Figure 5 Next, we will introduce the design and function of each part separately.

[0051] (1) The structure of the molecular-specific vocabulary system optimization module is as follows: Figure 3The core idea of ​​this module is to extend the tokenizer of the basic LMM (Low-Modal Matrix) to a specific domain, introducing a set of customized vocabulary units closely related to chemical structures and properties. This design aims to improve the model's ability to capture details of molecular structures and its depth of understanding of chemical semantics at the input level, thus laying a solid text representation foundation for subsequent cross-modal feature fusion and complex property prediction tasks. The extended molecular-specific vocabulary system mainly includes five categories of core chemical concept units. These units are carefully designed to efficiently and accurately encode molecular information at different levels of abstraction:

[0052] Atomic units represent basic chemical elements and their charge states. For example, [C] represents a neutral carbon atom, while [N+1] explicitly represents a nitrogen atom with one unit of positive charge. This design ensures the capture of elemental identity and its precise ionic state.

[0053] Bond types describe the specific way atoms are connected and are often associated with subsequent atoms. For example, [=C] might represent a double bond connected to a carbon atom, while [=O] represents a double bond connected to an oxygen atom. This helps the model directly identify the bonding mode and its local chemical environment.

[0054] Functional Groups and Hydrogenation State: These terms indicate the degree of hydrogenation of specific chemical functional groups or atoms in a molecule. For example, [OH1] represents a hydroxyl group (one oxygen atom with one hydrogen atom), and [CH3] represents a methyl group. These terms enable models to quickly identify key chemically active sites and structural units.

[0055] Topological structures describe the branching, ringing, and important stereochemical features of molecules. For example, [Branch1] and [Ring1] represent primary branching and ring structural units, respectively, while [C@@H1] encodes carbon centers with specific chirality (e.g., @@ indicates counterclockwise rotation) and the hydrogen atoms they are attached to. This allows the model to effectively capture the three-dimensional spatial configuration and complex connection patterns of molecules.

[0056] Modifiers: These are special units used to modify other tokens to enhance directional indication or provide more refined stereochemical information. For example, [\C] can represent a specific directional bond pointing to a carbon atom, and [\S@@] can be used to describe more complex information about sulfur-containing chiral centers. The introduction of these modifiers further improves the accuracy and information-expressive power of molecular representation.

[0057] Finally, these customized tokens are extended to the tokenizer of the large model, and the embedding layer and lm_head layer are fine-tuned using LoRA technology to achieve efficient adaptation to the chemical domain. LoRA updates parameters by decomposing the original full-rank matrix into the product of two low-rank matrices, thereby reducing the number of parameters and computational complexity. Simultaneously, LoRA introduces a scaling factor α, and the model uses α / r to scale the parameter update amount ΔW, adjusting the influence of low-rank matrices A and B on the original weight matrix W. The update process of W can be represented as follows:

[0058] W′=W+ΔW=W+A·B

[0059] in, ΔW represents the parameter update value of the corresponding matrix, and r is the rank of the low-rank matrix. The size of the rank will determine the performance of fine-tuning and the training time.

[0060] (2) The structure of the molecular graph multi-scale feature integration module is as follows: Figure 4 As shown. The core objective of this module is to extract semantically rich node features with multi-scale receptive fields from different levels of the GNN and effectively fuse them to generate a more comprehensive graph representation. This module is built upon a pre-trained GNN backbone network, which generates a hierarchical set of node features {C} through a multi-layer message passing mechanism. (1) C (2) ,…,C (L)},in, The node representation of the output of the l-th layer of the GNN, where N is the number of nodes (atoms) in the molecule, and d l Let L be the feature dimension of this layer, and L be the total number of layers in the GNN. This process constitutes a bottom-up feature extraction path, and then the multi-scale feature integration module further fuses these hierarchical features to achieve effective integration of multi-scale information.

[0061] Specifically, the module first bridges features from different abstraction levels in the GNN backbone network through lateral connections, and unifies their dimensions for subsequent fusion. The output feature C of each level... (l) Firstly, through an independent linear projection layer Process and transform to a unified target dimension d. fpn The lateral feature L generated by this process l It can be represented as:

[0062]

[0063] These lateral features L lThis not only fully preserves the original semantic information at each level but also lays the foundation for subsequent fusion with deeper features. The module then progressively constructs pyramid features through a top-down pathway and feature fusion mechanism. This process begins with the deepest (most semantically richest) lateral feature L in the GNN. L Starting with this, we first apply a smoothing operation and a non-linear activation function to generate a high-semantic feature representation at the top of the pyramid:

[0064] P (L) =ReLU(Smooth(L (L) ))

[0065] The Smooth operation further refines and enhances the local consistency of the fused features, making them more discriminative and more suitable for downstream tasks. This operation can be represented as:

[0066]

[0067] Where A is the adjacency matrix of the molecular graph. I is the identity matrix. yes The degree matrix, W smooth It is a learnable weight matrix, and σ is an activation function. This method can effectively capture the complex relationships between nodes and the topological information of the graph by utilizing the graph's topological structure.

[0068] For the remaining levels of the pyramid l = L-1, L-2, ..., 1, its high semantic feature P (l) By fusing the lateral features L of the corresponding level (l) and the feature P of the previous pyramid layer (l+1) To construct, use element-wise summation and fusion, this operation can be represented as:

[0069] Fraw (l) =L (l) +P (l+1)

[0070] P (l) =RELU(Smooth(Fraw) (l) ))

[0071] After constructing the features {P} of all pyramid levels (1) ,P (2) ,…,P (L) Afterwards, these node features, representing different scales and levels of abstraction, need to be aggregated into a unified, more information-rich node representation. This is for use by subsequent modules. This aggregation process employs an attention mechanism, assigning different importance weights to features at different scales. First, for each pyramid level's feature P... (l) Calculate a scalar attention score s (l) :

[0072]

[0073] in, and It is a learnable attention parameter, d attn This is the attention hidden dimension. Then, these scores are normalized along the hierarchical dimension using the Softmax function to obtain the attention weight α. (l) :

[0074]

[0075] Final aggregation node representation We obtain the following by weighted summation of all pyramid level features:

[0076]

[0077] (3) Enhanced feedforward pre-fusion module structure as follows Figure 5 As shown, to achieve deeper and more iterative interaction and information extraction between molecular graph structure representation and text semantic representation before the information is finally fed into the LMM (Large Language Model) backbone network, the system introduces an enhanced feedforward pre-fusion unit. This unit utilizes the powerful sequence processing capabilities of the decoder layer to perform early fusion and joint refinement of the concatenated multimodal feature sequences, aiming to generate representations with richer information and better intermodal alignment, thereby effectively alleviating the information dilution problem and improving the performance of downstream tasks.

[0078] Specifically, the pre-fusion unit receives two types of pre-optimized modal representations as input, namely graph modal representation H. graph and text modal representation E text H graph Aggregated node representations derived from the graph multi-scale feature integration module And through a graph modality-specific multilayer perceptron projection layer Proj graph Adjust its dimension to the same hidden space dimension d as the LMM backbone network. model E text It is a token embedding sequence obtained by passing molecular SELFIES, system prompts, and user prompts through the LMM word embedding layer after lexicalization by the molecular-specific vocabulary system optimization module.

[0079] To achieve cross-modal information flow in a specific direction, graph modal representation H graph and text modal representation E text The features are concatenated along the sequence dimension in a fixed order, with image features preceding text features. This forms a unified, mixed-modality input sequence S. concat :

[0080] S concat =Concat(H graph E text )

[0081] The pre-fusion unit consists of K pre The module consists of layers of Transformer decoders, each Transformer block having the same structure and hyperparameters as the LMM backbone and controlled by a causal mask. This module receives the concatenated multimodal sequence S. concat As its initial input (denoted as S) (0) ), after K pre After iterative processing by one Transformer decoder layer, the output is an enhanced sequence. This process can be represented as:

[0082]

[0083] Among them, TransformerDecoderStack causal (·) represents the entire decoder stack, where each layer operates independently, receiving the output of the previous layer as input. After iterative processing by the pre-fusion module, the final output sequence is obtained. It contains features resulting from deep, ordered cross-modal interactions. The sequence is then re-separated into its corresponding graph modal components H′ based on the boundaries established during the original concatenation. graph and text modal part E′ text This process can be represented as:

[0084]

[0085] At this time, H′ graph It is a graph representation refined by its own context, while E′ text This results in a context-aware text representation that deeply integrates graph structure information. Finally, these two modal representations, refined by the pre-fusion units, are provided as enhanced multimodal inputs to the LMM for subsequent understanding, reasoning, or generation. This preprocessing provides the LMM with higher-quality multimodal information, thereby improving its performance on inverse synthesis tasks.

[0086] (4) Loss Calculation: The loss of this model includes the cross-entropy loss from causal language modeling. The specific calculation formula is as follows:

[0087]

[0088] Among them, Logits LM Labels are the raw logits that LMM outputs at each position of the input sequence to predict the next word. target This refers to the true target word ID corresponding to the input sequence. The model evaluation metrics widely used in drug discovery include EXACT (exact match rate), BLEU (similarity between generated and reference sequences), LEVENSHTEIN (minimum number of single-character edits required to convert one string to another), RDKFTS (topological fingerprint similarity between the molecule and the target molecule), MACCS FTS (existence of chemical structural features corresponding to the MACCS fingerprint), and VALIDITY (chemical validity of the molecular representation generated by the model). The calculation formulas are as follows:

[0089]

[0090] Where, N exact N is the number of molecules generated that are exactly the same as the actual molecules. total It represents the total number of molecules participating in the comparison, BP is the short sentence penalty factor, and w n It is the weight of the n-gram, p n This is the corrected precision of n-grams, M is the total number of string pairs compared, Lev is the LEVENSHTEIN distance between the strings, and pred i For the predicted sequence, gt i Corresponding to the real label, M valid It is used to calculate the number of effective molecule pairs for fingerprint similarity. It is the RDKit fingerprint calculated from the i-th predicted molecule. It is the RDKit fingerprint calculated from the i-th real molecule. It is the MACCS fingerprint of the i-th predicted molecule. It is the MACCS fingerprint of the i-th real molecule, N valid N is the number of predictions that are successfully decoded and can be resolved into valid molecules by RDKit. total This represents the total number of predictions.

[0091] Step 3: Train a model for molecular retrosynthesis based on an information-enhanced multimodal large model.

[0092] The training process employs a two-stage approach. Stage 1 involves pre-training the multi-scale feature integration module and projection layer. The core objective of this stage is to train the newly added key components responsible for processing and transforming molecular graph information, enabling them to effectively extract multi-scale features from the graph structure and project them into a feature space compatible with LMM. Trainable parameters include the multi-scale feature integration module and the multimodal projection layer; frozen parameters are the LMM backbone and the graph neural network encoder. Stage 2 is end-to-end joint fine-tuning. This stage aims to jointly optimize the entire model. This includes further fine-tuning all newly added modules and using LoRA (Low-Rank Adaptation) technology to efficiently fine-tune the parameters of the LMM backbone, adapting the LMM to multimodal molecular graph data. Trainable parameters include all newly added modules and the LoRA adapter parameters of the LMM; frozen parameters are the original weights of the LMM backbone and the graph neural network encoder.

[0093] During training, four GPUs were used with a batch size of 128, ensuring each GPU received 32 data points. The AdamW optimizer was used. The initial learning rate for the first stage of training was set to 1e-3 with a weight decay of 0.05, and the initial learning rate for the second stage was set to 8e-5 with a weight decay of 0.05. Mixed precision was enabled during training, and multi-stage learning rate scheduling was employed. The maximum sequence length of the model was 2048. The maximum number of epochs in the first stage of training was 5, with validation performed every epoch and the current model saved. The maximum number of epochs in the second stage of training was 20, with validation performed every two epochs and the current model saved. In LoRA fine-tuning, the rank of the low-rank matrix was set to 64, matrix A was initialized using a random Gaussian distribution, and matrix B was initialized with a zero matrix.

[0094] Through the above two-stage training strategy, the model first efficiently learns molecular graph-specific representation extraction and alignment methods, and then, while maintaining the generality of LMM, uses LoRA technology to make the entire model adapt to multimodal molecular graph tasks at low cost, thereby improving performance while taking training efficiency into account.

[0095] Step 4: Test the predictive performance of this model.

[0096] When testing the model on the inverse synthesis task, the target product molecule to be predicted (given as the string "SELFIES") and the corresponding task instructions are first input into the trained model. Internally, the input target product "SELFIES" string is first parsed into a graph structure representation, and its initial text embedding is obtained through a molecule-specific vocabulary optimization module. Subsequently, a graph multi-scale feature integration module processes the molecular graph, extracting multi-scale graph features containing rich structural information. These graph features, along with the relevant text embeddings, are fed into an enhanced feedforward pre-fusion unit. Within this unit, deep, causally ordered cross-modal interactions and refinement are performed through a decoder stack, producing enhanced graph and text representations. Finally, these enhanced multimodal representations are combined into the final input sequence, which is then decoded by the backbone large-scale language model in an autoregressive manner to generate the predicted reactants (represented as the string "SELFIES"), thus completing the inverse synthesis prediction.

[0097] Specifically, during testing, target product molecules and corresponding retrosynthetic instructions are read in batches from the test set. Product SELFIES strings are decoded into SMILES and then converted into graph data objects (containing node features, edge features, and edge indices) for use by the model's graph encoder. Simultaneously, a complete text prompt containing graph embedding placeholders (DEFAULT_IMAGE_TOKEN) is constructed. The preprocessed graph data tensor and text prompts (after tokenization and ID conversion) are fed into the loaded model, which uses its generate method to infer and generate reactant SELFIES strings. This generation process is controlled by several hyperparameters, including temperature (temperature coefficient), top-p (core sampling threshold), and num-beams (bundle search width). For standardized chemical structure evaluation, these SELFIES strings are typically converted into SMILES strings and then normalized. During the evaluation phase, the model's output is quantitatively analyzed according to a series of standard evaluation metrics to comprehensively assess its performance on the retrosynthetic task, including EXACT and BLEU. Finally, the test results are visualized and saved. Take one example from all reactants in the test set, and visualize the result as follows: Figure 6 As shown.

[0098] On a test set containing 1000 target products, the method of this invention exhibits superior performance when the inference generation temperature is set to 0.1. Its key performance indicators are shown in Table 1.

[0099] Table 1

[0100] EXACT↑ BLEU↑ LEVENSHTEIN↓ MACCS FTS↑ MORGAN FTS↑ RDK FTS↑ InstructMol 0.407 0.941 13.967 0.852 0.714 0.753 This method 0.614 0.968 8.041 0.916 0.838 0.864

[0101] Compared to the baseline model (InstructMol) without the information enhancement strategy of this invention, our method achieves significant optimizations across all metrics: EXACT is improved by 0.207, BLEU by 0.027, LEVENSHTEIN distance is significantly reduced by 5.926, MACCS FTS is improved by 0.064, MORGAN FTS by 0.124, and RDK FTS by 0.111. These data strongly demonstrate that our method can significantly improve the accuracy, similarity, and effectiveness of retrosynthetic prediction. Compared to traditional methods, our method offers advantages such as high efficiency, accuracy, and automation. It can be adjusted according to different model parameters, significantly improving the prediction accuracy of retrosynthetic analysis and enhancing the integration of molecular maps and textual information, thereby providing strong technical support for drug discovery.

[0102] Example 2:

[0103] An electronic device, comprising a memory and a processor;

[0104] The memory is used to store computer programs;

[0105] The processor is configured to implement the method described in Embodiment 1 when executing the computer program.

[0106] Example 3:

[0107] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0108] Example 4:

[0109] A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.

[0110] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.

[0111] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0112] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0113] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0114] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0115] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0116] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0117] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A molecular retrosynthesis method based on an information-enhanced multimodal large model, characterized in that: Includes the following steps: S1: Collect inverse synthetic data and generate dialogue data for training multimodal large models, and clean and preprocess the data; S2: Construct a multimodal information processing model based on a multi-stage representation enhancement and feedforward fusion architecture, the model including: A molecular-specific vocabulary system optimization module is used to enhance the semantic accuracy of linear molecular representations; The multi-scale feature integration module provides a comprehensive understanding of complex molecular structures; Enhanced feedforward pre-fusion unit promotes cross-modal information flow and interaction, ensuring seamless integration of text and image information; The main module of the large language model receives input information after preprocessing and feature fusion, and performs multimodal reasoning and task execution; S3: Pre-trained weights are combined with LoRA fine-tuning technology to optimize model parameters, and the cross-entropy loss function is used to maximize the matching degree between the prediction results and the true labels. S4: Input the molecular diagram to be tested and the corresponding text information, output the prediction results of the retrosynthetic reaction, and evaluate the model performance through post-processing.

2. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: In step S1, the acquisition of retrosynthetic data and generation of dialogue data for training a multimodal large model includes: collecting chemical reaction data from a database, extracting information on target molecules and their reactants, generating diverse instructions through human-AI collaboration, and converting structured reaction data into a text instruction format containing inputs and outputs, wherein the input is a target molecule SELFIES representation, and the output is reactant SELFIES or a path description.

3. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The data cleaning and preprocessing process includes: using SELFIES as molecular descriptors to filter out noise and redundancy, and formatting the data into an instruction form containing task instructions, target molecules, and reactant SELFIES; then, the data is segmented into training, validation, and test sets.

4. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The molecular-specific vocabulary system optimization module enhances chemical semantic understanding by expanding the LMM segmenter, introducing chemical concept vocabulary units, and fine-tuning the embedding and lm_head layers.

5. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The graph multi-scale feature integration module extracts node features from different levels of GNN and constructs multi-scale pyramid features using lateral connections, top-down paths, and attention mechanisms.

6. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The representation of the enhanced feedforward pre-fusion unit stitched graph and the text representation are then fused early through a multi-layer Transformer decoder to generate a more aligned multimodal representation.

7. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: In step S3, a two-stage strategy is used for model training. First, the multi-scale feature integration module and projection layer are pre-trained. Then, LoRA fine-tuning technology is used for end-to-end joint fine-tuning, decomposing the original weight matrix into the product of two low-rank matrices.

8. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: During model testing, molecular diagrams and text information are input, and the model generates reactants in SELFIES format via autoregression. The generation process is controlled by the hyperparameters temperature, top_p, and num_beams.

9. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The post-processing involves: screening valid detection results, converting the SELFIES strings generated by the model prediction into SMILES strings, and then performing normalization processing for standardized chemical structure evaluation.

10. The molecular retrosynthesis method based on an information-enhanced multimodal large model according to claim 1, characterized in that: The model loss calculation uses cross-entropy loss modeled by causal language. Evaluation metrics include exact match rate (EXACT), similarity between generated and reference sequences (BLEU), minimum number of single-character edit operations required to convert one string to the other (LEVENSHTEIN), topological fingerprint structural similarity (RDK FTS) between the molecule and the target molecule, presence of chemical structural features corresponding to MACCS fingerprints (MACCS FTS), and chemical validity of the molecular representation generated by the model (VALIDITY).

Images