A method for generating cross-modal three-dimensional molecular geometry based on chemical grammar

Abstract

The application provides a chemical grammar-based cross-modal three-dimensional molecular geometry generation method. The method comprises: obtaining a chemical grammar representation of a target molecule; inputting the chemical grammar representation into a frozen-parameter one-dimensional molecular base model to obtain a one-dimensional semantic hidden state; using a learnable query vector and a trainable projector to extract key chemical knowledge from the hidden state and map it to a three-dimensional geometry space to obtain three-dimensional conditional features; and finally inputting the three-dimensional conditional features into a three-dimensional diffusion model to generate three-dimensional geometry coordinates of the target molecule. The application bridges one-dimensional chemical grammar and three-dimensional geometry structure through a unified framework, significantly improves the geometry fidelity, chemical effectiveness and structure stability of the generated molecules by using cross-modal knowledge transfer without retraining the one-dimensional molecular model, and solves the problem that the prior art is difficult to balance syntax rules and geometry structure.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence technology and materials science computing technology, and in particular, to a method and apparatus for generating cross-modal three-dimensional molecular geometry based on chemical syntax. Background Technology

[0002] Against the backdrop of the booming development of drug discovery and materials science, how to generate three-dimensional (3D) molecular geometries with high fidelity and chemical validity has become a core problem that urgently needs to be solved. Among related technologies, existing 3D molecular generation methods often face the contradiction between the validity of chemical syntax (usually represented by one-dimensional strings, such as SELFIES) and the fidelity of the geometric structure.

[0003] Specifically, existing generative methods often struggle to effectively transfer the rich semantic knowledge inherent in one-dimensional (1D) chemical language models to three-dimensional geometry generation tasks. While some techniques attempt to combine different modalities, they lack effective cross-modal alignment mechanisms, resulting in geometrically unstable or chemically inconsistent molecules. Furthermore, existing techniques often suffer from low computational efficiency and weak property control when performing de novo generation or conditional generation. Therefore, it is necessary to propose a more efficient method that can better integrate cross-modal knowledge to address the problem of three-dimensional molecular geometry generation. Summary of the Invention

[0004] This invention provides a cross-modal 3D molecular geometry generation method based on chemical grammar, which can achieve high-fidelity 3D molecular generation by utilizing the knowledge of a pre-trained one-dimensional molecular basic model through cross-modal knowledge transfer.

[0005] To achieve the above objectives, a first aspect of the present invention proposes a cross-modal three-dimensional molecular geometry generation method based on chemical syntax, the method comprising: 1. Obtaining a chemical syntax representation of a target molecule, wherein the chemical syntax representation includes a one-dimensional molecular sequence; 2. Input the chemical syntax representation into the one-dimensional molecular basis model (MoLLaMA) with frozen parameters to obtain the one-dimensional semantic hidden state; 3. Based on learnable query vectors and a trainable projector, key chemical knowledge is extracted from the one-dimensional semantic hidden state and mapped to a three-dimensional geometric space to obtain three-dimensional conditional features; 4. Input the three-dimensional conditional features into the three-dimensional diffusion model, and generate the three-dimensional geometric coordinates of the target molecule through a denoising process; 5. Output the three-dimensional geometric structure of the target molecule.

[0006] In some embodiments, the step of inputting the chemical syntax representation into a one-dimensional molecular basis model of frozen parameters to obtain a one-dimensional semantic hidden state includes: A MoLLaMA model trained on SELFIES strings is used as the basic model for a one-dimensional molecule. The model has learned the syntactic and semantic properties of the molecule. Keep the parameters of the one-dimensional molecular model frozen and do not update; The one-dimensional molecular sequence is input into the model to extract hidden layer feature representations containing rich chemical semantics.

[0007] In some embodiments, the step of extracting key chemical knowledge from the one-dimensional semantic hidden state and mapping it to a three-dimensional geometric space based on a learnable query vector and a trainable projector includes: Construct a set of dynamic, learnable query vectors to focus on and extract key features related to geometry generation from the one-dimensional semantic hidden state; The learnable query vector is interacted with the one-dimensional semantic hidden state using a cross-attention mechanism; By using a trainable projector network, the interactive features are projected and aligned from the one-dimensional semantic space to the three-dimensional geometric adjustment space, ensuring the consistency between syntactic information and geometric structure.

[0008] In some embodiments, inputting the three-dimensional conditional features into a three-dimensional diffusion model and generating the three-dimensional geometric coordinates of the target molecule through a denoising process includes: Construct a three-dimensional diffusion model based on the Transformer architecture; The three-dimensional conditional features are used as conditioning signals to guide the reverse denoising process of the diffusion model. Starting with Gaussian noise distribution, atomic coordinates and chemical bond features are predicted and updated step by step, ultimately generating a three-dimensional molecular structure that conforms to the chemical syntax representation.

[0009] A second aspect of the present invention provides a cross-modal three-dimensional molecular geometry generation device based on chemical syntax, comprising: The acquisition module is used to acquire the chemical syntax representation of the target molecule, such as the SELFIES string; The feature extraction module is used to extract semantic features using a frozen one-dimensional molecular basis model; The cross-modal transfer module is used to map one-dimensional features to three-dimensional conditional features using learnable query vectors and projectors. The generation module is used to generate three-dimensional molecular geometric coordinates based on the three-dimensional condition features and using a three-dimensional diffusion model.

[0010] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the cross-modal three-dimensional molecular geometry generation method based on chemical syntax described in the first aspect.

[0011] This invention provides a cross-modal 3D molecular geometry generation method (MolSculpt) based on chemical grammar. First, it obtains the chemical grammar representation of the molecule. Next, it extracts semantic knowledge using a frozen 1D molecular foundation model. Then, by introducing learnable query vectors and projectors, it achieves effective knowledge transfer from the 1D syntactic space to the 3D geometric space. Finally, it generates high-fidelity molecular structures using a 3D diffusion model. Compared with existing technologies, this invention bridges 1D chemical grammar and 3D geometric structures through a unified framework, eliminating the need to train a language model from scratch. It effectively solves the problem of balancing syntactic effectiveness and geometric fidelity, achieving significant improvements in molecular stability, effectiveness, and accuracy of property condition generation. This provides a new solution for drug discovery, materials science, and molecular design. Attached Figure Description

[0012] Figure 1 This is a flowchart of a cross-modal 3D molecular geometry generation method based on chemical syntax provided in an embodiment of the present invention, which shows the step logic from inputting chemical syntax (SELFIES) to finally outputting a 3D geometric structure; Figure 2 This is the overall architecture diagram of the MolSculpt model provided in the embodiments of the present invention, including the connection relationships of the MoLLaMA part, the learnable query and projector part, and the three-dimensional diffusion model part; Figure 3 This is a comparison table of generation effects provided by the embodiments of the present invention. This figure should show the comparison of the geometric rationality of the molecules generated by the method of the present invention and the baseline method. Figure 4 This is a schematic diagram of the functional modules of the cross-modal three-dimensional molecular geometry generation device based on chemical syntax provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0014] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0016] This invention presents a generative framework (named MolSculpt) for three-dimensional molecular generation based on cross-modal knowledge transfer. This method integrates a frozen one-dimensional molecular foundation model (MoLLaMA) with a three-dimensional diffusion model, utilizing learnable queries and projector mechanisms to effectively transform the semantic knowledge of one-dimensional chemical syntax into guiding signals for three-dimensional geometric generation, thereby improving the structural stability and property accuracy of the generated molecules.

[0017] In the field of drug discovery, existing 3D molecular generation methods face the challenge of balancing syntactic effectiveness and geometric fidelity. Related techniques are generally divided into two categories: one focuses on 1D sequence generation but lacks spatial information; the other focuses on 3D coordinate generation but often ignores complex chemical syntax rules. The following is an overview of these techniques: Existing 3D generation methods: While existing diffusion model-based methods (such as Diffusion-Mol, MDM) can generate 3D coordinates, they often struggle to guarantee the chemical validity and stability of complex molecules. Furthermore, these methods typically cannot directly utilize the knowledge from existing massive amounts of one-dimensional chemical text data.

[0018] This invention proposes a cross-modal 3D molecular geometry generation method based on chemical grammar, aiming to provide a solution for enhancing 3D generation by utilizing knowledge from pre-trained language models.

[0019] like Figure 1 As shown, the method flow in this embodiment begins with obtaining the chemical syntax string, followed by feature extraction and cross-modal mapping, ultimately outputting the three-dimensional structure from the diffusion model. The specific steps are as follows: S100, Obtain the target sample of the molecular chemical syntax representation and its corresponding learnable query vector.

[0020] In this embodiment, SELFIES (Self-Referencing Embedded Strings) is used as the chemical syntax representation of molecules. Compared to SMILES, SELFIES is 100% robust and can avoid generating invalid molecular strings.

[0021] S200. Input the target sample into a one-dimensional molecular basic model and extract the corresponding feature information.

[0022] In this step, keep the parameters of MoLLaMA unchanged. The hidden states are frozen and used only as a feature extractor. After the input sequence is encoded, the hidden states of the last one or more layers are extracted. ,in For sequence length, The dimension of the hidden layer (e.g., 1024).

[0023] S300: The feature information extracted from the target sample is transferred across modal knowledge through a learnable mapper and a feedforward neural network, and the cross-modal information is passed into the three-dimensional diffusion model as conditional information.

[0024] To address the mismatch between one-dimensional features and the dimensionality of the three-dimensional generated space, this invention designs a cross-modal bridging module. Specifically, a set of learnable query vectors is defined. Where N is the number of query tokens (e.g., 32 or 64). Cross-Attention mechanism is used to... Extract key information:

[0025] in, and Let C be the projection matrix. Then, through a cross-modal projector composed of a multilayer perceptron (MLP), the aggregated features C are mapped to the conditional space of the three-dimensional diffusion model.

[0026] S400: Based on the two-dimensional molecular features and cross-modal condition information of the target sample, input the three-dimensional molecular diffusion model to obtain the three-dimensional molecular output.

[0027] Molecular coordinates are generated using a 3D diffusion model. This model is based on the DiffusionTransformer (DiT) architecture. The diffusion process involves adding noise forward and denoising backward. At each step t of the backward generation, the model predicts the coordinates after noise or denoising. The model's prediction formula can be expressed as:

[0028] in, Let be the noisy molecular coordinates at the current time step, t be the time step embedding, and C be the conditional feature obtained from S300. In this way, one-dimensional chemical grammar knowledge directly guides the construction of three-dimensional geometry.

[0029] like Figure 3 As shown in the figure, the experimental results show that, compared with baseline models such as NeXt-Mol, the method of the present invention (MolSculpt) has significantly improved the FCD (fidelity), efficiency and stability of molecular generation, and the generated molecular conformation is closer to the real distribution.

[0030] Furthermore, this invention also supports conditional generation tasks, namely, generating molecules with specific properties (such as molecular weight, dipole moment, etc.). By incorporating property control signals at the input stage, this invention can control the physicochemical properties of the generated molecules more precisely than existing technologies.

[0031] In summary, this invention effectively solves the problem of the separation between one-dimensional syntax and three-dimensional geometry in the prior art by constructing a unified cross-modal framework. It can generate high-quality three-dimensional molecular structures without the need for expensive additional data collection, and has important application value.

[0032] like Figure 4 As shown, Figure 4 This is a functional module diagram of the cross-modal three-dimensional molecular geometry generation device based on chemical syntax provided in an embodiment of the present invention. The device includes: Module 1001 is used to acquire target samples of molecular chemical syntax representations and their corresponding learnable query vectors. Feature extraction module 1002 is used to extract corresponding feature information from the target sample; The cross-modal mapping module 1003 is used to transfer cross-modal knowledge from the feature information extracted from the target sample through a learnable mapper and a feedforward neural network, and to input the cross-modal information as conditional information into the three-dimensional diffusion model. Output module 1004 is used to obtain the target three-dimensional molecule output based on the two-dimensional molecular features and cross-modal condition information of the target sample.

[0033] The specific implementation of this device is basically the same as the specific embodiment of the cross-modal three-dimensional molecular geometry generation method based on chemical syntax described above, and will not be repeated here.

[0034] like Figure 5 As shown, Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device includes: The memory 1101 can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1101 can store the operating system and other application programs. When the technical solutions provided in the embodiments of the present invention are implemented through software or firmware, the relevant program code is stored in the memory 1101 and is called by the processor 1102 to execute the training method of the model in the embodiments of the present invention. The processor 1102 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. Input / output interface 1103 is used to implement information input and output; The communication interface 1104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1105 transmits information between various components of the device (e.g., processor 1101, memory 1102, input / output interface 1103, and communication interface 1104); The memory 1101, processor 1102, input / output interface 1103 and communication interface 1104 are connected to each other within the device via bus 1105.

[0035] This invention also provides a computer-readable storage medium storing one or more computer programs that can be executed by one or more processors to implement the above-described cross-modal three-dimensional molecular geometry generation method based on chemical syntax.

[0036] Finally, it should be noted that the above embodiments are only used to more clearly illustrate the technical solutions of the present invention and do not constitute a limitation on the technical solutions provided by the present invention. Those skilled in the art should understand that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the present invention are also applicable to similar technical problems. Furthermore, modifications to the technical solutions described in the above embodiments, or equivalent substitutions of some technical features, do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-modal three-dimensional molecular geometry generation method based on chemical grammar, characterized in that, The method includes: Obtain the chemical syntax representation of the target molecule, wherein the chemical syntax representation includes a one-dimensional molecular sequence; The chemical syntax representation is input into a one-dimensional molecular basis model with frozen parameters to obtain a one-dimensional semantic hidden state. Based on learnable query vectors and trainable projectors, key chemical knowledge is extracted from the one-dimensional semantic hidden state and mapped to a three-dimensional geometric space to obtain three-dimensional conditional features. The three-dimensional conditional features are input into the three-dimensional diffusion model, and the three-dimensional geometric coordinates of the target molecule are generated through a denoising process; Output the three-dimensional geometry of the target molecule.

2. The method according to claim 1, characterized in that, The step of inputting the chemical syntax representation into a one-dimensional molecular basis model with frozen parameters to obtain a one-dimensional semantic hidden state includes: Obtain a one-dimensional molecular basic model pre-trained based on the SELFIES string, which is used to learn the syntactic and semantic properties of the molecule; The parameters of the one-dimensional molecular model are not updated. The one-dimensional molecular sequence is input into the one-dimensional molecular basic model, and the hidden layer feature representation containing chemical semantics is extracted as the one-dimensional semantic hidden state.

3. The method according to claim 1, characterized in that, The process of extracting key chemical knowledge from the one-dimensional semantic hidden state and mapping it to a three-dimensional geometric space based on learnable query vectors and trainable projectors includes: Construct a set of dynamic, learnable query vectors; By utilizing a cross-attention mechanism, the learnable query vector interacts with the one-dimensional semantic hidden state to extract features related to geometry generation. By using a trainable projector network, the interactive features are projected and aligned from the one-dimensional semantic space to the three-dimensional geometric adjustment space to obtain the three-dimensional conditional features.

4. The method according to claim 1, characterized in that, The step of inputting the three-dimensional conditional features into a three-dimensional diffusion model and generating the three-dimensional geometric coordinates of the target molecule through a denoising process includes: Construct a three-dimensional diffusion model based on the Diffusion Transformer architecture; The three-dimensional conditional features are used as adjustment signals to guide the reverse denoising process of the three-dimensional diffusion model; Starting with a Gaussian noise distribution, atomic coordinates and chemical bond features are predicted and updated step by step until a three-dimensional molecular structure conforming to the chemical syntax is generated.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: While acquiring the chemical syntax representation of the target molecule, the property control signals of the target molecule are also acquired; The attribute control signal and the three-dimensional condition features are input into the three-dimensional diffusion model to control the physicochemical properties of the generated molecules.

6. A cross-modal three-dimensional molecular geometry generation device based on chemical grammar, characterized in that, The device includes: The acquisition module is used to acquire the chemical syntax representation of the target molecule; The feature extraction module is used to input the chemical syntax representation into the one-dimensional molecular basis model of the frozen parameters to obtain the one-dimensional semantic hidden state; The cross-modal transfer module is used to extract key chemical knowledge from the one-dimensional semantic hidden state and map it to a three-dimensional geometric space based on a learnable query vector and a trainable projector to obtain three-dimensional conditional features. The generation module is used to input the three-dimensional conditional features into the three-dimensional diffusion model and generate the three-dimensional geometric coordinates of the target molecule through a denoising process.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the cross-modal three-dimensional molecular geometry generation method based on chemical syntax as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cross-modal three-dimensional molecular geometry generation method based on chemical syntax as described in any one of claims 1 to 5.

Images