A multi-modal data fusion method for 3D molecule generation

Abstract

The present application relates to the technical field of data fusion, and particularly relates to a multi-modal data fusion method for 3D molecule generation, which performs cleaning, alignment and standardization on multi-modal data such as protein pockets, texts and fragments; extracts features with geometric consistency by using SE(3) and other variable map neural networks; dynamically fuses structure, text and fragment information through a hierarchical cross-attention mechanism to generate a unified conditional context vector; drives 3D molecule generation based on the vector, and combines multi-objective gradient guidance and confidence gating to improve generation quality; finally, through ablation analysis and visualization, multi-modal contribution evaluation and optimization closed loop are realized, physical equivariance and controllable generation are maintained, and thus the 3D molecule generation quality and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of data fusion technology, and in particular to a multimodal data fusion method for 3D molecular generation. Background Technology

[0002] Currently, 3D molecular generation methods typically rely on single or simple combinations of data modalities (such as SMILES and molecular graphs), which have the following limitations: isolated multimodal data preprocessing workflows lead to information loss and alignment errors; feature extraction lacks isotropic constraints on 3D spatial rotation and translation, requiring models to learn additional physical priors; cross-modal fusion mechanisms are simple and cannot dynamically adjust the weights of each modality based on the generation objective; and the generation results lack systematic attribution analysis, making it difficult to guide data and model optimization. Existing technologies have not yet formed a systematic solution that can effectively process, fuse, and analyze multimodal data while maintaining physical isotropic properties and generation controllability, thereby improving the quality and efficiency of 3D molecular generation. Summary of the Invention

[0003] The purpose of this invention is to provide a multimodal data fusion method for 3D molecular generation that can maintain physical isovariability and generation controllability, thereby improving the quality and efficiency of 3D molecular generation.

[0004] To achieve the above objectives, this invention provides a multimodal data fusion method for 3D molecule generation, comprising the following steps:

[0005] The raw multimodal data is acquired and standardized to generate a unified set of standardized tensors; wherein the raw multimodal data includes protein pocket structure data, ligand or fragment structure data, and text description data.

[0006] The standardized tensor set is processed based on the SE(3) equivariant graph neural network to extract atomic-level feature representations and global feature summaries with rotation and translation transformations;

[0007] Through a hierarchical cross-attention mechanism, the atomic-level feature representation, the global feature summary, the text embedding vector, and the fragment feature vector are dynamically fused to generate a unified conditional context vector.

[0008] Using the conditional context vector as conditional input, the 3D molecular generation model is driven to perform generation tasks, and the generation results are evaluated for multi-objective properties and analyzed for multi-modal contributions.

[0009] The standardization process for the original multimodal data includes:

[0010] The structural data is cleaned, aligned to coordinates, and cropped.

[0011] Perform terminology cleaning and encoding on text data.

[0012] This includes coordinate alignment and cropping of the structural data, including:

[0013] Establish a local coordinate system with the geometric center or weighted centroid of the protein pocket as the origin;

[0014] Translate all atomic coordinates of the protein and ligand / fragment to the local coordinate system;

[0015] With the origin as the center, a clipping radius is set, and only the atomic coordinates within the clipping radius are retained.

[0016] Among them, the normalized tensor set is processed based on the SE(3) equivariant graph neural network to extract atomic-level feature representations and global feature summaries with rotation and translation transformations, including:

[0017] The standardized tensor set is processed based on the SE(3) equivariant graph neural network;

[0018] Initialize scalar and vector features for each atom, where the vector features are the atom's three-dimensional coordinates.

[0019] The scalar features and the vector features are updated through multi-layer equivariant message passing, wherein the coordinate update amount is determined by the relative position vector between adjacent atoms and the scalar weight calculated based on the message, so as to ensure the equivariance of SE(3).

[0020] In the process of dynamically fusing the atomic-level feature representation, the global feature summary, the text embedding vector, and the fragment feature vector through a hierarchical cross-attention mechanism to generate a unified conditional context vector, the method further includes:

[0021] When calculating attention weights using the hierarchical cross-attention mechanism, semantic similarity and geometric distance are combined simultaneously.

[0022] The evaluation of the generated results includes multi-objective properties, including:

[0023] A set of constructed differentiable property predictors is used to score the properties of intermediate molecular states during the generation process;

[0024] Calculate the comprehensive guided loss based on multi-objective property scoring;

[0025] The integrated guiding loss is injected into the incremental sampling process of denoising diffusion in the form of a gradient of the current atomic coordinates. The gradient relative to the noise space or coordinate space is calculated by the chain rule to guide the generation direction.

[0026] In the gradient guidance process, a confidence gating mechanism based on uncertainty quantification is introduced, including:

[0027] For each property predictor, the Monte Carlo dropout method is used for multiple forward propagations to calculate the variance of the predicted values ​​as a measure of cognitive uncertainty.

[0028] Confidence weights are calculated based on the cognitive uncertainty measure, and these confidence weights are negatively correlated with uncertainty.

[0029] The confidence weights are used to weight the guiding gradient of the corresponding property predictor.

[0030] The multimodal contribution analysis was performed through ablation experiments, including:

[0031] Remove or disable the text modality, fragment modality, or multi-objective guidance module respectively, and perform the generation task under the same conditions;

[0032] The differences between the complete model and each ablation configuration in terms of the effectiveness of generated molecules, docking fraction, and drug-likeness indicators were compared to quantify the contribution of each modality or module.

[0033] The 3D molecule generation model is a conditional denoised diffusion probability model, and its training employs a classifier-free guidance strategy, including:

[0034] During training iterations, conditional information is replaced with null values ​​with a set probability, enabling the model to learn both conditional and unconditional generation simultaneously.

[0035] During sampling, linear extrapolation is performed on conditional noise prediction and unconditional noise prediction, and a guiding intensity coefficient is introduced to enhance the controllability of the generation process.

[0036] The training of the neural network employs a phased optimization strategy, including:

[0037] First, independently pre-trained multimodal encoders and property predictors;

[0038] Then, the pre-trained parameters are fixed, and the diffusion generation model is trained.

[0039] Finally, all model parameters are jointly fine-tuned end-to-end.

[0040] This invention provides a multimodal data fusion method for 3D molecule generation. It cleans, aligns, and standardizes multimodal data such as protein pockets, text, and fragments; extracts geometrically consistent features using a graph neural network such as SE(3); dynamically fuses structural, textual, and fragment information through a hierarchical cross-attention mechanism to generate a unified conditional context vector; drives 3D molecule generation based on this vector, and improves generation quality by combining multi-objective gradient guidance and confidence gating; finally, it achieves multimodal contribution evaluation and optimization loop through ablation analysis and visualization, maintaining physical isovariability and generation controllability, thereby improving the quality and efficiency of 3D molecule generation. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0042] Figure 1 This is a schematic diagram illustrating the steps of a multimodal data fusion method for 3D molecular generation provided by the present invention. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0044] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0045] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0046] Please see Figure 1 This invention provides a multimodal data fusion method for 3D molecule generation, comprising the following steps:

[0047] S1. Obtain the raw multimodal data, and perform standardization processing on the raw multimodal data to generate a unified set of standardized tensors; wherein, the raw multimodal data includes protein pocket structure data, ligand or fragment structure data, and text description data.

[0048] Specifically, the raw multimodal data is acquired, including protein pocket structure data, ligand or fragment structure data, and text description data. For the structure data, cleaning, coordinate alignment, and cropping are performed, specifically as follows:

[0049] For the 3D structures of protein pockets and ligands / fragments, perform rule-based and toolchain-based preprocessing:

[0050] PDB files were read using tools such as PDBFixer and Biopython to remove bulk water molecules, free ions, and obvious non-standard residues unrelated to ligand binding. Missing atoms and hydrogen atoms were completed based on crystallographic records and common valence states, and the formal valence states of metal centers and common cations / anions were standardized. For ligands or fragments, RDKit was used for structure normalization, generating one or more low-energy conformations as candidate inputs. All structures were rearranged using unique chain identifiers, residue numbers, and atom indices to ensure that subsequent graph construction and isotropic networks would not be ambiguous due to repeated or missing indices. The original structure input was denoted as P. raw ,L raw The cleaning operation can be viewed as a mapping:

[0051]

[0052] Among them, f clean It integrates a series of deterministic operations such as water / ion filtration, hydrogen replenishment, and valence state normalization. It is a structure cleaning and pretreatment function that performs a series of biochemical rule checks, including: removing solvent molecules and non-critical metal ions, replenishing missing residue side chain atoms, adding hydrogen under specific pH conditions, and correcting atomic valence states and bond orders. These are standard structure tuples after cleaning and normalization. P represents the processed protein pocket structure, and L represents the processed ligand or molecular fragment structure. Both possess complete atomic coordinates, explicit hydrogen atoms, and correct chemical valence states. P raw The original input was the three-dimensional structure data of the protein pocket. raw The original input is the three-dimensional structural data of the ligand or molecular fragment.

[0053] To eliminate the impact of global translation on subsequent learning and to limit the computational domain, this module constructs a local coordinate system based on the pocket centroid: the origin c is the geometric center or weighted centroid of the residues interacting with the ligand in the pocket. pocketPerform a translation on all atomic coordinates of the protein and ligand / fragment: x i ←x i -c pocket Set a radius r cut and the upper limit of the maximum number of atoms N max Only retain the area centered on the pocket with a radius of r cut All atoms within the range are truncated, and if the upper limit is exceeded, the coordinates are clipped by distance or B factor; the coordinates are normalized and scaled to a specified precision (e.g., Å) to ensure that the input coordinates are within a numerically stable range (e.g., [−5,5]).

[0054] For text modality standardization and encoding, this module performs terminology cleaning and embedding encoding for text modality T (including functional description, pharmacological mechanism, previous ligand features, etc.): natural language descriptions are segmented, case-normalized, and special character filtered; key terms are unified using a domain dictionary; the text is divided into token sequences {t1,...,tm} using the ChemBERTa pre-trained model; the context representation ei∈RdT of each token is calculated using a pre-trained Transformer encoder, and the global text embedding vector z is obtained using a [CLS] vector or a weighted average. T ∈RdT; For cases with multiple text segments or multiple labels, simple pooling or attention-weighted aggregation is used to form a unified text conditional representation. This process can be abstracted as mapping zT=ftext(T), and the output will be aligned with the graph representation of the structural modality in subsequent M2 / M3.

[0055] For the set of molecular fragments or skeletons S={sj} used as prior conditions, this module converts them into a unified graph representation: RDKit is used to parse each fragment, obtaining an atom list Vj, a bond list Ej, and an initial 3D conformation Xj. The atom list contains the atomic information obtained from parsing the fragment, typically corresponding to nodes in the graph structure. The bond list contains the bond information obtained from parsing the fragment, describing the topological connections between atoms, corresponding to edges in the graph structure. The initial 3D conformation refers to the spatial coordinates of atoms in the fragment, sometimes also including related geometric features. Discrete attributes such as atom type, bond type, hybridization state, and aromaticity are enumerated and encoded to form discrete feature vectors for nodes / edges. Potential attachment points are marked for each fragment, including replaceable positions on aromatic rings and positions where side chains can grow, denoted as set Aj. If necessary, the fragment conformation is minimized to ensure a reasonable geometric shape when used as prior input. The output can be denoted as a set of local graphs Gj=(Vj,Ej,Xj,Aj), providing structural templates and topological constraints for subsequent generation processes.

[0056] The atomic coordinates and discrete properties of protein pockets and ligands / fragments are organized into a coordinate matrix X∈RN×3 and a node feature matrix H∈RN×D; the fragment graph set {Gj} is packaged into a masked sparse adjacency tensor or edge list to support variable-size batch processing.

[0057] The text embedding zT is mapped to a space of the same dimension as the structural features (through a linear projection layer), which facilitates cross-modal attention fusion in subsequent modules; a unified data structure Sample is constructed, which contains standardized X, H, {Gj}, zT and necessary masks and meta-information (such as chain ID, pocket ID, etc.).

[0058] Through the above processing, the multimodal raw data is standardized into a set of inputs with a unified coordinate system, unified coding rules and unified tensor format, providing a stable and reproducible basis for subsequent SE(3) and other variable feature engineering.

[0059] S2. Based on SE(3) and other variable graph neural networks, the standardized tensor set is processed to extract atomic-level feature representations and global feature summaries with rotation and translation transformations.

[0060] Specifically, based on the normalized tensor output in step S1, a graph feature extraction network that is strictly equivariant to three-dimensional rotation and translation is constructed, so that the node-level representation can both preserve the local chemical environment and model the geometric pattern at the global scale. A message passing paradigm based on E(n) / SE(3) equivariant graph neural network is adopted. First, a combined representation consisting of scalar feature hi and vector feature xi is constructed for each atom / residue. The input is the normalized coordinate X and discrete feature H. After passing through multiple layers of SE(3) equivariant message passing, the node-level representation Z is output. V and global summary Z G Scalar characteristics Including element type, formal charge, hybrid state, aromaticity, and whether it belongs to a pocket residue / ligand atom, etc., encoded by a multilayer perceptron (MLP); vector features For the corresponding 3D coordinates, their update must satisfy the equivariance constraint. In the l-th layer equivariant message passing, the message is calculated for each edge (i,j):

[0061]

[0062] in, For edge attribute features, ϕ m Functions are built for messages, typically MLPs. No. The message vector that the layer passes from node j to node i. This is the index for the number of layers in the neural network. , For the first Scalar characteristics of layer nodes i and j. , For the first The three-dimensional coordinates of layer nodes i and j. It is the square of the Euclidean distance.

[0063] Node coordinates are updated using a linear combination of relative coordinates:

[0064]

[0065] Among them, ϕ x The coordinate update weight function (MLP) outputs scalar weights. For the first The updated coordinate vector of the layer (i.e., the first layer) +1 layer input coordinates). Given a relative position vector, the directionality (isotropy) of the update is guaranteed. Since the update depends only on the relative coordinates, this operation maintains isotropy for global translations and rotations.

[0066] Scalar feature updates are accomplished by aggregating the received messages:

[0067] Where t is an optional time step / layer index embedding, It can include global conditional summaries from text / fragments to inject cross-modal information during the feature extraction stage. By stacking several layers (L≈4–8), the network is able to capture long-range interactions and complex geometric patterns.

[0068] During the read-out phase, the final layer node representation is... With coordinates Converging into:

[0069] Node-level equivariant representation This serves as the query for subsequent cross-modal attention.

[0070] Global Invariant Summary Common implementations include weighted summation, attention pooling, or distance-based local pooling.

[0071] S3. Through a hierarchical cross-attention mechanism, the atomic-level feature representation, the global feature summary, the text embedding vector, and the fragment feature vector are dynamically fused to generate a unified conditional context vector.

[0072] Specifically, the structural modal node representation Z output by S2 is... V scalar part Through the learnable matrix W Q Project the query vector Q. Embed the text z.T and fragment graph encoding vector respectively through matrix , and , Projected as key vector K T K S Sum vector V T V S .

[0073] Where W· is the learnable projection matrix. After concatenating the key-value pairs of the text and the fragment, the output of the cross-attention is:

[0074] in, Let i be the local conditional context vector of node i. For normalized attention weights, Let j be the value vector of the j-th conditional element (from the text or fragment). For unnormalized attention scores, The denominator is summed using an index; the key-value vectors of the text and fragments are concatenated to obtain unified key-value pairs (K,V). For each structural node i and each conditional vector j, the attention score ψ is calculated. ij :

[0075]

[0076] Where g(·) can be either squared distance or radial basis function. For geometric anchor points (such as anchor atom positions in a fragment conformation) that are conditionally associated with the text or fragment, β is a learnable temperature parameter that adjusts the weights of the geometric distance. Let i be the query vector for node i. This is the transpose of the key vector of the condition element j. The feature dimension is used for scaling. Let be the coordinates of node i.

[0077] By employing multi-head cross-attention and several stacked layers, a fused node condition vector {Ci} can be obtained. Subsequently, pooling is used to obtain the global conditional context vector.

[0078]

[0079] in, This is the final global conditional context vector. For pooling functions, The global summary features of structural modes are used; pooling can employ weighted averaging, attention pooling, or local aggregation based on functional sites. The final C will serve as the unified conditional input for the diffusion generator and property predictor.

[0080] S4. Using the conditional context vector as conditional input, drive the 3D molecular generation model to perform generation tasks, and evaluate the multi-objective properties and analyze the multi-modal contribution of the generation results.

[0081] Specifically, the downstream task is driven by the conditional context vector C generated by S3, and an analysis feedback loop is constructed. The specific implementation is as follows:

[0082] Using C as a conditional input, a conditional 3D molecule generator p is driven. θ (M|C). This invention does not limit the specific generation architecture (such as diffusion models, flow models), but provides a standardized conditional interface. The generator outputs candidate molecules M=(H,X), i.e., a list of atom types and three-dimensional coordinates. A set of differentiable property predictors {Pk} is deployed to evaluate molecular properties (such as binding affinity, drug-likeness QED, and synthetic accessibility SA). During the generation process (especially based on diffusion model sampling), the multi-objective guidance loss L is calculated. guidance and its gradient L guidance Inject sampling steps to guide the generation direction toward optimizing the target properties. Confidence gating based on uncertainty quantification (such as Monte Carlo discarding) can be used to dynamically adjust the weights of the gradients for each property, enhancing the robustness of the guidance. By systematically removing or masking a modality (such as text, fragments) or module (such as multi-target guidance), changes in generation metrics (such as validity, Vina docking score, QED) can be quantitatively compared to evaluate the contribution of each component.

[0083] To effectively learn a conditional generative model that can simultaneously model 3D coordinates and atom types, a phased, multi-task training strategy is adopted, combined with classifier-free guidance technology to enhance controllability during sampling.

[0084] The model's total loss function L total It consists of three parts: the denoising loss L of the coordinates X Classification loss of atomic types L H And an optional auxiliary property predicting the loss L aux .

[0085] L total =L X +λ H L H +λ aux L aux

[0086] Where, λ H and λ aux It is a hyperparameter used to balance the weights of different tasks.

[0087] L X This is the standard loss of the diffusion model, using mean squared error (MSE) to measure the difference between the predicted noise and the actual injected noise:

[0088]

[0089] Among them, X t , It is the noisy state at time t. It is sampled Gaussian noise. It is the coordinate noise predicted by the model. It is the diffusion time step.

[0090] L H For discrete atom types, this paper uses cross-entropy loss to train the model to recover the original atom type. This can be viewed as a task of classifying each atom:

[0091] in, It is a logarithmic function. It is the probability distribution predicted by the model. It is a real atom type tag.

[0092] L aux To imbue the denoising network with some chemical knowledge, an auxiliary task can be introduced, allowing the network to predict a global property of the molecule while denoising. This helps improve the quality of the generated molecules.

[0093] To flexibly control the strength of conditional guidance during sampling, this paper employs a classifier-free guidance strategy during the training phase. In each training iteration, the conditional information C is sampled with a certain probability p. uncond It is replaced by a special null value ∅. At this point, the model learns to perform unconditional generation. Therefore, the denoising network ϵ θ In fact, two modes were learned:

[0094]

[0095] in, The empty condition vector represents the unconditional generation mode. During sampling, an enhanced conditional guidance effect can be obtained by linearly extrapolating the conditional and unconditional predictions. The corrected noise prediction calculation is as follows:

[0096] in, It is the corrected effective noise prediction. It is a scalar of guiding strength. >1 is the guiding strength coefficient. When w cWhen =1, it is a standard conditional generation; when When the value is greater than 1, the generation process places greater emphasis on the influence of condition C, thus generating molecules that are more closely related to the condition. This approach achieves flexible guided control without requiring the additional training of a classifier.

[0097] To ensure training stability, this paper employs a phased optimization process. First, the multimodal encoder and property predictor are pre-trained independently. Then, the parameters of these pre-trained models are fixed, focusing on training the core diffusion model. In the later stages of training, all parameters can be unfrozen for end-to-end joint fine-tuning to further improve performance. The optimizer uses AdamW, combined with learning rate warm-up and cosine annealing strategies to stabilize the training process.

[0098] The atomic coordinates and types output by the generative model are also chemically validated. This includes distance-based and GNN-based bond connectivity prediction, local geometric energy minimization (using a UFF force field), and valence and aromaticity checks and conflict backtracking resolution to ensure the chemical validity of the output molecules. First, a candidate bond set is generated based on the covalent bond radius and a preset tolerance τ. Then, a pre-trained graph neural network (GNN) classifier scores each candidate bond and outputs its probability p. ij All bonds with probabilities higher than the threshold (0.5) are initially accepted, forming an initial molecular map. For local regions with low confidence, the module triggers a constrained local energy minimization process. This process uses a universal force field (UFF) as the energy function, supplemented by harmonic potentials based on bond connection probabilities, to push bond lengths and bond angles towards more reasonable values. After geometry optimization, the module re-evaluates bond order, assigns aromaticity, and checks the valence states of all atoms. If conflicts are found, backtracking is performed based on connection probabilities and chemical heuristics, attempting to remove or modify low-confidence bonds until the structure satisfies all chemical constraints or reaches the maximum number of attempts. This systematic correction process ensures not only the generation of geometrically matched atomic clouds to the target but also the production of chemically effective molecular entities that can be directly used for downstream evaluation, significantly improving practicality.

[0099] To verify the technical effectiveness of this invention in multimodal data processing and fusion, multiple implementation examples were constructed on the CrossDocked2020 dataset. The process includes four stages: data preparation, model training, conditional sampling, and property evaluation. Key data and intermediate representations are recorded at each stage to facilitate subsequent analysis and attribution.

[0100] CrossDocked2020 contains a large number of protein-ligand complexes, providing a multimodal input source for this invention. In Example 1, protein pockets, ligands, and text / fragment conditions are uniformly converted into graph tensors and semantic vectors. To ensure the reproducibility of baseline comparisons, all candidate models use the same training set partitioning and hyperparameter settings, differing only in data processing and fusion methods. This setup fully highlights the innovation of this invention in "data preprocessing + fusion," that is, maintaining a consistent model architecture while only changing the aforementioned objects.

[0101] In Example 2, the method of the present invention was compared with three baselines: pocket splicing, pocket-only, and non-isovariant feature processing. Table 1 shows the performance of each method on key indicators such as effectiveness, novelty, diversity, Vina docking score, and QED. Since these indicators have different numerical scales, the Mann-Whitney U test, Clifsdelta, and Benjamini-Hochberg FDR were used as controls to ensure the statistical robustness of the conclusions. The results show that the method of the present invention significantly outperforms all baselines on all quantitative indicators, especially achieving improvements of approximately 1.4 kcal / mol and 0.10 on the Vima and QED indicators, respectively, indicating that the fused conditional vector C has a significant guiding effect on pharmaceutical properties.

[0102] Table 1 Comparison Table

[0103] Method Effectiveness Novelty Diversity Average Vina Average QED Average SA Full 72.4 68.1 0.54 -7.8 0.62 3.1 Pocket2Mol 58.9 55.2 0.49 -6.4 0.52 3.8 CVAE 41.2 37.5 0.47 -5.1 0.46 4.2 GraphAF 44.7 39.8 0.50 -5.3 0.48 4.0

[0104] Table 1 shows that the proposed method is significantly superior to the unconditional sequence / graph baseline (CVAE, GraphAF) in both effectiveness (%) and novelty, and also significantly better than Pocket2Mol in Vina score and QED index. For example, the median and mean of the Vina value of the proposed method are approximately 1.4-1.7 kcal / mol lower than those of Pocket2Mol; the Mann-Whitney U test yielded p < 0.01 at this experimental scale, and Clisdelta indicated a moderately high effect size (|6 ≈ 0.35). Regarding QED, the mean difference is approximately 0.10, which has high practical significance, as a higher QED indicates better suitability for medicinal chemistry; the corresponding effect size also exceeds 0.3, indicating that the difference is not accidental.

[0105] To avoid misjudgments caused by multiple comparisons, all p-values ​​for simultaneous tests were controlled using the Benjamini-Hochberg FDR with a significance threshold of 0.05. For each comparison, a confidence interval bootstrap (95% CI) with 1000 resampling iterations was provided, reporting the estimated uncertainty interval. These additional statistical treatments ensured the conclusions remained robust across multiple hypothesis testing frameworks.

[0106] The contribution of each modality was quantified through ablation experiments (Table 2): removing multi-target guidance or pocket conditions significantly reduced Vina and QED, respectively, verifying their necessity in dynamic fusion and analysis loop closure. Furthermore, attention weight visualization and feature perturbation experiments further demonstrated that the pocket modality dominates in determining binding sites, while the text modality plays a significant role in regulating the directionality of properties; the synthesis of both is key to achieving high-quality generation.

[0107] Model Architecture and Hyperparameters: The denoising network in this paper is based on the SE(3)-Transformer architecture, containing 6 equivariant layers and 128-dimensional latent features. In the multimodal encoder, the pocket encoder also adopts the SE(3)-Transformer, and the text encoder uses the pre-trained ChemBERTa. The property predictor is a simple graph neural network. The key hyperparameter settings are as follows:

[0108] Training parameters: Batch size is 32, AdamW optimizer is used, initial learning rate for diffusion model is 5x10⁻⁵, learning rate for property predictor is 1x10⁻⁵. -4 All models were trained for 200k steps on four NVIDIA A100 GPUs.

[0109] Diffusion parameters: Total diffusion steps T = 1000, using a linear noise scheduling scheme, β from 10 -4 Increased to 0.02.

[0110] Guiding parameters: Guiding weights are adjusted according to the target properties, typically set as follows: affinity 2.0, QEDw ged =1.0, SAWsa=0.5. The guiding scaling factor λ is small in the early stage of sampling and gradually increases in the later stage.

[0111] The sampling and post-processing algorithm uses DDIM for acceleration during the sampling process, with 100 steps. The core guided sampling algorithm is detailed in the algorithm documentation. In each denoising step, this algorithm first predicts the clean molecular state, then calculates the property gradient, and uses this gradient to correct the noise prediction, thereby guiding the generation process.

[0112] Table 2 Ablation Experiment Results

[0113] Setup Effectiveness Average Vina Average QED Explanation Full 72.4 -7.8 0.62 Full model NoGuidance 64.0 -6.9 0.57 Remove guidance NoPocket 59.3 -6.5 0.54 Remove pocket condition

[0114] Removing the NoGuidance module significantly reduced the optimization capabilities of Vina and QED, with an average decrease in Vina of approximately 1.3–1.5 kcal / mol and a reduction in QED of approximately 0.08–0.10. Combined with effect size measurements, this difference exhibits a moderate to large effect size, indicating that the guidance module provides quantifiable propulsion at steady state.

[0115] The removal of the target condition (NoPocket) mainly affects the biological relevance index of the method, and also causes a systematic decline in the Vina score, indicating that pocket condition information not only improves docking score, but also constrains the generation space to improve biological interpretability.

[0116] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0117] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A multimodal data fusion method for 3D molecular generation, characterized in that, Includes the following steps: The raw multimodal data is acquired and standardized to generate a unified set of standardized tensors; wherein the raw multimodal data includes protein pocket structure data, ligand or fragment structure data, and text description data. The standardized tensor set is processed based on the SE(3) equivariant graph neural network to extract atomic-level feature representations and global feature summaries with rotation and translation transformations; Through a hierarchical cross-attention mechanism, the atomic-level feature representation, the global feature summary, the text embedding vector, and the fragment feature vector are dynamically fused to generate a unified conditional context vector. Using the conditional context vector as conditional input, the 3D molecular generation model is driven to perform generation tasks, and the generation results are evaluated for multi-objective properties and analyzed for multi-modal contribution. Through a hierarchical cross-attention mechanism, the atomic-level feature representation, the global feature summary, the text embedding vector, and the fragment feature vector are dynamically fused to generate a unified conditional context vector, including: Representing structural modal nodes as Z V scalar part Through the learnable matrix W Q Projecting the query vector Q, embedding the text into z T and fragment graph encoding vector respectively through matrix , and , Projected as key vector K T K S Sum vector V T V S ; Where W· is the learnable projection matrix, and the output of the cross-attention after concatenating the key-value pairs of the text and the fragment is: in, Let i be the local conditional context vector of node i. For normalized attention weights, Let j be the value vector of the condition element. For unnormalized attention scores, The denominator is summed using an index; the key-value vectors of the text and fragments are concatenated to obtain unified key-value pairs (K,V); for each structural node i and each conditional vector j, the attention score ψ is calculated. ij : Where g(·) can be either squared distance or radial basis function. For geometric anchor points that are conditionally associated with text or fragments, β is a learnable temperature parameter that adjusts the weight of the geometric distance. Let i be the query vector for node i. This is the transpose of the key vector of the condition element j. The feature dimension is used for scaling. Let be the coordinates of node i; Through multi-head cross-attention and several layers of stacking, a set of fused node condition vectors {Ci} is obtained, which are then pooled to obtain the global conditional context vector: in, This is the final global conditional context vector. For pooling functions, The global summary features of the structural modes are used; pooling is performed using weighted average, attention pooling, or local aggregation based on functional sites, and the final C is used as the unified conditional input for the diffusion generator and property predictor. The text embedding vector generation process is as follows: Term cleaning and embedding encoding are performed on the text modality T: sentence segmentation, case normalization, and special character filtering are performed on the natural language description, and key terms are unified using a domain dictionary; the text is divided into a token sequence {t1,...,tm} using the ChemBERTa pre-trained model in the chemical domain; the context representation ei∈RdT of each token is calculated through a pre-trained Transformer encoder, and the global text embedding vector is obtained by using a [CLS] vector or a weighted average. The process of generating fragment feature vectors is as follows: For the set of molecular fragments or skeletons S={sj} that serves as a priori conditions, they are converted into a unified graph representation: Each fragment is parsed using RDKit to obtain an atom list Vj, a bond list Ej, and an initial three-dimensional conformation Xj. The atom list contains the atomic information obtained from parsing the fragment, which usually corresponds to nodes in the graph structure. The chemical bond list contains the bond information obtained from parsing the fragment, describing the topological connections between atoms, which corresponds to edges in the graph structure. The initial three-dimensional conformation refers to the spatial coordinate information of atoms in the fragment. Discrete attributes such as atom type, bond type, hybridization state, and aromaticity are enumerated and encoded to form discrete feature vectors of nodes / edges.

2. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, The raw multimodal data is standardized, including: The structural data is cleaned, aligned to coordinates, and cropped. Perform terminology cleaning and encoding on text data.

3. The multimodal data fusion method for 3D molecular generation as described in claim 2, characterized in that, Perform coordinate alignment and clipping on the structured data, including: Establish a local coordinate system with the geometric center or weighted centroid of the protein pocket as the origin; Translate all atomic coordinates of the protein and ligand / fragment to the local coordinate system; With the origin as the center, a clipping radius is set, and only the atomic coordinates within the clipping radius are retained.

4. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, The normalized tensor set is processed based on the SE(3) equivariant graph neural network to extract atomic-level feature representations and global feature summaries with rotation and translation transformations, including: The standardized tensor set is processed based on the SE(3) equivariant graph neural network; Initialize scalar and vector features for each atom, where the vector features are the atom's three-dimensional coordinates. The scalar features and the vector features are updated through multi-layer equivariant message passing, wherein the coordinate update amount is determined by the relative position vector between adjacent atoms and the scalar weight calculated based on the message, so as to ensure the equivariance of SE(3).

5. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, The generated results are evaluated for multi-objective properties, including: A set of constructed differentiable property predictors is used to score the properties of intermediate molecular states during the generation process; Calculate the comprehensive guided loss based on multi-objective property scoring; The integrated guiding loss is injected into the incremental sampling process of denoising diffusion in the form of a gradient of the current atomic coordinates. The gradient relative to the noise space or coordinate space is calculated by the chain rule to guide the generation direction.

6. The multimodal data fusion method for 3D molecular generation as described in claim 5, characterized in that, In the gradient guidance process, a confidence gating mechanism based on uncertainty quantization is introduced, including: For each property predictor, the Monte Carlo dropout method is used for multiple forward propagations to calculate the variance of the predicted values ​​as a measure of cognitive uncertainty. Confidence weights are calculated based on the cognitive uncertainty measure, and these confidence weights are negatively correlated with uncertainty. The confidence weights are used to weight the guiding gradient of the corresponding property predictor.

7. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, Multimodal contribution analysis was performed through ablation experiments, including: Remove or disable the text modality, fragment modality, or multi-objective guidance module respectively, and perform the generation task under the same conditions; The differences between the complete model and each ablation configuration in terms of the effectiveness of generated molecules, docking fraction, and drug-likeness indicators were compared to quantify the contribution of each modality or module.

8. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, The 3D molecule generation model is a conditional denoised diffusion probability model. The training of the 3D molecule generation model employs a classifier-free guidance strategy, including: During training iterations, conditional information is replaced with null values ​​with a set probability, enabling the model to learn both conditional and unconditional generation simultaneously. During sampling, linear extrapolation is performed on conditional noise prediction and unconditional noise prediction, and a guiding intensity coefficient is introduced to enhance the controllability of the generation process.

9. The multimodal data fusion method for 3D molecular generation as described in claim 1, characterized in that, The training of neural networks employs a phased optimization strategy, including: First, independently pre-trained multimodal encoders and property predictors; Then, the pre-trained parameters are fixed, and the diffusion generation model is trained. Finally, all model parameters are jointly fine-tuned end-to-end.

Images