A reaction optimization method based on dual-view contrast learning and double-layer attention

Abstract

The application discloses a reaction optimization method based on double-view contrast learning and double-layer attention, and relates to the technical field of chemical informatics and artificial intelligence. The method comprises the following steps: constructing a double-view molecular representation model, extracting molecular features from a graph structure view and a path sequence view, and pre-training the model through self-supervised contrast learning; constructing a chemical reaction prediction model; pre-training the chemical reaction prediction model by using the extracted molecular features in a chemical reaction data set; taking the reaction prediction model as a proxy model of Bayesian optimization; and realizing the collaborative and rapid optimization of material variables such as reactants, catalysts and solvents, and numerical condition variables such as temperature and concentration. The application effectively solves the problems of missing molecular representation information and low optimization efficiency of high-dimensional mixed variables in the prior art, and significantly speeds up the chemical research and development process.

Description

Technical Field

[0001] This invention belongs to the field of computational chemistry and organic synthesis, specifically relating to a reaction optimization method based on dual-view comparative learning and dual-layer attention. Background Technology

[0002] Predicting and optimizing chemical reactions are core issues in modern organic synthesis. Traditional optimization methods often rely on chemists' experience, intuition, or trial and error, which are not only inefficient but also fail to cover the vast space of chemical reactions. In recent years, with the development of machine learning technology, using data-driven methods to assist chemical synthesis has become a research hotspot.

[0003] However, existing technologies still face the following major bottlenecks in practical applications:

[0004] Insufficient molecular characterization capabilities: Existing molecular characterization is mostly based on a single perspective (such as using only fingerprints or only graph neural networks), which makes it difficult to simultaneously capture the topological information of molecules (graph structure perspective) and long-range dependence and connection patterns between functional groups (path sequence perspective).

[0005] Low prediction accuracy with small samples: Chemical experimental data acquisition is costly and often faces the "small sample" dilemma. Conventional deep learning models, without large-scale pre-training, struggle to achieve ideal generalization performance on small sample datasets (such as tens to hundreds of data points).

[0006] Limited optimization space: Traditional Bayesian optimization typically uses Gaussian processes (GP) as surrogate models. However, when dealing with high-dimensional spaces and mixed variables (containing both discrete chemical choices and continuous temperature / concentration variables), the computational complexity of GP increases cubically, and its fitting ability is limited, making it difficult to achieve co-optimization across the entire reaction space. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a response optimization method based on dual-view contrastive learning and dual-layer attention.

[0008] This invention integrates graph structure / path sequence dual-perspective molecular characterization, self-supervised contrastive learning, two-layer attention mechanism and Bayesian neural network, which can realize high-precision prediction of chemical reactions, quantitative assessment of uncertainty and intelligent closed-loop decision-making of reaction conditions, significantly improving the automation and intelligence level of new reaction development and process optimization.

[0009] To achieve the above objectives, this invention proposes a response optimization method based on dual-view contrastive learning and dual-layer attention, which includes the following steps:

[0010] 1) Construct a dual-perspective molecular representation model; construct a chemical molecular graph as the model input; the model uses a graph neural network to extract graph structure perspective features from the chemical molecular graph, and uses a Transformer with integrated edge features to extract path sequence perspective features from the chemical molecular graph;

[0011] 2) Chemical molecule data is used as pre-training data. The model is pre-trained using a self-supervised contrastive learning loss function to align the graph structure perspective features obtained by the model with the path sequence perspective features. The model then fuses the dual perspective features through a gated residual fusion module to obtain the molecular feature vector.

[0012] 3) Construct a chemical reaction prediction model, which includes a two-layer attention module and a Bayesian neural network layer: the first attention module is a substance interaction attention layer, used to capture the interaction between the feature vectors of each molecule in the reaction mixture; the second attention module is a transformation mapping attention layer, used to capture the mapping features of the transformation of reactant combinations into products; the Bayesian neural network layer is used for target prediction and uncertainty assessment.

[0013] 4) Construct the reaction feature matrix and pre-training target, and pre-train the chemical reaction prediction model;

[0014] 5) Use the chemical reaction prediction model trained in step 4) as a surrogate model for Bayesian optimization. For the chemical reaction to be optimized, the surrogate model performs an iterative search in the full reaction parameter space and outputs the optimal combination of reaction conditions.

[0015] According to a preferred embodiment of the present invention, the specific construction method of the two-layer attention module in step 3) includes:

[0016] A material interaction attention layer is constructed, which takes the molecular feature vectors of all participating substances in the reaction system as a set sequence and uses the self-attention mechanism to calculate the interdependence weights between reactants, catalysts, ligands, solvents and products, so as to update the feature representation of each component under a specific reaction environment.

[0017] A transformation mapping attention layer is constructed. When product information exists, a cross-attention mechanism is used to capture the mapping features from reaction atoms to product atoms by using the product feature vector as the query vector and the reaction mixture features updated by the material interaction layer as the bond vector and value vector. When product information does not exist, attention pooling is used to aggregate the global features of the reaction mixture.

[0018] According to a preferred embodiment of the present invention, the construction and prediction of the Bayesian neural network layer in step 3) specifically includes: inputting the feature vector output by the two-layer attention module into the Bayesian neural network, wherein the weight parameters in the network follow a probability distribution rather than fixed values; during the forward propagation process, specific weight values ​​are obtained by sampling the weight distribution and calculated in combination with variational inference methods;

[0019] The output of the chemical reaction prediction model contains two components: the expected value, which characterizes the target of the chemical reaction prediction, and the uncertainty variance, which characterizes the confidence level of the model prediction.

[0020] According to a preferred embodiment of the present invention, the construction of the reaction feature matrix in step 4) specifically includes: selecting a publicly available chemical reaction dataset as a pre-training dataset, using the dual-view molecular characterization model pre-trained in step 2) to extract molecular feature vectors of reactants, reagents, catalysts, solvents and products in the reaction system respectively; splicing and combining the extracted molecular feature vectors of each component according to the preset reaction rules to construct a reaction feature matrix containing complete reaction context information.

[0021] Through the above technical solution, the present invention has the following significant advantages:

[0022] 1. This invention achieves deep semantic fusion of molecular representation. The platform highly integrates the graph structure perspective (topological structure) and the path sequence perspective (long-range semantics) through self-supervised contrastive learning, effectively solving the problems of "long-range forgetting" and missing representation information in traditional single-perspective graph neural networks, and significantly improving the model's ability to understand complex organic molecules.

[0023] 2. This invention does not require complex quantum chemical calculations (DFT) or the artificial construction of physicochemical descriptors. It only requires the SMILES sequence of the reactants to directly drive the model, realizing end-to-end rapid prediction from molecular sequence to physical properties and reaction target values, which significantly reduces the threshold for use and computational cost.

[0024] 3. This invention designs a reaction prediction architecture that combines a two-layer attention mechanism with a Bayesian neural network. The two-layer attention mechanism can accurately capture key interactions within and between substances (such as reactants and reagents), while the BNN layer introduces probabilistic reasoning. The combination of the two significantly improves the model's ability to fit complex reaction patterns and its robustness.

[0025] 4. This invention overcomes the prediction bottleneck under small sample data. Through large-scale unsupervised molecular pre-training and reaction-level feasibility pre-training, this invention can maintain high target value prediction accuracy even with only a very small amount of experimental data, effectively solving the problem of expensive and scarce chemical experimental data.

[0026] 5. This invention utilizes the aforementioned reaction prediction network with uncertainty quantification capabilities as a surrogate model for Bayesian optimization. Compared to traditional Gaussian processes, this model can more efficiently balance "utilization" (utilizing known high-yield points) and "exploration" (trying unknown high-potential points) in a large-scale chemical space.

[0027] 6. This invention can simultaneously handle variables with different properties, achieving synergistic optimization of discrete chemical substances (such as catalysts, ligands, and solvents) and numerical process conditions (such as temperature, concentration, and equivalence ratio), helping users quickly identify the globally optimal reaction system. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0029] Figure 2 A diagram of a self-supervised contrastive learning network architecture with two perspectives (graph structure perspective + path sequence perspective);

[0030] Figure 3 A diagram showing the structure of a chemical reaction prediction network based on a two-layer attention mechanism;

[0031] Figure 4 These are candidate variables in the synthesis experiment of eltrombopag intermediate using the method of this invention. Detailed Implementation

[0032] The response optimization method based on dual-view contrastive learning and dual-layer attention of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The method of the present invention mainly includes the following steps:

[0033] 1) Construct a dual-perspective molecular representation model; construct a chemical molecular graph as the model input; the model uses a graph neural network to extract graph structure perspective features from the chemical molecular graph, and uses a Transformer with integrated edge features to extract path sequence perspective features from the chemical molecular graph;

[0034] 2) Chemical molecule data is used as pre-training data. The model is pre-trained using a self-supervised contrastive learning loss function to align the graph structure perspective features obtained by the model with the path sequence perspective features. The model then fuses the dual perspective features through a gated residual fusion module to obtain the molecular feature vector.

[0035] 3) Construct a chemical reaction prediction model, which includes a two-layer attention module and a Bayesian neural network layer: the first attention module is a substance interaction attention layer, used to capture the interaction between the feature vectors of each molecule in the reaction mixture; the second attention module is a transformation mapping attention layer, used to capture the mapping features of the transformation of reactant combinations into products; the Bayesian neural network layer is used for target prediction and uncertainty assessment.

[0036] 4) Construct the reaction feature matrix and pre-training target, and pre-train the chemical reaction prediction model;

[0037] 5) Use the chemical reaction prediction model trained in step 4) as a surrogate model for Bayesian optimization. For the chemical reaction to be optimized, the surrogate model performs an iterative search in the full reaction parameter space and outputs the optimal combination of reaction conditions.

[0038] Example 1: This example is a specific implementation case for steps 1) and 2) above.

[0039] This embodiment details the construction process of a molecular representation model based on dual-view self-supervised contrastive learning (dual-view molecular representation model), including the preprocessing of chemical molecular data, the construction details of the graph structure perspective and the path sequence perspective, the specific network configuration, and the pre-training strategy.

[0040] 1. Large-scale unsupervised pre-training data: 1 million molecules from the PubChem database were selected as unlabeled data, and their simplified molecular input line entry system (SMILES) was extracted.

[0041] The SMILES sequences of a molecule are converted into a molecular graph structure using cheminformatics tools (such as RDKit). The molecular graph includes:

[0042] Node feature matrix: dimension is ,in Number of atoms This represents the atomic feature dimension; each row corresponds to a feature vector of one atom, and the feature vector includes at least one or more combinations of atomic number, atomic connectivity, formal charge, number of connected hydrogen atoms, number of valence electrons, stereochemical configuration, ring characteristics, aromaticity, donor, and acceptor; in this example... ;

[0043] Edge feature matrix: dimension is ,in The number of chemical bonds, This represents the chemical bond feature dimension; each row corresponds to a feature vector of a chemical bond, and the feature vector contains at least one or more combinations of chemical bond types (single bond, double bond, triple bond, aromatic bond) and stereochemical features; in this example, ;

[0044] Adjacency matrix: Dimensions are , is used to characterize the topological connections between atoms, where the matrix elements indicate whether there are chemical bonds connecting the atomic nodes.

[0045] 2. Dual-Perspective Molecular Characterization Model Architecture Setup

[0046] The dual-view molecular characterization model consists of two parallel encoders, processing the graph structure view and the path sequence view respectively:

[0047] (1) Graph neural networks are used to extract graph structure perspective features from chemical molecular graphs. This invention constructs a graph structure perspective of molecules using a data augmentation strategy that combines full graph topological representation with random node masking; a graph isomorphic neural network (GIN) is used as a graph encoder to perform message passing and aggregation on atomic nodes and chemical bond edges in the chemical molecular graph, extracting deep topological structure features and generating graph structure perspective features. The random masking strategy includes: randomly selecting atomic nodes in the chemical molecular graph at a preset ratio (15%-25%) and replacing the node feature vectors with mask vectors.

[0048] In this embodiment, a Graph Isomorphism Network (GIN) is used as the backbone network, and its node message passing and aggregation formulas are as follows:

[0049] For the Nodes in the layer The feature aggregation process is represented as:

[0050]

[0051] in, Represents a node The neighborhood group, For nodes eigenvectors, For nodes With nodes The edges between them embed feature vectors, and MLP stands for Multilayer Perceptron Network.

[0052] The specific network configuration is as follows:

[0053] Network depth: Set the number of GIN layers to 3 to capture atomic neighborhood information within a 3-hop range.

[0054] Hidden layer dimension: The node feature embedding dimension and the hidden layer dimension are both set to 256.

[0055] Activation function: Use the ReLU activation function, combined with Layer Normalization for normalization.

[0056] Graph augmentation strategy (Masking): A random node masking strategy is adopted, randomly selecting 25% of the atomic nodes and replacing their feature vectors with all-zero features, forcing the model to use the surrounding topology to predict the type of the masked atom.

[0057] (2) Using Transformer with integrated edge features to extract path sequence perspective features from chemical molecular graph: a biased random walk algorithm guided by meta-path weight is used to sample and generate path sequences, and the self-attention mechanism with integrated edge features is used to extract path sequence perspective features.

[0058] The biased random walk algorithm guided by meta-path weights includes the following sub-steps:

[0059] S1. Construction of Global Meta-Path Library: Using the existing molecular database as a background knowledge base, perform a backtracking-free pure random walk on the molecules in the library to extract the original path sequence set;

[0060] S2. Meta-path optimization and weighting: Statistically analyze the frequency of occurrence of the original path sequence and filter out path patterns with lengths within a preset range; calculate the score of each path pattern based on path frequency and diversity balancing strategy, and select the 512 high-frequency path patterns with the highest scores to calculate the weights and form a meta-path library.

[0061] S3. Target molecule guided sampling: For the target molecule to be processed, it is mapped to the meta-path library; a biased random walk with backtracking is performed, where the transition probability between nodes is dynamically adjusted by the weights in the meta-path library, so that the walk trajectory tends to match high-scoring meta-paths;

[0062] S4. Sequence Generation: The number of samples is dynamically set according to the number of atoms in the target molecule to generate the final path sequence perspective features.

[0063] In this embodiment of the Transformer network incorporating edge features, the encoder processing the path sequence adopts the standard Transformer Encoder architecture, configured as follows:

[0064] Number of layers and attention heads: The number of encoder layers is set to 4, the number of heads of the multi-head self-attention mechanism is 8, and the embedding layer dimension is 256.

[0065] Edge feature embedding: To distinguish chemical bond types (single bond, double bond, aromatic bond, etc.), edge features are incorporated into the attention mechanism. The specific formula is as follows:

[0066]

[0067] in: This is the output of the attention mechanism; These represent the query vector, key vector, and value vector matrices, respectively. For the hidden layer dimension of the attention head, This is a scaling factor used to prevent gradient vanishing; This represents the original semantic relevance score between nodes; The edge feature matrix encodes the chemical bond type (such as single bond, double bond) and stereochemical properties of the connecting atom pairs in the path; This represents a topology mask matrix, used to indicate whether there are direct connections between nodes or whether they are located within a specific receptive field. Hadamard product (element-wise multiplication) is used to utilize topological structures. opposite edge features Perform filtering and weighting; This constitutes a structural bias term, which is directly added to the attention score.

[0068] 4. Pre-training process and parameter configuration

[0069] We constructed a dual-view dataset using the PubChem 1 million molecule dataset and trained it using self-supervised comparative learning.

[0070] Loss function: An improved InfoNCE contrastive learning loss function is adopted, including in-view contrastive loss within the graph structure perspective. And cross-perspective contrast loss between graph structure perspective and path sequence perspective. The specific calculation formula is as follows:

[0071]

[0072] in: For the extracted features of the original view, The extracted node mask view features. Features of the extracted path sequence. This is the balance coefficient for cross-model loss.

[0073] Optimizer: Use Optimizer, weight decay ( Set as .

[0074] Learning rate strategy: Initial learning rate set to , After decay to .

[0075] Batch Size: Set to 256 to provide a sufficient number of negative samples.

[0076] Training duration: Iterative training is performed on a core subset containing 1 million molecules for a total of 200 rounds.

[0077] Hardware environment: Model training was performed on a server configured with a 4090 (24GB) GPU.

[0078] Example 2: Validation of Feature Fusion and Molecular Property Prediction

[0079] The gated residual fusion module of this invention synthesizes the final representation vector through a weighted gated residual mechanism. Let the input features from the two perspectives be... and The output after fusion It can be represented as:

[0080]

[0081] in: It is a quality confidence score calculated based on the feature itself, used to suppress noise in low-quality features; It is a global gating weight calculated through cross-perspective interaction, used to dynamically adjust the information proportion of different perspectives; , These are learnable residual balance coefficients used to preserve the unique information of the original viewpoint during depth transformation; This is a non-linear mapping layer used for dimensionality compression.

[0082] To verify the representation effect, the pre-trained dual-view features were symmetrically fused, compressed to the prediction task dimension by MLP, and tested on the MoleculeNet benchmark dataset.

[0083] Table 1 - Comparison of RMSE between the method of this invention and other mainstream methods on molecular property prediction regression tasks.

[0084]

[0085] Table 2 - Comparison of ROC_AUC between the method of this invention and other mainstream methods on the molecular property prediction and classification task.

[0086]

[0087] As shown in Tables 1 and 2, the experimental results of this invention (Ours) demonstrate superior performance compared to mainstream comparative methods (MolCLR, MIFS) on multiple molecular benchmark datasets. This significant performance improvement is primarily attributed to the unique "graph-path collaboration" architecture design of this invention:

[0088] Multi-perspective complementary representation: Local topological structures of molecules are captured through GIN (graph structure perspective), and long program sequence dependencies are extracted in conjunction with Transformer (path sequence perspective). This strategy based on dual-perspective self-supervised contrastive learning ensures that the model can acquire complete molecular semantic information from different dimensions.

[0089] Adaptive feature fusion: The gated residual fusion module is used to dynamically weight and aggregate the heterogeneous features, which effectively solves the problems of noise interference and information misalignment from different perspectives.

[0090] In summary, this invention significantly enhances the model's ability to analyze the complex properties of molecules by combining "structure-sequence" dual representation with "adaptive fusion," thereby achieving optimal prediction accuracy and robustness in both regression and classification tasks.

[0091] Example 3: This example is a specific implementation case for steps 3) to 5) above.

[0092] This embodiment details the construction of the chemical reaction prediction model, the pre-training strategy using large-scale USPTO data, and the small-sample performance evaluation on the classic Buchwald-Hartwig coupling reaction dataset.

[0093] 1. Network architecture design for chemical reaction prediction models

[0094] This invention constructs a chemical reaction prediction model based on a two-layer attention mechanism and a Bayesian neural network (BNN). The input features are the feature vectors of each component (reactants, reagents, solvents, products, etc.) extracted from the dual-view molecular characterization model in Example 1.

[0095] (1) First layer: Matter interaction attention layer

[0096] Input: Concatenate the molecular feature vectors of all participating substances in the reaction system (reactant A, reactant B, catalyst, ligand, base, solvent, and product) into a matrix. Where p is the feature vector of the product molecule. This represents the molecular characteristic vectors of the reactants other than the products.

[0097] Mechanism: A multi-head self-attention mechanism is used to calculate the interaction matrix within the sequence.

[0098] This layer aims to simulate the "mixed environment" within the reactor, capturing solvent effects and catalyst-ligand-substrate interactions, and adjusting their eigenvectors to obtain a reaction feature matrix. .

[0099] (2) Second layer: Transformation mapping attention layer

[0100] Input: Adjusted response feature matrix .

[0101] Mechanism: Employs a cross-attention mechanism; specifically, Query ( ) is the feature vector of the product from the first layer output. (Represents the target structure), Key ( ) & Value ( ) represents the output from the first layer. (This indicates the reaction mixture environment).

[0102] This layer of forced modeling focuses on which specific substructures (Key) in the reactants contribute most to the formation of the final product (Query), thereby explicitly modeling atomic mapping and electron transfer processes and extracting higher-order reaction transformation features. .

[0103] (3) Prediction layer: Bayesian Neural Network (BNN)

[0104] Structure: Unlike traditional neural networks, the prediction layer in this embodiment does not use deterministic weight values. Instead, it introduces variational inference to model the weight parameters in the network as a probability distribution. Specifically, it assumes that the network weights... It follows a Gaussian distribution, that is... Since the true posterior distribution is difficult to calculate directly, the model uses a parameterized variational distribution. To approximate the true posterior distribution During forward propagation, sampling is performed using reparameterization techniques: .

[0105] Output: Through multiple random forward propagations, the model ultimately outputs the target predicted mean of the chemical reaction (as the final predicted value) and the predicted variance (as an estimate of the model's uncertainty). This allows the model to not only predict "what the target value is," but also "how confident it is of that prediction."

[0106] (4) Loss function: In order to train the Bayesian neural network constructed above, this embodiment designs a composite loss function. This loss function aims to minimize the difference between the prediction error and the uncertainty distribution of the model at the same time, so as to prevent overfitting while ensuring accuracy.

[0107] Specifically, the total loss function From the mean square error loss term and Kullback-Leibler (KL) divergence loss term The weighted summation yields the following mathematical expression:

[0108]

[0109] Among them: prediction error term ( )Calculation model predicts yield Compared with the actual experimental yield The mean squared error between the two. Its physical meaning is to encourage the model to fit the actual chemical reaction results as closely as possible at the data level, and it is calculated by the following formula: ,in The batch size is the number of samples. The Bayesian regularization term ( Calculate the variational posterior distribution Compared with the preset prior distribution The KL divergence between (usually assumed to be a standard Gaussian distribution) is a regularization term used to prevent overfitting. Its physical meaning lies in constraining the distribution of neural network weights to prevent deviations from prior assumptions, thus quantifying model uncertainty. The balance coefficient ( This is used to adjust the weights between prediction accuracy and distribution constraints. Smaller... This means the model focuses more on fitting the training data, while larger... This forces the model to pay more attention to the distribution characteristics of the weights, as in this embodiment. .

[0110] 2. Reaction pre-training: To address the problem of insufficient small sample reaction data, the chemical reaction prediction model is first initialized based on general chemical reaction knowledge.

[0111] Dataset: The dataset used is a publicly available dataset from the USPTO (United States Patent and Trademark Office), which was cleaned and retained to preserve approximately 1.45 million chemical reaction records.

[0112] Pre-training task: Construct a "response feasibility binary classification" task.

[0113] Positive samples: Real USPTO reaction records with a yield greater than 0 (marked as 1).

[0114] Negative samples: Randomly shuffle the correspondence between reactants and products; randomly delete a reactant; randomly replace a substance with a different substance from another reaction to generate a chemically illogical reaction (marked as 0). Negative sample data accounts for 30%.

[0115] Training objective: Freeze the underlying molecular characterization parameters and train only the two-layer attention module to learn to distinguish between reasonable reaction transformation logic and random combinations.

[0116] Training parameters: Initial learning rate is Epoch is set to 100, Batch size is set to 2048, and BCE_loss is used.

[0117] Training results: .

[0118] 3. Performance evaluation under small sample size: After pre-training, the classic Buchwald-Hartwig (BH) amination reaction dataset was used for testing.

[0119] Dataset Description: This dataset, released by Doyle et al., contains 3955 high-throughput experimental data points. It covers combinations of 15 aryl halides, 4 Buchwald ligands, 3 bases, and 23 isoxazole additives.

[0120] Experimental setup: A random split strategy was used, with four different training / test set ratios: 70 / 30: sufficient data (approximately 2770 training samples); 50 / 50: moderate data; 30 / 70: limited data; 2.5 / 97.5: extremely small sample size (only approximately 100 samples were used for training, simulating the scenario of extremely scarce data in the early stages of a real experiment).

[0121] Evaluation index: Coefficient of determination ( (The closer to 1, the better). The data partitioning recommended by Ahneman et al. was used, and the data was trained 10 times and the average value was taken.

[0122] 4. Experimental Results The comparison results of the present invention model (Ours) with the mainstream baseline models (DFT-Random Forest, One-hot-Random Forest, BERT-Yield) are shown in Table 3.

[0123] Table 3 - Comparison of yield prediction performance between the proposed model and the baseline model on the Buchwald-Hartwig dataset.

[0124]

[0125] Results Analysis: Given sufficient data (70%), the model of this invention... The model achieved an accuracy of 0.975 and an RMSE of only 0.043, demonstrating the powerful ability of dual-perspective molecular characterization and the bilayer attention mechanism to capture chemical reaction characteristics. Under moderate sample data conditions, the model of this invention... The results achieved Score of 0.962 and 0.935; even under the challenge of extremely small sample sizes (2.5%), the model of this invention, benefiting from the prior knowledge introduced by large-scale pre-training, still maintains an Score of 0.66. It demonstrates excellent few-shot learning capabilities. This lays a solid foundation for subsequent Bayesian optimization based on a very small amount of experimental data.

[0126] Example 4: Collaborative Optimization of Chemical Reaction Conditions Based on High-Throughput Datasets

[0127] This embodiment details the construction of the Bayesian optimization framework based on the reaction prediction network described in Embodiment 3, as well as the virtual screening and collaborative optimization process performed on three classic high-throughput experimental datasets using this method.

[0128] 1. Construction of Bayesian Optimized Proxy Model

[0129] 1) Proxy model: In order to efficiently find the optimal conditions in the huge chemical reaction space, this invention uses the "chemical reaction prediction model" trained in Example 3 as a Proxy model for Bayesian optimization.

[0130] 2) Uncertainty Quantification: Utilizing the variational inference capability of BNN layers, for any given combination of response conditions... The model not only outputs the mean of the predicted target It also outputs the prediction variance. .in This represents the model's expected objective under this condition. This represents the degree to which the model is unaware of the region (i.e., uncertainty).

[0131] 3) Acquisition Function: To balance "exploration" (i.e., prioritizing candidate points with higher predicted target values) and "exploration" (i.e., prioritizing unknown regions with greater model prediction uncertainty) during Bayesian optimization, this embodiment employs the Expected Improvement (EI) strategy as the acquisition function. The EI acquisition function measures the performance of a given candidate point... The expected improvement that can be obtained relative to the currently observed optimal objective value is defined as follows:

[0132]

[0133] Assume the target value follows a normal distribution. Under the premise that the above integral can be analytically derived into a closed form, i.e., the formula used in the actual code:

[0134]

[0135] in, Standardized variables:

[0136]

[0137] in,

[0138] : Candidate reaction conditions to be evaluated (including discrete material selection and continuous process parameters).

[0139] : The mean of the deep learning agent model's prediction of the response under this condition (representing the expected performance).

[0140] The standard deviation of the prediction by the deep learning agent model (representing uncertainty or confidence level).

[0141] The current global optimum, i.e., the highest yield or best metric recorded in the completed experimental dataset.

[0142] Exploration - Utilize a trade-off parameter, usually set to a very small positive number (such as 0.01), to prevent the search from getting stuck in local optima and to fine-tune the urgency of exploration.

[0143] : Cumulative distribution function (CDF) of the standard normal distribution.

[0144] : The probability density function (PDF) of the standard normal distribution.

[0145] 4) Batch sampling mechanism: To meet the needs of high-throughput chemical experiments or parallel computing, this invention introduces the Kriging Believer (KB) strategy to construct a batch parameter recommendation algorithm based on a single acquisition function (such as EI).

[0146] In traditional Bayesian optimization, the acquisition function outputs only a single optimal candidate point each time. To achieve parallel optimization (Batch Size > 1), this embodiment uses a KB strategy for serialized point generation, and the specific steps are as follows:

[0147] S1. Initial Selection: Based on the current agent model, maximize the EI acquisition function to determine the first candidate reaction condition in the batch. .

[0148] S2. Virtual observation: Since no physical experiment has been conducted at this time, it is impossible to obtain [data / information]. The actual reaction results Based on the KB strategy, a proxy model is directly adopted for... Predicted mean As a pseudo-observation.

[0149] S3. Virtual Update: Will The model is temporarily added to the historical dataset as a virtual sample, and the hyperparameters of the model are retrained to update the posterior distribution of the surrogate model.

[0150] S4. Iterative Generation: Based on the updated model posterior distribution, find new candidate points that maximize EI. .

[0151] S5. Loop Termination: Repeat the above "Predict-Virtual Fill-Update-Reselect" steps until the number of selected candidate points reaches the preset batch size. (Batch Size). Finally, this... All experimental conditions were submitted to the laboratory for parallel verification.

[0152] 2. Virtual simulation experiment dataset

[0153] To verify the generality of the method, a "virtual laboratory" was constructed using three classic high-throughput experimental datasets with different chemical characteristics. During the simulation, the model could only view the yield of the data points that had been "sampled," while the remaining data were not visible to the model.

[0154] Suzuki-Miyaura Coupling Reaction Dataset (Perera et al.): Features: Contains 5760 reaction combinations. Involves 12 ligands, 8 boric acids, 4 solvents, and other variables.

[0155] Aryl_animation Coupling Reaction Dataset (Ahneman / Doyle et al.): Features: This is the dataset used in Example 3, containing 3960 data points.

[0156] Direct Arylation dataset (Shields et al., Nature): Features: Contains 1728 reactions with numerical conditional variables.

[0157] 3. Simulation and optimization of experimental procedures

[0158] For each of the above datasets, the following standardized simulation optimization process is performed:

[0159] Step 1: Cold Start. Randomly select 5 combinations of reaction conditions from the entire reaction space, check their true yields, and use them as the initial training set. .

[0160] Step 2: Train the surrogate model. Using... The chemical reaction prediction model of this invention was fine-tuned.

[0161] Step 3: Condition Recommendation. The model predicts all remaining unexplored response combinations, calculates the EI score, and uses the KB algorithm to recommend the 5 condition combinations with the highest scores.

[0162] Step 4: Virtual Experiment. Examine the actual yield of these 5 recommendation conditions in the dataset, and add them to the training set to update the results. .

[0163] Step 5: Iteration. Repeat steps 2-4 for a total of 30 iterations (i.e., a total of 150 experiments).

[0164] 4. Comparison of baseline and experimental results

[0165] Our method is compared with two mainstream methods: Random Search (the most commonly used benchmark in industry, which selects conditions completely randomly) and Gaussian Process Bayesian Optimization (GP-BO), the standard Bayesian optimization method in academia, which uses descriptors provided by Shields as input to the kernel function. Comparative tests were conducted on three representative chemical reaction datasets (Aryl_animation, Direct_Arylation, and Suzuki_miyaura). Test metrics included whether the global maximum yield (Max_yield) was found, the number of iterations required to find the maximum yield (Round), and the hit rate of the Top-10 high-yield samples and their corresponding number of iterations. Experimental results are shown in Table 4.

[0166] Table 4 - Experimental results of co-optimization of chemical reaction conditions based on high-throughput datasets

[0167]

[0168] Analysis of experimental results:

[0169] 1. Analysis of Aryl Amination (Aryl Amination Reaction) Experimental Results

[0170] In this reaction system, the target global maximum yield is 99.9999%. Existing techniques, Random Search and GP-BO, failed to find the global maximum yield at the experimental cutoff, and exhibited low Top-10 hit rates (only 10% for Random Search) or slow convergence (GP-BO only reached 80% by the 21st round). In contrast, the method of this invention demonstrates extremely strong global optimization capabilities, successfully locking the theoretical maximum yield (99.9999%) in only the 5th and 7th rounds respectively in two independent experiments, and maintaining 80% high-yield sample coverage by the 30th round. This indicates that in the complex amination reaction space, this invention can quickly escape local optima and accurately locate extreme points.

[0171] 2. Analysis of Experimental Results of Direct Arylation

[0172] In this reaction system, the target global maximum yield is 100.0%. Random Search completely failed, failing to find the maximum yield and achieving a 0% Top-10 hit rate. While GP-BO found the maximum yield (average 10.5 rounds), its efficiency in traversing the Top-10 high-quality samples was low, only achieving 100% coverage by the end of the experiment (30th round). The method of this invention further improves the optimization speed, finding the global optimum in the 11th and 8th rounds respectively. More significantly, this invention achieves 100% coverage of the Top-10 high-quality samples in the 11th round, nearly three times more efficient than GP-BO (30th round). This demonstrates that this invention has extremely high search efficiency when exploring high-yield regions, significantly reducing experimental costs.

[0173] 3. Analysis of experimental results of Suzuki-miyaura coupling reaction

[0174] In this reaction system, the target global maximum yield is 100.0%. While Random Search stumbled upon the maximum in one experiment (round 13), it failed in all other experiments, exhibiting significant randomness and instability. GP-BO performed poorly in this task, failing to find the global maximum yield in multiple experiments. The method of this invention demonstrates superior robustness and stability, successfully finding the global maximum yield in 100% of three independent experiments, and completing optimization in as little as two rounds (averaging approximately seven rounds). Furthermore, this invention achieved a 70% Top-10 hit rate in round 25, outperforming GP-BO's 60% (round 24) and Random's 10%.

[0175] The experimental results from the three datasets above demonstrate that the chemical reaction yield prediction and optimization method proposed in this invention significantly outperforms existing technologies in different types of reaction spaces. In all cases where the optimal solution was successfully found, the number of iterations required by this invention was less than or equal to that of GP-BO. In complex reactions such as Aryl_animation and Suzuki_miyaura, which are difficult for GP-BO to handle, this invention is the only method that can stably obtain the global optimum. Furthermore, this invention can cover more Top-k high-yield conditions with fewer experiments, significantly reducing the trial-and-error cost of wet experiments.

[0176] Example 5: Collaborative Rapid Optimization Closed-Loop Scheme

[0177] Based on the aforementioned embodiments, this embodiment provides a collaborative and rapid optimization closed-loop process for the iterative search process of the surrogate model in the full reaction parameter space involved in step 5). This process includes:

[0178] Configuration phase: Users input the optimization goal, parameters to be optimized, and hyperparameters used for model training through the tool interface, and independently select the uncertainty evaluation mode (BNN mode) of the proxy model.

[0179] Recommendation phase: Based on the selected evaluation mode and acquisition function, the model searches and recommends the next optimal combination of reaction conditions within the full reaction condition space;

[0180] Feedback phase: Users receive recommended conditions, conduct wet experiments, and input the measured actual target value data into the tool;

[0181] Update phase: The model automatically aggregates new experimental data to update parameters online and enters the next round of recommendation cycle, forming a closed-loop optimization process of "configuration-recommendation-feedback-update".

[0182] Example 6: Synergistic Optimization of Chemical Reaction Conditions Based on Real Experiments

[0183] This embodiment describes the entire process of optimizing real wet experimental conditions for a synthetically challenging Suzuki-Miyaura coupling reaction using the method described in this invention.

[0184] 1. Experimental background and target reaction: Eltrombopag is an important drug for treating blood diseases; the conventional synthetic route of eltrombopag involves a key intermediate, 2´-methoxy-3´-nitrobiphenyl-3-carboxylic acid. Under conventional conditions, the yield of this intermediate is usually less than 50%, and there are many byproducts.

[0185] Baseline conditions: Commonly used laboratory conditions are employed. Under these conditions, the yield after 24 hours of reaction is 47%, which is insufficient to meet production demands.

[0186] 2. Optimization of the Variable Space (Search Space): To find the optimal conditions, a discrete reactant search space was established, comprising five dimensions: electrophile, nucleophile, catalyst / ligand, base, and solvent. The number of combinations is [number missing]. ; and a discrete conditional search space encompassing six dimensions: electrophile concentration, nucleophile equivalent, catalyst equivalent, ligand equivalent, base equivalent, and temperature, with a combination number of . A total of 3,102,624 possible reaction conditions; see appendix for specific variables. Figure 4 .

[0187] 3. Model Setup and Optimization Process: The chemical reaction prediction model pre-trained in Example 3 is used as a surrogate model.

[0188] Feature input: Use the SMILES of the above reactants, catalysts, ligands, bases, and solvents, along with their corresponding concentrations (or equivalent concentrations) and temperature values, as input features.

[0189] Acquisition function: EI-KB strategy is adopted.

[0190] Batch size: 10 (i.e., 10 experiments per round).

[0191] Initialization method: Random initialization.

[0192] 4. Experimental Results and Wet Experiment Verification

[0193] 1) Experimental verification method

[0194] To verify the effectiveness of the Bayesian optimization framework proposed in this invention, we conducted real-world synthesis experiments in the laboratory using the reaction conditions recommended by the trained chemical reaction prediction model for each batch. All reaction solutions underwent standard post-treatment after the reaction, and the actual yield of the target intermediate (2´-methoxy-3´-nitrobiphenyl-3-carboxylic acid) was accurately determined by high-performance liquid chromatography (HPLC) using the internal standard method. The actual yield was then fed back to the model for parameter updates.

[0195] 2) Iterative optimization process data

[0196] During a total of 7 rounds of optimization (70 experimental points), we observed a significant upward trend in the actual yield. Some experimental data are shown in the table below:

[0197] Table 5 - Results of Synergistic Optimization of Chemical Reaction Conditions Based on Real Experiments (Partial)

[0198]

[0199] 3) Comparison of optimal conditions with benchmark

[0200] After seven rounds of iteration, the optimal reaction conditions determined by this invention are as follows: using 1-bromo-2-methoxy-3-nitrobenzene (0.1 mol / L, 1 equivalent) and pinacol 3-carboxyphenylboronic acid (1.5 equivalent) as reaction substrates, using palladium acetate (0.1 equivalent) as catalyst, 2-(diphenylphosphine)-2,6-dimethoxybiphenyl (0.25 equivalent) as ligand, and sodium hydroxide (2.5 equivalent) as base, the reaction is carried out in methanol / water (4:1, volume ratio) at 100 °C for 24 hours.

[0201] Under these conditions, the experiment was repeated three times, and the average yield remained stable at over 95%. Compared with the baseline (yield 47%), the method of this invention increased the yield by more than two times while exploring only 0.002% of the search space, greatly improving the production efficiency of this pharmaceutical intermediate.

[0202] In summary, this invention combines topological information from a graph structure perspective with long-range semantics from a path sequence perspective, significantly improving the expressive power of molecular features. Through large-scale two-stage pre-training (molecular level + reaction level), this invention enables the model to maintain high prediction accuracy even with only a small amount of experimental data (e.g., 2.5% of the training set). This invention, far surpassing traditional machine learning methods, utilizes a deep learning surrogate model to replace the Gaussian process, overcoming the computational bottleneck of traditional Bayesian optimization in high-dimensional spaces with mixed variables. It enables rapid and collaborative screening of complex chemical reaction conditions, significantly reducing R&D costs.

[0203] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A response optimization method based on dual-view contrastive learning and dual-layer attention, characterized in that, Includes the following steps: 1) Construct a dual-perspective molecular representation model; construct a chemical molecular graph as the model input; the model uses a graph neural network to extract graph structure perspective features from the chemical molecular graph, and uses a Transformer with integrated edge features to extract path sequence perspective features from the chemical molecular graph; 2) Chemical molecule data is used as pre-training data. The model is pre-trained using a self-supervised contrastive learning loss function to align the graph structure perspective features obtained by the model with the path sequence perspective features. The model then fuses the dual perspective features through a gated residual fusion module to obtain the molecular feature vector. 3) Construct a chemical reaction prediction model, which includes a two-layer attention module and a Bayesian neural network layer: the first attention module is a substance interaction attention layer, used to capture the interaction between the feature vectors of each molecule in the reaction mixture; the second attention module is a transformation mapping attention layer, used to capture the mapping features of the transformation of reactant combinations into products; the Bayesian neural network layer is used for target prediction and uncertainty assessment. The specific construction methods of the two-layer attention module include: A material interaction attention layer is constructed, which takes the molecular feature vectors of all participating substances in the reaction system as a set sequence and uses the self-attention mechanism to calculate the interdependence weights between reactants, catalysts, ligands, solvents and products, so as to update the feature representation of each component under a specific reaction environment. A transformation mapping attention layer is constructed. When product information exists, a cross-attention mechanism is used to capture the mapping features from reaction atoms to product atoms by using the product feature vector as the query vector and the reaction mixture features updated by the material interaction layer as the bond vector and value vector. When product information does not exist, attention pooling is used to aggregate the global features of the reaction mixture. The construction and prediction of the Bayesian neural network layer specifically includes: inputting the feature vector output by the two-layer attention module into the Bayesian neural network, where the weight parameters follow a probability distribution rather than fixed values; during the forward propagation process, obtaining specific weight values ​​by sampling the weight distribution and calculating them using variational inference methods; The output of the chemical reaction prediction model contains two components: the expected value, which characterizes the target of the chemical reaction prediction, and the uncertainty variance, which characterizes the confidence level of the model prediction. 4) Construct the reaction feature matrix and pre-training target, and pre-train the chemical reaction prediction model; 5) Use the chemical reaction prediction model trained in step 4) as a surrogate model for Bayesian optimization. For the chemical reaction to be optimized, the surrogate model performs an iterative search in the entire reaction parameter space and outputs the optimal combination of reaction conditions. The collaborative optimization of chemical reaction conditions specifically includes: constructing a Bayesian optimization framework based on a deep learning surrogate model, using the pre-trained chemical reaction prediction model from step 4) as a surrogate model to replace the Gaussian process in Bayesian optimization; and performing target prediction and uncertainty assessment on unexplored reaction condition combinations within the full reaction parameter space to obtain the recommended reaction condition combinations for the current iteration. The proxy model can further update the actual target value data of the reaction obtained from the recommended combination of reaction conditions based on external feedback input, and output the recommended combination of reaction conditions for the next round of iteration. The iteration of feedback input and reaction condition recommendation is repeated until the optimal combination of reaction conditions that meets the preset requirements is obtained.

2. The method according to claim 1, characterized in that, Step 1) The construction of the chemical molecular map involves: converting the SMILES sequences of chemical molecules into a chemical molecular map using cheminformatics tools; the chemical molecular map includes: Node feature matrix: dimension is ,in Number of atoms For atomic feature dimensions; each row corresponds to the feature vector of one atom. Edge feature matrix: dimension is ,in The number of chemical bonds, This represents the feature dimension of chemical bonds; each row corresponds to a feature vector of a chemical bond. Adjacency matrix: dimension is , is used to characterize the topological connections between atoms, where the matrix elements indicate whether there are chemical bonds connecting the atomic nodes.

3. The method according to claim 1, characterized in that, In step 1), the model uses a graph neural network to extract graph structure perspective features from the chemical molecular graph, including: constructing the graph structure perspective of the molecule using a data augmentation strategy that combines full graph topological representation with random node masking; and using the graph isomorphic neural network GIN as a graph encoder to perform message passing and aggregation on the atomic nodes and chemical bond edges in the chemical molecular graph, extracting deep topological structure features, and generating graph structure perspective features. The random node masking strategy includes: randomly selecting atomic nodes in the chemical molecular diagram at a preset ratio and replacing the node feature vector with a mask vector.

4. The method according to claim 1, characterized in that, In step 1), the extraction of path sequence perspective features from the chemical molecular graph using the Transformer with integrated edge features includes: sampling and generating path sequences using a biased random walk algorithm guided by meta-path weights, and extracting path sequence perspective feature vectors using a self-attention mechanism with integrated edge features. The biased random walk algorithm guided by meta-path weights includes the following sub-steps: S1: Using an existing molecular database as a background knowledge base, perform a backtracking-free pure random walk on the molecules in the database to extract the original path sequence set; S2: Calculate the frequency of occurrence of the original path sequence and filter out the path patterns with lengths within a preset range; calculate the score of each path pattern based on the path frequency and diversity balancing strategy, and select the X high-frequency path patterns with the highest scores to calculate the weights and form a meta-path library. S3: For the target molecule to be processed, map it to the meta-path library; perform a biased random walk with backtracking, where the transition probability between nodes is dynamically adjusted by the weights in the meta-path library, so that the walk trajectory tends to match high-scoring meta-paths; S4: Dynamically set the sampling quantity based on the number of atoms in the target molecule to generate the final path sequence perspective features.

5. The method according to claim 1 or 4, characterized in that, In step 1), the edge features are incorporated into the self-attention mechanism using a structured bias injection strategy, specifically as follows: ； in: For the output of the self-attention mechanism, These represent the query vector, key vector, and value vector matrices, respectively. For the hidden layer dimension of the attention head, This is a scaling factor used to prevent gradient vanishing; This represents the original semantic relevance score between nodes; This represents the edge feature matrix, which encodes the chemical bond type and stereochemical properties of the connecting atomic pairs in the path; This represents a topology mask matrix, used to indicate whether there are direct connections between nodes or whether they are located within a specific receptive field. Hadamard product is used to utilize topological structures. opposite edge features Perform filtering and weighting; This constitutes a structural bias term, which is directly added to the attention score.

6. The method according to claim 1, characterized in that, In step 2), the loss function for self-supervised contrastive learning adopts the improved InfoNCE loss, which includes intra-view contrastive loss within the graph structure perspective and cross-view contrastive loss between the graph structure perspective and the path sequence perspective.

7. The method according to claim 1, characterized in that, In step 2), the gated residual fusion module calculates the similarity weight and gate weight between the two perspective features to perform weighted fusion of the features, and superimposes the original graph structure perspective features and path sequence perspective features back onto the fused features to obtain the final molecular feature vector.

8. The method according to claim 1, characterized in that, Step 4) The construction of the reaction feature matrix specifically includes: selecting a public chemical reaction dataset as a pre-training dataset, using the dual-view molecular characterization model pre-trained in step 2) to extract molecular feature vectors of reactants, reagents, catalysts, solvents and products in the reaction system respectively; splicing and combining the extracted molecular feature vectors of each component according to the preset reaction rules to construct a reaction feature matrix containing complete reaction context information.

9. The method according to claim 8, characterized in that, Step 4) The pre-training of the chemical reaction prediction model specifically includes: constructing a binary classification task of reaction thermodynamic feasibility as the pre-training target; training a two-layer attention module using the reaction feature sequence; and guiding the model to learn general chemical reaction rules and the mapping features from reactant atoms to product atoms by predicting whether a given combination of reaction components can undergo a chemical reaction, thereby completing the initialization of model parameters.

Images