A method for predicting multi-level organic chemical reactions based on double encoders

Abstract

The application provides a kind of based on double encoder's multilevel organic chemical reaction prediction method, belongs to machine learning assisted organic chemistry technical field.Solve the technical problem that the lack of attention to intermolecular interaction of Transformer model, encoding is insufficient and generalization ability is not strong.The technical scheme: including the following steps: S10, the chemical reaction SMILES sequence data set is pretreated;S20, the atomic encoder and molecular encoder are used to extract features from the input chemical reaction respectively;S30, multilevel feature fusion is used on atomic and molecular feature representation;S40, the loss of model prediction result and target and the loss of contrast learning are added with weight, and the model is trained;S50, the trained model is used to predict chemical product.The beneficial effects of the application are: improve the accuracy of chemical product prediction, improve the generalization ability of the model.

Description

Technical Field

[0001] This invention relates to the field of machine learning-assisted organic chemistry technology, and in particular to a method for predicting multilevel organic chemical reactions based on dual encoders. Background Technology

[0002] In recent years, large language modeling, such as Transformer, has demonstrated outstanding performance in organic chemistry prediction. Chemists can leverage the powerful modeling capabilities of machine learning to predict and discover new types of organic chemical reactions. Compared to traditional recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), Transformer models, through their self-attention mechanism, are better able to capture long-range dependencies and possess stronger feature representation and parallel computing capabilities. While large language models offer efficient data processing capabilities and can improve the accuracy of chemical reaction predictions, their performance is highly dependent on the quality and quantity of training data. Therefore, given limited computational resources and chemical datasets, extracting deeper chemical information to improve model prediction accuracy is an important direction for research improvement.

[0003] Currently, organic chemical reaction prediction methods segment chemical formulas represented by SMILES according to atoms and then pass them through a Transformer model. This only yields attention weights between atoms, ignoring intermolecular interactions. For example, the model proposed in "Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction" is entirely based on an attention mechanism, using the textual representations of reactants, reagents, and products for machine translation. However, the model does not focus on intermolecular information, leading to inaccurate predictions of organic chemical products. This invention uses molecular encoders and atomic encoders to extract intermolecular and atom-based interaction information respectively, and performs multi-level feature fusion to extract deeper features. Furthermore, the paper "Is Bert really robust? A strong baseline for natural language attack on text classification and entailment" points out that machine learning-based natural language processing models are susceptible to small and often imperceptible interferences. Therefore, avoiding the influence of synonyms and non-standardized data, as well as optimizing the extraction of different feature information simultaneously, are the challenges faced by this invention. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-level organic chemical reaction prediction method based on dual encoders. This method uses two encoders to characterize the atoms and molecules in the input chemical formula. The atomic encoder obtains the inter-atomic relationships through a self-attention mechanism, while the molecular encoder averages the atomic embedding information to obtain a characterization of the molecular embedding information, which is then processed through an attention mechanism to obtain the intermolecular interactions. Next, a multi-level feature fusion module fuses the outputs of the atomic and molecular encoders, and the result is input to a decoder. Then, the model's prediction result is dynamically weighted and fused with the loss from the target result and the loss from contrastive learning. This process improves the accuracy of the model in predicting organic chemical products and enhances the model's generalization ability. This invention will provide important reference for organic chemists in experimental design and synthetic pathway planning.

[0005] To achieve the aforementioned objectives, the present invention employs the following technical solution: a method for predicting multi-level organic chemical reactions based on dual encoders, comprising the following steps:

[0006] S10. Read the SMILES training, validation, and test datasets for organic chemical reactions sequentially, and process the reactants and reagents as source sentences x = {x1,...,x...} i ,...,x n The SMILES expression and its product are used as the target sentence y = {y1,...,y} to be predicted by the model. i ,...,y n The SMILES expression is then used. Next, regular expressions are used to process and label the data, followed by data augmentation to obtain more data samples.

[0007] S20. Using atomic encoders and molecular encoders, respectively process the input source sentence x = {x1, ..., x...} i ,...,x n Feature extraction is performed by first inputting the data into an atomic encoder, which converts it into corresponding IDs based on the source dictionary. Then, the data is mapped to X = {X1, ..., X} through an embedding layer. i ,...,X n The scaled X is then added with the location information encoding E. pos become Where pos represents the position, then After L iterations using a multi-head attention mechanism, the correlation matrix X between atoms is obtained. L The molecular encoder is described in the appendix of the instruction manual. Figure 2 For each chemical reaction, the embedding codes of reactants and reagents are averaged to obtain the molecular embedding matrix X′. Then, this matrix is ​​scaled and added to the new positional encoding matrix to obtain the molecular embedding matrix X′. After L iterations of the multi-head attention mechanism, the correlation matrix X′ between molecules is obtained. L ;

[0008] S30, Correlation Matrix between Atoms X L The correlation matrix X′ between molecules L The feature matrix is ​​concatenated along the last dimension, passed through a linear layer and a Softmax activation function to obtain a weight matrix λ. Then, the weight matrix is ​​used for weighted fusion to obtain a multi-level feature fusion matrix X″, which is then input into the decoder.

[0009] S40. Transform the target sentence y = {y1,...,y...} i ,...,y n The input is fed into the decoder, which employs Teacher Forcing and Masking techniques to process the entire sentence's embedding simultaneously during training. It predicts each token in parallel until a "sentence end marker" is generated or the maximum allowed prediction length is reached. A probability distribution is generated across the entire vocabulary space. The predicted and target products are compared using a multi-label smoothed cross-entropy loss function to obtain the loss X. loss The loss of contrastive learning The weights are summed, and the model's weights are optimized through backpropagation.

[0010] S50. Using a trained dual encoder model, predict the products of new organic chemical reactions.

[0011] As a method for predicting multi-level organic chemical reactions using dual encoders provided by the present invention, the specific steps of step S10 are as follows:

[0012] S11. Read the SMILES dataset of organic chemical reactions, retaining reactions without stereochemical information and atomic mappings. Then, following standard neural machine translation steps, use regular expressions to process and label the data. For the normalized dataset, use the regular expression Regex to perform sub-word segmentation on the normalized sequences. The definition of Regex is as follows:

[0013] Regex="(\[[^\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\(|\)|\.|=|#|-|\+|\\\\\ / |:|~|@|\?|>|\*|\$|\%[0-9]{2}|[0-9])"

[0014] For the original dataset, RDkit was used to convert the SMILES string into molecular objects. Then, all atoms in the molecular objects were iterated through. If a molAtomMapNumber attribute existed, the mapping numbers were removed using the Python code atom.ClearProp(“molAtom-MapNumber”), resulting in SMILES characters without atom mappings. Finally, regular expression Regex was used for word segmentation. Reactants and reagents were combined into the source string, with two reactants separated by a "." and reactants and reagents separated by a "". <separated>"Separation;

[0015] S12. Perform data augmentation on the training, testing, and validation datasets respectively. During the augmentation process, filter out non-standard and duplicate data to obtain more data samples.

[0016] As a method for predicting multi-level organic chemical reactions using dual encoders provided by the present invention, the specific steps of step S20 are as follows:

[0017] S21. The processed source sentence and target sentence are converted into corresponding IDs according to the source dictionary and target dictionary, respectively. Then, they are projected onto the D dimension through the embedding layer of the dual encoder to obtain the source embedding X = {X1,...,X...} i ,...,X n } and target embedding Y = {Y1,...,Y i ,...,Y n Then, after scaling, position information encoding E is added. pos The final atom embedding calculation formula is as follows:

[0018]

[0019]

[0020] in To process the source embedding, The processed target embedding, where 'a' is a scaling constant;

[0021] S22. For a Transformer model with dual encoders, The input is fed into the atomic encoder, passes through the multi-head self-attention module, and the weight matrix W is initialized with attention. Q W K W V The weight matrix is ​​dynamically adjusted to ensure that the weights reflect the importance of the samples, resulting in the following Q, K, V matrices:

[0022]

[0023]

[0024]

[0025] Q is the query matrix, K and V are the key and value matrices respectively, and b Q b k b v These are bias vectors, and these parameters are learned by the model during training;

[0026] S23. Finally, the attention matrix A is obtained. j :

[0027]

[0028] Where d k The dimension of the key matrix is ​​represented by the Q, K, and V matrices in the self-attention process, which undergo multiple linear transformations. The attention function is then executed in parallel, and the matrices are concatenated and projected to obtain the final result. The core of multi-head attention is scale-dot-product attention, and then the multi-head attention is concatenated using dot-product attention, calculated using the following formula:

[0029]

[0030] Next A j After level normalization, i.e., LN(A) j The calculation formula is as follows:

[0031]

[0032] LN(A j The model passes through a Dropout layer to prevent overfitting, then a feedforward network containing two linear transformations with a ReLU activation in between. Next, it goes through layer normalization and a Dropout layer. Finally, steps S22 and S23 are repeated L times to obtain the final correlation matrix X between atoms. L ;

[0033] S24. Convert the processed source sentences into corresponding IDs according to the source dictionary. The model obtains the output of the corresponding embedding layer based on the ID of each chemical formula. Then, refer to the appendix of the instruction manual. Figure 2 For each chemical reaction, the embedding codes of reactants and reagents are averaged, and the dot numbers and special symbols are retained to obtain the molecular embedding representation X′={X′1,...,X′ i ,...,X′ n Meanwhile, the padding matrix and position encoding matrix are recalculated, and then the new position encoding E′ is added after scaling. pos The final molecular embedding calculation formula is as follows:

[0034]

[0035] S25, molecular embedding The filling matrix is ​​input into the molecular encoder, and steps S22 to S23 are repeated L times to obtain the correlation matrix X′ between molecules. L ;

[0036] As a method for predicting multi-level organic chemical reactions using dual encoders provided by the present invention, the specific steps of step S30 are as follows:

[0037] S31, Correlation Matrix X between Atoms L The correlation matrix X′ between molecules L The data is concatenated along the last dimension, then passed through a linear layer and a Softmax activation function to obtain the weight matrix λ. This weight matrix is ​​then used for weighted fusion to obtain the multi-level feature fusion matrix X″, which is then input into the decoder. The relevant calculation formulas are as follows:

[0038] L = W·concat(X) L ,X′ L (10)

[0039] λ = softmax(L, dim = 1) (11)

[0040] X″=λX L +(1-λ)X′ L (12)

[0041] Where W is the weight matrix of the linear layer;

[0042] As a method for predicting multi-level organic chemical reactions using dual encoders provided by the present invention, the specific steps of step S40 are as follows:

[0043] S41. The decoder predicts the corresponding target symbol sequence one element at a time. Like the encoder, the decoder also consists of a stack of L identical layers. A third sublayer is inserted, which performs multi-head attention on the output of the encoder stack, utilizing the remaining connections around each sublayer. Then, layer normalization is performed. The model's ability to translate the source sentence x word-for-word into the target sentence y is calculated as follows:

[0044]

[0045] Where w(·) is a linear function that outputs the previously generated word y. i The probability, c i It is the i-th source representation, and the i-th decoded hidden state s i The calculation is as follows:

[0046] s i =f(s) i-1 ,y i-1 ,c i (14)

[0047] S42. The loss is obtained by applying a multi-label smoothed cross-entropy loss function to the predicted product and the target product. The calculation formula is as follows:

[0048]

[0049] Among them, L n It is the negative log-likelihood of the correct category label, L. s This represents the average of the negative log probabilities for all categories. Let ε represent the predicted probability of the i-th sample in the true class, and M represent the number of samples. To prevent overfitting when the model has too few training samples, label smoothing is applied to the multi-class cross-entropy loss function. After label smoothing, the true probability distribution becomes ε:

[0050]

[0051] Where η is a small hyperparameter, K represents the total number of multi-class categories, j represents the target label, and X is the label smoothing multi-class cross-entropy loss. loss as follows:

[0052] X loss =(1-η-ε)L n +εL s (18)

[0053] S43, N represents the number of samples in a batch. After data augmentation by n times, n pairs of positive samples are obtained, resulting in a total of nN samples. The contrastive loss first selects all positive classes for each sample in the batch, ensuring high embedding similarity between positive classes. Secondly, it selects the negative classes for all canonical samples in the batch, ensuring low embedding similarity between negative classes. The contrastive loss... Using InfoNCE loss, the calculation formula is as follows:

[0054]

[0055] Among them, z i z is the normalized sample of the i-th positive class in a batch. j It is a denormalized sample in a batch whose positive class is enhanced by a factor of n, where sim(·) is the cosine similarity, τ is the temperature parameter, and z k It is a normalized sample among the other (nN-1) samples in a batch;

[0056] S44. The loss X is obtained from the cross-entropy loss function for multi-label smoothing. loss Loss of contrastive learning The weights are summed, and the model weights are optimized through backpropagation. The calculation formula is as follows:

[0057]

[0058]

[0059] Among them, E α It is the current Epoch, E total It is the total epoch set, L z It is the final loss function;

[0060] As a prediction method for multi-level organic chemical reactions using dual encoders provided by the present invention, in step S50: the dual encoder model is trained based on steps S10 to S40, and the trained dual encoder model is used to predict the products of organic chemical reactions.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] 1. Deeper Feature Extraction: Traditional organic chemical product prediction methods ignore intermolecular interactions. The dual encoder model uses an atomic encoder and a word segmentation encoder to extract molecular and atomic information respectively. Based on a simple and effective molecular extraction algorithm, the model can simultaneously focus on information between atoms and molecules.

[0063] 2. Multi-level feature fusion: This invention proposes a fusion layer with automatic weight update. It adopts linear splicing to fuse the outputs of atomic and molecular encoders, batch normalization and softmax activation functions to obtain weight parameters, and then based on the fusion of gating units, it enhances the complementary information between features at different levels. Unlike simple connection, this design further extracts complementary information.

[0064] 3. Enhanced generalization ability of the model: The dual encoder model adopts an optimized multi-class cross-entropy loss function. On the basis of the smooth cross-entropy loss function, a contrastive learning loss is added. The two loss functions are weighted and fused by automatically updating the weight parameters. This method can be applied not only to the normalized SMILES chemical dataset, but also to the non-normalized SMILES dataset, thus improving the generalization ability of the model. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0066] Figure 1 This is an overall flowchart of the multi-level method for predicting organic chemical products using dual encoders in this invention.

[0067] Figure 2 This is a flowchart of the algorithm for molecular feature representation of the present invention.

[0068] Figure 3 This is a model framework diagram of the dual encoder multi-level method for predicting organic chemical products according to the present invention.

[0069] Figure 4 This is an example diagram of the multi-level dual encoder method for predicting organic chemical products according to the present invention. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0071] Example 1

[0072] See Figure 1 As shown, this embodiment provides a technical solution for predicting multi-level organic chemical reactions using dual encoders, comprising the following steps:

[0073] S10. Read the SMILES training, validation, and test datasets for organic chemical reactions sequentially, and process the reactants and reagents as source sentences x = {x1,...,x...} i ,...,x n The SMILES expression and its product are used as the target sentence y = {y1,...,y} to be predicted by the model. i ,...,y n The SMILES expression is then used. Next, regular expressions are used to process and label the data, followed by data augmentation to obtain more data samples.

[0074] S20. Using atomic encoders and molecular encoders, respectively process the input source sentence x = {x1, ..., x...} i ,...,x n Feature extraction is performed by first inputting the data into an atomic encoder, which converts it into corresponding IDs based on the source dictionary. Then, the data is mapped to X = {X1, ..., X} through an embedding layer. i ,...,X n The scaled X is then added with the location information encoding E. pos become Where pos represents the position, then After L iterations (L=6) using a multi-head attention mechanism, the correlation matrix X between atoms is obtained. L The molecular encoder is described in the appendix of the instruction manual. Figure 2 For each chemical reaction, the embedding codes of reactants and reagents are averaged to obtain the molecular embedding matrix X′. Then, this matrix is ​​scaled and added to the new positional encoding matrix to obtain the molecular embedding matrix X′. After L iterations of the multi-head attention mechanism, the correlation matrix X′ between molecules is obtained. L ;

[0075] S30, Correlation Matrix between Atoms X L The correlation matrix X′ between molecules L The feature matrix is ​​concatenated along the last dimension, passed through a linear layer and a Softmax activation function to obtain a weight matrix λ. Then, the weight matrix is ​​used for weighted fusion to obtain a multi-level feature fusion matrix X″, which is then input into the decoder.

[0076] S40. Transform the target sentence y = {y1,...,y...} i ,...,y n The input is fed into the decoder, which employs Teacher Forcing and Masking techniques to process the entire sentence's embedding simultaneously during training. It predicts each token in parallel until a "sentence end marker" is generated or the maximum allowed prediction length is reached. A probability distribution is generated across the entire vocabulary space. The predicted and target products are compared using a multi-label smoothed cross-entropy loss function to obtain the loss X. loss The loss of contrastive learning The weights are summed, and the model's weights are optimized through backpropagation.

[0077] S50. Using a trained dual encoder model, predict the products of new organic chemical reactions.

[0078] The specific steps of step S10 are as follows:

[0079] S11. This example uses the USPTO-Jin training, validation, and testing dataset, which is a publicly available chemical reaction dataset mined from US patents granted between 1976 and September 2016. A total of 479,035 reactions without stereochemical information and atom mappings are retained. First, the SMILES strings of the chemical reactions in the training set are read, and the source and target datasets are divided according to ">>". Then, RDkit is used to convert the SMILES strings in the source dataset into molecular objects. Next, all atoms in the molecular objects are traversed. If there is a molAtomMapNumber attribute, these mapping numbers are removed using the Python code at m.ClearPr-op("molAtomMapNumber") to obtain the SMILES characters without atom mappings. Finally, regular expression Regex is used for word segmentation. Reactants and reagents are combined into a source string, with reactants separated by "." and reagents separated by "". <separated>The separator, Regex, is defined as follows:

[0080] Regex="(\[[^\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\(|\)|\.|=|#|-|\+|\\\\\ / |:|~|@|\?|>|\*|\$|\%[0-9]{2}|[0-9])"

[0081] Since the original USPTO-Jin dataset contains only single products, after removing the atom mapping numbers, we use regular expressions (Regex) for word segmentation to obtain the final target dataset. The data processing methods for the validation set and the test set are the same.

[0082] S12. Perform root alignment data augmentation on the original datasets for training, validation and testing of USPTO-Jin respectively. During the augmentation process, filter out data that cannot be normalized and duplicates. An example of root alignment data augmentation is shown in Table 1.

[0083] Table 1. Steps to enhance root alignment data.

[0084]

[0085] In Table 1, the root atoms are in bold. First, a reaction is selected from the dataset, and an atom is randomly selected as the root atom. [CH:5] and C are selected here, resulting in reactants with specified root atoms. Then, the atom mapping is removed to obtain the final input. Next, the atom mapping that appears on the product SMILES is searched from left to right from the reactant SMILES. Once found, the atom is selected as the root of the reactant. [C:7] is selected here. Without performing atom mapping, the root atom SMILES representation of the product is obtained. Finally, the reactants and products are segmented.

[0086] The USPTO-Jin dataset was augmented by a factor of two using root-aligned data augmentation, resulting in the USPTO-Jin_R dataset. Details of both datasets are shown in Table 2.

[0087] Table 2 shows the data statistics for the two datasets.

[0088]

[0089] The specific steps of step S20 are as follows:

[0090] S21. The source and target sentences from the processed USPTO-Jin_R dataset are transformed into corresponding IDs based on the source and target dictionaries, respectively. The source dictionary contains 304 entries. After passing through the embedding layer of the dual encoder, the embeddings are projected to 512 dimensions to obtain the source embedding X = {X1,...,X...}. i ,...,X n } and target embedding Y = {Y1,...,Y i ,...,Y n Then, after scaling, position information encoding E is added. pos The final atom embedding calculation formula is as follows:

[0091]

[0092]

[0093] in To process the source embedding, For the processed target embedding, the value of a is 22.627416997969522;

[0094] S22. For a Transformer model with dual encoders, The input is fed into the atomic encoder, passes through the multi-head self-attention module, and the weight matrix W is initialized with attention. Q W K W V The weight matrix is ​​dynamically adjusted to ensure that the weights reflect the importance of the samples, resulting in the following Q, K, V matrices:

[0095]

[0096]

[0097]

[0098] Q is the query matrix, K and V are the key and value matrices respectively, and b Q b k b v These are bias vectors, and these parameters are learned by the model during training;

[0099] S23. Finally, the attention matrix A is obtained. j :

[0100]

[0101] Where d k The dimension of the key matrix is ​​represented by the Q, K, and V matrices in the self-attention process, which undergo multiple linear transformations. The attention function is then executed in parallel, and the matrices are concatenated and projected to obtain the final result. The core of multi-head attention is scale-dot-product attention, and then the multi-head attention is concatenated using dot-product attention, calculated using the following formula:

[0102]

[0103] Next A j After level normalization, i.e., LN(A) j The calculation formula is as follows:

[0104]

[0105] LN(A j After Dropout to prevent overfitting, the model passes through a feedforward network containing two linear transformations and a ReLU activation in between. This is followed by layer normalization and Dropout. Finally, steps S22 and S23 are repeated six times to obtain the final correlation matrix X between atoms. L At this point, the batch size is 36, and the maximum length of the SMILS string is 48. Specifically, X... L The Tensor values ​​are shown below:

[0106]

[0107] S24. Convert the processed source sentences into corresponding IDs according to the source dictionary. The model obtains the output of the corresponding embedding layer based on the ID of each chemical formula. Then, refer to the appendix of the instruction manual. Figure 2 For each chemical reaction, the embedding codes of reactants and reagents are averaged, and the dot numbers and special symbols are retained to obtain the molecular embedding representation X′={X′1,...,X′ i ,...,X′ n Meanwhile, the padding matrix and position encoding matrix are recalculated, and then the new position encoding E′ is added after scaling. pos At this point, the value of a is also 22.627416997969522, and the final molecular embedding calculation formula is as follows:

[0108]

[0109] S25, molecular embedding The filling matrix is ​​input into the molecular encoder, and steps S22 to S23 are repeated 6 times to obtain the correlation matrix X′ between molecules. L , specific X′ L The Tensor values ​​are shown below;

[0110]

[0111] The specific steps of step S30 are as follows:

[0112] S31, Correlation Matrix X between Atoms L The correlation matrix X′ between molecules L The layers are concatenated along the last dimension, then passed through a linear layer and a Softmax activation function to obtain the weight matrix λ. The specific values ​​of the Tensor for λ are as follows:

[0113]

[0114] Next, a weighted fusion is performed using a weight matrix to obtain a multi-level feature fusion matrix X″, which is then input into the decoder. The relevant calculation formulas are as follows:

[0115] L = W·concat(X) L ,X′ L (10)

[0116] λ = softmax(L, dim = 1) (11)

[0117] X″=λX L +(1-λ)X′ L (12)

[0118] Where W is the weight matrix of the linear layer, as shown below:

[0119]

[0120] The final Tensor value of X″ is as follows:

[0121]

[0122] The specific steps of step S40 are as follows:

[0123] S41. The decoder predicts the corresponding target symbol sequence one element at a time. Like the encoder, the decoder also consists of a stack of L identical layers. A third sublayer is inserted, which performs multi-head attention on the output of the encoder stack, utilizing the remaining connections around each sublayer. Then, layer normalization is performed. The model's ability to translate the source sentence x word-for-word into the target sentence y is calculated as follows:

[0124]

[0125] Where w(·) is a linear function that outputs the previously generated word y. i The probability, c i It is the i-th source representation, and the i-th decoded hidden state s i The calculation is as follows:

[0126] s i =f(s) i-1 ,y i-1 ,c i (14)

[0127] S42. The loss is obtained by applying a multi-label smoothed cross-entropy loss function to the predicted product and the target product. The calculation formula is as follows:

[0128]

[0129]

[0130] Among them, L n It is the negative log-likelihood of the correct category label, L. s p represents the average of the negative log probabilities for all categories. yi Let represent the predicted probability of the i-th sample in the true class, and M represent the number of samples. The USPTO-Jin_R training set has 817,446 samples. To prevent overfitting when the training samples are too few, label smoothing is applied to the multi-class cross-entropy loss function. After label smoothing, the true probability distribution becomes ε:

[0131]

[0132] Where η is a small hyperparameter, η is set to 0.1, K represents the total number of multi-class categories, j represents the target label, and X is the label smoothing multi-class cross-entropy loss. loss as follows:

[0133] X loss =(1-η-ε)L n +εL s (18)

[0134] At this time

[0135] S43 and N represent the number of samples in a batch. With N=36, after data augmentation by 1x, n=1, resulting in 2 pairs of positive samples. A batch contains 72 samples. The contrastive loss first selects all positive classes for each sample in the batch, ensuring high embedding similarity between positive classes. Secondly, it selects the negative classes for all canonical samples in the batch, ensuring low embedding similarity between negative classes. The contrastive loss... Using InfoNCE loss, the calculation formula is as follows:

[0136]

[0137] Among them, z i z is the normalized sample of the i-th positive class in a batch. j It is a denormalized sample in a batch whose positive class is enhanced by a factor of 1, where sim(·) is the cosine similarity, τ is the temperature parameter, and z k It is a normalized sample from the other 71 samples in a batch;

[0138] S44. The loss X is obtained from the cross-entropy loss function for multi-label smoothing. loss Loss of contrastive learning The weights are summed, and the model weights are optimized through backpropagation. The calculation formula is as follows:

[0139]

[0140] Among them, E α The current Epoch is 1, E total The total epochs are set to 500, L z It is the final loss function;

[0141] In step S50: The dual encoder model is trained based on steps S10 to S40. The model uses the Adam optimizer with an initial learning rate of 5e-4, which is automatically updated. Dropout is set to 0.3, and label_smoothing is set to 0.1. The trained dual encoder model is then used to predict the products of organic chemical reactions. The prediction accuracy is better than the baseline model and previous work, as shown in Tables 3 and 4.

[0142] Table 3 shows the test results of the model on the two datasets.

[0143]

[0144] Table 4 Comparison of related works on the USPTO_Jin dataset

[0145]

[0146] Example 2

[0147] USPTO-FULL is a larger dataset of chemical reactions, consisting of approximately 1,000,000. Using root-aligned data augmentation, the training, validation, and test sets of the USPTO-FULL dataset are augmented by a factor of 5, resulting in the diversified USPTO-FULL_5R dataset.

[0148] After S12 processing, a total of 4,803,620 chemical reaction datasets were obtained, including 3,843,150 training sets, 480,355 validation sets, and 480,115 test sets.

[0149] The hyperparameters for this experiment are set as follows: batch size is 128, the initial learning rate is 5e-4 using a learning rate scheduler based on the inverse square root, dropout is set to 0.3, and weight decay is 0.0001. The Adam optimizer is used in the loss function, with beta parameters of "0.9, 0.98".

[0150] To evaluate the proposed method, the model evaluation metrics are BLEU and accuracy (ACC), which are calculated using the following formulas:

[0151]

[0152] Wherein, the human translation is represented as s i,j M represents the total number of reference answers, and the translated text is represented as c. i E represents the total number of translations, d represents the d-th phrase, and h d (c i ) is the dth phrase in the translated text c i The number of times h appears in d (s i,j ) is the dth phrase in the standard answer S i,j The number of times it appears in the text.

[0153]

[0154] Where BP (Brevity Penalty) is the length penalty factor, Y C Y represents the length of the machine-translated text. S This indicates the length of the reference answer. When multiple reference translations exist, the length closest to the translated answer is selected. When the translated answer is longer than the reference translation, the penalty coefficient is 1, meaning no penalty is applied; a penalty factor is calculated only when the translated answer is less than or equal to the reference translation. The final formula for BLEU is shown below:

[0155]

[0156] Where Wn = 1 / n, the maximum accuracy is 4-gram, and the beam search used is 5. The accuracy of the model test results is calculated for TOP-1, TOP-3 and TOP-5 respectively. The accuracy calculation formula is as follows.

[0157]

[0158] For each reaction, the model predicts 5 products, where u represents the sum of the model's prediction accuracy in TOP-n.

[0159] Using the trained dual-encoder model, the product prediction accuracy for organic chemical reactions was better than that of the baseline model, as shown in Table 5.

[0160] Table 5 shows the test results of the model on the USPTO-FULL_5R dataset.

[0161]

[0162] Example 3

[0163] In Example 3, after Schwaller et al. removed duplicate and non-normalized reactions, a dataset of 1,088,170 reactions with only a single product was retained, which is called the USPTO-S dataset. Details are shown in Table 6.

[0164] Table 6 Data Statistics of the Dataset

[0165]

[0166] The experimental parameters for Example 3 were the same as those for Example 2, and the experimental comparison results are shown in Table 7:

[0167] Table 7 Comparison of related works on the USPTO_S dataset

[0168]

[0169] On the USPTO-S dataset, the model proposed in this embodiment outperformed the baseline model by 0.47, 0.43, and 0.72 points in Top-1, Top-3, and Top-5, respectively. This indicates that the model learned atomic feature information and the interactions between molecules, extracted richer information, and improved the accuracy of the model's product prediction.

[0170] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / separated> < / separated>

Claims

1. A method for predicting a multi-level organic chemical reaction based on dual-encoder, characterized in that, The method comprises the following steps: S10. Read the SMILES training, validation, and test datasets for organic chemical reactions sequentially, and process the reactants and reagents as source sentences x = {x1,...,x...} i ,...,x n The SMILES expression and its product are used as the target sentence y = {y1,...,y} to be predicted by the model. i ,...,y n The SMILES expression is then used, followed by regular expression processing and tagging of the data, and then data augmentation to obtain more data samples. S20, the input source sentence x = {x1,..., x i ,...,x n} is respectively extracted by atomic encoder and molecular encoder, first input into the atomic encoder to be converted into corresponding ID according to the source dictionary, then mapped into X = {X1,..., X i ,...,X n} through Embedding layer, and the scaled X is added with position information coding E pos to become wherein pos represents position, then go through L times of multi-head attention mechanism to obtain the correlation matrix X L between atoms, the molecular encoder averages the embedding coding of each chemical reaction reactant and reagent to obtain the molecular embedding matrix X', and then scales and adds the new position coding matrix to become go through L times of multi-head attention mechanism to obtain the correlation matrix X' between molecules L ; S30, the correlation matrix X between atoms L and the correlation matrix X' between molecules L Concatenate along the last dimension, pass through a linear layer and a Softmax activation function to get the weight matrix λ, then use the weight matrix for weighted fusion to get the multi-level feature fusion matrix X", and then input it to the decoder; S40, input the target sentence y = {y1,..., y i ,...,y n} into the decoder, use the Teacher Forcing technology and the Mask mechanism, so that the decoder processes the entire sentence Embedding at the same time during the training stage, predicts each molecular token in parallel, until the "sentence end token” is generated or the maximum length allowed for prediction is reached, and generates a probability distribution in the entire vocabulary space, the loss X loss is obtained by the cross-entropy loss function of the prediction product and the target product after multi-label smoothing , and the loss of the contrast learning The weights are added, and the weights of the model are optimized by back propagation. S50, using the trained double-encoder model to predict the product of the new organic chemical reaction.

2. The method of claim 1, wherein the method is a method of predicting a multi-level organic chemical reaction based on dual encoders. The specific steps of the step S10 are as follows: S11, reading the organic chemical reaction SMILES data set, retaining the reactions without stereochemical information and atomic mapping, then following the standard neural machine translation steps, using regular expressions to process and mark the data, for the normalized data set, using regular expressions Regex to perform sub-word segmentation on the normalized sequence, the definition of Regex is as follows: Regex = "(\[[^\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\(|\)|\.|=|#|-|\+|\\\\\ / |:|~|@|\?|>|\*|\$|\%[0-9]{2}|[0-9])" For the original dataset, the SMILES string is converted to a molecule object using RDkit, then all the atom mapping numbers in the molecule object are removed using the Python code atom.ClearProp, resulting in a SMILES character without atom mapping, and finally, the source string is combined by the reactants and reagents using regular expression Regex, and the two reactants are separated by ".” and the reactants and reagents are separated by " <separated>Separate;< / separated> S12, data augmentation is performed on the training, testing and validation data sets respectively, and the data that cannot be standardized and repeated is filtered during the augmentation process, so as to obtain more data samples.

3. The method for predicting multi-level organic chemical reactions based on dual encoders according to claim 1, characterized in that, The specific steps of the step S20 are as follows: S21, the processed source sentence and target sentence are respectively converted into corresponding IDs according to the source dictionary and the target dictionary, and then projected to D dimensions through the Embedding layer of the double encoder to obtain source embedding X = {X1,..., X i ,...,X n} and target embedding Y = {Y1,..., Y i ,...,Y n}, followed by scaling and adding position information coding E pos , to obtain the final atomic embedding. The calculation formula is as follows: wherein is the processed source embedding, is the processed target embedding, a is a scaling constant; S22、For the Transformer model with double encoders, the input is input to the atomic encoder, passed through the multi-head self-attention module, and the attention initialization weight matrix W is obtained Q , W K , W V , and the weight matrix is dynamically adjusted so that the weight reflects the importance of the sample, and finally the Q, K, and V matrices are obtained as follows: Q is the query matrix, K and V are the key and value matrices, respectively, b Q , b k , b v is the bias vector, and these parameters are learned by the model during training; S23, the attention matrix A is finally obtained j : where d k denotes the dimension of the key matrix, Q, K, V matrix in self-attention is linearly changed for multiple times, K T is to transpose the K matrix, then execute the attention function in parallel, and connect and project them again to get the final result, the core of multi-head attention is scale dot product attention, then the multi-head attention is concatenated through dot product attention, and the calculation formula is as follows: where W O is the weight matrix of the attention mechanism, and then A j is layer normalized, LN(A j ), using the following formula: LN(A j ) is passed through a Dropout layer to prevent overfitting of the model, then through a feedforward network comprising two linear transformations with a ReLU activation in between, followed by layer normalization and a Dropout layer, and the steps S22 to S23 are repeated L times to obtain the final inter-atomic correlation matrix X L ; S24. The processed source sentences are converted into corresponding IDs according to the source dictionary. The model obtains the output of the corresponding embedding layer based on the ID of each chemical formula. Then, according to Figure 2, the embedding codes of reactants and reagents in each chemical reaction are averaged, and the dot numbers and special symbols are retained to obtain the molecular embedding representation X′={X′1,...,X′ i ,...,X′ n Meanwhile, the padding matrix and position encoding matrix are recalculated, and then the new position encoding E′ is added after scaling. pos The final molecular embedding calculation formula is as follows: S25, molecular embedding and the filled matrix is input to the molecular encoder, S22 to S23 steps are repeated L times to obtain the correlation matrix X' between molecules L .

4. The method of claim 1, wherein the method is a method of predicting a multi-level organic chemical reaction based on dual encoders. The specific steps of the step S30 are as follows: S31, the correlation matrix X between atoms L and the correlation matrix X between molecules L ′Concatenate along the last dimension, pass through a linear layer and a Softmax activation function to get the weight matrix λ, then use the weight matrix to weight and fuse to get the multi-level feature fusion matrix X", and then input to decoding, the correlation calculation formula is as follows: L = W • concat(X L L (10)​ λ = softmax (L, dim = 1) (11) X" = λX L + (1 - λ)X' L (12) Wherein, W is the weight matrix of the linear layer.

5. The method of claim 1, wherein the method is a method of predicting a multi-level organic chemical reaction based on dual encoders, characterized in that, The specific steps of the step S40 are as follows: S41, the decoder predicts the corresponding target symbol sequence by one element at a time, like the encoder, the decoder is also composed of a stack of L same layers, and a third sub-layer is inserted, which performs multi-head attention on the output of the encoder stack, utilizes the remaining connections around each sub-layer, and then performs layer normalization, the ability of the model to translate from the source sentence x to the target sentence y word by word, and the calculation formula is as follows: where w(·) is a linear function that outputs the previously generated word y i the probability of the i-th source representation, c i is the i-th decoded hidden state s i is computed as: s i = f(s i-1 ,y i-1 ,c i ) (14) S42, the loss is obtained by the multi-label smoothed cross-entropy loss function of the predicted product and the target product, and the calculation formula is as follows: where L n is the negative log-likelihood of calculating the correct class label, L s represents the average value of calculating the negative log probability of all categories, p yi represents the predicted probability of the i-th sample on the real category, M represents the number of samples, in order to prevent the model from overfitting when the number of training samples is too small, the label smoothing is carried out on the basis of the multi-class cross-entropy loss function, and the real probability distribution after the label smoothing is increased to ε: wherein η is a small hyper-parameter, K represents the total number of multi-classification categories, j represents the target label, and the label-smoothed multi-classification cross-entropy loss X loss is as follows: X loss = (1 - η - ε) L n + ε L s (18) S43, N represents the number of samples in a batch, through data enhancement n times, n pairs of positive sample pairs are obtained, at this time nN samples are used, the contrast loss first selects all positive classes of each sample in the batch, so that the embedding similarity between each sample positive class is high, secondly, the negative class of all standard samples in the batch is selected, so that the embedding similarity between the negative classes of the standard samples is low, and the contrast loss Using InfoNCE loss, the calculation formula is as follows: where z i is the normalized sample of the i-th positive class in a batch, z j is the non-normalized sample of the i-th positive class augmented by n times in a batch, sim(·) is the cosine similarity, τ is the temperature parameter, z k is the normalized sample of the other (nN-1) samples in a batch; S44, the multi-label smoothed cross-entropy loss function obtains a loss X loss Loss of contrast learning The weights are added, and the weights of the optimization model are back-propagated. The calculation formula is as follows: where E α is the current Epoch, E total is the total Epochs set, L z is the final loss function.

6. The method of claim 1, wherein the method is a method of predicting a multi-level organic chemical reaction based on dual encoders. In the step S50, based on the steps S10 to S40, the double-encoder model is trained, and the trained double-encoder model is used to predict the product of the organic chemical reaction.

Images