Molecular representation and drug screening method based on transformer multi-translation model
By employing a molecular characterization method based on the Transformer multi-translation model, combined with various traditional molecular descriptor encodings, molecular characterizations are generated and a machine learning classifier is built. This solves the problem of complex molecular characterization acquisition in existing technologies, achieving high efficiency and accuracy in drug screening, and is suitable for drug development.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HAINAN UNIV
- Filing Date
- 2024-06-06
- Publication Date
- 2026-07-07
AI Technical Summary
Existing deep learning-based molecular characterization methods suffer from problems such as a single learning mode or complex molecular characterization acquisition methods, making it difficult to quickly and accurately represent molecules in drug development, resulting in low drug screening efficiency.
A molecular characterization method based on the Transformer multi-translation model is adopted. By combining multiple traditional molecular descriptor encodings and training the Transformer multi-translation model, molecular characterizations are generated. A machine learning classifier is then built for drug screening, which simplifies the method of obtaining molecular characterizations and improves the screening accuracy.
It enables the rapid generation of molecular characterizations without relying on additional data, improving the characterization capabilities of molecular characterizations and the accuracy of drug screening. It can quickly screen for potentially effective drugs and is suitable for the drug development process.
Smart Images
Figure CN118471372B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of machine learning technology, and in particular to a molecular characterization and drug screening method based on the Transformer multi-translation model. Background Technology
[0002] Drug development is a research process with long research cycles, high financial investment, and high risk of returns. Although the emergence of combinatorial chemistry and the rapid development of high-throughput screening technology have greatly accelerated the drug development process, high-throughput screening technology still requires a lot of time and economic costs when faced with millions of compounds. Over the decades, high-throughput experiments have accumulated a massive amount of biochemical data, making artificial intelligence-based virtual drug screening possible.
[0003] In virtual drug screening, a crucial step is effectively representing molecules. Before the advent of artificial intelligence, traditional molecular descriptors, such as SMILES (Simple Linear Input Canonical), InChI (International Compound Identifier), and fixed-length binary molecular fingerprints, were designed based on the structure or functional characteristics of molecules using different encoding methods. However, these traditional descriptors rely on the experience of human experts, are subjective, and are difficult to generalize. Deep learning networks, on the other hand, can autonomously learn to generate continuous molecular representations, reducing subjective intervention by humans in the design process. Combining deep neural networks with existing traditional molecular descriptors is currently the mainstream molecular representation learning method. However, many scholars have found that the potential of combining traditional molecular descriptors and deep neural networks remains to be explored. Comparing deep learning molecular representation methods with traditional methods, it has been found that the performance of deep learning molecular representation is not significantly better than that of traditional molecular descriptors. Some researchers have performed one-to-one transformation or reconstruction of traditional molecular descriptors, storing the information in the descriptors in fixed-length vectors and using them as molecular representations. However, these learning methods involve too many features, resulting in limited molecular representation capabilities, thus requiring more varied learning methods. Some researchers have synthesized a molecular representation by fusing multiple traditional molecular descriptors, or by obtaining the corresponding molecular representation through multi-task reconstruction. However, this many-to-one or many-to-many molecular representation learning method is highly dependent on the acquisition of traditional molecular descriptors. If the relevant properties or descriptors of certain molecules are difficult to obtain, it is impossible to successfully generate the ideal molecular representation.
[0004] Existing deep learning-based molecular characterization methods suffer from problems such as limited learning modes or complex methods of obtaining molecular representations. A good molecular characterization method should not only possess good characterization performance but also be easily accessible to all molecules. Summary of the Invention
[0005] To address the problems of existing deep learning-based molecular characterization methods, such as single learning modes or complex molecular characterization acquisition methods, this invention provides a molecular characterization and drug screening method based on the Transformer multi-translation model. By deeply mining the information in traditional molecular descriptors, a molecular characterization method is designed that can comprehensively cover the chemical space and has a simple molecular characterization acquisition method. Furthermore, drug screening using this molecular characterization method has a high screening accuracy, which can accelerate drug development.
[0006] The molecular characterization and drug screening method based on the Transformer multi-translation model proposed in this invention specifically includes the following steps:
[0007] S1: Download small molecule compounds from the PubChem database and perform molecular preprocessing on all small molecule compounds to obtain the molecular descriptor encoding set for each small molecule compound;
[0008] The molecular descriptor encoding set includes SMILES sequence encoding, 2D pharmacophore molecular fingerprint effective bit encoding, InChl sequence encoding, and PubChem molecular fingerprint effective bit encoding;
[0009] S2: Build a Transformer multi-translation model and train the Transformer multi-translation model using the molecular descriptor encoding set of each small molecule compound to obtain a trained Transformer multi-translation model. The Transformer multi-translation model includes an encoding structure and a decoding structure.
[0010] S3: Obtain small molecules with known activities for specific targets from the ChEMBL database, input the SMILE of each small molecule into the trained Transformer multi-translation model for processing, and obtain molecular representations corresponding to each small molecule after pooling the output of the encoding structure. Use each small molecule, the activity data of each small molecule, and the molecular representations corresponding to each small molecule to construct a small molecule dataset, and divide the small molecule dataset to obtain training set, validation set and test set.
[0011] S4: Build a machine learning classifier, and use the training set, test set and validation set to train the machine learning classifier and search for hyperparameters to obtain the final machine learning classifier.
[0012] S5: Input the SMILE of the drug to be tested into the trained Transformer multi-translation model to obtain the molecular characterization of the drug to be tested, and input the molecular characterization of the drug to be tested into the final machine learning classifier for processing to obtain the screening results of the drug to be tested.
[0013] Preferably, step S1 specifically includes the following steps:
[0014] S11: Remove small molecule compounds downloaded from the PubChem database that have a molecular weight of 12 to 600, a LogP value of -7 to 5, and a heavy atom number of 3 to 50.
[0015] S12: Use RDKit software to characterize the remaining small molecule compounds in step S11 as SMILES, and remove the small molecule compounds that cannot be characterized to obtain a small molecule compound dataset.
[0016] S13: Use RDKit and PyBioMed software to obtain the SMILES, 2D pharmacophore fingerprints, PubChem fingerprints and InChI sequences of all small molecule compounds in the small molecule compound dataset, and obtain the valid bits of the 2D pharmacophore fingerprint and the valid bits of the PubChem fingerprint according to the corresponding 2D pharmacophore fingerprint and PubChem fingerprint.
[0017] S14: Segment the SMILES, 2D pharmacophore fingerprint valid bits, PubChem fingerprint valid bits, and InChI sequences of all small molecule compounds included in the small molecule compound dataset. Then, based on the frequency of each word, encode the words in the SMILES, 2D pharmacophore fingerprint, PubChem fingerprint, and InChI sequences to obtain the SMILES sequence code, 2D pharmacophore molecular fingerprint valid bit code, InChI sequence code, and PubChem molecular fingerprint valid bit code.
[0018] Preferably, the encoding structure includes an embedding layer, a positional encoding layer, and two cascaded first attention mechanism modules. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing to obtain feature A1. Feature A1 is input into the two cascaded first attention mechanism modules for encoding processing, and the features obtained after encoding processing are pooled to obtain the molecular characterization of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character.
[0019] The first attention mechanism module includes a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. Feature A1 is linearly transformed to obtain a first Q matrix, a first K matrix, and a first V matrix. These matrices are then input into the multi-head attention mechanism for processing to obtain feature A2. Features A1 and A2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A3. Feature A3 is then input into the feedforward network for processing to obtain feature A4. Finally, features A3 and A4 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A5.
[0020] Preferably, the decoding structure includes a first decoder, a second decoder, a third decoder, and a fourth decoder, and the network structures of each decoder are the same. The first decoder includes an embedding layer, a positional encoding layer, two cascaded second attention mechanism modules, a linear layer, and a softmax function layer. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing to obtain feature B1. Feature B1 is input into the two cascaded second attention mechanism modules for processing. The processed features are sequentially input into the linear layer and the softmax function layer for prediction and mapping processing to obtain the word segmentation of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character.
[0021] The second attention mechanism module includes a mask multi-head attention mechanism, a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. It performs a linear transformation on feature B1 to obtain the second Q matrix, the second K matrix, and the second V matrix. These matrices are then input into the mask multi-head attention mechanism for processing to obtain feature B2. Features B1 and B2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature B3. Finally, the features obtained from the encoding processing of the two cascaded first attention mechanism modules are... Linear transformation is performed to obtain the third K matrix and the third V matrix. Feature B3 is then linearly transformed to obtain the third Q matrix. The third Q matrix, the third K matrix, and the third V matrix are input into a multi-head attention mechanism for processing to obtain feature B4. Features B4 and B3 are then sequentially input into a residual block and a regularization layer for residual and regularization processing to obtain feature B5. Feature B5 is then input into a feedforward neural network for processing to obtain feature B6. Features B5 and B6 are then sequentially input into a residual block and a regularization layer for residual and regularization processing to obtain feature B7.
[0022] Preferably, the input to the first decoder is the SMILES sequence code after adding the start character; the input to the second decoder is the effective bit code of the 2D pharmacophore molecular fingerprint after adding the start character; the input to the third decoder is the InChl sequence code after adding the start character; and the input to the fourth decoder is the effective bit code of the PubChem molecular fingerprint after adding the start character.
[0023] Preferably, in the encoded SMILES and encoded InChI sequences, atoms composed of two characters are treated as a single word unit during word segmentation; and in the encoded InChI sequence, the InChI=1S / segment representing the version number is treated as a single word unit during word segmentation.
[0024] Preferably, the network feature dimension of the Transformer multi-translation model is 256. Molecular descriptor codes with a length greater than 256 are truncated, and PAD characters are used to complete the molecular descriptor codes with a length less than 256, so that the length of each molecular descriptor code is equal to 256.
[0025] Preferably, the encoding structure and each decoder of the Transformer multi-translation model are trained using the cross-entropy loss function, and the sum of each cross-entropy loss function is used as the total loss function of the Transformer multi-translation model.
[0026] Cross-entropy loss function :
[0027] ;
[0028] Where p(x) represents the true probability distribution and q(x) is the predicted probability distribution. For the i-th sample, It is the cross-entropy function;
[0029] Total loss function Loss:
[0030] Loss=a×L1+b×L2+c×L3+d×L4;
[0031] Where a, b, c, and d are adjustable custom coefficients, and L i Let be the loss function of the i-th decoder.
[0032] Preferably, the total number of small molecule compounds participating in the training is no less than three million.
[0033] Compared with the prior art, the present invention can achieve the following beneficial effects:
[0034] This invention can rapidly generate molecular characterizations of all molecules without providing any additional data other than SMILES. The method of obtaining these characterizations is simple, and the molecular characterization method of this invention can achieve good results in related molecular property prediction tasks. That is, the molecular characterizations obtained by this method have strong characterization capabilities.
[0035] The molecular representation generated by this invention can mine information from SMILES in multiple ways. Each decoder in the Transformer multi-translation model attempts to extract useful information from the intermediate vector output by the encoding structure to complete its respective translation task. This requires that the intermediate vector contain as much molecular semantic information as possible to meet the different needs of multiple decoders. Furthermore, the task of the first decoding structure is to reconstruct the SMILES itself. Based on the principle of "reconstruction equals accuracy," different decoders can ensure the integrity of the information contained in the SMILES while mining the information they are interested in. This allows each decoder to learn grammatical features while learning semantic information. For targets with limited existing drug data, this molecular representation can be used for drug screening on machine learning classifiers with fewer parameters, which can quickly identify potential effective drugs. Attached Figure Description
[0036] Figure 1 This is a flowchart illustrating a molecular characterization and drug screening method based on a Transformer multi-translation model provided in an embodiment of the present invention.
[0037] Figure 2 This is a schematic diagram of the network structure of the Transformer multi-translation model provided in an embodiment of the present invention;
[0038] Figure 3 This is a performance comparison chart of molecular representation based on the Transformer multi-translation model, traditional molecular description amplitude, and deep learning molecular representation according to embodiments of the present invention.
[0039] Figure 4 This is a performance comparison chart of molecular characterization based on the Transformer multi-translation model and the end-to-end deep learning molecular property prediction model provided by an embodiment of the present invention. Detailed Implementation
[0040] In the following description, embodiments of the invention will be described with reference to the accompanying drawings. In the description below, the same modules are denoted by the same reference numerals. Where the same reference numerals are used, their names and functions are also the same. Therefore, their detailed description will not be repeated.
[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.
[0042] The molecular characterization and drug screening method based on the Transformer multi-translation model proposed in this invention can deeply mine the traditional descriptor information of small molecule compounds. Furthermore, by training a machine learning classifier and using the machine learning classifier for virtual drug screening, the screening accuracy can be greatly improved, and potential effective drugs can be screened out.
[0043] Figure 1 The flowchart of a molecular characterization and drug screening method based on a Transformer multi-translation model provided in an embodiment of the present invention is shown.
[0044] like Figure 1 As shown, the molecular characterization and drug screening method based on the Transformer multi-translation model proposed in this invention specifically includes the following steps:
[0045] S1: Download small molecule compounds from the PubChem database (an existing database), and perform molecular preprocessing on all small molecule compounds to obtain the molecular descriptor encoding set for each small molecule compound;
[0046] The molecular descriptor encoding set includes SMILES sequence encoding, 2D pharmacophore molecular fingerprint valid bit encoding, InChl (International Compound Identifier) sequence encoding, and PubChem molecular fingerprint valid bit encoding;
[0047] Step S1 specifically includes the following steps:
[0048] S11: Remove small molecule compounds downloaded from the PubChem database that have a molecular weight between 12 and 600, a LogP value between -7 and 5, and a heavy atom number between 3 and 15.
[0049] The total number of small molecule compounds participating in the training is no less than three million.
[0050] S12: Using RDKit software (RDKit software is a collection of cheminformatics and machine learning software written in C++ and Python), the remaining small molecule compounds in step S11 are characterized as SMILES, and the small molecule compounds that cannot be characterized are removed to obtain a small molecule compound dataset.
[0051] S13: Using RDKit and PyBioMed software (the best software for calculating descriptors of small molecules, proteins, DNA and their interactions in the Python environment), obtain the SMILES, 2D pharmacophore fingerprints, PubChem fingerprints and InChI sequences of all small molecule compounds in the small molecule compound dataset, and obtain the valid bits of the 2D pharmacophore fingerprint and the valid bits of the PubChem fingerprint according to the corresponding 2D pharmacophore fingerprint and PubChem fingerprint.
[0052] S14: Segment the SMILES, 2D pharmacophore fingerprint valid bits, PubChem fingerprint valid bits, and InChI sequences of all small molecule compounds included in the small molecule compound dataset. Then, based on the frequency of each word, encode the words in the SMILES, 2D pharmacophore fingerprint, PubChem fingerprint, and InChI sequences to obtain the SMILES sequence code, 2D pharmacophore molecular fingerprint valid bit code, InChI sequence code, and PubChem molecular fingerprint valid bit code.
[0053] Assume the molecular fingerprint is 010001; that is, the molecular fingerprint has 6 bits (the first bit is counted from 0), and its valid bits are 1 and 5.
[0054] For example, if the sequence of the 2D pharmacophore molecular fingerprint of a molecule is "011000011010000010000010000...", then the effective bit sequence is "1, 2, 7, 8, 10, 16, 22...".
[0055] By preserving the effective bits of the molecular fingerprint, the length of the molecular fingerprint can be compressed, making it non-sparse, reducing the length difference between the input and output of the Transformer multi-translation model, and improving the model's learnability.
[0056] SMILES and SMILES sequence encoding are normalized SMILES and normalized SMILES sequence encoding, respectively.
[0057] S2: Build a Transformer multi-translation model and train it using the molecular descriptor encoding set of each small molecule compound to obtain a trained Transformer multi-translation model. The Transformer multi-translation model includes an encoding structure and a decoding structure.
[0058] S3: Obtain small molecules with known activities for specific targets from the ChEMBL database, input the SMILE of each small molecule into the trained Transformer multi-translation model for processing, and obtain molecular representations corresponding to each small molecule after pooling the output of the encoding structure. Use each small molecule, its activity data, and the molecular representations corresponding to each small molecule to construct a small molecule dataset, and divide the small molecule dataset to obtain training set, validation set, and test set.
[0059] S4: Build a machine learning classifier, and use the training set, test set, and validation set to train the machine learning classifier and search for hyperparameters to obtain the final machine learning classifier.
[0060] Step S4 specifically includes the following steps:
[0061] S41: Input the training set into a machine learning classifier with different hyperparameter settings for training;
[0062] S42: Input the validation set into the trained machine learning classifier to obtain the validation set prediction results, and use the hyperparameter set that makes the machine learning classifier obtain the highest area under the receiver operating characteristic curve as the final hyperparameter set of the machine learning classifier.
[0063] S43: Input the test set into a machine learning classifier containing the final hyperparameter set for testing, and use the machine learning classifier that obtains the highest value of the area under the receiver operating characteristic curve as the final machine learning classifier.
[0064] S5: Input the SMILE of the drug to be tested into the trained Transformer multi-translation model to obtain the molecular characterization of the drug to be tested, and input the molecular characterization of the drug to be tested into the final machine learning classifier for processing to obtain the screening results of the drug to be tested.
[0065] The Transformer multi-translation model has a network feature dimension of 256. Molecular descriptor codes with a length greater than 256 are truncated, and PAD (padding sequence) characters are used to pad the molecular descriptor codes with a length less than 256, so that the length of each molecular descriptor code is equal to 256.
[0066] Figure 2 The network structure of the Transformer multi-translation model provided according to an embodiment of the present invention is shown.
[0067] like Figure 2 As shown, both the encoder and decoder employ a two-layer attention mechanism. The network feature dimension of the Transformer multi-translation model is 256. The encoder uses a hard-shared mode, with its output fed into the multi-head attention mechanisms of four structurally identical but parameter-distributed decoders. In both the encoder and decoder, an embedding layer is immediately followed by a positional encoding layer to learn positional information. The first layer of each attention mechanism in the decoder is a mask multi-head attention mechanism, ensuring that the Transformer multi-translation model relies solely on previous outputs during translation, preventing it from depending on future information.
[0068] The encoding structure includes an embedding layer, a positional encoding layer, and two cascaded first attention mechanism modules. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing. The resulting feature A1 is then input into the two cascaded first attention mechanism modules for encoding. The features obtained after encoding are then pooled to obtain the molecular characterization of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character.
[0069] The positional information of the input sequence is marked by a positional encoding (PE) layer. Transformer multi-translation models typically use sine and cosine functions to generate positional codes, as shown in the following equation:
[0070]
[0071] Where pos is the position number of each word in the sequence, i represents the dimension, 2i represents an even dimension in the vector, and 2i+1 represents an odd dimension, and d... model This indicates the dimension of the hidden layer.
[0072] The embedding layer transforms the input encoding into a tensor the size of the hidden layer, while the positional encoding layer uses sinusoidal positional encoding, which generates a positional code for each position using a sine function.
[0073] The first attention mechanism module includes a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. Feature A1 is linearly transformed to obtain a first Q matrix, a first K matrix, and a first V matrix. These matrices are then input into the multi-head attention mechanism for processing to obtain feature A2. Features A1 and A2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A3. Feature A3 is then input into the feedforward network for processing to obtain feature A4. Finally, features A3 and A4 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A5.
[0074] The decoding structure includes a first decoder, a second decoder, a third decoder, and a fourth decoder, and each decoder has the same network structure. The first decoder includes an embedding layer, a positional encoding layer, two cascaded second attention mechanism modules, a linear layer, and a softmax function layer. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing to obtain feature B1. Feature B1 is input into the two cascaded second attention mechanism modules for processing. The processed features are sequentially input into the linear layer and the softmax function layer for prediction and mapping processing to obtain the word segmentation of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character.
[0075] Word segmentation is a fundamental task in NLP, breaking down sentences and paragraphs into word units to facilitate subsequent processing and analysis.
[0076] The second attention mechanism module includes a mask multi-head attention mechanism, a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. It performs a linear transformation on feature B1 to obtain the second Q matrix, the second K matrix, and the second V matrix. These matrices are then input into the mask multi-head attention mechanism for processing to obtain feature B2. Features B1 and B2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature B3. Finally, the features obtained from the encoding processing of the two cascaded first attention mechanism modules are... Linear transformation is performed to obtain the third K matrix and the third V matrix. Feature B3 is then linearly transformed to obtain the third Q matrix. The third Q matrix, the third K matrix, and the third V matrix are input into a multi-head attention mechanism for processing to obtain feature B4. Features B4 and B3 are then sequentially input into a residual block and a regularization layer for residual and regularization processing to obtain feature B5. Feature B5 is then input into a feedforward neural network for processing to obtain feature B6. Features B5 and B6 are then sequentially input into a residual block and a regularization layer for residual and regularization processing to obtain feature B7.
[0077] The input to the first decoder is the SMILES sequence code with the start character added; the input to the second decoder is the effective bit code of the 2D pharmacophore molecular fingerprint with the start character added; the input to the third decoder is the InChl sequence code with the start character added; and the input to the fourth decoder is the effective bit code of the PubChem molecular fingerprint with the start character added.
[0078] In the encoded SMILES and encoded InChI sequences, atoms consisting of two characters are treated as a single word unit during word segmentation; and in the encoded InChI sequence, the fragment "InChI=1S / " representing the version number is treated as a single word unit during word segmentation.
[0079] The encoding structure and each decoder of the Transformer multi-translation model are trained using the cross-entropy loss function, and the sum of the cross-entropy loss functions is used as the total loss function of the Transformer multi-translation model.
[0080] Cross-entropy loss function L:
[0081] ;
[0082] Where p(x) represents the true probability distribution and q(x) is the predicted probability distribution. For the i-th sample, It is the cross-entropy function;
[0083] Total loss function Loss:
[0084] Loss=a×L1+b×L2+c×L3+d×L4;
[0085] Where a, b, c, and d are adjustable custom coefficients, and L i Let be the loss function of the i-th decoder.
[0086] Table 1
[0087]
[0088] To compare the performance gap between the proposed molecular characterization method based on the Transformer multi-translation model and existing technologies, the Transformer multi-translation model was trained using the training parameters shown in Table 1. To enable each decoder to better learn features, the weights of the loss function for each decoder were adjusted based on its loss value, resulting in a well-trained Transformer multi-translation model. Molecular characterization of small molecule compounds of the same type was performed using the Transformer multi-translation model, traditional molecular descriptor methods, deep learning molecular characterization methods, and end-to-end deep learning molecular property prediction models. The performance comparison results are as follows: Figures 3-4 As shown.
[0089] Compared to five traditional molecular fingerprinting methods (PubChem, MACCS, RDKitFP, MorganFP, and PharmErGFP), the molecular characterization method of this invention exhibits the best performance on eight tasks across twelve benchmark databases. The twelve benchmark datasets are shown in the table below. Furthermore, compared to two deep learning molecular characterization methods based on translation models (ST and cddd, where ST is the molecular characterization obtained by reconstructing SMILES, and cddd is the molecular characterization obtained by translating SMILES into normalized SMILES), the molecular characterization method of this invention demonstrates the best performance on all twelve tasks. This indicates that the molecular characterization method of this invention mines richer molecular information and outperforms traditional molecular descriptors. Compared to deep learning molecular characteristic prediction models (Chemprop and MMNB), the MT-FP (Multi-translation fingerprint) method of this invention shows outstanding performance on six tasks, such as... Figure 4 As shown above, all data were partitioned using the same method as MoleculeNet (existing technology), and the same classifier type and hyperparameter search method were used when comparing with traditional molecular fingerprints and deep learning molecular fingerprints. The drug screening accuracy of this invention can reach 70%, which is higher than traditional screening methods and other machine learning drug screening methods.
[0090] Table 2
[0091]
[0092] To verify the effectiveness of this invention, we screened potential NLRP3 inflammasome inhibitors from approved drugs and selected 11 drugs that were predicted as effective at least twice in three machine learning classifiers as candidate drugs. Ten available drugs were purchased based on availability for cellular-level validation. In this embodiment, a mouse neuroimmune cell-microglia inflammation model was used to verify the inhibitory effect of the drugs on the NLRP3 inflammasome. Seven drugs (doxazosin, indocyanine green, difluoroprednisolone, spironolactone, aztreonam, piperacillin, and homoharringtonine) were preliminarily validated as effective in the cellular model, indicating that the accuracy of the drug screening method used in this embodiment can reach 70%. Among them, four drugs (difluoroprednisolone, indocyanine green, spironolactone, and doxazosin) could reproduce similar results under different stimulus models, suggesting that these four drugs may have more superior therapeutic effects. Furthermore, in vitro animal experiments revealed that all four drugs exhibited some inhibitory effect on the abnormal activation of microglia, and two drugs reduced the levels of pro-inflammatory cytokines in the NLRP3 inflammasome-related pathway in peripheral serum. Finally, the animal model was expanded in this invention to verify the effect of the best-performing drug in an Alzheimer's disease model related to the NLRP3 inflammasome. This drug was found to inhibit neuroinflammatory pathology in the mouse brain, demonstrating a therapeutic effect superior to the currently recognized best-performing NLRP3 inflammasome inhibitor, MCC950. These results indicate that the molecular representation proposed in this invention can better represent molecules, possesses the ability to predict molecularly related properties, and the virtual drug screening using this molecular representation has high accuracy, helping to alleviate the current shortage of drugs for related diseases.
[0093] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.
[0094] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A molecular characterization and drug screening method based on a Transformer multi-translation model, characterized in that, Specifically, the steps include the following: S1: Download small molecule compounds from the PubChem database and perform molecular preprocessing on all small molecule compounds to obtain the molecular descriptor encoding set for each small molecule compound; The molecular descriptor encoding set includes SMILES sequence encoding, 2D pharmacophore molecular fingerprint effective bit encoding, InChl sequence encoding, and PubChem molecular fingerprint effective bit encoding. S2: Construct the Transformer multi-translation model and train the Transformer multi-translation model using the molecular descriptor encoding set of each small molecule compound to obtain a trained Transformer multi-translation model. The Transformer multi-translation model includes an encoding structure and a decoding structure. The encoding structure includes an embedding layer, a positional encoding layer, and two cascaded first attention mechanism modules. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing to obtain feature A1. Feature A1 is then input into the two cascaded first attention mechanism modules for encoding processing. The feature obtained after encoding processing is pooled to obtain the molecular characterization of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character. The decoding structure includes a first decoder, a second decoder, a third decoder, and a fourth decoder, and each decoder has the same network structure. The first decoder includes an embedding layer, a positional encoding layer, two cascaded second attention mechanism modules, a linear layer, and a softmax function layer. The SMILES sequence encoding after adding the start character is sequentially input into the embedding layer and the positional encoding layer for processing to obtain feature B1. Feature B1 is input into the two cascaded second attention mechanism modules for processing. The processed feature is sequentially input into the linear layer and the softmax function layer for prediction and mapping processing to obtain the word segmentation of the small molecule compound corresponding to the SMILES sequence encoding after adding the start character. The first decoder takes as input the SMILES sequence code with the start character added; the second decoder takes as input the 2D pharmacophore molecular fingerprint valid bit code with the start character added; the third decoder takes as input the InChl sequence code with the start character added; and the fourth decoder takes as input the PubChem molecular fingerprint valid bit code with the start character added. S3: Obtain small molecules with known activities for specific targets from the ChEMBL database, input the SMILE of each small molecule into the trained Transformer multi-translation model for processing, and obtain molecular representations corresponding to each small molecule after pooling the output of the encoding structure. Use each small molecule, the activity data of each small molecule and the molecular representations corresponding to each small molecule to construct a small molecule dataset, and divide the small molecule dataset to obtain a training set, a validation set and a test set. S4: Build a machine learning classifier, and use the training set, the test set and the validation set to train the machine learning classifier and search for hyperparameters to obtain the final machine learning classifier; S5: Input the SMILE of the drug to be tested into the trained Transformer multi-translation model to obtain the molecular characterization of the drug to be tested, and input the molecular characterization of the drug to be tested into the final machine learning classifier for processing to obtain the screening results of the drug to be tested.
2. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11: Remove small molecule compounds downloaded from the PubChem database that have a molecular weight of 12 to 600, a LogP value of -7 to 5, and a heavy atom number of 3 to 50. S12: Use RDKit software to characterize the remaining small molecule compounds in step S11 as SMILES, and remove the small molecule compounds that cannot be characterized to obtain a small molecule compound dataset. S13: Use the RDKit software and PyBioMed software to obtain the SMILES, 2D pharmacophore fingerprint, PubChem fingerprint and InChI sequence of all small molecule compounds contained in the small molecule compound dataset, and obtain the effective bits of the 2D pharmacophore fingerprint and the effective bits of the PubChem fingerprint according to the corresponding 2D pharmacophore fingerprint and PubChem fingerprint. S14: Perform word segmentation on the SMILES, 2D pharmacophore fingerprint valid bits, PubChem fingerprint valid bits, and InChI sequences of all small molecule compounds contained in the small molecule compound dataset. Based on the frequency of occurrence of each word, encode the words of the SMILES, 2D pharmacophore fingerprint, PubChem fingerprint, and InChI sequence to obtain the SMILES sequence code, 2D pharmacophore molecular fingerprint valid bit code, InChI sequence code, and PubChem molecular fingerprint valid bit code.
3. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 1, characterized in that, The first attention mechanism module includes a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. Specifically, feature A1 is linearly transformed to obtain a first Q matrix, a first K matrix, and a first V matrix. These matrices are then input into the multi-head attention mechanism for processing to obtain feature A2. Features A1 and A2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A3. Feature A3 is then input into the feedforward network for processing to obtain feature A4. Finally, features A3 and A4 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature A5.
4. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 3, characterized in that, The second attention mechanism module includes a mask multi-head attention mechanism, a multi-head attention mechanism, a feedforward network, a residual block, and a regularization layer. It performs a linear transformation on feature B1 to obtain a second Q matrix, a second K matrix, and a second V matrix. The second Q matrix, the second K matrix, and the second V matrix are then input into the mask multi-head attention mechanism for processing to obtain feature B2. Features B1 and B2 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature B3. Finally, the features obtained from the encoding processing of the two cascaded first attention mechanism modules are linearly transformed. The third K matrix and the third V matrix are obtained, and the feature B3 is linearly transformed to obtain the third Q matrix. The third Q matrix, the third K matrix, and the third V matrix are input into the multi-head attention mechanism for processing to obtain feature B4. Feature B4 and feature B3 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature B5. Feature B5 is input into the feedforward network for processing to obtain feature B6. Feature B5 and feature B6 are sequentially input into the residual block and the regularization layer for residual and regularization processing to obtain feature B7.
5. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 1, characterized in that, In the encoded SMILES and encoded InChI sequences, atoms consisting of two characters are treated as a single token during word segmentation; and in the encoded InChI sequence, the InChI=1S / fragment representing the version number is treated as a single token during word segmentation.
6. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 1, characterized in that, The Transformer multi-translation model has a network feature dimension of 256. Molecular descriptor codes with a length greater than 256 are truncated, and PAD characters are used to pad the molecular descriptor codes with a length less than 256, so that the length of each molecular descriptor code is equal to 256.
7. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 4, characterized in that, The encoding structure and each decoder of the Transformer multi-translation model are trained using the cross-entropy loss function, and the sum of the cross-entropy loss functions is used as the total loss function of the Transformer multi-translation model. The cross-entropy loss function : ； Where p(x) represents the true probability distribution and q(x) is the predicted probability distribution. For the i-th sample, It is the cross-entropy function; The total loss function Loss: Loss=a×L1+b×L2+c×L3+d×L4; Where a, b, c, and d are adjustable custom coefficients, and L i Let be the loss function of the i-th decoder.
8. The molecular characterization and drug screening method based on the Transformer multi-translation model according to claim 1, characterized in that, The total number of small molecule compounds participating in the training is no less than three million.