Small molecule cross-modal generation method based on scGPT
By using the scGPT model to generate gene transcription profiles and small molecule data across modalities, the problem of poor generalization ability in existing technologies has been solved, and high-accuracy small molecule generation has been achieved, especially in generating perturbation transcription profiles and gene inhibitors.
Patent Information
- Application Number
- CN202311370274.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-23
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2043-10-23
AI Technical Summary
Existing phenotype-based drug discovery methods are limited by the size and quality of the database, have poor generalization ability, and are difficult to effectively generate potential small molecule drugs.
A small molecule cross-modal generation method based on scGPT is adopted. Gene transcription profiles and small molecule data are encoded by gene encoders and molecular encoders, respectively. Modality matching is performed using fully connected layers and contrastive learning. Combined with an autoregressive model, gene modalities are converted into small molecule modalities, and finally molecular structures in SMILES and Mol formats are output.
It significantly improves the accuracy of generating small molecules and potential gene inhibitors with perturbation transcription profiles, outperforming existing technologies. Under the Tanimoto similarity, Fraggle similarity, and MACCS fingerprint similarity indices, the generated results have a higher similarity to actual small molecules and can completely generate small molecule backbones with complex structures.
Smart Images
Figure CN117316267B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of neural network applications, and particularly relates to a small molecule cross-modal generation method based on a single cell basic model (scGPT). BACKGROUND
[0002] Phenotype-based drug design is an important research content of bioinformatics, and the phenotype includes gene expression profiles and cell maps. The existing phenotype-based drug search compares the similarity between the input and the recorded gene expression differences to find potential small molecule drugs, which is limited by the size and quality of the database. SUMMARY
[0003] The application optimizes the poor generalization ability of the prior art, and proposes a small molecule cross-modal generation method based on scGPT, which obtains candidate molecules capable of inducing the required transcriptome profile by inputting the required gene expression profile and the control gene expression profile.
[0004] The application is implemented through the following technical scheme:
[0005] The application relates to a small molecule cross-modal generation method based on scGPT, which encodes and extracts features from the preprocessed gene transcriptome and small molecule data using a gene encoder and a molecule encoder, respectively, maps the two modalities to the same dimensional space through a full connection layer, and performs multi-modal matching of the gene modality and the small molecule modality through contrast learning, aligns the gene vector through an autoregressive model to convert the gene modality into a small molecule modality, and finally outputs the small molecule vector in SMILES and Mol formats.
[0006] The gene encoder adopts an open-source scGPT model.
[0007] The molecule encoder and generator adopt an open-source hierVAE model.
[0008] The preprocessing refers to selecting experimental data belonging to six cell lines (VCAP, PC3, A549, A375, HT29, and MCF7) in Level 3 in the L1000 public data set (GEO ID: GSE70138) and blank control data under the same conditions, then calculating the average gene expression level of each small molecule, and finally dividing the training set, test set and validation set according to the InChi-key main layer.
[0009] The multi-modal matching of the gene modality and the small molecule modality through contrast learning specifically includes:
[0010] 1) The small molecule encoding is mapped to a specific dimension through a fully connected layer, and the experimental group gene encoding is first subtracted from the control group encoding to obtain gene differential expression encoding, which is then mapped to the same dimension as the small molecule mapping through a fully connected layer;
[0011] 2) In one batch, each small molecule mapping vector forms a positive sample with its corresponding gene vector and a negative sample with other gene vectors. The present application needs to maximize the cosine similarity of the positive sample and minimize the similarity between the negative samples. The present application uses the InfoNCE loss function to achieve this goal;
[0012] 3) Repeat until the validation set loss does not decrease for 5 consecutive rounds.
[0013] The gene modality is converted into a small molecule modality through an autoregressive model, specifically comprising:
[0014] a) After dimensionality reduction of the small molecule modality before alignment, the small molecule modality after dimensionality reduction is flattened, 1024 intervals are divided, and each interval is labeled. The number of vectors belonging to the interval is replaced with the interval label, so that the small molecule encoding is discretized;
[0015] b) The aligned gene modality, the overall mean of the gene modality, and the discretized small molecule encoding form a sequence, and a transformer neural network is used to predict the small molecule encoding one by one, wherein: the transformer neural network only predicts the first sequence number of the small molecule according to the gene modality and the overall mean in the initial state, and then predicts the second sequence number of the small molecule by combining the gene modality, the overall mean, and the first sequence number, until the small molecule encoding is predicted complete;
[0016] c) Use PCA inverse conversion to convert the discretized small molecule encoding into continuous small molecule encoding.
[0017] Technical effects
[0018] The present application is significantly higher than the prior art in terms of Tanimoto similarity based on Morgan fingerprint, Tanimoto similarity based on MACCS fingerprint, and Fraggle similarity in the task of generating perturbed small molecules from small molecule perturbation transcriptome and generating potential gene inhibitors from gene knockout perturbation transcriptome. BRIEF DESCRIPTION OF DRAWINGS
[0019] Figure 1 The flowchart of the present application;
[0020] Figure 2 The overall structure diagram of the present application;
[0021] Figure 3To generate small molecule sketches from drug perturbed gene transcriptional profiles;
[0022] In the figure: (1)-(10) respectively show the generation results of the present application and 10 sketches of actual molecules;
[0023] Figure 4 To generate small molecule sketches from drug perturbed gene transcriptional profiles;
[0024] In the figure: (a)-(c) respectively represent the similarity distribution under the three indexes of MACCS fingerprint based Tanimoto similarity, Morgan fingerprint based Tanimoto similarity and Fraggle similarity;
[0025] Figure 5 To generate small molecule sketches from gene knockout perturbed gene transcriptional profiles;
[0026] In the figure: (1)-(10) respectively show the generation results of the present application and 10 sketches of the most similar known gene inhibitors;
[0027] Figure 6 To generate small molecule sketches from drug perturbed gene transcriptional profiles; DETAILED DESCRIPTION
[0028] As Figure 2 shown, a small molecule cross-modal generation system based on scGPT related to the present embodiment, comprising: a gene encoder, a small molecule encoder, a small molecule generator, a modal aligner and a modal converter, wherein: the gene encoder uses an open source scGPT model, the molecule encoder and generator use an open source hierVAE model; the modal aligner maps the two modalities into the same dimensional space using a fully connected layer and aligns using contrastive learning; the modal converter obtains the small molecule vector in an autoregressive manner from the aligned gene vector.
[0029] As Figure 1 shown, a small molecule cross-modal generation method based on scGPT based on the above system, comprising the following steps:
[0030] Step 1, preprocessing the gene expression data in the dataset, specifically: selecting the experimental data belonging to six cell lines (VCAP, PC3, A549, A375, HT29, MCF7) from Level 3 of the L1000 public dataset and the blank control data under the same conditions, and then taking the average gene expression level of each small molecule. The final data set size is 2x15156x978: 2 represents the experimental group and the control group, 15156 represents the sample size, and 978 represents the number of gene species. Then 300 (a total of 708 types of pharmacological small molecules) are randomly selected from the 6360 structures divided from the InChi-key main layer as the test set, 300 (a total of 691 types of pharmacological small molecules) are randomly selected as the validation set, and the remaining data are all used for model training.
[0031] Step 2, the gene encoder encodes the gene expression of the blank control group and the experimental group to obtain two gene encoding vectors.
[0032] Step 3, the small molecule encoder encodes the small molecule to obtain a small molecule encoding vector.
[0033] Step 3, let the blank control gene encoding vector and the experimental group gene encoding vector be different in the hidden space to obtain the gene expression difference.
[0034] Step 4, contrastive learning matches two modalities, this step only appears in the training stage, specifically including:
[0035] 4.1 Map the small molecule encoding through a fully connected layer to a specific dimension, then subtract the experimental group gene encoding from the control group encoding to obtain the gene differential expression encoding, and then map it through a fully connected layer to the same dimension as the small molecule mapping after the mapping;
[0036] 4.2 Use the InfoNCE loss function to maximize the cosine similarity of the positive samples and minimize the similarity between the negative samples, specifically:
[0037]
[0038] 4.3 To prevent model collapse, add auxiliary guidance to the last three layers of the gene encoder, specifically: use hierarchical unsupervised clustering to divide the small molecule encoding vector into 10, 100, and 1000 classes. Let the gene vector predict the corresponding small molecule class, and this purpose is achieved through the cross-entropy loss function.
[0039]
[0040] 4.4 Repeat iterations until the validation set loss does not decrease for 5 consecutive rounds.
[0041] Step 5, the autoregressive model converts from gene expression difference encoding to small molecule token, specifically including:
[0042] 5.1 Reduce the dimension of the pre-alignment small molecule mode by PCA, then flatten the reduced dimension small molecule mode, divide it into 1024 intervals, and label each interval. The number of vectors in this interval is replaced by the interval label, so that the small molecule encoding is discretized, called small molecule token;
[0043] 5.2 Discretize the gene mode after alignment, the overall mean of the gene mode, and the small molecule encoding to form a sequence, then use the transformer to predict the small molecule encoding one by one. Initially, the model only predicts the first sequence number of the small molecule according to the gene mode and the overall mean, and then combines the gene mode, the overall mean, and the first sequence number to predict the second sequence number of the small molecule. Until the small molecule encoding is predicted complete;
[0044] 5.3 Use PCA inverse conversion to convert discretized small molecule token to continuous small molecule encoding.
[0045] Step 6, input the converted small molecule encoding into the small molecule decoder to directly generate SMILES and Mol.
[0046] Through specific and actual experiments, the present application verifies the generalization ability of generating drug-like small molecules from gene expression profiles under compound perturbation and demonstrates the zero-shot learning ability of generating potential inhibitors from gene expression profiles under gene knockout perturbation.
[0047] Test indicators include:
[0048] Indicator 1, Tanimoto similarity based on MACCS keys, commonly used for similarity comparison of local features of molecules;
[0049] Indicator 2, Tanimoto similarity based on Morgan fingerprint: radius = 3, 1024 bits, commonly used for similarity comparison of global features of molecules;
[0050] Indicator 3, Fraggle similarity, commonly used for similarity comparison of local features of molecules.
[0051] To better reflect the generation ability of the model, the present embodiment is compared with three methods:
[0052] Method 1, calculate the cosine similarity of the gene expression change of the test set and the training set to find the closest small molecule;
[0053] Method 2, calculate the Euclidean distance between the gene expression changes of the test set and the training set to find the closest small molecules;
[0054] Method 3, random generation, that is, let the model directly sample from the small molecule hidden space to generate.
[0055] The specific results are shown as follows:
[0056] As shown in Figure 3 , the generated results of some models are compared with the structures of actual acting small molecules. It can be seen from the comparison that the model of the embodiment can completely generate most of the skeletons of the reference small molecules, can effectively generate long-chain molecules, and can process complex structures;
[0057] As shown in Figure 4 , the three types of index score distribution graphs of the model of the embodiment and the other three methods are shown. The results generated by the model are obviously better than the other three methods.
[0058] As shown in Figure 5 , the visualization comparison of the molecules generated by the model of the embodiment and the known inhibitors is shown. The similarity score shown is calculated based on the Taniomito of Morgan fingerprint. It can be seen that the model of the embodiment not only generates molecules that are very similar to known inhibitors as a whole, but more importantly, the model of the embodiment completely generates the skeleton part of them, which is convenient for scientists to perform subsequent optimization;
[0059] As shown in Figure 6 , the experimental results are completely shown by a box plot. It can be seen that the model of the embodiment is basically better than the other three methods.
[0060] Compared with the prior art, the application applies a gene large model to downstream cross-modal molecule generation. Unlike the one-dimensional representation SMILES used in the past small molecule cross-modal fusion, the application uses a two-dimensional molecular representation with more information. In the modeling process, the application uses auxiliary guidance to alleviate the model collapse problem and improve the effect of modality alignment. A modality converter based on an autoregressive form is used. The application achieves better results than existing methods in the tasks of generating small molecules with perturbed gene expression profiles and generating inhibitors of specified genes.
[0061] The above specific embodiments can be adjusted in different ways by those skilled in the art without departing from the principles and purposes of the application. The protection scope of the application is subject to the claims and is not limited by the above specific embodiments. Each implementation within the scope is subject to the constraints of the application.
Claims
1. An scGPT-based small molecule cross-modal generation system, characterized in that, include: The system consists of a gene encoder, a small molecule encoder, a small molecule generator, a modality aligner, and a modality converter. The gene encoder uses the open-source scGPT model, while the molecule encoder and generator use the open-source hierVAE model. The modality aligner maps two modalities to the same dimensional space using a fully connected layer and aligns them using contrastive learning. The modality converter obtains the small molecule vector from the aligned gene vector using an autoregressive approach. The aforementioned small molecule cross-modal generation refers to: encoding and extracting features from preprocessed gene transcription profiles and small molecule data using gene encoders and molecular encoders respectively; mapping the two modalities to the same dimensional space using a fully connected layer; and performing multimodal matching between gene modalities and small molecule modalities through contrastive learning; then converting the aligned gene vectors into small molecule modalities using an autoregressive model; and finally restoring the small molecule vectors to SMILES and Mol formats for output. The aforementioned multimodal matching of gene modalities and small molecule modalities through contrastive learning specifically includes: 1) The small molecule code is mapped to a specific dimension through a fully connected layer. Then, the gene codes of the experimental group and the control group are subtracted to obtain the differential gene expression codes. These differentially expressed codes are then mapped to the same dimension as the small molecule code through a fully connected layer. 2) In a batch, each small molecule's mapped vector forms a positive sample with its corresponding gene vector and a negative sample with other gene vectors. The InfoNCE loss function is used to maximize the cosine similarity of the positive samples and minimize the similarity between the negative samples. 3) Iterate repeatedly until the validation set loss no longer decreases for 5 consecutive rounds; The aforementioned method of converting gene modes into small molecule modes using an autoregressive model specifically includes: a) After reducing the dimensionality of the small molecule modes before alignment using PCA, all the reduced small molecule modes are flattened and divided into 1024 intervals. Each interval is labeled, and the vector values belonging to that interval are replaced with the interval label, thus discretizing the small molecule encoding. b) The aligned gene modalities, the overall mean of the gene modalities, and the discretized small molecule codes are combined into a sequence. The transformer neural network is used to predict the small molecule codes one by one. In the initial state, the transformer neural network only predicts the first sequence number of the small molecule based on the gene modalities and the overall mean. Then, it combines the gene modalities, the overall mean, and the first sequence number to predict the second sequence number of the small molecule, until the small molecule codes are completely predicted. c) Use PCA inverse transformation to convert discretized small molecule codes into continuous small molecule codes.
2. The scGPT-based small molecule cross-modality generation system of claim 1, wherein, The preprocessing refers to: selecting experimental data from Level 3 of GEO ID: GSE70138 in the L1000 public dataset, belonging to VCAP, PC3, A549, A375, HT29 and MCF7 cell lines respectively, and blank control data under the same conditions; then calculating the average gene expression level of each small molecule; and finally dividing the training set, test set and validation set according to the InChi-key principal layer.