A method for representing single-cell transcriptomes based on multi-omics

Abstract

The present invention relates to the field of bioinformatics, and specifically discloses a method for representing a single-cell transcriptome based on multi-omics assistance. The expression value of each gene or protein in each cell is regarded as a segmentation, and it is decided to use two MAE structures to respectively train the transcriptome and proteome data of CITE-seq to complete multi-omics auxiliary tasks. The training phase is divided into three stages. The first stage is the pre-training stage of multimodal data, the second stage is the fine-tuning stage of multimodal data, and the third stage is a unique prediction stage of unimodal data. The present invention proposes scMMAE (single-cell multimodal masked autoencoder), which uses pure MAE as the backbone network, plus a cross-attention mechanism, with a simple structure and powerful performance. The present invention applies this structure to CITE-seq data for the first time, and can also assist unimodal data in downstream tasks.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics, and in particular to a method for expressing single-cell transcriptomes based on multi-omics assistance. Background Art

[0002] Single-cell multi-omics refers to the technology of analyzing multiple biological molecules of cells at the single cell level at the same time. This technology can reveal differences between cells that are not easy to observe at the population level, such as the functions and states of different cell types. Through single-cell multi-omics analysis, scientists can understand cell behavior and cell-to-cell interactions more accurately, which is very valuable for revealing complex biological processes and disease mechanisms.

[0003] CITE-seq is an advanced single-cell multi-omics technology that combines single-cell transcriptomics (i.e., RNA expression analysis of cells) with protein surface marker analysis. In simple terms, CITE-seq allows researchers to not only read out the transcriptome information of cells, but also detect protein markers on the cell surface at the same time, which enables us to have a more comprehensive understanding of the identity and status of each cell.

[0004] Single-cell transcriptomics refers specifically to the analysis of RNA expression at the single-cell level. This technology can reveal subtle differences in gene expression levels between different cells, which may be overlooked in studies at the multi-cell or whole tissue level. Single-cell transcriptomics is an important branch of single-cell research and is crucial for understanding developmental biology, tumor heterogeneity, and cell fate decisions.

[0005] Single-cell proteomics is another important component of single-cell multi-omics, which involves analyzing protein expression and function at the single-cell level. Unlike single-cell transcriptomics, which focuses on RNA expression, single-cell proteomics focuses on the number, state, location of proteins within cells, and their interactions. This technology allows scientists to gain a detailed understanding of how proteins regulate cell behavior and their role in different biological processes. Through single-cell proteomics, researchers can identify protein fingerprints for specific cell states or types, which help reveal how cells respond to internal and external signals and their functional differences in health and disease. In addition, single-cell proteomics can also reveal the role of protein modifications (such as phosphorylation, ubiquitination, etc.) in regulating cell function, which is crucial for understanding cell signaling networks.

[0006] Currently, Transformer has been used in various fields of deep learning and has achieved good performance. However, there has been no research report on the model training of CITE-seq data using the image mask autoencoder (MAE) structure. Therefore, it is necessary to develop a representation method for single-cell transcriptomes assisted by multi-omics. Summary of the invention

[0007] The present invention aims to solve at least one of the technical problems existing in the above-mentioned prior art. To this end, the present invention proposes a method for representing single-cell transcriptomes based on multi-omics assistance.

[0008] The inventive concept of the present invention is: the expression value of each gene or protein in each cell is regarded as a token, and two MAE structures are used to train the transcriptome and proteome data of CITE-seq respectively to complete multi-omics auxiliary tasks. It should be noted that the model training phase of this application is divided into three stages. The first stage is the pre-training stage of multimodal data, the second stage is the fine-tuning stage of multimodal data, and the third stage is a unique prediction stage of unimodal data.

[0009] The first aspect of the present invention provides a method for representing single-cell transcriptomes based on multi-omics assistance.

[0010] Specifically, the following steps are included:

[0011] (1) In the pre-training phase, the transcriptome data were normalized by RPKM, and the proteome data were normalized by CLR;

[0012] (2) The transcriptome data and proteome data are inputted on a per-cell basis, and the data are passed through a linear layer to generate a segmentation sequence;

[0013] (3) Add a gene-based embedding vector and a protein-based embedding vector to the generated word segmentation sequence;

[0014] (4) During the training process, 5% to 15% of the word segmentation sequence is randomly masked, and then the unmasked vector is input into the encoder of the model for learning, the encoded gene expression vector and proteomics vector are fused, and the complete word segmentation sequence is reconstructed through the decoder. When the mean square error loss function tends to be stable, the pre-trained model is saved;

[0015] (5) The fine-tuning stage provides 25-35% of the annotated cell type labels. The fine-tuning stage uses the encoder of the pre-trained model. All data is input into the encoder without masking. The decoder is fine-tuned to output the correct label. When the cross entropy loss function tends to be stable, the fine-tuned model is saved.

[0016] (6) The prediction stage uses the single-cell transcriptome and proteome data, or only the single-cell transcriptomics data, to predict the type of a single cell using a fine-tuned model;

[0017] (7) Perform downstream task analysis on the predicted data.

[0018] Preferably, in step (1), the RPKM normalization is to divide the number of reads mapped to the gene by the number of reads mapped to all genomes; the CLR normalization is the central logarithmic ratio transformation, so that the proteomic data conforms to the normal distribution rule.

[0019] Preferably, in step (2), the word segmentation sequence includes a two-dimensional word segmentation sequence of transcriptome data and a one-dimensional word segmentation sequence of proteome data.

[0020] Preferably, in step (2), each cell is a single-channel matrix.

[0021] Preferably, in step (2), the shape of the two-dimensional segmentation n sequence of the transcriptome data is the number of cells*400*10.

[0022] Preferably, in step (2), the shape of the one-dimensional word segmentation sequence of the proteomic data is the number of cells*the number of proteins*1.

[0023] Preferably, in step (3), the shape of the gene-based embedding vector is 1*400*128; the shape of the protein-based embedding vector is 1*number of proteins*128.

[0024] Preferably, the encoder contains 6 Transformer blocks, the multi-head self-attention layer of each Transformer block contains 2 heads, and the output dimension is 128; the decoder contains 4 Transformer blocks, the multi-head self-attention layer of each Transformer block contains 2 heads, and the output dimension is 128.

[0025] Preferably, in step (4), the mean square error loss function is to calculate the reconstruction loss between the masked segmentation sequence and the restored segmentation sequence: Lrec , reconstruction loss Lrec The formula is

[0026] where Y i is the original transcript term, The segmentation of the transcriptome recovery, Y j The word for the original protein component, is the word segmentation of proteomic recovery; n is the number of training data, α is the weight of transcriptome reconstruction loss, and β is the weight of proteomic reconstruction loss.

[0027] Preferably, in step (5), the calculation formula of the cross entropy loss function is:

[0028] Where n is the number of cell types, Y i is the true label, is the predicted label.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0030] The present invention proposes scMMAE (single-cell multimodal masked autoencoder), which uses pure MAE as the backbone network and adds a cross-attention mechanism. It has a simple structure and powerful performance. The present invention applies this structure to CITE-seq data for the first time, and can also assist single-modal data in downstream tasks. The present invention designs a weighted multimodal data reconstruction loss function, which can find that the weight of a certain modality data is more important. The present invention improves the cell clustering effect of single-cell transcriptomes and provides further help for discovering new cell subpopulations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the method for expressing single-cell transcriptome based on multi-omics assistance of the present invention;

[0032] Figure 2 Evaluation and comparison results and analysis diagrams of multimodal analysis on 5 CITE-seq datasets;

[0033] Figure 3 Evaluation and comparison results and analysis diagrams of scMMAE on four scRNA-seq datasets. DETAILED DESCRIPTION

[0034] In order to make the technical scheme of the present invention more clearly understood by those skilled in the art, the following embodiments are listed for illustration. It should be pointed out that the following embodiments do not limit the protection scope of the present invention.

[0035] Unless otherwise specified, the raw materials, reagents or devices used in the following examples can be obtained from conventional commercial sources or by existing known methods.

[0036] Example 1

[0037] A single-cell transcriptome representation method based on multi-omics assistance.

[0038] The input data of the model is Figure 1 As shown in Figures A and B, adding additional modality data can help utilize different information from different omics for cell analysis. The unique features of the model are Figure 1As shown in C in the figure, after the model is trained with two modal data and saved, only one modal data is needed for the experiment.

[0039] The overall design flow chart of the present invention is as follows: Figure 1 As shown in D:

[0040] ScMMAE consists of three stages. In the first stage, a self-supervised learning approach is applied to pre-train the model with unlabeled CITE-seq data. This involves using masked expression embeddings and learnable gene and protein embeddings as input, which are then processed through encoder blocks. Cross-attention blocks are used to extract shared features, while salient features are retained by including residuals (⊕).

[0041] In the second stage, a small portion of CITE-seq data with cell information was input into scMMAE for multi-omics fusion.

[0042] In the third stage, scMMAE transfers the knowledge learned from multimodal omics fusion to enhance the performance of single-omics.

[0043] FC: fully connected layer. The encoder and decoder are composed of Transformer blocks, and each Transformer block is composed of four modules: multi-head attention layer, addition and normalization layer, multi-layer perception base layer and addition and normalization layer:

[0044] (1) Multi-head self-attention layer: This part mainly performs the operation of the attention mechanism. The dimensions of the three vectors q, k, and v are all 128. The formula is as follows:

[0045]

[0046] Among them, softmax represents the softmax function, which means that multiple values ​​are first exponentially calculated and then summed up to obtain the normalization operation. The multiple heads of the multi-head self-attention are divided into several groups to complete the attention operation, and the operation is consistent with the above formula.

[0047] (2) Addition and normalization layer: This layer mainly completes a residual connection, adds the original input to the output of the attention mechanism, and then performs a normalization operation. The normalization activation function used here is layer normalization (LayerNorm).

[0048] (3) Multi-layer perceptron layer, which includes two linear layers and one GELU activation layer. The two linear layers play the role of first increasing the dimension and then reducing the dimension.

[0049] (4) Addition and re-normalization layers, consistent with the second structure.

[0050] Validation experiments using transcriptome and proteome data:

[0051] The CITE-seq public datasets from the 10XGenomics platform were collected, including a 10k peripheral blood mononuclear cell dataset from healthy donors (PBMC10k), a 5k peripheral blood mononuclear cell dataset from healthy donors (PBMC5k), and a 10k cell dataset from MALT tumors (MALT10K). In addition, CITE-seq datasets (SPL111 and SPL206) for measuring spleen and lymph nodes from the literature written by Adam et al. were also collected. The above five datasets are stored in the github repository of Adam et al. (https: / / github.com / YosefLab / totalVI_reproducibility / tree / master / data).

[0052] In addition, scRNA-seq datasets from the 10XGenomics platform were also collected, including a 3k peripheral blood mononuclear cell dataset (PBMC3K) from healthy donors, a dataset measuring umbilical cord blood mononuclear cells (CBMC), a PBMC dataset containing interferon-stimulated cells and control cells (IFNB), and a dataset measuring bone marrow mononuclear cells (BMCITE). The above four datasets are stored on the Seurat platform (https: / / github.com / satijalab / seurat-data). Experiments were conducted on the 9 collected datasets, and downstream tasks such as cell clustering and cell annotation were performed for comparison. That is, experiments were conducted on five CITE-seq datasets (gene and protein data) such as SPL111, SPL206, PBMC5K, PBMC10K and MALT10K, and four scRNA-seq datasets (gene data) such as PBMC3K, IFNB, CBMC and BMCITE, and downstream tasks of cell clustering and cell annotation were performed for comparison.

[0053] The output data is the reconstruction of gene and protein data, which will be used together with the original data to calculate the loss function. The second is the embedding data of the cells, which is used for cell clustering. This application uses this embedded data to first run the K nearest neighbor algorithm to obtain an adjacency matrix for the cells, and then uses the Leiden clustering algorithm to cluster the cells into communities to obtain cell type labels.

[0054] The present invention and five other methods were compared on five CITE-seq datasets and four scRNA-seq datasets for the accuracy of predicting cell types. A total of 10 evaluation indicators were used for evaluation, and the model of the present invention achieved the best results in most of the evaluation indicators.

[0055] Figure 2 Evaluation and comparison of multimodal analysis on five CITE-seq datasets. (A) Comparison of scMMAE and other five baseline methods in ARI, NMI, FMI, average silhouette width, AMI, Jaccard index, SC, Calinski-Harabasz, and F-measure. (B) Comparison of the five datasets evaluated by DBI.

[0056] Figure 3 Evaluation and comparison of scMMAE on four scRNA-seq datasets. (A) Comparison of scMMAE and SOTA methods using metrics such as ARI, NMI, FMI, average silhouette width, AMI, Jaccard index, SC, Calinski-Harabasz, and F-measure. (B) Five methods were compared on four scRNA-seq datasets evaluated by DBI.

[0057] Specifically, in the CITE-seq dataset, the evaluation index Adjusted Rand Index (ARI) increased by 0.7 compared with the scMM model that also performed this task, and increased by 0.21 compared with the second place. The evaluation index Normalized Mutual Information (NMI) increased by 0.78 compared with the scMM model that also performed this task, and increased by 0.12 compared with the second place. In the scRNA-seq dataset, the average increase in the adjusted Rand index was about 0.20, and the average increase in the normalized mutual information was nearly 0.10 (the value range of the adjusted Rand coefficient ARI is [-1,1], and the larger the value, the more consistent the clustering result is with the actual situation. In a broad sense, ARI measures the degree of consistency between the two data distributions; the normalized mutual information NMI is an indicator used to measure the similarity of two clustering results. The NMI indicator is obtained by comparing the ratio of the number of identical categories in the two clustering results to the total number of categories.

[0058] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution obtained by any modification, equivalent replacement, improvement, etc. made by a person skilled in the art based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall be within the scope of protection determined by the claims.

Claims

1. A method for representing single-cell transcriptomes based on multi-omics, characterized in that: The following steps are involved: (1) In the pre-training stage, the transcriptome data was normalized by RPKM, and the proteome data was normalized by CLR; (2) The transcriptome data and proteome data are input on a per-cell basis, and the data is passed through a linear layer to generate a word segmentation sequence; (3) Add a gene-based embedding vector and a protein-based embedding vector to the generated word segmentation sequence; (4) During the training process, 5-15% of the word segmentation sequence is randomly masked, and then the unmasked vector is input into the encoder of the model for learning. The encoded gene expression vector and proteomics vector are fused, and the complete word segmentation sequence is reconstructed through the decoder. When the mean square error loss function tends to be stable, the pre-trained model is saved; (5) The fine-tuning stage provides 25-35% of the annotated cell type labels. The fine-tuning stage uses the encoder of the pre-trained model. All data is input into the encoder without masking. The decoder is fine-tuned to output the correct label. When the cross entropy loss function tends to be stable, the fine-tuned model is saved. (6) In the prediction stage, the type of a single cell is predicted using a fine-tuned model based on single-cell transcriptome and proteome data, or only single-cell transcriptomics data; (7) Perform downstream task analysis on the predicted data; The encoder contains 6 Transformer blocks, each Transformer block's multi-head self-attention layer contains 2 heads, and the output dimension is 128; the decoder contains 4 Transformer blocks, each Transformer block's multi-head self-attention layer contains 2 heads, and the output dimension is 128; In step (4), the mean square error loss function is to calculate the reconstruction loss between the masked word segmentation sequence and the restored word segmentation sequence. Lrec , reconstruction loss Lrec The formula is ,in is the original transcript term, The segmentation recovered for the transcriptome, The word for the original protein component, Participles recovered for the protein group; n is the number of training data, α is the weight of transcriptome reconstruction loss, and β is the weight of proteome reconstruction loss.

2. The method of representation according to claim 1, characterized in that: In step (1), the RPKM normalization is to divide the number of reads mapped to the gene by the number of reads mapped to all genomes; the CLR normalization is the central logarithmic ratio transformation, so that the proteomic data conforms to the normal distribution rule.

3. The representation method according to claim 1, characterized in that: In step (2), the word segmentation sequence includes a two-dimensional word segmentation sequence of the transcriptome data and a one-dimensional word segmentation sequence of the proteome data.

4. The representation method according to claim 1, characterized in that: In step (2), each cell is a single-channel matrix.

5. The representation method according to claim 1, characterized in that: In step (2), the shape of the two-dimensional segmentation n sequence of the transcriptome data is the number of cells*400*10.

6. The representation method according to claim 1, characterized in that: In step (2), the shape of the one-dimensional word segmentation sequence of the proteomic data is the number of cells*the number of proteins*1.

7. The representation method according to claim 1, characterized in that: In step (3), the shape of the gene-based embedding vector is 1*400*128; the shape of the protein-based embedding vector is 1*number of proteins*128.

8. The representation method according to claim 1, characterized in that: In step (5), the calculation formula of the cross entropy loss function is: , where n is the number of cell types, is the true label, is the predicted label.

Images