A cell type annotation method based on pre-trained single-cell multi-omics

Abstract

The present invention discloses a cell type annotation method based on pre-trained single-cell multi-omics, which belongs to the field of bioinformatics, including obtaining a gene expression matrix of paired scRNA-seq data and a chromatin accessibility peak matrix of scATAC-seq data, and pre-processing the gene expression matrix and the chromatin accessibility peak matrix, and then inputting a pre-trained feature extractor, and using its multi-head cross-attention network to obtain a query feature matrix of fused multi-omics; the feature extractor includes a multi-head cross-attention network and two classifiers; the query feature matrix is ​​input into a classification model, a linear layer is used for data splitting, and then a Transformer encoder is input for multi-omics genetic feature extraction, and then a classifier is input for cell type annotation. The cell type annotation method of this scheme solves the problem of deviation in cell type annotation in existing methods.

Description

Technical Field

[0001] The present invention relates to the field of bioinformatics, and in particular to a cell type annotation method based on pre-trained single-cell multi-omics. Background Art

[0002] Single cell type annotation is crucial for in-depth study of the tumor microenvironment and the development of precision medicine treatment plans. Although traditional wet-lab-based immunohistochemistry and fluorescence-activated cell sorting methods are reliable, their development cycle is long and costly. With the rapid development of high-throughput sequencing technology, a large amount of unannotated single-cell sequencing data has emerged. However, the limitations of traditional experimental protocols have severely restricted the application of these large-scale data. In contrast, the use of computer simulation methods for high-confidence cell type annotation has greatly improved the efficiency of the application of single-cell sequencing data.

[0003] In recent years, supervised classification methods based on deep learning have made rapid progress in the field of single-cell type annotation. This is due to the success of deep learning in other classification tasks, which enables large-scale validation in a relatively short period of time. Many of these methods are built from the perspective of single-cell RNA sequencing omics, integrating information about gene-gene interactions of biological cells into a unified end-to-end framework. These methods treat single-cell type annotation as a multi-class classification task and perform cell type prediction by feeding the data input forward to different deep encoder and decoder modules.

[0004] Despite promising progress made by some deep learning methods, existing deep learning-based methods still face two major challenges. First, it is very challenging to explicitly learn the genetic distribution patterns between single-cell multi-omics data. Currently, single-omics-based cell type annotation methods only partially reveal the specificity of cells. In complex biological systems, this may lead to biases in downstream analyses such as cell type annotation. The second challenge involves improving the prediction performance in datasets with large differences in the number of cell types to go beyond the learned distribution. Due to high-throughput sequencing of single cells in different biological tissue regions, the single cells that need to be predicted in real applications may be rare, and their specific gene expression may be quite different from that of common cells. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the cell type annotation method based on pre-training single-cell multi-omics provided by the present invention solves the problem of deviation in cell type annotation in the prior art.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0007] A method for annotating cell types based on pre-trained single-cell multi-omics is provided, comprising the steps of:

[0008] S1. Obtain the gene expression matrix of paired scRNA-seq data and the chromatin accessibility peak matrix of scATAC-seq data, and preprocess the gene expression matrix and the chromatin accessibility peak matrix;

[0009] S2. Input the preprocessed gene expression matrix and chromatin accessibility peak matrix into the pre-trained feature extractor, and use its multi-head cross attention network to obtain the query feature matrix integrating multi-omics; the feature extractor includes a multi-head cross attention network and two classifiers;

[0010] S3. Input the query feature matrix into the classification model, use the linear layer for data splitting, then input it into the Transformer encoder for multi-omics genetic feature extraction, and then input it into the classifier for cell type annotation.

[0011] Furthermore, the method for pre-training the feature extractor includes:

[0012] S21. Obtain gene expression matrices of paired scRNA-seq data of several known cell types and chromatin accessibility peak matrices of scATAC-seq data, and preprocess them;

[0013] S22, input the preprocessed gene expression matrix and chromatin accessibility peak matrix into two cross attention modules of the multi-head cross attention network respectively to obtain the Q matrix, the key K matrix and the value V matrix;

[0014] S23, based on the self-attention mechanism in the Transformer architecture, the Softmax function is used to calculate the cross feature matrix extracted by each attention head in the two cross attention modules;

[0015] S24, concatenating the cross feature matrices extracted by all attention heads corresponding to each cross attention module, and then concatenating the features input into the cross attention module in step S22 to obtain two cross feature data;

[0016] S25, inputting the two cross-feature data into a classifier respectively to obtain the cell type, and comparing the cell type with the true label to obtain the pre-training loss value of the corresponding channel of each classifier;

[0017] S26, summing the pre-training loss values ​​of the two channels to obtain the total loss of the feature extractor, and saving the pre-training model obtained after each pre-training is completed;

[0018] S27, when the number of training times does not reach the preset number of iterations, return to step S22; when the number of training times reaches the preset number of iterations, proceed to step S28;

[0019] S28. Compare the pre-trained models obtained from each training, and select the pre-trained model with the highest accuracy as the pre-trained feature extractor.

[0020] The beneficial effects of the above technical solution are as follows: the training of the feature extractor and classification model of this solution is two rounds of supervised learning. The first round of supervised learning, i.e., the feature extractor training, uses paired single-cell multi-omics data to predict cell types at their respective omics levels, thereby training a feature extractor for multi-omics fusion data. Specifically, the multi-head cross-attention network understands and integrates the genetic distribution laws of scRNA-seq data and scATAC-seq data, so that the original single-omics data obtains the characteristics of another omics. Then, the two data extracted with cross-features are independently input into the classifier, and the cell type is predicted with the cross-omics features learned by each of them. By reducing the common loss of the prediction results of the two classifiers, the single-cell multi-omics cross-features are more effectively extracted. This process not only improves the efficiency of feature extraction between omics, but also ensures the synergy between the respective omics data, providing a solid foundation for the prediction of cell types in the next round of supervised learning classification tasks.

[0021] Furthermore, the expression for calculating the cross-feature matrix is:

[0022]

[0023] Among them, X R and X A They are the preprocessed gene expression matrix and chromatin accessibility peak matrix, respectively; and X R and X A The cross feature matrix obtained by the i-th attention head of the corresponding cross attention module; and X R Q matrix, key K matrix and value V matrix; and X A The Q matrix, key K matrix and value V matrix of the attention head; T is the transpose; heads is the total number of attention heads; h is the feature dimension of the attention head; softmax(.) is the Softmax function.

[0024] Furthermore, the expression of the two cross-feature data is:

[0025]

[0026] Among them, Att R and Att A For all and all The cross attention matrix after splicing, X′ R and X′ A X R and X A Corresponding cross feature data; Concat(.) is the tensor concatenation function.

[0027] The beneficial effect of the above technical solution is: the use of a multi-head attention mechanism can reduce the computational complexity, and the calculation of the entire attention matrix is ​​completed by splicing the results of the calculation of each head.

[0028] Furthermore, the expression of the query feature matrix is:

[0029] X c =0.5X′ R +0.5X′ A

[0030] Among them, X c is the query feature matrix.

[0031] The beneficial effect of the above technical solution is: this is a feature fusion method, which adopts a 1:1 fusion method from two data ratios, and retains the tolerance of the two data distributions to the greatest extent.

[0032] Furthermore, the training method of the classification model includes:

[0033] S31, inputting the gene expression matrix and chromatin accessibility peak matrix preprocessed in step S21 into the pre-trained feature extractor, and using its multi-head cross attention network to obtain a query feature matrix integrating multi-omics;

[0034] S32, using the query feature matrix in step S31 to train the classification model, and saving the classifier trained each time, the classification model includes a linear layer, a Transformer encoder and a classifier connected in sequence;

[0035] S33. Determine whether the number of training times for the classification model reaches the preset number of training times. If so, output the classification model corresponding to the classifier with the lowest verification loss; otherwise, return to step S31.

[0036] The beneficial effects of the above technical solution are as follows: the second round of supervised learning, i.e., the training of the classification model, uses the paired multi-omics data involved in the first round of pre-training as the model input. After splitting, the data is input into the Transformer encoder to extract features from the single-cell multi-omics fusion data, aiming to improve the performance of further inputting the data into the classifier for cell type annotation tasks. Thanks to the learning effect of the cross-omics features of the cross-attention network in the first round of pre-training, the classifier can effectively capture the genetic distribution characteristics of the two omics modalities of scRNA-seq data and scATAC-seq data. This feature can enable the cell type annotation method of this scheme to have the ability to correct the characteristic deviations of complex biological systems to a certain extent. Therefore, when the paired scRNA-seq data and scATAC-seq data are combined for feature fusion, the model can output more accurate cell type annotation results.

[0037] Furthermore, in step S3, the expression for data splitting is:

[0038]

[0039] in, is the mth feature submatrix formed after the query feature matrix Xc is split; s is the total number of feature submatrices; d is the number of c The characteristic length of l is the length of the characteristic submatrix;

[0040] The expression of Transformer encoder for multi-omics genetic feature extraction is:

[0041]

[0042] Among them, X M is the feature extracted by the Transformer encoder; Mean(·) is the average pooling layer function; Concat(·) is the tensor concatenation function.

[0043] Furthermore, the loss function of the classifier in step S3 is:

[0044] Loss pred =-∑z j log(q j )

[0045] Among them, Loss pred is the loss function value; z j and q j They are the predicted result type and the type of its true label for single cell j, respectively.

[0046] Furthermore, the method for preprocessing the gene expression matrix and the chromatin accessibility peak matrix includes:

[0047] Regularization of gene expression matrix and chromatin accessibility peak matrix;

[0048] The gene expression matrix after regularization is screened using the screening method of high-variability genes to obtain high-variability genes;

[0049] The peak matrix was used to obtain the gene activity matrix in the chromatin accessibility peak matrix after regularization, and PCA dimensionality reduction was performed to make the dimensions of the gene activity matrix and the highly variable genes equal.

[0050] The beneficial effects of the present invention are as follows: the deep learning pre-training strategy proposed in this scheme uses a cross-attention network to promote mutual learning of features from paired multi-omics data, thereby enabling single cell type annotation tasks, thereby achieving accurate cell type annotation. This scheme combined with paired multi-omics data is conducive to solving the defect that single-omics data is difficult to directly use deep learning methods to accurately annotate cell types due to high-dimensional sparsity. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Flowchart of a cell type annotation method based on pre-trained single-cell multi-omics.

[0052] Figure 2 A model framework diagram corresponding to a cell type annotation method based on pre-trained single-cell multi-omics.

[0053] Figure 3 The performance comparison chart of this scheme method and nine comparison methods in the prior art on 10X-Multiome, ISSAAC-seq and SHARE-seq datasets is used to evaluate the accuracy index.

[0054] Figure 4 The figure shows the performance comparison of the proposed method and nine comparison methods in the prior art on the 10X-Multiome, ISSAAC-seq and SHARE-seq datasets using the balanced accuracy metric.

[0055] Figure 5 This is the confusion matrix diagram of the annotation using this method on the Multiome-BMMC dataset.

[0056] Figure 6 This is the confusion matrix diagram of the annotation using the CIForm method on the Multiome-BMMC dataset.

[0057] Figure 7 This is the confusion matrix diagram of the annotation using the scTransSort method on the Multiome-BMMC dataset. DETAILED DESCRIPTION

[0058] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0059] refer to Figure 1 , Figure 1 A flowchart of a cell type annotation method based on pre-trained single-cell multi-omics is shown; Figure 1 As shown, the method S includes steps S1 to S3.

[0060] In step S1, the gene expression matrix of the paired scRNA-seq data and the chromatin accessibility peak matrix of the scATAC-seq data are obtained, and the gene expression matrix and the chromatin accessibility peak matrix are preprocessed;

[0061] Methods for preprocessing gene expression matrices and chromatin accessibility peak matrices include:

[0062] Regularization of gene expression matrix and chromatin accessibility peak matrix;

[0063] The gene expression matrix after regularization was screened using a high-variance gene screening method to obtain high-variance genes; 4096 high-variance genes were retained during the screening, which is crucial for capturing cell heterogeneity and reducing feature dimensions.

[0064] The peak matrix is ​​used to obtain the gene activity matrix in the chromatin accessibility peak matrix after regularization, and PCA dimensionality reduction is performed to make the dimensions of the gene activity matrix and the highly variable genes equal. This scheme uses the PCA dimensionality reduction method to retain the first 4096 principal components.

[0065] This protocol uses the above preprocessing steps to not only retain the highly expressed gene information in scRNA-seq data, but also processes the high-dimensional sparse scATAC-seq data format into a format suitable for deep genetic feature mining using deep learning methods. By reducing the complexity of paired multi-omics data without affecting the relevance at the biological level, the efficiency of cell type annotation using paired single-cell multi-omics data is effectively improved.

[0066] In step S2, the preprocessed gene expression matrix and chromatin accessibility peak matrix are input into the pre-trained feature extractor, and its multi-head cross-attention network is used to obtain the query feature matrix integrating multi-omics; the feature extractor includes a multi-head cross-attention network and two classifiers.

[0067] like Figure 2 As shown, in one embodiment of the present invention, the method for pre-training the feature extractor includes:

[0068] S21. Obtain gene expression matrices of paired scRNA-seq data of several known cell types and chromatin accessibility peak matrices of scATAC-seq data, and preprocess them;

[0069] S22, input the preprocessed gene expression matrix and chromatin accessibility peak matrix into two cross attention modules of the multi-head cross attention network respectively to obtain the Q matrix, the key K matrix and the value V matrix;

[0070] For the preprocessed gene expression matrix, the expressions of its Q matrix, key K matrix and value V matrix are:

[0071]

[0072] in, and The weight parameter matrices of the three linear layers in the cross-attention module; X R and X A They are the preprocessed gene expression matrix and chromatin accessibility peak matrix, respectively; and X R The Q matrix, key K matrix and value V matrix of the criss-cross attention module; heads is the total number of attention heads of each criss-cross attention module.

[0073] For the preprocessed chromatin accessibility peak matrix, the expressions of its Q matrix, key K matrix and value V matrix are:

[0074]

[0075] in, and X A The Q matrix, key K matrix and value V matrix of .

[0076] S23. Based on the self-attention mechanism in the Transformer architecture, the Softmax function is used to calculate the cross feature matrix extracted by each attention head in the two cross attention modules:

[0077]

[0078] in, and X R and X A The cross feature matrix obtained by the i-th attention head of the corresponding cross attention module; T is the transpose; h = d / heads is the feature dimension of the attention head; d is X R The total dimension of ; softmax(.) is the Softmax function.

[0079] When calculating the cross-feature matrix, matrix transposition is used mainly to allow two matrices of the same shape to be multiplied with each other. For example, a matrix of shape N*M is transformed into M*N through transposition, so the matrices can be multiplied, and the result of the multiplication is N*N.

[0080] S24, concatenating the cross feature matrices extracted by all attention heads corresponding to each cross attention module, and then concatenating the features input into the cross attention module in step S22 to obtain two cross feature data;

[0081] During implementation, this solution preferably uses the expression of two cross-feature data as follows:

[0082]

[0083] Among them, Att R and Att A For all and all The cross attention matrix after splicing, X′ R and X′ A X R and X A Corresponding cross feature data; Concat(.) is the tensor concatenation function.

[0084] During implementation, the expression of the query feature matrix in step S2 of this solution is preferably:

[0085] X c =0.5X′ R +0.5X′ A

[0086] Among them, X c is the query feature matrix.

[0087] S25, inputting the two cross-feature data into a classifier respectively to obtain the cell type, and comparing the cell type with the true label to obtain the pre-training loss value of the corresponding channel of each classifier;

[0088] S26, summing the pre-training loss values ​​of the two channels to obtain the total loss of the feature extractor, and saving the pre-training model obtained after each pre-training is completed;

[0089] S27, when the number of training times does not reach the preset number of iterations, return to step S22; when the number of training times reaches the preset number of iterations, proceed to step S28;

[0090] S28. Compare the pre-trained models obtained from each training, and select the pre-trained model with the highest accuracy as the pre-trained feature extractor.

[0091] In this scheme, we propose to use the cross-attention mechanism to extract the cross-attention scores between the paired scRNA-seq data and scATAC-seq data, and then obtain the potential correlation between the two omics data, so as to improve the performance of this scheme in the single-cell type annotation task.

[0092] In step S3, the query feature matrix is ​​input into the classification model, a linear layer is used for data splitting, then input into the Transformer encoder for multi-omics genetic feature extraction, and then input into the classifier for cell type annotation.

[0093] During implementation, the preferred classification model training method of this solution includes:

[0094] S31, inputting the gene expression matrix and chromatin accessibility peak matrix preprocessed in step S21 into the pre-trained feature extractor, and using its multi-head cross attention network to obtain a query feature matrix integrating multi-omics;

[0095] S32, using the query feature matrix in step S31 to train the classification model, and saving the classifier trained each time, the classification model includes a linear layer, a Transformer encoder and a classifier connected in sequence;

[0096] S33. Determine whether the number of training times for the classification model reaches the preset number of training times. If so, output the classification model corresponding to the classifier with the lowest verification loss; otherwise, return to step S31.

[0097] In one embodiment of the present invention, in step S3, the expression for data splitting is:

[0098]

[0099] in, is the mth feature submatrix formed after the query feature matrix Xc is split; s is the total number of feature submatrices; d is the number of c The characteristic length of l is the length of the characteristic submatrix;

[0100] The expression of Transformer encoder for multi-omics genetic feature extraction is:

[0101]

[0102] Among them, X M is the feature extracted by the Transformer encoder; Mean(·) is the average pooling layer function; Concat(·) is the tensor concatenation function.

[0103] The loss function of the classifier in step S3 is:

[0104] Loss pred =-∑z j log(q j )

[0105] Among them, Loss pred is the loss function value; z j and q j They are the predicted result type and the type of its true label for single cell j, respectively.

[0106] This solution uses deep learning technology, especially Transformer-based models, to model and train the processed data. The Transformer model has the ability to capture complex relationships in the data and can effectively identify and distinguish different types of cells, including rare cells.

[0107] The effect of the cell type annotation method provided by this scheme (hereinafter referred to as scMoAnno) is evaluated in combination with specific examples:

[0108] Preparing the evaluation dataset

[0109] This example evaluates scMoAnno and nine state-of-the-art benchmark methods in the prior art, including CIForm, scTransSort, AtacAnnoR, GLUE, Seurat v3, scJoint, Conos, MAESTRO, and CellWalkR, on four public paired single-cell multi-omics datasets (10X-Multiome, ISSAAC-seq, SHARE-seq, and Multiome-BMMC).

[0110] The 10X-Multiome dataset was launched by the biotechnology company 10x Genomics and is a single-cell analysis platform that integrates multi-omics data. The ISSAAC-seq dataset is derived from an in situ sequencing heterogeneous RNA-DNA hybridization technology developed by Xu et al., which is used for analysis after transcriptome and chromatin sequencing. SHARE-seq is a dataset for studying single-cell gene expression and chromatin accessibility, which can be obtained from the GEO database. The Multiome-BMMC dataset is an analysis of bone marrow mononuclear cells, which is crucial for hematological diagnosis. The four benchmark datasets mentioned above all contain two types of omics data, paired scRNA-seq and scATAC-seq.

[0111] Model performance analysis

[0112] In order to evaluate the performance of scMoAnno, a single-cell multi-omics cell type annotation method of this scheme, reference training and query testing were performed on three paired multi-omics benchmark datasets: 10X-Multiome, ISSAAC-seq, and SHARE-seq. scMoAnno was compared with nine advanced comparison methods in the prior art. During the training process on these three benchmark datasets, the same hyperparameter settings, such as learning rate, number of training rounds, and number of attention heads, were maintained to ensure the generalization ability of the verification model.

[0113] like Figure 3 and Figure 4 Comprehensive visual evaluation results of ten methods on three datasets in terms of accuracy and balanced accuracy indicators are presented respectively.

[0114] First, scMoAnno was compared with nine other methods on the 10X-Multiome dataset. The comparison results can be found in the columns corresponding to the datasets 10X-Multiome, ISSAAC-seq, and SHARE-seq in Table 1.

[0115] Table 1 Comparative analysis results of scMoAnno and nine methods in the prior art

[0116]

[0117] As shown in Table 1, the scMoAnno method outperforms the nine comparison methods in the prior art in terms of accuracy. Compared with the comparison method CIForm with the best accuracy, the scMoAnno method of this solution is 2% higher in accuracy. In terms of the balanced accuracy index, it is equal to the best comparison method scTransSort and reaches a balanced accuracy of 90%. Therefore, this verifies the feasibility and effectiveness of the scMoAnno method.

[0118] Next, according to the parameter settings of the previous experiment, comparative experiments were conducted on the ISSAAC-seq dataset and the SHARE-seq dataset. The experimental results are also listed in Table 1. In terms of the accuracy evaluation indicators of these two datasets, the scMoAnno method exceeded the best comparison method GLUE by as much as 4% on the former dataset (ISSAAC-seq dataset), and was on par with the best comparison method scTransSort on the latter dataset (SHARE-seq dataset), achieving an accuracy of 83%.

[0119] In terms of the balanced accuracy index, the scMoAnno method leads the way in all aspects, being 3% higher than the best comparison method AtacAnnoR in the former dataset and 2% higher than the two best comparison methods CIForm and scTransSort in the latter dataset. The evaluation of these two datasets shows that the scMoAnno method proposed in this solution still shows improvement in indicators compared with the advanced methods, which further verifies that the scMoAnno method has a certain degree of generalization ability.

[0120] Finally, the balanced accuracy of the scMoAnno method and the other nine comparison methods on the three datasets are summarized in Table 1. The average accuracy and balanced accuracy of the scMoAnno method evaluated on the three datasets reached 0.88 and 0.86, respectively, which are significantly better than other methods. Therefore, it shows that the superiority of the scMoAnno method proposed in this scheme and its generalization ability have been further verified.

[0121] Rare cell identification

[0122] The key component of this solution is the multi-head cross-attention network, which is used to extract cross-genetic distribution features from scRNA-seq and scATAC-seq data. Under this premise, this example selects two models based on the Transformer self-attention mechanism, namely CIForm and scTransSort, for comparative experiments on the accuracy of rare cell identification.

[0123] In order to verify that scMoAnno has the ability to generalize between different cell type datasets, especially in the case of extreme differences in cell numbers, this embodiment designed a rare cell identification experiment and formulated rare cell division rules. Specifically, a division rule was formulated within the benchmark experimental dataset, that is, cells whose number of a certain cell type is less than or equal to 2% of the total number of cells in the dataset are defined as rare cells. Under this division rule, the Multiome-BMMC dataset divides 9 types of rare cells. The specific names of these rare cells and their proportions in their respective datasets are listed in Table 2.

[0124] Table 2 Rare cell identification on the Multiome-BMMC dataset

[0125]

[0126]

[0127] In order to clearly present the prediction results of the three methods (scMoAnno, CIForm and scTransSort) on all cell types in the dataset, the confusion matrix predicted by the three methods in the Multiome-BMMC dataset is plotted, as shown in Figure 2. Figures 5 to 7 As shown in the figure. The confusion matrix can intuitively understand the performance of the model on different categories by showing the comparison between the actual classification and the predicted classification of each category. By calculating the accuracy of each category, the confusion matrix can help identify which categories are correctly classified and which categories are misclassified, thereby evaluating and improving the overall performance of the classification model.

[0128] In addition, in order to more clearly observe the prediction results of the rare cell types divided in the Multiome-BMMC dataset, this embodiment summarizes the final prediction accuracy in Table 2. As can be seen from Table 2, the scMoAnno method has 5 types of rare cell prediction accuracy in the Multiome-BMMC dataset that exceeds the comparison method, and the average accuracy of all rare cells in the calculated dataset is 2% ahead of the best comparison method CIForm.

[0129] In summary, scMoAnno can ensure high recognition accuracy when applied to sequencing datasets with an unbalanced number of cell types or containing significantly scarce cell types.

Claims

1. A cell type annotation method based on pre-trained single-cell multi-omics, characterized in that: Includes steps: S1. Obtain the gene expression matrix of paired scRNA-seq data and the chromatin accessibility peak matrix of scATAC-seq data, and preprocess the gene expression matrix and the chromatin accessibility peak matrix; S2. Input the preprocessed gene expression matrix and chromatin accessibility peak matrix into the pre-trained feature extractor, and use its multi-head cross attention network to obtain the query feature matrix integrating multi-omics; the feature extractor includes a multi-head cross attention network and two classifiers; S3, the query feature matrix is ​​input into the classification model, a linear layer is used for data splitting, then input into the Transformer encoder for multi-omics genetic feature extraction, and then input into the classifier for cell type annotation; Methods for pre-training feature extractors include: S21. Obtain gene expression matrices of paired scRNA-seq data of several known cell types and chromatin accessibility peak matrices of scATAC-seq data, and preprocess them; S22, the preprocessed gene expression matrix and chromatin accessibility peak matrix are input into the two cross attention modules of the multi-head cross attention network respectively, and the Q Matrix, key K Matrices and Numbers V matrix; S23, based on the self-attention mechanism in the Transformer architecture, the Softmax function is used to calculate the cross feature matrix extracted by each attention head in the two cross attention modules; S24, concatenating the cross feature matrices extracted by all attention heads corresponding to each cross attention module, and then concatenating the features input into the cross attention module in step S22 to obtain two cross feature data; S25, inputting the two cross-feature data into a classifier respectively to obtain the cell type, and comparing the cell type with the true label to obtain the pre-training loss value of the corresponding channel of each classifier; S26, summing the pre-training loss values ​​of the two channels to obtain the total loss of the feature extractor, and saving the pre-training model obtained after each pre-training is completed; S27, when the number of training times does not reach the preset number of iterations, return to step S22; when the number of training times reaches the preset number of iterations, proceed to step S28; S28, comparing the pre-trained models obtained from each training, and selecting the pre-trained model with the highest accuracy as the pre-trained feature extractor; The training methods for classification models include: S31, inputting the gene expression matrix and chromatin accessibility peak matrix preprocessed in step S21 into the pre-trained feature extractor, and using its multi-head cross attention network to obtain a query feature matrix integrating multi-omics; S32, using the query feature matrix in step S31 to train the classification model, and saving the classifier trained each time, the classification model includes a linear layer, a Transformer encoder and a classifier connected in sequence; S33. Determine whether the number of training times for the classification model reaches the preset number of training times. If so, output the classification model corresponding to the classifier with the lowest verification loss; otherwise, return to step S31.

2. The cell type annotation method according to claim 1, characterized in that: The expression for calculating the cross-feature matrix is: Among them, X R and X A They are the preprocessed gene expression matrix and chromatin accessibility peak matrix, respectively; and X R and X A The corresponding criss-cross attention module i The cross feature matrix obtained by the attention heads; , and X R of Q Matrix, key K Matrices and Numbers V matrix; , and X A of Q Matrix, key K Matrices and Numbers V matrix; T is the transpose; heads is the total number of attention heads; h is the feature dimension of the attention head; softmax(.) is the Softmax function.

3. The cell type annotation method according to claim 2, characterized in that: The expression of two cross-feature data is: ， in, and For all and all The cross attention matrix after splicing, and X R and X A Corresponding cross feature data; Concat(.) is the tensor concatenation function.

4. The cell type annotation method according to claim 3, characterized in that: The expression of the query feature matrix is: Among them, X c is the query feature matrix.

5. The cell type annotation method according to claim 1, characterized in that: In step S3, the expression for data splitting is: ， in, The query feature matrix X c After being split, the m feature submatrices; s is the total number of feature submatrices; d is X c The characteristic length of l is the length of the feature submatrix; The expression of Transformer encoder for multi-omics genetic feature extraction is: in, Features extracted for the Transformer encoder; is the average pooling layer function; is the tensor concatenation function.

6. The cell type annotation method according to claim 1, characterized in that: The loss function of the classifier in step S3 is: in, is the loss function value; and For single cells j The type of predicted result and its true label type.

7. The cell type annotation method according to any one of claims 1 to 6, characterized in that: Methods for preprocessing gene expression matrices and chromatin accessibility peak matrices include: Regularization of gene expression matrix and chromatin accessibility peak matrix; The gene expression matrix after regularization is screened using the screening method of high-variability genes to obtain high-variability genes; The peak matrix was used to obtain the gene activity matrix in the chromatin accessibility peak matrix after regularization, and PCA dimensionality reduction was performed to make the dimensions of the gene activity matrix and the highly variable genes equal.

Images