Cell type annotation method based on self-supervised distillation learning

Abstract

The invention discloses a cell type annotation method based on self-supervised distillation learning, and belongs to the technical field of cell type annotation, and the method comprises the following steps: constructing a cell specificity high correlation network according to cosine similarity between cells; splicing the single cell transcriptome data tensor and the splicing tensor of each node in the cell specificity high correlation network to obtain spliced data; sending the spliced data into a global learning layer for self-supervised distillation learning, and extracting feature information of scRNA-seq data; and carrying out cell type annotation on the scRNA-seq data by adopting a KAN model based on the feature information. The scTCCHN is used as a downstream task model for deep feature extraction of scRNA-seq data, the universality of the model can be improved, the noise influence caused by the batch effect is overcome, and the robust and accurate capacity which is highly extensible in the aspect of cell type annotation is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field of cell type annotation, and in particular relates to a cell type annotation method based on self-supervised distillation learning. Background Art

[0002] Single-cell RNA sequencing (scRNA-seq) technology has revolutionized biomedical research because it can analyze gene expression at the level of individual cells. This technology has played a vital role in studying cellular heterogeneity, complexity, and cell differentiation. Accurate annotation of cell types is essential for understanding the regulatory mechanisms behind these biological processes and developing new medical treatment strategies. However, wet experimental annotation methods such as fluorescent chemical sorting are not only demanding but also costly. Therefore, computational tools are urgently needed to automatically predict and understand gene expression relationships, thereby achieving automatic and efficient cell type annotation.

[0003] In recent years, computational cell type annotation methods can be divided into three categories:

[0004] 1) Based on marker genes, cells are first clustered according to transcriptome-specific expression, and then clusters are identified using marker genes; 2) Based on correlation, the correlation between the query dataset and the reference dataset is calculated by a specific correlation coefficient to identify cell types; 3) Based on supervised classification, a neural network is used to train the relationship between gene expression profiles and cell type labels in the reference dataset to predict cell types. However, these methods still have limitations in model capacity, accuracy, robustness, or generalization ability.

[0005] Self-supervised learning has shown great research value in addressing these challenges. This method first pre-trains unlabeled single-cell transcriptome data, and then uses the labeled data for supervised fine-tuning in subsequent tasks. For example, scBERT has achieved excellent results in single-cell annotation tasks by pre-training with a large language model. This method effectively overcomes the limitations of traditional deep learning methods, opens up a new research paradigm for downstream analysis of scRNA-seq data, and shows great potential in single-cell gene characterization tasks. However, the current self-supervised pre-training models for single-cell transcriptome data mainly focus on learning gene expression of sequence data, so the learning ability of the model can be enhanced by introducing more knowledge representation. At the same time, how to introduce a screening strategy for highly variable genes (HVGs) in self-supervised learning to ensure that key genes are not ignored remains an urgent problem to be solved. Summary of the invention

[0006] The purpose of the present invention is to address the above-mentioned deficiencies in the prior art and provide a cell type annotation method based on self-supervised distillation learning to solve the problems of low model training efficiency and poor generalization ability in existing self-supervised learning applications of cell type annotation.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A cell type annotation method based on self-supervised distillation learning, comprising the following steps:

[0009] S1. Construct a cell-specific high correlation network based on the cosine similarity between cells;

[0010] S2, splicing the single-cell transcriptome data tensor and the splicing tensor of each node in the cell-specific high correlation network to obtain spliced ​​data;

[0011] S3, sending the spliced ​​data to the global learning layer for self-supervised distillation learning to extract the feature information of scRNA-seq data;

[0012] S4. Based on the feature information, the KAN model is used to annotate the cell types of the scRNA-seq data.

[0013] Furthermore, the S1 specifically includes the following sub-steps:

[0014] S1.1. Calculate the cosine similarity between cells:

[0015]

[0016] Among them, CosineSimilarity(x i ,x j ) is the cosine similarity between cells, x i is the gene expression vector of the ith cell, x j is the gene expression vector of the jth cell;

[0017] S1.2, select T cells related to the ith cell to form a sub-network containing T+1 nodes, where the nodes represent cells and the edges represent the cosine similarity between cells;

[0018] S1.3, calculating the cosine similarity between two nodes, and constructing a cosine similarity matrix according to the calculated cosine similarity;

[0019] S1.4. Based on the cosine similarity matrix, a cell-specific high correlation network is constructed using the Kruskal algorithm.

[0020] Furthermore, the Kruskal algorithm is used in S1.4 to construct a cell-specific high correlation network, including:

[0021]

[0022] Among them, MST is a cell-specific high correlation network, which means the maximum spanning tree; subgraph is a subgraph; S hl is an element in the cosine similarity matrix, representing the cosine similarity between the hth cell and the lth cell.

[0023] Furthermore, the splicing data obtained in S2 specifically includes:

[0024] X input =Concatenate(X scRNA-seq ,X correlation )

[0025] Among them, X input is the splicing data; X scRNA-seq is the single-cell transcriptome data tensor; X correlation It is the tensor concatenation of each node in the cell-specific highly correlated network; Concatenate() is the tensor concatenation function.

[0026] Furthermore, in S3, the concatenated data is sent to the global learning layer for self-supervised distillation learning, including:

[0027] The spliced ​​data X input Enter the teacher model in the global learning layer, which uses the global attention mechanism of the scLinformer model to learn the global feature representation of the input data:

[0028]

[0029] in, is the data assigned to the teacher model after passing through the global attention layer; GlobalAttention() is the global attention layer function;

[0030] The output of the student model in the global learning layer is:

[0031] Z student =StudentModel(0.5X input +0.5X scRNA-seq )

[0032] Among them, Z student is the data passed through the student model; StudentModel() is the function in the student model that processes the tensor data;

[0033] The teacher model is optimized by minimizing the mean square error loss function:

[0034]

[0035] in, is the mean square error loss function; N is the total number of cell samples; is the i-th data after the student model; ‖·‖ 2 is the square norm of the vector;

[0036] The Minkowski distance loss function is used to calculate the student model output Z student TensorX with single-cell transcriptome data scRNA-seq Consistency between:

[0037]

[0038] in, is the Minkowski distance loss function; is the input raw scRNA-seq data; ‖·‖ p represents the Minkowski norm.

[0039] Furthermore, the global attention mechanism of the scLinformer model is:

[0040]

[0041] Among them, Attention(Q,K,V) is the data after attention; softmax() is the activation function; are query, key, and value matrices, respectively, and d k are the dimensions of keys and values; E,F∈R k×n They are all projection matrices, n and k are the lengths of the E and F dimensions respectively.

[0042] Furthermore, the feature information of scRNA-seq data is extracted in S3, including:

[0043] z i =StudentModel(x i )

[0044] Among them, z i ∈R d Represents the d-dimensional feature vector of the i-th cell.

[0045] Furthermore, the S4 specifically includes:

[0046] In the KAN model, the d-dimensional feature vector z of the i-th cell is i Mapping to predicted cell types

[0047]

[0048] Among them, f(z i ) is the cell type prediction function; φ g and ψ gs is the learned nonlinear transformation function, m and p are the parameters of the KAN model, g is the index of the g-th feature activation function, s is the s-th input index of the g-th specific feature activation function;

[0049] The cross entropy loss function is used to optimize the KAN model:

[0050]

[0051] Among them, N is the number of cell samples, C is the number of cell type categories, and y i,c Indicates that the i-th sample belongs to the true label of category c, represents the probability that the i-th sample is predicted to be of category c, L cross is the cross entropy loss function;

[0052] By minimizing the cross entropy loss function, the parameters of the KAN model are optimized:

[0053]

[0054] Among them, θ is the parameter of the classifier, θ * is the optimized classifier parameter; is the cell type predicted by the classifier; y is the actual cell type.

[0055] Furthermore, g and ψ gs Represented as a linear transformation plus a nonlinear activation function:

[0056]

[0057] Among them, B gs and B g is the spline basis function, and u is the eigenvector.

[0058] The cell type annotation method based on self-supervised distillation learning provided by the present invention has the following beneficial effects:

[0059] 1. The present invention is different from the existing single-channel self-supervised learning paradigm. The method of the present invention makes up for the deficiency of only containing gene-level features in the single-cell transcriptome data channel through a cell-specific highly correlated network. During the pre-training process, in order to better learn the complementary information of the two channels, a strategy for enhancing data interaction is formulated. To this end, a global feature learning layer is introduced to jointly process the embedding from the two channel encoders. In practical applications, cell type annotation usually relies only on scRNA-seq data. In order to adapt to the data conditions of actual downstream tasks and reasonably utilize the cell-specific highly correlated network parameters in the pre-training model to improve the effectiveness of downstream tasks, the present invention converts the input of the two channels into a single-channel input through self-supervised distillation, divides the global learning layer into a teacher model and a student model, and jointly processes the cell-specific highly correlated network and transcriptome data. Among them, the teacher model with larger parameters learns the knowledge of scRNA-seq data and cell-specific highly correlated networks. At the same time, the student model with smaller parameters receives the original input of scRNA-seq and the knowledge of the teacher model, and jointly reconstructs the new expression of the two inputs. In this way, a feature extractor model with fewer parameters but capable of improving the performance of downstream cell type annotation tasks is trained.

[0060] 2. The present invention selects other cells that are most relevant to a single cell through its gene expression, and uses the maximum spanning tree algorithm to construct a cell-specific highly correlated network, making full use of the correlation between gene expressions of similar cells and solving the problem of insufficient utilization of prior information such as highly variable genes in gene-level expression.

[0061] 3. Based on the auxiliary role of cell-specific highly correlated networks, the present invention designs a distillation learning paradigm for self-supervised learning of transcriptome data, which can not only learn multi-scale single-cell gene expression features, but also ensure that the student model used as feature extraction has a moderate number of parameters, thereby showing high efficiency in downstream tasks.

[0062] 4. By using a large amount of unlabeled single-cell transcriptome data, the scTCHCN framework can learn the gene representation relationship of different types of cells and achieve state-of-the-art performance in a series of downstream tasks of cell type annotation. In addition, this method can not only better distinguish cell subtypes, but also show superior performance in the accuracy of identifying rare cells.

[0063] 5. The global learning layer of the present invention consists of a teacher model with a large parameter scale and a student model with a small parameter scale, forming a distillation learning paradigm for the joint feature data of long-sequence single-cell transcriptome data and cell-specific high-correlation networks; specifically, the teacher model receives the splicing features from scRNA-seq data and cell-specific high-correlation networks, and performs gene representation learning covering the global attention mechanism. Subsequently, the output data of the teacher model is passed to the student model, so that while only learning single-cell scRNA-seq data, it can also have a gene representation memory similar to these cells. In this way, the student model reduces the number of parameters while maintaining efficient cell-level and gene-level representation capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flowchart of a cell type annotation method based on self-supervised distillation learning in Example 1 of the present invention.

[0065] Figure 2 This is a scTCHCN framework diagram of Example 1 of the present invention.

[0066] Figure 3 Schematic diagram of the cell-specific high correlation network of Example 1 of the present invention.

[0067] Figure 4 This is a schematic diagram of the cell type annotation data flow of Example 1 of the present invention.

[0068] Figure 5 It is a simplified diagram of the KAN neural network and a distribution diagram of the KAN neural network B-spline function piecewise fitting curve function according to Example 1 of the present invention.

[0069] Figure 6 t-SNE graphs of three data sets in Example 2 of the present invention, wherein the left panel is colored according to the expert classification labels in the original experiment, and the right panel is colored according to the best prediction result of scTCHCN.

[0070] Figure 7 These are three clustering score indicator graphs of Example 2 of the present invention. The blue line represents the original scRNA-seq data, the red line represents the scRNA-seq data after deep features are extracted by the student model, and the purple bar graph represents the difference in clustering score improvement of each indicator after the student model is used on each data set. DETAILED DESCRIPTION

[0071] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0072] Example 1

[0073] This embodiment provides a cell type annotation method based on self-supervised distillation learning. The self-supervised distillation pre-training framework scTCHCN constructed by the method of this embodiment is referred to as Figure 2 , used to process single-cell transcriptome data and cell-specific highly correlated networks. The input data comes from the single-cell transcriptome. The input channel in the upper part of the figure uses the scRNA-seq expression data of a specific single cell, and the input channel in the lower part uses a highly correlated network constructed from other cells that are most relevant to the specific single cell. These two channels encode the data through the sequence encoder and the graph encoder respectively. The encoded data features are spliced ​​and sent to the backbone network - the global learning layer; the global learning layer further processes the data, disassembles the joint feature data and sends them to the sequence decoder and the graph decoder respectively. After the decoder reconstructs the data, it calculates the loss between the input original data and the reconstructed data, thereby optimizing the entire framework's learning ability for single-cell transcriptome data; through this self-supervised learning method, the scTCHCN framework can effectively integrate and utilize single-cell transcriptome data and highly correlated network information to improve the analysis and annotation performance of single-cell data, refer to Figure 1 , this embodiment specifically includes the following contents:

[0074] Step S1: Reference Figure 3 , constructing a cell-specific high correlation network, which specifically includes the following sub-steps:

[0075] Step S1.1: In order to capture the high correlation between cells in the scRNA-seq data, extract its gene expression data from the scRNA-seq matrix to calculate the cosine similarity between cells:

[0076]

[0077] Among them, CosineSimilarity(x i ,x j ) is the cosine similarity between cells, x i is the gene expression vector of the ith cell, x j is the gene expression vector of the jth cell.

[0078] Step S1.2, select T cells related to the ith cell to form a sub-network containing T+1 nodes, where the nodes represent cells and the edges represent the cosine similarity between cells;

[0079] As a preferred embodiment of this embodiment, this embodiment selects 19 cells that are most relevant to the ith cell to form a sub-network containing 20 nodes.

[0080] Step S1.3, calculate the cosine similarity between each of the 20 nodes to obtain a 20×20 cosine similarity matrix S;

[0081] Step S1.4: Based on the cosine similarity matrix S, the Kruskal algorithm is used to construct a cell-specific high correlation network, which is specifically:

[0082] The cosine similarity matrix S is solved for the most relevant relationship between the two nodes, and the maximum spanning tree is constructed. The maximum spanning tree is a subgraph containing all nodes, and the sum of the weights of its edges is the largest. The Kruskal algorithm is used to construct the maximum spanning tree. The construction process is as follows:

[0083] Sort all edges by cosine similarity from largest to smallest.

[0084] Select edges from the sorted edge set one by one, making sure that the selected edges do not form a cycle, until 19 edges are selected.

[0085] Finally, 20 nodes are connected in series through the maximum spanning tree to form a sparse network that captures the high correlation between cells. The construction process of the network is:

[0086]

[0087] Among them, MST is a cell-specific high correlation network, which means the maximum spanning tree; subgraph is a subgraph; S hl is an element in the cosine similarity matrix, representing the cosine similarity between the hth cell and the lth cell

[0088] Step S2: In order to ensure that the model can take into account both the gene expression of the cell and its interaction with similar cells, the single-cell transcriptome data tensor and the splicing tensor of each node in the cell-specific high correlation network are spliced ​​to obtain spliced ​​data, which is specifically:

[0089] X input =Concatenate(X scRNA-seq ,X correlation )

[0090] Among them, X input is the splicing data; XscRNA-seq is the single-cell transcriptome data tensor; X correlation It is the tensor concatenation of each node in the cell-specific highly correlated network; Concatenate() is the tensor concatenation function.

[0091] Step S3: Send the spliced ​​data to the global learning layer for self-supervised distillation learning to extract feature information of scRNA-seq data, which specifically includes the following contents:

[0092] The spliced ​​data X input Enter the teacher model in the global learning layer, which uses the global attention mechanism of the scLinformer model to learn the global feature representation of the input data:

[0093]

[0094] in, is the data assigned to the teacher model after passing through the global attention layer; GlobalAttention() is the global attention layer function;

[0095] In this way, the teacher model of this embodiment can capture important associations and global information in the data, and provide effective feature representation for the subsequent student model;

[0096] The student model is designed to be a scLinformer model with smaller parameters to reduce computational complexity. The output of the student model in the global learning layer is:

[0097] Z student =StudentModel(0.5X input +0.5X scRNA-seq )

[0098] Among them, Z student is the data passed through the student model; StudentModel() is the function in the student model that processes the tensor data;

[0099] Its ability to learn the complex cell phenotype features output by the teacher model is optimized by minimizing the mean square error loss function:

[0100]

[0101] in, is the mean square error loss function; N is the total number of cell samples; is the i-th data after the student model; ‖·‖ 2 is the square norm of the vector;

[0102] The Minkowski distance loss function is used to calculate the student model output Z studentTensorX with single-cell transcriptome data scRNA-seq Consistency between:

[0103]

[0104] in, is the Minkowski distance loss function; is the input raw scRNA-seq data; ‖·‖ p represents the Minkowski norm, and p is a hyperparameter, usually 2, which represents the Euclidean distance. The reference of the mean square error loss function and the Minkowski distance loss function enables the student model to retain the original gene expression information as much as possible, while optimizing its learning ability of complex cell phenotypic characteristics through the guidance of the teacher model.

[0105] This embodiment is based on the design of Linformer (a model architecture improved based on the Transformer model) and builds a scLinformer model for processing scRNA-seq data (a model architecture specifically proposed for single-cell RNA sequencing data based on the Linformer model). This model uses low-rank approximation to reduce the computational complexity of the self-attention mechanism to linear, thereby optimizing the performance of the Transformer model.

[0106] In the standard Transformer, the attention calculation expression is:

[0107]

[0108] Among them, Attention(Q,K,V) is the data after attention; softmax() is the activation function; are query, key, and value matrices, respectively, and d k are the dimensions of keys and values; E,F∈R k×n They are all projection matrices, n and k are the lengths of the E and F dimensions respectively.

[0109] In this embodiment, two projection matrices E and F are introduced in the scLinformer model, and their size is n×k, where k<<n. Through these projection matrices, the keys and values ​​can be projected into a low-dimensional space, and the reconstructed attention mechanism is:

[0110]

[0111] Among them, Attention(Q,K,V) is; softmax() is; are query, key, and value matrices, respectively, and d kare the dimensions of keys and values; E,F∈R k×n Both are projection matrices.

[0112] In this embodiment, the projection matrix E,F∈R k×n The learning of , which significantly reduces the computational complexity of processing scRNA-seq data, that is, the matrix after projection EK is denoted as K', and FV is denoted as V'. This allows the scTCHCN framework to make extensive use of the scLinformer model to capture the potential structure of sequence data. The training target domain is the same as the traditional Transformer, but the computing and memory consumption requirements are greatly reduced.

[0113] This embodiment uses a pre-trained student model through self-supervised distillation learning to extract features from scRNA-seq data. The student model learns rich multi-level feature representations from the data through distillation learning and extracts feature information of scRNA-seq data, which is specifically expressed as:

[0114] z i =StudentModel(x i )

[0115] Among them, z i ∈R d Represents the d-dimensional feature vector of the i-th cell.

[0116] Step S4: Based on the feature information, the KAN model is used to annotate the cell types of the scRNA-seq data, referring to Figure 4 and Figure 5 , which specifically includes the following:

[0117] This embodiment uses the KAN model as the classifier model. The KAN model is a model that replaces the multi-layer perceptron (MLP) and has stronger expression ability based on the Kolmogorov-Arnold representation theorem. The theorem shows that any continuous function can be represented as a finite superposition of a series of one-dimensional continuous functions.

[0118] Specifically, the Kolmogorov-Arnold representation theorem states that there exist several one-dimensional continuous functions φ j and ψ jk , so that any continuous function f can be expressed as:

[0119]

[0120] In the KAN model, a similar representation method is used, where the d-dimensional feature vector z of the i-th cell is i Mapping to predicted cell types

[0121]

[0122] Among them, f(z i ) is the cell type prediction function; φ g and ψ gs is the learned nonlinear transformation function, m and p are the parameters of the KAN model respectively; g is the index of the g-th feature activation function; s is the s-th input index of the g-th specific feature activation function.

[0123] Specifically, φ g and ψ gs It can be expressed as a linear transformation plus a nonlinear activation function:

[0124]

[0125] Among them, B gs and B g is the spline basis function, and u is the eigenvector.

[0126] Finally, the classifier model will output the prediction of cell type The prediction result for the i-th cell and the true label y i , the prediction model is optimized using the cross entropy loss function, which is:

[0127]

[0128] Among them, N is the number of cell samples, C is the number of cell type categories, and y i,c Indicates that the i-th sample belongs to the true label of category c, It represents the probability that the i-th sample is predicted to be of category c.

[0129] By minimizing the cross entropy loss function, the parameters of the classifier model are optimized to improve the accuracy of cell type prediction. The optimization process can be expressed as:

[0130]

[0131] Among them, θ is the parameter of the classifier, θ * is the optimized classifier parameter; is the cell type predicted by the classifier; y is the actual cell type.

[0132] Example 2

[0133] In order to verify the method in Example 1 of the present invention, this example conducted experiments in the task of single cell type annotation, specifically including the following five aspects:

[0134] First, data preparation;

[0135] The experimental training is divided into two stages: the first stage is self-supervised pre-training of a large amount of unlabeled data; the second stage is supervised training of data containing labeled cells to achieve efficient cell type annotation. In order to avoid data leakage, the data used in these two stages come from different sources. The data used for self-supervised distillation comes from the PanglaoDB website (a foreign website for collecting single-cell RNA sequencing databases), which integrates 305 sample data sets covering 74 human tissues and a total of 1,126,580 single cells. These data come from different experimental sequencing and are integrated according to different sequencing standards. In the supervised classification task of scTCHCN, the following 4 data sets were used for detailed analysis: 10X-Multiome, ISSAAC-seq, SHARE-seq, and Zheng68k.

[0136] Second, for cell type prediction;

[0137] The method of this embodiment is compared with three supervised classification methods (CIForm, scTransSort and scGAD) and a marker gene-based method Seurat. The t-SNE diagram of the expert annotation results and the best prediction results of the present invention is shown in Figure 2. Figure 6 As shown. Table 1 shows the mean and standard deviation of Accuracy and F1-score of each method under five-fold cross validation. The results show that the Transformer-based supervised classification methods (CIForm and scTransSort) outperform other traditional supervised classification methods scGAD and marker gene methods Seurat. Among them, the scTCHCN framework method proposed in the present invention performed best on three data sets (10X-Multiome, ISSAAC-seq, and SHARE-seq). For example, in the 10X-Multiome data set, the average values ​​of Accuracy and F1-score of scTCHCN were 0.962±3.7e-3 and 0.962±3.6e-3, respectively, which were the best among all methods.

[0138] Table 1. Performance evaluation of SCTCHCN and other methods. The table shows the mean and standard deviation of accuracy and F1-score of each method;

[0139]

[0140] Third, for the identification of rare cells;

[0141] The scTCHCN proposed in the present invention and two other excellent Transformer-based comparison methods (CIForm and scTransSort) were evaluated on the Zheng68k dataset for rare cell recognition capabilities. In this dataset, the distribution of cell types is unbalanced, and there is a strong correlation between cell types, which increases the difficulty of distinction. Therefore, the present invention lists cell types that account for less than 5.0% of the dataset as rare cell types for comparative experiments. As can be seen from Table 2, scTCHCN occupies two-thirds of the types in the recognition of rare cell types to achieve the highest recognition accuracy. And the average recognition accuracy of all rare cell types is 0.673, leading the second-place comparison method CIForm by 6.3%. These results show that scTCHCN exhibits excellent performance on datasets with imbalanced categories.

[0142] Table 2. Performance evaluation of different methods on rare cell types on the Zheng68K dataset. The table shows the cell counting rate and accuracy of each method.

[0143]

[0144] Fourth, verify the students’ model feature extraction capabilities;

[0145] In order to verify the effectiveness of the student model in extracting deep features of transcriptome data, three clustering score indicators, NMI (Normalized Mutual Information), ARI (Adjusted Rand Index) and Purity (clustering purity), were used to compare and analyze the clustering performance of the original scRNA-seq data and the data extracted by the student model. Figure 7 It can be observed that after the student model feature extraction, the NMI, ARI and Purity curves of the data all show an upward trend. This shows that the student model has significantly improved the feature extraction ability of scRNA-seq data, thereby facilitating the analysis of downstream tasks such as cell type annotation, and further verifies the success of the scTCHCN model in self-supervised distillation learning of transcriptome data.

[0146] Fifth, the best performing model architecture for predicting cell types;

[0147] This experiment also selected the Zheng68k dataset with an unbalanced category distribution to explore the optimal architecture of the model. Through five-fold cross-validation, the prediction results of the final prediction model scTCHCN and other variants were obtained, as shown in Table 3. The combination of (student model + KAN) performed best, with an accuracy of 0.8% higher than the second best combination of (student model + MLP (multi-layer perceptron)) and an F1-score of 9.2% higher than the second best combination of (Transformer + KAN). The experimental results show that the student model and KAN neural network combined with self-supervised distillation learning of cell-specific high correlation networks do have a significant improvement in improving the architecture of cell type prediction models. This further shows that this paradigm has the potential to be extended to other downstream tasks.

[0148] Table 3. Module ablation study results on the Zheng68k dataset - use is indicated by a check mark (√), and non-use is indicated by a cross (×).

[0149] The table shows the mean accuracy and F1-score along with their standard deviation.

[0150]

[0151] In summary, the present invention uses scTCHCN as a downstream task model for deep feature extraction of scRNA-seq data, which can improve the versatility of the model, overcome the noise impact caused by batch effects, and provide highly scalable, robust and accurate capabilities in cell type annotation. In short, scTCHCN is a framework for using cell-level specific high correlation networks in self-supervised distillation learning tasks of transcriptome data, which provides unlimited possibilities for more supervised fine-tuning downstream tasks based on transcriptome data in the future.

[0152] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A cell type annotation method based on self-supervised distillation learning, characterized in that: The following steps are involved: S1. Construct a cell-specific high correlation network based on the cosine similarity between cells; S2, splicing the single-cell transcriptome data tensor and the splicing tensor of each node in the cell-specific high correlation network to obtain spliced ​​data; S3, sending the spliced ​​data to the global learning layer for self-supervised distillation learning to extract the feature information of scRNA-seq data; S4. Based on the feature information, the KAN model is used to annotate the cell types of the scRNA-seq data.

2. The cell type annotation method based on self-supervised distillation learning according to claim 1, characterized in that: The S1 specifically includes the following sub-steps: S1.

1. Calculate the cosine similarity between cells: Among them, CosineSimilarity(x i ,x j ) is the cosine similarity between cells, x i is the gene expression vector of the ith cell, x j is the gene expression vector of the jth cell; S1.2, select T cells related to the ith cell to form a sub-network containing T+1 nodes, where the nodes represent cells and the edges represent the cosine similarity between cells; S1.3, calculating the cosine similarity between two nodes, and constructing a cosine similarity matrix according to the calculated cosine similarity; S1.

4. Based on the cosine similarity matrix, a cell-specific high correlation network is constructed using the Kruskal algorithm.

3. The cell type annotation method based on self-supervised distillation learning according to claim 1, characterized in that: The Kruskal algorithm is used in S1.4 to construct a cell-specific high correlation network, including: Among them, MST is a cell-specific high correlation network, which means the maximum spanning tree; subgraph is a subgraph; S hl is an element in the cosine similarity matrix, representing the cosine similarity between the hth cell and the lth cell.

4. The cell type annotation method based on self-supervised distillation learning according to claim 1, characterized in that: The splicing data obtained in S2 specifically includes: X input =Concatenate(X scRNA-seq ,X correlation ) Among them, X input is the splicing data; X scRNA-seq is the single-cell transcriptome data tensor; X correlation It is the tensor concatenation of each node in the cell-specific highly correlated network; Concatenate() is the tensor concatenation function.

5. The cell type annotation method based on self-supervised distillation learning according to claim 4, characterized in that: In S3, the concatenated data is sent to the global learning layer for self-supervised distillation learning, including: The spliced ​​data X input Enter the teacher model in the global learning layer, which uses the global attention mechanism of the scLinformer model to learn the global feature representation of the input data: in, is the data assigned to the teacher model after passing through the global attention layer; GlobalAttention() is the global attention layer function; The output of the student model in the global learning layer is: From student =StudentModel(0.5X input +0.5X scRNA-seq ) Among them, Z student is the data passed through the student model; StudentModel() is the function in the student model that processes the tensor data; The teacher model is optimized by minimizing the mean square error loss function: in, is the mean square error loss function; N is the total number of cell samples; is the i-th data after the student model; ||·|| 2 is the square norm of the vector; The Minkowski distance loss function is used to calculate the student model output Z student TensorX with single-cell transcriptome data scRNA-seq Consistency between: in, is the Minkowski distance loss function; is the input raw scRNA-seq data; ||·|| p represents the Minkowski norm.

6. The cell type annotation method based on self-supervised distillation learning according to claim 5, characterized in that: The global attention mechanism of the scLinformer model is: Among them, Attention(Q, K, V) is the data after attention; softmax() is the activation function; are query, key, and value matrices, respectively, and d k are the dimensions of keys and values; E, F∈R k×n They are all projection matrices, n and k are the lengths of the E and F dimensions respectively.

7. The cell type annotation method based on self-supervised distillation learning according to claim 5, characterized in that: The feature information of scRNA-seq data extracted in S3 includes: from i =StudentModel(x i ) Among them, z i ∈R d Represents the d-dimensional feature vector of the i-th cell.

8. The cell type annotation method based on self-supervised distillation learning according to claim 1, characterized in that: The S4 specifically includes: In the KAN model, the d-dimensional feature vector z of the i-th cell is i Mapping to predicted cell types Among them, f(z i ) is the cell type prediction function; φ g and ψ gs is the learned nonlinear transformation function, m and p are the parameters of the KAN model, g is the index of the g-th feature activation function, s is the s-th input index of the g-th specific feature activation function; The cross entropy loss function is used to optimize the KAN model: Among them, N is the number of cell samples, C is the number of cell type categories, and y i，c Indicates that the i-th sample belongs to the true label of category c, represents the probability that the i-th sample is predicted to be of category c, L cross is the cross entropy loss function; By minimizing the cross entropy loss function, the parameters of the KAN model are optimized: Among them, θ is the parameter of the classifier, and θ* is the optimized classifier parameter; is the cell type predicted by the classifier; y is the actual cell type.

9. The cell type annotation method based on self-supervised distillation learning according to claim 8, characterized in that: φ g and ψ gs Represented as a linear transformation plus a nonlinear activation function: Among them, B gs and B g is the spline basis function, and u is the eigenvector.

Images