A method for constructing a large model of a heart

Abstract

This invention discloses a method for constructing a large-scale cardiac model, belonging to the field of bioinformatics. It addresses the problem of improving the model's ability to analyze cell subpopulations in cardiac tissue. The method involves collecting human cardiac single-cell atlas data, preprocessing and labeling it to obtain a cardiac reference dataset, and constructing a gene expression matrix for the reference data. A base model is selected and fine-tuned using the gene expression matrix of the reference data, with the fine-tuned model serving as the encoder. Cell embeddings are extracted and randomly inactivated, resulting in randomly inactivated cell embeddings that are input into a fully connected network. The output is a fully connected network-processed cell embedding, which is then input into a hierarchical classifier for classification. A loss function is constructed to train the hierarchical classifier, thus completing the construction of a large-scale cardiac model. This invention enhances the ability to analyze cell subpopulations in cardiac tissue.

Description

Technical Field

[0001] This invention belongs to the field of bioinformatics technology, specifically relating to a method for constructing a large model of the heart. Background Technology

[0002] Single-cell basic models can effectively learn the potential associations of gene expression in different cell types from single-cell transcriptome data, demonstrating excellent performance in cell type prediction. However, existing models have the following limitations and shortcomings:

[0003] 1. Lack of heart-specific annotation models. Existing large models (such as scBERT, Geneformer, scKEPLM, etc.) are pre-trained on whole-body tissues, and are not specifically trained for heart tissue.

[0004] 2. Lack of high-precision cell subpopulation annotation models. Existing machine learning annotation techniques ignore the hierarchical structure of cell types, leading to inconsistent annotation results at different levels. Current single-cell annotation methods have limitations in resolving fine-grained cell subpopulations. Due to the high similarity of expression features among similar cell subpopulations, the task of accurate cell typing places significantly higher demands on the model's feature discrimination and fine-grained representation learning capabilities. These basic models can only be fine-tuned based on a single level of annotation labels, ignoring the hierarchical structure inherent in cell annotation itself. Summary of the Invention

[0005] The problem this invention aims to solve is to improve the model's ability to analyze cell subpopulations in cardiac tissue, and proposes a method for constructing a large cardiac model.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for constructing a large cardiac model includes the following steps:

[0008] S1. Collect human heart single-cell atlas data, perform preprocessing and labeling to obtain a heart reference dataset, and obtain the gene expression matrix of the reference data. Each row is a cell, each column is a gene, and the value is the original expression value of the gene in that cell.

[0009] S2. Select a base model, including the BERT model scBERT applied to single-cell data, the Geneformer model based on the Transformer architecture for processing gene data, and the single-cell knowledge-enhanced pre-trained language model scKEPLM. Use the gene expression matrix of the reference data obtained in step S1 to fine-tune the base model respectively, and use the fine-tuned model as the encoder.

[0010] S3. Input the gene expression matrix of the reference data obtained in step S1 into the encoder obtained in step S2, extract the cell embedding, and perform random inactivation processing on the cell embedding, that is, by independently setting each number of the cell embedding to zero with a probability of 0.2, the cell embedding after random inactivation processing is obtained.

[0011] S4. Input the cell embeddings obtained in step S3 after random inactivation into a fully connected network. The fully connected network consists of two fully connected layers. Batch normalization, ReLU activation and random inactivation are performed after each fully connected layer. The cell embeddings processed by the fully connected network are output.

[0012] S5. The cell embeddings obtained in step S4 are input into a hierarchical classifier for classification. The hierarchical classifier includes a first-level classifier head, a second-level classifier head, and a third-level classifier head. Then, standard cross-entropy loss functions are constructed for each of the three classifier heads and trained to complete the construction of a large heart model.

[0013] Furthermore, in step S1, the study by Kanemaru et al. is used as the cardiac reference dataset; the cell level is divided into 3 levels, and 8 primary cell type labels, 19 secondary cell type labels, and 62 tertiary cell type labels are established; then the gene expression matrix of the reference data is obtained.

[0014] Furthermore, the method for fine-tuning the base model in step S2 is to freeze all parameters of the normalization layer and the penultimate layer parameters of the Performer attention module for scBERT; freeze two layers of transformer parameters for Geneformer; and freeze the first two layers of BERT, the first two layers of knowledge graph embedding layer, and the first two layers of cross attention layer parameters for scKEPLM, with the remaining parameters participating in gradient updates.

[0015] Furthermore, the specific implementation method of step S3 is as follows: using the fine-tuned model as an encoder, inputting the gene expression matrix of the reference data, sequentially extracting cell embeddings from the three encoders as cell representations, and then performing random dropout on the cell embeddings, as expressed by:

[0016]

[0017]

[0018] Among them, X ref This is the gene expression matrix of the reference data, M fine-tune It is a model finely tuned based on different base models; It is the cell embedding output from the fine-tuned base model; It is a random deactivation function; p dropIt is the probability of dropout. It is the cell embedding after random dropout.

[0019] Furthermore, the expression for outputting the cell embedding in step S4 is:

[0020]

[0021]

[0022] in, , These are the weights of the first fully connected layer and the weights of the second fully connected layer, respectively. , These are the biases of the first fully connected layer and the second fully connected layer, respectively. Representatives criticized normalization. Represents ReLU activation. Represents dropout. yes Cell embedding after the first fully connected layer It is the cell embedding of h after the second fully connected layer.

[0023] Furthermore, the specific implementation method of step S5 includes the following steps:

[0024] S5.1. Perform first-level classification on the cell embeddings obtained in step S4. The embedding features are output as an 8-dimensional vector through a linear layer. The first-level probability distribution is obtained through softmax. The category with the highest probability is selected as the first-level prediction result.

[0025] S5.2. After primary classification, the cell embeddings obtained in step S4 are further classified into secondary categories. Each primary category has an independent secondary branch. The embedded features are processed through a linear layer to output a 19-dimensional vector. Based on the primary classification results, the vector values ​​corresponding to the secondary branches of non-primary categories are set to a minimum value of -10. 9 This makes the probability of the class approach zero during the softmax process, and selects the class with the highest probability as the secondary prediction result.

[0026] S5.3. After secondary classification of the cell embeddings obtained in step S4, a one-dimensional convolutional neural network is introduced to enhance the features. Through convolution, batch normalization, ReLU activation, and max pooling operations, local feature patterns are extracted for tertiary classification. The expression is as follows:

[0027]

[0028] in, , These represent convolution, batch normalization, activation, max pooling, and flattening, respectively. yes Feature-enhanced embedding;

[0029] A contrastive learning strategy is used at the three-level classification head. For each anchor cell, cells in the same subgroup are classified as positive sample cells, and cells in different subgroups are classified as negative sample cells. The loss function for contrastive learning is constructed as follows:

[0030]

[0031] in, It is cell embedding after projection. It is the cell embedding of a positive sample with the same third-level label as cell i. τ is the cell embedding of a negative sample from a different subpopulation than cell i, where τ is the temperature coefficient and N is the number of cells in the uniform batch. It is the loss function for contrastive learning. The set of positive samples with the same third-level label for cell i;

[0032] S5.4. Loss Function of the Model The sum of the cross-entropy loss function of the three-level classifiers and the loss function of contrastive learning is expressed as:

[0033]

[0034]

[0035] Where N is the total number of cells in a batch, C i It is the number of categories for the i-th task. It is the logit value of the nth cell belonging to category c on the i-th task. It is the logit of the nth cell corresponding to the true label category on the i-th task; The cross-entropy loss corresponds to the first-level, second-level, and third-level classifications, respectively. To compare the losses These are the parameters of the contrastive learning loss function.

[0036] The beneficial effects of this invention are:

[0037] The method for constructing a large cardiac model described in this invention improves the three-level classification metrics of the optimized model. The accuracy of GeneFormer increases from 77.66% to 86.31%, the accuracy of scBERT increases from 77.93% to 79.39%, and the accuracy of scKEPLM increases from 73.56% to 77.43%, enhancing the ability of these three models to analyze cell subpopulations in cardiac tissue. Attached Figure Description

[0038] Figure 1 This is a diagram showing the processing of the reference dataset for this invention;

[0039] Figure 2 This is an architectural diagram of a method for constructing a large cardiac model according to the present invention;

[0040] Figure 3 This is a performance comparison of the data integration capabilities of the model of this invention;

[0041] Figure 4 This is a comparison chart of the cell type annotation performance of the model in this invention;

[0042] Figure 5 This is a flowchart of a method for constructing a large cardiac model according to the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0044] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0045] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 - Appendix Figure 5 Detailed explanation is as follows:

[0046] Example 1:

[0047] A method for constructing a large cardiac model includes the following steps:

[0048] S1. Collect human heart single-cell atlas data, perform preprocessing and labeling to obtain a heart reference dataset, and obtain the gene expression matrix of the reference data. Each row is a cell, each column is a gene, and the value is the original expression value of the gene in that cell.

[0049] Furthermore, in step S1, the study by Kanemaru et al. is used as the cardiac reference dataset; the cell level is divided into 3 levels, and 8 primary cell type labels, 19 secondary cell type labels, and 62 tertiary cell type labels are established; then the cell-gene expression matrix of the reference data is obtained.

[0050] Furthermore, this reference dataset includes 25 healthy donors with no history of heart disease or arrhythmia, aged between 20 and 75 years. Sampling locations covered all eight regions of the heart (left and right ventricles, left and right atria, interventricular septum, atrial septum, sinoatrial node, and atrioventricular node). Sequencing was performed using the 10X Genomics method. After sample processing and quality control, a total of 704,296 cells and 32,732 genes were obtained, constructing a high-resolution cardiac reference atlas with complete cell annotation. This patent standardizes and classifies the cell types, resulting in three levels of cell type tags (…). Figure 1 Cells with fewer than 100 tertiary cell subpopulation tags and those marked "unclassified" were removed, resulting in 689,998 cells with clearly defined identities for training. Eight primary cell type tags were obtained, indicating different cell identities and origins (including adipocytes, cardiomyocytes, endothelial cells, fibroblasts, lymphocytes, myeloid cells, neurons, and vascular wall cells); 19 secondary cell type tags, based on these primary tags, provided preliminary functional classification (e.g., further subdividing lymphocytes into innate lymphocytes, B cells, and T cells; cardiomyocytes into atrial cardiomyocytes, ventricular cardiomyocytes, and conduction cells; and myeloid cells into dendritic cells, mast cells, macrophages, and monocytes); and 62 further detailed tertiary cell type tags were obtained from these secondary classifications, representing more refined subpopulations with higher functional specificity (e.g., further subdividing B cells into B cells and plasma cells; and T cells into CD4 and CD8 expressing different characteristic genes or performing different functions). T cells, such as CD8 toxic T cells and CD8 effector memory T cells; innate lymphocytes are divided into classical NK cells, non-classical NK cells, and ILC cells; myocardial conduction cells are further divided into Purkinje cells, atrioventricular node conduction cells, and sinoatrial node conduction cells; endothelial cells are further divided into endocardial endothelium, vascular endothelium, capillary endothelium, venous endothelium, arterial endothelium, and lymphatic endothelium.

[0051] S2. Select a base model, including the BERT model scBERT applied to single-cell data, the Geneformer model based on the Transformer architecture for processing gene data, and the single-cell knowledge-enhanced pre-trained language model scKEPLM. Use the gene expression matrix of the reference data obtained in step S1 to fine-tune the base model respectively, and use the fine-tuned model as the encoder.

[0052] Furthermore, the method for fine-tuning the base model in step S2 is to freeze all parameters of the normalization layer and the penultimate layer parameters of the Performer attention module for scBERT; freeze two layers of transformer parameters for Geneformer; and freeze the first two layers of BERT, the first two layers of knowledge graph embedding layer, and the first two layers of cross attention layer parameters for scKEPLM, with the remaining parameters participating in gradient updates.

[0053] Furthermore, the reference dataset was randomly divided according to cell type, with 80% of the data used for training and 20% for testing. Results showed that the average accuracy of these basic models at levels 1 and 2 both reached over 97%, while the average accuracy at level 3 was only around 75%. Geneformer achieved an accuracy of 77.66%, scBERT 77.93%, and scKEPLM 73.56%. This indicates that these basic models can effectively resolve the major cell types in cardiac tissue, but their performance in predicting cell subpopulations is worse. To construct a high-precision cardiac subpopulation annotation model, existing large language models were optimized (…). Figure 2 After inputting the gene expression matrix of cardiac reference data into the base model, the model is fine-tuned based on cell type labels to learn the data distribution characteristics specific to cardiac tissue. To compare the performance of different models, the fine-tuned model is used as a data encoder, and the gene expression matrix of the reference data is input. Cell embeddings are extracted sequentially from three encoders as cell representations.

[0054] S3. Input the gene expression matrix of the reference data obtained in step S1 into the encoder obtained in step S2 to extract the cell embeddings. To improve the robustness and effectiveness of the model, the cell embeddings are randomly deactivated, i.e., each number in the cell embedding is independently set to zero with a probability of 0.2 to obtain the randomly deactivated cell embeddings.

[0055] Furthermore, the specific implementation method of step S3 is as follows: using the fine-tuned model as an encoder, inputting the gene expression matrix of the reference data, sequentially extracting cell embeddings from the three encoders as cell representations, and then performing random dropout on the cell embeddings, as expressed by:

[0056]

[0057]

[0058] Among them, X ref This is the gene expression matrix of the reference data, M fine-tune It is a model finely tuned based on different base models; It is the cell embedding output from the fine-tuned base model; It is a random deactivation function; p drop It is the probability of dropout. It is the cell embedding after random dropout.

[0059] S4. Input the cell embeddings after random inactivation treatment obtained in step S3 into a fully connected network. The fully connected network consists of two fully connected layers. After each fully connected layer, batch normalization, ReLU activation and random inactivation treatment are performed, and the cell embeddings processed by the fully connected network are output.

[0060] Furthermore, the expression for outputting the cell embedding in step S4 is:

[0061]

[0062]

[0063] in, , These are the weights of the first fully connected layer and the weights of the second fully connected layer, respectively. , These are the biases of the first fully connected layer and the second fully connected layer, respectively. Representatives criticized normalization. Represents ReLU activation. Represents dropout. yes Cell embedding after the first fully connected layer It is the cell embedding of h after the second fully connected layer.

[0064] S5. The cell embeddings output in step S4 are input into a hierarchical classifier for classification. The hierarchical classifier includes a first-level classifier head, a second-level classifier head, and a third-level classifier head. Then, a standard cross-entropy loss function is constructed for each of the three classifier heads for training, thus completing the construction of a large cardiac model.

[0065] Furthermore, the specific implementation method of step S5 includes the following steps:

[0066] S5.1. Perform first-level classification on the cell embedding features obtained in step S4. The embedding features are passed through a linear layer to output an 8-dimensional vector. The first-level probability distribution is obtained through softmax, and the category with the highest probability is selected as the first-level prediction result.

[0067] S5.2. Based on the cell embedding features obtained in step S4, perform secondary classification after primary classification; each primary category has an independent secondary branch. The embedding features are processed through a linear layer to output a 19-dimensional vector. According to the results of primary classification, the vector values ​​corresponding to the secondary branches that are not classified in the primary category are set to the minimum value -10. 9 This makes the probability of the class approach zero during the softmax process, and selects the class with the highest probability as the secondary prediction result.

[0068] S5.3. After secondary classification of the cell embedding features obtained in step S4, a one-dimensional convolutional neural network is introduced to enhance the features. Through convolution, batch normalization, ReLU activation, and max pooling operations, local feature patterns are extracted for tertiary classification. The expression is as follows:

[0069]

[0070] in, , These represent convolution, batch normalization, activation, max pooling, and flattening, respectively. yes Feature-enhanced embedding;

[0071] A contrastive learning strategy is used at the three-level classification head. For each anchor cell, cells in the same subgroup are classified as positive sample cells, and cells in different subgroups are classified as negative sample cells. The loss function for contrastive learning is constructed as follows:

[0072]

[0073] in, It is cell embedding after projection. It is the cell embedding of a positive sample with the same third-level label as cell i. This refers to cell embedding from a different subpopulation than cell i, where τ is a temperature coefficient and N is the number of cells in the same batch. It is the loss function for contrastive learning. The set of positive samples with the same third-level label for cell i;

[0074] During the prediction process, the effective class range of the next level is dynamically constrained based on the prediction results of the previous level, ensuring that the model only performs classification within the effective subclass space. To better distinguish between different subgroups, a contrastive learning strategy is used at the three-level classification head. For each "anchor cell," cells from the same subgroup are designated as "positive cells," and cells from different subgroups are designated as "negative cells." An InfoNCE loss function is constructed to maximize the similarity between positive sample pairs and minimize the similarity between negative sample pairs.

[0075] S5.4. Loss Function of the Model The sum of the cross-entropy loss function of the three-level classifiers and the loss function of contrastive learning is expressed as:

[0076]

[0077]

[0078] Where N is the total number of cells in a batch, C i It is the number of categories for the i-th task. It is the logit value of the nth cell belonging to category c on the i-th task. It is the logit of the nth cell corresponding to the true label category on the i-th task; The cross-entropy loss corresponds to the first-level, second-level, and third-level classifications, respectively. To compare losses These are the parameters of the contrastive learning loss function.

[0079] Furthermore, by comparing the classification metrics of the base model and the hierarchical classification optimization model, the optimized model subgroup training metrics all improved. The accuracy of Geneformer increased from 77.66% to 86.31%, the accuracy of scBERT increased from 77.93% to 79.39%, and the accuracy of scKEPLM increased from 73.56% to 77.43%, indicating that the optimization method is suitable for these models, as shown in Table 1.

[0080] Table 1

[0081]

[0082] Since Geneformer's level 3 accuracy improved from 77.66% to 86.31%, demonstrating good performance, the optimized Geneformer was used as the benchmark for scCardioFM for performance comparison and downstream applications. First, model benchmarking—integration evaluation and predictive efficacy evaluation—was conducted, comparing scCardioFM with common integration models (geneformer, scanvi, sckeplm, scgpt, scpoli, scvi). The model-generated embeddings showed varying degrees of improvement in biological conservation and batch effect removal. Figure 3 Especially compared to scvi, the model performance shows significant improvement across multiple metrics. Comparing scCardioFM with common cell type annotation models (scanvi, scbert, geneformer, scgpt, sckeplm, scibet, scmap), the model's accuracy, precision, recall, and F1-score are all higher than the other models. Figure 4 ).

[0083] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for constructing a large cardiac model, characterized in that, Includes the following steps: S1. Collect human heart single-cell atlas data, perform preprocessing and labeling. Select a single-cell dataset with complete cell labeling and clear cell hierarchy as a reference dataset, and obtain its gene expression matrix, where each row represents a cell, each column represents a gene, and the value is the original expression value of the gene in that cell; S2. Select a base model, including the BERT model scBERT applied to single-cell data, the Geneformer model based on the Transformer architecture for processing gene data, and the single-cell knowledge-enhanced pre-trained language model scKEPLM. Use the gene expression matrix of the reference data obtained in step S1 to fine-tune the base model respectively, and use the fine-tuned model as the encoder. S3. Input the gene expression matrix of the reference data obtained in step S1 into the encoder obtained in step S2, extract the cell embedding, and perform random inactivation processing on the cell embedding, that is, by independently setting each number of the cell embedding to zero with a probability of 0.2, the cell embedding after random inactivation processing is obtained. S4. Input the cell embeddings obtained in step S3 after random inactivation into a fully connected network. The fully connected network consists of two fully connected layers. After each fully connected layer, batch normalization, ReLU activation and random inactivation are performed, and the cell embeddings of the fully connected network are output. S5. The cell embeddings of the fully connected network obtained in step S4 are input into a hierarchical classifier for classification. The hierarchical classifier includes a first-level classifier head, a second-level classifier head, and a third-level classifier head. Then, a loss function is constructed and a standard cross-entropy loss function is constructed for each of the three classifier heads for training, thus completing the construction of a large heart model.

2. The method for constructing a large cardiac model according to claim 1, characterized in that, In step S1, the study by Kanemaru et al. was used as the cardiac reference dataset; the cell level was divided into 3 levels, and 8 primary cell type labels, 19 secondary cell type labels, and 62 tertiary cell type labels were established; then the gene expression matrix of the reference data was obtained.

3. The method for constructing a large cardiac model according to claim 2, characterized in that, In step S2, the method for fine-tuning the basic model is to freeze all parameters of the normalization layer and the penultimate layer parameters of the Performer attention module for scBERT; freeze two layers of transformer parameters for Geneformer; and freeze the first two layers of BERT, the first two layers of knowledge graph embedding layer, and the first two layers of cross attention layer parameters for scKEPLM, with the remaining parameters participating in gradient updates.

4. The method for constructing a large cardiac model according to claim 3, characterized in that, The specific implementation method of step S3 is as follows: using the fine-tuned model as an encoder, inputting the gene expression matrix of the reference data, extracting cell embeddings sequentially from the three encoders as cell representations, and then performing random dropout on the cell embeddings, as expressed in the following expression: Among them, X ref This is the gene expression matrix of the reference data, M fine-tune It is a model finely tuned based on different base models; It is the cell embedding output from the fine-tuned base model; It is a random deactivation function that independently sets each number to zero with a probability of 0.2 for each cell embedding; p drop It is the probability of dropout. It is the cell embedding after random dropout.

5. The method for constructing a large cardiac model according to claim 4, characterized in that, The expression for the cell embedding processed by the fully connected network in step S4 is as follows: in, , These are the weights of the first fully connected layer and the weights of the second fully connected layer, respectively. , These are the biases of the first fully connected layer and the second fully connected layer, respectively. Representatives criticized normalization. Represents ReLU activation. Represents dropout. yes Cell embedding after the first fully connected layer It is the cell embedding of h after the second fully connected layer.

6. The method for constructing a large cardiac model according to claim 5, characterized in that, The specific implementation method of step S5 includes the following steps: S5.

1. Perform first-level classification on the cell embeddings obtained in step S4. The embedding features are passed through a linear layer to output an 8-dimensional vector. The first-level probability distribution is obtained through softmax, and the category with the highest probability is selected as the first-level prediction result. S5.

2. Perform secondary classification on the cell embeddings obtained in step S4; each primary category has an independent secondary branch. The embedded features are output as a 19-dimensional vector through a linear layer. Based on the primary classification results, the vector values ​​corresponding to the secondary branches of non-primary categories are set to the minimum value -10. 9 This makes the probability of the class approach zero during the softmax process, and selects the class with the highest probability as the secondary prediction result. S5.

3. For the cell embeddings obtained in step S4, a one-dimensional convolutional neural network is introduced to enhance the features. Through convolution, batch normalization, ReLU activation, and max pooling operations, local feature patterns are extracted and a three-level classification is performed. The expression is: in, , , These represent convolution, batch normalization, activation, max pooling, and flattening, respectively. yes Embedding after feature enhancement; A contrastive learning strategy is used at the three-level classification head. For each anchor cell, cells in the same subgroup are classified as positive sample cells, and cells in different subgroups are classified as negative sample cells. The loss function for contrastive learning is constructed as follows: in, It is cell embedding after projection. It is a cell embedding that is identical to the cell's third-level tag. This refers to cell embedding from a different subpopulation than cell i, where τ is a temperature coefficient and N is the number of cells in the same batch. It is the loss function for contrastive learning. The set of positive samples with the same third-level label for cell i; S5.

4. Loss Function of the Model The sum of the cross-entropy loss function of the three-level classifiers and the loss function of contrastive learning is expressed as: Where N is the total number of cells in a batch, C i It is the number of categories for the i-th task. It is the logit value of the nth cell belonging to category c on the i-th task. It is the logit of the nth cell corresponding to the true label category on the i-th task; The cross-entropy loss corresponds to the first-level, second-level, and third-level classifications, respectively. To compare the losses, These are the parameters of the contrastive learning loss function.

Images