Single cell type annotation model based on pre-trained virtual adversarial domain adaptation network

Abstract

The application discloses a single cell type annotation model pscAdapt based on a pre-training virtual adversarial domain self-adaptive network. The application firstly proposes a pre-training strategy to complete initialization of model parameters to learn data distribution of a source domain, combines an adversarial learning strategy to train a domain self-adaptive network to realize domain-level alignment and reduce differences between adjacent domains. Secondly, a structural similarity loss is designed to realize cell-level alignment and enhance the distinguishability of cell categories. Finally, a virtual adversarial training module is used to add random disturbance to generate adversarial samples to enhance the generalization ability of the model. Experiments are conducted on 44 data sets to verify the effectiveness of the pscAdapt model. The experimental results show that the proposed pscAdapt model is superior to current popular cell type annotation models in terms of cell type annotation and batch correction.

Description

Technical Field

[0001] This invention relates to the field of data mining, specifically to a single-cell type annotation model based on a pre-trained virtual adversarial domain adaptive network. Background Technology

[0002] With the rapid development of single-cell technology, abundant single-cell RNA sequencing data helps to gain a deeper understanding of cellular differences and functions at the molecular level, revealing important mechanisms of cells in development, disease, and treatment. However, most current single-cell data is unlabeled; therefore, it is crucial to annotate unlabeled single-cell data using existing labeled single-cell data. Single-cell type annotation can discover new cell types, subtypes, and states, as well as the characteristics and functions of these cells, providing more accurate information for disease diagnosis and treatment.

[0003] Traditional single-cell annotation tasks are often completed based on professional knowledge or experience, making them susceptible to the influence of personal subjectivity and experience, which in turn affects downstream analysis tasks. Currently, many supervised methods have been developed to automate cell annotation tasks. Existing cell annotation models can be broadly classified into three categories: similarity-based models, machine learning-based models, and deep learning-based models.

[0004] Similarity-based models primarily annotate cell types by comparing the correlation between individual cells in the source data and cell features in the target data. SingleR calculates the similarity between the target and source data using Spearman correlation based on source data with known labels, thus completing the target data annotation. SCMAP combines cosine distance, Pearson correlation, and Spearman correlation based on source data with known labels to calculate the similarity between the target and source data, thereby completing the cell annotation of the target data. The CelliD algorithm's main idea is to use multiple correspondence analysis to map the source data into a low-dimensional space and complete the cell annotation of the target data based on a pre-established list of markers. Similarity-based models have advantages such as computational simplicity and speed, but they cannot consider complex nonlinear relationships between genes and are sensitive to batch effects. Machine learning-based models mainly use machine learning algorithms to train the model and use the trained model to complete the cell type annotation task. SingleCellNet converts the source data into a binary matrix to train a random forest classifier, and uses the trained classifier to complete the target data annotation task. Seurat maps source and target data into a low-dimensional space using PCA or CCA methods. It then uses the MNN algorithm to find the nearest neighbor cells (anchor cells) between two different batches of data. Based on these anchor cells, a weighted voting classifier is trained using the known labels of the source data to annotate the cells in the target data. scClassify calculates the similarity between the target data to be classified and the source data cells of known categories using the Spearman correlation coefficient. For each cell in the target data, the KNN algorithm is used to find the most similar cell. Cells. Based on the selected... Cell annotation of target data is accomplished by selecting the nearest neighbor cell types. Machine learning-based models are also sensitive to batch effects. Deep learning-based models can automatically extract latent feature representations and then use a classifier to complete the cell annotation task. scAdapt uses a virtual adversarial domain adaptive network to train an enhanced classifier using labeled source data and unlabeled target data, and aligns the labeled source centroids and pseudo-labeled target centroids to generate joint embeddings for cell annotation of target data. Although deep learning-based models can generally improve the accuracy of single-cell classification tasks, their ability to handle batch effects still has room for improvement. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a single-cell type annotation model based on a pre-trained virtual adversarial domain adaptive network, abbreviated as pscAdapt.

[0006] A single-cell type annotation model based on a pre-trained virtual adversarial domain adaptive network includes the following steps:

[0007] Step 1: Propose a pre-trained domain adaptation module. Use the pre-training strategy to initialize the model parameters to learn the data distribution of the source domain. Combine the adversarial learning strategy to train the domain adaptation network to achieve domain-level alignment and reduce the differences between neighboring domains.

[0008] Step 2: Introduce a virtual adversarial training module for joint training. By maximizing the model's sensitivity to small input perturbations during training, adversarial examples are generated to improve the model's performance on unlabeled samples and enhance its generalization ability to new data.

[0009] Step 3: Design a structural similarity alignment module. By considering that cells of the same type have similar structural features, the module brings cells of the same type closer together in the feature space and widens the distance between cells of different types in the feature space, thereby achieving cell-level alignment and enhancing the discriminability of cell categories. Attached Figure Description

[0010] The invention will now be further described with reference to the accompanying drawings, in which:

[0011] Figure 1 This is a schematic diagram illustrating the process of implementing cell type annotation in this invention.

[0012] Figure 2 This is a comparison of pscAdapt with other comparative methods on seven simulated datasets of different batch intensities, showing the average accuracy of cell type prediction.

[0013] Figure 3 This is a box plot comparison of the cell type prediction accuracy of pscAdapt on a cross-platform dataset with that of the comparative methods.

[0014] Figure 4 This is a comparison of the cell type annotation performance of pscAdapt on cross-species datasets with comparative methods.

[0015] Figure 5 This is a comparison of contour scores between pscAdapt and other integration methods on cross-platform datasets.

[0016] Figure 6 This is a comparison of profile scores between pscAdapt and other fusion methods on cross-species datasets. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0018] A single-cell type annotation model based on a pre-trained virtual adversarial domain adaptive network, such as Figure 1As shown, it includes the following steps:

[0019] Step 1: Propose a pre-trained domain adaptation module. Use the pre-training strategy to initialize the model parameters to learn the data distribution of the source domain. Combine the adversarial learning strategy to train the domain adaptation network to achieve domain-level alignment and reduce the differences between neighboring domains.

[0020] Step 2: Introduce a virtual adversarial training module for joint training. By maximizing the model's sensitivity to small input perturbations during training, adversarial examples are generated to improve the model's performance on unlabeled samples and enhance its generalization ability to new data.

[0021] Step 3: Design a structural similarity alignment module. By considering that cells of the same type have similar structural features, the module brings cells of the same type closer together in the feature space and widens the distance between cells of different types in the feature space, thereby achieving cell-level alignment and enhancing the discriminability of cell categories.

[0022] Furthermore, step 1 specifically includes:

[0023] Step 1.1: Gene expression matrix of source domain data As input, it passes through the feature extractor Extracting cell feature representation Using a label classifier Generate source domain to predict cell categories By minimizing the true cell categories in the source domain data With predicting cell categories The pre-training of the domain adaptive network is completed using the loss between the two sides; where cross-entropy loss is used as the objective function.

[0024] For the problem of minimizing the classification error of the source dataset, the standard cross-entropy loss function is used:

[0025] ,

[0026] Among them, if cell tag It is a cell The correct label is The value is 1, otherwise =0; Indicates the number of cell types. This indicates that the cell belongs to the cell tag. The predicted probability;

[0027] Step 1.2: Gene expression matrix of source domain data Gene expression matrix of target domain data As input, features are extracted using a parameter-shared feature extractor. Domain discriminator predicts the domain category of input data. The objective function is binary cross-entropy loss.

[0028] ,

[0029] in, Indicates feature extractor, The domain discriminator is represented; to maximize the loss function, the domain adaptive network is used in the feature extractor. Domain discriminant A gradient inversion layer is introduced between the two layers. During the backpropagation process, the gradient of the domain classification loss is automatically inverted before it is backpropagated to the feature extractor.

[0030] Furthermore, step 2 specifically includes:

[0031] Step 2.1: The virtual adversarial training module uses the feature matrix of the target domain. As input, in Random perturbations generated nearby Find the maximum virtual perturbation ;

[0032] Step 2.2: In the original input Add maximum virtual perturbation Generate adversarial examples ;

[0033] Step 2.3: Convert the original input and adversarial examples The inputs are fed into the trained classifier. In the middle, predict the probability distribution of the original input. Probability distribution of adversarial examples ,use Divergence calculates the difference between two distributions;

[0034] Step 2.4: By minimizing distribution differences, enhance the local smoothness of the model and improve its generalization ability;

[0035] ,

[0036] ,

[0037] in, This represents a virtual adversarial perturbation that maximizes the difference between the model output and the input without the perturbation. These are model parameters; Represents the original input The output distribution; Indicate adversarial examples The output distribution; express Divergence is used to measure the difference between two distributions. This represents a penalty term used to improve the discriminability and diversity of predictions. It is the nuclear norm.

[0038] Furthermore, step 3 specifically includes:

[0039] Step 3.1: Using the characteristic matrix of the source domain With real cell tags Feature matrix of the target domain With predicted pseudo-labels As input to the structural similarity alignment module; through the feature matrix of the source domain medium cells Corresponding feature vector Refactor Construct the feature structure matrix ;

[0040] Step 3.2: Use cosine similarity Calculate the characteristic structure matrix To obtain the corresponding cells Structural similarity matrix ;

[0041] Step 3.3: Feature matrix of the target domain medium cells Corresponding feature vector Repeat steps 3.1 and 3.2 above to obtain the structural similarity matrix of the cells. ;

[0042] Step 3.4: Utilize the real cell class labels of the source domain And the predicted pseudo-labels of the target domain Information is used to ensure that the source and target domains select the same type of cells for cell alignment; the following loss is used as the objective function:

[0043] ,

[0044] in, This indicates that the cell types in the source and target domains are... The minimum number of cells, Indicates the first in the source domain A structural similarity matrix of individual cells Indicates the first in the target domain A structural similarity matrix of individual cells express Norm; by minimizing structural similarity loss, it enhances the model's ability to distinguish cell types, attracting cells of the same type in the feature space and distancing cells of different types in the feature space.

[0045] To verify the effectiveness of the present invention, this embodiment tests the performance of the present invention from two aspects: cell type annotation and batch correction.

[0046] 1. Dataset and Evaluation Metrics

[0047] (1) Simulated dataset

[0048] The simulated datasets generate single-cell RNA-seq data with varying batch effect intensities by adjusting parameters with different batch effect scales. The batch effect intensity represents the degree of difference in batch effect between the source and target domain data, with values ​​ranging from {0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4}. Each simulated dataset consists of source domain data with 2000 cells and target domain data with 1000 cells, with each cell containing 10000 genes. The cell data is also divided into four equal parts to simulate different cell types. The differential expression variation range in the simulated data is set to 0.2, incorporating a differential expression factor. For each batch effect intensity, five simulated datasets are randomly generated. Using statistical methods, the average of the five sets of experimental results is taken as the performance of the model for cell type annotation at that batch effect intensity. Unless otherwise specified, default values ​​are used for other parameters.

[0049] (2) Cross-platform datasets

[0050] The cross-platform dataset PbmcBench consists of single-cell transcriptome data from human peripheral blood mononuclear cells collected from different sequencing platforms. Data from the 10x Chromium (v2) and 10x Chromium (v3) platforms were used as the source domain, while data from the CEL-Seq2, Drop-seq, inDrop, Smart-seq2, and Seq-Well platforms were used as the target domain. Based on this data, five cross-platform datasets were constructed: 10x_CEL-Seq2, 10x_Drop-seq, 10x_inDrop, 10x_Seq_Well, and 10x_Smart-seq2.

[0051] (3) Cross-species dataset

[0052] The cross-species dataset consists of pancreatic data from humans and mice. Four cross-species datasets were constructed in a source-target domain format: Baron_mouse_Baron_human, Baron_mouse_segerstolpe, Baron_human_Baron_mouse, and Baron_mouse_combination. The homology gene table provided by SingleCellNet was used to convert gene names between humans and mice, retaining only the gene sets that overlap between the two species.

[0053] The performance of each cell annotation method was evaluated using the accuracy metric; higher accuracy indicates more precise cell annotation. Throughout the evaluation process, the cell type annotations provided by the original dataset were used as the baseline.

[0054]

[0055] in, Indicates cell category The prediction accuracy Indicates cell type Predict the number of correct cell tags. Indicates cell type The number of truly correct cell tags.

[0056]

[0057] in, This represents the average accuracy of the model's predictions for each category.

[0058] The contour score is used as an evaluation metric to assess the degree of separation of different cell types after batch correction. The higher the contour score, the better the separation of different cell types.

[0059]

[0060] in, Represents cells The average distance between cells and other cells within the same cluster reflects the cohesion within the cluster. Represents cells The average distance between cells in the nearest cluster reflects the separation between clusters.

[0061] 2. Comparative Analysis of Experimental Results

[0062] (1) Cell type annotation experiment

[0063] (a) Simulated dataset

[0064] The classification capabilities of pscAdapt and the contrast method under different batch effect strengths were evaluated with a differential expression factor parameter value of 0.2. Figure 2 As shown, with increasing batch effect strength, the performance of all cell type annotation methods decreased to varying degrees, demonstrating that the batch effect significantly impacts cell type annotation performance. The proposed pscAdapt model achieved the best cell type annotation performance; even with increasing batch effect strength, the accuracy of pscAdapt cell annotation remained above 0.99. When the batch effect strength exceeded 0.4, the performance of methods such as SingleR significantly decreased. When the batch effect strength was 1.4, the performance of the comparative methods such as SingleR dropped below 0.6. Experimental results indicate that pscAdapt can effectively address the batch effect problem.

[0065] (b) Cross-platform datasets

[0066] Figure 3 This paper demonstrates the performance of pscAdapt and contrasting methods in cell type annotation on cross-platform datasets. pscAdapt achieves an average cell annotation accuracy of 88%, which is higher than the contrasting methods. Compared with scAdapt, pscAdapt improves performance by 3%. By pre-training on source domain data to initialize the parameters of the feature extractor and label classifier, the domain adaptation module learns prior knowledge of the source domain data, thus enhancing pscAdapt's cross-platform capability.

[0067] (c) Cross-species dataset

[0068] The differences in gene expression levels among the same cell type across different species increase the challenge of cell type annotation tasks on cross-species datasets. The performance of pscAdapt and all comparative methods was evaluated on four pairs of cross-species datasets, such as... Figure 4 As shown, pscAdapt achieved the best cell type annotation performance on four cross-species datasets, with an average accuracy of 0.97, which is 7% higher than the second-ranked method.

[0069] (2) Batch calibration experiment

[0070] (a) Cross-platform dataset

[0071] Figure 5This paper compares the contour scores of pscAdapt with various integration methods on cross-platform datasets. pscAdapt achieves an average contour score of 0.49 on the cross-platform data, which is 2% higher than the second-ranked scAdapt method and significantly higher than the other comparison methods. This demonstrates that pscAdapt can effectively separate different cell types, proving the role of the structural similarity alignment module. The results indicate that pscAdapt possesses good batch correction capabilities and can effectively handle the batch effect problem introduced by cross-platform integration.

[0072] (b) Cross-species dataset

[0073] Figure 6 The comparison of contour scores between pscAdapt and various integration methods on cross-species datasets is presented. pscAdapt achieves an average contour score of 0.66 on cross-species datasets, significantly higher than other comparative methods, demonstrating that pscAdapt possesses the best batch correction capability on cross-species data.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a single-cell type annotation model based on a pre-trained virtual adversarial domain adaptive network, characterized in that: Step 1: Propose a pre-trained domain adaptation module. Use the pre-training strategy to initialize the model parameters to learn the data distribution of the source domain. Combine the adversarial learning strategy to train the domain adaptation network to achieve domain-level alignment and reduce the differences between neighboring domains. Step 2: Introduce a virtual adversarial training module for joint training. By maximizing the model's sensitivity to small input perturbations during training, adversarial examples are generated to improve the model's performance on unlabeled samples and enhance its generalization ability to new data. Step 3: Design a structural similarity alignment module. By considering that cells of the same type have similar structural features, the module shortens the distance between cells of the same type in the feature space and widens the distance between cells of different types in the feature space, thereby achieving cell-level alignment and enhancing the discriminability of cell categories. Step 1 specifically includes: Step 1.1: Gene expression matrix of source domain data As input, it passes through the feature extractor Extracting cell feature representation Using a label classifier Generate source domain to predict cell categories By minimizing the true cell categories in the source domain data With predicting cell categories The pre-training of the domain adaptive network is completed using the loss between the two sides; where cross-entropy loss is used as the objective function. For the problem of minimizing the classification error of the source dataset, the standard cross-entropy loss function is used: Among them, if cell tag It is a cell The correct label is The value is 1, otherwise =0; Indicates the number of cell types. This indicates that the cell belongs to the cell tag. The predicted probability; Step 1.2: Gene expression matrix of source domain data Gene expression matrix of target domain data As input, features are extracted using a parameter-shared feature extractor. Domain discriminator predicts the domain category of input data. The objective function is binary cross-entropy loss. in, Indicates feature extractor, The domain discriminator is represented; to maximize the loss function, the domain adaptive network is used in the feature extractor. Domain discriminant A gradient inversion layer is introduced between the layers, and the gradient of the domain classification loss is automatically inverted before it is backpropagated to the feature extractor during the backpropagation process. Step 2 specifically includes: Step 2.1: The virtual adversarial training module uses the feature matrix of the target domain. As input, in Random perturbations generated nearby Find the maximum virtual perturbation ; Step 2.2: In the original input Add maximum virtual perturbation Generate adversarial examples ; Step 2.3: Convert the original input and adversarial examples The inputs are fed into the trained classifier. In the middle, predict the probability distribution of the original input. Probability distribution of adversarial examples ,use Divergence calculates the difference between two distributions; Step 2.4: By minimizing distribution differences, enhance the local smoothness of the model and improve its generalization ability; in, This represents a virtual adversarial perturbation that maximizes the difference between the model output and the input without the perturbation. These are model parameters; Represents the original input The output distribution; Indicate adversarial examples The output distribution; express Divergence is used to measure the difference between two distributions. This represents a penalty term used to improve the discriminability and diversity of predictions. It is the nuclear norm; Step 3 specifically includes: Step 3.1: Using the characteristic matrix of the source domain With real cell tags Feature matrix of the target domain With predicted pseudo-labels As input to the structural similarity alignment module; through the feature matrix of the source domain medium cells Corresponding feature vector Refactor Construct the feature structure matrix ; Step 3.2: Use cosine similarity Calculate the characteristic structure matrix To obtain the corresponding cells Structural similarity matrix ; Step 3.3: Feature matrix of the target domain medium cells Corresponding feature vector Repeat steps 3.1 and 3.2 above to obtain the structural similarity matrix of the cells. ; Step 3.4: Utilize the real cell class labels of the source domain And the predicted pseudo-labels of the target domain Information is used to ensure that the source and target domains select the same type of cells for cell alignment; the following loss is used as the objective function: in, This indicates that the cell types in the source and target domains are... The minimum number of cells, Indicates the first in the source domain A structural similarity matrix of individual cells Indicates the first in the target domain A structural similarity matrix of individual cells express Norm; by minimizing structural similarity loss, it enhances the model's ability to distinguish cell types, attracting cells of the same type in the feature space and distancing cells of different types in the feature space.

Images