A Single-Cell Incremental Annotation Method Based on Distribution and Expression-Aware Revisit

Abstract

This invention discloses a single-cell incremental annotation method based on distribution and expression-aware revisiting, belonging to the technical field of single-cell type annotation. The method includes: inputting the single-cell expression profile sample to be processed into a diffusion model based on distribution-aware conditions to generate gene expression profile data of old cell samples that participated in the diffusion model training; integrating the gene expression profile data of the old cell samples with the gene expression profile data of newly collected cell samples; and inputting the integrated data into two different feature extractors in an expression-aware knowledge distillation model to perform multi-view gene expression alignment and output the corresponding cell type annotation results. This invention effectively solves the core challenge of incremental annotation of long-tailed and high-dimensional sparse single-cell data, enabling efficient data replay and continuous learning, thereby significantly improving the performance and robustness of incremental annotation.

Description

Technical Field

[0001] This invention belongs to the technical field of single-cell type annotation, specifically relating to a single-cell incremental annotation method based on distribution and expression-aware revisiting. Background Technology

[0002] Single-cell type annotation holds significant cutting-edge value in targeted therapy research on the tumor microenvironment. However, with the exponential growth in the scale of single-cell sequencing data, existing methods face three core challenges: First, directly iteratively training existing models leads to severe catastrophic forgetting problems, such as... Figure 1 As shown in (a); secondly, batch-trained independent models are difficult to meet the diverse cell annotation needs in open environments, such as Figure 1 As shown in (b) above; furthermore, the data categories exhibit a significant long-tail distribution. These limitations essentially reflect the inadequacy of existing methods in supporting single-cell incremental annotation, severely restricting the widespread application of this technology in precision medicine clinical practice. Therefore, overcoming the scientific challenge of single-cell long-tail incremental annotation is not only of significant theoretical value but also of urgent practical importance.

[0003] Currently, other fields mainly focus on incremental scenarios with balanced data distribution. However, single-cell gene expression faces a severe long-tail incremental challenge due to its high-dimensional sparsity, high specificity, and extreme imbalance between cell types. Under such extreme distribution, traditional incremental methods struggle to balance the learning of new and old cell types, further exacerbating catastrophic forgetting and efficiency bottlenecks. To address the long-tail phenomenon, existing research often employs data augmentation strategies to alleviate the limited sample size by expanding data sources. However, in real-world scenarios, single-cell data typically originates from the open world, and its long-tail distribution is more consistent with biological and natural laws. Therefore, it is essential to explore more practical strategies to address the negative impacts of long-tail distribution. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned shortcomings of existing technologies by providing a single-cell incremental annotation method based on distribution and expression-aware revisiting. This addresses the problem that existing traditional incremental annotation methods struggle to balance the learning of new and old cell types, further exacerbating catastrophic forgetting and efficiency bottlenecks.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A single-cell incremental annotation method based on distribution and expression-aware revisiting includes the following steps:

[0007] S1. Input the single-cell expression profile sample to be processed into the diffusion model based on distribution-aware conditions to generate gene expression profile data of old cell samples that have participated in the training of the diffusion model.

[0008] S2. Integrate the gene expression profile data of the old cell samples with the gene expression profile data of the newly collected cell samples;

[0009] S3. Input the integrated data into two different feature extractors in the knowledge distillation model of expression perception to perform multi-view gene expression alignment and output the corresponding cell type annotation results.

[0010] Furthermore, S1 specifically includes the following steps:

[0011] S11. Represent the single-cell expression profile samples to be processed as a matrix form of the true expression profile. ;

[0012] S12, Authentic Expression Spectrum As input, it is fed into the sparse feature extraction module to extract sparse masks used to describe the expression regions of key genes;

[0013] S13. Based on the conditional probability distribution of the diffusion model, the true expression spectrum... Perform noise addition operation;

[0014] S14. Take the result of the noise addition operation as input and repeat S12~S13 until the final sparse mask is extracted. and the noise expression spectrum corresponding to the final noise addition operation result. ;

[0015] S15, Noise Expression Spectrum As the starting input for diffusion inversion, the generated expression spectrum is gradually recovered through multi-step inverse denoising operations. .

[0016] Furthermore, S14 specifically includes: at each time step of the diffusion model The input data is constrained by a dynamic sparse mask, specifically represented as follows:

[0017]

[0018] in,

[0019]

[0020] In the formula, This is represented as the conditional probability distribution of the diffusion model; Indicates time step Dynamic sparse mask; Indicates to Give The weights of the probabilities make Noise distribution guided by sparse distribution; Represents a normal distribution; Indicates time step The degree of preservation of original data at that time; The covariance representing noise is a diagonal matrix; For the Sigmoid function; and It is an adjustable parameter. Genes in dynamic sampling data The amount of expression; It is a constant.

[0021] Furthermore, in S15, the conditional probability distribution of inverse denoising is expressed as:

[0022]

[0023] In the formula, The conditional probability distribution representing inverse denoising; and These represent the direction and magnitude of the noise reduction process, respectively.

[0024] Furthermore, in S15, a sparsity constraint loss is introduced in each step of the diffusion inversion recovery. To maintain a consistent sparse structure between the generated expression profile and the real expression profile;

[0025] Among them, sparsity constraint loss Represented as:

[0026]

[0027] In the formula, They are time steps Noise expression spectra after denoising and after adding noise ; and These are the weighting coefficients for sparse and non-sparse regions; This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise; for quantile threshold; To measure the error between the generated expression profile data and the actual expression profile data.

[0028] Furthermore, S3 specifically includes the following steps:

[0029] S31. Copy the integrated data into two copies and record them as basic data respectively. With new types of data ;

[0030] S32, Transfer basic class data The new class data is input into the base class feature extractor in the knowledge distillation model. The input is fed into the new class feature extractor in the knowledge distillation model, and the high-level features of the base class are output respectively. and new advanced features Simultaneously, multiple levels of basic class attention maps are extracted from the multi-scale attention modules in both the basic class feature extractor and the novel class feature extractor. And new class attention graphs with multiple levels ;

[0031] S33. Employ methods that minimize attention alignment loss. Attention graphs for basic classes And new class attention graphs with multiple levels Align the corresponding attention distributions between them;

[0032] S34. Minimize the characterization alignment loss. Advanced features of basic classes and new advanced features Perform structural alignment in feature space;

[0033] S35, satisfying the minimum representation alignment loss Basic Class Advanced Features The input is fed into the base annotator to obtain the corresponding base annotation prediction. ; New advanced features Input into the new class annotator to obtain the new class annotation prediction. ;

[0034] S36. Using category prediction to predict consistency loss Predicting basic annotations And new class annotation prediction Align the annotation distribution.

[0035] Furthermore, in S32, the multi-scale attention module contains multiple levels of base class attention graphs. And new class attention graphs with multiple levels The calculation expression is:

[0036]

[0037] In the formula, An attention map representing each attention layer; , , represent linear mappings of the input data, , and represent the query, key, and value matrices, respectively; d is the dimension of the key; and i is the index.

[0038] Furthermore, in S33, the attention alignment loss is minimized. Represented as:

[0039]

[0040] In S34, the representation alignment loss is minimized. Represented as:

[0041]

[0042] In S36, the category prediction consistency loss Represented as:

[0043]

[0044] Based on minimizing attention alignment loss Minimize representation alignment loss Consistency loss between categories Perform multi-perspective gene expression re-alignment:

[0045]

[0046] In the formula, Dimension of the attention map matrix; This is the attention map matrix output by the basic feature extractor; The attention map matrix output by the new type of feature extractor; The feature matrix is ​​the output of the basic class feature extractor; The feature matrix output by the new type of feature extractor; The logits output after being processed by the base class annotator; The logits output after being processed by the new class annotator; , , Indicates the alignment weight between regulatory features and the gene map; This represents the gene expression back-visit alignment loss function from multiple perspectives.

[0047] Furthermore, fuzzy guidance constraints are introduced into the basic class feature extractor to suppress false recalls of the basic class feature extractor on unseen cell types, while guiding the new class feature extractor to enhance its adaptability to new types;

[0048] The fuzzy guiding constraint is expressed as follows:

[0049]

[0050] In the formula, This represents the fuzzy guided loss function; The cardinality of the incremental sample set; and These are weight parameters; The preset minimum false recall threshold; Use the Sigmoid activation function; Representing the The predicted logit for each sample.

[0051] Furthermore, according to , and Calculate the sum of training losses for incremental annotations:

[0052]

[0053] in,

[0054]

[0055] In the formula, Represents the cross-entropy loss function; This is an operation on the cross-entropy loss function; This indicates data concatenation operation. The resulting combined input; This is cell type information.

[0056] The single-cell incremental annotation method based on distribution and expression-aware regression provided by this invention has the following beneficial effects:

[0057] 1. This invention innovatively designs two perception paradigms for revisit learning: the conditional diffusion generation paradigm of distributed perception and the fuzzy incremental guidance paradigm of expression perception. By deeply integrating the above perception paradigms with the core module, data replay and continuous learning can be efficiently realized, thereby significantly improving the performance and robustness of incremental annotation. Moreover, this invention can be seamlessly integrated into other conventional annotation models, thereby giving full play to its wide applicability and advantages in continuous learning.

[0058] 2. The incremental annotation process of this invention first utilizes a conditional diffusion model, leveraging its distribution-aware capabilities, to generate replay data of cell samples from the old task in a near-lossless manner. This replay data is then jointly trained with samples of new cell types. The jointly trained samples are input into an incremental knowledge distillation architecture, achieving multi-perspective gene expression revisit alignment through expression awareness, and incorporating fuzzy guidance constraints to eliminate interference from spurious recalls. This design effectively solves the core challenge of incremental annotation of long-tailed, high-dimensional, and sparse single-cell data. Attached Figure Description

[0059] Figure 1 This is a comparison between conventional cell annotation and incremental cell annotation methods. Figure 1 In the example (a), the independently trained annotation model is used to learn unseen cell types; Figure 1 (b) in the diagram represents a pre-trained annotation model used for transfer learning of new cell types; Figure 1 (c) in the figure represents the incremental cell annotation framework proposed in Embodiment 1 of the present invention.

[0060] Figure 2 This refers to the single-cell incremental annotation process in Example 1; Figure 2 (a) is a flowchart of the distribution-sensing conditional diffusion generation of old cell gene expression profiles; Figure 2 (b) is a flowchart of the incremental annotation of single-cell long-tailed classes expressing perceptual knowledge distillation and fuzzy guidance.

[0061] Figure 3 The process of forward noise addition and reverse noise reduction for generating single-cell expression profiles based on sparse distribution sensing in Example 1.

[0062] Figure 4 The multi-perspective gene expression perception back-visit alignment process under the knowledge distillation architecture in Example 1 includes multi-scale attention feature map alignment, high-level feature alignment, and annotation type representation alignment.

[0063] Figure 5 This is a schematic diagram of fuzzy guided constraints in Example 1.

[0064] Figure 6 This is a hyperparameter analysis of gene expression sensing alignment on the Vento dataset to mitigate forgetting in Example 2.

[0065] Figure 7 This is the result of multi-scale gene expression regression analysis using distribution sensing in Example 2; due to the large number of genes, the gene expression vectors are reshaped into rectangles for display in the figure. Figure 7 (a) in the dataset represents the expression regression results of PMSC cells in the He dataset; Figure 7 (b) in the figure shows the expression regression results of NK cells in the Madissoon dataset; Figure 7 (c) in the figure represents the expression regression results of Endothelium cells in the Stewart dataset; Figure 7 (d) in the figure represents the expression regression results of dS1 cells in the Vento dataset.

[0066] Figure 8 In Example 2, the effect of scLTCIA distribution sensing on gene expression levels generated by revisiting the He, Madissoon, Stewart, and Vento datasets was evaluated.

[0067] Figure 9 These are three key visualization results on the Zheng68k dataset from Example 2; among them, Figure 9(a) shows the experimental setup and the long-tail distribution of cell types, reflecting the extremely imbalanced label distribution characteristics of this dataset in the last incremental phase; Figure 9 (b) in the figure shows the multi-view alignment visualization results based on gene expression perception. It compares the changes in feature distribution of the Base model and the Novel model before and after alignment on the old and new cell types in the second incremental stage, and verifies the effectiveness of the feature space alignment strategy. Figure 9 (c) in the figure shows the difference in gene expression distribution between the generated sample and the real sample for the extremely rare cell type CD34+, which further verifies the accuracy and bioconsistency of expression re-inquiry.

[0068] Figure 10 This is a flowchart of the single-cell incremental annotation method based on distribution and expression perception revisiting in Embodiment 1 of the present invention. Detailed Implementation

[0069] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0070] Example 1

[0071] This embodiment provides a single-cell incremental annotation method based on distribution-aware revisiting. First, a distribution-aware generative framework is designed to accurately revisit old data in the model's memory and efficiently replay single-cell data with high-dimensional sparsity for incremental annotation. Simultaneously, an expression-aware knowledge distillation architecture is designed. Through multi-perspective gene representation alignment, it significantly improves the perception and representation capabilities for novel type-specific expressions in a small number of samples while strengthening the retention of mainstream category memory. Furthermore, fuzzy incremental guidance constraints are introduced to effectively eliminate false recall of novel cell types during the incremental process, thereby mitigating the negative impact of long-tailed distributions on incremental annotation.

[0072] The method in this embodiment corresponds to the single-cell incremental annotation framework scLTCIA, see reference. Figure 1 (c) Figure 2 and Figure 10 Specifically, it includes the following:

[0073] S1. Input the single-cell expression profile sample to be processed into the diffusion model based on distribution-aware conditions to generate gene expression profile data of old cell samples that have participated in the training of the diffusion model.

[0074] like Figure 2 As shown in (a), this embodiment uses a pre-trained diffusion model with distribution-aware conditions to generate gene expression profile data of old cell samples that have participated in the training, based on the type and number information of single cells. Then, the generated gene expression profile data is integrated with the gene expression profile data of newly collected cell samples and input together. Figure 2 (b) shows the incremental learning framework.

[0075] Generative replay, with its advantage of avoiding large-scale storage consumption, has become an effective strategy to mitigate catastrophic forgetting in incremental learning. However, the high-dimensional sparse expression profile distribution in single-cell incremental annotation severely restricts the quality of generative replay. Therefore, this embodiment proposes a distribution-aware expression profile generation paradigm, referencing... Figure 3 Specifically, it includes the following steps:

[0076] S11. Extract real single-cell expression profile data;

[0077] The single-cell expression profile samples to be processed are represented as a matrix form of the true expression profile. This matrix has high dimensionality and sparse properties, and serves as the input data for the model.

[0078] S12. Extract sparse representation structure information;

[0079] Authentic Expression Spectrum As input, it is fed into the sparse feature extraction module to extract sparse masks used to describe key gene expression regions. To capture the effective information regions in the expression spectrum;

[0080] S13, Perform the noise addition operation of the diffusion model;

[0081] Conditional probability distribution based on diffusion model , for the true expression spectrum Perform noise addition (noise perturbation processing) to generate the result of the first step of noise addition. To simulate the gradual perturbation process of data in the potential space;

[0082] S14. Repeated sparse mask extraction and noise addition process;

[0083] The result of the noise addition operation is used as the new input to replace the true representation spectrum input in S12. And repeat steps S12~S13, that is, the intermediate noise-adding operation results generated in each step (such as...) , Until Repeated Input Sparse Extraction Module Extract its sparse mask and through conditional probability Noise is continuously added until the final sparse mask is obtained. and the noise expression spectrum corresponding to the final noise addition operation result. .

[0084] More specifically, in order to preserve the sparse statistical characteristics of real data, this embodiment introduces a dynamic sparse mask in the diffusion model. This dynamic sparse mask maintains the sparse region and strengthens the noise in the non-sparse region during the forward diffusion operation, and ensures that the sparse distribution of the denoised data approximates the real distribution during the reverse operation, thus achieving both the generation quality and biological consistency.

[0085] Among them, dynamic sparse mask generation is sampled from the true representation spectrum. and the data after noise addition at each subsequent time step. The sparsity statistics, specifically express Step-by-step genes The mask value is determined by the sparse distribution of gene expression at that location. If the gene... If the expression level is very low in the sampled data, then Approaching 0; conversely, if the gene If the expression level is high, then The process of approaching 1 can be represented as:

[0086]

[0087] At each time step of the diffusion model The input data is constrained by a dynamic sparse mask, specifically represented as follows:

[0088]

[0089] In the formula, This is represented as the conditional probability distribution of the diffusion model; Indicates time step Dynamic sparse mask; Indicates to Give The weights of the probabilities make Noise distribution guided by sparse distribution; Represents a normal distribution; Indicates time step The degree of preservation of original data at that time; The covariance representing noise is a diagonal matrix (i.e., each dimension is independent and identically distributed). For the Sigmoid function; and It is an adjustable parameter. Genes in dynamic sampling data The amount of expression; It is a constant.

[0090] S15. Initialize the reverse diffusion process;

[0091] noise expression spectrum As the starting input for diffusion inversion, it enters the inverse generation process constructed by Diffusion U-Net, and through multi-step inverse denoising operations, the generated expression spectrum is gradually recovered. It is used to gradually restore the biological expression profile structure from pure noise.

[0092] Specifically, this embodiment examines the true expression spectrum. Sparse pattern learning is optimized through normalization and sparse masking. Cell type information. With sample identifier Conditional embeddings are jointly encoded using a pre-trained Text2vec model. And linearly mapped to the embedding space. , as initial conditions and embed standard Gaussian noise What was obtained later As the initial input.

[0093] During the reverse reasoning phase, the model extracts from the noise expression spectrum. Starting from [the point], through multiple denoising steps, sequentially from [the point] The expression profile is gradually recovered and generated. At each step, the expression profile distribution at the current time is predicted by the diffusion model. That is, the target data is generated step by step by denoising and then the expression profile is generated. To detect the sparsity of gene expression data, a dynamic sparse mask is introduced. This guides the updating of specific regions during the denoising process. Specifically, for each time step... and genes sparse mask The amplitude and region of denoising are dynamically adjusted to ensure that the generated data retains the sparse distribution characteristics consistent with the real data; the conditional probability distribution of inverse denoising is represented as:

[0094]

[0095] In the formula, The conditional probability distribution representing inverse denoising; and These represent the direction and magnitude of the noise reduction process, respectively.

[0096] To minimize the difference between generated and real data, this embodiment introduces a sparse regularization loss for structural constraints. A quantile threshold method is used to dynamically distinguish between sparse and non-sparse regions, and the error is calculated for each. Specifically, a sparsity constraint loss is introduced in each step of the diffusion inversion recovery. To maintain a consistent sparse structure between the generated and real expression spectra, thereby improving the model's expressive power in high-dimensional sparse scenarios; sparsity constraint loss. Represented as:

[0097]

[0098] In the formula, They are time steps Noise expression spectra after denoising and after adding noise ; and These are the weighting coefficients for sparse and non-sparse regions; This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise; for The quantile threshold (e.g., 50th percentile) is used to dynamically distinguish between sparse and non-sparse regions.

[0099] To measure the error between the generated expression profile data and the true expression profile data, it is represented as:

[0100]

[0101] In the formula, express and The squared Euclidean distance between the generated data and the real data; this constraint ensures global and local sparsity consistency between the generated data and the real data.

[0102] This embodiment introduces a cell type recognition module for semantic constraints, inputting the intermediate expression profile during the diffusion process into the cell type recognizer. The model predicts cell types and optimizes the generation model (diffusion model) based on classification loss feedback to ensure the generated spectrum is biologically plausible and distinguishable. Based on the above process, the final output of this embodiment is... It simultaneously meets the requirements of sparse expression features and cell type semantic information, and can be used for downstream single-cell analysis tasks, such as cell annotation and lineage inference.

[0103] S2. Integrate the gene expression profile data of the old cell samples with the gene expression profile data of the newly collected cell samples;

[0104] Specifically, the old data generated by the diffusion model With new data As input in knowledge distillation, it is processed through data concatenation. Integration to form a joint input .

[0105] S3. Knowledge distillation, as a classic regularization strategy, has demonstrated excellent performance in continuous learning. However, traditional methods mainly focus on feature representation learning, making it difficult to effectively address the incremental annotation challenges posed by specific gene expression and long-tailed distribution in single-cell data. To address this, this embodiment proposes an innovative multi-perspective gene attention mechanism and fuzzy guided constraints, constructing an expression-aware incremental annotation paradigm.

[0106] Specifically, in this embodiment, the integrated data is input into two different feature extractors in the expression-aware knowledge distillation model to perform multi-view gene expression alignment and output corresponding cell type annotation results. (See reference...) Figure 4 Specifically, it includes the following steps:

[0107] S31. Constructing multi-view input expression profile data: Integrating the data ( Duplicate the data into two copies and record each copy as a basic data type. With new types of data ;

[0108] S32, Transfer basic class data The new class data is input into the base class feature extractor in the knowledge distillation model. The input is fed into the new class feature extractor in the knowledge distillation model, and the high-level features of the base class are output respectively. and new advanced features ;

[0109] Meanwhile, both the basic feature extractor and the novel feature extractor consist of multiple attention layers (with query, key, and value structures). Specifically, the multi-scale attention module generates multiple levels of basic attention maps. And new class attention graphs with multiple levels It is used to capture gene expression patterns at different scales;

[0110] Specifically, the multi-scale attention module contains multiple levels of basic class attention graphs. And new class attention graphs with multiple levels The calculation expression is:

[0111]

[0112] In the formula, This indicates the different levels of downsampling. An attention map representing each attention layer; , respectively, represent linear mappings to the input data, and , respectively, represent the query, key, and value matrices; d is the dimension of the key; i For indexing.

[0113] S33, Perform multi-scale attention feature alignment;

[0114] Minimize attention alignment loss Attention graphs for basic classes And new class attention graphs with multiple levels Align the corresponding attention distributions between them to achieve consistency in attention response distribution between the base class and the new class, thereby improving cross-class expressive perception capabilities;

[0115] Minimize attention alignment loss Represented as:

[0116]

[0117] S34. Output high-level feature representations and perform representation alignment;

[0118] Minimize the characterization alignment loss Advanced features of basic classes and new advanced features Performing structural alignment in the feature space is beneficial for feature transfer of new classes;

[0119] Minimize representation alignment loss Represented as:

[0120]

[0121] S35. Enter the annotator to predict cell types respectively;

[0122] This will satisfy the condition of minimizing the representation alignment loss. Basic Class Advanced Features The input is fed into the base annotator to obtain the corresponding base annotation predictions, which will then be used to generate high-level features of the new class. Input into the new class annotator to obtain the new class annotation prediction. ;

[0123] S36, Execution type prediction alignment;

[0124] Using category prediction consistency loss Predicting basic annotations And new class annotation prediction The annotation distribution is aligned to ensure that the new class can obtain a stable and reliable classification decision boundary even without a large number of labeled samples;

[0125] Category prediction consistency loss Represented as:

[0126]

[0127] After training with the old type, the basic annotation model has accumulated rich and detailed gene expression data. and cell type characterization This can serve as a foundational model to guide representation alignment within the knowledge distillation architecture of new models. Furthermore, in incremental annotation scenarios, immature new models can revisit the gene expression "worldview" of the foundational model and dynamically align their receptive fields. To accurately capture hierarchical information about specific gene expression in single cells, the alignment constraint of knowledge distillation is formally defined as:

[0128]

[0129] In the formula, Dimension of the attention map matrix; This is the attention map matrix output by the basic feature extractor; The attention map matrix output by the new type of feature extractor; The feature matrix is ​​the output of the basic class feature extractor; The feature matrix output by the new type of feature extractor; The logits output after being processed by the base class annotator; The logits output after being processed by the new class annotator; , , Indicates the alignment weight between regulatory features and the gene map; This represents the gene expression back-visit alignment loss function from multiple perspectives.

[0130] This embodiment introduces fuzzy-guided long-tailed incremental annotations into the base model (diffusion model);

[0131] As a preferred embodiment, refer to Figure 6 In the distillation architecture, the predictive representation of the basic model The dimension corresponds to the number of categories of the basic cell type. Although the basic model can effectively revisit the representation of the basic type during the replay of class increments, its prediction of new types often leads to misclassification because the probability peak is distributed among the basic types. To improve the performance of incremental annotation, a fuzzy guided constraint is proposed, such as... Figure 5 As shown, this study aims to suppress false recall of the base model as a teacher on unseen cell types, while effectively guiding the new model to enhance its adaptability to new types, avoid forgetting, and improve its learning ability on long-tail data.

[0132] The fuzzy guiding constraint is expressed as follows:

[0133]

[0134] In the formula, This represents the fuzzy guided loss function; The cardinality of the incremental sample set; and These are weight parameters; The preset minimum false recall threshold; Use the Sigmoid activation function; Representing the The predicted logit for each sample. In the fuzzy guided constraint formula, the first term exceeds the squared penalty constraint. The first term provides a high-confidence prediction, while the second term provides stable gradient guidance through a smooth logarithmic penalty; the maximum value of the logit predicted by the base model is constrained to not exceed [a certain value]. This allows the gene to be classified as an unknown type, thus guiding the new model to perceive novel type-specific genes in incremental tasks. This method not only enhances the deep perception of novel type-specific genes but also significantly improves the performance of incremental annotation for single-cell long-tailed classes.

[0135] Incremental annotation reasoning process;

[0136] This invention presents a two-stage incremental annotation framework: first, data replay is achieved through a sparse-aware conditional diffusion model; then, long-tail class incremental annotation is performed using gene-aware and fuzzy guidance. The old data generated by the diffusion model... With new data As input in knowledge distillation, it is processed through data concatenation. Forming joint input Subsequently, multi-perspective gene expression re-alignment was performed. Secondly, a fuzzy guidance mechanism was introduced into the base model (diffusion model) to suppress [the expression of gene expression]. False memories, when guiding new models, are addressed using the cross-entropy loss function. Calculate loss It significantly enhances the ability to perceive genes specifically expressed in single cells under long-tailed distributions, effectively alleviates the forgetting problem, and enables continuous and accurate annotation of rare cell types.

[0137] This embodiment mines gene relationships at different scales, integrates low-level local gene expression with high-level global features, and constructs attention maps for each layer of the output extractor. , and cell type characterization of the logit output by the annotator. ,in This indicates a comment operation.

[0138] Based on this, the total training loss for incremental annotation in this embodiment is:

[0139]

[0140] in,

[0141]

[0142] In the formula, Represents the cross-entropy loss function; This is an operation on the cross-entropy loss function; This indicates data concatenation operation. The resulting combined input; This is cell type information.

[0143] Example 2

[0144] This embodiment verifies the performance and effectiveness of the method in Embodiment 1 through experiments, specifically including the following:

[0145] Experimental setup

[0146] Benchmark dataset;

[0147] This embodiment is based on the analysis of five single-cell datasets: He (containing 14 cell types and 15,680 cells), Madissoon (containing 25 cell types and 57,020 cells), Stewart (containing 43 cell types and 26,628 cells), Vento (containing 32 cell types and 64,734 cells), and Zheng68k (containing 11 cell types and 65,943 cells). The first four datasets are divided into five incremental tasks, with the specific incremental tasks for each dataset at each stage as shown in the formula. As shown, the number of cell types in the dataset is split into the first stage. One category and four subsequent incremental phases The Zheng68k dataset, due to its significant long-tail distribution, is divided into three incremental phases for research such as model comparison and validation, computational efficiency evaluation, and visualization analysis under more stringent comparison conditions. For detailed information on the specific division of these datasets, please refer to Table 1 for the dataset splitting.

[0148]

[0149] Baseline comparison methods;

[0150] This embodiment selects a single-cell type annotation baseline method that is easy to modularly expand and embeds it into the single-cell incremental annotation framework scLTCIA proposed in Embodiment 1 as the core feature extraction module:

[0151] CIForm: This method is based on the Transformer architecture, and its sub-vector self-attention mechanism has a good modular design, which makes it easy to split the model into an encoder (for feature extraction) and a classification head (for cell type prediction), so as to flexibly embed it into the incremental training process of the continuous learning framework.

[0152] scTransSort: This method uses a Transformer model to extract features from unordered input data, effectively alleviating the sparsity of single-cell data. It requires no manual annotation or reliance on external reference data, demonstrating excellent cell type recognition performance. Its end-to-end feature modeling capability makes it a strong candidate for a high-quality feature extraction module in continuous learning frameworks.

[0153] scPred: This method uses Singular Value Decomposition (SVD) for feature dimensionality reduction and selection, and combines it with Support Vector Machine (SVM) to build a high-precision classification model. It performs well in cell annotation tasks and has good generalization ability, making it suitable as a core feature extraction module in continuous learning frameworks.

[0154] Furthermore, to further verify the comparative performance of the method of this invention with representative algorithms in the general field of continuous learning, two classic continuous learning methods widely used in image classification tasks were selected for more rigorous evaluation on the Zheng68k dataset. During this process, the scRNA-seq data was reshaped (an operation that changes the shape of tensors) into an "image format" suitable for convolution operations, enabling these methods to run on the data of this invention, thus achieving a fair cross-sectional comparison.

[0155] iCaRL: This method innovatively integrates the nearest neighbor mean classification strategy, a sample selection mechanism based on herding (a method name in the iCaRL paper), and a feature learning method that combines sample replay and knowledge distillation. It can simultaneously optimize feature representation and classifier in incremental learning scenarios, effectively alleviate catastrophic forgetting, and improve overall performance.

[0156] DER: This method optimizes network predictions (logits) by combining sample replay, knowledge distillation, and regularization constraints, rather than directly relying on real labels. This makes it more in line with the General Continuous Learning (GCL) paradigm, has better model calibrability, and demonstrates leading performance in multiple benchmark tests.

[0157] Evaluation indicators;

[0158] Classification accuracy was used to evaluate the performance of baseline models in incremental annotation tasks, specifically using three core metrics: B represents the annotation accuracy for known cell types (baseline), N represents the annotation accuracy for newly introduced cell types (new), and B+N represents the overall accuracy for all cell types. Furthermore, the resource overhead of different baseline methods was compared and analyzed in terms of training time, GPU memory usage, and the number of model parameters, reported in minutes (mins), gigabytes (G), and millions of parameters (M), respectively. In the distribution-aware gene expression replay experiment, two further evaluation metrics, MAE (mean absolute error) and SSIM (structural similarity index), were introduced to measure the average error between the generated replay data and the real gene expression data.

[0159] Specific implementation details;

[0160] The evaluation experiment was designed in two tiers: standard benchmark testing and enhanced benchmark testing.

[0161] In the standard benchmark test, the four public datasets (He, Madissoon, Stewart and Vento) were divided into five task stages, and two task construction strategies were used for evaluation: (1) Ordered task: The long-tail class samples were divided in order of decreasing quantity to simulate an idealized incremental learning scenario; (2) Mixed task: The class order was randomly shuffled while keeping the overall class distribution unchanged to simulate the random and unstructured long-tail distribution that may occur in real biological sequencing, so as to evaluate the robustness and generalization ability of the model in a more complex environment.

[0162] Under the two classification strategies mentioned above, three mainstream single-cell annotation methods (scPred, CIForm, and scTransSort) are integrated into the incremental annotation framework scLTCIA proposed in this invention, and their performance in long-tail incremental annotation tasks is systematically evaluated. To ensure the fairness of the comparison, the initial learning rate of all models is uniformly set to 1e-4 in the first three incremental stages and 1e-2 in the last two stages, with a maximum of 100 training epochs, and an early stopping mechanism based on validation set loss is adopted (allowing a maximum of 15 epochs of no improvement tolerance).

[0163] In the enhanced benchmark test, the large-scale dataset Zheng68k was selected and divided into three task stages, which were distributed in descending order of the number of class samples: the first stage contained 5 classes, and the second and third stages each contained 3 classes, to simulate a more sparse and challenging incremental learning scenario. Under this setting, the performance of three methods was compared: (1) the performance of scPred, CIForm, and scTransSort, three single-cell annotation methods integrated in the incremental framework; (2) the direct training results of CIForm, which performed relatively well, under the non-incremental setting; and (3) the adaptability of representative general continuous learning methods (iCaRL and DER). In addition to accuracy, the consumption of each method in terms of training time, GPU memory usage, and model parameter quantity was further compared to comprehensively evaluate the performance-resource efficiency trade-off of different methods.

[0164] Standard benchmark results

[0165] Mixed-order long-tail incremental annotation;

[0166] The experimental results are shown in Table 2. The scLTCIA method of this invention outperforms other baseline models in mixed-order long-tail incremental tasks across all datasets. Compared to other methods, scLTCIA exhibits stronger stability during long-tail incremental annotation, with significantly smaller fluctuations in accuracy between new and old types at each stage. For example, in the final stages of the Madissoon and Vento datasets, scLTCIA's accuracy in learning new cell types exceeds that of the suboptimal methods by 7.86% and 6.68%, respectively, highlighting its significant advantage in learning new knowledge. Regarding anti-forgetting capabilities, scLTCIA achieves an accuracy of 88.51% for old types in the final stage of the He dataset, 1.95% higher than the suboptimal method scTransSort, further demonstrating the significant advantage of the framework of this invention in maintaining the integrity of old knowledge. These results indicate that scLTCIA can not only efficiently learn new types in long-tailed distributed data with biological randomness but also effectively mitigate the problem of catastrophic forgetting, demonstrating its strong adaptability and robustness in complex biological scenarios.

[0167]

[0168] Sequential long-tail incremental annotation

[0169] The experimental results are shown in Table 3. In mixed-order annotation tasks, tail categories (with very few samples) are often neglected because they have little impact on overall accuracy. However, in sequential long-tail incremental tasks, by learning in descending order of sample quantity, the incremental performance of the model on tail categories can be evaluated more effectively, revealing its ability to handle few-sample types. For example, in the final stage of the Madissoon dataset, scLTCIA achieved an annotation accuracy of 85% for novel types with very few samples, significantly higher than CIForm's 55.08% and scPred's 40.53%. Furthermore, on the Stewart dataset, scLTCIA consistently maintained an annotation accuracy of over 92% for tail categories, and the suboptimal method scTransSort, which extends the framework of this invention, performed close to scLTCIA in all metrics. These results demonstrate that scLTCIA exhibits excellent plasticity and scalability in sequential long-tail incremental annotation scenarios.

[0170]

[0171] Ablation Research

[0172] Table 4 shows that the scLTCIA framework has significant advantages in incremental annotation of a very small number of samples in the tail class, especially in the final stage of the Madissoon dataset, where it exhibits high plasticity for a very small number of cell types. Ablation experiments were used to quantitatively evaluate the contribution of scLTCIA to the three key constraints for incremental annotation on the Madissoon dataset. As shown in Table 4, compared with using only… compared to, and This significantly improves the plasticity of incremental annotation, thanks to the knowledge distillation architecture's enhanced focus on novel types through gene expression awareness. Furthermore, the introduction of... The fuzzy-guided incremental constraint significantly improved the annotation accuracy of new types of long-tailed distribution tails in the final stage, indicating that it has a significant effect on the uncertainty increment of a very small number of samples.

[0173]

[0174] Gene expression perception alignment analysis;

[0175] The knowledge distillation system of expression perception is crucial for maintaining memories of revisiting old cells. Therefore, using the best-performing Vento dataset, we analyzed the hyperparameter sensitivity of two alignment operations involved in the expression perception paradigm: multi-scale gene feature map alignment and cell type representation alignment. Experimental results are as follows: Figure 6 As shown in the figure. In the nine experimental settings, by comparing the baseline accuracy evaluation indicators of the four incremental stages, gene expression perception showed the best effect in preventing forgetting when the accuracy was 1 and 2.

[0176] Multi-scale diffusion follow-up

[0177] Figure 7 This paper demonstrates the effectiveness of the proposed distribution-aware diffusion model in revisiting cell expression patterns in historical memory. Using four datasets—He, Madissoon, Stewart, and Vento—and selecting four cell types (PMSC, NK, Endothelium, and dS1) at three different gene-level scales, scLTCIA almost perfectly reconstructed the cell expression features in historical memory. Notably, the figure shows that the gene expression data exhibits high sparsity across all scales, with a large number of zero values. However, even under such extreme sparsity, the model maintains extremely high revisit accuracy, fully demonstrating the robustness and effectiveness of the proposed sparse distribution-aware module.

[0178] Visualization of gene expression during follow-up visits;

[0179] A retrospective visualization analysis of gene expression in four datasets (He, Madissoon, Stewart, and Vento) was performed using T-SNE dimensionality reduction. The differences in gene regularization expression distribution between cells generated by the scLTCIA framework and the real data were compared after the second incremental annotation session. Figure 8 As shown, except for the Madissoon dataset, the samples generated from the latent space in the other datasets are highly consistent with the true distribution after dimensionality reduction, demonstrating good distribution fidelity. However, in the Madissoon dataset, the generated samples exhibit a distribution pattern that is roughly a mirror image of the true data. This discrepancy may stem from subtle differences between the generated data and the true distribution in the high-dimensional space, especially slight shifts in the sparse structure. Considering the high sensitivity of T-SNE to local structural changes, these high-dimensional micro-differences may be amplified during dimensionality reduction. Overall, these results indicate that scLTCIA can effectively model high-dimensional sparse representation distributions, possesses strong distribution awareness during revisiting generated data, and is consistent with... Figure 7 The accuracy of the follow-up visits shown in the figures corroborates each other.

[0180] Strengthening benchmark experiments

[0181] Additional baseline comparison;

[0182] To validate on a more stringent dataset, the Zheng68k dataset, widely used as a benchmark for its highly pronounced long-tailed distribution, was used for testing. The long-tailed distribution of cell types in the Zheng68k dataset is shown below. Figure 9As shown in (a), the dataset was divided into three incremental sessions: five initial classes in the first round, and three incremental classes in the second and final rounds. CIForm, scPred, and scTransSort were integrated into our scLTCIA framework as baseline feature extraction modules to compare the performance of incremental annotation. Simultaneously, CIForm, the baseline method that best generalized to unseen cells in the final round, was selected from the three cell annotation models and trained and tested separately for non-incremental and incremental evaluation. To enhance the theoretical support for continuous learning in the experiments, two well-regarded traditional continuous learning methods in the image domain, iCaRL and DER, were introduced for evaluation.

[0183] As shown in Table 5, the framework of this invention exhibits the best performance in terms of overall incremental annotation accuracy. It achieves excellent performance, almost on par with traditional DER methods in the field of continuous learning, in terms of accuracy and computational cost. Furthermore, a comparison between the non-incremental annotation CIForm-DA method and the incremental CIForm-CIA method shows that the incremental implementation method performs better in terms of the overall accuracy of the final B+N. Figure 9 (a) The data distribution also shows that the three cell types used in the Zheng68k dataset in the last incremental session accounted for only 3.2% of the total number of cells, exhibiting a very extreme long-tail trend. Therefore, it can be considered that the CIForm-DA baseline method, which is directly trained under supervision, still has room for improvement in annotating rare cells. At the same time, CIForm-CIA, by leveraging the fuzzy guidance mechanism in our scLTCIA incremental annotation framework, prevented new cells from being incorrectly predicted as the numerous common cells, achieving an accuracy of 56.09% for annotating these rare cells in the last round, which is quite remarkable.

[0184] From an overall computational cost perspective, the scLTCIA framework consumes comparable GPU memory to traditional continuous learning frameworks. Although its computational cost is slightly higher than that of directly training but annotating cell models, it achieves better annotation accuracy for rare cells in incremental scenarios, and its computational training time and cost are well controlled, making it a commendable achievement overall. Therefore, the scLTCIA framework is an excellent architecture that balances annotation performance and computational cost control.

[0185]

[0186] Visualize the alignment of the expression distribution;

[0187] Furthermore, to verify the multi-perspective alignment capability of the student model with the teacher model in the knowledge distillation architecture for gene expression perception, in this invention, Figure 8(b) Visualizes the feature distribution of the old and new classes in the second session of the Zheng68k dataset, demonstrating the representation of the same cell type by the base model and the new model before and after alignment. As shown in the figure, before alignment, there are significant differences in the feature point distribution generated by the base model (red) and the new model (blue). The embedding distribution of old-class cells (circles) and new-class cells (squares) in the two models is inconsistent, exhibiting a clear separation. After applying the distillation alignment strategy, the feature points generated by the two models almost completely overlap, with red and blue dots highly overlapping, indicating that the student model has successfully aligned to the feature space of the teacher model, and the distribution of the old and new cell types is consistent between them. These phenomena intuitively demonstrate the effectiveness of the proposed multi-view distillation alignment strategy based on gene expression awareness in incremental learning, providing solid support for the generalization of rare cell types.

[0188] Visualization of rare cell expression re-examination;

[0189] This embodiment further selects the tenth category of CD34+ cells, containing only 188 samples, from the Zheng68k dataset as an extremely rare cell type. A sparse distribution-aware expression regression generation strategy is employed to compare and analyze the differences in gene expression between the generated samples and the real samples. The results are as follows: Figure 8 (c) shows the regions where the generated cells and real cells show significant differences in gene expression, marked with green boxes. The results show that these regions generally capture the significant sparse expression trend in scRNA-seq data, but there are still some gaps in the reproduction of expression levels at certain specific gene loci, requiring further optimization of simulation accuracy. This phenomenon is presumably due to the extremely low overall proportion of CD34+ in Zheng68k, only 0.29%. Therefore, achieving a preliminary reconstruction of gene expression distribution characteristics under such sparse sample conditions is already quite remarkable, demonstrating the potential of the distribution-aware revisit strategy. The proposed fuzzy-guided incremental strategy also strives to prevent incremental annotation performance issues caused by long-tailed distribution and revisit accuracy problems, and will continue to improve its generation accuracy in extremely low-resource scenarios in the future.

[0190] Although specific embodiments of the invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of this patent. Various modifications and variations that can be made by a person skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of this patent.

Claims

1. A single-cell incremental annotation method based on distribution and expression-aware revisiting, characterized in that, Includes the following steps: S1. Input the single-cell expression profile sample to be processed into the diffusion model based on distribution-aware conditions to generate gene expression profile data of old cell samples that have participated in the training of the diffusion model. S1 specifically includes the following steps: S11. Represent the single-cell expression profile samples to be processed as a matrix form of the true expression profile. ; S12, Authentic Expression Spectrum As input, it is fed into the sparse feature extraction module to extract sparse masks used to describe the expression regions of key genes; S13. Based on the conditional probability distribution of the diffusion model, the true expression spectrum... Perform noise addition operation; S14. Take the result of the noise addition operation as input and repeat S12~S13 until the final sparse mask is extracted. and the noise expression spectrum corresponding to the final noise addition operation result. ; S15, Noise Expression Spectrum As the starting input for diffusion inversion, and through multi-step inverse denoising operations, the generated expression spectrum is gradually recovered. ; S2. Integrate the gene expression profile data of the old cell samples with the gene expression profile data of the newly collected cell samples; S3. Input the integrated data into two different feature extractors in the expression-aware knowledge distillation model to perform multi-view gene expression alignment and output the corresponding cell type annotation results. S3 specifically includes the following steps: S31. Copy the integrated data into two copies and record them as basic data respectively. With new types of data ; S32, Transfer basic class data The new class data is input into the base class feature extractor in the knowledge distillation model. The input is fed into the new class feature extractor in the knowledge distillation model, and the high-level features of the base class are output respectively. and new advanced features Simultaneously, multiple levels of basic class attention maps are extracted from the multi-scale attention modules in both the basic class feature extractor and the novel class feature extractor. And new class attention graphs with multiple levels ; S33. Employ methods that minimize attention alignment loss. Attention graphs for basic classes And new class attention graphs with multiple levels Align the corresponding attention distributions between them; S34. Minimize the alignment loss. Advanced features of basic classes and new advanced features Perform structural alignment in feature space; S35, satisfying the minimum representation alignment loss Basic Class Advanced Features The input is fed into the base annotator to obtain the corresponding base annotation prediction. ; New advanced features Input into the new class annotator to obtain the new class annotation prediction. ; S36. Using category prediction to predict consistency loss Predicting basic annotations And new class annotation prediction Align the annotation distribution; In S33, the attention alignment loss is minimized. Represented as: In S34, the representation alignment loss is minimized. Represented as: In S36, the category prediction consistency loss Represented as: Based on minimizing attention alignment loss Minimize representation alignment loss Consistency loss between categories Perform multi-perspective gene expression re-alignment: In the formula, Dimension of the attention map matrix; This is the attention map matrix output by the basic feature extractor; The attention map matrix output by the new type of feature extractor; The feature matrix is ​​the output of the basic class feature extractor; The feature matrix output by the new type of feature extractor; This refers to the logits output by the base class annotator; The logits output after being processed by the new class annotator; , , Indicates the alignment weight between regulatory features and the gene map; Represents the gene expression backtracking alignment loss function from multiple perspectives; Fuzzy guidance constraints are introduced into the basic class feature extractor to suppress false recall of the basic class feature extractor on unseen cell types, while guiding the new class feature extractor to enhance its adaptability to new types; The fuzzy guiding constraint is represented as follows: In the formula, This represents the fuzzy guided loss function; The cardinality of the incremental sample set; and These are weight parameters; The preset minimum false recall threshold; Use the Sigmoid activation function; Representing the The predicted logit for each sample.

2. The single-cell incremental annotation method based on distribution and expression-aware revisiting according to claim 1, characterized in that, S14 specifically includes: at each time step of the diffusion model The input data is constrained by a dynamic sparse mask, specifically represented as follows: in, In the formula, This is represented as the conditional probability distribution of the diffusion model; Indicates time step Dynamic sparse mask; Indicates to Give The weights of the probabilities make Noise distribution guided by sparse distribution; Represents a normal distribution; Indicates time step The degree of preservation of original data at that time; The covariance representing noise is a diagonal matrix; For the Sigmoid function; and It is an adjustable parameter. Genes in dynamic sampling data The amount of expression; It is a constant.

3. The single-cell incremental annotation method based on distribution and expression-aware revisiting according to claim 2, characterized in that, In step S15, the conditional probability distribution for reverse denoising is expressed as follows: In the formula, The conditional probability distribution representing inverse denoising; and These represent the direction and magnitude of the noise reduction process, respectively.

4. The single-cell incremental annotation method based on distribution and expression-aware revisiting according to claim 2, characterized in that, In step S15, a sparsity constraint loss is introduced in each step of the diffusion inversion recovery. To maintain a consistent sparse structure between the generated expression profile and the real expression profile; Among them, sparsity constraint loss Represented as: In the formula, They are time steps Noise expression spectra after denoising and after adding noise ; and These are the weighting coefficients for sparse and non-sparse regions; This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise; for quantile threshold; To measure the error between the generated expression profile data and the actual expression profile data.

5. The single-cell incremental annotation method based on distribution and expression-aware revisiting according to claim 1, characterized in that, In S32, the multi-scale attention module contains multiple levels of basic class attention graphs. And new class attention graphs with multiple levels The calculation expression is: In the formula, An attention map representing each attention layer; , respectively, represent linear mappings to the input data, and , respectively, represent the query and the key; d is the dimension of the key; i For indexing.

6. The single-cell incremental annotation method based on distribution and expression-aware revisiting according to claim 1, characterized in that, according to , and Calculate the sum of training losses for incremental annotations: in, In the formula, Represents the cross-entropy loss function; This is an operation on the cross-entropy loss function; This indicates data concatenation operation. The resulting combined input; This is cell type information.

Images