Single cell type annotation method and system based on diffusion transformer and adaptive representation learning

Abstract

This invention relates to a single-cell type annotation method and system based on diffusion Transformer and adaptive representation learning. The method includes: acquiring single-cell RNA sequencing data for preprocessing and principal component analysis dimensionality reduction; training a diffusion Transformer network using an optimal transport flow matching strategy; freezing the network weights after training; inputting noise-free latent variables and fixed time-step parameters into the frozen-weight network; and performing deterministic feature extraction using a classifier-free approach; identifying and suppressing large-scale activation dimensions of intermediate layer features to obtain biological features; dynamically calculating the Softmax normalized weights of each layer through a learnable scalar mixture mechanism and dynamically weighting and fusing them to obtain aggregated features; performing lightweight fine-tuning using low-rank adaptive fine-tuning technology; and obtaining cell type annotation results through a multilayer perceptron classification head. This method can improve the accuracy and robustness of cell type annotation while resisting strong noise and batch effects.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics, and in particular to a single-cell type annotation method and system based on diffusion Transformer and adaptive representation learning. Background Technology

[0002] Accurate cell type annotation is the cornerstone of single-cell RNA sequencing (scRNA-seq) analysis, supporting key tasks such as trajectory inference, differential expression analysis, and disease mechanism research. Cell type annotation is a core component of the single-cell RNA sequencing (scRNA-seq) data analysis workflow. In downstream analysis tasks using scRNA-seq data, annotation accuracy directly determines the reliability of downstream tasks such as trajectory inference, differential expression analysis, cell lineage construction, and disease mechanism analysis. Efficiently and robustly extracting biologically meaningful features from high-dimensional, sparse, and noisy gene expression data is crucial for achieving accurate cell annotation. The accuracy of cell type annotation fundamentally depends on the effective characterization of cell biological features. However, due to the inherent high-dimensionality, sparseness, and noise of the data, extracting robust biological features from scRNA-seq data remains a significant challenge.

[0003] Currently, existing technologies in the field of single-cell annotation can be mainly divided into: traditional statistical and shallow machine learning methods: early methods relied on traditional statistics and machine learning, as well as mapping methods based on reference maps; deep learning-based discriminative networks: discriminative networks based on multilayer perceptrons and graph neural networks were introduced to improve the classification accuracy of complex cell subtypes through nonlinear functions or graph structures; large-scale basic models: inspired by natural language processing, large models such as scBERT and scGPT have emerged, which are pre-trained on massive single-cell maps and aim to learn general cell representations through unsupervised learning; and diffusion model-based generative methods: the bioinformatics community has begun to explore the use of diffusion models for scRNA-seq, mainly to generate high-quality pseudo data to expand the training set of downstream classifiers. However, while traditional shallow models perform well on small datasets, they often struggle to capture the complex nonlinear relationships in gene expression. Existing deep learning discriminative models typically focus on defining rigid decision boundaries in the feature space, failing to truly learn the intrinsic manifold structure of the data distribution. This makes the models very vulnerable to noisy data or unknown cell types. Most existing single-cell diffusion model studies limit this technique to data augmentation or imputation, but a diffusion model capable of high-fidelity generation has deeply internalized the biological essence of cells and the laws of gene regulation. Such models often require generating pseudo-data first, then using this data for data augmentation or imputation, and finally annotating the cells.

[0004] Therefore, traditional single-cell annotation methods often suffer from insufficient annotation accuracy and weak noise resistance and generalization ability because they are difficult to capture the complex nonlinear relationships in gene expression and the generative models are only used for data augmentation and do not fully utilize their feature modeling capabilities. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems, a single-cell type annotation method and system based on diffusion Transformer and adaptive representation learning is provided, which can improve the accuracy and robustness of cell type annotation while resisting strong noise and batch effects.

[0006] A single-cell type annotation method based on diffusion Transformer and adaptive representation learning, the method comprising:

[0007] The single-cell RNA sequencing data to be annotated is preprocessed and subjected to principal component analysis for dimensionality reduction to obtain latent variables in the low-dimensional latent space. Based on the latent variables, a diffusion Transformer network is trained using the optimal transport flow matching strategy.

[0008] The weights of the trained diffusion Transformer network are frozen, and the noise-free latent variables and fixed time step parameters are input into the diffusion Transformer network with frozen weights. Deterministic feature extraction is performed using a classifier-free approach.

[0009] Identify and suppress large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network, and remove non-semantic dimension signals to obtain biological features;

[0010] The Softmax normalized weights of each layer are dynamically calculated through a learnable scalar mixing mechanism, and the biological features of the multi-layer outputs of the diffusion Transformer network are dynamically weighted and fused according to the normalized weights to obtain aggregated features.

[0011] The aggregated features are lightly fine-tuned using low-rank adaptive fine-tuning technology. The fine-tuned feature vector is then input into the multilayer perceptron classification head to obtain cell type annotation results.

[0012] In one embodiment, the single-cell RNA sequencing data to be annotated is preprocessed and subjected to principal component analysis for dimensionality reduction to obtain latent variables in a low-dimensional latent space, including:

[0013] Obtain the single-cell RNA sequencing data to be annotated, and perform log normalization on the single-cell RNA sequencing data to be annotated to screen out hypervariable genes.

[0014] Principal component analysis was performed on the hypervariable genes, and the results were projected onto a low-dimensional latent space of a preset dimension to obtain the corresponding latent variables.

[0015] In one embodiment, a diffusion Transformer network is trained using an optimal transport stream matching strategy based on the latent variables, including:

[0016] Construct a continuous probability path from the standard Gaussian noise distribution to the latent variable data distribution, and define the time step according to the standard Gaussian noise distribution and the latent variable data distribution;

[0017] For each latent variable sample and its corresponding noise sample, an intermediate state is constructed by linear interpolation, and the optimal velocity field on the continuous probability path is calculated.

[0018] A diffusion Transformer network is constructed. Based on the time step and the optimal velocity field, the parameters of the diffusion Transformer network are optimized using the flow matching loss function to complete the training of the diffusion Transformer network.

[0019] In one embodiment, the weights of the trained diffusion Transformer network are frozen, the noise-free latent variables and fixed-time-step parameters are input into the frozen-weight diffusion Transformer network, and deterministic feature extraction is performed using a classifier-free approach, including:

[0020] Set all parameters of the trained Diffusion Transformer network to a non-updateable state to complete the weight freeze of the Diffusion Transformer network.

[0021] Select fixed time step parameters and convert them into time embedding vectors and noise-free latent variables as inputs to the diffusion Transformer network, and set the classifier-guided conditional inputs to empty labels;

[0022] Forward propagation is performed using a diffusion Transformer network to output the feature representations of each hidden layer, thereby extracting deterministic features.

[0023] In one embodiment, large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network are identified and suppressed, and non-semantic dimension signals are removed to obtain biological features, including:

[0024] Calculate the activation magnitude of the intermediate layer features of the diffusion Transformer network in each dimension;

[0025] The activation amplitude is compared with a preset threshold to identify large-scale activation dimensions;

[0026] The large-scale activation dimensions are zeroed or masked to remove non-semantic dimension signals related to the diffusion time step and generation conditions, thus obtaining biological features.

[0027] In one embodiment, the Softmax normalized weights of each layer are dynamically calculated using a learnable scalar mixing mechanism, and the biological features output from multiple layers of the diffusion Transformer network are dynamically weighted and fused according to the normalized weights to obtain aggregated features, including:

[0028] During the forward propagation of the diffusion Transformer network, biological features output from each intermediate layer are collected, and learnable scalar parameters are assigned to each biological feature.

[0029] The learnable scalar parameters are input into the Softmax function for normalization to obtain the normalized weights for each layer.

[0030] The biological features of each layer are multiplied element-wise with their corresponding normalized weights, and the results are summed to obtain the weighted and fused aggregated features.

[0031] In one embodiment, a low-rank adaptive fine-tuning technique is used to perform lightweight fine-tuning on the aggregated features. The fine-tuned feature vector is then input into the multilayer perceptron classification head to obtain cell type annotation results, including:

[0032] Freeze the pre-trained weight matrix of the diffusion Transformer network, use low-rank adaptive fine-tuning technique to perform lightweight fine-tuning on the aggregated features, and train only the low-rank parameters to obtain the fine-tuned feature vector.

[0033] The finely tuned feature vector is input into the multilayer perceptron classification head and mapped to the cell type category space through a nonlinear transformation.

[0034] Based on the cell type category space, the predicted probability of each cell type is calculated using the Softmax function, and the category corresponding to the highest probability is determined as the cell type annotation result.

[0035] In one embodiment, the multilayer perceptron classification head includes a hidden projection layer, a batch normalization layer, a ReLU activation function, and a linear layer mapped to the number of cell types.

[0036] A single-cell type annotation system based on diffusion Transformer and adaptive representation learning, the system comprising:

[0037] The data processing and network training module is used to acquire the single-cell RNA sequencing data to be annotated, perform preprocessing and principal component analysis for dimensionality reduction, obtain latent variables in the low-dimensional latent space, and train the diffusion Transformer network based on the latent variables using the optimal transport flow matching strategy.

[0038] The deterministic feature extraction module is used to freeze the weights of the trained diffusion Transformer network, input the noise-free latent variables and fixed time step parameters into the frozen weights of the diffusion Transformer network, and perform deterministic feature extraction using classifier-free guidance.

[0039] An activation suppression module is used to identify and suppress large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network, eliminate non-semantic dimension signals, and obtain biological features.

[0040] The learnable scalar mixing module is used to dynamically calculate the Softmax normalized weights of each layer through a learnable scalar mixing mechanism, and to dynamically weight and fuse the biological features output by the multi-layer of the diffusion Transformer network according to the normalized weights to obtain aggregated features.

[0041] The fine-tuning and classification module is used to perform lightweight fine-tuning of the aggregated features using low-rank adaptive fine-tuning technology, and input the fine-tuned feature vector into the multilayer perceptron classification head to obtain cell type annotation results.

[0042] The aforementioned single-cell type annotation method and system based on diffusion Transformer and adaptive representation learning can compress high-dimensional sparse single-cell expression data and suppress noise through preprocessing and principal component analysis dimensionality reduction. By training the diffusion Transformer network with an optimal transport flow matching strategy, it can accurately learn the probabilistic flow structure of cell data and the nonlinear dependencies between genes. By freezing network weights and combining them with fixed time steps, noise-free input, and classifier-free guidance for deterministic feature extraction, it can extract high signal-to-noise ratio intrinsic biological features without introducing additional perturbations. By identifying and suppressing large-scale activation dimensions, it can effectively remove non-semantic interference signals related to the generation task. By dynamically fusing multi-layer features through a learnable scalar mixing mechanism, it can adaptively integrate shallow structural information and deep semantic information. By employing low-rank adaptive lightweight fine-tuning and using a multilayer perceptron classification head, it can improve the accuracy and robustness of cell type annotation while resisting strong noise and batch effects. Attached Figure Description

[0043] Figure 1 This is an application environment diagram of a single-cell type annotation method based on diffusion Transformer and adaptive representation learning in one embodiment;

[0044] Figure 2 This is a flowchart illustrating a single-cell type annotation method based on diffusion Transformer and adaptive representation learning in one embodiment.

[0045] Figure 3 This is a schematic diagram of the workflow framework of a single-cell type annotation method based on diffusion Transformer and adaptive representation learning in one embodiment.

[0046] Figure 4 This is a block diagram of a single-cell type annotation system based on diffusion Transformer and adaptive representation learning in one embodiment;

[0047] Figure 5 This is a schematic diagram showing the accuracy of cell processing compared with other single-cell annotation methods after the experiment.

[0048] Figure 6 This is a schematic diagram comparing the high-quality feature clustering and manifold decoupling capabilities obtained after the experiment.

[0049] Figure 7 , Figure 8 This is a schematic diagram comparing the effectiveness of the core components and the adaptive feature fusion obtained after the experiment;

[0050] Figure 9 This is a schematic diagram comparing the adaptive retention and integration batch effects obtained after the experiment;

[0051] Figure 10 This is a schematic diagram showing the comparison of superior robustness in scenarios with sparse data and few samples obtained after the experiment.

[0052] Figure 11 This is a schematic diagram comparing the unknown cell detection capability and high scalability obtained after the experiment.

[0053] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] The single-cell type annotation method based on diffusion Transformer and adaptive representation learning provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1As shown, the application environment includes computer device 110. Computer device 110 can acquire single-cell RNA sequencing data to be annotated, perform preprocessing and principal component analysis for dimensionality reduction, obtain latent variables in a low-dimensional latent space, and train a diffusion Transformer network based on the latent variables using an optimal transport flow matching strategy. Computer device 110 can freeze the weights of the trained diffusion Transformer network, input the noise-free latent variables and fixed time-step parameters into the frozen-weight diffusion Transformer network, and perform deterministic feature extraction using a classifier-free approach. Computer device 110 can identify and suppress large-scale activation dimensions of the intermediate layer features of the diffusion Transformer network, eliminating non-semantic dimension signals to obtain biological features. Computer device 110 can dynamically calculate the Softmax normalized weights of each layer through a learnable scalar mixing mechanism, and dynamically weight and fuse the biological features output from multiple layers of the diffusion Transformer network according to the normalized weights to obtain aggregated features. Computer device 110 can use low-rank adaptive fine-tuning technology to perform lightweight fine-tuning of the aggregated features, and input the fine-tuned feature vector into the multilayer perceptron classification head to obtain cell type annotation results. Among them, computer equipment 110 may include, but is not limited to, various personal computers, laptops, smartphones, robots, unmanned aerial vehicles, tablets, and other devices.

[0056] In one embodiment, such as Figure 2 As shown, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning is provided, including the following steps:

[0057] Step 202: Obtain the single-cell RNA sequencing data to be annotated, perform preprocessing and principal component analysis to reduce dimensionality, obtain latent variables in the low-dimensional latent space, and train the diffusion Transformer network based on the latent variables using the optimal transport flow matching strategy.

[0058] Computer devices can map the gene expression matrix of single-cell RNA sequencing to be annotated from a high-dimensional, sparse original space to a low-dimensional, compact, and continuous latent space, providing a high-quality input representation for the generative pre-training and feature extraction of the subsequent Diffusion Transformer (DiT) model.

[0059] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include data processing and dimensionality reduction processes. Specifically, the process includes: acquiring the single-cell RNA sequencing data to be annotated, performing log normalization on the single-cell RNA sequencing data to be annotated, and screening out highly variable genes; performing principal component analysis on the highly variable genes, projecting them onto a low-dimensional latent space of a preset dimension, and obtaining the corresponding latent variables.

[0060] Among them, the single-cell RNA sequencing data scRNA-seq to be annotated has extremely high sparsity and high dimensionality. Direct modeling in the original gene space is not only computationally expensive but also highly susceptible to noise interference. In addition, scRNA-seq data has permutation invariance in gene arrangement order, and therefore does not possess the spatial inductive bias upon which convolutional neural networks rely. Linear projection can replace complex MLP or CNN encoders in certain mask reconstruction tasks, and PCA itself is the standard processing flow for scRNA-seq data analysis, widely used in mainstream analysis tools such as Seurat and Scanpy. Therefore, in this embodiment, PCA can be used as a non-parametric linear encoder.

[0061] Computer equipment can address the high dimensionality and extreme sparsity of single-cell RNA sequencing data by using principal component analysis (PCA) to project the raw gene expression profile into a compact, continuous latent space. Before performing PCA dimensionality reduction, standard preprocessing can be performed on the raw gene expression matrix, including log-normalization, and the top 2000 highly variable genes (HVGs) can be extracted.

[0062] Specifically, the computer equipment can acquire the raw expression matrix of single-cell RNA sequencing (scRNA-seq) for cell type annotation. This matrix is ​​a high-dimensional sparse matrix, where rows represent cells, columns represent genes, and the values ​​within the matrix represent the expression levels of the corresponding genes in the cells. Next, the computer equipment can perform logarithmic normalization on the gene expression data to compress the dynamic range of expression levels and reduce the impact of extreme values; it can screen the top 2000 highly variable genes (HVGs) from all genes, removing low-expression, low-variance noise genes; and it can center the preprocessed expression vectors by subtracting the mean gene expression value to ensure the data distribution meets the input requirements for principal component analysis.

[0063] Next, the computer equipment can use principal component analysis to project the preprocessed high-dimensional gene expression vector into a continuous, compact low-dimensional latent space. That is, the preprocessed high-dimensional expression vector... Projected into a low-dimensional dense hidden space in, Including the front One principal component (in this model) ), The mean value of the genes effectively suppresses technical noise, providing a compact and continuous input manifold for subsequent generative models. Furthermore, since this model utilizes the internal representations of the generative model for discrimination rather than generation, there is no need to train a corresponding decoder.

[0064] Computer devices can train diffusion Transformer (DiT) networks using the optimal transport flow matching (OT-FM) strategy in the latent space, forcing the model to fit the distribution of the original data and learn the probability flow of cells in the latent space as well as the complex nonlinear dependencies between gene programs.

[0065] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include a network training process, specifically including: constructing a continuous probability path from a standard Gaussian noise distribution to a latent variable data distribution, and defining a time step according to the standard Gaussian noise distribution and the latent variable data distribution; constructing an intermediate state for each latent variable sample and its corresponding noise sample through linear interpolation, and calculating the optimal velocity field on the continuous probability path; constructing a diffusion Transformer network, and optimizing the parameters of the diffusion Transformer network using a flow matching loss function based on the time step and the optimal velocity field, thereby completing the training of the diffusion Transformer network.

[0066] To overcome the problems of slow sampling speed and unstable training trajectories in traditional diffusion models based on stochastic differential equations, this embodiment introduces the Optimal Transport Flow Matching (OT-FM) framework. A computer device can use OT-FM to train a Diffusion Transformer (DiT) network, enabling the network to learn a deterministic probabilistic flow path from noise in a standard normal distribution to the real data distribution, thereby deeply internalizing the biological essence of cells and the laws of gene regulation.

[0067] Computer devices can construct a continuous probability path from a standard Gaussian noise distribution to a latent variable data distribution, defining time steps. Where t=0 corresponds to the noise distribution and t=1 corresponds to the data distribution. At t=0, the data represents the true latent cell variables. At t=1, the data completely transforms into standard Gaussian noise. .

[0068] Specifically, the traditional diffusion model defines a variance-preserving stochastic differential equation SDE: in, It is standard Brownian motion; Let be the particle state at time t; It is a noisy scheduling function. Current research has proven that for any such SDE, there exists an equivalent deterministic probabilistic flow ODE, which at any time... Marginal probability density In accordance with SDE, that is: This ODE describes the deterministic trajectory of a particle flowing from the data distribution (t=0) to the noise distribution (t=1); and the core objective is to directly model this vector field.

[0069] During conditional flow matching, the computer device can define a time-dependent probability density path. satisfy and The evolution of this density follows the continuity equation (mass conservation): Direct optimization of vector fields Satisfying this equation is extremely difficult; Conditional Flow Matching (CFM) uses regression to fix the data sample. Vector field with conditions To simplify the problem, the objective function is:

[0070] To ensure the most efficient training and straight-line trajectory generation, in this embodiment, the computer device constructs an optimal transmission displacement path corresponding to a geodesic at the Wasserstein-2 distance. That is, it defines a conditional flow. For noise and data Linear interpolation between: in, For time steps; Standard normal noise; These are real data samples, i.e., the latent variables after PCA dimensionality reduction. (Regarding time...) Differentiate and derive the objective conditional vector field. For constants: This indicates that the particle's velocity from noise to data is constant, and its trajectory is a straight line, representing the optimal path under optimal transmission conditions. Therefore, the current DiT model... They are trained to predict this constant velocity field.

[0071] Next, the computer device can construct a diffusion Transformer network. First, the one-dimensional cellular latent variables are... Treating it as a sequence, positional encoding is added to each dimension to preserve the order information between dimensions. A multi-layer Transformer decoder structure containing multi-head self-attention (MSA) and feedforward neural network (MLP) is constructed, which can capture the long-range dependencies between dimensions in the latent variables. Then, continuous time steps t are embedded into high-dimensional vectors through sinusoidal positional encoding or MLP as conditional inputs to the network. The temporal embedding vector and cell type label are injected into each layer of DiT through cross-attention mechanism or adaptive layer normalization (AdaLN), so that the network can adjust its output according to the current time step and conditional information.

[0072] Specifically, in this embodiment, a modified DiffusionTransformer (DiT) architecture for 1D ensemble data is adopted, which includes: Input Embedding: Latent Vectors Linearly projected onto the hidden dimension It abandons the Patchify operation in the image domain to maintain global dependencies between genes. Transformer module: The backbone network consists of... It consists of several modules, each containing a multi-head self-attention (MSA) network and a feedforward network (FFN). The AdaLN-Zero conditional mechanism is used to efficiently inject time steps. and category conditions Adaptive layer normalization with zero initialization (AdaLN) is used, and the modulation parameters are... The result is obtained from the regression of the sum of temporal embedding and category embedding: This makes the scaling and translation of latent features explicitly dependent on biological conditions and diffusion time.

[0073] During network training, random sampling of time step t and noise samples ϵ is performed to calculate intermediate states. ,Will The input is a DiT network with input t. The network's output objective is to predict the theoretical velocity field at that location. Then, a loss function under the flow matching framework is used to minimize the L2 distance between the network's predicted velocity field and the theoretical optimal velocity field. The loss function is: in Finally, the gradient can be calculated using the backpropagation algorithm, and the parameters of the DiT network can be updated using the optimizer until convergence, thus completing the training of the Diffusion Transformer network.

[0074] Step 204: Freeze the weights of the trained diffusion Transformer network, input the noise-free latent variables and fixed time step parameters into the frozen weights of the diffusion Transformer network, and perform deterministic feature extraction using classifier-free guidance.

[0075] During the feature extraction stage, the computer device can freeze the pre-trained DiT network weights, transforming it from a generator into a feature extractor. To maximize the signal-to-noise ratio of biological semantics, clean latent variables are input during inference, with fixed time step parameters (e.g., ...) injected. Furthermore, it employs classifier-free guidance (Null-CFG, i.e., conditions are set to null). This strategy puts the model into a sensitive "ready to denoise" state, effectively extracting unconditional, intrinsic features without introducing actual noise variance.

[0076] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include a deterministic feature extraction process. Specifically, this process includes: setting all parameters of the trained diffusion Transformer network to a non-updateable state to freeze the diffusion Transformer network weights; selecting fixed time-step parameters and converting them into temporal embedding vectors and noise-free latent variables as inputs to the diffusion Transformer network, setting the classifier-guided conditional input to an empty label; performing forward propagation through the diffusion Transformer network to output the feature representations of each hidden layer and extract deterministic features.

[0077] The computer device can set all parameters of the diffusion Transformer network to an untrainable state to freeze the network weights, ensuring that the network's internal representation remains fixed during subsequent forward propagation and does not change due to new inputs. Next, the computer device can convert noise-free latent variables and preset fixed-time-step parameters into temporal embedding vectors via sine and cosine encoding and inject them into the adaptive layer normalization module of the diffusion Transformer network. The conditional input guided by the classifier is set to an empty label, allowing the diffusion Transformer network to perform forward propagation without class bias, thereby extracting the feature representations of each intermediate layer and the output layer, achieving deterministic, high signal-to-noise ratio biological feature extraction.

[0078] Step 206: Identify and suppress large-scale activation dimensions of intermediate layer features in the diffuse Transformer network, remove non-semantic dimension signals, and obtain biological features.

[0079] To eliminate the interference of the generation process on downstream classification tasks, a feature optimization strategy for DiT is introduced to actively identify the maximal activation values ​​in intermediate layer features caused by Adaptive Layer Normalization (AdaLN) and concentrated on specific dimensions. These high-frequency signals mainly encode information related to time steps or global generation conditions, rather than local semantics used to distinguish cell types. By masking or zeroing these dimensions where large-scale activations occur, non-semantic signals are accurately stripped away, significantly improving the signal-to-noise ratio of the remaining features used for the discrimination task.

[0080] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include a feature extraction process, specifically including: calculating the activation amplitude of the intermediate layer features of the diffusion Transformer network in each dimension; comparing the activation amplitude with a preset threshold to identify large-scale activation dimensions; zeroing or masking the large-scale activation dimensions to remove non-semantic dimension signals related to the diffusion time step and generation conditions, thereby obtaining biological features.

[0081] The computer device can calculate the activation statistics of each dimension (channel) of the feature tensor output by the intermediate layer of the diffusion Transformer network on the full dataset or the current batch of data. It sets a threshold to identify dimensions that exhibit consistently high activation in most input samples—these are called large-scale activation dimensions. Next, the computer device can perform correlation analysis between the activation patterns of these feature dimensions and known biological data. Based on the analysis results, it can suppress the identified large-scale activation dimensions by either zeroing out or masking them. This either sets the activation values ​​of all cells in that dimension to zero or masks out the feature transmission of that dimension without affecting the feature information of other normal dimensions. After suppressing large-scale activation dimensions, non-semantic interference signals are removed, retaining only high signal-to-noise ratio features used to distinguish cell types—these are the pure biological features used for subsequent fusion.

[0082] Specifically, the computer device can capture hierarchical information based on different layers of DiT, and the computer device can employ the learnable scalar mixing mechanism used in ELMo feature extraction. That is, in this embodiment, let... For the first The feature representation of each layer is used to calculate the Softmax normalized weights for each layer. Final expression This is the result of weighted aggregation: in, It is a learnable scalar. It is a global scaling parameter.

[0083] Because the features contain a very small number of abnormally large activation values, known as massive activations, these massive activations are not random noise but are persistently concentrated on a very small number of fixed dimensions of the feature vector. This phenomenon is essentially related to the Adaptive Layer Normalization (AdaLN) mechanism, where these high-amplitude, specific dimensions are primarily responsible for encoding and timing. This could be global information related to global generation conditions, rather than local semantic information used to distinguish cell types. Without processing, these dominant high-frequency signals would completely overwhelm crucial biological features.

[0084] Therefore, in this embodiment, for the first Layer characteristics Computer devices can identify sets of dimensions that exhibit large-scale activation and either zero them out or mask them: In this way, interference signals that are strongly coupled with the generation task can be accurately removed, significantly improving the signal-to-noise ratio of the remaining features in the downstream discrimination task.

[0085] Specifically, during feature extraction, the model can be placed in a deterministic intermediate state: injecting a fixed time... This puts the model in a sensitive state ready for denoising, without introducing actual noise variance; and it inputs clean latent vectors. Rather than noise Simultaneously set (CFG token) to extract unconditional, intrinsic biological characteristics.

[0086] Step 208: Softmax normalized weights of each layer are dynamically calculated using a learnable scalar mixing mechanism, and biological features of the multi-layer outputs of the diffusion Transformer network are dynamically weighted and fused according to the normalized weights to obtain aggregated features.

[0087] Considering that the optimal feature layers of different datasets vary significantly and that batch effects can cause systematic distortion of intermediate representations, this embodiment proposes a learnable scalar mixing mechanism. Based on the features of the input data, the Softmax normalization weights are dynamically calculated, and the feature representations of different layers of DiT are automatically weighted and fused. This mechanism can automatically select the layer combination containing the richest biological semantics, and adaptively mitigate batch interference while preserving basic biological variations.

[0088] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include a feature aggregation process. Specifically, this process includes: during the forward propagation of the diffusion Transformer network, collecting biological features output from each intermediate layer and assigning learnable scalar parameters to each biological feature; inputting the learnable scalar parameters into a Softmax function for normalization to obtain normalized weights for each layer; performing element-wise multiplication of the biological features of each layer with their corresponding normalized weights, and summing the results to obtain weighted and fused aggregated features.

[0089] The computer device can extract biological features from the intermediate outputs of multiple layers of a diffusion Transformer network after large-scale activation suppression processing. Shallow features retain cell data details and structural information, while deep features encode higher-order cell type semantic information. Next, a learnable scalar parameter can be assigned to each layer of the diffusion Transformer feature to characterize its importance for the cell type annotation task. All learnable scalars participate in end-to-end model training. The computer device can input the learnable scalar parameters corresponding to each layer into a Softmax function to normalize them into non-negative weight coefficients that sum to 1, obtaining the normalized fusion weights for each layer's features. Then, according to the Softmax normalized weights, the multi-layer biological features are weighted and fused to obtain aggregated features that take into account multi-level information and adaptively resist batch interference, which are used for subsequent lightweight fine-tuning and cell classification.

[0090] Step 210: Use low-rank adaptive fine-tuning technology to perform lightweight fine-tuning on the aggregated features, and input the fine-tuned feature vector into the multilayer perceptron classification head to obtain cell type annotation results.

[0091] To avoid catastrophic forgetting and reduce computational costs when adapting to annotation tasks specific to biological tissues, a low-rank adaptive (LoRA) fine-tuning technique can be applied to the fused features to enhance the model's ability to discriminate specific cell types without disrupting the general cell manifold structure learned during pre-training. Finally, the fine-tuned feature vectors are fed into a multilayer perceptron (MLP) classification head containing hidden projection layers, batch normalization, and ReLU activation functions, outputting the final cell type prediction probability.

[0092] In one embodiment, a single-cell type annotation method based on diffusion Transformer and adaptive representation learning may further include a fine-tuning and classification process. Specifically, this process includes: freezing the pre-trained weight matrix of the diffusion Transformer network; using low-rank adaptive fine-tuning technology to perform lightweight fine-tuning on the aggregated features, training only the low-rank parameters to obtain the fine-tuned feature vector; inputting the fine-tuned feature vector into a multilayer perceptron classification head, mapping it to the cell type category space through a nonlinear transformation; and calculating the predicted probability of each cell type based on the cell type category space using the Softmax function, determining the category corresponding to the highest probability as the cell type annotation result.

[0093] Before fine-tuning, the computer device can set the weight parameters of the diffusion Transformer network that generates aggregated features and its feature fusion module to a non-updateable state, i.e., "frozen", to ensure that the general biological knowledge learned in the pre-training stage is preserved and only the newly added adaptation modules are trained.

[0094] Computer devices can introduce low-rank adaptive techniques into the path of aggregated feature input to subsequent classification networks. During forward propagation, low-rank branch outputs are superimposed on the original feature calculations to achieve task-adaptive feature enhancement. The fine-tuning process does not disrupt the pre-trained cellular manifold structure, avoiding catastrophic forgetting, while significantly reducing computational and parameter costs. Specifically, during backpropagation, only the gradient of the low-rank matrix is ​​calculated and updated, while the frozen backbone network parameters do not participate in gradient updates, greatly reducing memory usage and computational overhead.

[0095] The pre-trained DiT contains general knowledge of cellular manifolds. To adapt to specific annotation tasks without catastrophic forgetting and with minimal computational cost, LoRA fine-tuning is implemented in this embodiment.

[0096] Specifically, for the pre-trained weight matrix in DiT ,freeze And introduce low-rank update ,in, and Is the rank Learnable matrix (here) Forward propagation becomes: in, It is a constant scaling factor. In this embodiment, a Gaussian distribution is used for initialization. Initialize with zero Ensure the initial stage of fine-tuning .

[0097] Next, the computer device can input the feature vector, which has undergone low-rank adaptive fine-tuning, into a multilayer perceptron (MLP) classification head. The MLP further extracts discriminative information from the features through nonlinear transformations. Finally, the last layer of the MLP maps the features to the cell type category space, outputs the probability distribution of each cell belonging to each cell type through the Softmax function, and selects the category with the highest probability as the final prediction result, thus completing the automatic cell type annotation for single-cell data.

[0098] In one embodiment, a multilayer perceptron classification head includes a hidden projection layer, a batch normalization layer, a ReLU activation function, and a linear layer mapped to the number of cell types. The multilayer perceptron classification head typically includes one or more fully connected layers, each followed by a non-linear activation function and a Dropout layer to prevent overfitting.

[0099] Specifically, the aggregated feature vector The data is fed into an MLP classification head for final cell type prediction. The classifier contains a hidden projection layer (2048 dimensions), BatchNormalization, a ReLU activation function, and a mapping to... The final linear layer of each cell type can be mathematically represented as: in, This is the weight matrix. This represents the ReLU activation function.

[0100] Model parameter optimization is achieved by minimizing the predicted probability distribution. With real labels This is achieved through the cross-entropy loss function between (One-hot encodings): in, For batch size, The total number of cell types is used as the basis for this discriminative training process, which enables the model to fully utilize the deep biological semantics extracted by DiT to achieve high-precision cell annotation.

[0101] In one embodiment, the specific process of the single-cell type annotation method scDiTA based on diffusion Transformer and adaptive representation learning is as follows: Figure 3 As shown, it mainly includes several parts: pre-trained diffusion generative model, adaptive guided generative feature extraction, LoRA fine-tuning and classification, and cell feature visualization.

[0102] Specifically, in the pre-trained diffusion generative model part, the original single-cell RNA sequencing data is input, and the high-dimensional single-cell data is reduced to a low-dimensional latent space through the PCA encoder to obtain latent variables; noise is added to the latent variables through the forward diffusion process, and then the diffusion Transformer model is used for denoising; the diffusion Transformer model is used to learn how to recover the original data from the noise, and outputs a pre-trained diffusion Transformer model that can generate high-quality latent variables.

[0103] In the adaptive guided generative feature extraction part, noise-free latent variables and fixed time step parameters are input. The diffusion Transformer model trained in the first step is used, but its weights are frozen so that they are no longer updated. In the inference stage, the generation process is adjusted by a guiding factor to enhance the expression of specific features, thereby performing deterministic feature extraction and outputting multi-layer features extracted from the diffusion Transformer network.

[0104] In the LoRA fine-tuning and classification section, the extracted multi-layer features are input. By identifying and suppressing the dimensions of large-scale activation in the intermediate layer features of the diffusion Transformer network, signals with non-semantic dimensions are removed to obtain purer biological features. Through a learnable scalar mixing mechanism, the Softmax normalized weights of each layer feature are dynamically calculated, and the biological features output from multiple layers are dynamically weighted and fused to obtain an aggregated feature. A low-rank adaptive fine-tuning technique is used to perform lightweight fine-tuning on the aggregated feature, which can greatly reduce the number of parameters that need to be trained. The fine-tuned feature vector is input into a multi-layer perceptron classification head to finally obtain the cell type annotation results. The cell type annotation is output.

[0105] In the cell feature visualization section, the cell features obtained after input processing are used to reduce the dimensionality of the high-dimensional cell features through the UMAP algorithm, so that they can be visualized in two-dimensional or three-dimensional space. The output UMAP map can intuitively show the distribution of different cell types in the feature space and verify the effectiveness of the method.

[0106] In this embodiment, by transforming the pre-trained Diffusion Transformer (DiT) from a generative model into a feature extractor, the high-dimensional manifold structure of single-cell data is captured in depth, thereby improving the accuracy and robustness of cell type annotation while resisting strong noise and batch effects.

[0107] It should be understood that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart above may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0108] In one embodiment, such as Figure 4 As shown, a single-cell type annotation system based on diffusion Transformer and adaptive representation learning is provided, including: a data processing and network training module 410, a deterministic feature extraction module 420, an activation and inhibition module 430, a learnable scalar mixture module 440, and a fine-tuning and classification module 450, wherein:

[0109] The data processing and network training module 410 is used to acquire the single-cell RNA sequencing data to be annotated, perform preprocessing and principal component analysis for dimensionality reduction, obtain latent variables in the low-dimensional latent space, and train the diffusion Transformer network based on the latent variables using the optimal transport flow matching strategy.

[0110] The deterministic feature extraction module 420 is used to freeze the weights of the trained diffusion Transformer network, input the noise-free latent variables and fixed time step parameters into the frozen weights of the diffusion Transformer network, and perform deterministic feature extraction using classifier-free guidance.

[0111] The activation suppression module 430 is used to identify and suppress the large-scale activation dimension of the intermediate layer features of the diffuse Transformer network, remove non-semantic dimension signals, and obtain biological features.

[0112] The learnable scalar mixing module 440 is used to dynamically calculate the Softmax normalized weights of each layer through a learnable scalar mixing mechanism, and dynamically weight and fuse the biological features of the multi-layer outputs of the diffusion Transformer network according to the normalized weights to obtain aggregated features.

[0113] The fine-tuning and classification module 450 is used to perform lightweight fine-tuning of aggregated features using low-rank adaptive fine-tuning technology. The fine-tuned feature vector is then input into the multilayer perceptron classification head to obtain cell type annotation results.

[0114] In one embodiment, the data processing and network training module 410 is further used to acquire single-cell RNA sequencing data to be annotated, and to perform log normalization on the single-cell RNA sequencing data to be annotated to screen out hypervariable genes; to perform principal component analysis on the hypervariable genes, and to project them onto a low-dimensional latent space of a preset dimension to obtain the corresponding latent variables.

[0115] In one embodiment, the data processing and network training module 410 is further configured to construct a continuous probability path from the standard Gaussian noise distribution to the latent variable data distribution, and define a time step based on the standard Gaussian noise distribution and the latent variable data distribution; for each latent variable sample and its corresponding noise sample, construct an intermediate state through linear interpolation, and calculate the optimal velocity field on the continuous probability path; construct a diffusion Transformer network, and optimize the parameters of the diffusion Transformer network using a flow matching loss function based on the time step and the optimal velocity field, thereby completing the training of the diffusion Transformer network.

[0116] In one embodiment, the deterministic feature extraction module 420 is further configured to set all parameters of the trained diffusion Transformer network to a non-updateable state, thereby completing the weight freezing of the diffusion Transformer network; select fixed time step parameters and convert them into temporal embedding vectors and noise-free latent variables as inputs to the diffusion Transformer network, set the conditional input guided by the classifier to an empty label; perform forward propagation through the diffusion Transformer network, output the feature representations of each hidden layer, and extract deterministic features.

[0117] In one embodiment, the activation suppression module 430 is further configured to calculate the activation amplitude of the intermediate layer features of the diffusion Transformer network in each dimension; compare the activation amplitude with a preset threshold to identify large-scale activation dimensions; and perform zeroing or masking on the large-scale activation dimensions to remove non-semantic dimension signals related to the diffusion time step and generation conditions, thereby obtaining biological features.

[0118] In one embodiment, the learnable scalar fusion module 440 is further configured to collect biological features output from each intermediate layer during the forward propagation of the diffuse Transformer network, and assign learnable scalar parameters to each biological feature; input the learnable scalar parameters into the Softmax function for normalization processing to obtain the normalized weights corresponding to each layer; perform element-wise multiplication operations on the biological features of each layer and the corresponding normalized weights, and sum the results to obtain the weighted fused aggregated features.

[0119] In one embodiment, the fine-tuning and classification module 450 is further used to freeze the pre-trained weight matrix of the diffusion Transformer network, use low-rank adaptive fine-tuning technology to perform lightweight fine-tuning on the aggregated features, and train only the low-rank parameters to obtain the fine-tuned feature vector; input the fine-tuned feature vector into the multilayer perceptron classification head, and map it to the cell type category space through nonlinear transformation; based on the cell type category space, calculate the predicted probability of each cell type through the Softmax function, and determine the category corresponding to the highest probability as the cell type annotation result.

[0120] In one embodiment, the multilayer perceptron classification head includes a hidden projection layer, a batch normalization layer, a ReLU activation function, and a linear layer mapped to the number of cell types.

[0121] In one embodiment, to verify the provided single-cell type annotation method and system based on diffusion Transformer and adaptive representation learning, the experimental procedure is as follows:

[0122] Data preprocessing and feature dimensionality reduction: The original single-cell gene expression matrix was preprocessed according to standard methods (log normalization and screening of the top 2000 hypervariable genes). Then, principal component analysis (PCA) was used to project the sparse high-dimensional gene expression profile into a compact continuous latent space to provide input for the generative model.

[0123] Generative pre-training of the DiT model: Based on latent variables, the diffusion Transformer (DiT) network is trained using the optimal transport flow matching (OT-FM) strategy. The model learns the probabilistic flow structure of cells and the complex nonlinear dependencies between genes in the latent space by constructing smooth training trajectories.

[0124] Setting up denoising and feature filtering: Freeze the trained DiT weights, input clean latent variables with fixed time step parameters, and set the classifier guide to null-CFG. Then, identify and suppress the large-scale activation dimension caused by the AdaLN mechanism in the network, and filter out non-semantic generated signals.

[0125] Adaptive Scalar Mixing of Features: The system calculates the Softmax normalized weights of different hidden layers in DiT through a learnable scalar mixing mechanism. This mechanism dynamically assigns weights based on the input data features, automatically fusing shallow details with deep biological semantics to output aggregated features.

[0126] Lightweight fine-tuning and terminal classification prediction: The aggregated features are input into a low-rank adaptive (LoRA) module for lightweight fine-tuning to adapt to the specific single-cell annotation task. The final processed feature vector is then fed into a multilayer perceptron (MLP) classification head, where the predicted probability distribution of each cell type is calculated using the Softmax function to obtain the final annotation result.

[0127] Experiments show that, for example Figure 5 As shown, compared with other single-cell annotation methods, scDiTA in this application significantly outperforms mainstream methods such as scRGCL and ACTINN in both accuracy and macro F1 score on multiple benchmark datasets. In particular, it maintains extremely high accuracy when dealing with ultra-large-scale data of more than 170,000 cells.

[0128] like Figure 6 As shown, scDiTA has gained a deep understanding of the intrinsic manifold structure of cell data through generative pre-training. The extracted features perform well on clustering metrics (such as ASW and ARI), enabling cells of the same type to form well-defined and compact clusters in the feature space, thus overcoming the shortcomings of existing models with fuzzy feature spaces.

[0129] Ablation experiment such as Figure 7 , 8 As shown, the introduction of LoRA fine-tuning and a learnable scalar hybrid strategy can adaptively fuse the best features from different layers. Combined with an inference strategy with a fixed time step (e.g., t=500), the model enters the optimal denoising state, significantly improving feature extraction efficiency.

[0130] like Figure 9 As shown, unlike traditional methods that force alignment, scDiTA adaptively preserves the batch structure of the original data (iLISI scores are almost identical to the original data). This effectively avoids the loss of biological heterogeneity caused by forced fusion and still achieves high-precision annotation under strong batch interference.

[0131] like Figure 10 As shown, even when dealing with extremely sparse data with up to 50% noise, or with very few samples using only 10% of the training data, scDiTA still maintains extremely high annotation accuracy thanks to the manifold prior knowledge obtained through pre-training, and its noise resistance and generalization ability far exceed those of the comparison models.

[0132] like Figure 11 As shown, when encountering cell types not seen in the training set, the model readily and significantly reduces its prediction confidence, effectively avoiding the risk of misclassification. Furthermore, the framework exhibits extremely high scalability, achieving near-state-of-the-art performance even when using only a simple linear classification head.

[0133] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a single-cell type annotation method based on diffusion Transformer and adaptive representation learning. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0134] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement steps of a single-cell type annotation method based on diffusion Transformer and adaptive representation learning.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a single-cell type annotation method based on diffusion Transformer and adaptive representation learning.

[0137] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A single-cell type annotation method based on diffusion Transformer and adaptive representation learning, characterized in that, The method includes: The single-cell RNA sequencing data to be annotated is preprocessed and subjected to principal component analysis for dimensionality reduction to obtain latent variables in the low-dimensional latent space. Based on the latent variables, a diffusion Transformer network is trained using the optimal transport flow matching strategy. The weights of the trained diffusion Transformer network are frozen, and the noise-free latent variables and fixed time step parameters are input into the diffusion Transformer network with frozen weights. Deterministic feature extraction is performed using a classifier-free approach. Identify and suppress large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network, and remove non-semantic dimension signals to obtain biological features; The Softmax normalized weights of each layer are dynamically calculated through a learnable scalar mixing mechanism, and the biological features of the multi-layer outputs of the diffusion Transformer network are dynamically weighted and fused according to the normalized weights to obtain aggregated features. The aggregated features are lightly fine-tuned using low-rank adaptive fine-tuning technology. The fine-tuned feature vector is then input into the multilayer perceptron classification head to obtain cell type annotation results.

2. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, Single-cell RNA sequencing data to be annotated were preprocessed and subjected to principal component analysis for dimensionality reduction to obtain latent variables in the low-dimensional latent space, including: Obtain the single-cell RNA sequencing data to be annotated, and perform log normalization on the single-cell RNA sequencing data to be annotated to screen out hypervariable genes. Principal component analysis was performed on the hypervariable genes, and the results were projected onto a low-dimensional latent space of a preset dimension to obtain the corresponding latent variables.

3. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, Based on the aforementioned latent variables, a diffusion Transformer network is trained using an optimal transport flow matching strategy, including: Construct a continuous probability path from the standard Gaussian noise distribution to the latent variable data distribution, and define the time step according to the standard Gaussian noise distribution and the latent variable data distribution; For each latent variable sample and its corresponding noise sample, an intermediate state is constructed by linear interpolation, and the optimal velocity field on the continuous probability path is calculated. A diffusion Transformer network is constructed. Based on the time step and the optimal velocity field, the parameters of the diffusion Transformer network are optimized using the flow matching loss function to complete the training of the diffusion Transformer network.

4. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, The weights of the trained diffusion Transformer network are frozen. The noise-free latent variables and fixed-time-step parameters are input into the frozen-weight diffusion Transformer network, and deterministic feature extraction is performed using a classifier-free approach, including: Set all parameters of the trained Diffusion Transformer network to a non-updateable state to complete the weight freeze of the Diffusion Transformer network. Select fixed time step parameters and convert them into time embedding vectors and noise-free latent variables as inputs to the diffusion Transformer network, and set the classifier-guided conditional inputs to empty labels; Forward propagation is performed using a diffusion Transformer network to output the feature representations of each hidden layer, thereby extracting deterministic features.

5. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, Identify and suppress large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network, remove non-semantic dimension signals, and obtain biological features, including: Calculate the activation magnitude of the intermediate layer features of the diffusion Transformer network in each dimension; The activation amplitude is compared with a preset threshold to identify large-scale activation dimensions; The large-scale activation dimensions are zeroed or masked to remove non-semantic dimension signals related to the diffusion time step and generation conditions, thus obtaining biological features.

6. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, The Softmax normalized weights of each layer are dynamically calculated using a learnable scalar mixing mechanism, and the biological features output from multiple layers of the diffusion Transformer network are dynamically weighted and fused based on these normalized weights to obtain aggregated features, including: During the forward propagation of the diffusion Transformer network, biological features output from each intermediate layer are collected, and learnable scalar parameters are assigned to each biological feature. The learnable scalar parameters are input into the Softmax function for normalization to obtain the normalized weights for each layer. The biological features of each layer are multiplied element-wise with their corresponding normalized weights, and the results are summed to obtain the weighted and fused aggregated features.

7. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 1, characterized in that, The aggregated features are lightly fine-tuned using a low-rank adaptive fine-tuning technique. The fine-tuned feature vector is then input into the multilayer perceptron classification head to obtain cell type annotation results, including: Freeze the pre-trained weight matrix of the diffusion Transformer network, use low-rank adaptive fine-tuning technique to perform lightweight fine-tuning on the aggregated features, and train only the low-rank parameters to obtain the fine-tuned feature vector. The finely tuned feature vector is input into the multilayer perceptron classification head and mapped to the cell type category space through a nonlinear transformation. Based on the cell type category space, the predicted probability of each cell type is calculated using the Softmax function, and the category corresponding to the highest probability is determined as the cell type annotation result.

8. The single-cell type annotation method based on diffusion Transformer and adaptive representation learning according to claim 7, characterized in that, The multilayer perceptron classification head includes a hidden projection layer, a batch normalization layer, a ReLU activation function, and a linear layer mapped to the number of cell types.

9. A single-cell type annotation system based on diffusion Transformer and adaptive representation learning, characterized in that, The system includes: The data processing and network training module is used to acquire the single-cell RNA sequencing data to be annotated, perform preprocessing and principal component analysis for dimensionality reduction, obtain latent variables in the low-dimensional latent space, and train the diffusion Transformer network based on the latent variables using the optimal transport flow matching strategy. The deterministic feature extraction module is used to freeze the weights of the trained diffusion Transformer network, input the noise-free latent variables and fixed time step parameters into the frozen weights of the diffusion Transformer network, and perform deterministic feature extraction using classifier-free guidance. An activation suppression module is used to identify and suppress large-scale activation dimensions of the intermediate layer features of the diffuse Transformer network, eliminate non-semantic dimension signals, and obtain biological features. The learnable scalar mixing module is used to dynamically calculate the Softmax normalized weights of each layer through a learnable scalar mixing mechanism, and to dynamically weight and fuse the biological features output by the multi-layer of the diffusion Transformer network according to the normalized weights to obtain aggregated features. The fine-tuning and classification module is used to perform lightweight fine-tuning of the aggregated features using low-rank adaptive fine-tuning technology, and input the fine-tuned feature vector into the multilayer perceptron classification head to obtain cell type annotation results.

Images