Single cell clustering method and system based on multi-granularity Transform

Abstract

The invention belongs to the technical field of single cell clustering, and particularly relates to a single cell clustering method and system based on multi-granularity Transform. In order to solve the problems of insufficient cell genome relation extraction, limited clustering accuracy and the like in scRNA-seq data, the method provides a multi-granularity feature learning framework, hierarchical Token division is carried out on gene expression data, multi-level association from local gene collaboration to a global function module is captured by utilizing an attention mechanism, and the clustering accuracy is improved. And thus, cell global representation with higher discrimination is constructed. Furthermore, by introducing a cluster contrast learning mechanism, feature collision possibly caused by negative sample pairs in traditional contrast learning is avoided, and the clustering friendliness of a feature space is enhanced.

Description

A Single-Cell Clustering Method and System Based on Multi-Granularity Transformer Technical Field

[0001] This invention belongs to the field of single-cell clustering technology, specifically relating to a single-cell clustering method and system based on multi-granularity Transformer. Background Technology

[0002] Intracellular gene expression is a core indicator characterizing cellular state and function. In recent years, breakthroughs in single-cell RNA sequencing (scRNA-seq) technology have enabled high-precision measurement of the entire transcriptome expression profile at the individual cell level, overcoming the limitations of traditional batch sequencing and providing a powerful tool for understanding key biological issues such as cellular heterogeneity, type diversity, state dynamics, and lineage evolution. In this technological workflow, single-cell clustering is a crucial step, aiming to classify a large number of cells into biologically significant categories based on the similarity of gene expression patterns. The results directly impact the reliability of downstream analyses such as cell type annotation, differential gene identification, and trajectory inference, serving as the cornerstone of the overall analytical process.

[0003] However, the inherent technical characteristics and biological complexity of scRNA-seq data present a severe dual challenge to clustering tasks. First, there is the issue of high dimensionality: each cell requires the simultaneous detection of the expression levels of tens of thousands of genes, forming a data matrix with extremely high feature dimensions (typically 2000-50000 genes) and a relatively limited sample size. This not only leads to the "curse of dimensionality," rendering traditional metrics such as Euclidean distance ineffective and drastically increasing computational complexity, but more seriously, in high-dimensional space, irrelevant noise features may drown out meaningful biological signals, thus masking the true expression differences between cell types. Second, there is the sparsity and high dropout rate of the data: limited by the detection sensitivity of current scRNA-seq technology, a large number of low-abundance mRNA molecules are not successfully captured during library construction, resulting in numerous "false zeros" (dropout events), making the expression matrix highly sparse. This technical noise severely interferes with the accurate assessment of true biological similarity between cells and simulates non-existent cell subpopulations. Faced with these challenges, traditional clustering methods (such as K-means, hierarchical clustering, DBSCAN, etc.) are often limited in their performance when dealing with high-dimensional, sparse and noisy scRNA-seq data because they were originally designed for low-dimensional, dense and well-distributed data. They are difficult to effectively identify complex and continuous cell population structures.

[0004] To address these challenges, deep learning techniques have been gradually introduced into the field of single-cell clustering. In recent years, contrastive learning-based single-cell clustering methods have constructed different expression views of cells as positive sample pairs through data augmentation strategies, enabling the model to learn feature representations robust to dropout noise, thereby improving feature quality. However, existing methods generally neglect a crucial source of information: the associations between genes (such as functional synergy or sequence homology). This type of genomic-level association information is crucial for cell type discrimination; its absence leads to insufficient discriminative power of extracted cell features, limiting further improvements in clustering performance. The Transformer architecture, with its self-attention mechanism, has demonstrated excellent relational modeling capabilities in fields such as natural language processing and computer vision. Introducing it into single-cell clustering tasks and designing attention mechanisms adapted to genomic data holds promise for fully exploring the complex genomic associations between cells, thereby enhancing feature discriminative power, suppressing the impact of noise, and providing a new technical path for improving clustering accuracy. Therefore, how to effectively integrate genomic association information from scRNA-seq data to guide the model in learning more discriminative cell embedding representations has become a core issue in improving the accuracy of single-cell clustering. Summary of the Invention

[0005] To address the issues of insufficient extraction of cellular genomic relationships and limited clustering accuracy in scRNA-seq data, this invention provides a single-cell clustering method and system based on multi-granularity Transformer. Through multi-granularity feature extraction, dual-encoder training, and hybrid loss optimization, high-quality cell clustering is achieved. Specifically, this invention proposes a multi-granularity feature learning framework. By performing hierarchical tokenization on gene expression data, it utilizes an attention mechanism to capture multi-level associations from local gene collaboration to global functional modules, thereby constructing a more discriminative global cell representation. This method effectively alleviates the feature learning difficulties caused by the high dimensionality, sparsity, and noise interference of scRNA-seq data. Furthermore, by introducing a cluster contrastive learning mechanism, it avoids feature collisions that may be caused by negative sample pairs in traditional contrastive learning, enhancing the clustering friendliness of the feature space.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] This invention provides a single-cell clustering method based on multi-granularity Transformer, comprising the following steps:

[0008] Step 1: Preprocess scRNA-seq data to screen high-quality samples and key gene features from the raw scRNA-seq data;

[0009] Step 2: Augment the preprocessed scRNA-seq data;

[0010] Step 3: Construct a multi-granularity gene coding model based on multi-granularity Transformer, including a multi-granularity online gene encoder, a multi-granularity gene target encoder, and a feature reprojection head;

[0011] Step 4: Train the single-cell clustering model, and design an optimizer and loss function to update the single-cell clustering model;

[0012] Step 5: Use the trained single-cell clustering model to perform cell clustering based on scRNA-seq data.

[0013] Furthermore, step 1 specifically includes:

[0014] Standardized scRNA-seq data were collected and screened. Low-quality cells and low-expression genes were filtered out, and highly variable genes were screened based on the mean-dispersion model to construct a high-quality input matrix. The matrix was then standardized to eliminate technical bias, resulting in a standardized expression matrix. .

[0015] Furthermore, step 2 specifically includes:

[0016] Step 21, standardize the expression matrix. Perform batch partitioning;

[0017] Step 22: Augment scRNA-seq data. For each cell in the original cellular gene expression profile, Gaussian noise is randomly added with probability p to generate corresponding augmented samples. The normalized expression matrix is ​​then processed. Perform two augmentations to obtain augmented data. , .

[0018] Furthermore, step 3 specifically includes:

[0019] Step 31: Construct and initialize a multi-granularity online gene encoder. Using TNT as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is designed. First, gene features are segmented into coarse-grained tokens to extract global gene associations. Then, the coarse-grained tokens are segmented into fine-grained tokens to capture local gene synergies.

[0020] Step 32: Construct and initialize a multi-granularity gene target encoder. Using TNT as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is included to capture the association features between genes at different granularities. The parameter update method is shown in Equation (1):

[0021]

[0022] in, The parameters represent the multi-granularity online gene encoder. Parameters representing multi-granularity gene target encoders, This is a smoothing coefficient used to control the magnitude of parameter updates;

[0023] Step 33: Construct and initialize the feature reprojection head : This is a multi-layer nonlinear perceptron network containing 4 linear layers. The first, second, and third linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer. The fourth linear layer maps the feature dimension back to the original input dimension and initializes the parameters of the feature reprojection head.

[0024] Furthermore, step 4 specifically involves:

[0025] Step 41, process the scRNA-seq data from step 22. Input the multi-granularity online gene encoder constructed in step 31 In, and augment the data in step 22. Input the multi-granularity gene target encoder constructed in step 32 In this process, a dual-path encoding mechanism is used to obtain the original feature representation from the input raw data. 1. Input augmented data to obtain online vector representations With target vector representation ;

[0026] Step 42, based on the online vector representation obtained in step 41 The soft k-means algorithm is used to calculate the probability distribution of each cell belonging to each cluster, generating a soft assignment matrix. The matrix is ​​separated from the computation graph and transformed into a static soft-assignment tensor. Used for subsequent loss calculation;

[0027] Step 43, represent the online vector from step 41. With target vector representation By inputting multiple different reprojection heads, diverse reprojection feature representations are generated. , , , By representing the original features The reprojected feature representation is then subjected to an equal-weighted average, and multi-view information is fused to generate the final weighted feature. , As shown in the following formula:

[0028]

[0029]

[0030] Step 44, based on the weighted features generated in step 43 , and the soft allocation tensor of step 42 Calculate the cluster centers of the two groups respectively. , As the basis for prototype contrastive learning, cluster centers are obtained through a soft-assigned probability weighted average, as shown in the following formula:

[0031]

[0032]

[0033] in, Represents cells Belongs to cluster The soft assignment probability, Represents cells Feature representation;

[0034] Step 45, calculate the mean squared error loss. : By calculating weighted features under the enhanced view The mean square error between them is shown in the following formula:

[0035]

[0036] in, Represents a cell characteristic index. This represents the number of cell samples in the current training batch.

[0037] Step 46, Calculate the prototype contrast loss For each cluster center, a prototype is generated using a multi-granularity online gene encoder. As the anchor point, its corresponding target prototype As positive samples, the remaining target prototypes As negative samples, a contrastive loss function based on cosine similarity is used, as shown in the following equation:

[0038]

[0039] in, Represents cosine similarity. For temperature hyperparameters, The total number of clusters, For clustering index;

[0040] Step 47, Calculate the overall loss of the model. The mean squared error loss is added to the prototype comparison loss, and the calculation formula is as follows:

[0041]

[0042] in, To balance the hyperparameters of the two loss weights;

[0043] Step 48: Repeat steps 41 to 47 iteratively to train the entire multi-granularity gene coding model end-to-end. The optimization process uses stochastic gradient descent or Adam optimizer. Set the initial learning rate and weight decay rate, and dynamically adjust the learning rate according to the training progress. The model is trained until the preset maximum number of iterations is reached or the loss function converges, and finally a multi-granularity gene coding model that can extract discriminative feature representations is obtained.

[0044] Furthermore, step 41 specifically includes:

[0045] Augmented matrix and It is divided into T coarse-grained tokens to form tokenized data. As shown in the following formula:

[0046]

[0047] Each coarse-grained token is... A series of highly variable genes constitute the structure shown in formula (9), while a linear projection layer is formed. Map each token to the embedding space to obtain a coarse-grained embedding. ;

[0048]

[0049] in, Indicates the first A coarse-grained token;

[0050] For fine-grained partitioning, each coarse-grained token is further subdivided to construct fine-grained tokenized data. As shown in the following formula:

[0051]

[0052] in, The number of genes for fine-grained tokens. The number of times each coarse-grained token is split. Indicates which fine-grained token;

[0053] Through the internal projection layer Will Mapping to fine-grained embedding At a fine-grained level, a multi-head self-attention mechanism is introduced to... Processing is performed using a learnable weight matrix. Calculate query ,key ,value The vector is shown in the following equation:

[0054]

[0055] The attention weights and output are calculated as follows:

[0056]

[0057] in, For scaling dot product attention functions, Where is the dimension of the key vector, and T is the number of coarse-grained tokens;

[0058] Let the number of attention heads be Then the multi-head attention output fine-grained feature representation As shown in the following formula:

[0059]

[0060]

[0061] in, Indicates the attention head index, This is a vector concatenation operation. The output projection matrix is ​​trainable;

[0062] The enhanced fine-grained representation is obtained through residual connections and layer normalization operations: Remodeled into Through linear layer Projecting onto the coarse-grained dimension yields , With coarse-grained embedding Perform residual fusion to obtain the fused representation. , fusion Further apply coarse-grained multi-head self-attention and calculate the attention output. After further processing via residual connections, layer normalization, and feedforward networks, the output is finally reshaped into a cell-level online vector representation. ;

[0063] Multi-granularity gene target encoders process augmented data using the same structure Obtain the target vector representation .

[0064] The present invention also provides a single-cell clustering system based on multi-granularity Transformer, including a computer processor, a single-cell dataset storage unit, a data processing unit, a multi-granularity gene coding model training unit, and a clustering unit;

[0065] The single-cell dataset storage unit stores the constructed scRNA-seq dataset and loads it into computer memory;

[0066] The data processing unit retrieves the scRNA-seq dataset from computer memory and performs augmentation processing to obtain augmented data;

[0067] The multi-granularity gene coding model training unit trains the model, determines the model training parameters, and obtains the multi-granularity gene coding model.

[0068] The clustering unit performs cell clustering to obtain the result of single-cell clustering;

[0069] The specific data processing and calculation work in all units is performed by the computer processor.

[0070] Compared with the prior art, the present invention has the following advantages:

[0071] 1. Unlike existing methods that typically treat genes as independent features or perform only single-scale analysis, this invention innovatively constructs scRNA-seq data as multi-level tokens: "complete genome → coarse-grained gene fragments → fine-grained gene fragments." Through a complementary dual attention mechanism, the model can simultaneously and explicitly capture global functional associations at the coarse-grained level and local co-expression patterns at the fine-grained level. This multi-scale information integration from macro to micro levels enables the model to learn more discriminative and biologically meaningful cellular feature representations, effectively overcoming the challenges of signal sparsity and noise interference in high-dimensional data.

[0072] 2. This invention can effectively handle single-cell clustering tasks. By establishing a multi-granularity genome representation to construct global features, it can effectively realize cell feature representation and improve the accuracy of clustering.

[0073] 3. This invention effectively improves the robustness and clustering performance of the model, enabling the model to learn both fine-grained local gene patterns and coarse-grained global cell structures simultaneously, significantly improving clustering accuracy and feature representation quality. Attached Figure Description

[0074] Figure 1 is a structural diagram of the single-cell clustering model of the present invention;

[0075] Figure 2 is a structural diagram of the encoder of the present invention;

[0076] Figure 3 is a flowchart of the single-cell clustering method based on multi-granularity Transformer of the present invention;

[0077] Figure 4 is a structural diagram of the single-cell clustering system based on multi-granularity Transformer of the present invention. Detailed Implementation

[0078] To further illustrate the technical solution of the present invention, the present invention will be further described below through embodiments.

[0079] This embodiment uses the QS_Diaph dataset (Quake Smart-seq2 Diaphragm) as an example to specifically illustrate the method of the present invention. This dataset is derived from mouse diaphragm tissue, obtained by the Quake laboratory using Smart-seq2 sequencing technology. It contains approximately 870 single cells and about 23,000 genes, covering five cell types. The QS_Diaph dataset is constructed based on single-cell transcriptome sequencing results and mainly includes four types of information: single-cell gene expression matrix, cell type annotation information, tissue origin and experimental batch information, and gene dictionary information. In the implementation of this invention, the single-cell gene expression matrix in this dataset was preprocessed to support subsequent analysis and model construction.

[0080] As shown in Figure 1, a single-cell clustering method based on multi-granularity Transformer in this embodiment includes the following steps:

[0081] Step 1: Preprocess scRNA-seq data. Select high-quality samples and key gene features from the raw scRNA-seq data to construct a standardized dataset suitable for model training. Specifically, this includes extracting and parsing gene expression matrix, cell observation information (including cell type annotation), gene variable information, and unstructured metadata from the HDF5 format dataset.

[0082] Standardized scRNA-seq data from authoritative international databases were collected and screened. Low-quality cells and low-expression genes were filtered out, and highly variable genes (HVGs) were screened based on a mean-dispersion model to construct a high-quality input matrix. The expression matrix was then standardized to eliminate technical bias, resulting in a final standardized expression matrix. ,in Cell count, The number of highly variable genes is typically 1000-2000.

[0083] Step 2 involves augmenting the preprocessed scRNA-seq data, specifically as follows:

[0084] Step 21: To balance the computational efficiency and generalization performance of model training, the standardized representation matrix needs to be adjusted according to the dataset size and hardware resources. Perform reasonable batch partitioning, make full use of computing resources, and set an appropriate batch size for the entire dataset. The scRNA-seq data was divided into... One batch;

[0085] Step 22: Augment scRNA-seq data. For each cell in the original cell gene expression profile, Gaussian noise with a mean of 0 and a standard deviation of 0.5 is randomly added with a probability of p=0.7 or 0.8 to generate corresponding augmented samples. The normalized expression matrix is ​​then... Perform two augmentations to obtain augmented data. , This provides ample data support for the subsequent dual encoder training and comparative learning of the single-cell clustering model based on multi-granularity Transformer.

[0086] Step 3: Construct a multi-granularity gene coding model based on multi-granularity Transformer (Figure 2), including a multi-granularity online gene encoder (Figure 3), a multi-granularity gene target encoder (Figure 3), and a feature reprojection head, specifically as follows:

[0087] Step 31: Construct and initialize a multi-granularity online gene encoder. Using TNT (Transformer in Transformer) as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is designed. First, gene features are segmented into coarse-grained tokens to extract global gene associations. Then, the coarse-grained tokens are segmented into fine-grained tokens to capture local gene synergies.

[0088] Step 32: Construct and initialize a multi-granularity gene target encoder. Using TNT as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is included to capture the association features between genes at different granularities. The parameter update method is shown in Equation (1):

[0089]

[0090] in, The parameters represent the multi-granularity online gene encoder. Parameters representing multi-granularity gene target encoders, This is a smoothing coefficient used to control the magnitude of parameter updates;

[0091] Step 33: Construct and initialize the feature reprojection head : This is a multi-layer nonlinear perceptron network containing 4 linear layers. The first, second, and third linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer. The fourth linear layer maps the feature dimension back to the original input dimension and initializes the parameters of the feature reprojection head.

[0092] Step 4: Train the single-cell clustering model. Design an optimizer and loss function to update the single-cell clustering model. In this embodiment, the initial learning rate is set to 5e-4, and a cosine annealing scheduler is used for adjustment. The weight decay is set to 1e-5 to prevent overfitting. The momentum update coefficients of the target encoder are... The value was set to 0.99 to ensure smooth parameter changes. The process employed an end-to-end training approach, combining contrastive learning and clustering constraints, training for a total of 500 epochs. The model stabilized after approximately 300 epochs. Specifically:

[0093] Step 41, process the scRNA-seq data from step 22. Input the multi-granularity online gene encoder constructed in step 31 In, and augment the data in step 22. Input the multi-granularity gene target encoder constructed in step 32 In this process, a dual-path encoding mechanism is used to obtain the original feature representation from the input raw data. 1. Input augmented data to obtain online vector representations With target vector representation The specific steps are as follows:

[0094] First, it undergoes multi-granularity structuring: augmented matrix and It is divided into T=50 coarse-grained tokens to form tokenized data. Where N=870, T=50, and S=30 is the number of genes contained in each coarse-grained token. The sum of the products equals the number of hypervariable genes, as shown in the following formula:

[0095]

[0096] Each coarse-grained token is... =30 consecutive highly variable genes constitute the structure shown in formula (9), and a linear projection layer = Each token is mapped to the embedding space to obtain a coarse-grained embedding. Where N=870, T=50, and E=30 represent the coarse-grained embedding dimensions, as shown in the following formula:

[0097]

[0098] in, Indicates the first A coarse-grained token;

[0099] For fine-grained partitioning, simply repeat the same operation based on any of the methods described above. Taking the sequence block enumeration method as an example, each coarse-grained token is further partitioned to construct fine-grained tokenized data. Where N=870, T=50, U=3, I=10, as shown in the following formula:

[0100]

[0101] in, The number of genes for fine-grained tokens. The number of times each coarse-grained token is split. This indicates the nth fine-grained token. Through two levels of "division," a hierarchical feature extraction structure is constructed, from the complete genome → coarse-grained gene fragments → fine-grained gene fragments. This is the core of the single-cell clustering method based on multi-granularity Transformer.

[0102] Through the internal projection layer Will Mapping to fine-grained embedding 4 represents the fine-grained embedding dimension. At the fine-grained level, a multi-head self-attention mechanism is introduced. Processing is performed using a learnable weight matrix. Calculate query ,key ,value The vector is shown in the following equation:

[0103]

[0104] The attention weights and output are calculated as follows:

[0105]

[0106] in, The scaling dot product attention function's core function is to dynamically calculate attention weights based on query-key matching and to weight and fuse value vectors, thereby enhancing the model's ability to capture fine-grained nonlinear associations between genes. This allows each gene segment to adaptively aggregate global contextual information based on its semantic relevance to other segments in the sequence, effectively improving the robustness of feature representations to dropout noise. Where is the dimension of the key vector, and T is the number of coarse-grained tokens;

[0107] Let the number of attention heads be Then the multi-head attention output fine-grained feature representation As shown in the following formula:

[0108]

[0109]

[0110] in, Indicates the attention head index, This is a vector concatenation operation, responsible for... The heterogeneous features learned by each attention head in different representation subspaces are integrated to form a unified multi-view feature representation. This is then achieved through a trainable output projection matrix. The concatenated high-dimensional features are fused and dimensionality reduced to finally output a fine-grained feature representation that is both discriminative and compact. ;

[0111] The enhanced fine-grained representation is obtained through residual connections and layer normalization operations: Remodeled into Through linear layer Projecting onto the coarse-grained dimension yields , With coarse-grained embedding Perform residual fusion to obtain the fused representation. , fusion Further apply coarse-grained multi-head self-attention and calculate the attention output. After further processing via residual connections, layer normalization, and a feedforward network (FFN), the output is finally reshaped into a cell-level online vector representation. ,in This represents the final feature dimension of the cell;

[0112] Multi-granularity gene target encoders process augmented data using the same structure Obtain the target vector representation .

[0113] Step 42, based on the online vector representation obtained in step 41 The soft k-means algorithm is used to calculate the probability distribution of each cell belonging to each cluster, generating a soft assignment matrix. Where 870 represents cells (samples) and 5 represents cluster centers (categories), this matrix is ​​separated from the computational graph and converted into a static soft-assignment tensor. Used for subsequent loss calculations, thereby avoiding gradient updates from interfering with the stability of cluster centers;

[0114] Step 43, represent the online vector from step 41. With target vector representation By inputting multiple different reprojection heads (composed of multilayer sensing mechanisms), diverse reprojection feature representations are generated. , , , Representation of original features By performing an equal-weighted average with the reprojected feature representation and fusing multi-view information, a more robust final weighted feature is generated. , As shown in the following formula:

[0115]

[0116]

[0117] Step 44, based on the weighted features generated in step 43 , and the soft allocation tensor of step 42 Calculate the cluster centers of the two groups respectively. , As the basis for prototype contrastive learning, cluster centers are obtained through a soft-assigned probability weighted average, as shown in the following formula:

[0118]

[0119]

[0120] in, Represents cells Belongs to cluster The soft assignment probability, Represents cells Feature representation;

[0121] Step 45, calculate the mean squared error loss. This loss term is used to constrain the consistency of features extracted by the online encoder and the target encoder in the representation space, by calculating weighted features under the augmented view. The mean squared error between them makes the model invariant to noise interference and improves the stability of the feature representation, as shown in the following equation:

[0122]

[0123] in, Represents a cell characteristic index. This represents the number of cell samples in the current training batch.

[0124] Step 46, Calculate the prototype contrast loss This loss function aims to bring the representations of the same cluster prototype closer together under different views and to push apart the representations of different cluster prototypes, thereby promoting the formation of cluster structures. Specifically, for each cluster center, a prototype generated by a multi-granularity online gene encoder is used. As the anchor point, its corresponding target prototype As positive samples, the remaining target prototypes As negative samples, a contrastive loss function based on cosine similarity is used, as shown in the following equation:

[0125]

[0126] in, Represents cosine similarity. For temperature hyperparameters, The total number of clusters, For clustering index;

[0127] Step 47, Calculate the overall loss of the model. The mean squared error loss is added to the prototype contrast loss to form a multi-task learning objective, enabling the model to simultaneously learn instance-level representation consistency and cluster-level discriminative structure. The calculation formula is as follows:

[0128]

[0129] in, The hyperparameters that balance the weights of the two losses can be adjusted according to the actual task requirements;

[0130] Step 48: Repeat steps 41 to 47 iteratively to train the entire multi-granularity gene coding model end-to-end. The optimization process uses stochastic gradient descent (SGD) or Adam optimizer. Set the initial learning rate, weight decay rate and other hyperparameters, and dynamically adjust the learning rate according to the training progress. The model is trained until the preset maximum number of iterations is reached or the loss function converges, and finally a multi-granularity gene coding model that can extract discriminative feature representations is obtained.

[0131] Step 5: Using the trained single-cell clustering model, perform cell clustering based on scRNA-seq data to obtain the clustering results, specifically:

[0132] The scRNA-seq data were preprocessed using the preprocessing method described in step 1. The processed time-series medical data were then input into the model, and single-cell features were output for single-cell clustering.

[0133] To verify the effectiveness of this method, it was evaluated using the ACC, NMI, and ARI metrics based on the QS_Diaph database, and compared with single-cell clustering methods KMeans, SC, scziDesk, scTPF, scTAG, scDCC, scDECL, scSimGCL, and CLEAR. ACC measures the accuracy of the clustering results, NMI assesses the consistency between the clustering results and the true labels, and ARI measures the similarity between the clustered data and the true assignments. Higher values ​​for ACC, NMI, and ARI indicate more accurate clustering results, as shown in Table 1.

[0134] Table 1 Single-cell clustering results

[0135]

[0136] As shown in Table 1, our proposed method significantly outperforms traditional algorithms such as K-Means and spectral clustering (SC), highlighting the inherent advantages of deep learning models in processing high-dimensional, sparse scRNA-seq data. It also surpasses a series of deep clustering models proposed in recent years, such as scDECL, scTPF, and scSimGCL. This performance improvement is mainly due to the core innovation of our method: a multi-granularity feature extraction mechanism. Unlike models that only perform feature modeling at a single granularity (global or local), this method captures both global associations and local synergies of the genome through "coarse-grained - fine-grained" dual attention, thereby constructing a richer and more robust cellular embedding representation.

[0137] Example 2

[0138] As shown in Figure 4, this example illustrates a single-cell clustering system based on multi-granularity Transformer, which includes a computer processor, a single-cell dataset storage unit, a data processing unit, a multi-granularity gene coding model training unit, and a clustering unit.

[0139] The single-cell dataset storage unit stores the constructed scRNA-seq dataset and loads it into computer memory;

[0140] The data processing unit retrieves the scRNA-seq dataset from computer memory and performs augmentation processing to obtain enhanced data;

[0141] The multi-granularity gene coding model training unit trains the model, determines the model training parameters, and obtains the multi-granularity gene coding model.

[0142] The clustering unit performs cell clustering to obtain the result of single-cell clustering;

[0143] The specific data processing and calculation work in all units is performed by the computer processor.

[0144] The data flow and computation tasks between the various units of the system are all uniformly scheduled and executed by the computer processor, realizing fully automated processing from data loading, preprocessing, model training to cluster analysis.

[0145] The foregoing has shown and described the main features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0146] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A single-cell clustering method based on multi-granularity Transformer, characterized in that, Includes the following steps: Step 1: Preprocess scRNA-seq data to screen high-quality samples and key gene features from the raw scRNA-seq data; Step 2: Augment the preprocessed scRNA-seq data; Step 3: Construct a multi-granularity gene coding model based on multi-granularity Transformer, including a multi-granularity online gene encoder, a multi-granularity gene target encoder, and a feature reprojection head; Step 4: Train the multi-granularity gene coding model, and design an optimizer and loss function to update the multi-granularity gene coding model; Step 5: Use the trained multi-granularity gene coding model to perform cell clustering based on scRNA-seq data to obtain clustering results.

2. The single-cell clustering method based on multi-granularity Transformer according to claim 1, characterized in that, Step 1 specifically involves: collecting and screening standardized scRNA-seq data, filtering out low-quality cells and low-expression genes, and screening for highly variable genes based on the "mean-dispersion" model to construct a high-quality input matrix. The matrix is ​​then standardized to eliminate technical bias, ultimately yielding a standardized expression matrix. 。 3. The single-cell clustering method based on multi-granularity Transformer according to claim 2, characterized in that, Step 2 specifically includes: Step 21, standardizing the expression matrix. Perform batch partitioning; Step 22, augment scRNA-seq data: for each cell in the original cell gene expression profile, randomly add Gaussian noise with probability p to generate corresponding augmented samples, and normalize the expression matrix. The augmented data was obtained by performing augmentation twice. ， 。 4. The single-cell clustering method based on multi-granularity Transformer according to claim 3, characterized in that, Step 3 specifically includes: Step 31, constructing and initializing a multi-granularity online gene encoder. Using TNT as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is designed. First, gene features are segmented into coarse-grained tokens to extract global gene associations. Then, the coarse-grained tokens are segmented into fine-grained tokens to capture local gene synergies. Step 32: Construct and initialize a multi-granularity gene target encoder. Using TNT as the backbone network, a dual attention mechanism of "coarse-grained and fine-grained" is included to capture the association features between genes at different granularities. The parameter update method is shown in Equation (1): ;in, Parameters representing the multi-granularity online gene encoder, Parameters representing multi-granularity gene target encoders, The smoothing coefficient is used to control the magnitude of parameter updates; Step 33, construct and initialize the feature reprojection head. : This is a multi-layer nonlinear perceptron network containing 4 linear layers. The first, second, and third linear layers are followed by a BatchNormal normalization layer and a ReLU nonlinear activation layer. The fourth linear layer maps the feature dimension back to the original input dimension and initializes the parameters of the feature reprojection head.

5. The single-cell clustering method based on multi-granularity Transformer according to claim 4, characterized in that, Step 4 specifically involves: Step 41, processing the scRNA-seq data from step 22... Input the multi-granularity online gene encoder constructed in step 31 In, and augment the data in step 22. Input the multi-granularity gene target encoder constructed in step 32 In this process, a dual-path encoding mechanism is used to obtain the original feature representation from the input raw data.

1. Input augmented data to obtain online vector representations With target vector representation Step 42, based on the online vector representation obtained in step 41 The soft k-means algorithm is used to calculate the probability distribution of each cell belonging to each cluster, generating a soft assignment matrix. The matrix is ​​separated from the computation graph and transformed into a static soft-assignment tensor. Used for subsequent loss calculation; Step 43, the online vector representation from Step 41 is used. With target vector representation By inputting multiple different reprojection heads, diverse reprojection feature representations are generated. 、 、 、 By representing the original features The reprojected feature representation is then subjected to an equal-weighted average, and multi-view information is fused to generate the final weighted feature. 、 As shown in the following formula: ； Step 44, based on the weighted features generated in step 43 、 and the soft allocation tensor of step 42 Calculate the cluster centers of the two groups respectively. 、 As the basis for prototype contrastive learning, cluster centers are obtained through a soft-assigned probability weighted average, as shown in the following formula: ； ;in, Represents cells Belongs to cluster The soft assignment probability, Represents cells Feature representation; Step 45, calculate mean squared error loss. : By calculating weighted features under the enhanced view The mean square error between them is shown in the following formula: ;in, Represents a cell characteristic index. The number of cell samples in the current training batch; Step 46, calculate the prototype contrast loss. For each cluster center, a prototype is generated using a multi-granularity online gene encoder. As the anchor point, its corresponding target prototype As positive samples, the remaining target prototypes As negative samples, a contrastive loss function based on cosine similarity is used, as shown in the following equation: ;in, Represents cosine similarity. For temperature hyperparameters, The total number of clusters, For clustering index; Step 47, calculate the overall model loss. The mean squared error loss is added to the prototype comparison loss, and the calculation formula is as follows: ;in, To balance the hyperparameters of the two loss weights; in step 48, steps 41 to 47 are repeated iteratively to train the entire multi-granularity gene coding model end-to-end. The optimization process uses stochastic gradient descent or Adam optimizer, sets the initial learning rate and weight decay rate, and dynamically adjusts the learning rate according to the training progress. The model is trained until the preset maximum number of iterations is reached or the loss function converges, and finally a multi-granularity gene coding model that can extract discriminative feature representations is obtained.

6. The single-cell clustering method based on multi-granularity Transformer according to claim 5, characterized in that, Step 41 specifically involves: augmenting the matrix. and It is divided into T coarse-grained tokens to form tokenized data. As shown in the following formula: Each coarse-grained token is composed of... A series of highly variable genes constitute the structure shown in formula (9), while a linear projection layer is formed. Map each token to the embedding space to obtain a coarse-grained embedding. ； ;in, Indicates the first Each coarse-grained token is initially defined; for fine-grained partitioning, each coarse-grained token is further subdivided to construct fine-grained tokenized data. As shown in the following formula: ;in, The number of genes for fine-grained tokens. The number of times each coarse-grained token is split. Indicates which fine-grained token; through the internal projection layer. Will Mapping to fine-grained embedding At a fine-grained level, a multi-head self-attention mechanism is introduced to... Processing is performed using a learnable weight matrix. Calculate query ,key ,value The vector is shown in the following equation: The attention weights and output are calculated as follows: ;in, For scaling dot product attention functions, Let T be the dimension of the key vector, and T be the number of coarse-grained tokens; let the number of attention heads be... Then the multi-head attention output fine-grained feature representation As shown in the following formula: ； ;in, Indicates the attention head index, This is a vector concatenation operation. The output projection matrix is ​​trainable; the enhanced fine-grained representation is obtained through residual connections and layer normalization operations: Remodeled into Through linear layer Projecting onto the coarse-grained dimension yields ， With coarse-grained embedding Perform residual fusion to obtain the fused representation. , fusion Further apply coarse-grained multi-head self-attention and calculate the attention output. After further processing via residual connections, layer normalization, and feedforward networks, the output is finally reshaped into a cell-level online vector representation. Multi-granularity gene target encoders process augmented data using the same structure. Obtain the target vector representation 。 7. A single-cell clustering system based on multi-granularity Transformer, characterized in that, It includes a computer processor, a single-cell dataset storage unit, a data processing unit, a multi-granularity gene coding model training unit, and a clustering unit; the single-cell dataset storage unit stores the constructed scRNA-seq dataset and loads it into computer memory; the data processing unit retrieves the scRNA-seq dataset from computer memory and performs augmentation processing to obtain augmented data; The multi-granularity gene coding model training unit trains the model, determines the model training parameters, and obtains the multi-granularity gene coding model; the clustering unit performs cell clustering to obtain the results of single-cell clustering; the specific data processing and calculation work in all units is completed by the computer processor.

Images