Self-supervised clustering method for massive single-cell data

Abstract

The present invention discloses a self-supervised clustering method for massive single-cell data, the method comprising the following steps: 1) downloading single-cell RNA sequencing data from a public gene expression database; 2) preprocessing the single-cell RNA-seq data, including filtering, standardization and feature selection; 3) using a fast Louvain algorithm to perform preliminary clustering on the preprocessed single-cell RNA sequencing data; 4) using the "cluster representativeness" indicator to screen out representative cell points from the preliminary clustering results; 5) after screening out representative cell points in each preliminary cluster, using a Transformer model to obtain the final clustering results. This method combines the fast Louvain algorithm and the Transformer neural network, significantly improving the efficiency and accuracy of clustering analysis, and providing an efficient and accurate solution for the analysis of massive single-cell data.

Description

Technical Field

[0001] The present invention belongs to the field of bioinformatics and computer science, and relates to a single-cell data clustering method, and specifically to a self-supervised clustering method for massive single-cell data combining a fast Louvain algorithm with a Transformer neural network. Background Art

[0002] Single-cell RNA sequencing (scRNA-seq) technology has developed rapidly in the past decade, greatly promoting the understanding of cellular heterogeneity and dynamic processes in biological systems. This technology enables researchers to perform high-resolution analysis of gene expression at the single-cell level, thereby promoting breakthroughs in many fields such as developmental biology, cancer research, and immunology. However, with the continuous maturity of scRNA-seq technology and the continuous expansion of experimental scale, the size of the data set has grown from the initial thousands of cells to hundreds of thousands or even millions of cells today. Faced with such a large data set, traditional clustering analysis methods began to expose obvious shortcomings.

[0003] Most current single-cell data clustering methods rely on a combination of dimensionality reduction and clustering strategies, such as principal component analysis (PCA) combined with K-means or graph-based clustering methods. These methods perform well when processing small and medium-sized data sets, but as the size of the data set increases, the contradiction between computing time and clustering accuracy gradually emerges. On the one hand, clustering algorithms need to complete the processing of large-scale data within a limited time, but complex algorithms may lead to a sharp increase in computing time; on the other hand, in order to ensure the biological significance and reliability of the clustering results, the algorithm must accurately capture the subtle differences in the data, which usually requires higher computational complexity.

[0004] Although existing analysis tools such as Seurat and Scanpy have optimized the clustering process, they still face performance bottlenecks when processing ultra-large-scale data sets. Traditional algorithms such as Louvain and Leiden perform well on medium-sized data sets, but when the data set size exceeds hundreds of thousands or millions of cells, the clustering time increases rapidly and the accuracy of the clustering results is difficult to guarantee, and problems such as too many clusters may occur.

[0005] In recent years, with the development of deep learning technology, clustering methods based on deep learning have shown great potential in processing high-dimensional complex data. These methods can extract multi-level features from massive data by constructing complex neural network structures, and to a certain extent alleviate the limitations of traditional algorithms. However, many existing deep learning methods still face problems such as high computational complexity, long training time, and insufficient biological interpretability when facing ultra-large-scale single-cell data. Therefore, how to design a clustering algorithm that is both efficient and accurate is still an important direction of current research. Summary of the invention

[0006] In order to solve the shortcomings of existing single-cell RNA sequencing data analysis methods in processing large-scale and complex biological data, the present invention provides a self-supervised clustering method for massive single-cell data. This method combines the fast Louvain algorithm and the Transformer neural network to significantly improve the efficiency and accuracy of clustering analysis, providing an efficient and accurate solution for the analysis of massive single-cell data, and providing technical support for medical analysis, biological cell research and other fields.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A self-supervised clustering method for massive single-cell data comprises the following steps:

[0009] Step 1) Obtain single-cell RNA-seq data: Download single-cell RNA sequencing data from a public gene expression database;

[0010] Step 2) Data preprocessing: Preprocess the single-cell RNA-seq data, including filtering, normalization, and feature selection. The specific steps are as follows:

[0011] Step 2.1) Calculate the expression of each gene in all cells, retain genes that are expressed in at least 1% of cells, and filter out genes whose expression is lower than this standard;

[0012] Step 2.2) Convert the gene expression matrix to a Seurat object and standardize the data using the SCTransform function;

[0013] Step 2.3) Use the FindVariableFeatures function to select the 2000 genes with the highest variability in the dataset;

[0014] Step 3) Preliminary clustering using the fast Louvain algorithm: The preprocessed single-cell RNA sequencing data are preliminarily clustered using the fast Louvain algorithm. The specific steps are as follows:

[0015] Step 3.1) Perform PCA dimensionality reduction on the gene expression matrix to project the high-dimensional data into a low-dimensional feature space;

[0016] Step 3.2) Generate representative anchor points: Generate a number p of representative anchor points to approximate the underlying geometric structure of the data, thereby constructing a sparse matrix 500≤p≤3000;

[0017] Step 3.3) Establishment of neighbor relationship and KNN screening: After generating anchor points, use KNN algorithm to determine the k nearest anchor points of each cell, and establish sparse connection 5≤k≤20 by selecting the nearest neighbor relationship;

[0018] Step 3.4) Generate an N×p sparse similarity matrix B: By using a locally scaled Gaussian kernel function, the distance between the cell and the anchor point is converted into a similarity measure. Each row of the sparse similarity matrix B corresponds to a cell, and each column corresponds to an anchor point. An element in the matrix represents a similarity measure between a pair of cells and anchor points. Each row only retains the similarity values ​​of the k anchor points closest to the current cell, and the other elements are set to 0, thereby generating an N×p sparse similarity matrix, where N is the number of cells;

[0019] Step 3.5) Construct a sparse adjacency matrix: Based on the sparse similarity matrix B, generate a (N+p)×(N+p) sparse adjacency matrix W:

[0020]

[0021] Among them: the upper left part is an N×N zero matrix, the lower right part is a p×p zero matrix, B is the sparse similarity matrix between cells and anchor points, B T is the transposed matrix of B, representing the similarity between the anchor point and the cell;

[0022] Step 3.6) Generation and standardization of sparse weight matrix: Standardize the weights in the sparse adjacency matrix W and convert the sparse adjacency matrix W into a standardized sparse weight matrix A;

[0023] Step 3.7) Apply the Louvain algorithm for preliminary clustering: Substitute the standardized sparse weight matrix A into the Louvain algorithm for community division;

[0024] Step 4) Screening representative cell spots: Use the "cluster representativeness" indicator to screen representative cell spots from the preliminary clustering results. The specific steps are as follows:

[0025] Step 4.1) Cluster representativeness calculation: Based on the preliminary clustering results and sparse weight matrix generated by the Louvain algorithm, the cluster representativeness of each cell point in its cluster is calculated. The cluster representativeness calculation formula is as follows:

[0026]

[0027] In the formula, represents the cluster representativeness of node i in the cluster to which it belongs, represents the set of nodes that node i is in the same cluster (community), is the element in the standardized sparse weight matrix A, representing the weight of the edge between node i and node j;

[0028] Step 4.2) Representative cell screening: For each preliminary cluster, select the cell point with the highest cluster representativeness as the representative cell of the cluster. The number of representative cells is 2-5% of the total number of cells in the cluster.

[0029] Step 5) Use the Transformer model to obtain the final clustering results: After selecting representative cell points in each preliminary cluster, use the Transformer model to obtain the final clustering results. The specific steps are as follows:

[0030] Step 5.1) Model training: Use the selected representative cell points as training sets to train the Transformer model;

[0031] Step 5.2) Model prediction: After training is completed, the trained Transformer model is used to predict the entire gene expression dataset to obtain the final clustering results.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] This invention significantly improves the efficiency and accuracy of cluster analysis of single-cell RNA sequencing data by combining the fast Louvain algorithm with the deep learning model, solves the bottleneck of existing methods in large-scale single-cell data analysis, provides a powerful solution for efficient analysis of large-scale biological data, and promotes the further development of life science and biomedicine. The specific effects are as follows:

[0034] 1. Improve computational efficiency: This invention effectively compresses the similarity matrix of the original data and generates a sparse adjacency matrix based on the introduction of anchor point generation strategy and sparse matrix construction method. This method greatly reduces the amount of computation of the Louvain algorithm when processing large-scale data sets, allowing preliminary clustering analysis to be completed in a shorter time. Compared with traditional methods, this invention can quickly complete data preprocessing and preliminary community division when processing large-scale data, significantly improving the efficiency of overall clustering analysis.

[0035] 2. Enhanced clustering accuracy: By combining the Louvain algorithm with the Transformer neural network, the complex patterns and subtle differences in the data can be better captured. Compared with traditional deep learning methods, the clustering results generated by this invention are more accurate when processing large-scale and highly heterogeneous biological data, and can effectively cope with the diversity in the data.

[0036] 3. Applicable to large-scale data sets: The present invention is particularly suitable for the analysis of large-scale single-cell RNA sequencing data. It can effectively overcome the performance bottleneck problem of traditional algorithms when processing data of hundreds of thousands or millions of cells, and ensure the feasibility and accuracy of clustering results in large-scale data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall flow chart of the self-supervised clustering method for massive single-cell data;

[0038] Figure 2 This is the clustering effect diagram of the dataset GSE223414 after preliminary clustering using the fast Louvain algorithm, where each data point represents a cell and each color represents a preliminary cluster;

[0039] Figure 3 This is the clustering effect diagram of the dataset GSE223414 after preliminary clustering using the fast Louvain algorithm. Figure 2 Red dots are added on the basis to represent representative cell spots selected from each preliminary clustering;

[0040] Figure 4 This is the final clustering effect diagram of the dataset GSE223414 obtained by the Transformer model, where each data point represents a cell and each color represents a cluster;

[0041] Figure 5This is a bar chart comparing the clustering results obtained by the method of the present invention (MSCLT) and the two deep learning models scMAE and CellVGAE for the dataset GSE223414. The three indicators used are ARI (Adjusted Rand Index), NMI (Normalized Mutual Information) and ACC (Clustering Accuracy).

[0042] Figure 6 This is a bar chart comparing the total clustering time consumed by the method of the present invention (MSCLT) and the two deep learning models scMAE and CellVGAE for the dataset GSE223414. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0044] The present invention provides a self-supervised clustering method for massive single-cell data. First, a fast Louvain algorithm is designed to perform rapid preliminary clustering of massive single-cell data to generate preliminary community division results. Subsequently, the indicator of "cluster representativeness" is proposed to measure the representativeness of each cell point in its preliminary cluster, thereby screening out representative cell points in each preliminary cluster. Afterwards, based on these representative cell points, a Transformer neural network is used for self-supervised learning training to automatically extract useful features from the data and further optimize the clustering effect. The Transformer neural network effectively captures complex patterns in large-scale data through its self-attention mechanism to ensure that the clustering results are more accurate. Finally, all single-cell data are input into the trained model for prediction, further improving the accuracy of the clustering effect, thereby generating the final clustering results. The present invention not only greatly improves the computational efficiency, but also improves the accuracy of the clustering results, ensures the accuracy of the clustering results, and is particularly outstanding in processing large-scale single-cell RNA sequencing data, providing a powerful tool for the analysis of large-scale single-cell data, and providing strong technical support for life science and biomedical research.

[0045] like Figure 1 As shown, the specific steps include:

[0046] Step 1) Obtain single-cell RNA-seq data: In order to perform cluster analysis of single-cell RNA sequencing data, obtain public single-cell RNA sequencing datasets from gene expression databases such as the GEO (Gene Expression Omnibus) database and the 10x Genomics platform. The downloaded dataset is usually a gene expression matrix, in which each row represents a gene and each column represents a cell. The downloaded data can include cell data from different tissues or experimental conditions to ensure data diversity and coverage. The gene expression level of each cell is standardized through preprocessing operations. These datasets cover gene expression data from different organisms and cell types, including cells in various physiological and pathological states, providing rich diversity and extensive biological information.

[0047] Step 2) Data preprocessing: After obtaining the single-cell RNA-seq data, preprocessing operations are performed on it, including filtering, standardization, and feature selection. The specific steps are as follows:

[0048] Step 2.1) Filter out highly expressed genes by calculating the expression of each gene in all cells. Keep genes that are expressed in at least 1% of cells and filter out genes whose expression is lower than this standard. This step reduces the noise in the data and retains relatively important genes.

[0049] Step 2.2) Convert the gene expression matrix to a Seurat object and use the SCTransform function to normalize the data. This step removes technical noise through normalization and correction techniques while retaining biological differences so that expression levels between different cells are comparable.

[0050] Step 2.3) Use the FindVariableFeatures function to select the 2000 genes with the highest variability in the dataset. Highly variable genes are more important in subsequent dimensionality reduction and clustering analysis because they contain the main biological differences in the dataset.

[0051] These steps together complete the filtering, standardization and feature selection of data, laying the foundation for subsequent clustering analysis. The purpose of this part of preprocessing is to reduce data noise, highlight important biological signals and ensure the quality and analyzability of the data.

[0052] Step 3) Preliminary clustering using fast Louvain algorithm: In order to solve the problem of low efficiency of the traditional Louvain algorithm in processing large-scale single-cell RNA sequencing data, the present invention innovatively designs a fast Louvain algorithm and applies the algorithm to perform preliminary clustering of single-cell RNA sequencing data after a series of preprocessing steps. The core idea of ​​the algorithm is to reduce redundant calculations through anchor point selection strategy and sparse matrix generation strategy before clustering, thereby significantly improving calculation efficiency. The specific steps are as follows:

[0053] Step 3.1) Perform PCA (Principal Component Analysis) on the gene expression matrix to reduce the dimension and project the high-dimensional data into a low-dimensional feature space. PCA, as a classic linear dimensionality reduction method, can convert high-dimensional gene expression data into a low-dimensional feature matrix by finding the direction of the most significant change in the data. This transformation of PCA not only reduces the dimensionality while retaining the main features of the data, but also enhances the effect of subsequent analysis (such as clustering or visualization).

[0054] Step 3.2) Generate representative anchor points: In order to avoid the high cost of calculating the original large-scale and dense similarity matrix, the present invention generates representative anchor points to approximate the underlying geometric structure of the data, thereby constructing a sparse matrix to reduce subsequent operations. Specifically, given a gene expression matrix, first randomly select p0 candidate cells from all N cells, and then divide the candidate cells into p clusters through k-means clustering, where p0 is 10~20% of the total number, 500≤p≤3000. The final anchor points are the centroids of these p clusters, denoted as r 1, r 2,……, r p , r p Represents the pth anchor point. It should be noted that the purpose of this step of selecting candidate cells is to reduce the time overhead on large-scale data. For datasets with a small number of cells, this step can be skipped and k-means can be directly applied to all N cells to identify anchor points.

[0055] K-means is a commonly used clustering algorithm. Its basic idea is to distribute data points to k different clusters through repeated iterations so that the distance between the data points in each cluster and their centroid is minimized. The final cluster centroid can well represent the overall characteristics of the data points in the cluster.

[0056] Step 3.3) Establishment of neighbor relationship and KNN (K-Nearest Neighbors) screening: After generating anchor points, in order to establish neighbor relationships between cells and anchor points and further compress the data scale, the present invention uses the KNN algorithm to determine each cell and its nearest k anchor points, and establishes sparse connections 5≤k≤20 by selecting the nearest neighbor relationship. KNN is a commonly used classification and regression algorithm based on distance metrics. Its basic idea is to predict or classify based on the distance relationship between a data point and its nearest k neighbors in the feature space. Specifically, KNN calculates the distance between a data point and other points (usually using Euclidean distance), selects the nearest k points, and makes inferences based on the distribution or labels of these neighbor points. The KNN algorithm is simple and intuitive and can effectively capture the local features of the data when processing small-scale data.

[0057] First, all p anchor points are divided into o clusters using the K-means algorithm and marked as ω1, ω2, ω o , thereby reducing the similarity calculation between anchor points to 10≤o≤100. Calculate each cell X i The Euclidean distance to all cluster centroids is calculated and each cell is assigned to the cluster with the closest distance to its centroid. Find cell X i The closest anchor point in the cluster is denoted by r i . The KNN algorithm is used to further obtain the k0 nearest neighbors of each anchor point and based on X i With r i The Euclidean distance between the k0 nearest anchor points determines the cell X i The k nearest anchor points k0=10k.

[0058] For two points A(x1, x2, x3, ..., x n ) and B(y1, y2, y3,..., y n ), the formula for Euclidean distance is as follows:

[0059] Formula 1

[0060] In the formula, Represents the Euclidean distance between point A and point B.

[0061] Step 3.4) Generate an N×p sparse similarity matrix B: By using a locally scaled Gaussian kernel function, the distance between the cell and the anchor point can be converted into a similarity measure. Each row of the sparse similarity matrix B corresponds to a cell, and each column corresponds to an anchor point. An element in the matrix represents a similarity measure between a pair of cells and anchor points, which is converted from the Euclidean distance between the two. Each row only retains the similarity values ​​of the k anchor points closest to the current cell, and the other elements are set to 0, thereby generating a sparse similarity matrix. By converting the relationship between cells and anchor points into a sparse similarity matrix, the amount of calculation can be greatly reduced, and time can be saved while maintaining the main relationship structure, thereby improving the efficiency of the overall algorithm.

[0062] Used to calculate the elements of the sparse similarity matrix B The formula for the locally scaled Gaussian kernel function is as follows: Formula 2

[0063] In the formula, Represents cells With anchor The similarities between Represents cells With anchor The Euclidean distance between Represents the parameter controlling the width of the Gaussian kernel.

[0064] Example of sparse similarity matrix B (each row has only k non-zero elements):

[0065]

[0066] Step 3.5) Construct a sparse adjacency matrix: Based on the sparse similarity matrix B, generate a (N+p)×(N+p) sparse adjacency matrix W, which is defined as follows:

[0067]

[0068] Among them: the upper left part is an N×N zero matrix (there is no direct connection between cells), the lower right part is a p×p zero matrix (there is no direct connection between anchor points), B is the sparse similarity matrix between cells and anchor points, B T is the transposed matrix of B, representing the similarity between the anchor point and the cell.

[0069] Example of a sparse adjacency matrix W:

[0070]

[0071] Step 3.6) Generation and standardization of sparse weight matrix: After generating the sparse adjacency matrix W, further standardization is required to generate a sparse weight matrix A that can be used in the Louvain algorithm. The non-zero elements in the sparse adjacency matrix W represent the similarity between cells and anchors or between anchors, but since these weight values ​​may have large numerical differences that affect the accuracy of the community division results, these weight values ​​need to be standardized. The standardization formula is as follows:

[0072] Formula 3

[0073] In the formula, is an element in the sparse weight matrix A, representing the cell With anchor The standardized weight value between ; is an element in the sparse adjacency matrix W, representing the cell With anchor The original weight value between ; and are the minimum and maximum values ​​of all weights in the sparse adjacency matrix W, respectively.

[0074] After this standardization process, all weight values All are compressed to [0, 1], thus ensuring that the similarity measures between all cells and anchor points can be consistently considered during community division, eliminating the impact of different value ranges on clustering results. The standardized sparse weight matrix A will be used as the input matrix of the Louvain algorithm to optimize modularity.

[0075] Step 3.7) Apply the Louvain algorithm for preliminary clustering: Substitute the standardized sparse weight matrix A into the Louvain algorithm for community division. Since the matrix has been sparsely processed, the Louvain algorithm can converge more quickly and complete clustering.

[0076] The Louvain algorithm achieves the division of community structure by maximizing the modularity Q, using the function formula as follows:

[0077] Formula 4

[0078] In the formula, is the weight value in the weight matrix A, indicating the similarity strength between cell i and cell j, and are the degrees of node i and node j respectively, is the sum of all edge weights, , is the community label of node i and node j, that is, the community number assigned to them. is the Kronecker indicator function, when The value is 1 when , otherwise it is 0.

[0079] The Louvain algorithm first considers each node as an independent community and merges adjacent communities by iteratively optimizing the modularity until the modularity Q is no longer significantly improved. After multiple rounds of iterations, the community label matrix C of all nodes is obtained, thereby achieving preliminary clustering of cell data.

[0080] Step 4) Screening representative cell points: During the clustering process of the Louvain algorithm, different cell points will be assigned to different preliminary clusters, and each cluster may contain different numbers of cells. In order to better represent the biological characteristics of each cluster, improve clustering accuracy, and reduce the computational complexity of subsequent deep learning model training, the present invention innovatively defines the indicator of cluster representativeness to further screen representative cell points in each cluster. The definition of cluster representativeness is different from the traditional modularity or cluster centrality. It aims to measure the representativeness of each cell point in its cluster to identify cell points that have important contributions to the cluster characteristics, and significantly reduce the amount of data required to participate in model training, thereby improving overall computational efficiency. The specific steps are as follows:

[0081] Step 4.1) Cluster representativeness calculation: Based on the preliminary clustering results and sparse weight matrix generated by the Louvain algorithm, calculate the cluster representativeness of each cell point in its cluster. The higher the cluster representativeness, the more representative the expression of the overall characteristics of the cell point in the cluster to which it belongs, and the better it can reflect the core characteristics of the cluster. The calculation formula of cluster representativeness is as follows:

[0082] Formula 5

[0083] In the formula, represents the cluster representativeness of node i in the cluster to which it belongs, represents the set of nodes that node i is in the same cluster (community), is the element in the standardized sparse weight matrix A, representing the weight of the edge between node i and node j, which is obtained by normalizing the sparse neighbor matrix W between the cell and the anchor point generated in the fast Louvain algorithm.

[0084] Step 4.2) Representative cell screening: For each preliminary cluster, select the cell point with the highest cluster representativeness as the representative cell of the cluster, and the number of representative cells is 2-5% of the total number of cells in the cluster. These cell points can not only better express the core characteristics of the cluster in subsequent analysis, but also effectively reduce the data scale of model training, reduce training time and computing overhead, and thus improve the clustering effect and analysis efficiency of the overall model by screening fewer and more representative cell points.

[0085] Step 5) Use the Transformer model to obtain the final clustering results: After selecting representative cell points in each preliminary cluster, use the Transformer model to obtain the final clustering results. The specific steps are as follows:

[0086] Step 5.1) Define the Transformer model: Define the Transformer model for classification. The key parameters of this model include input feature dimension, number of attention heads, multi-layer encoder, and classifier.

[0087] The Transformer model first converts the input feature dimension into the feature dimension inside the model through a linear layer. The formula is as follows:

[0088] Formula 6

[0089] In the formula, represents the gene expression data matrix, Represents the input matrix After a linear transformation, the result is It means to standardize the data so that its mean is 0 and its variance is 1. Represents the feature representation after linear transformation and Layer Normalization.

[0090] The core of each layer in the Transformer model is the multi-head attention mechanism, which can capture global information. The formula is as follows:

[0091] Formula 7

[0092] In the formula, represent the query, key, and value vectors obtained through different projection matrices, respectively. represents the scaling factor of attention, It is a normalization function, commonly used in probability distribution calculations. It converts the input into a range from 0 to 1, and the sum of all output values ​​is 1. The value of is a weight distribution representing the correlation between different parts of the input.

[0093] Step 5.2) Model training: Use the selected representative cell points as the training set to train the Transformer model. The goal of the Transformer model is to learn the feature representation of each cell and improve the classification performance of the Transformer model by optimizing the cross entropy loss function.

[0094] For classification tasks, the loss function generally uses cross entropy loss:

[0095] Formula 8

[0096] In the formula, represents the input label in the i-th category, It indicates the probability that the cell point predicted by the model belongs to the i-th category, is the number of clusters obtained by preliminary clustering.

[0097] During the training process, the Transformer model is optimized to capture more complex patterns and features by adjusting the learning rate and batch size. The learning rate determines the size of the step size each time the model parameters are updated. A reasonable learning rate can accelerate convergence. The batch size determines how many samples are used each time the model is updated. A larger batch size can update the parameters more stably but at a higher computational cost.

[0098] Step 5.3) Model prediction: After training is completed, the trained Transformer model is used to predict the entire gene expression dataset to obtain the final clustering results.

[0099] In the prediction stage, the Transformer model processes the input gene expression matrix X through a series of Transformer layers to obtain the feature representation of each cell, and outputs the probability distribution of each cell belonging to each category through the fully connected layer:

[0100] Formula 9

[0101] In the formula, represents the feature representation of each cell, represents the weight of the classifier, Used to calculate the probability of each category.

[0102] Finally, the category corresponding to the maximum probability is selected as the predicted label:

[0103] Formula 10

[0104] In the formula, Indicates the maximum value in the input data. is the label of the final clustering result, indicating which cluster cell i belongs to. Through this process, a final clustering label can be assigned to each cell, thereby completing the clustering task of the entire gene expression dataset.

[0105] Example:

[0106] 1) Dataset download: Download single-cell RNA sequencing data from a public gene expression database. This example uses the GSE223414 dataset from the GEO database, which analyzes the gene expression changes of induced pluripotent stem cells during cardiac differentiation through single-cell RNA sequencing, and provides the gene expression matrix data required for clustering, including 45,460 cells and 25,639 genes.

[0107] 2) Data preprocessing: Data needs to be preprocessed before cluster analysis to improve data quality and reduce noise. The preprocessing step is completed using R language. The specific steps are as follows:

[0108] 2.1) Gene screening: Low-expression genes are screened out by calculating the expression ratio of each gene in all cells. Specifically, genes with an expression ratio of less than 1% in all cells are eliminated, and genes expressed in at least 1% of cells are retained. This step can remove noise data and retain genes that may have biological significance, thereby ensuring the effectiveness of model clustering.

[0109] 2.2) Standardization: The gene expression matrix is ​​passed into the CreateSeuratObject() function to convert it into a Seurat object, and then the data is standardized using the SCTransform function in the R language. This method can effectively remove technical noise (such as batch effects) while retaining biological differences in the data. After standardization, the gene expression levels between different cells can be compared.

[0110] 2.3) Feature selection: In order to highlight the representative biological features in the dataset, the FindVariableFeatures function in R language was used to select the 2000 genes with the highest variability in the dataset. These genes reflect important biological differences between cells and are therefore crucial for subsequent dimensionality reduction and clustering analysis.

[0111] The preprocessed gene expression matrix contained 45,460 cells and 2,000 genes.

[0112] 3) Preliminary clustering using the fast Louvain algorithm: After data preprocessing, the single-cell RNA sequencing data is preliminarily clustered using a custom fast Louvain algorithm. The calculation process of the traditional Louvain algorithm is optimized through the anchor point selection strategy and sparse matrix generation strategy. The specific steps are as follows:

[0113] 3.1) PCA dimensionality reduction: First, use the RunPCA function in the Seurat package of the R language to perform principal component analysis (PCA) dimensionality reduction on the high-dimensional gene expression matrix. This function automatically selects the most representative gene expression features as input without manually specifying parameters. This step can extract the main variation features of the data and compress the data into a low-dimensional feature space, thereby reducing the computational complexity. In this embodiment, the gene expression matrix after PCA dimensionality reduction contains 45,460 cells and 50 genes.

[0114] All subsequent steps of this embodiment are implemented using Python language.

[0115] 3.2) Anchor point selection and adjacency matrix construction: In order to avoid directly calculating the similarity between all cells based on the reduced-dimensional data, an anchor point selection strategy is introduced to approximate the potential structure of the data and construct the adjacency matrix through a sparse similarity matrix. The specific steps are as follows:

[0116] Generate anchor points: Use the k-means algorithm to divide all cells into 2000 clusters, and the centroid of each cluster is used as the anchor point. The number of anchor points 2000 is much smaller than the total number of all cells 45460, which greatly reduces the subsequent similarity calculation burden.

[0117] Select anchor points: All 2000 anchor points are divided into 44 clusters using the K-means algorithm to reduce the amount of similarity calculation between anchor points. Calculate each cell X i Find the Euclidean distance to all cluster centroids and assign each cell to the cluster to which it is closest. i The closest anchor point in the cluster is denoted by r i The KNN algorithm is used to further obtain the 150 nearest neighbors of each anchor point and i With r i The Euclidean distances between the 150 nearest anchor points determine the 15 nearest anchor points of cell Xi.

[0118] Similarity calculation between cells and anchor points: To reduce the computational complexity, each cell is only calculated for similarity with its 15 nearest anchor points, rather than directly calculating similarity with all other cells. A locally scaled Gaussian kernel function is used to calculate the similarity between cells and anchor points. The parameter controlling the width of the Gaussian kernel is Set to the average distance between each cell and its 15 nearest anchors.

[0119] Sparse adjacency matrix construction: A 45460×2000 sparse similarity matrix B is constructed. Each row of the matrix B corresponds to a cell and each column corresponds to an anchor point. Each element in the matrix represents the similarity between a cell and the corresponding anchor point. In order to further reduce the amount of calculation, each row in the matrix only retains the similarity of the 15 anchor points closest to the cell, and the other elements are set to zero.

[0120] Expand the adjacency matrix: Expand the 45460×2000 sparse similarity matrix B to a 47460×47460 sparse adjacency matrix W.

[0121] 3.3) Generation and standardization of weight matrix: After obtaining the sparse adjacency matrix W, the weights in the matrix are further standardized to convert the sparse adjacency matrix W into a standardized sparse weight matrix A. After standardization, all edge weight values ​​are compressed to [0, 1], thus avoiding clustering bias caused by numerical differences.

[0122] 3.4) Louvain algorithm clustering: Based on the standardized sparse weight matrix A, the Louvain algorithm is applied to perform community division to obtain preliminary clustering results. This algorithm maximizes the modularity to find the optimal community division and obtain preliminary clustering results.

[0123] 4) Screening representative points: In order to optimize the training effect of the Transformer model and improve the computational efficiency, a custom clustering representative index is used to screen representative cell points from the preliminary clustering results. The specific steps for screening representative cell points are as follows:

[0124] 4.1) Cluster representativeness calculation: Based on the clustering results of the Louvain algorithm, the cluster representativeness of each cell point in the preliminary cluster to which it belongs is calculated. The cluster representativeness is calculated based on the connection strength between the cell point and other cell points in the same cluster. The higher the cluster representativeness, the better the cell point can reflect the overall characteristics of the cluster.

[0125] 4.2) Select representative cells: For each cluster, select several cell points with the highest representativeness, which is 2% of the total number of cells, as the cell points representing the cluster. These cell points will be used as training data for the Transformer model to further improve clustering accuracy and effectively reduce data redundancy, thereby improving overall computing efficiency.

[0126] 5) Use the Transformer model to obtain the final clustering results: After screening out representative cell points, use the Transformer neural network model for final clustering. The specific steps are as follows:

[0127] 5.1) Define the Transformer model for classification tasks: The key parameters of the model include the number of attention heads, the number of encoder layers, the hidden dimension, and the drop ratio. In this example, these parameters are set to 16, 8, 512, and 0.3, respectively. These parameters can capture complex patterns in the data. The model converts the input features into the internal representation of the model through a linear layer to ensure that the features of each cell can effectively participate in subsequent processing.

[0128] 5.2) Model training: The selected representative cell points are input into the Transformer model as training sets. Since these representative cell points have been screened by the Louvain algorithm for cluster representation, it is ensured that they can fully represent the characteristics of their respective clusters.

[0129] 5.3) Model prediction: After the model training is completed, the trained Transformer model is used to predict the entire gene expression dataset. In order to perform cluster prediction, the model will output the probability distribution of each cell point belonging to each cluster. The category corresponding to the maximum probability is selected as the prediction result.

[0130] Through the above process, a cluster label is assigned to each cell to complete the final clustering of the entire data set.

[0131] Figure 2 This is a clustering effect diagram of the dataset GSE223414 after preliminary clustering using the fast Louvain algorithm, where each data point represents a cell and each color corresponds to a preliminary cluster. After processing with the fast Louvain algorithm, the dataset was divided into 10 categories. The preliminary clustering results show that each cluster has a relatively clear separation in the scatter plot, indicating that the differences in gene expression of cell populations have been preliminarily identified and distinguished, which provides a solid foundation for subsequent more refined clustering analysis, helps to deeply explore the potential heterogeneity or subtypes between cells and lays an important foundation for subsequent functional analysis and biological research.

[0132] Figure 3 This is the clustering effect diagram of the dataset GSE223414 after preliminary clustering using the fast Louvain algorithm. Figure 2On the basis of the above data, red dots are added to represent the representative cell points selected from each preliminary cluster. The selection of representative cells is based on the strength of their connection with other cell points in the same cluster. The higher the cluster representativeness, the better the cell point can reflect the overall characteristics of the cluster. These selected representative cell points will be input into the Transformer model as a training set for training. By selecting the most representative cell points, it can be ensured that the training data can more accurately capture the core features of each cluster, thereby improving the model training effect and prediction performance. In addition, this screening method can not only reduce the amount of calculation but also enhance the generalization ability of the model in subsequent cell classification or other biological analysis.

[0133] Figure 4 The final clustering effect diagram of the dataset GSE223414 obtained by the Transformer model is shown, in which each data point represents a cell and each color represents a cluster. The dataset is divided into 10 categories. By comparing with the preliminary clustering effect, it can be clearly seen that the Transformer model significantly improves the clustering results. Compared with the preliminary clustering, the cells in the clusters are more closely distributed and the boundaries are clearer, indicating that the Transformer model performs well in cell feature learning and clustering accuracy. Through the training of the Transformer model, the representation of cells in the feature space is optimized, enabling the model to better capture the subtle differences between different clusters, thereby improving the quality and reliability of clustering. This result further verifies the effectiveness and advantages of the model in processing complex biological data.

[0134] Figure 5 This is a bar chart comparing the clustering results obtained by the method of the present invention (MSCLT) and the two deep learning models scMAE and CellVGAE for the dataset GSE223414. The three indicators used are ARI (Adjusted Rand Index), NMI (Normalized Mutual Information) and ACC (Clustering Accuracy). The results show that the MSCLT method is superior to the comparison method in all three indicators, and performs outstandingly in clustering precision, consistency and accuracy. This shows that the MSCLT method can more effectively capture the potential characteristics of cells, thereby significantly improving the quality of clustering results. Compared with scMAE and CellVGAE, MSCLT has obvious advantages in cell type classification and identification.

[0135] Figure 6The bar chart shows the total clustering time consumed by the method of the present invention (MSCLT) and the two deep learning models of scMAE and CellVGAE for the dataset GSE223414. It can be seen that the actual time consumed by the method of the present invention (MSCLT) when clustering the dataset is significantly less than that of the two comparison methods. This result shows that the MSCLT method can complete the clustering task more efficiently while maintaining a high clustering accuracy, saving a lot of computing time. This is especially important for processing large-scale datasets, which can greatly improve the efficiency of analysis, especially when it is necessary to quickly obtain results in practical applications. The MSCLT method has a significant time advantage. In contrast, scMAE and CellVGAE consume more time in the clustering process and may face bottlenecks in computing resources and time on large-scale datasets.

Claims

1. A self-supervised clustering method for massive single-cell data, characterized in that The method comprises the following steps: Step 1) Obtain single-cell RNA-seq data: Download single-cell RNA sequencing data from a public gene expression database; Step 2) Data preprocessing: Preprocess the single-cell RNA-seq data, including filtering, normalization, and feature selection; Step 3) Preliminary clustering using the fast Louvain algorithm: The preprocessed single-cell RNA sequencing data are preliminarily clustered using the fast Louvain algorithm. The specific steps are as follows: Step 3.1) Perform PCA dimensionality reduction on the gene expression matrix to project the high-dimensional data into a low-dimensional feature space; Step 3.2) Generate representative anchor points: Generate a number of representative anchor points p to approximate the underlying geometric structure of the data, thereby constructing a sparse matrix, 500≤p≤3000; Step 3.3) Establishment of neighbor relationship and KNN screening: After generating anchor points, use the KNN algorithm to determine the k nearest anchor points for each cell, and establish sparse connections by selecting the nearest neighbor relationship, 5≤k≤20; Step 3.4) Generate an N×p sparse similarity matrix B: By using a locally scaled Gaussian kernel function, the distance between the cell and the anchor point is converted into a similarity measure. Each row of the sparse similarity matrix B corresponds to a cell, and each column corresponds to an anchor point. An element in the matrix represents a similarity measure between a pair of cells and anchor points. Each row only retains the similarity values ​​of the k anchor points closest to the current cell, and the other elements are set to 0, thereby generating an N×p sparse similarity matrix, where N is the number of cells; Step 3.5) Construct a sparse adjacency matrix: Based on the sparse similarity matrix B, generate a (N+p)×(N+p) sparse adjacency matrix W: Among them: the upper left part is an N×N zero matrix, the lower right part is a p×p zero matrix, B is the similarity matrix between cells and anchor points, B T is the transposed matrix of B, representing the similarity between the anchor point and the cell; Step 3.6) Generation and standardization of weight matrix: Standardize the weights in the sparse adjacency matrix W and convert the sparse adjacency matrix W into a standardized sparse weight matrix A; Step 3.7) Apply the Louvain algorithm for preliminary clustering: Substitute the standardized sparse weight matrix A into the Louvain algorithm for community division; Step 4) Screening representative cell spots: Use the "cluster representativeness" indicator to screen representative cell spots from the preliminary clustering results. The specific steps are as follows: Step 4.1) Cluster representativeness calculation: Based on the preliminary clustering results and weight matrix generated by the Louvain algorithm, calculate the cluster representativeness of each cell point in its cluster; Step 4.2) Representative cell screening: For each preliminary cluster, select the cell point with the highest cluster representativeness as the representative cell of the cluster. The number of representative cells is 2-5% of the total number of cells in the cluster. Step 5) Use the Transformer model to obtain the final clustering results: After selecting representative cell points in each preliminary cluster, use the Transformer model to obtain the final clustering results.

2. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that The specific steps of step 2) are as follows: Step 2.1) Calculate the expression of each gene in all cells, retain genes that are expressed in at least 1% of cells, and filter out genes whose expression is lower than this standard; Step 2.2) Convert the gene expression matrix to a Seurat object and standardize the data using the SCTransform function; Step 2.3) Use the FindVariableFeatures function to select the 2000 genes with the highest variability in the dataset.

3. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that In step 3.4), the element B used to calculate the sparse similarity matrix B ij The formula for the locally scaled Gaussian kernel function is as follows: In the formula, Represents cells With anchor The similarities between Represents cells With anchor The Euclidean distance between Represents the parameter controlling the width of the Gaussian kernel.

4. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that In step 3.6), the standardization formula is as follows: In the formula, is an element in the sparse weight matrix A, representing the cell With anchor The standardized weight value between ; is an element in the sparse adjacency matrix W, representing the cell With anchor The original weight value between ; and are the minimum and maximum values ​​of all weights in the sparse adjacency matrix W, respectively.

5. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that In step 3.7), the Louvain algorithm realizes the division of community structure by maximizing the modularity Q, and the function formula used is as follows: In the formula, is the weight value in the sparse weight matrix A, indicating the similarity strength between cell i and cell j, and are the degrees of node i and node j respectively, is the sum of all edge weights, , is the community label of node i and node j, that is, the community number assigned to them. is the Kronecker indicator function, when = The value is 1 when , otherwise it is 0.

6. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that In step 4.1), the cluster representativeness calculation formula is as follows: In the formula, represents the cluster representativeness of node i in the cluster to which it belongs, represents the set of nodes that node i is in the same cluster, is the element in the standardized sparse weight matrix A, which represents the weight of the edge between node i and node j.

7. The self-supervised clustering method for massive single-cell data according to claim 1, characterized in that The specific steps of step 5) are as follows: Step 5.1) Model training: Use the selected representative cell points as training sets to train the Transformer model; Step 5.2) Model prediction: After the training is completed, the trained Transformer model is used to predict the entire gene expression dataset to obtain the final clustering results.

8. The self-supervised clustering method for massive single-cell data according to claim 7, characterized in that In step 5.1), the Transformer model first converts the input feature dimension into the feature dimension inside the model through a linear layer. The formula is as follows: In the formula, represents the gene expression data matrix, Represents the input matrix After a linear transformation, the result is It means to standardize the data so that its mean is 0 and its variance is 1. Represents the feature representation after linear transformation and Layer Normalization; The core of each layer in the Transformer model is the multi-head attention mechanism, the formula is as follows: In the formula, , , represent the query, key, and value vectors obtained through different projection matrices, respectively. represents the scaling factor of attention, is the normalization function, The value of is a weight distribution, representing the correlation between different parts of the input; In step 5.2), for the classification task, the loss function uses the cross entropy loss: In the formula, represents the input label in the i-th category, It indicates the probability that the cell point predicted by the model belongs to the i-th category, is the number of clusters obtained by preliminary clustering.

Images