A method and system for cross-modal deep structural clustering of scRNA-seq data with knowledge enhancement for large models.

Abstract

This invention discloses a method and system for cross-modal deep structural clustering of scRNA-seq data with large-model knowledge enhancement, belonging to the field of bioinformatics. The method includes: generating a cell-level text embedding matrix for cell groups based on a large language model; generating a cell-level structural embedding matrix for cell groups based on a structural encoder; wherein the structural encoder includes a depth map clustering encoder; performing cross-modal alignment of the cell-level text embedding matrix and the cell-level structural embedding matrix using a nonlinear projection head, and fusing the aligned cell-level text embedding matrix and cell-level structural embedding matrix; and obtaining the clustering results of the cell groups based on the fusion result. This invention can endow the clustering results with clear biological attribution capabilities, improving the discrimination accuracy and biological interpretation depth of single-cell analysis.

Description

Technical Field

[0001] This invention belongs to the field of bioinformatics technology, specifically relating to a method and system for cross-modal deep structural clustering of scRNA-seq data with large model knowledge enhancement. Background Technology

[0002] Single-cell RNA sequencing (scRNA-seq) technology, with its high-resolution cellular atlas, has become a key technique for characterizing tissue heterogeneity. Cell clustering, as a core component of the scRNA-seq analysis workflow, aims to define the functional identity of cells based on the similarity of gene expression among cells, laying an important foundation for downstream biological mechanism discovery and functional analysis.

[0003] Looking back at the evolution of single-cell clustering algorithms, its essence is a continuous pursuit of better numerical representations. From early statistical analyses based on linear dimensionality reduction (such as Principal Component Analysis (PCA) and t-distributed random neighborhood embedding (t-SNE)) to nonlinear embeddings based on deep autoencoders (such as scDeepCluster and scNAME), and further to structured learning that incorporates graph structure constraints (such as scGNN and scCDCG), existing methods have made relatively mature progress in characterizing the statistical distribution and topological structure of data. All of these methods share a fundamental limitation: the learned representations lack the support of biological semantic knowledge. In these models, gene expression is often treated as a numerical matrix in a high-dimensional feature space. The model can capture numerical changes in gene expression levels, but lacks an explicit understanding of gene function and its biological relationships. This disconnect between numerical representation and biological semantics limits the interpretability of clustering results and also affects the model's reasoning ability in complex biological scenarios to some extent.

[0004] To address the aforementioned semantic limitations, bioinformatics research has begun to introduce the Foundation Models (FMs) paradigm. Inspired by Large Language Models (LLMs), researchers have likened genes to tokens and cells to sentences, utilizing large-scale single-cell maps for pre-training to learn representation spaces with semantic expressive capabilities. Models such as scBERT and Geneformer pioneered the introduction of the BERT architecture into single-cell data modeling, learning the contextual dependencies of genome-wide expression through mask modeling. scGPT and scFoundation further expanded the model scale, attempting to construct generative representation models applicable across species. To overcome the barrier of inconsistent gene symbols between species, UCE innovatively uses protein sequence homology as alignment anchors to achieve zero-sample cross-species migration; addressing the problem of pure data-driven approaches neglecting biological structure, GeneCompass attempts to incorporate knowledge of gene regulatory networks, while GenePT explores the possibility of using text embeddings to replace numerical inputs; the recently proposed scMamba aims to solve the computational overhead problem faced by the Transformer architecture when processing ultra-long gene sequences by leveraging its linear computational complexity.

[0005] Although these emerging foundational models demonstrate strong potential for general representation, they still face numerous challenges when directly transferred to a specific downstream task like single-cell clustering. (1) Inconsistency between training objectives of generative pre-trained foundational models and discriminative clustering tasks: Existing models (such as scGPT) rely on generative pre-training paradigms, and their optimization focuses on modeling the overall expression distribution rather than discriminating cell subpopulation boundaries. In clustering tasks that require identifying fine-grained cell heterogeneity (discriminative objectives), general models often lack targeted boundary constraints, resulting in overly smooth representations that are difficult to distinguish between cell subpopulations in a developmental transition state. (2) Lack of biological semantic knowledge support in numerical representation: Although existing methods represent genes as tokens, the models mainly learn statistical co-occurrence relationships in expression data, while the utilization of gene functional attributes (such as regulatory relationships or pathway information) is relatively limited. Therefore, in high-noise data environments, the models may produce expression patterns or cell type predictions that are inconsistent with biological knowledge. (3) Insufficient cross-modal alignment mechanisms: Current methods generally lack mechanisms for systematically integrating biological textual knowledge with transcriptome numerical features. This separation between information sources means that clustering results often only represent mathematical labels, making it difficult to directly link them to specific biological processes or pathways.

[0006] In summary, developing a deep structural clustering method that can deeply integrate biological semantic knowledge with complex transcriptome topology and achieve cross-modal representation alignment has significant scientific research value and application prospects for improving the discrimination accuracy and biological interpretation depth of single-cell analysis. Summary of the Invention

[0007] This invention discloses a large-scale model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method and system (scLLM-DSC), which can endow the clustering results with clear biological attribution ability and improve the discrimination accuracy and biological interpretation depth of single-cell analysis.

[0008] To achieve the above objectives, the technical solution of the present invention includes the following:

[0009] A cross-modal deep structural clustering method for large-scale model knowledge enhancement scRNA-seq data, the method comprising: Generate cell-level text embedding matrices for cell groups based on a large language model; A cell-level structure embedding matrix of cell groups is generated based on a structure encoder; wherein, the structure encoder includes a depth map clustering encoder; A nonlinear projection head is used to perform cross-modal alignment of the cell-level text embedding matrix and the cell-level structure embedding matrix, and the aligned cell-level text embedding matrix and cell-level structure embedding matrix are then fused. Based on the fusion results, the clustering results of the cell groups are obtained.

[0010] Furthermore, a cell-level text embedding matrix for cell groups is generated based on a large language model, including: Encoding the textual semantic information contained in gene function descriptions using a large language model yields a global knowledge representation matrix for cell groups. The semantic information of the text includes official symbols, full name, main source, gene type, species origin, and functional summary. Obtain the whole genome expression matrix of the cell group And based on gene expression abundance, the whole genome expression matrix was analyzed. Screening was performed to obtain the cell expression matrix. ; Based on cell expression matrix Global knowledge representation matrix The elements in the matrix are filtered to obtain the global knowledge representation matrix. ; Cell expression matrix With the global knowledge representation matrix Linear aggregation is performed to obtain the first cell-level text embedding matrix; Cell expression matrix based on gene expression intensity Arrange in descending order to obtain the cell expression matrix. And based on the cell expression matrix The corresponding gene name generates the word sequence of the cell group; The word sequence is encoded using a large language model encoder to obtain the second-cell-level text embedding matrix; The first cell-level text embedding matrix is ​​integrated with the second cell-level text embedding matrix to obtain the cell-level text embedding matrix.

[0011] Furthermore, based on a large language model, the knowledge information of genes is encoded to obtain a global knowledge representation matrix of the cell group. ,include: For each gene contained in each cell in the cell group, the textual semantic information contained in the gene function description is converted into text input through cue word engineering; The text input is fed into a large language model, and the hidden layer representation of the large language model is extracted as the semantic embedding vector of the gene. By stacking the semantic embedding vectors into matrices, we obtain the global knowledge representation matrix of the cell group. .

[0012] Furthermore, a cell-level structure embedding matrix for cell groups is generated based on the depth map clustering encoder, including: Obtain the whole genome expression matrix of the cell group ; whole genome expression matrix The data is input into a depth map clustering encoder to obtain a cell-level structure embedding matrix.

[0013] Furthermore, the process of training the structure encoder and the nonlinear projection head includes: The structural encoder of the structural backbone network is initialized using normalized shear loss and reconstruction loss; The structure encoder and the nonlinear projection head are jointly trained end-to-end based on cross-modal alignment loss, normalized cut loss, reconstruction loss, and KL divergence loss; wherein the cross-modal alignment loss is used to optimize the parameters of the nonlinear projection head, and the normalized cut loss, reconstruction loss, and KL divergence loss are used to optimize the parameters of the structure encoder.

[0014] Furthermore, the cross-modal alignment loss is based on a bidirectional alignment loss and a variance regularization term, and the calculation process of the bidirectional alignment loss includes: The aligned cell-level text embedding matrix and cell-level structure embedding matrix are normalized to a unit hypersphere, and the cross-modal cosine similarity matrix is ​​calculated by scaling the dot product. Based on the cross-modal cosine similarity matrix, a symmetric information noise contrast estimation loss is introduced to obtain the bidirectional alignment loss.

[0015] A large-model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering system, the system comprising: The text embedding matrix generation module is used to generate cell-level text embedding matrices for cell groups based on a large language model. A structure embedding matrix generation module is used to generate a cell-level structure embedding matrix of cell groups based on a structure encoder; wherein, the structure encoder includes a depth map clustering encoder; The fusion module is used to perform cross-modal alignment of the cell-level text embedding matrix and the cell-level structure embedding matrix using a non-linear projection head, and then fuse the aligned cell-level text embedding matrix and cell-level structure embedding matrix. The clustering module is used to obtain the clustering results of cell groups based on the fusion results.

[0016] An electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method described in any of the preceding claims.

[0017] A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method described above.

[0018] A computer program product, characterized in that, when the computer program product is run on a computer device, the computer device executes the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method described in any of the preceding claims.

[0019] Compared with the prior art, the present invention has at least the following beneficial effects.

[0020] 1) This invention establishes a new clustering paradigm driven by both knowledge and data: by introducing NCBI gene function knowledge, biological semantic information is combined with the structural features of expression data, thereby expanding the traditional clustering method based on numerical representation.

[0021] 2) This invention implements a targeted optimization strategy for clustering tasks: In view of the difference between the goals of generative base models and clustering tasks, an optimization goal of semantic alignment and discriminative constraint collaboration is explicitly constructed to improve the ability to identify cell subpopulations.

[0022] 3) This invention constructs a unified and interpretable representation space: Utilizing a cross-modal alignment mechanism, a unified latent space is constructed that simultaneously possesses transcriptome topological fidelity and biological semantic connotation. This representation not only achieves high-precision clustering but also provides a more interpretable representation foundation for subsequent biological analyses.

[0023] 4) This invention has good data visualization capabilities.

[0024] 5) The characterization matrix learned by this invention can support downstream tasks in a variety of biological fields. Attached Figure Description

[0025] Figure 1 The overall flowchart of this invention.

[0026] Figure 2 A schematic diagram of the model framework of this invention.

[0027] Figure 3 Visualization results of the present invention and five baseline methods on Mauro Pancreas. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the objectives, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0029] This invention aims to address key technical bottlenecks in existing single-cell clustering technologies, such as the lack of semantic representation, the inconsistency between training objectives between generative pre-trained models and discriminative clustering tasks, and the lack of cross-modal alignment mechanisms, by constructing a collaborative architecture that integrates knowledge-driven, structure-aware, and semantically aligned approaches. By synergistically integrating LLM knowledge enhancement with deep structural clustering, this invention endows clustering results with clear biological attribution capabilities, establishing a new analytical paradigm for moving from simple data partitioning to in-depth analysis of biological mechanisms.

[0030] Specifically, the scRNA-seq data cross-modal deep structural clustering method provided by this invention, such as... Figure 1 and Figure 2 As shown, it includes the following steps.

[0031] Step S1: Generate cell-level text embedding matrices for cell groups based on the large language model.

[0032] Step S11: Encode the textual semantic information contained in the gene function description based on the large language model to obtain the global knowledge representation matrix of the cell group. .

[0033] This invention constructs a gene semantic encoding knowledge base that incorporates prior biological knowledge. This module maps discrete gene feature dimensions to a continuous semantic space with functional consistency by introducing textual information from an external knowledge base, thus providing a biological semantic reference for subsequent cross-modal alignment. As shown in Figure 1, gene embedding generation first retrieves structured metadata at the whole-genome scale from the NCBI database. Unlike some methods that only model based on gene expression ranking, this module focuses on utilizing the textual semantic information contained in gene function descriptions. For any target gene (such as AKT1), this invention extracts its official symbol, full name, main source, gene type, species origin, and core function summary, and transforms this knowledge into semantically coherent text input through prompt word engineering. This process, while maintaining the accuracy of biological information, transforms unstructured information from the database into data suitable for pre-trained language models. (e.g., GPT) processed text format. Then, the text input... Input the model and extract its hidden layer representation as a gene semantic embedding vector : (Formula 1) in This represents the semantic embedding dimension. Based on this, through genome-wide analysis... Each gene is encoded in parallel and stacked into matrices to obtain a global knowledge representation matrix. (i.e., the gene semantic coding knowledge base): (Formula 2) in, This represents the total number of genome-wide genes retrieved from the NCBI database (typically exceeding 30,000), thus ensuring... It can cover relatively complete gene function information. It should be noted that this is a gene semantic coding knowledge base. The construction of this matrix does not rely on a specific cell expression matrix, but is introduced as external semantic information to provide a stable functional reference for subsequent model learning. In the subsequent model, this matrix is ​​used to establish a link between cell expression features and gene function information, thereby forming a correlation between numerical features and biological interpretation.

[0034] Step S12: Obtain the whole genome expression matrix of the cell group And based on gene expression abundance, the whole genome expression matrix was analyzed. Screening was performed to obtain the cell expression matrix. .

[0035] To highlight the role of key genes in cell characterization and reduce the impact of sequencing noise, this invention first arranges the genes of each cell in descending order according to gene expression abundance, and then selects the top genes... One highly expressed gene (set in the experiment) ).make (Where N is the number of cell samples) represents the cell expression matrix after screening and sorting in descending order based on gene expression abundance, where each row contains only the most representative genes from that cell. The expression values ​​of each gene; Step S13: Based on the cell expression matrix Global knowledge representation matrix The elements in the matrix are filtered to obtain the global knowledge representation matrix. .

[0036] In one embodiment, to reflect the importance of gene function information in cell characterization, let This represents a cell expression matrix containing the top 2,048 highly expressed genes. Accordingly, the following is defined: To obtain from the global gene knowledge matrix The subset extracted based on the above filtering index.

[0037] Step S14: Analyze the cell expression matrix With the global knowledge representation matrix Linear aggregation is performed to obtain the first cell-level text embedding matrix.

[0038] By performing expression-weighted linear aggregation, this invention constructs a first-path cell-level text embedding representation. The calculation formula is as follows: in This represents a weighted summary of gene functional semantics at the cellular level. Its physical significance lies in giving dominant weight to highly expressed marker genes in the cellular semantic definition, thereby capturing the overall biological functional profile of the cell.

[0039] Step S15: Analyze the cell expression matrix according to gene expression intensity. Arrange in descending order to obtain the cell expression matrix. And based on the cell expression matrix The corresponding gene name generates the word sequence of the cell group.

[0040] To capture non-linear dependencies and co-occurrence patterns among genes, this invention adopts the sequencer strategy of the Cell2Sentence method, converting highly expressed genes into ordered gene name sequences according to their expression intensity, thereby constructing cell description statements. Specifically, it utilizes the set of highly expressed genes identified in the previous section. The 2,048 genes in each cell were sorted in descending order of gene expression intensity to obtain... The gene name was extracted as ordered morphemes to construct a text description that depicts the unique transcriptional characteristics of the cell.

[0041] Step S16: Encode the word sequence using a large language model encoder to obtain the second cell-level text embedding matrix.

[0042] This embodiment utilizes a large language model encoder. The sequence is encoded to obtain the second semantic embedding matrix. The calculation formula is as follows: in, This represents the sequence semantic features extracted by the language model. This is a matrix composed of the top 2,048 highly expressed genes arranged in descending order of expression level. This sequenced representation paradigm not only reflects the co-expression relationships between genes and their contextual information, but also effectively alleviates the shortcomings of traditional statistical methods in perceiving the dynamic regulatory relationships of cell development.

[0043] Step S17: Integrate the first cell-level text embedding matrix with the second cell-level text embedding matrix to obtain the cell-level text embedding matrix.

[0044] To obtain a cell representation that simultaneously contains gene function information and expression sequence information, this invention designs a semantic fusion operator to integrate the embeddings obtained based on gene knowledge. Embeddings obtained from sequence modeling By integrating the data, a unified cell-level text embedding matrix is ​​obtained. : in, This is the balance coefficient. This fusion characterization It can serve as a semantic anchor point for subsequent structure-aware alignment. In the experiment, it will be used as a semantic anchor point. Set to 0.5. In this fusion characterization, It provides a semantic context based on genome-wide functional description, while The regulation logic based on representation ranking was captured. The fused representation. It will serve as a semantic reference in the subsequent structure learning module, and will be used to jointly model the structure information of the representation data.

[0045] In summary, this invention utilizes a large language model to jointly encode text-based gene knowledge embeddings and sequenced cell expression information to construct a cell semantic representation with a biological interpretable basis. This process uses semantic embedding vectors as a foundation and establishes a link between sequencing expression values ​​and gene functional information through expression level weighting and sequence modeling.

[0046] Step S2: Generate a cell-level structure embedding matrix of cell groups based on the structure encoder; wherein the structure encoder includes a depth map clustering encoder.

[0047] While constructing a semantic representation space, this invention characterizes the statistical distribution features and potential manifold structures among cells. It employs a deep graph clustering encoder (scCDCG) as the backbone network for structural feature extraction. For the input whole-gene expression value matrix... (The original expression matrix of the whole genome), this backbone network acts as a nonlinear mapping encoder. The structural features of the cells are extracted and embedded, and the calculation formula is as follows: The encoding process is constrained by three complementary unsupervised optimization objectives: (1) Topology preservation ( ): It adopts a differentiable normalized loss and extracts high-order topological information by capturing the long-range dependencies of the data. While effectively preserving the global manifold structure, it avoids the oversmoothing defect of traditional graph convolutional networks. (2) Authenticity of expression Based on the reconstruction loss of the autoencoder, it ensures that the obtained potential embeddings can retain the core information of gene expression; (3) Clustering optimization By using KL divergence loss, the soft clustering assignment results are kept consistent with the target distribution obtained based on optimal transmission, thereby providing global distribution constraints during training and reducing the clustering degradation problem that may occur in deep clustering.

[0048] The final generated structural feature embedding It deeply integrates the nonlinear characteristics of gene expression with the topological structure of cell populations, and can serve as the basis for structural representation in subsequent cross-modal alignment modules.

[0049] Step S3: Use a nonlinear projection head to perform cross-modal alignment of the cell-level text embedding matrix and the cell-level structure embedding matrix, and then fuse the aligned cell-level text embedding matrix and cell-level structure embedding matrix.

[0050] After passing through the dual-path cell semantic encoding and structure-aware cell feature encoding modules, cell semantic embeddings carrying biological semantic information were obtained. and cell structure embedding containing transcriptome topological features The two types of embeddings exhibit intermodal heterogeneity due to differences in representational dimensions and information types. To break down these modal barriers and achieve the fusion of the two types of embeddings, this module uses contrastive learning as its core technical approach to integrate semantic embeddings. With structural embedding Alignment is performed to a unified latent space, while regularization mechanisms are used to avoid the dimensionality collapse problem in the feature mapping process, ensuring the effectiveness of the unified latent space. The specific implementation process includes five key steps: projection and similarity calculation, bidirectional alignment loss construction, variance regularization constraints, overall loss definition, and joint optimization.

[0051] 3.1 Projection and Similarity Calculation.

[0052] To eliminate the dimensional differences between the text semantic space and the transcription feature space, and to achieve co-mapping between the two, this invention first introduces nonlinear projection heads. Specifically, for semantic embedding... With structural embedding Two-layer multilayer perceptron (MLP) transformations based on the ReLU activation function are applied respectively, denoted as... (⋅) and (⋅), thereby obtaining a shared potential space (dimension is) Alignment representation in ) Subsequently, to eliminate the influence of the vector magnitude, the projected embedding is normalized to the unit hypersphere (i.e., ), and calculate the cross-modal cosine similarity matrix by scaling the dot product. . in The temperature parameter was set to 0.1 in the experiment.

[0053] 3.2 Bidirectional alignment loss function.

[0054] To establish a correspondence between expressed features and semantic information while preserving cell population structure information, this invention introduces a symmetric information noise contrastive estimation loss (InfoNCE contrastive loss). For cells containing... This invention utilizes small batches of data from individual cells to characterize the same cell under different modalities. Each cell is considered a positive sample pair, while the representation of each cell is considered a negative sample pair along with the representations of all other cells. The loss function is defined as follows: The first option aims to align the text view to the structure view, while the second option pulls the structure view towards the text view. Indicates the similarity between positive sample pairs. This represents the negative sample similarity from text to structure. This represents the negative sample similarity from structure to text. This indicates the batch size. This two-way constraint ensures that the embedded vectors generated by the model can not only distinguish cell heterogeneity, but also interpret the clustering results through semantic association.

[0055] 3.3 Variance Regularization.

[0056] To avoid the projection head mapping significantly different cells to the same feature point, i.e., the dimensionality collapse problem, this invention introduces a variance regularization term based on Hinge Loss. By constraining the standard deviation of each dimension's features to remain above a threshold, we ensure that the projection space has sufficient discriminative power and avoids information redundancy. in, Indicates the first in this batch The column vectors corresponding to each feature dimension It is a constant set to ensure numerical stability.

[0057] 3.4 Overall loss function of this module.

[0058] Combining contrastive loss and regularization term, the overall optimization objective of the cross-modal alignment module is defined as: in This is a hyperparameter used to balance the weights of contrast alignment loss and variance regularization loss in the overall optimization.

[0059] 3.5 Joint optimization.

[0060] This invention, scLLM-DSC, employs an end-to-end joint training model, achieving deep synergy between data-driven features and knowledge-driven semantics by constructing a composite loss function. The global optimization objective is defined as follows: α, β, γ, and δ are responsible for adjusting the contribution weights of cross-modal semantic alignment, global topology preservation, data reconstruction fidelity, and cluster refinement tasks, respectively.

[0061] In terms of semantic space construction, this invention employs OpenAI's text-embedding-3-small model to generate vector representations of gene functions. Cell-level semantic representations are obtained by aggregating the top k = 2048 hypervariable genes by expression level, and weighted summation is performed using the corresponding gene expression values ​​as weights to ensure that the semantic representation reflects the transcriptional expression characteristics of the cell. Model training employs a two-stage process: first, pre-training the structural encoder, followed by joint training of contrastive learning and clustering. In the first stage, only Normalized Cut Loss (NCut) and Reconstruction Loss (MSE) are used to initialize the structural backbone network to learn the basic manifold structure of the data; in the second stage, a cross-modal contrastive learning module is introduced to jointly fine-tune the overall model under the guidance of Equation 12 until convergence. The training epochs for both stages are set to 200 epochs, and the Adam optimizer is used for parameter updates. The final experimental results are reported as the average and variance of 5 independent runs to reduce the impact of random initialization.

[0062] Step S4: Based on the fusion results, obtain the clustering results of the cell groups.

[0063] After the model converges, this invention obtains the final unified embedding representation by fusing the aligned semantic view and structural view. The formula is as follows: Ultimately, based on this unified embedding Discrete cell clustering assignment results C are generated directly using K-Means or Leiden algorithms. Through this strategy, scLLM-DSC successfully anchors static external functional knowledge onto dynamic native expression patterns. This construction method not only generates cell representations with both robust structure and clear semantics, but also provides a reliable computational foundation for downstream accurate clustering and functional attribution analysis.

[0064] To evaluate the clustering performance of this invention and to clarify the biological rationale of the clustering results, two aspects of experiments are described below.

[0065] (a) Clustering performance evaluation.

[0066] This experiment uses three recognized clustering evaluation metrics to quantify algorithm performance: Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI). ACC and NMI are strictly defined within the range of [0, 1], while ARI has a theoretical range of [-1, 1] and may exhibit negative values ​​when the clustering performance is inferior to random assignment. For all the above metrics, higher values ​​indicate higher biological fidelity and partitioning accuracy.

[0067] For the data, researchers selected six highly representative scRNA-seq datasets provided by the scCluBench dataset platform. These datasets not only cover both human and mouse species and span various tissues including the pancreas, liver, lung, testis, and brain, but also originate from mainstream sequencing platforms such as CEL-seq2, 10X Genomics, and Smart-seq2. All data retain complete genomic meta-information, laying a solid foundation for constructing semantic representations using external knowledge bases. Detailed statistical profiles of each dataset (including cell size, gene dimension, and sparsity) are shown in Table 1.

[0068] Table 1. Overview of the single-cell RNA sequencing datasets used to evaluate scLLM-DSC performance In the experiments, the scLLM-DSC framework was implemented using the Python 3.12 programming language and the PyTorch ≥ 2.9.0 deep learning framework, and relied on CUDA 13.0 for parallel computing support. To ensure sufficient GPU memory and computational efficiency during large-scale single-cell data processing, all experiments were conducted on high-performance computing nodes equipped with NVIDIA A800-80GB GPUs.

[0069] The researchers also selected 11 representative existing algorithms as benchmark models. These methods range from classic deep nonlinear embedding models to state-of-the-art general single-cell basic models and can be divided into three main categories: (1) Deep nonlinear embedding models: scDeepCluster, scMAE, scNAME, scziDesk; (2) Deep structural clustering models: scGNN, scDSC, scSiameseClu, scCDCG; (3) Single-cell general basic models: scGPT, GeneFormer, GeneCompass.

[0070] To ensure the fairness and reproducibility of the comparative experiments, all benchmark models were deployed using their official open-source Python code libraries. The experiments strictly followed the parameter configurations recommended in the original paper, and hyperparameters were fine-tuned based on specific datasets to record the optimal performance metrics of each model under the current experimental conditions.

[0071] The experimental results are shown in Tables 2 and 3. In the results, the best result is indicated in bold, and the second-best result is indicated by an underline. OOM indicates failure due to insufficient video memory; Rank is the average ranking of the three metrics: ACC, NMI, and ARI (scDeepCluster and scziDesk are tied for 6th). Table 2 Clustering Performance Comparison: scLLM-DSC vs. Deep Learning Benchmarks Table 3. Clustering Performance Comparison: scLLM-DSC vs. Basic Biological Model Benchmark Analysis of experimental results.

[0072] (1) Comparative Analysis with Deep Clustering Benchmark Models: Table 2 details the performance comparison of scLLM-DSC with existing deep nonlinear embedding models and deep structural clustering models on six benchmark datasets. Overall, scLLM-DSC ranks first in the comprehensive ranking, with ACC, NMI, and ARI reaching 88.80%, 85.35%, and 83.04% respectively, achieving the current best performance. This performance improvement is mainly due to the model's successful deep collaboration between semantic anchors derived from large language models and global structural manifolds. Compared with deep nonlinear embedding models, the advantages of scLLM-DSC clearly demonstrate that manifold learning based solely on numerical features has limitations, highlighting the necessity of cross-modal fusion of text semantics and topological structure. It is worth noting that scLLM-DSC and scCDCG are not simply performance iterations, but rather complementary relationships based on differences in data attributes. scCDCG addresses purely data-driven scenarios lacking biological information or annotations. Through a refined normalized tiling strategy, it accurately captures the manifold structure of cell populations at the topological level, even with only gene expression matrices available. This is why scLLM-DSC chose it as its structured backbone, demonstrating that robust topological modeling can guarantee the lower bound of clustering performance in the absence of external knowledge. In contrast, scLLM-DSC is geared towards datasets with rich biological annotations or meta-information. In such scenarios, purely numerical features often lead to blurred boundaries of complex subpopulations due to a lack of semantic information. The core advantage of scLLM-DSC lies in its ability to activate accompanying biological information, transforming it into high-dimensional semantic anchors to assist clustering. For example, the Muris Brain dataset (13,417 cells, cluster number k = 2) exhibits a large sample size but a small number of clusters. Several benchmark models (such as scziDesk) show limited performance in this scenario: some suffer from memory overflow (OOM) due to their inability to handle large-scale graph structures; others, while achieving high ACC, exhibit near-zero NMI and ARI. This discrepancy in metrics indicates that the model falls into "trivial solution collapse," where most cells are classified into the majority class without recognizing the true biological boundaries. The root cause lies in the fact that in high-throughput and class-imbalanced scenarios, the original numerical features lack sufficient spacing to define clear boundaries, leading to over-smoothing in structured models. scLLM-DSC addresses this by injecting LLM-based semantic priors as anchors, providing necessary contrastive signals to maintain inter-cluster separation, aligning the global manifold with true biological logic, and effectively avoiding cluster collapse.

[0073] (2) Comparative analysis with general biological models: Table 3 shows the performance comparison between scLLM-DSC and current mainstream single-cell general models. Experimental results show that scLLM-DSC provides more robust and consistent clustering results on most datasets, reflecting the core concept proposed in this invention, namely, correcting the inconsistency of training objectives of general models in specific analysis scenarios. General models (such as GeneFormer and scGPT) are usually based on generative pre-training tasks such as mask reconstruction. Their main ability is to learn continuous and generalized transcriptome representations, rather than directly processing discrete category divisions. In contrast, scLLM-DSC designs a dedicated structured optimization objective for clustering tasks, more accurately capturing clear cell type boundaries that are difficult for general models to resolve. It should be noted that although GeneFormer and GeneCompass achieve near-perfect scores on the Sonya Liver dataset, this does not mean that their generalization ability is superior. Analysis shows that this local performance peak may be due to the overlap of pre-training data, that is, the samples of this dataset may be included in its large pre-training corpus, causing the model performance to be biased towards memory rather than reasoning. After removing potential data leakage factors, scLLM-DSC exhibits more consistent and robust performance on unseen datasets. This result demonstrates that in the analysis of complex cellular heterogeneity, relying solely on general pre-trained embeddings is insufficient; it is necessary to introduce specialized structured objective functions designed for clustering tasks to achieve an effective transition from microscopic analysis to macroscopic representation.

[0074] (ii) Visualization of clustering results.

[0075] To elucidate the biological rationale behind the clustering results, this invention utilizes the t-SNE algorithm to correlate the low-dimensional embedding representations obtained through concatenation learning. Nonlinear dimensionality reduction is performed to construct a distribution map of cells in a two-dimensional manifold space. In the resulting visualized manifold map, each data point corresponds to a single-cell sample, and the spatial distribution of the points reflects the transcriptomic similarity among cells; different color codes are used to distinguish the cell subpopulation categories predicted by the model, thereby achieving high-confidence verification of clustering performance and cell heterogeneity.

[0076] Specifically, to obtain an intuitive biological interpretation of the clustering results, this invention uses t-SNE to extract and visualize the clustering embeddings learned in two-dimensional space. The distribution of cells. Each point on the visualization represents a cell, and each color represents a predicted cell type. Figure 3The projection results of the present invention (scLLM-DSC) and five representative benchmark models in a two-dimensional latent space are shown. Overall, scLLM-DSC shows significant advantages in two key dimensions: inter-cluster boundary clarity and intra-cluster compactness. (1) Inter-cluster boundary clarity: Observing the projection results of the benchmark models, it can be found that they faithfully preserve the "transition bridging" structure naturally formed between cell subpopulations due to transcriptome similarity. This shows that the pure data-driven structured model can accurately capture the continuity and gradual features of the original data manifold. On this basis, scLLM-DSC introduces a "semantic anchoring" mechanism for scenarios that require clear biological boundaries. By injecting external priors into the semantic space of the large language model, the model provides additional discriminative dimensions for transition regions with highly similar numerical values. This mechanism effectively enhances the separability of different functional subpopulations, and while maintaining the authenticity of the underlying topology, it achieves semantic decoupling of the manifold, thereby establishing sharper and more biologically meaningful inter-class boundaries. (2) Intra-cluster compactness: From the perspective of intra-cluster distribution, scLLM-DSC exhibits higher aggregation characteristics in the manifold space. Compared with the relatively loose clusters of the baseline model, the cluster structure generated by this method is more compact, forming a robust topological morphology. This dual improvement in "boundary separation" and "intra-cluster compactness" strongly verifies the effectiveness of the cross-modal alignment mechanism: introducing high-dimensional semantic constraints into the latent space enables the model to filter non-specific background noise, thereby accurately capturing fine-grained cellular heterogeneity.

[0077] Although the present invention has been described through the above preferred embodiments, those skilled in the art will understand that various modifications in form and detail can be made therein without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A cross-modal deep structural clustering method for scRNA-seq data with knowledge enhancement for large models, characterized in that, The method includes: Generate cell-level text embedding matrices for cell groups based on a large language model; A cell-level structure embedding matrix of cell groups is generated based on a structure encoder; wherein, the structure encoder includes a depth map clustering encoder; A nonlinear projection head is used to perform cross-modal alignment of the cell-level text embedding matrix and the cell-level structure embedding matrix, and the aligned cell-level text embedding matrix and cell-level structure embedding matrix are then fused. Based on the fusion results, the clustering results of the cell groups are obtained.

2. The method according to claim 1, characterized in that, Cell-level text embedding matrices for cell groups are generated based on a large language model, including: Encoding the textual semantic information contained in gene function descriptions using a large language model yields a global knowledge representation matrix for cell groups. The semantic information of the text includes official symbols, full name, main source, gene type, species origin, and functional summary. Obtain the whole genome expression matrix of the cell group And based on gene expression abundance, the whole genome expression matrix was analyzed. Screening was performed to obtain the cell expression matrix. ; Based on cell expression matrix Global knowledge representation matrix The elements in the matrix are filtered to obtain the global knowledge representation matrix. ; Cell expression matrix With the global knowledge representation matrix Linear aggregation is performed to obtain the first cell-level text embedding matrix; Cell expression matrix based on gene expression intensity Arrange in descending order to obtain the cell expression matrix. And based on the cell expression matrix The corresponding gene name generates the word sequence of the cell group; The word sequence is encoded using a large language model encoder to obtain the second-cell-level text embedding matrix; The first cell-level text embedding matrix is ​​integrated with the second cell-level text embedding matrix to obtain the cell-level text embedding matrix.

3. The method according to claim 2, characterized in that, Encoding gene knowledge information using a large language model yields a global knowledge representation matrix for cell groups. ,include: For each gene contained in each cell in the cell group, the textual semantic information contained in the gene function description is converted into text input through cue word engineering; The text input is fed into a large language model, and the hidden layer representation of the large language model is extracted as the semantic embedding vector of the gene. By stacking the semantic embedding vectors into matrices, we obtain the global knowledge representation matrix of the cell group. .

4. The method according to claim 1, characterized in that generating a cell-level structure embedding matrix of cell groups based on a depth map clustering encoder includes: Obtain the whole genome expression matrix of the cell group ; whole genome expression matrix The data is input into a depth map clustering encoder to obtain a cell-level structure embedding matrix.

5. The method according to claim 1, characterized in that, The process of training the structure encoder and the nonlinear projection head includes: The structural encoder of the structural backbone network is initialized using normalized shear loss and reconstruction loss; The structure encoder and the nonlinear projection head are jointly trained end-to-end based on cross-modal alignment loss, normalized cut loss, reconstruction loss, and KL divergence loss; wherein the cross-modal alignment loss is used to optimize the parameters of the nonlinear projection head, and the normalized cut loss, reconstruction loss, and KL divergence loss are used to optimize the parameters of the structure encoder.

6. The method according to claim 5, characterized in that, The cross-modal alignment loss is based on a bidirectional alignment loss and a variance regularization term. The calculation process of the bidirectional alignment loss includes: The aligned cell-level text embedding matrix and cell-level structure embedding matrix are normalized to a unit hypersphere, and the cross-modal cosine similarity matrix is ​​calculated by scaling the dot product. Based on the cross-modal cosine similarity matrix, a symmetric information noise contrast estimation loss is introduced to obtain the bidirectional alignment loss.

7. A cross-modal deep structural clustering system for large-scale model knowledge enhancement scRNA-seq data, characterized in that, The system includes: The text embedding matrix generation module is used to generate cell-level text embedding matrices for cell groups based on a large language model. A structure embedding matrix generation module is used to generate a cell-level structure embedding matrix of cell groups based on a structure encoder; wherein, the structure encoder includes a depth map clustering encoder; The fusion module is used to perform cross-modal alignment of the cell-level text embedding matrix and the cell-level structure embedding matrix using a non-linear projection head, and then fuse the aligned cell-level text embedding matrix and cell-level structure embedding matrix. The clustering module is used to obtain the clustering results of cell groups based on the fusion results.

8. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device performs the large model knowledge-enhanced scRNA-seq data cross-modal deep structural clustering method as described in any one of claims 1-6.

Images