A single-cell multi-modal data integration method based on attention mechanism and graph variational autoencoder

Abstract

The application discloses a single-cell multi-modal data integration method based on an attention mechanism and a graph variational autoencoder, and comprises the following steps: step 1, multi-modal data preprocessing and cell relationship graph construction; step 2, cross-modal adjacency matrix fusion based on multi-head attention; step 3, graph variational autoencoder training and multi-objective optimization; step 4, multi-objective loss calculation and model joint optimization; and step 5, cell low-dimensional embedding extraction and clustering analysis. The method fuses single-cell transcriptome and epigenetic group data through a multi-head attention mechanism, and learns a low-dimensional embedding representation of cells by using a graph variational autoencoder, so that efficient integration and clustering analysis of single-cell multi-modal data are realized.

Description

Technical Field

[0001] This invention belongs to the field of bioinformatics technology and relates to a method for integrating single-cell multimodal data, specifically a method for integrating single-cell multimodal data based on attention mechanism and graph variational autoencoder. Background Technology

[0002] Single-cell RNA sequencing (scRNA-seq) and single-cell chromatin accessibility sequencing (scATAC-seq) are core tools for elucidating cellular heterogeneity and gene regulatory networks. scRNA-seq captures the complete expression profile of the cellular transcriptome, while scATAC-seq precisely reveals the open state of chromatin and potential regulatory elements. These two technologies have been widely applied in cutting-edge fields such as developmental biology, the tumor microenvironment, and immune responses, greatly deepening our understanding of life processes and disease mechanisms. However, single-modality data only reflects one dimension of cellular state and cannot construct a complete link from "chromatin openness → gene regulation → transcriptional expression → cellular function," thus having significant limitations in elucidating intercellular interactions and complex epigenetic mechanisms. Therefore, achieving efficient integration of scRNA-seq and scATAC-seq data, and on this basis, accurately identifying cell subpopulations through unsupervised clustering, has become a core challenge in the field of single-cell multi-omics, and is of great significance for revealing disease mechanisms and promoting precision medicine.

[0003] Early research on single-cell clustering analysis methods mainly focused on the parsing of single-modality data, forming a technical system centered on traditional statistics and machine learning. In the field of single-cell RNA sequencing data analysis, the Seurat method proposed by Stuart T. et al. (2019) in "Comprehensive Integration of Single-Cell Data" published in *Cell* is a landmark tool. Its core lies in constructing a nearest neighbor graph based on inter-cell expression similarity and using community detection algorithms for clustering. Although it has good scalability and ease of use, it has limitations in nonlinear data modeling and parameter sensitivity. The SC3 method proposed by Vladimir Yu Kiselev et al. (2017) in "SC3: consensus clustering of single-cell RNA-seq data" published in *Nature Methods* is innovative in that it adopts an ensemble learning approach, forming a final consensus by aggregating multiple basic clustering results obtained in different dimensionality reduction spaces, which significantly improves clustering stability. However, its robustness in high-noise environments is insufficient, and its computational cost is relatively high. The SIMLR method, proposed by Bo Wang et al. (2017) in *Nature Methods*, titled "SIMLR: A Tool for Large-Scale Single-Cell RNA-seq Data Analysis by Multi-Kernel Learning," aims to improve clustering by learning a customized similarity matrix that reflects the intrinsic structure of the data. It provides a more efficient input for spectral clustering by constructing multiple kernel matrices and learning their optimal combination, but it also faces challenges of noise sensitivity and high computational complexity. Overall, while these classic single-modal methods have advanced single-cell clustering research, they still face bottlenecks in high-dimensional feature modeling, noise tolerance, and large-scale data processing.

[0004] With the rise of single-cell multi-omics technologies, clustering methods relying solely on single-modal data can no longer meet the urgent need for "cross-modal information integration." Multimodal clustering methods have achieved breakthroughs within the traditional linear model framework. These early methods, with linear modeling and matrix factorization as their core technical approaches, aimed to solve the challenges of fusing data from different modalities. The MOFA+ method, proposed by Ricard Argelaguet et al. (2020) in *Genome Biology*, titled "MOFA+: a statistical framework for comprehensive integration of multi-modal single-cell data," employs a probabilistic generative model framework. It aims to decouple shared and specific latent factors from multiple omics datasets, but its linear assumptions result in weak modeling capabilities for complex nonlinear relationships between data. The LIGER method, described by Jialin Liu et al. (2020) in *Nature Protocols* in their paper "Jointly defining cell types from multiple single-cell datasets using LIGER," utilizes integrated nonnegative matrix factorization to achieve joint alignment in low-dimensional space by identifying shared metabolisms across datasets. It has shown outstanding performance in integrating scRNA-seq and scATAC-seq data. The bindSC method, proposed by Jinzhuang Dou et al. (2022) in *Genome Biology* in their paper "Bi-order multimodal integration of single-cell data," employs a bidirectional integration strategy to achieve modality alignment by considering the correlation between both cell and feature dimensions. While it demonstrates significant effectiveness in heterogeneity correction, it suffers from drawbacks such as noise sensitivity and high computational burden. The Harmony method, proposed by Ilya Korsunsky et al. (2019) in *Nature Methods* in their paper "Fast, sensitive and accurate integration of single-cell data with Harmony," utilizes a mild iterative maximization algorithm to embed cells into a low-dimensional space shared by multiple omics studies, thereby correcting batch effects and achieving integration. However, its linear processing logic struggles to adapt to complex nonlinear data characteristics. These methods mark a strategic shift in cluster analysis from "single-modal analysis" to "multi-modal integration," but the inherent limitations of linear models still restrict further improvements in analytical accuracy.

[0005] The development of deep learning technology has provided crucial support for overcoming the bottleneck of linear models, and multimodal clustering methods based on neural networks have gradually become the mainstream research approach. These methods, with their powerful nonlinear modeling and feature learning capabilities, effectively address the shortcomings of traditional methods in capturing cross-modal relationships, noise tolerance, and complex feature extraction, significantly improving the accuracy of cell subpopulation identification. In the scenario of integrating transcriptomic and epigenetic data, various deep learning models demonstrate unique advantages. The scEMC method proposed by Hu Dong et al. (2024) in "Effective multi-modal clustering method via skip aggregation network for parallel scRNA-seq and scATAC-seq data" published in Briefings in Bioinformatics designs a multi-task learning network with skip connections, which can process bimodal data in parallel and effectively capture its shared and specific information. The scAI method proposed by Suoqin Jin et al. (2020) in "scAI: an unsupervised approach for the integrative analysis of parallel single-cell transcriptomic and epigenomic profiles" published in Genome Biology constructs a joint model based on matrix factorization, which simultaneously maps cells to a shared latent space and a modality-specific latent space, thereby achieving integration and interpretation of biomarkers at the same time.A series of methods based on variational autoencoders have further enriched the technical system: The scVAE method proposed by Romain Lopez et al. (2018) in "Deep generative modeling for single-cell transcriptomics" published in Nature Methods explored the modeling of the generation process of single-cell transcriptome counting data using variational autoencoders, laying the foundation for subsequent research; The scMVAE method proposed by Chunman Zuo et al. (2020) in "Deep-joint-learning analysis model of single cell transcriptome and openchromatin accessibility data" published in Briefings in Bioinformatics designed a multi-view variational autoencoder, which learns a unified representation of cells by jointly optimizing the reconstruction loss and maximizing the consistency between views. Although it improves the integration accuracy, the training efficiency of large-scale datasets still needs to be optimized.

[0006] Furthermore, the scMCs method proposed by Liangrui Pan et al. (2023) in "scMCs: a framework for single cell multi-omics data integration and multipleclusterings" published in Bioinformatics adopts a deep embedding clustering strategy, placing multimodal data integration and cluster label assignment in a unified end-to-end framework for learning. Yuhan Hao et al. (2021) in "Integrated analysis of multimodal single-cell data" published in Cell introduces the Seurat V4 tool, which comprehensively considers features from different modalities when calculating cell nearest neighbors by introducing the "weighted nearest neighbor" algorithm, thus achieving flexible and robust multimodal integration. For scenarios involving partially paired multimodal data, Boying Guo et al. (2021), in their paper "Cobolt: integrative analysis of multimodal single-cell sequencing data" published in *Genome Biology*, proposed the Cobolt method. This method constructs a unified probabilistic generation framework based on a hybrid model, which can naturally handle missing modalities and learn shared latent features across modalities, significantly improving clustering accuracy and biological interpretability. In summary, deep learning-driven multimodal clustering methods have become a core tool for analyzing single-cell cross-omics data; however, balancing model accuracy, computational efficiency, and generalization ability remains a key area for future research.

[0007] While the aforementioned multi-omics methods have made significant contributions to the clustering of single-cell sequencing data in various aspects, challenges remain in single-cell clustering. These methods still have room for improvement in robustness to noisy data, identification of rare cell subpopulations, and comprehensive analysis of complex biological relationships. Especially in large-scale datasets, due to high data dimensionality, significant batch effects, and strong heterogeneity, existing methods may face limitations in computational efficiency and result consistency. Furthermore, how to more efficiently integrate biological prior knowledge and experimental data, and improve the biological interpretability of clustering results, remains a crucial issue that current multi-omics clustering research on single cells needs to address. Therefore, developing more robust, flexible, and biologically driven methods remains a direction for future research. Summary of the Invention

[0008] The purpose of this invention is to provide a method for integrating single-cell multimodal data based on attention mechanisms and graph variational autoencoders. This method integrates single-cell transcriptome and epigenetic data through a multi-head attention mechanism and uses graph variational autoencoders to learn the low-dimensional embedding representation of cells, thereby achieving efficient integration and cluster analysis of single-cell multimodal data.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A method for integrating single-cell multimodal data based on attention mechanism and graph variational autoencoder includes the following steps:

[0011] Step 1: Multimodal data preprocessing and cell relationship map construction:

[0012] The raw sequencing data was transformed into a standardized feature matrix, and cell-cell relationship network diagrams were independently constructed based on each modality of data. The specific steps are as follows:

[0013] Step 1-1: scRNA-seq data preprocessing: Load the raw gene expression count matrix from single-cell RNA sequencing, normalize the library size, and perform logarithmic transformation to stabilize the data variance; subsequently, use analysis of variance to screen for highly variable genes, obtaining a standardized gene expression feature matrix. ;

[0014] Steps 1-2: scATAC-seq data preprocessing: The raw peak signals from single-cell chromatin accessibility sequencing are mapped to gene promoter regions to generate a gene activity matrix; this matrix is ​​then normalized, logarithmically transformed, and subjected to TF-IDF transformation to enhance cell type-specific patterns, resulting in a standardized gene activity feature matrix. ;

[0015] Steps 1-3: Construction of RNA modal cell relationship map: based on gene expression feature matrix The similarity between cells (such as the Pearson correlation coefficient) is calculated, and a k-nearest neighbor graph is constructed to obtain the adjacency matrix representing the cell connectivity relationships in the RNA modality. ;

[0016] Steps 1-4: Construction of ATAC modal cell relationship map: based on gene activity feature matrix The similarity between cells (such as cosine similarity) is calculated, and the same strategy is used to construct a k-nearest neighbor graph to obtain the adjacency matrix representing the cell connectivity relationships in the ATAC mode. ;

[0017] Step 2: Cross-modal adjacency matrix fusion based on multi-head attention:

[0018] The RNA modal cell relationship map and the ATAC modal cell relationship map obtained in step 1 are fused to generate a unified, noise-reduced consensus cell relationship map. The specific steps are as follows:

[0019] Step 2-1, Unify the definition of edge set: Integration and All non-zero edges in the set form a unified edge set;

[0020] Step 2-2, Edge Feature Construction and Projection: For each edge in the unified edge set, extract its features... and The connection strength values ​​in the vector form the original feature vector of the edge; then, these features are projected into a shared latent space through a modality-specific linear transformation layer.

[0021] Steps 2-3: Multi-head attention fusion calculation: The projected edge features are input into a multi-head self-attention module. This module dynamically calculates the attention weights from different modalities for each edge, thereby adaptively fusing bimodal information to generate a fused edge feature representation.

[0022] Steps 2-4: Fusion Graph Construction and Post-processing: The fused edge features are mapped to single fused connection weights using a decoder; a sparse fused adjacency matrix is ​​constructed based on the unified edge set and the calculated weights. Finally, regarding Symmetry and normalization processes were performed to obtain the final consensus cell relationship diagram;

[0023] Step 3: Training and multi-objective optimization of the graph variational autoencoder:

[0024] Using the fused adjacency matrix The topological structure information is used to guide the graph variational autoencoder model in learning a low-dimensional, smooth, and information-rich embedding representation of cells. The specific steps are as follows:

[0025] Step 3-1, Model Input Preparation: Take the model obtained in Step 1... and The features are concatenated to form a multimodal feature matrix. and the result obtained in step 2 Together they are used as model input;

[0026] Step 3-2, Graph structure feature smoothing: utilizing ,right Perform graph convolution operations to aggregate the feature information of each cell and its neighboring nodes to obtain a smoothed feature representation, making the representation of similar cells more consistent;

[0027] Step 3-3, Variational Encoding and Sampling: The encoder outputs the parameters of the latent space distribution for each cell; a reparameterization technique is used to sample from this distribution to obtain a low-dimensional random embedding vector for each cell. ;

[0028] Steps 3-4: Feature Reconstruction: Convert the sampled low-dimensional random embedding vector... The input is fed into a decoder network, which reconstructs the original multimodal feature matrix. Generate reconstructed data ;

[0029] Step 4: Multi-objective loss calculation and joint model optimization:

[0030] The loss function guides the end-to-end training of the entire model, ensuring that the learned representations simultaneously satisfy high reconstruction fidelity, spatial normality, and multimodal consistency. The specific steps are as follows:

[0031] Step 4-1, Calculation of Reconstruction Loss: Calculate and The difference between them (such as mean squared error) is used as the reconstruction loss. , aimed at forcing Preserve key information from the original data;

[0032] Step 4-2, KL divergence loss calculation: Calculate the KL divergence between the latent distribution of the encoder output and the standard normal prior distribution, as the regularization loss. The aim is to standardize the structure of the latent space, improve the generalization ability of the model, and promote the separability of the embeddings;

[0033] Step 4-3, Consistency Loss Calculation: Define a consensus edge set, which contains... and Strongly connected cell pairs exist in all of them; calculation The difference between the connection strength on this consensus edge set and the ideal value (strong connection) serves as the consistency loss. The aim is to force fusion graphs to retain reliable cross-modal biological associations;

[0034] Step 4-4, Joint Optimization: Reconstruction Loss KL divergence loss and consistency loss The total loss is obtained by weighting the sums according to predetermined weights; the total loss is minimized by using the backpropagation algorithm and gradient descent optimizer, and all parameters of the attention fusion module and graph variational autoencoder are jointly updated.

[0035] Step 5: Low-dimensional cell embedding extraction and cluster analysis:

[0036] The final representation of the cell is obtained using the trained model, and the cell subpopulation is divided. The specific steps are as follows:

[0037] Step 5-1, Latent Representation Extraction: After the model training is complete, for each cell, the low-dimensional random embedding vector is no longer used. Instead, it directly uses the mean vector output by the encoder. As a stable and deterministic low-dimensional potential representation;

[0038] Step 5-2, Cell Subpopulation Clustering: Cluster all cells As input, unsupervised clustering algorithms (such as K-means) are used to group cells, enabling accurate identification and classification of different subtypes or states in complex cell populations.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] (1) A paradigm shift from "feature-level integration" to "relationship-level integration" has been achieved: The core innovation lies in elevating the integration level from the traditional cellular feature space to the cellular relationship network space. By first constructing cellular similarity graphs for each modality and then fusing them along the "edges" of the graphs, the model can more fundamentally capture and reconcile the intercellular biological associations revealed by different omics data, thereby generating an intrinsic data structure foundation with lower noise and stronger consistency. This advantage is inherent to the model architecture and does not depend on specific downstream clustering metrics.

[0041] (2) The model is endowed with the intelligence to dynamically weigh modal credibility through the attention mechanism: the multi-head self-attention module can adaptively learn the evidence weights from RNA and ATAC modalities for each specific cell relationship. This means that the model itself has the ability to handle the quality imbalance and different noise levels between modalities without the need to preset fixed weights or perform tedious manual parameter tuning. This adaptive ability is an inherent robustness feature endowed by the model design.

[0042] (3) Co-optimizing representation learning and clustering objectives within a unified framework greatly enhances the discriminativeness of the embedding: Unlike the traditional process of performing dimensionality reduction and clustering separately, this model integrates data reconstruction, latent space regularization, and multimodal consensus alignment into an end-to-end training objective through a graph variational autoencoder framework. This makes the optimization direction of the model naturally tend to form a well-structured and easily separable latent space when learning the low-dimensional representation of cells. This is the "innate" design advantage of the model embedded to achieve high-quality clustering.

[0043] (4) The "consistency loss" provides a provable reliability constraint for the multimodal fusion process: The consensus consistency loss term introduced in the model's loss function imposes a clear mathematical constraint on the optimization objective, requiring the fusion result to faithfully retain the strong correlations supported by the original bimodalities. This mechanism ensures that the fusion process will not lose key cross-modal consensus information in principle, enhancing the interpretability and theoretical reliability of the model output. This is a guarantee advantage at the algorithm design level. Attached Figure Description

[0044] Figure 1 This is the overall framework diagram;

[0045] Figure 2 For data reading and multimodal data preprocessing;

[0046] Figure 3 For multimodal graph structure fusion;

[0047] Figure 4 Training of the graph variational autoencoder;

[0048] Figure 5 The clustering visualization comparison of the method of this invention with 12 other single / multimodal clustering methods on a human peripheral blood mononuclear cell dataset is shown, where each point represents a cell and each colored block represents a cell cluster.

[0049] Figure 6 The clustering visualization comparison of the method of this invention with 12 other single / multimodal clustering methods on a lymphoma dataset is shown, where each point represents a cell and each colored block represents a cell cluster.

[0050] Figure 7 This is a heatmap of marker genes for each cell type identified after data optimization and dimensionality reduction of a human peripheral blood mononuclear cell dataset using the method of this invention;

[0051] Figure 8 The figures show a comparison of developmental trajectory analysis using the original data and the data optimized by the method of this invention, respectively. (a) shows the original data, and (b) shows the optimized data.

[0052] Figure 9 Convergence curves for reconstruction loss, KL divergence loss, consistency loss, and total loss on the human external mononuclear cell dataset. Detailed Implementation

[0053] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0054] This invention provides a single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder, the main technical features of which include:

[0055] (1) "Graph-Graph Fusion" integration paradigm: abandoning the traditional feature splicing or linear weighting method, instead first constructing independent cell relationship graphs (k-nearest neighbor graphs) for scRNA-seq and scATAC-seq data respectively, and then performing deep integration at the level of cell relationship network.

[0056] (2) Adaptive fusion based on edge-level multi-head attention: The connection strength of corresponding edges in different modal graphs is used as a feature. The weight of each edge during fusion is dynamically calculated using the multi-head self-attention mechanism, which automatically strengthens the consistent association between the two modalities and weakens noise or inconsistent association.

[0057] (3) Representation learning guided by graph variational autoencoder: The consensus cell relationship graph generated by fusion and the multimodal feature matrix are input into the graph variational autoencoder, the neighbor information is aggregated by the graph structure, and the low-dimensional probability distribution representation of cells is learned in the latent space.

[0058] (4) Multi-objective joint optimization with the introduction of consensus loss: In model training, in addition to the classic reconstruction loss and KL divergence loss, a consensus loss is specially designed to force the fusion graph to retain the strong connection relationship supported by both original modes, thereby actively protecting reliable cross-modal evidence.

[0059] (5) Modality-specific preprocessing and measurement: Based on the different statistical characteristics of transcriptome and epigenetics data, differentiated standardization procedures (such as TF-IDF transformation for ATAC data) and similarity measurement standards (Pearson correlation for RNA and cosine similarity for ATAC) are used to construct the initial modality map.

[0060] (6) End-to-end trainable architecture: Multimodal graph construction, attention fusion, representation learning and clustering optimization are integrated into a complete, differentiable deep learning framework. All parameters are jointly trained by gradient descent to achieve end-to-end learning from raw data to clustering results.

[0061] like Figure 1 As shown, the method includes the following steps:

[0062] Step 1: Multimodal data preprocessing and cell relationship map construction:

[0063] This step aims to transform the raw sequencing data into a standardized feature matrix and independently construct intercellular relationship network diagrams based on each modality of data. For example... Figure 2 As shown, the specific steps are as follows:

[0064] Step 1-1: scRNA-seq data preprocessing: Load the original gene expression count matrix and normalize the gene expression counts for each cell to the library size (to a total count of 10). 4 Then, a log1p transformation was applied to stabilize the variance; the 2000 most variable genes were screened out through analysis of variance, resulting in a standardized gene expression matrix. ;

[0065] Steps 1-2: scATAC-seq data preprocessing: The original chromatin openness peak signal is mapped to the gene promoter region to generate a gene activity matrix; this matrix is ​​then subjected to CPM normalization, log1p transformation, and TF-IDF transformation to enhance cell type-specific chromatin openness patterns, resulting in a standardized gene activity matrix. ;

[0066] Steps 1-3: Construction of RNA modal cell relationship map: based on Calculate the Pearson correlation coefficient between cells, set all negative correlation values ​​to zero, construct a k-nearest neighbor graph, and perform symmetry and self-join removal operations to obtain the RNA adjacency matrix. ;

[0067] Steps 1-4: Construction of ATAC modal cell maps: based on The cosine similarity between cells is calculated using the same k-nearest neighbor graph construction strategy, followed by symmetry and self-connection removal operations to obtain the ATAC adjacency matrix. .

[0068] Step 2: Cross-modal adjacency matrix fusion based on multi-head attention:

[0069] This step aims to intelligently fuse the two modal cell relationship graphs obtained in Step 1 to generate a unified, denoised consensus cell relationship graph. For example... Figure 3 As shown, the specific steps are as follows:

[0070] Step 2-1, Unify the definition of edge set: Integration and Non-zero edges in the array form a unified edge set. , For cells Connection weights in RNA modalities For cells The connection weights on the ATAC mode are used as the objects for subsequent fusion operations;

[0071] Step 2-2, Edge Feature Construction and Projection: For Each edge in Construct its feature vector Each scalar feature is projected onto a d-dimensional shared latent space through a modality-specific linear layer to obtain the cell. Projection characteristics in mode m , and stacked as ;

[0072] Steps 2-3: Multi-head attention fusion calculation: Input a multi-head self-attention layer, compute the query, key and value matrix, independently compute the output of each attention head through a scaled dot product attention mechanism, and finally concatenate all heads and linearly project to obtain the fused features;

[0073] Steps 2-4: Fusion Graph Construction and Post-processing: The fused features are passed through a multilayer perceptron decoder containing the ReLU activation function, and the fused edge weights are output using the Sigmoid function. Construct a sparse fusion adjacency matrix based on the edge set ϵ. Symmetry processing Normalization of sum-degree matrix D is the degree matrix.

[0074] Step 3: Training and multi-objective optimization of the graph variational autoencoder:

[0075] This step aims to utilize the merged adjacency matrix. The topological information guides the graph variational autoencoder model constructed in this method to learn low-dimensional, smooth, and information-rich embedding representations of cells. For example... Figure 4 As shown, the specific steps are as follows:

[0076] Step 3-1, Model Input Preparation: Prepare the standardized multimodal feature matrix. With the fusion adjacency matrix As model input , , For cell number, For feature dimensions;

[0077] Step 3-2, Graph Feature Smoothing: Using the fused adjacency matrix Perform two-layer graph convolution operation on the input features , To achieve the aggregation of neighbor information, among which, , , For the hidden layer dimension, As a potential spatial dimension;

[0078] Step 3-3, Variational encoder forward propagation: Convolve the features of the graph By cutting, the mean vector μ and log-variance vector of the latent space are output. ;

[0079] Steps 3-4: Latent variable sampling: Use reparameterization techniques to sample latent variables from the learned distribution. , and For cells The mean vector and variance, For random noise, obtain a low-dimensional random embedding vector for each cell. ;

[0080] Steps 3-5: Data Reconstruction: Reconstructing low-dimensional random latent variables The original feature data is reconstructed using a decoder constructed from a multilayer perceptron and the Sigmoid activation function. .

[0081] Step 4: Multi-objective loss calculation and joint model optimization:

[0082] This step aims to guide the end-to-end training of the entire model using a comprehensive loss function, ensuring that the learned representations simultaneously satisfy high reconstruction fidelity, spatial normality, and multimodal consistency. The specific steps are as follows:

[0083] Step 4-1, Reconstruction Loss Calculation: Calculate the reconstruction data With the original input Mean square error between , It is the first The cells in the first The original input values ​​for each feature. It is the first The cells in the first Model reconstruction values ​​on each feature;

[0084] Step 4-2, KL divergence loss calculation: Calculate the KL divergence between the latent distribution and the standard normal prior. , It is the first The cells in the first The mean vector over each feature It is the first The cells in the first Variance over each feature;

[0085] Step 4-3, Consistency Loss Calculation: Define Consensus Edge Set , As the strength threshold, calculate the degree to which the fusion graph retains consensus connections:

[0086]

[0087] In the formula, For cells Merged adjacency weights;

[0088] Step 4-4, Joint Optimization: The three losses are weighted and summed, and all parameters of the attention fusion module and the graph variational autoencoder are jointly optimized through the backpropagation algorithm.

[0089] Step 5: Low-dimensional cell embedding extraction and cluster analysis:

[0090] This step aims to obtain the final representation of cells using the trained model and to classify cell subpopulations. The specific steps are as follows:

[0091] Step 5-1, Latent Representation Extraction: After the model training is completed, the mean vector μ output by the encoder is used as the final low-dimensional latent representation of the cell.

[0092] Step 5-2, Cell Subpopulation Classification: Apply the K-means clustering algorithm to the low-dimensional latent representation, setting the number of clusters to the known number of cell types to achieve accurate cell subpopulation classification. The clustering results are as follows: Figure 5 and Figure 6 As shown.

[0093] Step 5 - 3, Downstream Biological Analysis: Using the data reconstructed by the method of this invention, marker gene identification is performed in immune system cells. The identification results are as follows: Figure 7 As shown; developmental trajectory inference was performed within the breast cancer tumor environment, and the inference results are as follows. Figure 8 As shown, it reveals a more refined distribution of cell states and transition pathways. Figure 9 The convergence curves of each loss term in the method's loss function demonstrate the stability of the model's operation. These results validate the application value of this invention in complex biological environments.

Claims

1. A method for integrating single-cell multimodal data based on attention mechanism and graph variational autoencoder, characterized in that... The method includes the following steps: Step 1: Multimodal data preprocessing and cell relationship map construction: The raw sequencing data was transformed into a standardized feature matrix, and intercellular relationship network diagrams were independently constructed based on the data of each modality. Step 2: Cross-modal adjacency matrix fusion based on multi-head attention: The RNA modal cell relationship map obtained in step 1 and the ATAC modal cell relationship map are fused to generate a consensus cell relationship map; Step 3: Training and multi-objective optimization of the graph variational autoencoder: Using the fused adjacency matrix The topological information guides the graph variational autoencoder model to learn a low-dimensional, smooth, and information-rich embedding representation of cells; Step 4: Multi-objective loss calculation and joint model optimization: The entire model is trained end-to-end using a comprehensive loss function. Step 5: Low-dimensional cell embedding extraction and cluster analysis: The trained model is used to obtain the final representation of the cell and to complete the division of cell subpopulations.

2. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 1, characterized in that... The specific steps of step 1 are as follows: Step 1-1: scRNA-seq data preprocessing: Load the raw gene expression count matrix from single-cell RNA sequencing, and perform library size normalization and logarithmic transformation; subsequently, use analysis of variance to screen for highly variable genes, obtaining a standardized gene expression feature matrix. ; Steps 1-2: scATAC-seq data preprocessing: The raw peak signals from single-cell chromatin accessibility sequencing are mapped to gene promoter regions to generate a gene activity matrix; this matrix is ​​then standardized, logarithmically transformed, and subjected to TF-IDF transformation to obtain the standardized gene activity feature matrix. ; Steps 1-3: Construction of RNA modal cell relationship map: based on gene expression feature matrix The similarity between cells is calculated, and a k-nearest neighbor graph is constructed to obtain the adjacency matrix representing the cell connectivity relationships in the RNA modality. ; Steps 1-4: Construction of ATAC modal cell relationship map: based on gene activity feature matrix The similarity between cells is calculated, and the same strategy is used to construct a k-nearest neighbor graph to obtain the adjacency matrix representing the cell connectivity relationships in the ATAC mode. .

3. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 2, characterized in that... The specific steps of step 2 are as follows: Step 2-1, Unify the definition of edge set: Integration and All non-zero edges in the set form a unified edge set; Step 2-2, Edge Feature Construction and Projection: For each edge in the unified edge set, extract its features... and The connection strength values ​​in the vector form the original feature vector of the edge; then, these features are projected into a shared latent space through a modality-specific linear transformation layer. Steps 2-3: Multi-head attention fusion calculation: The projected edge features are input into a multi-head self-attention module. This module dynamically calculates the attention weights from different modalities for each edge, thereby adaptively fusing bimodal information to generate a fused edge feature representation. Steps 2-4: Fusion Graph Construction and Post-processing: The fused edge features are mapped to single fused connection weights using a decoder; a sparse fused adjacency matrix is ​​constructed based on the unified edge set and the calculated weights. Finally, regarding Symmetry and normalization were performed to obtain a consensus cell relationship diagram.

4. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 3, characterized in that... The specific steps of step 3 are as follows: Step 3-1, Model Input Preparation: Take the model obtained in Step 1... and The features are concatenated to form a multimodal feature matrix. and the result obtained in step 2 Together they are used as model input; Step 3-2, Graph structure feature smoothing: utilizing ,right Perform graph convolution operations to aggregate the feature information of each cell and its neighboring nodes to obtain a smoothed feature representation; Step 3-3, Variational Encoding and Sampling: The encoder outputs the parameters of the latent space distribution for each cell; a reparameterization technique is used to sample from this distribution to obtain a low-dimensional random embedding vector for each cell. ; Steps 3-4: Feature Reconstruction: Convert the sampled low-dimensional random embedding vector... The input is fed into a decoder network, which reconstructs the original multimodal feature matrix. Generate reconstructed data .

5. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 1, characterized in that... The specific steps of step 4 are as follows: Step 4-1, Calculation of Reconstruction Loss: Calculate and The difference between them serves as the reconstruction loss. ; Step 4-2, KL divergence loss calculation: Calculate the KL divergence between the latent distribution of the encoder output and the standard normal prior distribution, as the regularization loss. ; Step 4-3, Consistency Loss Calculation: Define a consensus edge set and calculate... The difference between the connection strength on this consensus edge set and the ideal value serves as the consistency loss. ; Step 4-4, Joint Optimization: Reconstruction Loss KL divergence loss and consistency loss The total loss is obtained by summing the values ​​according to predetermined weights. The total loss is then minimized using the backpropagation algorithm and gradient descent optimizer, and all parameters of the attention fusion module and graph variational autoencoder are jointly updated.

6. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 5, characterized in that... The reconstruction loss The calculation formula is: ； In the formula, It is the first The cells in the first The original input values ​​for each feature. It is the first The cells in the first Model reconstruction values ​​on each feature, Let be the number of cells, and D be the degree matrix.

7. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 5, characterized in that... The KL divergence loss The calculation formula is: ； In the formula, For cell number, As a potential spatial dimension, It is the first The cells in the first The mean vector over each feature It is the first The cells in the first Variance over each feature.

8. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 5, characterized in that... The consistency loss The calculation formula is: ； In the formula, For consensus edge set, For cells The fusion adjacency weight.

9. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 8, characterized in that... The , The intensity threshold, For cells Connection weights in RNA modalities For cells Connection weights in the ATAC mode.

10. The single-cell multimodal data integration method based on attention mechanism and graph variational autoencoder according to claim 1, characterized in that... The specific steps of step 5 are as follows: Step 5-1, Latent Representation Extraction: After the model training is complete, for each cell, the mean vector output by the encoder is used. As a stable and deterministic low-dimensional potential representation; Step 5-2, Cell Subpopulation Clustering: Cluster all cells As input, an unsupervised clustering algorithm is used to group cells, enabling accurate identification and classification of different subtypes or states in complex cell populations.

Images