A unified representation learning method and system for spatial multi-omics and a storage medium

Abstract

This invention belongs to the interdisciplinary field of bioinformatics and artificial intelligence, specifically involving a unified representation learning method for spatial multi-omics. This method first constructs a spatial affinity graph, extracts latent embeddings of each modality through a modality-specific encoder, and applies orthogonal constraints; it introduces a graph Laplacian regularization term to maintain spatial topological smoothness; it employs cross-modal contrastive learning to construct a unified semantic space and fuses high-order interaction features through a tensor outer product mechanism; finally, it utilizes an adaptive embedding clustering module to perform end-to-end fine-tuning based on KL divergence, outputting the spatial domain partitioning results. This invention effectively solves the problems of representation collapse, shallow alignment, and structural fragmentation in traditional methods, significantly improving the noise resistance and anatomical fidelity of multi-omics integration, and providing high-quality unified representations for downstream space biology analysis.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of bioinformatics and artificial intelligence, specifically relating to a unified representation learning method for spatial multi-omics. Background Technology

[0002] The rapid development of spatially resolved transcriptomics has provided an unprecedented perspective for analyzing cellular heterogeneity and spatial topology within the tissue microenvironment. Whether it's sequencing-based spatial transcriptomics or high-resolution imaging-based spatial proteomics or epigenomics, their core value lies in anchoring molecular expression profiles to real physical spatial coordinates. However, the functional operation of organisms depends on the high degree of synergy among multiple layers of molecular signals, including the transcriptome, proteome, epigenome, and cell morphology. Single-modality data often suffers from information blind spots and inherent sparsity, making it difficult to comprehensively characterize the biological state of complex tissues. Therefore, how to deeply integrate multi-source, heterogeneous spatial multi-omics data with high-resolution histological images to construct a unified and coherent tissue molecular atlas has become a key requirement in the field of biomedical computing.

[0003] For the integrated analysis of spatial multi-omics data, existing computational methods mainly follow two technical routes, but both have significant limitations. The first route, represented by early spatial analysis frameworks, focuses primarily on extracting local spatial features from single or limited modalities (such as spatial transcriptomics alone), lacking the ability to fuse heterogeneous signals across multiple omics layers and failing to establish a global view of molecular interactions. The second route relies on non-spatial multi-omics integration algorithms. These methods excel in aligning the statistical distributions of different molecular modalities, but their mathematical models do not explicitly incorporate spatial coordinate constraints or prior knowledge of tissue structure. When directly applied to spatial domain recognition tasks, the neglect of physical neighborhood relationships often leads to a loss of spatial continuity in the integrated feature representation, making it difficult to reconstruct the true anatomical boundaries of tissues.

[0004] In recent years, some studies have attempted to combine spatial constraints with multi-omics integration, achieving some progress. However, in-depth analysis reveals that existing spatial multi-omics integration frameworks still face two major technical bottlenecks in practical applications. First, in cross-modal alignment strategies, existing methods mostly rely on pre-computed modality-specific view splicing or post-mapping mechanisms. This approach can only capture superficial linear correspondences between modalities and cannot deeply model the intrinsic biological connections between gene expression and protein products, chromatin accessibility, etc., making it difficult for models to construct a truly shared semantic space, and severely weakening the synergistic effect of cross-modal information. Second, in terms of spatial structure preservation, existing frameworks generally lack explicit point-level constraint mechanisms to enforce local topological consistency. Due to the limitations of sequencing depth and the influence of imaging artifacts, spatial multi-omics data itself has a high level of technical noise, and embedding representations lacking spatial smoothness constraints are prone to fragmentation. This fragmentation not only amplifies noise interference but also blurs the feature boundaries between different tissue regions, resulting in severe structural breaks or overlaps in downstream spatial domain identification and cell type annotation results. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a unified representation learning method, system, and storage medium for spatial multi-omics, which solves the technical problems of shallow cross-modal alignment, lack of shared semantic space, fragmented embedded representations due to lack of spatial constraints, and poor noise resistance in existing spatial multi-omics integration methods.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A unified representation learning method for spatial multi-omics includes the following steps:

[0008] S1. Acquire spatial multi-omics data, wherein the data includes at least spatial molecular data and histological image data, and construct a spatial affinity map based on the physical spatial coordinates of each data point;

[0009] S2. Feature extraction is performed on each modality data using a modality-specific encoder to generate a latent embedding representation for each modality, and orthogonal parameterization constraints are applied to the latent embedding representation.

[0010] S3. Calculate the graph Laplacian regularization term based on the spatial affinity graph to constrain the topological smoothness of spatial neighbor nodes in the potential embedding representation;

[0011] S4. Employ a cross-modal contrastive learning strategy to semantically align the latent embedding representations and construct a unified cross-modal shared semantic space;

[0012] S5. Calculate the high-order interaction features of the latent embedding representations of each modality after alignment through the tensor outer product mechanism, and then fuse them with the original embeddings after dimensionality reduction and compression to generate a composite representation vector.

[0013] S6. Input the composite representation vector into the adaptive embedding clustering module, perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution, and output the biological spatial domain partitioning result with clear spatial boundaries.

[0014] Further design optimization, step S1 specifically includes:

[0015] High-variability gene screening, logarithmic transformation, and Z-score normalization were performed on the spatial molecular data.

[0016] Histological image data is segmented into blocks, and hierarchical morphological features are extracted using a hierarchical visual Transformer and aggregated into point-level image embeddings.

[0017] Based on the two-dimensional spatial coordinates of each data point, a fully connected spatial affinity graph is constructed by calculating the affinity between points using a Gaussian kernel function, or a sparse spatial affinity graph is constructed by retaining only the k nearest neighbors with the closest physical distance based on a preset k value.

[0018] Further design optimization, step S2 specifically includes:

[0019] The high-dimensional feature matrices of each modality are projected onto a preset low-dimensional subspace using principal component analysis for initial denoising.

[0020] The dimensionality-reduced features are input into a multilayer perceptron to learn modality-specific deep nonlinear representations.

[0021] An orthogonal parameterization constraint is imposed on the output matrix of the multilayer perceptron, which forces the product of the output matrix and its transpose to be equal to the identity matrix, in order to preserve the geometric volume of the feature space and prevent the collapse of the multimodal representation.

[0022] Further design optimization, step S3 specifically includes:

[0023] Construct the graph Laplacian matrix and calculate the squared Euclidean distance between neighboring nodes in the latent embedding representation in the spatial graph;

[0024] The squared Euclidean distance is added as a penalty term to the objective function, forcing the model to learn a continuous and smooth latent manifold representation in physical space, thereby filtering out local technical noise and salt-and-pepper noise.

[0025] Further design optimization, step S4 specifically includes:

[0026] Based on the symmetric InfoNCE loss function, the mutual information between paired modal signals is maximized, thus narrowing the feature distance between corresponding points in the unified semantic space.

[0027] A distance regularization term based on margin is introduced to explicitly constrain the Euclidean distance between non-corresponding points to be greater than the preset safety margin, so as to overcome the limitation of cosine similarity that only considers angular correlation.

[0028] Further design optimization, step S5 specifically includes:

[0029] Perform tensor outer product operation on the two modal latent embeddings of each data point to generate a high-dimensional interaction tensor that captures all pairwise correlations;

[0030] The high-dimensional interaction tensor is compressed using principal component analysis, and the compressed interaction features are concatenated with the original embedding representation to form a composite representation vector that includes modality-specific details and combined synergistic effects.

[0031] Further optimizing the design, step S6 specifically includes implementing a two-stage training strategy:

[0032] The first stage is the multi-objective pre-training stage, which jointly optimizes the modality reconstruction loss, graph Laplacian structure preservation loss and cross-modal contrastive loss to complete representation learning and semantic alignment.

[0033] The second stage is the joint fine-tuning stage, which initializes the cluster centroids, activates the adaptive embedding clustering module, uses the model's own high confidence prediction to construct the target distribution, and performs end-to-end joint optimization through KL divergence loss to enhance intra-class compactness and inter-class separation.

[0034] Further design optimization also includes step S7: based on the latent embeddings obtained from the unified representation learning, perform downstream spatial analysis tasks, including UMAP high-dimensional visualization, cell type annotation, spatially variable gene detection, or cross-modal spatial correlation map construction.

[0035] On the other hand, the present invention also provides a unified representation learning system for spatial multi-omics, comprising:

[0036] The data preprocessing and graph construction module is used to acquire and preprocess spatial multi-omics data and construct spatial affinity graphs.

[0037] The modality coding module is used to extract the latent embedding representation of each modality data through a modality-specific encoder, and to apply orthogonal constraints to the latent embedding representation;

[0038] A structure preservation module is used to construct a graph Laplacian regularization term based on the spatial affinity graph to constrain the topological consistency of spatially neighboring points in the potential embedding representation;

[0039] The semantic alignment module is used to align the latent embedding representations using a cross-modal contrastive learning strategy to construct a unified semantic space.

[0040] The interactive fusion module is used to explicitly compute cross-modal high-order interactive features through tensor outer product mechanism and compress and fuse them to obtain a composite representation vector; The clustering refinement module is used to input the composite representation vector into the adaptive embedding clustering module, and perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution to output the spatial domain partitioning result.

[0041] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described above.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. This invention applies orthogonal parameterization constraints to the output of a modality-specific encoder, forcing the product of the output matrix and its transpose to equal the identity matrix. This feature mathematically maintains the orthogonality and independence of the geometric volume and latent dimensions of the feature space, avoiding information redundancy and feature confusion. It ensures that the latent embeddings extracted from each modality have high fidelity and strong discriminative power, laying a reliable data foundation for subsequent cross-modal deep integration.

[0044] 2. This invention constructs a graph Laplacian regularization term based on spatial affinity graphs, incorporating the squared Euclidean distance between physically neighboring points in the latent embedding representation as a penalty term into the objective function. This design forces the model to learn a physically continuous and smooth latent manifold, effectively filtering out local technical artifacts and salt-and-pepper noise, ensuring a high degree of consistency between the distribution of molecular features and the physical continuity of tissue anatomy, and significantly improving the model's structural fidelity in low-quality or high-noise data scenarios.

[0045] 3. This invention employs a cross-modal contrastive learning strategy based on symmetric InfoNCE loss and margin constraints. This strategy brings corresponding spatial points closer together and pushes away non-corresponding points in a unified semantic space, overcoming the limitation of cosine similarity, which only focuses on angular correlation. Simultaneously, it explicitly computes high-order interaction features of potential inter-modal embeddings through a tensor outer product mechanism, supplemented by dimensionality reduction compression and splicing fusion. This combined feature breaks through the bottleneck of traditional linear fusion, realistically modeling the nonlinear synergistic relationship between gene expression, protein products, and morphological background, achieving a leap from physical splicing to semantic unification of heterogeneous data.

[0046] 4. This invention introduces an Adaptive Embedded Clustering (AEC) module, employing a two-stage training strategy: after semantic alignment is completed during pre-training, a self-supervised target distribution update mechanism based on KL divergence is activated, utilizing the model's own high-confidence predictions as internal supervision signals for end-to-end joint fine-tuning. This feature effectively suppresses the interference of low-confidence noise, enhances intra-class compactness and inter-class separation, forces adjacent spatial points to maintain logical consistency in the semantic space, and ultimately outputs spatial domain partitioning results with clear anatomical boundaries and explicit biological significance.

[0047] 5. The refined latent embeddings generated by this invention deeply integrate multi-omics molecular signals and morphological context, while faithfully preserving spatial topology and functional structure. This high-quality unified characterization possesses high generalization and interpretability, and can be directly used as the input basis for UMAP high-dimensional visualization, fine cell type annotation, spatially variable gene (SVG) detection, and cross-modal spatial correlation map construction. It significantly improves the accuracy of downstream analysis and the efficiency of biological discovery, providing stable and efficient technical support for analyzing complex tissue microenvironments. Attached Figure Description

[0048] Figure 1 The SpaAlign framework diagram provides a unified representation learning method for spatial multi-omics in accordance with embodiments of the present invention;

[0049] Figure 2 This is a schematic diagram illustrating the extraction of pathological image features according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of four downstream tasks in an embodiment of the present invention;

[0051] Figure 4 is a schematic diagram of the real spatial domain annotation of the simulated dataset used for robustness verification in an embodiment of the present invention;

[0052] Figure 5 is a comparison chart of spatial domain clustering results of various comparative methods in the embodiments of the present invention on a simulated dataset;

[0053] Figure 6 is a schematic diagram of the actual annotation of another set of noise test simulation dataset with a layered ring structure according to an embodiment of the present invention;

[0054] Figure 7 is a line graph comparing the quantitative indices of ARI and NMI for each method under different noise intensities in the embodiments of the present invention;

[0055] Figure 8 shows the spatial clustering visualization results of all comparison methods when the intrinsic noise level of the embodiment of the present invention is 0.2;

[0056] Figure 9 is a visualization of the spatial clustering results of all comparative methods under a high-noise scenario with an intrinsic noise level of 1.0 according to the embodiments of the present invention;

[0057] Figure 10 is a schematic diagram of the artificial gold standard annotation of the tissue anatomy structure of human lymph node spatial multi-omics dataset according to an embodiment of the present invention;

[0058] Figure 11 shows the spatial domain clustering output results of all baseline comparison methods in this embodiment of the invention on the lymph node dataset;

[0059] Figure 12 is a quantitative bar chart comparing the six clustering evaluation indicators of each method when the number of clusters is fixed at the gold standard of 10 in the embodiment of the present invention.

[0060] Figure 13 is a box plot comparing the stability of six evaluation indicators of each method within the clustering resolution range of 8–12 in the embodiments of the present invention;

[0061] Figure 14 is a visualization of the distribution of UMAP dimensionality reduction of latent features learned by all comparison methods in the embodiments of the present invention;

[0062] Figure 15 is a schematic diagram of the gold standard annotation of the anatomical structure of the Allen brain atlas of mouse coronal sections in an embodiment of the present invention;

[0063] Figure 16 is a visualization of spatial anatomical region segmentation on a mouse brain epigenesis-transcriptome dataset using the comparative methods of the embodiments of the present invention.

[0064] Figure 17 is a quantitative bar chart comparing the six clustering indicators of each method when the number of clusters is fixed at the gold standard of 14 in the embodiment of the present invention.

[0065] Figure 18 is a box plot comparing the stability of six evaluation indicators of each method within the clustering resolution range of 12–16 in the embodiments of the present invention;

[0066] Figure 19 is a visualization of UMAP dimensionality reduction clustering of latent features on a mouse brain dataset using all baseline methods of the present invention.

[0067] Figure 20 is a schematic diagram of manually annotated 10 classes of spatial micro-regions in the H&E slice of the human tonsil trimodal dataset according to an embodiment of the present invention;

[0068] Figure 21 is a bar chart showing the quantitative comparison of F1 scores for the germinal center identification task in various methods of the embodiments of the present invention;

[0069] Figure 22 is a comparison of the spatial domain clustering and segmentation results of all baseline methods in the embodiments of the present invention and SpaAlign on the tonsil dataset;

[0070] Figure 23 shows the SpaAlign clustering results and the expression distribution of germinal center characteristic marker genes RGS13, MME, and SERPIA9 in the embodiments of the present invention.

[0071] Figure 24 is a comparison of the heatmap of similarity between single-modal features (genes / proteins / pathological images) in the embodiments of the present invention and the heatmap of similarity between SpaAlign multi-omics fusion characterization.

[0072] Figure 25 is a schematic diagram comparing the gold standard annotation of the simulated dataset in the embodiment of the present invention and the clustering results of three methods: GLUE, GraphST, and SpaAlign.

[0073] Figure 26 is a quantitative bar chart comparing the seven clustering evaluation indicators of SpaAlign and all baseline methods on the simulated dataset of the embodiment of the present invention;

[0074] Figure 27 shows the visualization of GLUE, GraphST, and SpaAlign clustering at an intrinsic noise level of 0.2 in this embodiment of the invention, compared with the gold standard.

[0075] Figure 28 is a visualization of GLUE, GraphST, and SpaAlign clustering in a high-noise scenario with an intrinsic noise level of 1.0 according to an embodiment of the present invention, compared with the gold standard.

[0076] Figure 29 is a visualization of the distribution of UMAP dimensionality reduction of all comparison methods under the low-noise (0.2) simulated dataset of the embodiment of the present invention;

[0077] Figure 30 is a visualization of the distribution of UMAP dimensionality reduction of all comparison methods under the high-noise (1.0) simulation dataset of the embodiment of the present invention;

[0078] Figure 31 is a comprehensive chart showing the clustering visualization and quantitative comparison of six indicators of the human lymph node dataset GLUE, GraphST, and SpaAlign in an embodiment of the present invention.

[0079] Figure 32 is a quantitative bar chart comparing the clustering indices of the SpaAlign model module ablation experiment of the human lymph node dataset in an embodiment of the present invention.

[0080] Figure 33 is a line graph showing the sensitivity analysis of hyperparameters α, γ, σ, and tol in the human lymph node dataset of an embodiment of the present invention.

[0081] Figure 34 is a comprehensive comparison of the clustering visualization and six quantitative indicators of the mouse brain dataset GLUE, GraphST, and SpaAlign in the embodiment of the present invention.

[0082] Figure 35 is a schematic diagram comparing the gold standard annotation of the human tonsil dataset and the clustering and segmentation results of GLUE, GraphST, and SpaAlign in an embodiment of the present invention;

[0083] Figure 36 is a visualization of the distribution of all baseline methods and SpaAlign multi-omics fusion latent features UMAP in the human amygdala trimodal dataset according to an embodiment of the present invention.

[0084] Figure 37 is a bar chart comparing the encoder accuracy, recall, and F1 score of ablation quantification for four types of pathological images under the human tonsil germinal center identification task in an embodiment of the present invention.

[0085] Figure 38 is a comparison chart of ablation analysis indicators of different modal combination fusion strategies for human tonsil datasets in an embodiment of the present invention;

[0086] Figure 39 is a visualization of the results of clustering and segmentation of all baseline methods and SpaAlign spatial domain in the human breast cancer dataset according to an embodiment of the present invention.

[0087] Figure 40 is a quantitative bar chart comparing the six clustering evaluation indicators of all methods in the human breast cancer dataset of the present invention. Detailed Implementation

[0088] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0089] This application provides a unified representation learning method for spatial multi-omics, including: S1. Acquire spatial multi-omics data, wherein the data includes at least spatial molecular data and histological image data, and construct a spatial affinity map based on the physical spatial coordinates of each data point; S2. Feature extraction is performed on each modality data using a modality-specific encoder to generate a latent embedding representation for each modality, and orthogonal parameterization constraints are applied to the latent embedding representation. S3. Calculate the graph Laplacian regularization term based on the spatial affinity graph to constrain the topological smoothness of spatial neighbor nodes in the potential embedding representation; S4. Employ a cross-modal contrastive learning strategy to semantically align the latent embedding representations and construct a unified cross-modal shared semantic space; S5. Calculate the high-order interaction features of the latent embedding representations of each modality after alignment through the tensor outer product mechanism, and then fuse them with the original embeddings after dimensionality reduction and compression to generate a composite representation vector. S6. Input the composite representation vector into the adaptive embedding clustering module, perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution, and output the biological spatial domain partitioning result with clear spatial boundaries; S7. Based on the latent embeddings obtained from the unified representation learning, perform downstream spatial analysis tasks, including UMAP high-dimensional visualization, cell type annotation, spatially variable gene detection, or cross-modal spatial correlation map construction.

[0090] Specifically, the process includes the following steps:

[0091] 1. Data preprocessing and graph construction

[0092] To bridge the gap between high-throughput molecular precision and multi-scale morphological context, SpaAlign requires a rigorous preprocessing pipeline tailored to the unique statistical characteristics of each data modality. Molecular modality: For raw counting matrices, such as those from spatial transcriptomics or proteomics, we employ Scanpy to mitigate technical noise and missing events. The pipeline includes log transformation and selection of the top 3,000 highly variable genes (HVGs) to focus the model’s attention on biological information signals. Z-score normalization is then applied to ensure cross-sample comparability and maintain gradient stability during deep learning optimization. Histological images: To extract high-level morphological features from H&E stained images, we utilize the Hierarchical Image Pyramid Transformer (HIPT)

[28] , which uses a hierarchical self-attention mechanism to capture local cellular details and global tissue structure. Local 256×256 pixel blocks are encoded as 576-dimensional labels and then aggregated into point-level embeddings via global context-aware ViT and average pooling.

[0093] Graph Construction: To capture the spatial relationships between capture points, we construct a mode-specific affinity graph. Given the two-dimensional spatial coordinates pi of point i, the affinity between points i and j is calculated using a Gaussian kernel:

[0094]

[0095] Where σ is the kernel bandwidth that controls the spatial decay rate. The full-point-pair formula is used by default because distant points receive negligible weights, naturally achieving local smoothing. When memory or computational constraints arise with a large number of points, we optionally sparsify the graph by retaining only the k-nearest neighbors in terms of physical distance:

[0096]

[0097] 2. Modality-Specific Feature Encoding

[0098] The primary goal of the encoding phase is to transform heterogeneous modality-specific information into a machine-interpretable low-dimensional format. SpaAlign employs independent encoders for each modality to accommodate their distinct feature spaces and dynamic ranges. First, the high-dimensional feature matrix X(m) is projected to a 128-dimensional subspace using principal component analysis (PCA). This linear dimensionality reduction serves as an initial denoising step, extracting the main biological variance while discarding technical artifacts. This is followed by a two-layer multilayer perceptron (MLP) that learns a deep nonlinear representation capable of capturing complex gene-gene or gene-image correlations.

[0099]

[0100]

[0101] in As the final potential embedding. To address the general problem of representation collapse—that is, multi-omics models learning degeneracy to map different biological states to the same output vector—we... Introduce orthogonal parameterization. This is achieved through mandatory constraints. We ensure that the encoding process preserves the geometric volume of the feature space and maintains the independence of latent dimensions, preventing the loss of crucial modality-specific information during alignment.

[0102] 3. Intramodal structure retention

[0103] A key technological innovation of SpaAlign is the requirement that latent embeddings must maintain a connection to their physical organizational context. Since biological tissues exhibit strong spatial autocorrelation—neighbors are more likely to share functional states—we introduce a graph Laplacian regularizer to stabilize the latent manifold. This module penalizes significant latent differences between spatial neighbors, encouraging the model to learn smooth representations on the spatial graph.

[0104]

[0105] By minimizing SpaAlign filters out localized technical artifacts and salt-and-pepper noise, which often plague purely feature-based clustering methods. This step ensures that the learned embeddings are not only molecularly rich but also topologically consistent with the underlying anatomy.

[0106] 4. Cross-modal contrast alignment

[0107] The core of the integration lies in aligning latent spaces from different modalities into a shared semantic framework. SpaAlign bridges the modal gap through a dual-objective contrastive framework that maximizes the mutual information between paired molecules and morphological signals. We optimize the joint alignment loss. Balancing global semantic consistency with local geometric accuracy. Global alignment is achieved through symmetric InfoNCE loss. Enforcement:

[0108]

[0109] The temperature τ is set to 1.0. To further elaborate, we propose distance regularization based on margins. To address the limitations of cosine similarity, which only considers angular correlation, we explicitly constrain the Euclidean distance:

[0110]

[0111] With the margin δ fixed at 1.0, we bring corresponding points closer together while ensuring that non-corresponding points are separated by the minimum safe margin. This dual-constraint strategy provides a well-structured and highly discriminative foundation for subsequent joint clustering.

[0112] 5. Cross-modal interaction fusion

[0113] Simple fusion methods, such as splicing, often fail to capture the synergistic relationships and complex crosstalk between different molecular layers or morphological cues. To explicitly model these high-order interactions, we utilize a tensor outer product mechanism. For each point i, we compute an interaction tensor.

[0114]

[0115] This operation captures all possible pairwise correlations between the 32-dimensional latent features of the two modalities, producing a 1024-dimensional interaction graph. To maintain computational efficiency and prevent the introduction of redundant noise, the interaction graph is compressed using PCA and integrated with the original embeddings to form a comprehensive feature vector: This holistic representation ensures that downstream analysis benefits from modality-specific details and their combined synergistic effects.

[0116] 6. Clustering Refinement Based on AEC

[0117] The ultimate goal of SpaAlign is to transform integrated features into a biologically discrete spatial domain. We employ Adaptive Embedding Clustering (AEC), which adapts to the feature manifold:

[0118]

[0119] To achieve clear, anatomically consistent boundaries, we minimize High-confidence target distribution KL divergence between:

[0120]

[0121] in It emphasizes high-confidence assignments while suppressing low-confidence noise. This self-supervised refinement iteratively improves the latent space, forcing the model to converge to strongly biological assignments that exhibit high internal homogeneity and clear spatial boundaries.

[0122] 7. Optimization and Training Strategies

[0123] SpaAlign's stability and performance are guaranteed through a two-stage training strategy. In Stage 1 (multi-objective pre-training), the model learns modality-specific representations and cross-modal alignment by minimizing the following objectives:

[0124]

[0125] in The model was trained using the Adam optimizer with a learning rate of 10⁻³. In Phase 2 (joint fine-tuning), the cluster centroids were initialized using KMeans, followed by activation of the AEC module. The network was then optimized end-to-end using the following formula:

[0126]

[0127] Where γ = 0.5. Fine-tuning is performed using the Adam optimizer with a learning rate of... Early cessation guidelines .

[0128] Specifically, SpaAlign aims to identify biologically meaningful spatial domains by generating deep integrated representations from spatial multi-omics and histological data. Figure 1 At its core, SpaAlign employs a two-stage analysis framework. For histological input, the Vision Transformer (ViT) extracts hierarchical features from 256×256 pixel blocks to capture key morphological context. Figure 2 In the first stage, cross-modal representations are aligned through multi-objective pre-training. In the second stage, these embeddings are jointly optimized and refined through deep clustering.

[0129] In the first phase, SpaAlign constructs a separate autoencoder for each modality. These autoencoders are trained using reconstruction terms to preserve modality-specific information; structure-preserving terms to maintain internal local topology; and CLIP-inspired cross-modal contrastive loss to achieve precise alignment. To capture the nonlinear cooperative relationships across these modalities, SpaAlign explicitly computes the outer product of pre-trained embeddings to form a comprehensive composite representation.

[0130] In the second stage, a deep clustering module fine-tunes the entire model end-to-end. By integrating a clustering loss based on Kullback-Leibler (KL) divergence, this joint optimization utilizes the model's own high-confidence predictions as a supervision signal. This enforces spatial consistency and guides the embeddings toward a clearer clustering structure.

[0131] The final output is a refined latent embedding that integrates all modalities while faithfully preserving spatial and functional structure. These robust embeddings can serve as the basis for a variety of downstream analyses, including UMAP visualization, cell type identification, domain-enriched gene analysis, and spatial correlation construction. Figure 3 ).

[0132] 1. Evaluate the integration performance and robustness of SpaAlign on simulated data.

[0133] We first benchmark SpaAlign against competing spatial multi-omics integration methods using simulated data with known real labels. The simulated dataset used in this benchmark comes from SpatialGlue (Long et al., 2024)

[18] and contains two modalities simulating the transcriptome and proteome, respectively. Modal 1 follows a zero-inflated negative binomial (ZINB) distribution, encoding spatial factors 1, 3, and 4, while modal 2 follows a negative binomial (NB) distribution, uniquely providing information about factor 2 ( Figure 4 ), ), simulating scenarios where complementary but non-redundant biological signals are captured at different omics layers. Successful recovery in this scenario therefore requires effective cross-modal integration. Performance was evaluated using a set of supervised clustering metrics, including mutual information (MI)

[21] , normalized mutual information (NMI)

[22] , adjusted mutual information (AMI)

[23] , Fowlkes-Mallows index (FMI)

[24] , adjusted Rand index (ARI)

[25] , V-Measure

[26] , and integrity

[26] . Baseline methods included MISO

[10] , MultiVI

[16] , MUSE

[13] , PAST

[14] , PRAGA

[19] , SpatialGlue

[18] , STAGATE

[11] , GraphST

[12] , GLUE

[17] , and totalVI

[15] , each using its standard configuration.

[0134] Qualitatively, SpaAlign recovers all four spatial factors that are closely aligned with the true values ​​at their boundaries. Figure 5 ; Figure 25 In contrast, competing methods exhibit varying degrees of fragmentation or blurred boundaries. This qualitative advantage is supported by quantitative analysis, which identifies SpaAlign as the best-performing method. Figure 26 SpaAlign uniquely achieved perfect scores (1.0) on NMI, AMI, and ARI, accurately restoring the true labels.

[0135] To further assess robustness, we generated another simulated dataset using the scMultiSim R package

[27] , which simulates a multi-omics single-cell dataset containing RNA and ATAC modalities based on gene regulatory networks (GRNs) and differentiation trees. The dataset contains six cell clusters with different transcriptomic and epigenomic profiles ( Figure 6 To assess robustness under different noise conditions, we varied the intrinsic noise parameter from 0.2 to 1.0, which controls the level of intrinsic transcriptomic randomness during count generation. Higher intrinsic noise increases random variations in gene expression and cross-modal alignment, making domain identification more challenging.

[0136] SpaAlign maintained top robustness across all noise levels, with its ARI only slightly decreasing from 0.814 to 0.767. In contrast, GraphST showed a significant drop in ARI, from an initial 0.724 to 23.2% with increasing noise levels. GLUE consistently performed poorly across all settings. Notably, STAGATE collapsed under high noise conditions, with its ARI dropping to 0.006 ( Figure 7 Qualitatively, SpaAlign maintains high structural consistency across all noise extremes. Figure 8 9; Figure 27 ,28), avoiding the severe fragmentation seen in the baseline output. UMAP visualization further confirms that SpaAlign uniquely maintains well-separated embeddings under all conditions ( Figure 29 , 30 ).

[0137] 2. SpaAlign depicts complex multilayered tissue structures on human lymph node data.

[0138] We benchmarked SpaAlign on 10x Visium human lymph node data containing spatially matched RNA and protein measurements. This dataset includes manually annotated key anatomical structures based on H&E—such as the cortex, medulla, capsule, and follicles—providing robust ground truth for spatial domain reconstruction. Figure 10 ).

[0139] SpaAlign exhibits the highest anatomical fidelity, uniquely reproducing the basic capsule-cortex-medulla structure and fine-scale domains such as follicles and pericapsular adipose tissue. Figure 11 ; Figure 31 In contrast, the baseline exhibits significant limitations: MultiVI and totalVI lack consistent spatial organization, MUSE and STAGATE produce fragmented or merged segments, and PAST and PRAGA show blurred boundaries and omissions of internal details.

[0140] Quantitatively, at the reference resolution (n = 10), SpaAlign consistently outperforms all baselines across all six metrics. Figure 12 ; Figure 31 In particular, SpaAlign achieved the highest ARI (0.312), outperforming MISO, GraphST, and GLUE by 4.7%, 8.7%, and 15.6%, respectively. SpaAlign also demonstrated superior stability across varying clustering resolutions (n ​​= 8–12). Figure 13 It exhibits low variance. Furthermore, UMAP visualization ( Figure 14 This confirms that SpaAlign produces compact, well-separated clusters, while the baseline produces diffuse, elongated (STAGATE, totalVI) or overlapping (MISO, SpatialGlue) embeddings.

[0141] An ablation study ( Figure 32 This confirms the indispensable role of SpaAlign's core components. Removing the CLIP-based contrast alignment module resulted in a 4.8% decrease in ARI. Jointly removing the CLIP and AEC deep clustering modules resulted in a 7.8% decrease in ARI, with all other metrics showing significant declines. These findings validate the complementary nature of their functions in achieving superior embedding quality.

[0142] In addition, hyperparameter sensitivity analysis of human lymph node datasets ( Figure 33 The results show that SpaAlign's performance remains stable across a wide range of key hyperparameters (α, γ, σ, and tol), confirming the robustness of the proposed framework.

[0143] 3. SpaAlign integrates sparse epigenome-transcriptome data to resolve fine structure.

[0144] We next evaluated SpaAlign on the spatial ATAC-RNA dataset from coronal sections of the P22 mouse brain (9,215 cells), a challenging testing platform due to its high anatomical complexity and chromatin sparsity. Using the Allen mouse brain atlas as a reference, we assessed its ability to resolve fine anatomical structures (e.g., cortex, caudate putamen, lateral ventricles). Figure 15 ).

[0145] like Figure 16 and Figure 34 As shown, SpaAlign successfully reconstructed biologically consistent brain regions, uniquely resolving the cortical layer and fine subcortical structures (e.g., cp, vl, lpo, aca) with high fidelity. Conversely, the baseline suffered from severe fragmentation (MUSE, totalVI, STAGATE) or blurred boundaries (SpatialGlue, PRAGA). This superior organization seamlessly extends into the latent space: UMAP visualization ( Figure 17 This study confirms that SpaAlign produces compact, anatomically aligned clusters, in stark contrast to the diffusion (MUSE) or severely entangled (STAGATE, totalVI) embeddings of competing methods.

[0146] Quantitatively, at the reference resolution (n = 14), SpaAlign outperforms all baselines across all six metrics. Figure 17 ; Figure 35 SpaAlign achieved the highest NMI (0.407, a 3.3% improvement over MUSE) and ARI (0.287 vs. 0.276). Furthermore, SpaAlign maintained top-level stability even under suboptimal settings (n = 12–16). Figure 18 The competitive method degenerates significantly under suboptimal settings.

[0147] 4. SpaAlign identifies biological microstructures through trimodal data integration.

[0148] To evaluate the trimodal integration for identifying tissue microstructures, we applied SpaAlign to a human tonsil dataset that includes spatial transcriptomics, proteomics, and H&E histology. This dataset is well-annotated with histological and molecular markers, serving as a robust benchmark for spatial localization. Figure 20 ).

[0149] We benchmarked SpaAlign against eight other methods, categorized by input capacity: only SpaAlign, MISO, and MUSE utilized all three modalities, while the others were limited to RNA and protein because they lacked architectural support for integrating direct histological images. Notably, despite their general competitiveness, PRAGA and SpatialGlue's inability to integrate image-derived structural cues limited their performance in fine-grained tasks such as germinal center delineation.

[0150] SpaAlign achieves the most accurate germinal center (GC) depiction with clear boundaries and minimal overflow, while MUSE, STAGATE, and totalVI produce highly fragmented or overlapping clusters. Figure 22 ; Figure 36 The F1 score is used as a GC identification metric. Figure 21 SpaAlign achieved the highest score (0.94), outperforming MISO (0.90) and all other methods. Notably, methods excluding histology (e.g., GraphST, 0.87; Glue, 0.86; PRAGA, 0.84) performed moderately, while methods excluding space (e.g., SpatialGlue, 0.67; MultiVI, 0.31) showed significantly weaker correspondences. These results highlight the advantages of SpaAlign's contrastive alignment and interaction-based design in fusing trimodal data to achieve fine granularity.

[0151] The analysis of domain-enriched genes further validated the biological significance of SpaAlign's recognition of spatial domains. Figure 23 Differential expression analysis of the original gene expression matrix was performed using the Wilcoxon rank-sum test to identify domain-specific marker genes. The identified GC regions showed strong enrichment for known GC markers such as RGS13 and MME, while SERPINA9 exhibited a distinct complementary pattern in the surrounding mantle region. These biologically meaningful spatial expression patterns suggest that SpaAlign accurately reconstructs the different transcriptionally distinct microenvironments.

[0152] To further investigate the structural organization of the learned representations, we visualized the latent space using UMAP and computed a point-to-point Pearson correlation heatmap from the learned embeddings. Figure 24 While unimodal embeddings appear diffuse and lack clear structural separation, SpaAlign's integrated representations exhibit a distinct block-diagonal correlation structure, indicating strong intra-domain similarity and clear inter-domain separation. These results suggest that multimodal integration facilitates the recovery of more consistent and spatially organized representations, thus supporting more accurate spatial domain identification.

[0153] To validate the architecture design of SpaAlign, we conducted ablation experiments on the tonsil dataset. Without histological data, SpaAlign outperformed all competing methods, achieving an F1 score of 0.9109. Adding histological data further improved performance, with an F1 score of 0.9383, mainly due to improved recall (from 0.8473 to 0.8977) while maintaining high accuracy. In addition, we used four state-of-the-art visual models (HIPT

[28] , omiCLIP

[29] , Prov-GigaPath

[30] , and UNI

[31] ) as feature encoders ( Figure 37 The results showed that HIPT achieved the highest F1 score (0.94) by providing the best balance between precision (0.98) and recall (0.90), outperforming alternatives such as omiCLIP (0.90) and UNI (0.90), which, despite their high precision, had lower recall (0.70). Therefore, HIPT was selected as the default image encoder for all subsequent analyses.

[0154] To quantify the unique contribution of protein modalities to spatial cell type clustering, we conducted a systematic ablation study on the human tonsil dataset, evaluating the performance under different modality combinations. Compared to RNA-only analysis, integrating surface protein measurements improved ARI and F1 scores by 2.3% and 9.6%, respectively, with similar sustained improvements observed in NMI, AMI, FMI, V-measure, and integrity. These results highlight the complementary role of protein information in mitigating the inherent sparsity of transcriptomic profiles. Furthermore, the best overall performance was achieved by jointly integrating transcriptomics, proteomics, and spatial imaging information. Figure 38 Compared with RNA analysis alone, the ARI and F1 scores improved by 5.8% and 15.3%, respectively. These findings suggest that synergistic multimodal integration is crucial for accurately depicting heterogeneous tissue microenvironments.

[0155] In addition, we extended our evaluation to another trimodal spatial multi-omics dataset—the human breast cancer dataset, which also includes spatial transcriptomics, proteomics, and H&E histology

[32] . SpaAlign produced biologically consistent spatial domains with clearly separated boundaries ( Figure 39 Quantitatively, SpaAlign achieved the best performance across all six supervised clustering metrics, with an ARI of 0.561, exceeding the second-best method, PAST, by 11.9%. Figure 40 These results confirm that SpaAlign's trimodal integration advantage is robust across different tissue types.

[0156] This embodiment provides a unified representation learning system for spatial multi-omics, used to execute the aforementioned unified representation learning method for spatial multi-omics. The system includes: The data preprocessing and graph construction module is used to acquire and preprocess spatial multi-omics data and construct spatial affinity graphs. The modality coding module is used to extract the latent embedding representation of each modality data through a modality-specific encoder, and to apply orthogonal constraints to the latent embedding representation; A structure preservation module is used to construct a graph Laplacian regularization term based on the spatial affinity graph to constrain the topological consistency of spatially neighboring points in the potential embedding representation; The semantic alignment module is used to align the latent embedding representations using a cross-modal contrastive learning strategy to construct a unified semantic space. The interactive fusion module is used to explicitly compute cross-modal high-order interactive features through tensor outer product mechanism and compress and fuse them to obtain a composite representation vector; The clustering refinement module is used to input the composite representation vector into the adaptive embedding clustering module, and perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution to output the spatial domain partitioning result.

[0157] The data flow and logical control relationships between the modules correspond completely to steps S1 to S7, and will not be elaborated here.

[0158] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the aforementioned unified representation learning method for spatial multi-omics. The storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A unified representation learning method for spatial multi-omics, characterized in that, Includes the following steps: S1. Acquire spatial multi-omics data, wherein the data includes at least spatial molecular data and histological image data, and construct a spatial affinity map based on the physical spatial coordinates of each data point; S2. Feature extraction is performed on each modality data using a modality-specific encoder to generate a latent embedding representation for each modality, and orthogonal parameterization constraints are applied to the latent embedding representation. S3. Calculate the graph Laplacian regularization term based on the spatial affinity graph to constrain the topological smoothness of spatial neighbor nodes in the potential embedding representation; S4. Employ a cross-modal contrastive learning strategy to semantically align the latent embedding representations and construct a unified cross-modal shared semantic space; S5. Calculate the high-order interaction features of the latent embedding representations of each modality after alignment through the tensor outer product mechanism, and then fuse them with the original embeddings after dimensionality reduction and compression to generate a composite representation vector. S6. Input the composite representation vector into the adaptive embedding clustering module, perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution, and output the biological spatial domain partitioning result with clear spatial boundaries.

2. The unified representation learning method for spatial multi-omics as described in claim 1, characterized in that, Step S1 specifically includes: High-variability gene screening, logarithmic transformation, and Z-score normalization were performed on the spatial molecular data. Histological image data is segmented into blocks, and hierarchical morphological features are extracted using a hierarchical visual Transformer and aggregated into point-level image embeddings. Based on the two-dimensional spatial coordinates of each data point, a fully connected spatial affinity graph is constructed by calculating the affinity between points using a Gaussian kernel function, or a sparse spatial affinity graph is constructed by retaining only the k nearest neighbors with the closest physical distance based on a preset k value.

3. The unified representation learning method for spatial multi-omics as described in claim 2, characterized in that, Step S2 specifically includes: The high-dimensional feature matrices of each modality are projected onto a preset low-dimensional subspace using principal component analysis for initial denoising. The dimensionality-reduced features are input into a multilayer perceptron to learn modality-specific deep nonlinear representations. An orthogonal parameterization constraint is imposed on the output matrix of the multilayer perceptron, which forces the product of the output matrix and its transpose to be equal to the identity matrix, in order to preserve the geometric volume of the feature space and prevent the collapse of the multimodal representation.

4. The unified representation learning method for spatial multi-omics as described in claim 3, characterized in that, Step S3 specifically includes: Construct the graph Laplacian matrix and calculate the squared Euclidean distance between neighboring nodes in the latent embedding representation in the spatial graph; The squared Euclidean distance is added as a penalty term to the objective function, forcing the model to learn a continuous and smooth latent manifold representation in physical space, thereby filtering out local technical noise and salt-and-pepper noise.

5. A unified representation learning method for spatial multi-omics as described in claim 4, characterized in that, Step S4 specifically includes: Based on the symmetric InfoNCE loss function, the mutual information between paired modal signals is maximized, thus narrowing the feature distance between corresponding points in the unified semantic space. A distance regularization term based on margin is introduced to explicitly constrain the Euclidean distance between non-corresponding points to be greater than the preset safety margin, so as to overcome the limitation of cosine similarity that only considers angular correlation.

6. The unified representation learning method for spatial multi-omics as described in claim 5, characterized in that, Step S5 specifically includes: Perform tensor outer product operation on the two modal latent embeddings of each data point to generate a high-dimensional interaction tensor that captures all pairwise correlations; The high-dimensional interaction tensor is compressed using principal component analysis, and the compressed interaction features are concatenated with the original embedding representation to form a composite representation vector that includes modality-specific details and combined synergistic effects.

7. A unified representation learning method for spatial multi-omics as described in claim 6, characterized in that, Step S6 specifically includes implementing a two-stage training strategy: The first stage is the multi-objective pre-training stage, which jointly optimizes the modality reconstruction loss, graph Laplacian structure preservation loss and cross-modal contrastive loss to complete representation learning and semantic alignment. The second stage is the joint fine-tuning stage, which initializes the cluster centroids, activates the adaptive embedding clustering module, uses the model's own high confidence prediction to construct the target distribution, and performs end-to-end joint optimization through KL divergence loss to enhance intra-class compactness and inter-class separation.

8. A unified representation learning method for spatial multi-omics as described in claim 1, characterized in that, It also includes step S7: based on the latent embeddings obtained by the unified representation learning, perform downstream spatial analysis tasks, including UMAP high-dimensional visualization, cell type annotation, spatially variable gene detection, or cross-modal spatial correlation map construction.

9. A unified representation learning system for spatial multi-omics, characterized in that, include: The data preprocessing and graph construction module is used to acquire and preprocess spatial multi-omics data and construct spatial affinity graphs. The modality coding module is used to extract the latent embedding representation of each modality data through a modality-specific encoder, and to apply orthogonal constraints to the latent embedding representation; A structure preservation module is used to construct a graph Laplacian regularization term based on the spatial affinity graph to constrain the topological consistency of spatially neighboring points in the potential embedding representation; The semantic alignment module is used to align the latent embedding representations using a cross-modal contrastive learning strategy to construct a unified semantic space. The interactive fusion module is used to explicitly compute cross-modal high-order interactive features through tensor outer product mechanism and compress and fuse them to obtain a composite representation vector; The clustering refinement module is used to input the composite representation vector into the adaptive embedding clustering module, and perform end-to-end fine-tuning by minimizing the KL divergence between the clustering distribution and the high-confidence target distribution to output the spatial domain partitioning result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images