Spatial transcriptome region identification method driven by multi-modal graph fusion

Abstract

The invention discloses a spatial transcriptome region identification method driven by multi-modal graph fusion, and belongs to the field of biomedical informatics and computational biology. The method comprises the following steps: step 1, data acquisition; 2, data processing; step 3, generating a multi-modal adjacent matrix; 4, carrying out the structure fusion of the differentiable multi-modal graph; step 5, variational graph self-encoder training is carried out; and step 6, clustering. According to the method, the spatial domain recognition framework based on multi-modal graph fusion and the variational autoencoder is constructed, so that the accuracy, continuity and biological interpretability of spatial transcriptome clustering analysis are remarkably improved, and the defects of an existing method in the aspects of multi-modal fusion, graph structure optimization and feature discrimination are effectively overcome; and a more reliable calculation tool is provided for deeply revealing tissue spatial heterogeneity in the field of advanced biomedical research such as brain science, tumor microenvironment analysis and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of biomedical informatics and computational biology, and relates to a spatial transcriptome data analysis method, in particular to a spatial domain recognition method based on multi-modal graph fusion and dynamic structure optimization. BACKGROUND

[0002] The rapid development of spatial transcriptomics technology provides strong technical support for analyzing the microenvironment of tissues in situ. Through advanced technology platforms such as 10x Visium, MerFISH and seqFISH, researchers can obtain the whole genome expression profile while preserving the spatial location information, greatly advancing the depth of understanding of the structural heterogeneity of tissues. Accurate identification of the "spatial domain" with specific functions in tissues has become a key link to understanding the physiological functions and pathological states of tissues. However, spatial transcript data has high dimensionality, sparsity, high noise and multi-modal characteristics, making precise spatial domain recognition face serious technical challenges. Existing spatial domain recognition methods have obvious evolution process and development bottlenecks when dealing with complex spatial transcript data.

[0003] Early methods mainly borrowed clustering strategies from single-cell RNA sequencing, directly using algorithms such as K-means, Louvain and Leiden, and only relying on gene expression similarity for region division. This method completely ignores spatial location information, resulting in fragmented clustering results in complex tissues, which cannot reflect the true biological tissue structure.

[0004] To make up for the lack of spatial information, researchers began to try to introduce spatial constraints into the clustering process. BayesSpace builds a hierarchical Bayesian model and introduces a spatial smoothing prior, forcing adjacent points to tend to belong to the same spatial domain, which improves the spatial coherence of the clustering results to some extent. However, this method only realizes the shallow fusion of spatial information and fails to establish a deep connection between gene expression and spatial location, and is prone to over-smoothing in the boundary region of the tissue.

[0005] With the development of graph neural network technology, researchers began to explore deeper information fusion methods. By regarding each measurement point as a node in the graph and constructing adjacency relationships based on spatial coordinates, a leap from single-modal to dual-modal fusion was achieved. SpaGCN made an important attempt in this direction, integrating gene expression and spatial coordinates in the graph convolution network framework for joint learning. Subsequent methods such as STAGATE further introduced graph attention mechanisms, which can adaptively learn the importance between adjacent nodes, enhancing the modeling ability of spatial context.

[0006] However, the above methods still have obvious shortcomings in information integrity - they only integrate two modalities of spatial location and gene expression, but ignore the histological images that contain rich histological structure information. To overcome this limitation, researchers have begun to explore solutions for three-modal fusion. The EfNST method represents an important development in this direction, by using a pre-trained EfficientNet-B0 network to extract deep features of tissue sections, and combining graph attention mechanisms to achieve cross-modal feature interaction. Although this method has made progress in the utilization of morphological information, its feature extraction and graph learning processes are separate, and the pre-trained image features and gene expression features are separated in the semantic space, making it impossible to achieve true end-to-end collaborative optimization.

[0007] The common technical difficulties faced by current multi-modal fusion methods mainly manifest in three aspects: First, different modal data have heterogeneous characteristics, with spatial coordinates being continuous numerical values, gene expression being high-dimensional sparse matrices, and tissue images being two-dimensional pixel arrays. Traditional splicing or weighted fusion is difficult to fully exploit their deep connections. Second, existing methods mostly use static graph structures, which cannot dynamically adjust the fusion weights between modalities during training. Finally, the separation of feature extraction and fusion learning prevents deep interaction and complementary enhancement of modal information in a unified framework.

[0008] Existing spatial transcriptomic region identification methods have obvious limitations in fully utilizing multi-modal information and optimizing graph structure modeling. Specifically:

[0009] (1) Insufficient multi-modal information fusion: Most existing methods can only cooperatively utilize spatial location and gene expression modalities, and fail to effectively integrate tissue section images that contain rich morphological information, or only use simple feature splicing and other shallow fusion strategies, which cannot achieve deep interaction and adaptive collaboration of cross-modal information, limiting the model's comprehensive analysis ability of complex biological systems.

[0010] (2) Graph structure modeling is rigid: Graph neural network-based methods heavily rely on pre-defined, static adjacency relationships (such as fixed graph construction based on spatial K-nearest neighbors). This fixed graph structure cannot dynamically evolve and optimize based on learned features during model training, limiting the model's ability to capture local heterogeneity and complex interaction patterns in tissues.

[0011] (3) Weak discriminability of feature representation: Due to the limitations of insufficient fusion and rigid graph structure, the latent embedding learned by existing methods is not discriminative enough, making it difficult to clearly distinguish between functionally distinct but spatially adjacent fine tissue structures (such as different functional subregions of tumor margins), resulting in blurred spatial domain boundaries and reduced resolution.

[0012] (4) Sensitive to noise and data sparsity: In high-dimensional, high-noise spatial transcriptome data with a large number of zero values, the existing methods are not robust enough and are easily affected by technical noise, resulting in fragmentation of clustering results or errors in region identification. Summary of the Invention

[0013] To overcome the limitations of existing spatial transcriptome analysis techniques in multimodal information fusion and static graph structure, this invention provides a multimodal graph fusion-driven spatial transcriptome region identification method. This method proposes a spatial domain identification framework based on joint optimization of multimodal graph fusion and variational autoencoders. By constructing a unified trimodal graph structure of spatial location, gene expression, and tissue morphology images, it completely breaks through the limitations of traditional single or bimodal analysis. Specifically, a differentiable edge-by-edge fusion mechanism is designed, dynamically learning the optimal weight ratio between spatial proximity, gene expression correlation, and morphological similarity through a multilayer perceptron (MLP), overcoming the limitations of traditional fixed weights or manually set fusion coefficients. At the model architecture level, this invention innovatively integrates variational graph autoencoders and multimodal graph fusion modules, achieving end-to-end joint training. This framework synchronously updates graph structure parameters and node embedding representations through a unified optimization process, transforming the adjacency matrix from a static preset to a dynamic evolution, significantly improving the model's adaptability to complex tissue structures. In particular, the model ensures both the spatial separability of potential embeddings and the biological consistency of feature representations through the synergistic optimization of reconstruction loss and KL divergence.

[0014] The objective of this invention is achieved through the following technical solution:

[0015] A multimodal graph fusion-driven method for spatial transcriptome region identification includes the following steps:

[0016] Step 1, Data Acquisition:

[0017] Download spatial transcriptome data from publicly available databases;

[0018] Step 2: Data Processing

[0019] After the data is read, preprocessing operations are performed, including gene expression data normalization and gene screening. The specific steps are as follows:

[0020] Step 2-1, Gene Expression Data Normalization: Perform a logarithmic transformation on the original gene expression matrix and normalize the data using the SCANPY package;

[0021] Step 2-2, Gene Screening: Perform z-score normalization on the normalized expression matrix and screen for the top gene expression. Using highly variable genes as input features, a preprocessed gene expression matrix is ​​constructed. , For the number of genes, Number of spots;

[0022] Step 3: Generation of multimodal adjacency matrix:

[0023] Spatial adjacency matrices were constructed from three modalities: spatial location, gene expression, and histological image. Gene expression adjacency matrix Image adjacency matrix Spatial feature matrix Gene expression feature matrix and image feature matrix The specific steps are as follows:

[0024] Step 3-1: Spatial Adjacency Matrix Construction: Using the sine and cosine position coding method, the two-dimensional spatial coordinates are mapped into high-dimensional features to obtain the spatial feature matrix. and based on The spatial adjacency matrix is ​​constructed using the K-nearest neighbor algorithm. ;

[0025] Step 3-2, Gene Expression Adjacency Matrix Construction: First, the preprocessed expression matrix... Spatial smoothing enhancement is performed to obtain the spatially enhanced representation matrix. Subsequently Principal component analysis was performed to reduce the dimensionality and obtain the gene expression feature matrix. Based on the dimensionality-reduced features, the K-nearest neighbor algorithm is used to construct the gene expression adjacency matrix. ;

[0026] Step 3-3: Image Adjacency Matrix Construction: A pre-trained convolutional neural network is used to extract deep features of organized image patches centered at each point. Principal component analysis is then used to reduce the dimensionality, resulting in a low-dimensional image feature matrix. ,based on The Euclidean distance is used to construct the image adjacency matrix using the radius nearest neighbor method. ;

[0027] Step 4: Differentiable multimodal graph structure fusion:

[0028] Step 4-1: Convert the spatial feature matrix Gene expression feature matrix Image feature matrix By concatenating the features along the feature dimension, a comprehensive feature representation matrix is ​​obtained. ;

[0029] Step 4-2: The adjacency matrices and their features of spatial, gene expression, and image modalities are fused using an edge-by-edge multilayer perceptron (MLP) to generate a multimodal adjacency matrix. ;

[0030] Step 5: Variational graph autoencoder training:

[0031] Step 5-1, Data Preparation: Prepare the multimodal adjacency matrix. and comprehensive feature expression matrix The input variational graph autoencoder (VAE) is used for training;

[0032] Step 5-2, Encoder Training: In the encoder part, the node features are first processed through graph convolutional layers, and a multimodal adjacency matrix is ​​used. The encoder propagates neighborhood features to obtain the latent features of each node; then, it approximates the posterior distribution of the latent space using a Gaussian distribution and calculates the mean vector of each node. and standard deviation vector ;

[0033] Step 5-3, Decoder Training: The decoder receives the latent variables output by the encoder. The latent embeddings are mapped back to the input feature space; the decoder progressively expands the dimension of the latent embeddings through multiple fully connected layers and enhances the nonlinear expressive power through the LeakyReLU activation function; finally, the decoder restores the latent features to the original input features.

[0034] Step 5-4, Loss function calculation: To optimize the latent space and feature reconstruction, two parts of loss are considered simultaneously during training: reconstruction loss and KL divergence loss;

[0035] Step 5-5, Optimization process: Optimize the parameters of the encoder and decoder by minimizing the weighted sum of the reconstruction loss and KL divergence loss, and jointly update the model parameters using the Adam optimizer. In each round of training, the model performs forward propagation, calculates the loss, and updates the parameters through backpropagation.

[0036] Step 6, Clustering:

[0037] For latent variables The K-Means algorithm was used for clustering to complete the spatial domain clustering of the spatial transcriptome.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) Multimodal information graph fusion: This invention constructs a unified framework for multimodal information sources by integrating spatial coordinates, gene expression data, and H&E stained tissue images. On this basis, adjacency relationships are effectively mapped to the same graph structure, enabling information from different modalities to be collaboratively optimized in the same space.

[0040] (2) Dynamic adaptive graph fusion mechanism: The present invention implements a differentiable graph fusion mechanism through an edge-by-edge multilayer perceptron, which enables the graph structure to be adaptively adjusted according to the learned features during the training process, thereby optimizing the adjacency relationship and significantly improving the adaptability of the graph model to complex organizational structures.

[0041] (3) Deep Multimodal Feature Collaboration: This invention utilizes deep multimodal feature collaboration to standardize and fuse spatial, gene expression, and morphological data at the feature level, generating a comprehensive node feature representation. This method not only enhances the complementarity of information from each modality but also enables the model to achieve deep collaboration at the feature and structural levels, improving the discriminative ability of potential features and thus increasing the recognition accuracy in the spatial domain.

[0042] (4) Variational graph autoencoder joint optimization to handle noise: This invention utilizes variational graph autoencoder joint optimization to ensure the separability and regularity of the latent space by simultaneously optimizing the reconstruction loss and KL divergence. In this process, the model can effectively suppress the influence of noise and maintain high robustness and clustering quality under conditions of high noise and sparse data.

[0043] Through the above technical solutions, this invention effectively addresses the shortcomings of existing methods in multimodal fusion, graph structure optimization, and feature discriminability. These improvements significantly enhance the accuracy, continuity, and biological consistency of spatial domain identification, providing a more reliable computational tool for in-depth understanding of tissue spatial heterogeneity in cutting-edge biomedical research fields such as brain science and tumor microenvironment analysis. Attached Figure Description

[0044] Figure 1 This is the overall framework diagram;

[0045] Figure 2 For data input and multimodal information processing;

[0046] Figure 3 For the fusion of multimodal features and graph structure;

[0047] Figure 4 For variational autoencoder training and spatial domain recognition;

[0048] Figure 5 For the visualization of spatial domain identification results, (a) manually annotated cortical hierarchies (L1-L6) and white matter (WM) regions, (b) clustering results of the method of this invention, and (c) clustering results of other methods;

[0049] Figure 6 For clustering performance comparison (ARI score);

[0050] Figure 7 Visualize UMAP and PAGA;

[0051] Figure 8 This is a spatial domain visualization comparison diagram of the method of the present invention and eight comparative methods on Human_breast_cancer;

[0052] Figure 9 For spatial domain analysis on a mouse forebrain dataset, (a) region results annotated by pathologists, (b) spatial clustering results of the method of this invention, (c) identification results of domains 17 and 36, and (d) region identification results of eight other methods. Detailed Implementation

[0053] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0054] This invention provides a multimodal graph fusion-driven method for spatial transcriptome region identification, such as... Figure 1 and Figure 2 As shown, the method includes the following steps:

[0055] Step 1, Data Acquisition:

[0056] Download spatial transcriptome data from publicly available databases.

[0057] Step 2: Data Processing

[0058] After the data is read, preprocessing operations are performed:

[0059] Step 2-1: Gene expression data normalization: Perform a logarithmic transformation on the original gene expression matrix and normalize the data using the SCANPY package.

[0060] Step 2-2, Gene Screening: Perform z-score normalization on the normalized expression matrix and screen for the top gene expression. Using highly variable genes as input features, a preprocessed gene expression matrix is ​​constructed. ( For the number of genes, (Number of spots). For datasets with fewer highly variable genes (such as the MerFish dataset), all available genes are used as input.

[0061] Step 3: Generation of multimodal adjacency matrix:

[0062] Construct adjacency matrices and feature matrices from three modalities: spatial location, gene expression, and histological image.

[0063] Step 3-1: Spatial Adjacency Matrix Construction: Using the sine and cosine position coding method, the two-dimensional spatial coordinates are mapped into high-dimensional features to obtain the spatial feature matrix. Based on this feature, the K-nearest neighbor algorithm is used to construct a spatial adjacency matrix. .

[0064] Step 3-2, Gene Expression Adjacency Matrix Construction: First, the preprocessed expression matrix... Spatial smoothing enhancement is performed to obtain the spatially enhanced representation matrix. Subsequently, the enhanced expression matrix was analyzed. Principal component analysis was performed to reduce the dimensionality and obtain the gene expression feature matrix. Based on the dimensionality-reduced features, the K-nearest neighbor algorithm is used to construct the gene expression adjacency matrix. .

[0065] Step 3-3: Image Adjacency Matrix Construction: A pre-trained convolutional neural network is used to extract deep features of organized image patches centered at each point. Principal component analysis is then used to reduce the dimensionality, resulting in a low-dimensional image feature matrix. Based on the Euclidean distance of this feature, the radius nearest neighbor method is used to construct the image adjacency matrix. .

[0066] Step 4: Differentiable multimodal graph structure fusion:

[0067] Step 4-1: Convert the spatial feature matrix Gene expression feature matrix Image feature matrix By concatenating the features along the feature dimension, a comprehensive feature representation matrix is ​​obtained. .

[0068] Step 4-2: The adjacency matrices and their features of spatial, gene expression, and image modalities are fused using an edge-by-edge multilayer perceptron (MLP) to generate a multimodal adjacency matrix. In this process, the relationships between different modalities are optimized by weighted learning of the edge features of different modalities, and a comprehensive graph structure is generated.

[0069] Step 5: Variational graph autoencoder training:

[0070] Step 5-1, Data Preparation: Prepare the multimodal adjacency matrix. and comprehensive feature expression matrix The input is trained using a variational graph autoencoder (VAE). The encoder part receives these inputs and propagates neighborhood information through a graph convolutional network (GCN) to aggregate first-order neighborhood features.

[0071] Step 5-2, Encoder Training: In the encoder part, the node features are first processed through graph convolutional layers, and a multimodal adjacency matrix is ​​used. The neighborhood features are propagated to obtain the latent features of each node. Next, the encoder approximates the posterior distribution of the latent space using a Gaussian distribution and calculates the mean vector for each node. and standard deviation vector These parameters are learned by the neural network through training. A reparameterization technique is employed to make the latent embeddings differentiable, facilitating optimization via gradient backpropagation.

[0072] Step 5-3, Decoder Training: The decoder receives the latent variables output by the encoder. The latent embeddings are then mapped back to the input feature space. The decoder progressively expands the dimension of the latent embeddings through multiple fully connected layers and enhances the nonlinear expressive power through the LeakyReLU activation function. Ultimately, the decoder restores the latent features to the original input features, thereby minimizing the reconstruction error.

[0073] Step 5-4, Loss Function Calculation: To optimize the latent space and feature reconstruction, two parts of the loss are considered simultaneously during training: reconstruction loss and KL divergence loss. The KL divergence term constrains the distribution of the latent space, making the latent features approximate a standard normal distribution, thereby ensuring the separability and consistency of the latent space.

[0074] Step 5-5, Optimization Process: Optimize the encoder and decoder parameters by minimizing the weighted sum of the reconstruction loss and the KL divergence loss. The Adam optimizer is used to jointly update the model parameters. In each training round, the model performs forward propagation, calculates the loss, and updates the parameters via backpropagation.

[0075] Step 6, Clustering:

[0076] For latent variables The K-Means algorithm was used for clustering to complete the spatial domain clustering of the spatial transcriptome.

[0077] This invention significantly improves the accuracy, continuity, and biological interpretability of spatial transcriptome clustering analysis by constructing a spatial domain recognition framework based on multimodal graph fusion and variational autoencoders. Specifically:

[0078] (1) Dynamic graph structure optimization: Through a differentiable edge-by-edge MLP fusion mechanism, the optimal weight ratio between spatial proximity, gene expression similarity and morphological features is adaptively learned to generate a graph structure that continuously evolves during training, breaking through the limitations of traditional static graph modeling of complex organizational structures.

[0079] (2) Deep multimodal feature collaboration: Spatial coordinates, gene expression and tissue image three-modal features are spliced ​​and jointly learned under a unified framework, realizing deep interaction and complementary enhancement of cross-modal information, which significantly improves the discriminative ability and biological consistency of potential embeddings.

[0080] (3) Enhanced robustness by variational autoencoder: The fused graph structure is trained end-to-end using variational graph autoencoder. By jointly optimizing the reconstruction loss and KL divergence, the model learns a regular and separable low-dimensional representation while ensuring data fidelity, which effectively improves the model’s adaptability to high noise and high sparse data.

[0081] Example:

[0082] Step 1: Data Acquisition.

[0083] Five spatial transcriptome datasets were obtained from public databases (such as 10X Genomics and MerFish), covering samples from the DLPFC (human dorsolateral prefrontal cortex), human breast cancer, and mouse brain. All datasets include baseline clustering labels. The datasets contain the following information: spatial location coordinates (two-dimensional coordinates of each spot), gene expression matrices (rows represent genes, columns represent spots), and H&E-stained tissue slice images.

[0084] Step 2: Data processing.

[0085] The original gene expression matrix was logarithmically transformed and normalized using the SCANPY tool; the normalized expression matrix was then z-score standardized. The top 2000 highly variable genes were selected as input features to construct the preprocessed expression matrix. ( For the number of genes, (Number of spots).

[0086] Step 3: Construct the multimodal adjacency matrix.

[0087] Step 3-1, Spatial Adjacency Matrix Construction: A sine-cosine positional encoding method is used to map the two-dimensional spatial coordinates into high-dimensional features, resulting in a matrix of dimensions... Spatial feature matrix Specifically, for each dimension, using a one-dimensional frequency band, the point encoding... for:

[0088]

[0089] (1)

[0090] in, Represented as a point spatial coordinates, , Spatial characteristic matrix The frequency band selection ensures that each coordinate dimension can be effectively encoded at different frequencies.

[0091] Subsequently, based on the encoding space of that location, a graph is constructed using K-nearest neighbors to generate a spatial adjacency matrix. :

[0092] (2)

[0093] in, These are elements of the spatial adjacency matrix, representing points. and points The relationship between them in spatial coordinate modes, i.e. and Does a spatial connection exist? Indicates the midpoint of a spatial location of A set of nearest neighbors; It is one of the points in space, indicating its position relative to other points. Spatial relationships.

[0094] Step 3-2, Gene Expression Adjacency Matrix Construction: Based on Preprocessed Expression Matrix Identify each spot in coordinate space The Euclidean nearest neighbors are calculated. Then, the mean gene expression within the neighborhood of each location is calculated and linearly superimposed using weights α to generate a spatially enhanced expression matrix. :

[0095] (3)

[0096] in, The enhanced feature representation is augmented using features from weighted neighborhood points. At the gene expression level Feature representation; for of A set of spatial neighborhoods; For the number of spots, Index for a given spot location.

[0097] Next, respectively Perform principal component analysis (PCA) on rows and columns, and take the top row and column row. One principal component is used to obtain the gene expression feature matrix. and Subsequently, regarding Perform k-nn to obtain the gene expression adjacency matrix. :

[0098] (4)

[0099] in, Point and points The relationship between them under gene expression modalities Indicates the midpoint of gene expression data of A nearest neighbor set It is one of the points in space, indicating its position relative to other points. Spatial relationships.

[0100] Step 3-3: Image Adjacency Matrix Construction: Morphological information from tissue slices is used to characterize the relationships between spatial points. First, based on the coordinates of each spot in the spatial transcriptome data, local image patches centered on each spot are obtained. Then, PCA dimensionality reduction is performed to obtain the image feature matrix. :

[0101] (5)

[0102] in, Number of spots; For image level Feature representation; The image feature dimension corresponding to each spot; The node features of the image modality are input into the downstream model, and the image adjacency matrix is ​​further obtained. :

[0103] (6)

[0104] in, Point and points The relationship between them in the image modality, i.e., points and points Are they adjacent in the image space? express Image feature vectors, express Image feature vectors, express and Euclidean distance in image feature space This is the radius threshold.

[0105] Step 4: Differentiable multimodal graph structure fusion.

[0106] Step 4-1, Adjacency Matrix Stacking: Stack the adjacency matrices of the three modalities along the edge dimension to construct a fused edge feature vector that includes spatial modality, gene expression modality, and image modality. The edge feature represents the node. and nodes Similarity across different modalities. Specifically, it is represented as follows:

[0107] (7)

[0108] in, Point and points The relationship between them under gene expression modalities Point and points The relationship between them in spatial coordinate modes Point and points The relationship between them in the image modality.

[0109] Step 4-2, MLP Fusion: The input is fed into an edge-by-edge multilayer perceptron (MLP) to learn the fusion weights for cross-modal edges. This MLP consists of two fully connected layers, each followed by a LeakyReLU activation function (with a negative half-axis slope). The first layer is followed by Dropout regularization to mitigate overfitting. Finally, the output is compressed to the [0,1] interval using the Sigmoid function to obtain the fused edge weights.

[0110] (8)

[0111] (9)

[0112] (10)

[0113] (11)

[0114] in, , This represents the intermediate representation after processing with the LeakyReLU activation function. } represents the set of parameters to be learned. This represents the Sigmoid function. For activation functions with a negative slope of 0.2, This corresponds to the bias term of the weight matrix, and the final output multimodal adjacency matrix is... This process involves pattern measurement entirely through training, thus enabling backpropagation of gradients during training. Perform end-to-end optimization.

[0115] Step 4-3, Feature Fusion: In the feature-level fusion, the feature matrix of the three modes is... , , Each feature is standardized (zero mean, unit variance) to eliminate scale differences between different modalities. Then, they are concatenated along the feature dimensions to obtain the comprehensive feature representation matrix. :

[0116] (12)

[0117] Step 5: Training the variational graph autoencoder.

[0118] Step 5-1, Encoder Part Training: The goal of the encoder part is to transform the input comprehensive feature representation matrix. and multimodal adjacency matrix The distribution is transformed into a latent space. The encoder propagates the neighborhood information of nodes through graph convolutional layers (GCN) and approximates the posterior distribution of the latent space using a Gaussian distribution. The specific process is as follows:

[0119] (1) Graph convolutional layer: In the graph convolutional layer of the encoder, each graph convolutional operation propagates the features of the nodes through the adjacency matrix. :

[0120] (13)

[0121] in, This is a multimodal adjacency matrix; L=1,2, The weight matrix is ​​a learnable matrix. For the feature dimension of the previous layer, The feature dimension of the current layer. For activation functions with a negative slope of 0.2, It is the bias term of the Lth layer, used to adjust the output.

[0122] (2) Gaussian distribution approximation: The encoder approximates the posterior distribution of the latent space using a Gaussian distribution. The latent features of each spot are represented by a mean vector and a standard deviation vector, and these two parameters are learned by the encoder through a neural network.

[0123] (14)

[0124] in, Represent a Gaussian distribution; and It is calculated through the encoder's neural network, representing the nodes. The mean and standard deviation of the latent characteristics; Representing latent variables, at this point, It is the node feature matrix after graph convolution processing.

[0125] (3) Reparameterization Technique: To make the sampling process of the latent variables differentiable and to enable optimization through gradient backpropagation, a reparameterization technique is employed. This involves using the mean... and standard deviation and noise variables sampled from the standard normal distribution. Latent variables can be obtained. Sampling process:

[0126] (15)

[0127] in, Representing latent variables The Each sample value, Let the noise variable represent a standard normally distributed noise variable. This represents a standard normal distribution.

[0128] Step 5-2, Encoder Decoding: Decoding the latent variables in the latent space The input data is mapped back to the original data space and reconstructed. The decoder maps the latent space to the data space through layer-by-layer linear transformations and nonlinear activation functions.

[0129] Step 5-3, Loss Function: The training objective is to minimize the weighted sum of the reconstruction loss and the KL divergence loss. During training, the following loss function is minimized:

[0130] To measure the difference between the decoded features and the original input, we define the reconstruction loss:

[0131] (16)

[0132] in, The expression matrix reconstructed by the decoder, This represents the Frobenius norm, used to measure the element-wise reconstruction error of a matrix.

[0133] However, relying solely on reconstruction error is insufficient to constrain the distribution of the latent space. To avoid disordered distribution or overfitting in the latent space, a KL divergence term is further introduced to measure the approximate posterior. With standard Gaussian prior Differences:

[0134] (17)

[0135] in, Embedded vector Dimensions For the first The mean of the 10 latent variables, For the first The variance of each latent variable. The KL divergence term can constrain the latent embedding to approximate a standard normal distribution, thereby ensuring its distribution regularity and separability, and enhancing the separability and consistency among latent variables.

[0136] Ultimately, the overall optimization objective function combines the reconstruction loss and the KL divergence:

[0137] (18)

[0138] in, These are the weight parameters for the KL divergence. During training, the Adam optimizer is used to optimize the parameter set. A joint update is performed. In each iteration, the model performs forward propagation, calculates the loss function, and backpropagates to update the parameters.

[0139] Step 6: Latent variable generation and clustering.

[0140] After training, in order to perform cluster analysis, the latent embeddings are... The K-Means algorithm was used for clustering to complete the spatial domain clustering of the spatial transcriptome.

[0141] pass Figure 5 It can be seen that the method of the present invention clearly distinguishes each cortical layer and white matter region, and is superior to other methods in terms of layer differentiation.

[0142] pass Figure 6 It can be seen that the method of this invention performs best and has strong stability. The closer the homogeneity and normalized mutual information (NMI) scores are to 1, the better the effect. The results show that the method of this invention performs excellently across multiple metrics, and its highest ARI score reaches 0.62, outperforming other comparative methods.

[0143] pass Figure 7 It can be seen that the method of the present invention presents a clear trajectory from L1 to L6 and then to white matter, while the hierarchical boundaries of other methods are more blurred.

[0144] pass Figure 8As can be seen, the method of this invention can better identify the boundaries of the spatial domain, and generally presents a more coherent and clear spatial division. In contrast, EfNST and DeepST divide the IDC_5 region into multiple regions, while SpaGCN, Seurat, and STAGATE are somewhat disordered in their overall division.

[0145] pass Figure 9 A visual comparison of the methods shown reveals that the method of the present invention, as a whole, clearly separates the components into blocks.

Claims

1. A method for spatial transcriptomic region identification driven by multi-modal graph fusion, characterized in that The method comprises the following steps: Step 1, data acquisition: Downloading spatial transcriptome data from a public database; Step 2: data processing: After data reading, pre-processing operations are performed, including gene expression data normalization and gene screening; Step 3, multi-modal adjacency matrix generation: Constructing spatial adjacency matrix from three modalities of spatial position, gene expression and histological image respectively gene expression adjacency matrix image adjacency matrix spatial feature matrix gene expression feature matrix and image feature matrix ; Step 4, differentiable multi-modal graph structure fusion: Step 4-1, obtaining a spatial feature matrix , a gene expression feature matrix , an image feature matrix Splicing in the feature dimension to obtain a comprehensive feature expression matrix ; Step 4-2, the adjacency matrices of the space, gene expression and image modalities and their features are fused by edge-wise multilayer perceptron to generate a multi-modal adjacency matrix ; Step 5, variational graph autoencoder training: Step 5-1, Data Preparation: Constructing the multi-modal adjacency matrix and the integrated feature representation matrix to train the input variational graph autoencoder; Step 5-2, Encoder Training: In the encoder part, the node features are first processed through graph convolutional layers, and a multimodal adjacency matrix is ​​used. The encoder propagates neighborhood features to obtain the latent features of each node; then, it approximates the posterior distribution of the latent space using a Gaussian distribution and calculates the mean vector of each node. and standard deviation vector ; Step 5-3, Decoder Training: The decoder receives the latent variable output by the encoder and maps it back to the input feature space. and maps it back to the input feature space. The decoder gradually expands the dimension of the latent embedding through multiple fully connected layers and enhances the non-linear representation capability through the LeakyReLU activation function; finally, the decoder restores the latent features to the original input features; Step 5-4, loss function calculation: to optimize the latent space and feature reconstruction, two parts of loss are considered simultaneously during the training process: reconstruction loss and KL divergence loss; Step 5-5, optimization process: by minimizing the weighted sum of reconstruction loss and KL divergence loss, the parameters of the encoder and decoder are optimized, and the Adam optimizer is used to update the model parameters jointly; in each round of training, the model performs forward propagation, calculates the loss and updates the parameters through back propagation; Step 6, clustering: Potential variables Clustering was performed using the K-Means algorithm to complete the spatial domain clustering division of the spatial transcriptome.

2. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 1, wherein The specific steps of step 2 are as follows: Step 2-1, gene expression data normalization: log transformation is performed on the original gene expression matrix, and data normalization is performed using the SCANPY package; Step 2-2, gene screening: z-score standardization is performed on the normalized expression matrix, and the top 1000 high-variation genes are screened as input features to form the preprocessed gene expression matrix .​ 3. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 1, wherein The specific steps of step 3 are as follows: Step 3-1, space adjacency matrix construction: using the sine and cosine position encoding method to map the two-dimensional space coordinates into high-dimensional features to obtain a space feature matrix , and based on K-neighbor algorithm is used to construct the space adjacency matrix ; Step 3-2, Gene Expression Adjacency Matrix Construction: First, the preprocessed expression matrix... Spatial smoothing enhancement is performed to obtain the spatially enhanced representation matrix. Subsequently Principal component analysis was performed to reduce the dimensionality and obtain the gene expression feature matrix. Based on the dimensionality-reduced features, the K-nearest neighbor algorithm is used to construct the gene expression adjacency matrix. ; Step 3-3, image adjacency matrix construction: the deep features of the tissue image block centered at each point are extracted by using a pre-trained convolutional neural network, and dimension reduction is performed by principal component analysis to obtain a low-dimensional image feature matrix , based on the Euclidean distance, the image adjacency matrix is constructed by using the radius neighbor method .

4. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 3, wherein The step 3-1, for each dimension, uses a one-dimensional frequency band, then the point is encoded is: , ； wherein, the spatial coordinates of a point are represented as , the dimension of the spatial feature matrix is Subsequently, using K-Nearest Neighbors, a spatial adjacency matrix is generated based on the encoding space of the locations : ； wherein, represents the relationship between points and points in a spatial coordinate modality; represents a set of nearest neighbors of a point in a spatial location; is one of the points in a spatial location representing a spatial relationship to other points .

5. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 3, wherein The specific steps of step 3-2 are as follows: Based on pre-processed expression matrix In the coordinate space, for each spot, identify its Euclidean neighbors; The average of gene expression in the neighborhood of each point is calculated, and the space-enhanced expression matrix is generated by linear superposition with weight α : ； wherein, to enhance the generated enhanced feature representation by weighting features of neighboring points, at the gene expression level of the feature representation; to of the a set of spatial neighbors; is the number of spots, is a spot position index; respectively, and the first principal components of the rows and columns are analyzed, and the first principal components are taken to obtain the gene expression feature matrix and ; To perform knn, get gene expression adjacency matrix : ； in, Point and points The relationship between them under gene expression modalities Indicates the midpoint of gene expression data of A nearest neighbor set It is one of the points in space, indicating its position relative to other points. Spatial relationships.

6. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 3, wherein The specific steps of step 3-3 are as follows: According to the coordinates of each spot in the spatial transcriptome data, a local image block centered on each spot is obtained; By PCA dimension reduction, the image feature matrix is obtained : ； wherein, is the number of spots; is a feature representation of the image level of the image. input the node features of the image modality into the downstream model, and further obtain an image adjacency matrix : ； wherein, a point and a point represent a relationship between the points in an image modality, represent an image feature vector of the point represent an image feature vector of the point represent and a Euclidean distance between the points in the image feature space, is a radius threshold.

7. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 1, wherein The specific steps of step 4-2 are as follows: Adjacent matrix stacking: stack the adjacent matrices in the three modalities in the edge dimension to construct a fusion edge feature vector containing the spatial modality, gene expression modality and image modality : ； wherein, represents the relationship between points and points in a gene expression modality, represents the relationship between points and points in a spatial coordinate modality, represents the relationship between points and points in an image modality; MLP fusion: is input into a multi-layer perceptron to learn the fusion weights of cross-modal edges, and the output is compressed to the interval [0, 1] by a Sigmoid function to obtain the fusion edge weights: ； ； ； ； wherein, , denotes an intermediate representation processed by a LeakyReLU activation function, } is a set of parameters to be learned, denotes a Sigmoid function, is an activation function with a negative slope of 0.2, and the final output multi-modal adjacency matrix is .

8. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 1, wherein The specific steps of step 5-2 are as follows: (1) Graph convolution layer: In the graph convolution layer of the encoder, each layer of graph convolution operation propagates the features of nodes through an adjacency matrix : ； wherein, is a multi-modal adjacency matrix; L = 1, 2, is a learnable weight matrix, is an activation function with negative slope of 0.2, is a bias term for the Lth layer to adjust the output; (2) Gaussian distribution approximation: the encoder approximates the posterior distribution of the latent space by a Gaussian distribution : ； where, represents a Gaussian distribution; and represents the mean and standard deviation of the latent feature of a node ; represents a latent variable; (3) Reparameterization trick: Obtain a sampling process for the latent variable z by means of the mean and standard deviation and a noise variable sampled from the standard normal distribution : ； wherein represents a latent variable of the th sample value, represents a noise variable of a standard normal distribution, represents a standard normal distribution.

9. The multi-modal graph fusion driven spatial transcriptomic region identification method of claim 1, wherein In step 5-4, reconstructing the loss is defined as: ； wherein is the representation matrix for the decoder reconstruction, denotes the Frobenius norm; KL divergence loss is defined as: ； wherein, is the dimension of the embedding vector , is the mean of the th latent variable, is the variance of the th latent variable.

Images