Spatiotemporal omics image processing method and device based on spatial dynamic neural network

Abstract

The present application discloses a spatiotemporal omics image processing method and device based on a spatial dynamic neural network, the main purpose of which is to solve the problem that an effective tissue gene representation map cannot be generated in the existing method. It includes obtaining spatial node graph data of tissue genes and cell-to-cell communication pairs based on spatiotemporal omics images; performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes model training, and performing convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, and the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; the output representations of the node layer and the edge layer are integrated through the attention mechanism to generate the representation map of the tissue genes.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a spatiotemporal omics image processing method and device based on spatial dynamic neural network. Background Art

[0002] The study of complex tissues is not limited to the analysis of molecular characteristics of a large number of cells, but also requires an understanding of how the spatial environment affects cell state or function. The transcriptional state of cells can be regulated by the Gene Regulatory Network (GRN), which is a collection that reflects the regulatory relationship between transcription factors (TF) and their downstream target genes. In recent years, breakthroughs in spatially resolved transcriptomics (SRT) technology have made it possible to perform transcriptome feature analysis while retaining positional information, thus providing unprecedented opportunities to capture transcriptional states with spatial context. The spatial arrangement of cells in tissues is essential for the formation of higher-order spatial functional units. Therefore, understanding the cellular ecosystem and its interactions in complex cellular tissues is crucial to advancing multicellular biology research and analyzing spatial functional landscapes.

[0003] Currently, existing tissue genes are usually represented through multimodal images, which integrate gene expression and other modal information to reveal the spatial function of tissue genes, or detect the fine-grained structure of tissue genes by comparing the transcriptome signal of each point with its spatial neighbors. However, integrating gene expression and other modal information or comparing the transcriptome signal of each site with its spatial neighbors ignores the functional associations and interactions between cell types in the tissue microenvironment, greatly limiting the ability to identify cell populations in key specific areas from the graph structure, and affecting the accuracy of combining cell information in the tissue context, making it impossible to generate an effective tissue gene characterization map. Summary of the invention

[0004] In view of this, the present application provides a spatiotemporal omics image processing method and device based on spatial dynamic neural network, the main purpose of which is to solve the problem that it is currently impossible to generate effective tissue gene representation maps.

[0005] According to one aspect of the present application, a spatiotemporal omics image processing method based on a spatial dynamic neural network is provided, comprising: Obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; The spatial node graph data is convolved based on the node layer in the spatial dynamic neural network that has completed model training, and the cell communication pairs are convolved based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; The output representations of the node layer and the edge layer are integrated through an attention mechanism to generate a representation graph of the tissue gene.

[0006] Furthermore, before performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that has completed model training, and performing convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, the method further includes: Acquire gene representation sample data, wherein the gene representation sample data includes node sample data and edge sample data; Constructing a spatial dynamic neural network that integrates the output layer based on an attention mechanism, wherein the spatial dynamic neural network includes a node layer and an edge layer, and the next-level input samples of the node layer and the edge layer include the node sample data and the edge sample data output by the previous level; The spatial dynamic neural network is model trained based on the gene representation sample data to obtain a spatial dynamic neural network that has completed model training.

[0007] Furthermore, before obtaining the gene characterization sample data, the method further includes: Obtaining the original gene expression matrix and adjacency matrix of the tissue genes; Performing enhancement processing on the original gene expression matrix through the adjacency matrix to obtain an enhanced expression matrix, and performing principal component analysis on the enhanced expression matrix to obtain a principal component matrix; Node sample data is constructed based on the adjacency matrix and the principal component matrix.

[0008] Furthermore, before obtaining the original gene expression matrix and adjacency matrix of the tissue genes, the method further includes: Acquiring a spatial omics data set, and determining an original gene expression matrix of genes in different tissues from the spatial omics data set; A similarity matrix is ​​calculated according to the low-dimensional matrix of the original gene expression matrix, and the adjacency matrix is ​​constructed through the similarity matrix.

[0009] Furthermore, before obtaining the gene characterization sample data, the method further includes: Calculating the total communication intensity of genes in different tissues based on the ligand receptor expression levels, and determining an edge feature matrix based on the total communication intensity; Determine an enhanced edge point matrix based on the edge feature matrix and the point connection state; The edge sample data is constructed based on the gene site communication weight and the enhanced site matrix.

[0010] Further, the performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes the model training, and the performing convolution processing on the cell communication pair based on the edge layer in the spatial dynamic neural network includes: Determining a node adjacency matrix based on the enhanced expression features corresponding to the spatial node graph data, and determining a normalized adjacency matrix based on the enhanced point feature matrix of the cell communication pairs; Determine the output representation of the node at the previous level of the node layer by performing operations on the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the first learnable weight matrix; Determine the edge output representation of the previous level of the edge layer by performing operations on the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the second learnable weight matrix; Determine the node output representation of the previous layer and the edge output representation of the previous layer as input representations of the next node layer and the next edge layer respectively, wherein the input representations include the edge feature matrix of the next node layer and the enhanced expression features of the next edge layer; After completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained.

[0011] Furthermore, the method further comprises: After obtaining the representation graph, determining the allocation probability of different clusters of each gene locus, and determining the allocation probability distribution; The target distribution is obtained by normalizing the clustering frequency and the corresponding clustering probability of the target cluster; The assignment probability distribution and the target distribution are subjected to divergence minimization processing to determine a clustering loss, so as to cluster the gene sites in the representation graph based on the clustering loss.

[0012] According to another aspect of the present application, a spatiotemporal omics image processing device based on a spatial dynamic neural network is provided, comprising: An acquisition module, used to obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; A processing module, configured to perform convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes model training, and to perform convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; A generation module is used to integrate the output representations of the node layer and the edge layer through an attention mechanism to generate a representation graph of the tissue gene.

[0013] Furthermore, the device also includes: a construction module, a training module, The acquisition module is further used to acquire gene representation sample data, wherein the gene representation sample data includes node sample data and edge sample data; The construction module is further used to construct a spatial dynamic neural network that integrates the output layer based on an attention mechanism, wherein the spatial dynamic neural network includes a node layer and an edge layer, and the next-level input samples of the node layer and the edge layer include the node sample data and the edge sample data output by the previous level; The training module is also used to perform model training on the spatial dynamic neural network based on the gene representation sample data to obtain a spatial dynamic neural network that has completed model training.

[0014] Further, The acquisition module is also used to acquire the original gene expression matrix and adjacency matrix of the tissue genes; The processing module is further used to perform enhancement processing on the original gene expression matrix through the adjacency matrix to obtain an enhanced expression matrix, and perform principal component analysis on the enhanced expression matrix to obtain a principal component matrix; The construction module is also used to construct node sample data based on the adjacency matrix and the principal component matrix.

[0015] Furthermore, the device also includes: The determination module is also used to obtain a spatial omics data set and determine the original gene expression matrix of genes in different tissues from the spatial omics data set; calculate the similarity matrix according to the low-dimensional matrix of the original gene expression matrix, and construct the adjacency matrix through the similarity matrix.

[0016] Further, The determination module is further used to calculate the total communication intensity of genes in different tissues based on the ligand receptor expression level, and determine the edge feature matrix based on the total communication intensity; The determination module is further used to determine an enhanced edge point matrix based on the edge feature matrix and the point connection state; The construction module is also used to construct edge sample data based on the gene site communication weight and the enhanced site matrix.

[0017] Further, The processing module is specifically used to determine the node adjacency matrix based on the enhanced expression features corresponding to the spatial node graph data, and determine the normalized adjacency matrix based on the enhanced point feature matrix of the cell communication pair; determine the node output representation of the previous level of the node layer by performing operations on the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression features, the binary transformation matrix and the first learnable weight matrix; determine the edge output representation of the previous level of the edge layer by performing operations on the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression features, the binary transformation matrix and the second learnable weight matrix; determine the node output representation of the previous level and the edge output representation of the previous level as the input representation of the next node layer and the next edge layer respectively, and the input representation includes the edge feature matrix of the next node layer and the enhanced expression features of the next edge layer; after completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained.

[0018] Further, The processing module is also used to determine the allocation probability of different clusters of each gene point after obtaining the representation map, and determine the allocation probability distribution; perform normalization processing based on the clustering frequency of the target cluster and the corresponding clustering probability to obtain the target distribution; perform divergence minimization processing on the allocation probability distribution and the target distribution to determine the clustering loss, so as to cluster the gene points in the representation map based on the clustering loss.

[0019] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned spatiotemporal omics image processing method based on spatial dynamic neural network.

[0020] According to another aspect of the present application, a terminal is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned spatiotemporal omics image processing method based on spatial dynamic neural network.

[0021] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages: The present application provides a spatiotemporal omics image processing method and device based on a spatial dynamic neural network. Compared with the prior art, the embodiments of the present application obtain spatial node graph data of tissue genes and cell-to-cell communication pairs based on spatiotemporal omics images; perform convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes model training, and perform convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, and the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; the output representations of the node layer and the edge layer are integrated through the attention mechanism to generate a representation graph of the tissue genes, and by constructing a dynamic graph convolution network, the dynamic relationship of cell-cell communication is combined with gene expression data to achieve dynamic updating of node and edge outputs, more effectively revealing spatial functional areas, and significantly improving the ability of spatial domain detection and expression denoising, enhancing the effect of parsing the spatial structure of genes from the perspective of cell-to-cell interactions, greatly improving the accuracy of combining cell information in the tissue context, and accurately generating an effective tissue gene representation graph.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A flow chart of a spatiotemporal omics image processing method based on a spatial dynamic neural network provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of a convolutional neural network architecture provided in an embodiment of the present application is shown; Figure 3 A block diagram of a spatiotemporal omics image processing device based on a spatial dynamic neural network provided in an embodiment of the present application is shown; Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0025] The present application embodiment provides a spatiotemporal omics image processing method based on a spatial dynamic neural network, such as Figure 1 As shown, the method includes: 101. Obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images.

[0026] In the embodiments of the present application, the current execution subject, as the processing end for generating the expression map, can be a terminal device or a cloud server, etc., to execute the generation step of the tissue gene representation map, which is not specifically limited in the embodiments of the present application. Among them, the tissue base is the gene image of the tissue cells obtained by segmenting different species in different dimensions in the large-scale spatiotemporal omics image data set (spatial transcriptomics database), or it can be the gene image of the tissue cells collected in real time, which is not specifically limited in the embodiments of the present application. The spatial node graph data is constructed based on the multimodal graph structure data. At this time, the multimodal graph structure data includes vector data, image data, and spatial coordinate data, etc. The structural diagram of the characterization of tissue genes in different modes is not specifically limited in the embodiments of the present application. In addition, the intercellular communication pair is COMMOT (COMMunication analysis by Optimal Transport) - communication analysis technology for optimal transport. At this time, the COMMOT method uses the contribution of the ligand-receptor complex to infer spatial cell communication (CCC) through collective optimal transport (COMOT). It can be based on spatial transcriptomics data or spatially annotated scRNA-seq data, while considering a large number of ligand-receptor pairs to infer spatial cell communication (CCC), which is not specifically limited in the embodiments of the present application.

[0027] 102. Perform convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that has completed model training, and perform convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network.

[0028] In the embodiment of the present application, a spatial dynamic neural network with a node layer and an edge layer is pre-established, and convolution processing is performed on the spatial node graph data through the node layer, and at the same time, convolution processing is performed on the cell communication pair based on the edge layer, such as Figure 2As shown, the spatial node graph data is transmitted to the node layer as the input parameter of the model input layer for convolution processing. At the same time, the cell communication pair is transmitted to the edge layer as the input parameter of the model input layer for convolution processing, and then transmitted to the output layer after at least three layers of convolution calculation. In the embodiment of the present application, during the convolution processing of the spatial dynamic neural network, the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately, such as Figure 2 As shown, the first node layer is , the second node layer is , the third node layer is , the first edge layer is , the second edge layer is , the third edge layer is , indicating that the input of the current node layer includes the output of the previous node layer and the output of the previous edge layer. At the same time, the input of the current edge layer includes the output of the previous node layer and the output of the previous edge layer, so as to perform convolution calculations in the current node layer and the edge layer respectively. The embodiment of the present application does not make specific limitations.

[0029] 103. Integrate the output representations of the node layer and the edge layer through an attention mechanism to generate a representation graph of the tissue gene.

[0030] In the embodiment of the present application, after each node layer and edge layer completes the convolution processing, the result is output to the output layer of the spatial dynamic neural network. At this time, the attention mechanism is referenced in the output layer, that is, by dynamically adjusting the attention weight, the important information in the output layer is highlighted and the unimportant information is ignored. Among them, the attention weight can be calculated based on the shared attention vector of the edge layer output and the node layer output, so that the edge layer output and the node layer output are integrated using the attention weight to obtain a representation map of the tissue gene.

[0031] In another embodiment of the present application, for further definition and explanation, before the step of performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes the model training and performing convolution processing on the cell communication pair based on the edge layer in the spatial dynamic neural network, the method further includes: Obtaining genetic characterization sample data; Construct a spatial dynamic neural network that integrates the output layer based on the attention mechanism; The spatial dynamic neural network is model trained based on the gene representation sample data to obtain a spatial dynamic neural network that has completed model training.

[0032] In order to analyze the gene structure of the multimodal graph of tissue genes based on convolutional neural networks, thereby generating a tissue gene representation graph with higher recognition, the current execution end first obtains the gene representation sample data. At this time, the gene representation sample data includes node sample data and edge sample data, which can be obtained from the spatial transcriptomics SRT database. The SRT database includes STOmics (Spatiotemporal Omics Database), SOAR, SpatialDB (Spatial Transcriptomics Resource), CROST (Comprehensive Repository of Spatial Transcriptomics, spatial transcriptomics database), 10xGenomics website and gene Census database and other data, for example, including 96,700,729 cells / points from 7,367 tissue sections, covering 365 tissue cells, including lungs, skin, brain, liver, kidneys, spinal cord and embryos, and covering normal tissues and various diseases, such as pancreatic ductal adenocarcinoma, amyotrophic lateral sclerosis, non-small cell lung cancer and hepatocellular carcinoma, which are not specifically limited in the embodiments of the invention. At the same time, a spatial dynamic neural network is constructed to integrate the output layer based on the attention mechanism. The spatial dynamic neural network is constructed based on the convolutional neural network. At this time, the spatial dynamic neural network includes a node layer and an edge layer, and the next-level input samples of the node layer and the edge layer include the node sample data and edge sample data output by the previous level, so that the spatial dynamic neural network is model trained based on the gene representation sample data to obtain a spatial dynamic neural network with completed model training.

[0033] It should be noted that the constructed spatial dynamic neural network includes a node layer and an edge layer. The node sample data and the edge sample data are convolved as the node layer and the edge layer respectively, and output through the output layer. At this time, an attention mechanism is embedded in the output layer to integrate the output of the node layer and the output of the edge layer. The embodiments of the present application do not make specific limitations.

[0034] In another embodiment of the present application, for further definition and explanation, before the step of obtaining the gene characterization sample data, the method further includes: Obtaining the original gene expression matrix and adjacency matrix of the tissue genes; Performing enhancement processing on the original gene expression matrix through the adjacency matrix to obtain an enhanced expression matrix, and performing principal component analysis on the enhanced expression matrix to obtain a principal component matrix; Node sample data is constructed based on the adjacency matrix and the principal component matrix.

[0035] Since spatial transcriptomics data lacks adjacency information between gene spots, in order to use the gene edge structure and gene spot structure in multimodal data to analyze the location structure of genes in cells and generate effective representation graphs, the current execution end first obtains the original gene expression matrix and adjacency matrix when obtaining node sample data. It can be directly obtained from the SRT database and the adjacency matrix can be determined Furthermore, the original gene expression matrix is ​​enhanced based on the adjacency matrix to obtain the enhanced expression matrix, which is expressed as ,in, In order to control the influence of spatial neighborhood similarity on the spatial domain, the parameters can be adjusted based on the generation requirements of the representation graph. Build node sample data When the enhanced expression matrix and the adjacency matrix Used to construct node sample data for input to the node layer , , L is the number of samples, where Enhanced Expression Matrix Perform principal component analysis (PCA) and obtain dimensional matrix, n is the node, d m ×N n Expression Matrix The matrix generated after principal component analysis, N n is the number of distinct points, i.e., the number of points in a spatial transcriptome slice.

[0036] In another embodiment of the present application, for further definition and explanation, before the step of obtaining the original gene expression matrix and the adjacency matrix of the tissue gene, the method further includes: Acquiring a spatial omics data set, and determining an original gene expression matrix of genes in different tissues from the spatial omics data set; A similarity matrix is ​​calculated according to the low-dimensional matrix of the original gene expression matrix, and the adjacency matrix is ​​constructed through the similarity matrix.

[0037] In order to enhance the gene expression of each gene locus and thus improve the accuracy of expression map generation, the current execution end generates enhanced spatial expression. Specifically, the adjacency matrix A is first created. At this time, the original gene expression matrix X0 of genes in different tissues is determined from the spatial omics data set, and the original gene expression matrix X0 is PCA-reduced to obtain a low-dimensional feature matrix U. , and then, the similarity matrix D is calculated according to the low-dimensional matrix U of the original gene expression matrix, and the adjacency matrix A is constructed by the similarity matrix, which is expressed as: ; ; in, , i and j are different gene sites in the spatial transcriptome slice, which is the adjacency matrix obtained by normalizing D.

[0038] In another embodiment of the present application, for further definition and explanation, before the step of obtaining the gene characterization sample data, the method further includes: Calculating the total communication intensity of genes in different tissues based on the ligand receptor expression levels, and determining an edge feature matrix based on the total communication intensity; Determine an enhanced edge point matrix based on the edge feature matrix and the point connection state; The edge sample data is constructed based on the gene site communication weight and the enhanced site matrix.

[0039] In order to use intercellular communication as the basis for identifying the cell gene structure and thus improve the accuracy of constructing the cell gene representation map, the current execution end calculates the total communication intensity of genes in different tissues based on the expression level of ligand receptors, and determines the edge feature matrix based on the total communication intensity, which can be expressed as ,in, is the expression level of ligand L at site i, is the expression level of receptor R at site j, and the product is the total communication intensity . Then, the edge feature matrix is ​​determined based on the total communication strength. At this time, the elements in the edge feature matrix B are It represents the total communication strength between site i and site j, which can be obtained by summing the total mutual communication strength of all ligand i-receptor j, expressed as In the graph representing the edge structure, the enhanced edge point matrix can be determined based on the edge feature matrix B and the point connection state. , , , e is the edge graph identifier (edge), and the point connection state is determined by whether the two edges in the edge feature matrix B are connected to the same node, that is, it is determined by inference through COMMOT, and the embodiment of the present application does not make specific limitations.

[0040] It should be noted that after obtaining the enhanced edge site matrix, the edge sample data is constructed based on the gene site communication weight and the enhanced site matrix, wherein the gene site communication weight is composed of the Pearson correlation and cosine similarity between sites calculated based on the contribution of the ligand-receptor complex. , expressed as Among them, the Pearson correlation between points is expressed as , is the covariance of the eigenvectors of point i and point j, Represent the standard deviation of gene expression at point i and point j respectively. Cosine similarity can be expressed as , It is expressed as the dot product of the feature vectors of point i and point j, are the eigenvector norms of site i and site j respectively. Combining the Pearson correlation and cosine similarity between sites, we can get the eigenvector of the initial edge, i.e., the gene site communication weight , to construct edge sample data .

[0041] In another embodiment of the present application, for further definition and explanation, the steps of performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes the model training, and performing convolution processing on the cell communication pair based on the edge layer in the spatial dynamic neural network include: Determining a node adjacency matrix based on the enhanced expression features corresponding to the spatial node graph data, and determining a normalized adjacency matrix based on the enhanced point feature matrix of the cell communication pairs; Determine the output representation of the node at the previous level of the node layer by performing operations on the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the first learnable weight matrix; Determine the edge output representation of the previous level of the edge layer by performing operations on the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the second learnable weight matrix; Determine the node output representation of the previous layer and the edge output representation of the previous layer as input representations of the next node layer and the next edge layer respectively, wherein the input representations include the edge feature matrix of the next node layer and the enhanced expression features of the next edge layer; After completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained.

[0042] After the model training is completed, in order to generate a more accurate structural representation diagram between gene sites, in the application scenario, the current execution end obtains the spatial node graph data and the cell communication pair, and then generates an enhanced expression feature based on the spatial node graph data. Determine the node adjacency matrix , and based on the enhanced point feature matrix of the cell communication pair Determine the normalized adjacency matrix Among them, the node adjacency matrix Expressed as , for The diagonal matrix of is the unit matrix, A is the enhanced expression feature The adjacency matrix of Represents the degree of connection between node i and node j. Normalized adjacency matrix Expressed as ,in, for The diagonal matrix of is the identity matrix, It should be noted that when performing convolution processing in the node layer, the input is Specifically, through the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, and the binary conversion matrix And the first learnable weight matrix is ​​operated to determine the output representation of the node in the previous layer of the node layer. The (l+1)th layer convolution propagation rule is expressed as , , ,get , ,in, ,element Is node i connected to edge v? At this time, since each edge in an undirected graph (or simple graph) connects exactly two different nodes, for the binary transformation matrix representing the "node-edge" relationship, the rows of the matrix correspond to nodes and the columns correspond to edges. Among them, when and only when a node is connected to an edge (that is, the node is an endpoint of this edge), the corresponding matrix element takes the value of 1, otherwise it is 0. At the same time, since each edge has only two endpoints, in the column corresponding to that edge, only the matrix elements corresponding to the rows where the two endpoints are located will be 1, and the rest of the rows will be, so there are exactly two values ​​1 in each column, corresponding to the two nodes connected by the edge. represents the edge feature matrix of layer l, is the first learnable weight matrix, is the node layer learnable weight matrix, represents the Hadamard product, which is used to integrate edge information into the node layer input. is the influence parameter of the edge control node, Represents the activation function.

[0043] When performing convolution in the edge layer, the input is Specifically, the edge output representation of the previous level of the edge layer is determined by calculating the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix, and the second learnable weight matrix. The (l+1)th layer convolution propagation rule is expressed as , ,in, is the second learnable weight matrix, is the learnable weight matrix of the edge layer, and T is the transpose, so that the data in the edge layer can update the parameters by propagating information from the node layer, ensuring the bidirectional integration of nodes and edges.

[0044] It should be noted that in each layer of convolution processing, Figure 2 As shown in the figure, the node output representation of the previous layer and the edge output representation of the previous layer can be determined as the input representation of the next node layer and the next edge layer respectively. At this time, the input representation includes the edge feature matrix of the next node layer and the enhanced expression feature of the next edge layer, so that after completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained. Specifically, the edge feature matrix in the final output layer and enhanced expression characteristics To integrate, characterize as H, first, the edge output can be decomposed by SVD Converted to a graph structure and unified to the node dimension, represented as , where for each point i node layer output and edge layer output Using shared attention vectors Calculate the attention value: ,in, is the weight matrix, is the bias vector and the attention weights are represented as a diagonal matrix: and Finally, the output of the node layer and the output of the edge layer are integrated in the output layer to obtain the joint representation H. , and construct a neighboring network Z for the joint representation H to obtain the output representation G (H, Z). In addition, in order to optimize the graph representation of each point, a denoised expression spectrum can also be generated, that is, This optimization enhances the clarity of spatial expression patterns and improves domain specificity. Ultimately, the resulting denoised data can be used to identify biologically meaningful domain-specific differentially expressed genes.

[0045] In another embodiment of the present application, for further definition and explanation, the steps further include: After obtaining the representation graph, determining the allocation probability of different clusters of each gene locus, and determining the allocation probability distribution; The target distribution is obtained by normalizing the clustering frequency and the corresponding clustering probability of the target cluster; The assignment probability distribution and the target distribution are subjected to divergence minimization processing to determine a clustering loss, so as to cluster the gene sites in the representation graph based on the clustering loss.

[0046] In order to make the representation graph of tissue genes more adaptable in the gene clustering scenario and improve the effectiveness of spatial clustering, the current execution end determines the allocation probability of different clusters of each gene point after obtaining the representation graph, and determines the distribution of the allocation probability. Specifically, based on the joint representation H, the Student's distribution is used to assign the probability of each gene point belonging to a specific cluster j, and the allocation probability ,in, is the center of the jth cluster determined by K-means clustering, is the i-th column of the joint representation H, is the degrees of freedom of the Student t distribution, That is, the distribution probability of all points. Then, based on the clustering frequency of the target cluster and the corresponding clustering probability, normalization is performed to obtain the target distribution, which is expressed as ,in, is the soft cluster frequency of cluster j, and the target distribution P emphasizes high confidence assignments by centering all data points to their respective cluster centers.

[0047] It should be noted that in order to align the distributions Q and P, the Kullback-Leibler (KL) divergence between Q and P can be minimized, thereby enhancing the confidence of the clustering assignment. The clustering loss is defined as At this time, the target distribution P can be iteratively supervised based on the clustering contrast mechanism to update the probability distribution Q, so that it becomes more cohesive within the cluster.

[0048] The present application provides a spatiotemporal omics image processing method based on spatial dynamic neural networks. By constructing a dynamic graph convolutional network, the dynamic relationship of cell-cell communication is combined with gene expression data to achieve dynamic update of node and edge outputs, more effectively revealing spatial functional areas, and significantly improving the ability of spatial domain detection and expression denoising, enhancing the effect of parsing the spatial structure of genes from the perspective of cell-to-cell interactions, greatly improving the accuracy of combining cell information in the context of tissues, and accurately generating effective tissue gene representation maps.

[0049] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the embodiment of the present application provides a spatiotemporal omics image processing device based on a spatial dynamic neural network, such as Figure 3 As shown, the device comprises: An acquisition module 21, for acquiring spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; A processing module 22 is used to perform convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes model training, and to perform convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; The generation module 23 is used to integrate the output representations of the node layer and the edge layer through an attention mechanism to generate a representation graph of the tissue gene.

[0050] Furthermore, the device also includes: a construction module, a training module, The acquisition module is further used to acquire gene representation sample data, wherein the gene representation sample data includes node sample data and edge sample data; The construction module is further used to construct a spatial dynamic neural network that integrates the output layer based on an attention mechanism, wherein the spatial dynamic neural network includes a node layer and an edge layer, and the next-level input samples of the node layer and the edge layer include the node sample data and the edge sample data output by the previous level; The training module is also used to perform model training on the spatial dynamic neural network based on the gene representation sample data to obtain a spatial dynamic neural network that has completed model training.

[0051] Further, The acquisition module is also used to acquire the original gene expression matrix and adjacency matrix of the tissue genes; The processing module is further used to perform enhancement processing on the original gene expression matrix through the adjacency matrix to obtain an enhanced expression matrix, and perform principal component analysis on the enhanced expression matrix to obtain a principal component matrix; The construction module is also used to construct node sample data based on the adjacency matrix and the principal component matrix.

[0052] Furthermore, the device also includes: The determination module is also used to obtain a spatial omics data set and determine the original gene expression matrix of genes in different tissues from the spatial omics data set; calculate the similarity matrix according to the low-dimensional matrix of the original gene expression matrix, and construct the adjacency matrix through the similarity matrix.

[0053] Further, The determination module is further used to calculate the total communication intensity of genes in different tissues based on the ligand receptor expression level, and determine the edge feature matrix based on the total communication intensity; The determination module is further used to determine an enhanced edge point matrix based on the edge feature matrix and the point connection state; The construction module is also used to construct edge sample data based on the gene site communication weight and the enhanced site matrix.

[0054] Further, The processing module is specifically used to determine the node adjacency matrix based on the enhanced expression features corresponding to the spatial node graph data, and determine the normalized adjacency matrix based on the enhanced point feature matrix of the cell communication pair; determine the node output representation of the previous level of the node layer by performing operations on the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression features, the binary transformation matrix and the first learnable weight matrix; determine the edge output representation of the previous level of the edge layer by performing operations on the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression features, the binary transformation matrix and the second learnable weight matrix; determine the node output representation of the previous level and the edge output representation of the previous level as the input representation of the next node layer and the next edge layer respectively, and the input representation includes the edge feature matrix of the next node layer and the enhanced expression features of the next edge layer; after completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained.

[0055] Further, The processing module is also used to determine the allocation probability of different clusters of each gene point after obtaining the representation map, and determine the allocation probability distribution; perform normalization processing based on the clustering frequency of the target cluster and the corresponding clustering probability to obtain the target distribution; perform divergence minimization processing on the allocation probability distribution and the target distribution to determine the clustering loss, so as to cluster the gene points in the representation map based on the clustering loss.

[0056] The present application provides a spatiotemporal omics image processing device based on a spatial dynamic neural network. By constructing a dynamic graph convolutional network, the dynamic relationship of cell-cell communication is combined with gene expression data to achieve dynamic update of node and edge outputs, more effectively revealing spatial functional areas, and significantly improving the ability of spatial domain detection and expression denoising, enhancing the effect of parsing the spatial structure of genes from the perspective of cell-to-cell interactions, greatly improving the accuracy of combining cell information in the tissue context, and accurately generating effective tissue gene representation maps.

[0057] According to one embodiment of the present application, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the spatiotemporal omics image processing method based on spatial dynamic neural network in any of the above method embodiments.

[0058] Figure 4 A schematic diagram of the structure of a terminal provided according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the terminal.

[0059] like Figure 4 As shown, the terminal may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0060] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0061] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0062] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above-mentioned embodiment of the spatiotemporal omics image processing method based on spatial dynamic neural network.

[0063] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0064] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0065] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0066] The program 310 may be specifically configured to enable the processor 302 to perform the following operations: Obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; The spatial node graph data is convolved based on the node layer in the spatial dynamic neural network that has completed model training, and the cell communication pairs are convolved based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; The output representations of the node layer and the edge layer are integrated through an attention mechanism to generate a representation graph of the tissue gene.

[0067] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0068] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A spatiotemporal omics image processing method based on spatial dynamic neural network, characterized in that: include: Obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; The spatial node graph data is convolved based on the node layer in the spatial dynamic neural network that has completed model training, and the cell communication pairs are convolved based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; The output representations of the node layer and the edge layer are integrated through an attention mechanism to generate a representation graph of the tissue gene.

2. The method according to claim 1, characterized in that Before performing convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that has completed model training, and performing convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, the method further includes: Acquire gene representation sample data, wherein the gene representation sample data includes node sample data and edge sample data; Constructing a spatial dynamic neural network that integrates the output layer based on an attention mechanism, wherein the spatial dynamic neural network includes a node layer and an edge layer, and the next-level input samples of the node layer and the edge layer include the node sample data and the edge sample data output by the previous level; The spatial dynamic neural network is model trained based on the gene representation sample data to obtain a spatial dynamic neural network that has completed model training.

3. The method according to claim 2, characterized in that Before obtaining the gene characterization sample data, the method further includes: Obtaining the original gene expression matrix and adjacency matrix of the tissue genes; Performing enhancement processing on the original gene expression matrix through the adjacency matrix to obtain an enhanced expression matrix, and performing principal component analysis on the enhanced expression matrix to obtain a principal component matrix; Node sample data is constructed based on the adjacency matrix and the principal component matrix.

4. The method according to claim 3, characterized in that Before obtaining the original gene expression matrix and adjacency matrix of the tissue genes, the method further comprises: Acquiring a spatial omics data set, and determining an original gene expression matrix of genes in different tissues from the spatial omics data set; A similarity matrix is ​​calculated according to the low-dimensional matrix of the original gene expression matrix, and the adjacency matrix is ​​constructed through the similarity matrix.

5. The method according to claim 2, characterized in that: Before obtaining the gene characterization sample data, the method further includes: Calculating the total communication intensity of genes in different tissues based on the ligand receptor expression levels, and determining an edge feature matrix based on the total communication intensity; Determine an enhanced edge point matrix based on the edge feature matrix and the point connection state; The edge sample data is constructed based on the gene site communication weight and the enhanced site matrix.

6. The method according to claim 1, characterized in that The convolution processing of the spatial node graph data based on the node layer in the spatial dynamic neural network that has completed model training, and the convolution processing of the cell communication pairs based on the edge layer in the spatial dynamic neural network include: Determining a node adjacency matrix based on the enhanced expression features corresponding to the spatial node graph data, and determining a normalized adjacency matrix based on the enhanced point feature matrix of the cell communication pairs; Determine the output representation of the node at the previous level of the node layer by performing operations on the node adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the first learnable weight matrix; Determine the edge output representation of the previous level of the edge layer by performing operations on the normalized adjacency matrix, the edge feature matrix in the cell communication pair, the enhanced expression feature, the binary conversion matrix and the second learnable weight matrix; Determine the node output representation of the previous layer and the edge output representation of the previous layer as input representations of the next node layer and the next edge layer respectively, wherein the input representations include the edge feature matrix of the next node layer and the enhanced expression features of the next edge layer; After completing the convolution processing of all node layers and edge layers, the output representation of the output layer is obtained.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: After obtaining the representation graph, determining the allocation probability of different clusters of each gene locus, and determining the allocation probability distribution; The target distribution is obtained by normalizing the clustering frequency and the corresponding clustering probability of the target cluster; The assignment probability distribution and the target distribution are subjected to divergence minimization processing to determine a clustering loss, so as to cluster the gene sites in the representation graph based on the clustering loss.

8. A spatiotemporal omics image processing device based on spatial dynamic neural network, characterized in that: include: An acquisition module, used to obtain spatial node graph data of tissue genes and intercellular communication pairs based on spatiotemporal omics images; A processing module, configured to perform convolution processing on the spatial node graph data based on the node layer in the spatial dynamic neural network that completes model training, and to perform convolution processing on the cell communication pairs based on the edge layer in the spatial dynamic neural network, wherein the propagation between the node graph after the convolution processing of the node layer and the edge graph after the convolution processing of the edge layer is updated alternately; A generation module is used to integrate the output representations of the node layer and the edge layer through an attention mechanism to generate a representation graph of the tissue gene.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

Images