Spatial transcriptomic data annotation method and device based on graph convolutional neural network

Abstract

The application discloses a kind of based on graph convolutional neural network's spatial transcriptome data annotation method and device, comprising: obtaining the spatial transcriptome data of cancer boundary zone slice to be annotated and standardization, extract high variation gene in the spatial transcriptome data after standardization as feature expression matrix;Based on the position information between the regions of spatial transcriptome slice and feature expression matrix, an initial graph structure is constructed;According to each region of spatial transcriptome slice, H&E image is cut, and the corresponding H&E image block is obtained;The corresponding H&E image block is input into image feature extraction module, and the corresponding image feature is extracted, and the corresponding image feature is integrated into initial graph structure, to obtain final graph structure;Final graph structure is input into trained spatial transcriptome data annotation model, and the annotation result of each region in the spatial transcriptome data of cancer boundary zone slice to be annotated is obtained.The application can effectively improve the accuracy of prediction annotation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of spatial transcriptomic data annotation, and in particular to a spatial transcriptomic data annotation method and device based on a graph convolutional neural network. BACKGROUND

[0002] Spatial transcriptomics technology is a comprehensive analysis of the transcriptome in tissue samples through high-throughput sequencing, and can retain the spatial location information of cells in the tissue. Unlike traditional transcriptomic techniques, spatial transcriptomics not only reveals the overall profile of gene expression, but also accurately tracks the changes in gene expression in different tissue regions, cell types, and microenvironments. This provides an unprecedented perspective for studying the interactions between cells, the relationship between tissue structure and function, and the occurrence and development of diseases.

[0003] In particular, in the fields of oncology, immunology, neuroscience, etc., the spatial distribution and relative position of different cells in the tissue are crucial for understanding the mechanisms of disease occurrence, progression, and treatment effects. Spatial information can reveal how cells respond to external stimuli in a specific tissue environment, how they communicate with surrounding cells, and even how they regulate gene expression in a specific microenvironment. Therefore, spatial transcriptomics technology provides new tools and methods for early diagnosis, precise treatment, and pathological research of diseases, and helps to better understand the behavior of cells in their microenvironment and lay the foundation for the development of precision medicine.

[0004] In the study of spatial transcriptomics, the most important step is to annotate the obtained spatial domain. The main steps are divided into two steps: identifying spatially variable genes that show spatial expression patterns and detecting spatial domains with consistent gene expression. Annotating spatial domains can better understand tissue structure and functional partitioning and explore disease progression.

[0005] Currently, the annotation of spatial domains mainly uses unsupervised clustering methods such as SpaGCN. Unsupervised clustering methods require the spatial domain to be segmented first, and then the spatially variable genes are identified to annotate the spatial domain.

[0006] The above prior art has the following defects and obstacles:

[0007] (1) Since the identified spatially variable genes mostly use statistical modeling-based methods to statistically analyze the significance of genes, the spatial distribution of gene expression is measured. This makes it difficult to explain the spatial distribution characteristics of genes. And for these unsupervised methods, the segmentation of spatial domains and the identification of spatially variable genes are treated as two separate tasks, while there is a close relationship between spatially variable genes and spatial domains in biology.

[0008] (2) Current annotation methods do not fully utilize all the information of spatial transcriptome data. Spatial transcriptome sequencing data contains spatial location information of tissue regions, gene expression and corresponding image information. Since the corresponding image information in the current database is mainly low-resolution image information, and is limited by the resolution of spatial transcriptome sequencing technology, the published annotation methods mainly use gene information and ignore image information. Some emerging methods such as SpaGCN only consider the color information of the image. For methods that use image information for identification, the resolution of the image is a decisive factor for whether the annotation can be performed, and the identification effect is not good for low-resolution images. SUMMARY

[0009] The purpose of the present application is to propose a spatial transcriptome data annotation method and device based on a graph convolutional neural network for the above-mentioned technical problems.

[0010] In a first aspect, the present application provides a spatial transcriptome data annotation method based on a graph convolutional neural network, comprising the following steps:

[0011] Obtain the spatial transcriptome data of the cancer boundary section to be annotated and standardize it to obtain the standardized spatial transcriptome data, extract the high-variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression amounts of one gene in one region of the spatial transcriptome section;

[0012] Construct an initial graph structure based on the position information between the regions of the spatial transcriptome section and the feature expression matrix; construct an image feature extraction module based on a pre-trained deep neural network, obtain the H&E image corresponding to the spatial transcriptome data of the cancer boundary section to be annotated, and cut it according to each region of the spatial transcriptome section to obtain the H&E image block corresponding to each region of the spatial transcriptome section; input the H&E image block corresponding to each region of the spatial transcriptome section into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome section, and integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome section into the initial graph structure to obtain a final graph structure;

[0013] Construct a spatial transcriptome data annotation model based on a graph convolutional neural network model, and perform pan-domain training on the spatial transcriptome data annotation model to obtain a trained spatial transcriptome data annotation model;

[0014] Input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation results of each region in the spatial transcriptome data of the cancer boundary section to be annotated.

[0015] Preferably, the feature expression matrix is represented as:

[0016]

[0017] wherein express_matrix is the feature expression matrix, E ij represents the expression amount of the jth gene of the ith region of the spatial transcriptome slice, i∈{1,2,…,m}, j∈{1,2,…,n}.

[0018] As preferred, the initial graph structure is constructed based on the positional information between the regions of the spatial transcriptome slice and the feature expression matrix, specifically comprising:

[0019] Each region of the spatial transcriptome slice is taken as a node in the initial graph structure, and the Euclidean distance is used to calculate the distance between each two regions of the spatial transcriptome slice;

[0020] It is judged whether the distance between two regions of the spatial transcriptome slice exceeds the distance threshold, if yes, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, and thus the adjacency matrix of the initial graph structure is constructed.

[0021] As preferred, the node in the initial graph structure stores the gene expression feature of the corresponding region in the spatial transcriptome slice, and the gene expression feature of the corresponding region in the spatial transcriptome slice is the row vector of the corresponding region in the feature expression matrix; the node in the final graph structure stores the gene expression feature of the corresponding region in the spatial transcriptome slice and the image feature of the H&E image block of the corresponding region.

[0022] As preferred, the pre-trained deep neural network comprises ResNet50, and the graph convolutional neural network model comprises an input layer, a hidden layer and an output layer, and the calculation process of the input layer is as follows:

[0023]

[0024] wherein X represents the feature obtained by splicing the gene expression feature and the image feature stored on each node, W1 represents the weight matrix of the input layer, is the normalized adjacency matrix, ReLU is the relu activation function, and Z1 represents the output feature of the input layer;

[0025] The calculation process of the hidden layer is as follows:

[0026]

[0027] wherein Dropout represents the dropout operation, W2 represents the weight matrix of the hidden layer, and Z2 represents the output feature of the hidden layer;

[0028] The calculation process of the output layer is as follows:

[0029]

[0030] wherein Softmax represents a Softmax function, W3 represents a weight matrix of the output layer, and Z3 represents a prediction score;

[0031] The output layer uses two neurons as two classification heads, and therefore the prediction score is a two-dimensional vector

[0032] Z3 = [Score normal , Score cancer ], wherein Score normal represents a probability score of a normal cell, and Score cancer represents a probability score of a malignant cell, and the annotation result is output using the following formula:

[0033] Class prediction = argmax (Score normal , Score cancer );

[0034] wherein argmax represents a maximum value, and Class prediction represents an annotation result, which is 0 or 1, 1 indicating a malignant cell and 0 indicating a normal cell.

[0035] As a preferred, the spatial transcriptome data annotation model is pan-domain trained to obtain a trained spatial transcriptome data annotation model, specifically comprising:

[0036] Collect spatial transcriptome data and corresponding true labels and H&E images from different tissues and different distributions, and divide them into different domain training sets according to different cancer types;

[0037] Standardize the spatial transcriptome data of each spatial transcriptome slice in the training set of each domain, extract the corresponding feature expression matrix and construct an initial graph structure; cut the H&E image of each spatial transcriptome slice in the training set of each domain according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice, and input the H&E image block corresponding to each region of the spatial transcriptome slice into an image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice;

[0038] Integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the corresponding node in the initial graph structure for storage to obtain a final graph structure, input the final graph structure into the spatial transcriptome data annotation model, and obtain a predicted label;

[0039] The malignant ratio in each spatial transcriptome slice is calculated based on the true label of each spatial transcriptome slice in the training set of each domain, as shown in the following formula:

[0040]

[0041] wherein, malignant ratio represents the malignant ratio, spot malignant represents the number of regions labeled as malignant cells in the true label of each spatial transcriptome slice, and sum(spot) represents the number of all regions of each spatial transcriptome slice;

[0042] If the malignant ratio is less than or equal to the ratio threshold, the focal loss function is calculated based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain, otherwise the cross-entropy loss function is calculated based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain;

[0043] The average loss value of all spatial transcriptome slices in the training set of each domain is calculated, as shown in the following formula:

[0044]

[0045] wherein, Loss_domain[h] represents the loss value of the hth spatial transcriptome slice in the training set of one domain, and the total number of spatial transcriptome slices in the training set of the domain is H;

[0046] The penalty item penalty is calculated based on the average loss value, as shown in the following formula:

[0047]

[0048] The total loss function Total_loss of each domain is calculated using the following formula:

[0049] Total loss = μ + penalty_weight * penalty;

[0050] Wherein, penalty_weight represents the penalty item weight;

[0051] The spatial transcriptome data annotation model is trained in a random gradient descent manner and based on the total loss function on each domain to obtain a trained spatial transcriptome data annotation model.

[0052] In a second aspect, the present application provides a spatial transcriptome data annotation device based on a graph convolutional neural network, comprising:

[0053] a data processing module configured to obtain spatial transcriptome data of a cancer border section to be annotated and perform standardization to obtain standardized spatial transcriptome data, extract high-variable genes in the standardized spatial transcriptome data as a feature expression matrix, and elements in the feature expression matrix being expression amounts of one gene in one region of the spatial transcriptome section;

[0054] a graph structure construction module configured to construct an initial graph structure based on position information between regions of the spatial transcriptome section and the feature expression matrix, construct a graph image feature extraction module based on a pre-trained deep neural network, obtain an H&E image corresponding to the spatial transcriptome data of the cancer border section to be annotated and cut the H&E image according to each region of the spatial transcriptome section to obtain an H&E image block corresponding to each region of the spatial transcriptome section, input the H&E image block corresponding to each region of the spatial transcriptome section into the graph image feature extraction module to extract image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome section, and integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome section into the initial graph structure to obtain a final graph structure;

[0055] a model construction module configured to construct a spatial transcriptome data annotation model based on a graph convolutional neural network model, perform pan-domain training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model;

[0056] a prediction module configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain annotation results of each region in the spatial transcriptome data of the cancer border section to be annotated.

[0057] In a third aspect, the present application provides an electronic device, including one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementation manners of the first aspect.

[0058] In a fourth aspect, the present application provides a computer-readable storage medium, having a computer program stored thereon, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0059] In a fifth aspect, the present application provides a computer program product, including a computer program, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0060] Compared with the prior art, the present application has the following beneficial effects:

[0061] (1) The spatial transcriptome data annotation method based on the graph convolutional neural network uses a graph structure to represent spatial transcriptome data, models the gene expression of spots as the features of graph nodes, and models the direct spatial distance of spots as the distance of the edges of the graph structure. According to the position of the spot, the H&E image is segmented, the features are extracted using a deep neural network, and are integrated with the gene features, so that the gene expression, position information and image information in the spatial transcriptome data can be fully utilized, the prediction accuracy of the model can be improved, and the occurrence of irregular discrete spots can be effectively reduced. Compared with the existing method, the present method reuses the spatial position information and image information of the spot, and has higher annotation accuracy for malignant regions in the spatial transcriptome data.

[0062] (2) The spatial transcriptome data annotation method based on the graph convolutional neural network uses a supervised graph convolutional neural network model for training, realizes an end-to-end spatial domain prediction effect, and the supervised training method improves the consistency of the spatial domain recognition process and can explore the biological interpretability of the result. The main spatial domain recognition method at present is an unsupervised method, which lacks consistency in the process of dividing the spatial domain and recognizing the variable genes.

[0063] (3) The spatial transcriptome data annotation method based on the graph convolutional neural network performs pan-domain training on the graph convolutional neural network model, can be applied to the annotation of spatial transcriptome data of different types of cancers, and can realize high model prediction accuracy on a small training data set. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Figure 1 The flowchart of the spatial transcriptome data annotation method based on the graph convolutional neural network of the embodiments of the present application is shown in the figure.

[0066] Figure 2 The flowchart of the spatial transcriptome data annotation method based on the graph convolutional neural network of the embodiments of the present application is shown in the figure.

[0067] Figure 3 The feasibility verification diagram of the spatial transcriptome data annotation method based on the graph convolutional neural network of the embodiments of the present application is shown in the figure, wherein (a) represents the construction of a graph structure using only gene information, (b) represents the construction of a graph structure integrating position information and gene information, and (c) represents a labeled region.

[0068] Figure 4 The visualization result figures of the spatial transcriptome data annotation method based on the graph convolutional neural network of the embodiments of the present application are for different kinds of cancer boundary section slices, wherein the left side in each box is the annotation area of the pathologist, the right side is the model annotation result, the red area is the area of malignant cells, and the yellow area is the area of normal cells, (a) represents breast cancer, (b) represents colorectal cancer, (c) represents renal clear cell carcinoma, and (d) represents hepatocellular carcinoma.

[0069] Figure 5 The result figure of the prediction accuracy of the spatial transcriptome data annotation method based on the graph convolutional neural network of the embodiments of the present application;

[0070] Figure 6 The schematic diagram of the spatial transcriptome data annotation device based on the graph convolutional neural network of the embodiments of the present application;

[0071] Figure 7 The hardware structure schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0072] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0073] Figure 1 A spatial transcriptome data annotation method based on a graph convolutional neural network is shown, which comprises the following steps:

[0074] S1, obtaining spatial transcriptome data of a cancer boundary section slice to be annotated and performing standardization to obtain standardized spatial transcriptome data, extracting high-variable genes in the standardized spatial transcriptome data as a feature expression matrix, and elements in the feature expression matrix being expression amounts of a gene in a region of a spatial transcriptome slice.

[0075] In specific embodiments, the feature expression matrix is expressed as:

[0076]

[0077] wherein express_matrix is the feature expression matrix, E ij represents the expression amount of the jth gene in the ith region of the spatial transcriptome slice, i∈{1,2,…,m}, j∈{1,2,…,n}.

[0078] In particular, with reference to Figure 2 Embodiments of the present application first normalize the spatial transcriptome data and extract high-variable genes as a feature expression matrix. Each element in the feature expression matrix corresponds to the expression of one gene in one region (spot) of the spatial transcriptome slice. The i-th row in the feature expression matrix corresponds to the expression of n genes in the i-th spot. The feature expression matrix extracts the gene expression information and location information in the spatial transcriptome data.

[0079] S2, based on the location information between the regions of the spatial transcriptome slice and the feature expression matrix, an initial graph structure is constructed; an image feature extraction module based on a pre-trained deep neural network is constructed, an H&E image corresponding to the spatial transcriptome data of the border slice of the cancer to be annotated is obtained, and the H&E image is cut according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, and the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice are integrated into the initial graph structure to obtain a final graph structure.

[0080] In specific embodiments, the initial graph structure is constructed based on the location information between the regions of the spatial transcriptome slice and the feature expression matrix, and specifically includes:

[0081] Each region of the spatial transcriptome slice is used as a node in the initial graph structure, and the Euclidean distance is used to calculate the distance between each two regions of the spatial transcriptome slice;

[0082] It is determined whether the distance between two regions of the spatial transcriptome slice exceeds a distance threshold. If yes, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, and an adjacency matrix of the initial graph structure is finally constructed.

[0083] In specific embodiments, the nodes in the initial graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slice, and the gene expression features of the corresponding regions in the spatial transcriptome slice are the row vectors in the feature expression matrix; the nodes in the final graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slice and the image features of the H&E image blocks of the corresponding regions.

[0084] Specifically, the initial graph structure is constructed based on the position information between spots, the Euclidean distance is used to calculate the distance between each two spots, and the distance threshold is set to limit the connectivity between the nodes corresponding to the two spots. The nodes corresponding to the two spots with a distance exceeding the distance threshold are considered not to be connected, which is based on the fact that the information exchange between two spots with a long actual distance is weak. Finally, the elements A ik : wherein, spot i represents the node corresponding to the i-th region, and spot k represents the node corresponding to the k-th region.

[0085] Further, an image feature extraction module based on a pre-trained deep neural network is first constructed, and then the H&E image is preprocessed, mainly including color normalization and contrast enhancement. The gene expression data of the spatial transcriptome slice is aligned with the corresponding H&E image in space, the H&E image corresponding to the spatial transcriptome data is cut, and the gene expression information of each spot is corresponded to the H&E image to obtain each H&E image block. The image features of each H&E image block are extracted using the pre-trained deep neural network ResNet50, the initial graph structure and the image features are integrated, and the corresponding final graph structure is constructed, which contains gene expression features, position information and image features.

[0086] S3, constructing a spatial transcriptome data annotation model based on a graph convolutional neural network model, training the spatial transcriptome data annotation model in a general domain to obtain a trained spatial transcriptome data annotation model.

[0087] In specific embodiments, the pre-trained deep neural network includes ResNet50, the graph convolutional neural network model includes an input layer, a hidden layer and an output layer, and the calculation process of the input layer is as follows:

[0088]

[0089] wherein, X represents the features obtained by splicing the gene expression features and image features stored on each node, W1 represents the weight matrix of the input layer, is the normalized adjacency matrix, ReLU is the relu activation function, and Z1 represents the output features of the input layer.

[0090] The calculation process of the hidden layer is as follows:

[0091]

[0092] wherein Dropout denotes a dropout operation, W2 denotes a weight matrix of the hidden layer, and Z2 denotes an output feature of the hidden layer;

[0093] The calculation process of the output layer is as follows:

[0094]

[0095] wherein Softmax denotes a Softmax function, W3 denotes a weight matrix of the output layer, and Z3 denotes a prediction score;

[0096] The output layer uses two neurons as two classification heads, and thus the prediction score is a two-dimensional vector

[0097] Z3 = [Score normal , Score cancer ], wherein Score normal denotes a probability score of a normal cell, and Score cancer denotes a probability score of a malignant cell, and the annotation result is output by using the following formula:

[0098] Class prediction = argmax (Score normal , Score cancer );

[0099] wherein argmax denotes a maximum value, and Class prediction denotes an annotation result, and the value is 0 or 1, 1 indicating a malignant cell and 0 indicating a normal cell.

[0100] Specifically, the spatial transcriptome data annotation model used by the embodiment of the present application is a three-layer graph convolutional neural network model, which is used to extract features from the final graph structure and perform classification. The model design combines the gene expression information, position information and H&E image information of the node, and learns the complex relationship between different spatial regions through graph convolution operation. The input layer of the model is the final graph structure constructed for each slice, and the final graph structure embeds the position information of the slice by calculating the Euclidean distance between spots, and the node feature is the feature obtained by splicing the gene expression feature and the image feature. Each layer of the model updates the node representation through graph convolution operation, and applies dropout to prevent overfitting.

[0101] In one of the embodiments, in the input layer of the graph convolutional neural network model, the input feature dimension d in = 5000 is mapped to the output of the same dimension, that is, the input dimension is kept, for extracting initial local spatial relationship features. In the hidden layer, 5000 dimensions are mapped to the dimension d hid= 512, and dropout is applied, which randomly discards part of the neurons, and the specific proportion is determined by the dropout rate, to avoid overfitting. In the output layer, the dimension of the hidden layer is further reduced to the dimension d of the output layer out = 2, to generate a prediction score, according to which the annotation result can be determined, i.e., whether the region is a malignant cell or a normal cell region.

[0102] In specific embodiments, the spatial transcriptome data annotation model is trained in a pan-domain manner to obtain a trained spatial transcriptome data annotation model, specifically including:

[0103] Spatial transcriptome data and corresponding true labels and H&E images from different tissues and different distributions are collected, and are divided into different domain training sets according to different cancer types;

[0104] The spatial transcriptome data of each spatial transcriptome slice in the training set of each domain is standardized, and the corresponding feature expression matrix is extracted and an initial graph structure is constructed; the H&E images of each spatial transcriptome slice in the training set of each domain are cut according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice, and the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice;

[0105] The image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice are integrated into the corresponding node in the initial graph structure for storage to obtain a final graph structure, and the final graph structure is input into the spatial transcriptome data annotation model to obtain a predicted label;

[0106] The malignant proportion in each spatial transcriptome slice is calculated based on the true label of each spatial transcriptome slice in the training set of each domain, as shown in the following formula:

[0107]

[0108] wherein, malignant ratio represents the malignant proportion, spot malignant represents the number of regions labeled as malignant cells in the true label of each spatial transcriptome slice, and sum(spot) represents the number of all regions of each spatial transcriptome slice;

[0109] If the proportion of malignancy is less than or equal to the proportion threshold, a focal loss function is calculated according to the predicted label and the true label to obtain a loss value of each spatial transcriptome slice in the training set of each domain, otherwise a cross-entropy loss function is calculated according to the predicted label and the true label to obtain a loss value of each spatial transcriptome slice in the training set of each domain.

[0110] An average loss value of all spatial transcriptome slices in the training set of each domain is calculated, as shown in the following formula:

[0111]

[0112] wherein Loss_domain[h] represents a loss value of an hth spatial transcriptome slice in the training set of one of the domains, and the total number of spatial transcriptome slices in the training set of the domain is H;

[0113] A penalty item penalty is calculated based on the average loss value, as shown in the following formula:

[0114]

[0115] A total loss function Total_loss of each domain is calculated using the following formula:

[0116] Total loss = μ + penalty_weight * penalty.

[0117] wherein penalty_weight represents a penalty item weight;

[0118] The spatial transcriptome data annotation model is trained based on the total loss function of each domain in a manner of stochastic gradient descent to obtain a trained spatial transcriptome data annotation model.

[0119] Specifically, the spatial transcriptome data from different organizations and different distributions is divided into different domains, each kind of cancer is a domain, and the data is labeled according to the true label of the training set. The true label is a malignant cell or a non-malignant cell. The corresponding final graph structure is constructed for the spatial transcriptome data in the training set using the steps of steps S1 and S2. The spatial transcriptome data annotation model uses a three-layer graph convolutional neural network model, and the graph convolutional neural network model is trained through the final graph structure and the true label corresponding to the training set of each domain.

[0120] In one of the embodiments, 16 human spatial transcriptome cancer boundary data are collected in the GEO database, and the data are divided into seven domains according to the cancer type. The annotation provided by the GEO database is used as the true label of the spatial transcriptome data. In the training process, the breast cancer spatial transcriptome data are fixedly selected as the verification set; the data of each domain are traversed in turn as the test set; and the remaining data are used as the training set.

[0121] The high-variable genes are screened using the variance of expression. The gene expression of all normalized spatial transcriptomes is sorted according to the variance difference between the data, and the top 5000 genes are selected as the high-variable genes. The variance sorting method is as follows: the gene expression of the spatial transcriptome slices in the training set (the row name is the gene, and the column name is the spot) is combined according to the gene name, so that the expression data of all spots of a certain gene in each row of the training set can be obtained. The variance of the jth row is calculated as follows: wherein, var j represents the variance of the jth row, x ji represents the expression of the ith column of the jth row, represents the mean value of all expressions of the jth row;

[0122] After the variance of each row is calculated, the variance is sorted from large to small. In the construction of the initial graph structure, the relative spatial position of the spot in the spatial transcriptome is used as the measurement, and by adjusting the distance threshold between different spots, it is found that

[0123] X 2 +Y 2 <=2 2 (X, Y are the horizontal and vertical coordinates of the relative position of the spot) can construct an initial graph structure with simple structure and high efficiency. Further, the image features are integrated to construct the final graph structure.

[0124] During training, the final graph structures of the training sets from different domains are input into the graph convolutional neural network model for classification to obtain predicted labels. First, the malignant ratio in the spatial transcriptome slices is calculated based on the number of regions with malignant cells and the total number of regions in the true labels. For samples where the malignant and non-malignant regions differ significantly (malignant ratio ≤ 0.1), a focal loss function is used to calculate the loss value of the spatial transcriptome slice to reduce losses caused by sample imbalance. For other cases, a cross-entropy loss function is used to calculate the loss value of the spatial transcriptome slice. The graph convolutional neural network model is trained using stochastic gradient descent. The training process aims to reduce the loss differences of the model across multiple domains (i.e., different data distributions, such as different tissue slices or patient data) and enhances the performance of the graph convolutional neural network model in unseen domains through additional penalty terms. Finally, a trained spatial transcriptome data annotation model is obtained. The embodiments of this application perform training and generalization across multiple domains simultaneously, so the resulting trained spatial transcriptome data annotation model can predict a variety of cancers without requiring retraining for a specific cancer.

[0125] To confirm that the final graph structure constructed using the embodiments of this application can effectively improve the model's predictive performance, comparative verification was performed on the embodiments of this application: using only gene information to construct the graph structure, and integrating location information with gene information to construct the graph structure for model training. (Reference) Figure 3 Compared to simply using gene information to construct a graph structure, the graph structure constructed by integrating location information and gene information reconstructs the spatial location of tissue slices, which can effectively improve the predictive performance of the model and reduce the occurrence of discrete malignant tissue outside the malignant region.

[0126] Through comparative verification experiments, this application demonstrates the significant effectiveness of the method for constructing graph structures based on the actual location information of spots in spatial transcriptome slices. Furthermore, to investigate the impact of node feature selection in the graph structure, this application conducted ablation experiments. The results are shown in Table 1 and... Figure 4 As shown, the experiment was divided into three groups: location + image features, location + gene features, and location + image features + gene features. All three groups used graph structures built based on location information, the difference being that different feature types were assigned to the nodes.

[0127] The experimental results show that, among the total of 12 slices, the change in prediction performance of the position + image feature + gene feature group compared with the position + gene feature group is that the performance of 4 slices decreases, the performance of 6 slices increases, and the performance of 2 slices does not change significantly. According to this conclusion, after adding image features, the prediction ability of the model is generally slightly improved. Further analysis found that the results of the position + image feature group revealed that the noise in the image features was the main bottleneck affecting the improvement of the model performance. Therefore, if less noisy image features are used, the prediction effect may be more obviously improved.

[0128] Table 1

[0129]

[0130] S4, inputting the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation result of each region in the spatial transcriptome data of the cancer boundary slice to be annotated.

[0131] Specifically, the trained spatial transcriptome data annotation model is deployed, and in the inference process, for an unknown cancer boundary slice, the final graph structure obtained through the above steps S1 and S2 is input into the trained spatial transcriptome data annotation model to obtain the annotation result of each region. As shown in Figure 5 , the annotation result of the spatial transcriptome data of different cancer types predicted by the trained spatial transcriptome data annotation model of the present application is consistent with the region labeled by the pathologist, which shows that the annotation effect of the trained spatial transcriptome data annotation model of the present application is good.

[0132] Further referring to Figure 6 , as an implementation of the method shown in the above figures, the present application provides an embodiment of a spatial transcriptome data annotation device based on a graph convolutional neural network. The device embodiment corresponds to the method embodiment shown in Figure 1 , and the device can be specifically applied to various electronic devices.

[0133] The present application embodiment provides a spatial transcriptome data annotation device based on a graph convolutional neural network, which comprises:

[0134] The data processing module 1 is configured to obtain spatial transcriptome data of a cancer boundary slice to be annotated and perform standardization to obtain standardized spatial transcriptome data, extract high-variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression amounts of one gene in one region of the spatial transcriptome slice.

[0135] The graph structure construction module 2 is configured to construct an initial graph structure based on position information between regions of a spatial transcriptome slice and a feature expression matrix; construct an image feature extraction module based on a pre-trained deep neural network, acquire an H&E image corresponding to the spatial transcriptome data of the cancer boundary slice to be annotated, and cut the H&E image according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; input the H&E image block corresponding to each region of the spatial transcriptome slice into the image feature extraction module to extract image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the initial graph structure, and obtain a final graph structure;

[0136] The model construction module 3 is configured to construct a spatial transcriptome data annotation model based on a graph convolutional neural network model, perform pan-domain training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model.

[0137] The prediction module 4 is configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain an annotation result of each region in the spatial transcriptome data of the cancer boundary slice to be annotated.

[0138] Figure 7 A hardware structure schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 7. Figure 7 As shown in FIG. 7, the electronic device of the embodiment includes a processor 701 and a memory 702; the memory 702 is used to store computer execution instructions; the processor 701 is used to execute the computer execution instructions stored in the memory to realize each step performed by the electronic device in the above-mentioned embodiment. For details, please refer to the related description in the foregoing method embodiment.

[0139] Optionally, the memory 702 can be independent or integrated with the processor 701.

[0140] When the memory 702 is independently arranged, the electronic device further includes a bus 703 for connecting the memory 702 and the processor 701.

[0141] The embodiment of the present application further provides a computer storage medium, and the computer storage medium stores computer execution instructions; when the processor 701 executes the computer execution instructions, the method as above is realized.

[0142] The embodiment of the present application further provides a computer program product, and the computer program product includes a computer program; when the computer program is executed by the processor 701, the method as above is realized.

[0143] In the embodiments of the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, another division can be used. For example, a plurality of modules can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the modules shown or discussed can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or in other forms.

[0144] The modules described as separate components can or can not be physically separate, and the components shown as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to implement the embodiments of the present application.

[0145] In addition, the functional modules in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware or in the form of hardware plus software functional units.

[0146] The integrated modules realized in the form of software functional modules can be stored in a computer readable storage medium. The software functional modules stored in the storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or the processor 701 to execute part of the steps of the method of each embodiment of the present application.

[0147] It should be understood that the processor 701 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor 701 can be any conventional processor 701, etc. The steps of the method disclosed in the present application can be directly embodied as the execution of the processor 701 in hardware, or executed by a combination of hardware and software modules in the processor 701.

[0148] The memory 702 can include a high-speed RAM memory and can also include a non-volatile storage NVM, for example at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic or optical disk, etc.

[0149] The bus 703 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus 703 in the drawings of the present application does not limit to only one bus 703 or one type of bus 703.

[0150] The storage medium described above can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0151] An exemplary storage medium is coupled to the processor 701, so that the processor 701 can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be an integral part of the processor 701. The processor 701 and the storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor 701 and the storage medium can also exist as discrete components in an electronic device or a host device.

[0152] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction-related hardware. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes various storage media that can store program codes, such as ROM, RAM, magnetic disk or optical disk, etc.

[0153] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A spatial transcriptome data annotation method based on graph convolutional neural networks, characterized in that, Includes the following steps: Spatial transcriptome data of cancer junction slices to be annotated are obtained and standardized to obtain standardized spatial transcriptome data. Hypervariable genes in the standardized spatial transcriptome data are extracted as feature expression matrix. The elements in the feature expression matrix are the expression levels of one gene in one region of the spatial transcriptome slice. An initial graph structure is constructed based on the positional information between regions of the spatial transcriptome slices and the feature expression matrix. An image feature extraction module based on a pre-trained deep neural network is then constructed to acquire H&E images corresponding to the spatial transcriptome data of the cancer border region slices to be annotated, and to cut the spatial transcriptome slices according to each region, obtaining H&E image blocks corresponding to each region of the spatial transcriptome slices. The H&E image blocks corresponding to each region of the spatial transcriptome slices are input into the image feature extraction module to extract the image features corresponding to the H&E image blocks corresponding to each region of the spatial transcriptome slices. The image features corresponding to the H&E image blocks corresponding to each region of the spatial transcriptome slices are then integrated into the initial graph structure to obtain the final graph structure. A spatial transcriptome data annotation model based on a graph convolutional neural network model is constructed, and the spatial transcriptome data annotation model is trained by generalization to obtain a trained spatial transcriptome data annotation model. The final graph structure is input into the trained spatial transcriptome data annotation model to obtain the annotation results for each region in the spatial transcriptome data of the cancer junction slice to be annotated.

2. The spatial transcriptome data annotation method based on graph convolutional neural networks according to claim 1, characterized in that, The feature representation matrix is ​​expressed as follows: Where express_matrix is ​​the feature representation matrix, E ij Let represent the expression level of the j-th gene in the i-th region of a spatial transcriptome slice, where i∈{1,2,…,m} and j∈{1,2,…,n}.

3. The spatial transcriptome data annotation method based on graph convolutional neural networks according to claim 1, characterized in that, An initial graph structure is constructed based on the positional information between regions of the spatial transcriptome slices and the feature expression matrix, specifically including: Each region of the spatial transcriptome slice is used as a node in the initial graph structure, and the distance between every two regions of the spatial transcriptome slice is calculated using Euclidean distance. If the distance between two regions of a spatial transcriptome slice exceeds a distance threshold, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, thus constructing the adjacency matrix of the initial graph structure.

4. The spatial transcriptome data annotation method based on graph convolutional neural networks according to claim 3, characterized in that, The initial graph structure contains gene expression features of the corresponding regions in the node storage space transcriptome slices, where the gene expression features of the corresponding regions are the row vectors of the corresponding regions in the feature expression matrix; the final graph structure contains gene expression features of the corresponding regions in the node storage space transcriptome slices, as well as image features of the corresponding H&E image patches.

5. The spatial transcriptome data annotation method based on graph convolutional neural networks according to claim 4, characterized in that, The pre-trained deep neural network includes ResNet50, and the graph convolutional neural network model includes an input layer, a hidden layer, and an output layer. The calculation process of the input layer is as follows: Where X represents the feature obtained by concatenating the gene expression features and image features stored at each node, and W1 represents the weight matrix of the input layer. Z1 represents the normalized adjacency matrix, ReLU is the ReLU activation function, and Z1 represents the output feature of the input layer. The calculation process for the hidden layer is as follows: Where Dropout represents the dropout operation, W2 represents the weight matrix of the hidden layer, and Z2 represents the output feature of the hidden layer; The calculation process for the output layer is as follows: Where Softmax represents the Softmax function, W3 represents the weight matrix of the output layer, and Z3 represents the prediction score; The output layer uses two neurons as two classification heads, therefore the predicted score is a two-dimensional vector Z3 = [Score]. normal Score cancer ], where Score normal Score represents the probability score of normal cells. cancer The probability fraction of malignant cells is represented by the following formula, and the annotation results are output: Class prediction =argmax(Score normal ,Score cancer ); Where argmax represents taking the maximum value, Class prediction This indicates the annotation result, with a value of 0 or 1, where 1 indicates a malignant cell and 0 indicates a normal cell.

6. The spatial transcriptome data annotation method based on graph convolutional neural networks according to claim 1, characterized in that, The spatial transcriptome data annotation model is subjected to generalization training to obtain a trained spatial transcriptome data annotation model, specifically including: Spatial transcriptome data from different tissues and distributions, along with their corresponding ground truth labels and H&E images, were collected and divided into training sets for different domains according to different cancer types. The spatial transcriptome data of each spatial transcriptome slice in the training set of each domain is standardized, and the corresponding feature expression matrix is ​​extracted and an initial graph structure is constructed. The H&E image of each spatial transcriptome slice in the training set of each domain is cut according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice. The H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice. The image features corresponding to the H&E image blocks of each region of the spatial transcriptome slice are integrated into the corresponding nodes in the initial graph structure for storage to obtain the final graph structure. The final graph structure is then input into the spatial transcriptome data annotation model to obtain the predicted labels. The malignancy rate in each spatial transcriptome slice is calculated based on the true labels of each spatial transcriptome slice in the training set for each domain, as shown in the following formula: Wherein, malignant ratio represents the proportion of malignant cases, spot malignant This represents the number of regions labeled as malignant cells in the true labels of each spatial transcriptome slice, and sum(spot) represents the total number of regions in each spatial transcriptome slice. Determine whether the malignancy ratio is less than or equal to the ratio threshold. If so, calculate the focus loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain. Otherwise, calculate the cross-entropy loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain. The average loss value of all spatial transcriptome slices in the training set for each domain is calculated as follows: Where Loss_domain[h] represents the loss value of the h-th spatial transcriptome slice in the training set of one of the domains, and the total number of spatial transcriptome slices in the training set of this domain is H; The penalty term is calculated based on the average loss value, as shown in the following formula: The total loss function, Total_loss, for each domain is calculated using the following formula: Total loss =μ+penalty_weight*penalty; Where penalty_weight represents the weight of the penalty term; The spatial transcriptome data annotation model is trained using stochastic gradient descent and based on the total loss function over each domain, resulting in a trained spatial transcriptome data annotation model.

7. A spatial transcriptome data annotation device based on graph convolutional neural networks, characterized in that, include: The data processing module is configured to acquire spatial transcriptome data of cancer junction slices to be annotated and standardize it to obtain standardized spatial transcriptome data. The module then extracts hypervariable genes from the standardized spatial transcriptome data as a feature expression matrix, where each element in the feature expression matrix represents the expression level of a gene in one region of the spatial transcriptome slice. The graph structure construction module is configured to construct an initial graph structure based on the positional information between regions of the spatial transcriptome slice and the feature expression matrix; construct an image feature extraction module based on a pre-trained deep neural network to acquire H&E images corresponding to the spatial transcriptome data of the cancer border region slice to be annotated, and cut the spatial transcriptome slice according to each region to obtain H&E image blocks corresponding to each region of the spatial transcriptome slice; input the H&E image blocks corresponding to each region of the spatial transcriptome slice into the image feature extraction module to extract the image features corresponding to the H&E image blocks corresponding to each region of the spatial transcriptome slice; integrate the image features corresponding to the H&E image blocks corresponding to each region of the spatial transcriptome slice into the initial graph structure to obtain the final graph structure; The model building module is configured to build a spatial transcriptome data annotation model based on a graph convolutional neural network model, and to perform generalization training on the spatial transcriptome data annotation model to obtain a trained spatial transcriptome data annotation model. The prediction module is configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation results for each region in the spatial transcriptome data of the cancer junction slice to be annotated.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images