Space transcriptome data annotation method and device based on graph convolutional neural network

Abstract

The invention discloses a spatial transcriptome data annotation method and device based on a graph convolutional neural network, and the method comprises the steps: obtaining and standardizing the spatial transcriptome data of a to-be-annotated cancer junction region slice, and extracting a hypervariant gene in the standardized spatial transcriptome data as a feature expression matrix; constructing an initial graph structure based on the position information between the areas of the spatial transcriptome slices and the feature expression matrix; the method comprises the following steps: performing Hamp; e, cutting the image according to each region of the spatial transcriptome slice to obtain corresponding Hamp; e image blocks; the corresponding Hamp is processed; e, inputting the image blocks into an image feature extraction module, extracting to obtain corresponding image features, and integrating the corresponding image features into the initial image structure to obtain a final image structure; and inputting the final graph structure into a trained spatial transcriptome data annotation model to obtain an annotation result of each region in the spatial transcriptome data of the to-be-annotated cancer junction region slice. According to the method, the annotation prediction accuracy can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of spatial transcription data annotation, and in particular to a method and device for annotating spatial transcriptome data based on a graph convolutional neural network. Background Art

[0002] Spatial transcriptomics technology uses high-throughput sequencing to comprehensively analyze the transcriptome in tissue samples and can preserve the spatial location information of cells in tissues. Unlike traditional transcriptomics technology, spatial transcriptomics can not only reveal the overall spectrum of gene expression, but also accurately track the changes in gene expression in different tissue regions, cell types and microenvironments. This provides an unprecedented perspective for studying the interactions between cells, the relationship between tissue structure and function, and the occurrence and development of diseases.

[0003] Especially in the fields of oncology, immunology, neuroscience, etc., the spatial distribution and relative position of different cells in tissues are crucial to understanding the occurrence mechanism, progression and treatment effect of diseases. Spatial information can reveal how cells respond to external stimuli in a specific tissue environment, how they communicate with surrounding cells, and even how they regulate gene expression in a specific microenvironment. Therefore, spatial transcriptome technology provides new tools and methods for early diagnosis, precision treatment and pathological research of diseases, which helps to deeply understand the behavior of cells in their microenvironment and lay the foundation for the development of precision medicine.

[0004] In the study of spatial transcriptome, the most important step is to annotate the obtained spatial domains. The main steps are divided into two steps, identifying spatially variable genes that show spatial expression patterns and detecting spatial domains with consistent gene expression. Annotating spatial domains can better understand tissue structure and functional zoning and explore disease progression.

[0005] Currently, the annotation of spatial domain mainly adopts unsupervised clustering methods, such as SpaGCN, etc. Unsupervised clustering methods need to segment the spatial domain first, and then identify the spatially variable genes to annotate the spatial domain.

[0006] The above-mentioned prior art has the following defects and obstacles:

[0007] (1) Most of the spatially variable genes identified use statistical modeling-based methods to calculate the significance of genes and thus measure the spatial distribution of gene expression. This makes it difficult to explain the spatial distribution characteristics of genes. In addition, for these unsupervised methods, the segmentation of the spatial domain and the identification of spatially variable genes are treated as two separate tasks, while in biology there is a close connection between spatially variable genes and spatial domains.

[0008] (2) Current annotation methods do not fully utilize all the information of spatial transcriptome data. Spatial transcriptome sequencing data contains spatial location information of tissue regions, gene expression, and corresponding image information. Since the corresponding image information in the current database is mainly low-resolution image information and is limited by the resolution of spatial transcriptome sequencing technology, published annotation methods mainly use gene information and ignore image information. Some emerging methods such as SpaGCN only consider the color information of the image. For methods that use image information for recognition, the resolution of the image is the decisive factor in whether annotation can be performed, and the recognition effect is not good for low-resolution images. Summary of the invention

[0009] The purpose of this application is to propose a spatial transcriptome data annotation method and device based on graph convolutional neural network to address the above-mentioned technical problems.

[0010] In a first aspect, the present invention provides a spatial transcriptome data annotation method based on a graph convolutional neural network, comprising the following steps:

[0011] The spatial transcriptome data of the cancer border zone slice to be annotated are obtained and standardized to obtain the standardized spatial transcriptome data, and the highly variable genes in the standardized spatial transcriptome data are extracted as a feature expression matrix, wherein the elements in the feature expression matrix are the expression levels of one gene in one region of the spatial transcriptome slice;

[0012] An initial graph structure is constructed based on the position information and feature expression matrix between the regions of the spatial transcriptome slice; an image feature extraction module based on a pre-trained deep neural network is constructed to obtain an H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated and cut according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, and the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice are integrated into the initial graph structure to obtain a final graph structure;

[0013] Construct a spatial transcriptome data annotation model based on a graph convolutional neural network model, perform generalized training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model;

[0014] The final graph structure is input into the trained spatial transcriptome data annotation model to obtain the annotation results of each region in the spatial transcriptome data of the cancer border zone slices to be annotated.

[0015] Preferably, the feature expression matrix is ​​expressed as:

[0016]

[0017] Among them, express_matrix is ​​the feature expression matrix, E ij Represents the expression level of the jth gene in the ith region of the spatial transcriptome slice, i∈{1,2,…,m}, j∈{1,2,…,n}.

[0018] Preferably, the initial graph structure is constructed based on the position information and feature expression matrix between regions of the spatial transcriptome slice, specifically including:

[0019] Each region of the spatial transcriptome slice is used as a node in the initial graph structure, and the distance between each two regions of the spatial transcriptome slice is calculated using the Euclidean distance;

[0020] It is determined whether the distance between the two regions of the spatial transcriptome slice exceeds the distance threshold. If so, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, so as to construct the adjacency matrix of the initial graph structure.

[0021] Preferably, the nodes in the initial graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices, and the gene expression features of the corresponding regions in the spatial transcriptome slices are the row vectors of the corresponding regions in the feature expression matrix; the nodes in the final graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices and the image features of the H&E image blocks of the corresponding regions.

[0022] Preferably, the pre-trained deep neural network includes ResNet50, and the graph convolutional neural network model includes an input layer, a hidden layer, and an output layer. The calculation process of the input layer is as follows:

[0023]

[0024] Among them, X represents the features obtained by splicing the gene expression features and image features stored on each node, W1 represents the weight matrix of the input layer, is the normalized adjacency matrix, ReLU is the relu activation function, and Z1 represents the output features of the input layer;

[0025] The calculation process of the hidden layer is as follows:

[0026]

[0027] Among them, Dropout represents the discarding operation, W2 represents the weight matrix of the hidden layer, and Z2 represents the output feature of the hidden layer;

[0028] The calculation process of the output layer is as follows:

[0029]

[0030] Among them, Softmax represents the Softmax function, W3 represents the weight matrix of the output layer, and Z3 represents the prediction score;

[0031] The output layer uses two neurons as two classification heads, so the prediction score is a two-dimensional vector

[0032] Z3=[Score normal ,Score cancer ], where Score normal Indicates the probability score of normal cells, Score cancer Represents the probability score of malignant cells, and the annotation results are output using the following formula:

[0033] Class prediction = argmax(Score normal ,Score cancer );

[0034] Among them, argmax means taking the maximum value, Class prediction Indicates the annotation result, the value is 0 or 1, 1 represents malignant cells, and 0 represents normal cells.

[0035] Preferably, the spatial transcriptome data annotation model is subjected to generalized training to obtain a trained spatial transcriptome data annotation model, specifically comprising:

[0036] Collect spatial transcriptome data from different tissues and distributions and their corresponding true labels and H&E images, and divide them into training sets of different domains according to different cancer types;

[0037] The spatial transcriptome data of each spatial transcriptome slice in the training set of each domain are standardized, the corresponding feature expression matrix is ​​extracted and the initial graph structure is constructed; the H&E image of each spatial transcriptome slice in the training set of each domain is cut according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice, and the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice;

[0038] Integrate the image features corresponding to the H&E image blocks corresponding to each area of ​​the spatial transcriptome slice into the corresponding nodes in the initial graph structure for storage to obtain the final graph structure, and input the final graph structure into the spatial transcriptome data annotation model to obtain the predicted label;

[0039] The malignancy proportion in each spatial transcriptome slice is calculated based on the true label of each spatial transcriptome slice in the training set of each domain as shown in the following formula:

[0040]

[0041] Among them, malignant ratio means the malignant ratio, spot malignant represents the number of regions marked as malignant cells in the true label of each spatial transcriptome slice, and sum(spot) represents the number of all regions in each spatial transcriptome slice;

[0042] Determine whether the malignancy ratio is less than or equal to the ratio threshold. If so, calculate the focal loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain. Otherwise, calculate the cross entropy loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain.

[0043] The average loss value of all spatial transcriptome slices in the training set of each domain is calculated as follows:

[0044]

[0045] Wherein, Loss_domain[h] represents the loss value of the h-th spatial transcriptome slice in the training set of one of the domains, and the total number of spatial transcriptome slices in the training set of this domain is H;

[0046] The penalty term penalty is calculated based on the average loss value, as shown in the following formula:

[0047]

[0048] The total loss function Total_loss for each domain is calculated using the following formula:

[0049] Total loss =μ+penalty_weight*penalty;

[0050] Among them, penalty_weight represents the weight of the penalty item;

[0051] The spatial transcriptome data annotation model is trained by stochastic gradient descent based on the total loss function on each domain to obtain a trained spatial transcriptome data annotation model.

[0052] In a second aspect, the present invention provides a spatial transcriptome data annotation device based on a graph convolutional neural network, comprising:

[0053] A data processing module is configured to obtain and standardize the spatial transcriptome data of the cancer border zone slice to be annotated, obtain the standardized spatial transcriptome data, extract the highly variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression levels of one of the genes in one of the regions of the spatial transcriptome slice;

[0054] A graph structure construction module is configured to construct an initial graph structure based on the position information and feature expression matrix between regions of the spatial transcriptome slice; construct an image feature extraction module based on a pre-trained deep neural network, obtain an H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated, and cut it according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; input the H&E image block corresponding to each region of the spatial transcriptome slice into the image feature extraction module, extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, and integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the initial graph structure to obtain a final graph structure;

[0055] A model building module is configured to build a spatial transcriptome data annotation model based on a graph convolutional neural network model, perform generalized training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model;

[0056] The prediction module is configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation results of each region in the spatial transcriptome data of the cancer border zone slice to be annotated.

[0057] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0058] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0059] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] (1) The spatial transcriptome data annotation method based on graph convolutional neural network proposed in the present invention uses graph structure to characterize spatial transcriptome data, models the gene expression of spots as the features of graph nodes, and models the direct spatial distance of spots as the distance of graph structure edges. H&E images are segmented according to the positions of spots, and features are extracted using deep neural networks and integrated with gene features. This can make full use of gene expression, position information and image information in spatial transcriptome data, improve the accuracy of model prediction, and effectively reduce the appearance of irregular discrete spots. Compared with existing methods, this method reuses the spatial position information and image information of spots, and has a higher annotation accuracy for malignant areas in spatial transcriptome data.

[0062] (2) The spatial transcriptome data annotation method based on graph convolutional neural network proposed in the present invention adopts a supervised graph convolutional neural network model for training to achieve end-to-end spatial domain prediction effect. The supervised training method improves the consistency of the spatial domain identification process and can explore the biological interpretability of the results. At present, the main spatial domain identification method is an unsupervised method, which lacks consistency in the process of dividing spatial domains and identifying variable genes.

[0063] (3) The spatial transcriptome data annotation method based on graph convolutional neural network proposed in the present invention performs generalized training on the graph convolutional neural network model, which can be applied to the annotation of spatial transcriptome data of different types of cancers and can achieve high model prediction accuracy on a smaller training data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 A schematic diagram of the process of a spatial transcriptome data annotation method based on a graph convolutional neural network according to an embodiment of the present application;

[0066] Figure 2 A flowchart of a spatial transcriptome data annotation method based on a graph convolutional neural network according to an embodiment of the present application;

[0067] Figure 3 Schematic diagram of feasibility verification of the spatial transcriptome data annotation method based on graph convolutional neural network according to an embodiment of the present application, wherein (a) indicates that only gene information is used to construct a graph structure, (b) indicates that position information and gene information are integrated to construct a graph structure, and (c) indicates a marked area;

[0068] Figure 4 This is a visualization result diagram of the spatial transcriptome data annotation method based on graph convolutional neural network for different types of cancer border area slices in an embodiment of the present application, wherein the left side of each box is the pathology expert annotation area, the right side is the model annotation result, the red is the area of ​​malignant cells, the yellow is the area of ​​normal cells, (a) represents breast cancer, (b) represents colorectal cancer, (c) represents renal clear cell carcinoma, and (d) represents hepatocellular carcinoma;

[0069] Figure 5 A result diagram of the prediction accuracy of the spatial transcriptome data annotation method based on a graph convolutional neural network according to an embodiment of the present application;

[0070] Figure 6 A schematic diagram of a spatial transcriptome data annotation device based on a graph convolutional neural network according to an embodiment of the present application;

[0071] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Figure 1 A spatial transcriptome data annotation method based on a graph convolutional neural network provided in an embodiment of the present application is shown, comprising the following steps:

[0074] S1, obtain the spatial transcriptome data of the cancer border zone slice to be annotated and standardize it to obtain the standardized spatial transcriptome data, extract the highly variable genes in the standardized spatial transcriptome data as the feature expression matrix, and the elements in the feature expression matrix are the expression levels of one of the genes in one of the regions of the spatial transcriptome slice.

[0075] In a specific embodiment, the feature expression matrix is ​​expressed as:

[0076]

[0077] Among them, express_matrix is ​​the feature expression matrix, E ij Represents the expression level of the jth gene in the ith region of the spatial transcriptome slice, i∈{1,2,…,m}, j∈{1,2,…,n}.

[0078] Specifically, refer to Figure 2 , the embodiment of the present application first standardizes the spatial transcriptome data and extracts highly variable genes as a characteristic expression matrix. Each element in the characteristic expression matrix corresponds to the expression of one of the genes in one of the regions (spot) of the spatial transcriptome slice, and the i-th row in the characteristic expression matrix corresponds to the expression of n genes in the i-th spot, and the gene expression information and position information in the spatial transcriptome data are extracted through the characteristic expression matrix.

[0079] S2, constructing an initial graph structure based on the position information and feature expression matrix between the regions of the spatial transcriptome slice; constructing an image feature extraction module based on a pre-trained deep neural network, obtaining the H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated and cutting it according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice; inputting the H&E image block corresponding to each region of the spatial transcriptome slice into the image feature extraction module, extracting the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, integrating the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the initial graph structure, and obtaining the final graph structure.

[0080] In a specific embodiment, the initial graph structure is constructed based on the position information and feature expression matrix between regions of the spatial transcriptome slice, specifically including:

[0081] Each region of the spatial transcriptome slice is used as a node in the initial graph structure, and the distance between each two regions of the spatial transcriptome slice is calculated using the Euclidean distance;

[0082] It is determined whether the distance between the two regions of the spatial transcriptome slice exceeds the distance threshold. If so, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, and finally the adjacency matrix of the initial graph structure is constructed.

[0083] In a specific embodiment, the nodes in the initial graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices, and the gene expression features of the corresponding regions in the spatial transcriptome slices are the row vectors of the corresponding regions in the feature expression matrix; the nodes in the final graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices and the image features of the H&E image blocks of the corresponding regions.

[0084] Specifically, the initial graph structure is constructed based on the location information between spots. The distance between each two spots is calculated using the Euclidean distance, and a distance threshold is set to limit the connectivity between the nodes corresponding to the two spots. The nodes corresponding to the two spots whose distance exceeds the distance threshold are considered to be disconnected, based on the fact that the information exchange between the two spots with a long distance is weak. Finally, the element A of the adjacency matrix A representing the initial graph structure is generated. ik : Among them, spot i Indicates the node corresponding to the i-th region, spot k Represents the node corresponding to the kth region.

[0085] Furthermore, an image feature extraction module based on a pre-trained deep neural network was first constructed, and then the H&E image was preprocessed, mainly including color normalization and contrast enhancement. The gene expression data of the spatial transcriptome slices were spatially aligned with the H&E images of the corresponding spatial transcriptome slices, and the H&E images corresponding to the spatial transcriptome data were cut. The gene expression information of each spot was matched with the H&E image to obtain each H&E image block. The image features of each H&E image block were extracted using the pre-trained deep neural network ResNet50, and the initial graph structure and image features were integrated to construct the corresponding final graph structure, which contained gene expression features, location information, and image features.

[0086] S3, construct a spatial transcriptome data annotation model based on the graph convolutional neural network model, perform generalized training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model.

[0087] In a specific embodiment, the pre-trained deep neural network includes ResNet50, the graph convolutional neural network model includes an input layer, a hidden layer and an output layer, and the calculation process of the input layer is as follows:

[0088]

[0089] Among them, X represents the features obtained by splicing the gene expression features and image features stored on each node, W1 represents the weight matrix of the input layer, is the normalized adjacency matrix, ReLU is the relu activation function, and Z1 represents the output features of the input layer;

[0090] The calculation process of the hidden layer is as follows:

[0091]

[0092] Among them, Dropout represents the discarding operation, W2 represents the weight matrix of the hidden layer, and Z2 represents the output feature of the hidden layer;

[0093] The calculation process of the output layer is as follows:

[0094]

[0095] Among them, Softmax represents the Softmax function, W3 represents the weight matrix of the output layer, and Z3 represents the prediction score;

[0096] The output layer uses two neurons as two classification heads, so the prediction score is a two-dimensional vector

[0097] Z3=[Score normal ,Score cancer ], where Score normal Indicates the probability score of normal cells, Score cancer Represents the probability score of malignant cells, and the annotation results are output using the following formula:

[0098] Class prediction = argmax(Score normal ,Score cancer );

[0099] Among them, argmax means taking the maximum value, Class prediction Indicates the annotation result, the value is 0 or 1, 1 represents malignant cells, and 0 represents normal cells.

[0100] Specifically, the spatial transcriptome data annotation model adopted in the embodiments of the present application is a three-layer graph convolutional neural network model, which is used to extract features from the final graph structure and classify them. The model design combines the gene expression information, position information and H&E image information of the nodes, and learns the complex relationship between different spatial regions through graph convolution operations. The input layer of the model is the final graph structure constructed for each slice. The final graph structure embeds the position information of the slice by calculating the Euclidean distance between spots, and the node features are the features obtained by splicing the gene expression features and image features. Each layer of the model updates the node representation through graph convolution operations, and applies dropout to prevent overfitting.

[0101] In one embodiment, in the input layer of the graph convolutional neural network model, from the input feature dimension d in =5000 is mapped to the output of the same dimension, that is, the input dimension is maintained to extract the initial local spatial relationship features. In the hidden layer, the dimension d is mapped from 5000 to the hidden layer. hid=512, and apply dropout to randomly discard some neurons. The specific proportion is determined by the dropout rate to avoid overfitting. In the output layer, the dimension of the hidden layer is further reduced from 512 to the dimension of the output layer d out =2, a prediction score is generated, and the annotation result can be determined based on the prediction score, that is, whether the region is a malignant cell or a normal cell region.

[0102] In a specific embodiment, the spatial transcriptome data annotation model is subjected to generalized training to obtain a trained spatial transcriptome data annotation model, specifically comprising:

[0103] Collect spatial transcriptome data from different tissues and distributions and their corresponding true labels and H&E images, and divide them into training sets of different domains according to different cancer types;

[0104] The spatial transcriptome data of each spatial transcriptome slice in the training set of each domain are standardized, the corresponding feature expression matrix is ​​extracted and the initial graph structure is constructed; the H&E image of each spatial transcriptome slice in the training set of each domain is cut according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice, and the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice;

[0105] Integrate the image features corresponding to the H&E image blocks corresponding to each area of ​​the spatial transcriptome slice into the corresponding nodes in the initial graph structure for storage to obtain the final graph structure, and input the final graph structure into the spatial transcriptome data annotation model to obtain the predicted label;

[0106] The malignancy proportion in each spatial transcriptome slice is calculated based on the true label of each spatial transcriptome slice in the training set of each domain as shown in the following formula:

[0107]

[0108] Among them, malignant ratio means the malignant ratio, spot malignant represents the number of regions marked as malignant cells in the true label of each spatial transcriptome slice, and sum(spot) represents the number of all regions in each spatial transcriptome slice;

[0109] Determine whether the malignancy ratio is less than or equal to the ratio threshold. If so, calculate the focal loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain. Otherwise, calculate the cross entropy loss function based on the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain.

[0110] The average loss value of all spatial transcriptome slices in the training set of each domain is calculated as follows:

[0111]

[0112] Wherein, Loss_domain[h] represents the loss value of the h-th spatial transcriptome slice in the training set of one of the domains, and the total number of spatial transcriptome slices in the training set of this domain is H;

[0113] The penalty term penalty is calculated based on the average loss value, as shown in the following formula:

[0114]

[0115] The total loss function Total_loss for each domain is calculated using the following formula:

[0116] Total loss =μ+penalty_weight*penalty;

[0117] Among them, penalty_weight represents the weight of the penalty item;

[0118] The spatial transcriptome data annotation model is trained by stochastic gradient descent based on the total loss function on each domain to obtain a trained spatial transcriptome data annotation model.

[0119] Specifically, the spatial transcriptome data from different tissues and different distributions are divided into different domains, each cancer is a domain, and the data is annotated according to the true label of the training set. The true label is malignant cells or non-malignant cells. The spatial transcriptome data in the training set are constructed using steps S1 and S2 to construct the corresponding final graph structure. The spatial transcriptome data annotation model uses a three-layer graph convolutional neural network model, which is trained by the final graph structure and true label corresponding to the training set of each domain.

[0120] In one embodiment, 16 human spatial transcriptome cancer boundary region data were collected in the GEO database, and the data were divided into seven domains according to the cancer type. The annotations provided by the GEO database were used as the true labels of the spatial transcriptome data. During the training process, the breast cancer spatial transcriptome data was fixedly selected as the validation set; the data of each domain was traversed and used as the test set in turn; and the remaining data was used as the training set.

[0121] Highly variable genes are screened using the variance of expression. The gene expression of all normalized spatial transcriptomes is sorted according to the variance difference between the data, and the top 5,000 genes are selected as highly variable genes. The specific variance sorting method is: the gene expression of the spatial transcriptome slices in the training set (row name is gene, column name is spot) is merged according to the gene name, so that the expression data of all spots of a gene in each row of all training sets can be obtained. Variance calculation of the jth row: Among them, var j represents the variance of the jth row, x ji represents the expression of the jth row and the ith column, represents the mean of all expression levels in the jth row;

[0122] After calculating the variance of each row, we sorted them from large to small. When constructing the initial graph structure, we used the relative spatial position of the spot in the spatial transcriptome as a metric. By adjusting the distance threshold between different spots, we found that

[0123] X 2 +Y 2 <=2 2 (X, Y are the horizontal and vertical coordinates of the relative position of the spot) can build a simple and efficient initial graph structure. The image features are further integrated to build the final graph structure.

[0124] During the training process, the final graph structures of the training sets in different domains are input into the graph convolutional neural network model for classification to obtain the predicted labels; first, the malignant ratio in the spatial transcriptome slice is calculated based on the number of regions of malignant cells annotated in the true label and the number of all regions. For samples with a large difference between the malignant region and the non-malignant region, that is, malignant ratio ≤ 0.1, the focal loss function is used to calculate the loss value of the spatial transcriptome slice to reduce the loss caused by sample imbalance; for other cases, the cross entropy loss function is used to calculate the loss value of the spatial transcriptome slice. The graph convolutional neural network model is trained using the stochastic gradient descent algorithm. The training process aims to reduce the loss difference of the model in multiple domains (i.e., different data distributions, such as different tissue slices or patient data), and to enhance the performance of the graph convolutional neural network model in unseen domains through additional penalty terms. Finally, the trained spatial transcriptome data annotation model is obtained. The embodiments of the present application simultaneously perform training generalization for multiple domains, so the obtained trained spatial transcriptome data annotation model can predict multiple cancers without the need for retraining for a specific cancer.

[0125] In order to confirm that the final graph structure constructed using the embodiments of the present application can effectively improve the prediction performance of the model, the embodiments of the present application conducted a comparison verification: only using gene information to construct the graph structure, and integrating location information and gene information to construct the graph structure for model training. Figure 3 Compared with the simple use of genetic information to construct a graph structure, the graph structure constructed by integrating position information and genetic information reconstructs the spatial position of tissue slices, which can effectively improve the prediction performance of the model and reduce the occurrence of discrete malignant tissues outside the malignant area.

[0126] Through comparative verification experiments, this application proves that the method of constructing a graph structure based on the actual location information of spots in spatial transcriptome slices is significantly effective. Furthermore, in order to study the impact of node feature selection in the graph structure, this application conducted an ablation experiment. The results are shown in Table 1 and Figure 4 As shown in Figure 1, the experiment is divided into three groups: position + image feature group, position + gene feature group, and position + image feature + gene feature group. All three groups use a graph structure built based on position information, but the difference is that different feature types are assigned to nodes.

[0127] The experimental results show that among the total of 12 slices, the change in prediction performance of the position + image feature + gene feature group compared with the position + gene feature group is as follows: the performance of 4 slices decreased, the performance of 6 slices improved, and the performance of 2 slices did not change significantly. Based on this, it is concluded that after adding image features, the prediction ability of the model has improved slightly overall. Further analysis found that the results of the position + image feature group revealed that the noise in the image features is the main bottleneck affecting the improvement of model performance. Therefore, if image features with less noise are used, the prediction effect may be more significantly improved.

[0128] Table 1

[0129]

[0130] S4, input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation results of each region in the spatial transcriptome data of the cancer border zone slice to be annotated.

[0131] Specifically, the trained spatial transcriptome data annotation model is deployed. During the inference process, for unknown cancer border area slices, the final graph structure obtained through the above steps S1 and S2 is input into the trained spatial transcriptome data annotation model to obtain the annotation results of each area. Figure 5 As shown, the annotation results of spatial transcriptome data of different cancer types predicted by the trained spatial transcriptome data annotation model of the present application are consistent with the areas annotated by pathology experts, indicating that the annotation effect of the trained spatial transcriptome data annotation model of the present application is good.

[0132] Further references Figure 6 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a spatial transcriptome data annotation device based on a graph convolutional neural network. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0133] The present application embodiment provides a spatial transcriptome data annotation device based on a graph convolutional neural network, comprising:

[0134] The data processing module 1 is configured to obtain the spatial transcriptome data of the cancer border zone slice to be annotated and perform standardization to obtain the standardized spatial transcriptome data, extract the highly variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression levels of one gene in one region of the spatial transcriptome slice;

[0135] A graph structure construction module 2 is configured to construct an initial graph structure based on the position information and feature expression matrix between the regions of the spatial transcriptome slices; construct an image feature extraction module based on a pre-trained deep neural network, obtain an H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated, and cut it according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; input the H&E image block corresponding to each region of the spatial transcriptome slice into the image feature extraction module, extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, and integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the initial graph structure to obtain a final graph structure;

[0136] A model building module 3 is configured to build a spatial transcriptome data annotation model based on a graph convolutional neural network model, perform generalized training on the spatial transcriptome data annotation model, and obtain a trained spatial transcriptome data annotation model;

[0137] The prediction module 4 is configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation results of each region in the spatial transcriptome data of the cancer border zone slice to be annotated.

[0138] Figure 7 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 7 As shown, the electronic device of this embodiment includes: a processor 701 and a memory 702; wherein the memory 702 is used to store computer-executable instructions; the processor 701 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0139] Optionally, the memory 702 may be independent or integrated with the processor 701 .

[0140] When the memory 702 is independently provided, the electronic device further includes a bus 703 for connecting the memory 702 and the processor 701 .

[0141] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 701 executes the computer execution instructions, the above method is implemented.

[0142] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 701, the above method is implemented.

[0143] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0144] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0145] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0146] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 701 to perform some steps of the methods of various embodiments of the present application.

[0147] It should be understood that the processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 701 may be any conventional processor 701, etc. The steps of the method disclosed in the invention may be directly embodied in the execution of the hardware processor 701, or may be executed by a combination of hardware and software modules in the processor 701.

[0148] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0149] The bus 703 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 703 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 703 in the drawings of the present application is not limited to only one bus 703 or one type of bus 703.

[0150] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0151] An exemplary storage medium is coupled to the processor 701, so that the processor 701 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 701. The processor 701 and the storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as ASIC). Of course, the processor 701 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0152] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatial transcriptome data annotation method based on graph convolutional neural network, characterized in that: The following steps are involved: Acquire the spatial transcriptome data of the cancer border zone slice to be annotated and standardize it to obtain the standardized spatial transcriptome data, extract the highly variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression levels of one gene in one region of the spatial transcriptome slice; An initial graph structure is constructed based on the position information between the regions of the spatial transcriptome slice and the feature expression matrix; an image feature extraction module based on a pre-trained deep neural network is constructed, an H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated is obtained, and the H&E image corresponding to each region of the spatial transcriptome slice is cut according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module, image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice are extracted, and image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice are integrated into the initial graph structure to obtain a final graph structure; Constructing a spatial transcriptome data annotation model based on a graph convolutional neural network model, and performing generalized training on the spatial transcriptome data annotation model to obtain a trained spatial transcriptome data annotation model; The final graph structure is input into the trained spatial transcriptome data annotation model to obtain the annotation result of each region in the spatial transcriptome data of the cancer border zone slice to be annotated.

2. The spatial transcriptome data annotation method based on graph convolutional neural network according to claim 1, characterized in that: The feature expression matrix is ​​expressed as: Among them, express_matrix is ​​the feature expression matrix, E ij Represents the expression level of the jth gene in the ith region of the spatial transcriptome slice, i∈{1,2,…,m}, j∈{1,2,…,n}.

3. The spatial transcriptome data annotation method based on graph convolutional neural network according to claim 1, characterized in that: Constructing an initial graph structure based on the position information between the regions of the spatial transcriptome slice and the feature expression matrix specifically includes: Taking each region of the spatial transcriptome slice as a node in the initial graph structure, and calculating the distance between each two regions of the spatial transcriptome slice using Euclidean distance; It is determined whether the distance between the two regions of the spatial transcriptome slice exceeds the distance threshold. If so, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are not connected in the initial graph structure; otherwise, it is determined that the two nodes corresponding to the two regions of the spatial transcriptome slice are connected in the initial graph structure, thereby constructing an adjacency matrix of the initial graph structure.

4. The spatial transcriptome data annotation method based on graph convolutional neural network according to claim 3, characterized in that: The nodes in the initial graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices, and the gene expression features of the corresponding regions in the spatial transcriptome slices are the row vectors of the corresponding regions in the feature expression matrix; the nodes in the final graph structure store the gene expression features of the corresponding regions in the spatial transcriptome slices and the image features of the H&E image blocks of the corresponding regions.

5. The spatial transcriptome data annotation method based on graph convolutional neural network according to claim 4, characterized in that: The pre-trained deep neural network includes ResNet50, the graph convolutional neural network model includes an input layer, a hidden layer and an output layer, and the calculation process of the input layer is as follows: Among them, X represents the features obtained by splicing the gene expression features and image features stored on each node, W1 represents the weight matrix of the input layer, is the normalized adjacency matrix, ReLU is the relu activation function, and Z1 represents the output feature of the input layer; The calculation process of the hidden layer is as follows: Wherein, Dropout represents a drop operation, W2 represents a weight matrix of a hidden layer, and Z2 represents an output feature of the hidden layer; The calculation process of the output layer is as follows: Among them, Softmax represents the Softmax function, W3 represents the weight matrix of the output layer, and Z3 represents the prediction score; The output layer uses two neurons as two classification heads, so the prediction score is a two-dimensional vector Z3=[Score normal ,Score cancer ], where Score normal Represents the probability score of normal cells, Score cancer Represents the probability score of malignant cells, and the annotation results are output using the following formula: Class prediction =argmax(Score normal ,Score cancer ); Among them, argmax means taking the maximum value, Class prediction Indicates the annotation result, the value is 0 or 1, 1 represents malignant cells, and 0 represents normal cells.

6. The spatial transcriptome data annotation method based on graph convolutional neural network according to claim 1, characterized in that: The spatial transcriptome data annotation model is subjected to generalized training to obtain a trained spatial transcriptome data annotation model, specifically comprising: Collect spatial transcriptome data from different tissues and distributions and their corresponding true labels and H&E images, and divide them into training sets of different domains according to different cancer types; The spatial transcriptome data of each spatial transcriptome slice in the training set of each domain are standardized, the corresponding feature expression matrix is ​​extracted and the initial graph structure is constructed; the H&E image of each spatial transcriptome slice in the training set of each domain is cut according to each region of the spatial transcriptome slice to obtain the H&E image block corresponding to each region of the spatial transcriptome slice, and the H&E image block corresponding to each region of the spatial transcriptome slice is input into the image feature extraction module to extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice; Integrate the image features corresponding to the H&E image blocks corresponding to each region of the spatial transcriptome slice into the corresponding nodes in the initial graph structure for storage to obtain a final graph structure, and input the final graph structure into the spatial transcriptome data annotation model to obtain a predicted label; The malignancy ratio in each spatial transcriptome slice is calculated based on the true label of each spatial transcriptome slice in the training set of each domain as shown in the following formula: Among them, malignant ratio means the malignant ratio, spot malignant represents the number of regions marked as malignant cells in the true label of each spatial transcriptome slice, and sum(spot) represents the number of all regions in each spatial transcriptome slice; Determine whether the malignancy ratio is less than or equal to the ratio threshold, if so, calculate the focal loss function according to the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain, otherwise calculate the cross entropy loss function according to the predicted label and the true label to obtain the loss value of each spatial transcriptome slice in the training set of each domain; The average loss value of all spatial transcriptome slices in the training set of each domain is calculated as follows: Wherein, Loss_domain[h] represents the loss value of the h-th spatial transcriptome slice in the training set of one of the domains, and the total number of spatial transcriptome slices in the training set of this domain is H; The penalty term penalty is calculated based on the average loss value, as shown in the following formula: The total loss function Total_loss for each domain is calculated using the following formula: Total loss =μ+penalty_weight*penalty; Among them, penalty_weight represents the weight of the penalty item; The spatial transcriptome data annotation model is trained by adopting a stochastic gradient descent method and based on the total loss function on each domain to obtain a trained spatial transcriptome data annotation model.

7. A spatial transcriptome data annotation device based on graph convolutional neural network, characterized in that: include: A data processing module is configured to obtain and standardize the spatial transcriptome data of the cancer border zone slice to be annotated, obtain the standardized spatial transcriptome data, extract the highly variable genes in the standardized spatial transcriptome data as a feature expression matrix, and the elements in the feature expression matrix are the expression levels of one of the genes in one of the regions of the spatial transcriptome slice; A graph structure construction module is configured to construct an initial graph structure based on the position information between the regions of the spatial transcriptome slice and the feature expression matrix; construct an image feature extraction module based on a pre-trained deep neural network, obtain an H&E image corresponding to the spatial transcriptome data of the cancer border zone slice to be annotated, and cut it according to each region of the spatial transcriptome slice to obtain an H&E image block corresponding to each region of the spatial transcriptome slice; input the H&E image block corresponding to each region of the spatial transcriptome slice into the image feature extraction module, extract the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice, and integrate the image features corresponding to the H&E image block corresponding to each region of the spatial transcriptome slice into the initial graph structure to obtain a final graph structure; A model building module is configured to build a spatial transcriptome data annotation model based on a graph convolutional neural network model, and perform generalized training on the spatial transcriptome data annotation model to obtain a trained spatial transcriptome data annotation model; The prediction module is configured to input the final graph structure into the trained spatial transcriptome data annotation model to obtain the annotation result of each region in the spatial transcriptome data of the cancer border zone slice to be annotated.

8. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images