A gene space expression prediction method based on tumor pathological images
By combining deep learning models of convolutional neural networks and graph neural networks, pixel intensity and structural features in tumor pathological images are extracted, solving the problems of insufficient prediction accuracy and stability in existing technologies, and achieving high-precision and stable gene spatial expression prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-07-04
- Publication Date
- 2026-07-31
AI Technical Summary
Existing gene spatial expression prediction methods based on tumor pathology images are inadequate in terms of prediction accuracy, stability, and scalability, and cannot effectively utilize pixel intensity and structural features in the images.
By combining convolutional neural networks and graph neural networks, a deep learning model is constructed by extracting pixel intensity and cell structure features from tumor pathological images, mapping pathological images to gene expression space, and predicting gene spatial expression.
It significantly improves the accuracy and stability of gene spatial expression prediction, enhances the generalization performance of the model, demonstrates superior predictive performance in different solid tumor types, and extends to the application of hematoxylin-eosin stained full-field digital slides.
Smart Images
Figure CN116844631B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of tumor pathological image analysis, and in particular to a method for predicting gene spatial expression based on tumor pathological images. Background Technology
[0002] The tumor microenvironment (TME) is crucial for characterizing tumor status, predicting prognosis, and making treatment decisions. Spatial transcriptome sequencing provides gene expression, spatial location, and corresponding hematoxylin-eosin stained histopathological images, and is frequently used to explore the complex cellular composition and TME of various cancers. However, spatial transcriptome sequencing is very expensive and has not yet been widely adopted in clinical practice. In contrast, hematoxylin-eosin stained histopathological images are relatively inexpensive and widely used in clinical practice. Pathological images of tumor regions contain rich information about tumor morphology and its surrounding microenvironment. Some deep learning techniques have successfully predicted key molecular biological markers, such as estrogen receptor status (ERS) and microsatellite instability (MSI), from histopathological images. Using tumor pathological images to predict high-dimensional gene expression in corresponding regions holds promise for improving our understanding of the TME and its impact on patient prognosis without relying on expensive and specialized sequencing technologies.
[0003] Recent studies have utilized deep learning methods to predict gene spatial expression levels from hematoxylin and eosin (HE) stained images, including ST-Net (a deep learning algorithm proposed by Bryan Hed et al.), HisToGene (a deep learning algorithm proposed by Minxing Pang et al.), and Hist2ST (a deep learning algorithm proposed by Yuansong Zeng et al.). These methods extract information from histopathological images using image-based deep learning. Gene spatial expression is highly correlated with structural features in hematoxylin-eosin stained images, including tissue structure and cell distribution. However, these structural features cannot be fully extracted using image-based deep learning methods; therefore, existing methods have room for further improvement in prediction accuracy, stability, and scalability.
[0004] Therefore, there is an urgent need to design a gene spatial expression prediction method based on tumor pathology images with high prediction accuracy. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a gene spatial expression prediction method based on tumor pathological images. This method comprehensively utilizes the information contained in the pixel intensity and structural features in the image, and uses convolutional neural networks (CNN) and graph neural networks (GNN) to map the hematoxylin-eosin stained pathological images of the tumor region to the gene expression space, which significantly improves the prediction accuracy and further helps to explore the tumor microenvironment from the perspective of image analysis.
[0006] The objective of this invention can be achieved through the following technical solutions:
[0007] This invention provides a method for predicting gene spatial expression based on tumor pathological images, the method comprising:
[0008] Acquire pathological images and gene expression data from the spatial transcriptome and perform preprocessing;
[0009] Convolutional neural networks and graph neural networks were used to extract tissue texture information and cell structure features, respectively. A gene spatial expression prediction model based on deep learning model was constructed, and the model was trained using preprocessed pathological images and gene expression data.
[0010] Gene spatial expression prediction is performed using a trained gene spatial expression prediction model.
[0011] Preferably, the preprocessing of the pathological images includes:
[0012] The hematoxylin-eosin stained histopathological images from the spatial transcriptome were segmented into small image blocks. The cell nucleus regions in the images were segmented, and relevant features at the cellular level were extracted to construct a cell nucleus map.
[0013] Preferably, the step of segmenting the hematoxylin-eosin stained histopathological images from the spatial transcriptome into small image blocks, segmenting the cell nucleus regions in the images, extracting relevant features at the cellular level, and constructing a cell nucleus map specifically involves:
[0014] Based on the coordinates of different sequencing sites in the spatial transcriptome, the entire hematoxylin-eosin stained pathological image was divided into small image blocks with a set pixel value and resolution, and then color normalization was performed.
[0015] The trained HoVer-Net model (a network based on the prediction of horizontal and vertical distances) was used for cell nucleus region segmentation and cell type classification.
[0016] One-hot encoding is used to map the classification label of each segmentation kernel;
[0017] After extracting and standardizing the features of each cell nucleus region, multidimensional cell nucleus features are obtained. These features include morphology, Fourier shape descriptors, intensity, gradient, and Haralick features (Haralick features are a method for image texture analysis that can effectively extract texture information from images. It was proposed by computer scientist Robert Haralick in 1973 and is widely used in fields such as medical image analysis and machine vision).
[0018] Preferably, for each of the image blocks, a nucleus map is constructed to characterize the geometric topology between cells in the tissue, with the nucleus serving as a node in the map, and the node features being the multidimensional nucleus features of each nucleus.
[0019] Preferably, if the distance between two cell nuclei is less than a set distance, an edge is added between the two nodes.
[0020] Preferably, the preprocessing of the gene expression data specifically includes:
[0021] Spatially variable genes were selected using the SPARK-X (Spatial Pattern Recognition via Kernels eXpedited) method, and then spatially variable genes expressed in more than k% of sequencing sites in all tissue sections were selected as target genes to be predicted.
[0022] The original gene count data is transformed to make it approximately follow a normal distribution.
[0023] Preferably, the transformation of the original target gene count data to make it approximately follow a normal distribution includes: 1) adding 1 to each gene count data to construct a pseudo-count; 2) performing gene expression normalization count for each sequencing site, dividing the pseudo-count of each gene by the total pseudo-count of all genes at that site; 3) scaling, multiplying all normalized count values by 1,000,000; and 4) logarithmic transformation.
[0024] Preferably, the step of using convolutional neural networks and graph neural networks to extract tissue texture information and cell structure features respectively, and constructing a gene spatial expression prediction model based on a deep learning model, specifically involves:
[0025] The gene spatial expression prediction model includes two input branches and one fused output branch. The input branches include an image branch and a graph branch; wherein:
[0026] The image branch contains a deep convolutional neural network with ResNet18 (18-layer Residual Net) convolutional layers to extract tissue texture information of pathological image blocks of a set size, and applies average pooling to output features of a set dimension.
[0027] The graph branch uses a four-layer graph isomorphic network to aggregate neighbor node information in the nucleus graph, and uses global pooling of maximum and average values to calculate the feature representation of each graph isomorphic network GIN layer;
[0028] The fusion output branch, through a two-layer fully connected neural network, fuses the texture features extracted by the image branch and the geometric features extracted by the graph branch to output the spatial gene expression level of the target to be predicted.
[0029] Preferably, the loss function used in training the gene spatial expression prediction model based on the deep learning model is mean squared error, and the Adam optimizer is used for optimization.
[0030] Preferably, the gene spatial expression prediction model based on the deep learning model is trained and optimized using a weight decay and early stopping strategy.
[0031] Compared with the prior art, the present invention has the following beneficial effects:
[0032] 1) High prediction accuracy: This invention combines convolutional neural networks and graph neural networks to simultaneously mine pixel intensity and structural features in pathological images, overcoming the shortcomings of existing technologies that only model based on pixel intensity features of images and ignore structural features such as cell distribution, resulting in higher prediction accuracy;
[0033] 2) Good stability: This invention uses the SPARK-X method to select spatially variable genes as target genes to be predicted, and transforms the original count data of the target genes to make them approximately follow a normal distribution. The mean squared error is used as the loss function for model training, and the weight decay and early stopping strategies are used for training optimization. The model has strong generalization performance and shows superior and stable performance in a variety of solid tumor types.
[0034] 3) High scalability: This invention segments the hematoxylin-eosin stained histopathological images in the spatial transcriptome into small image blocks, divides the cell nucleus regions in the images, extracts relevant features at the cellular level, constructs cell nucleus maps, and predicts the gene expression of the corresponding tissue regions based on each small image block and its cell nucleus map. It shows excellent performance on both 10X Visium and ST technology data, and has high scalability. It can be further extended to spatial gene prediction of hematoxylin-eosin stained full-view digital slices. Attached Figure Description
[0035] Figure 1 This is a flowchart of pathological image preprocessing and Nuclei-Graph construction in the embodiments;
[0036] Figure 2This is a schematic diagram of the architecture of the IGI-DL (integrated graph and image deep learning) model in the embodiment.
[0037] Figure 3 The example shows a violin plot comparing the performance of the IGI-DL model and existing technologies on a colorectal cancer dataset. The evaluation metric is the Pearson correlation coefficient. Subplot (a) shows the results on the internal validation set, and subplot (b) shows the results on the external test set.
[0038] Figure 4 The example shows a violin plot comparing the performance of the IGI-DL model and existing technologies on a breast cancer dataset. The evaluation metric is the Pearson correlation coefficient. Subplot (a) shows the results on the internal validation set, and subplot (b) shows the results on the external test set.
[0039] Figure 5 The example shows a violin plot comparing the performance of the IGI-DL model and existing techniques on a cutaneous squamous cell carcinoma dataset. The evaluation metric is the Pearson correlation coefficient. Subplot (a) shows the results on the internal validation set, and subplot (b) shows the results on the external test set. Detailed Implementation
[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0041] Example
[0042] This embodiment presents a method for predicting gene spatial expression based on tumor pathology images by combining convolutional and graph neural networks. The method includes:
[0043] Acquire pathological images and gene expression data from the spatial transcriptome and perform preprocessing;
[0044] Convolutional neural networks and graph neural networks were used to extract tissue texture information and cell structure features, respectively. A gene spatial expression prediction model based on deep learning model was constructed, and the model was trained using preprocessed pathological images and gene expression data.
[0045] Gene spatial expression prediction is performed using a trained gene spatial expression prediction model.
[0046] The method of this embodiment will now be described in detail.
[0047] Step 1: Dataset Preparation
[0048] Spatial transcriptomics data from tumor samples were collected for model training and evaluation. Spatial transcriptome sequencing was performed using either 10X Visium or ST technology. Histopathological images of the sequenced tissue regions were all stained with hematoxylin and eosin (HE). In 10X Visium, the distance between adjacent sequencing sites was 100 μm, and the diameter of each site was 55 μm; in ST technology, the distance between adjacent sequencing sites was 200 μm, and the diameter of each site was 100 μm.
[0049] Step 2: Preprocessing of pathological images stained with hematoxylin and eosin:
[0050] The entire hematoxylin-eosin stained pathological image was segmented into 200×200 pixel blocks with a resolution of 0.5 μm / pixel based on the coordinates of different sequencing sites in the spatial transcriptome. Reinhard color normalization was used to eliminate differences in staining between different tissue sections. A HoVer-Net model trained on the pan-cancer pathological image dataset PanNuke was used for nuclear region segmentation and cell type classification. Each segmented nucleus was assigned one of the following classification labels: background, tumor cells, inflammatory cells, connective tissue cells, dead cells, and non-tumor epithelial cells. One-Hot encoding was used to map the classification label of each segmented nucleus into a 6-dimensional vector; for example, tumor cells were represented as [0,1,0,0,0,0]. Features of each nuclear region were extracted using the Python package HistomicsTK, including 79-dimensional numerical features across multiple aspects such as morphology (size and shape), Fourier shape descriptor (FSD), intensity, gradient, and Haralick features. At the patient level, these 79-dimensional numerical features were normalized by removing the mean and scaling. Ultimately, 85 features were extracted for each cell nucleus.
[0051] Step 3: Construction of the cell nucleus map:
[0052] For each image patch obtained from segmenting the entire hematoxylin-eosin stained pathological image, a nucleus map was constructed to represent the geometric topology between cells in the tissue. The nucleus served as a node in the map, and the node features were 85-dimensional features of each nucleus. Considering communication between neighboring cells, if the distance between two nuclei was less than 20 μm (40 pixels), an edge was added between the two nodes. Figure 1 The flowcharts for pathological image preprocessing and cell nucleus map construction are shown.
[0053] Step 4: Gene expression data preprocessing:
[0054] Spatially variable genes (SVGs) in spatial transcriptome sequencing were selected using the SPARK-X method. SVGs expressed in more than 30% of sequencing sites across all tissue sections were then selected as target genes for prediction. The raw gene count data were transformed to approximate a normal distribution. The steps included: first, adding 1 to each gene count to construct a pseudo-count to avoid zero values during logarithmic transformation; second, performing gene expression normalization counting for each sequencing site, dividing the pseudo-count of each gene by the total pseudo-count of all genes at that site; third, scaling, multiplying all normalized count values by 1,000,000; and finally, logarithmic transformation.
[0055] Step 5, Model Architecture:
[0056] The constructed deep learning model IGI-DL contains two input branches and one fused output branch. The detailed architecture is provided by [details omitted]. Figure 2 The image branch uses a deep convolutional neural network with 18 ResNet layers to extract tissue texture information from 200×200 pixel pathological image patches and applies average pooling to output 512-dimensional features. The graph branch uses a four-layer graph isomorphic network (GIN) to aggregate neighbor node information in the nucleus graph and uses max & average global pooling to calculate the feature representation of each GIN layer. Each layer has a feature dimension of 256. The features from each layer are summed and passed through two fully connected neural networks with 128 and 64 neurons respectively to obtain 64-dimensional geometric features. In the fusion output branch, the geometric features extracted from the graph and the texture features extracted from the image are concatenated and then passed through three fully connected neural networks with 512, 256, and 256 neurons respectively, ultimately outputting the expression level of the target gene to be predicted.
[0057] Step Six: Model Training
[0058] The loss function used for model training is mean squared error (MSE), optimized using the Adam optimizer. The initial learning rate is set to 0.0002, and the training epochs are set to 300. Considering training speed and computer memory, the mini-batch size is set to 512. To avoid overfitting, a weight decay and early stopping strategy are employed. The weight decay is set to 0.0001, and training is interrupted if the loss function on the validation set does not decrease within 30 epochs.
[0059] Step 7: Model Evaluation
[0060] The trained deep learning model was used to predict the multivariate gene expression in different regions of tumor pathology images. The results were compared with the actual values obtained by sequencing each target gene. The Pearson correlation coefficient was calculated to evaluate the performance of the gene spatial expression prediction model. The Wilcoxon signed-rank test was used to compare the performance differences between different methods.
[0061] To demonstrate the high accuracy of the method proposed in this embodiment, experimental verification was conducted using spatial transcriptome data from actual tumor patients:
[0062] 1) Datasets: The data includes spatial transcriptomics data from three different cancers for model training and evaluation: colorectal cancer, breast cancer, and cutaneous squamous cell carcinoma. For colorectal cancer, spatial transcriptomics data from 6 patients with 10X Visium sequencing was used as the internal leave-one-out validation set, and 1 publicly available 10X Visium spatial transcriptomics dataset from colorectal cancer was used as the external test set. For breast cancer, publicly available spatial transcriptomics data from 36 tissue regions from 8 patients with ST sequencing was used as the internal leave-one-out validation set, and 4 tissue regions from 1 patient with ST sequencing was used as the external test set. For cutaneous squamous cell carcinoma, publicly available spatial transcriptomics data from 12 tissue regions from 4 patients with cutaneous squamous cell carcinoma was used as the internal leave-one-out validation set, and 4 tissue regions from 1 patient with 10X Visium spatial transcriptomics was used as the external test set. During the leave-one-out validation, data from one patient is selected as the internal test set, while data from the remaining patients are used as the training set. 20% of these samples are randomly selected as the validation set for the early stop method. This process is repeated for each patient in the leave-one-out validation set. A model is trained using all data from the internal leave-one-out validation set and then tested on the external test set to evaluate its performance.
[0063] 2) Statistical Analysis: For the above dataset, the model IGI-DL of this invention and three existing technology models, ST-Net, HisToGene, and Hist2ST, were trained. Then, the gene spatial expression prediction accuracy was compared. To compare the IGI-DL model with the three existing technology models, this example used the Pearson correlation coefficient as the evaluation index and the Wilcoxon signed-rank test to compare the differences in performance between different methods and calculate the p-value. All statistical analyses were performed using R software.
[0064] 3) Results:
[0065] 3.1 Prediction of spatial expression of genes in colorectal cancer
[0066] This embodiment first trains the IGI-DL model to predict the spatial expression of genes in colorectal cancer based on pathological images, and then compares it with three existing technology models. The model is trained using data from five patients in a leave-one-out validation set for colorectal cancer. Predictions are then performed on the remaining patient in the leave-one-out validation set, repeated six times. All patients are used as the internal test set. The average Pearson correlation coefficients of the 323 genes predicted by the IGI-DL model and the three comparative models are compared. The results are as follows: Figure 3 Subfigure a shows the results. In the data from 6 patients, the mean Pearson correlation coefficient of IGI-DL was 0.161, significantly better than other models, with an average increase of 0.094. For the external test set of colorectal cancer, the mean Pearson correlation coefficient between the true values of all target genes obtained by sequencing and the predicted values of IGI-DL was 0.146, which was an average improvement of 0.101 compared to the three comparative models. Figure 3 (as shown in subgraph b).
[0067] 3.2 Prediction of spatial expression of breast cancer genes
[0068] This embodiment trained an IGI-DL model to predict the spatial expression of genes in breast cancer based on pathological images, and then compared it with three existing technology models. The model was trained using data from seven patients in a leave-one-out validation set for breast cancer. Predictions were then performed on the remaining patient in the leave-one-out validation set, and this process was repeated eight times until all patients were included as the internal test set. The average Pearson correlation coefficients of the 321 genes predicted by the IGI-DL model and the three comparative models were compared. The results are as follows: Figure 4 Subfigure a is shown in the figure. In data from 36 tissue regions across 8 patients, the average Pearson correlation coefficient of IGI-DL was 0.200, significantly better than existing models, with an average improvement of 0.063. Of the 321 predicted genes in the internal validation set for breast cancer, 317 genes also existed in the external test set. The average Pearson correlation coefficient of these genes predicted by the IGI-DL model in the external test set was 0.213, significantly higher than the ST-Net and Hist2ST models, and with similar prediction accuracy to the HisToGene model. Figure 4 (as shown in subgraph b).
[0069] 3.3 Prediction of spatial expression of genes in cutaneous squamous cell carcinoma
[0070] This embodiment trains an IGI-DL model to predict the spatial expression of genes in cutaneous squamous cell carcinoma based on pathological images, and then compares it with three existing technology models. The model was trained using data from three patients in a leave-one-out validation set for cutaneous squamous cell carcinoma. Predictions were then performed on the remaining patient in the leave-one-out validation set, repeated four times until all patients were included as the internal test set. The average Pearson correlation coefficients of the 487 genes predicted by the IGI-DL model and the three comparative models were compared. The results are as follows: Figure 5 As shown in subfigure a, IGI-DL predicted target genes with an average Pearson correlation coefficient of 0.188 across 12 tissue regions from 4 patients, performing best among all models and improving upon other comparative models by an average of 0.129. Of the 487 predicted genes in the internal validation set for cutaneous squamous cell carcinoma, 467 were also present in the external test set. In the four samples of the external test set for cutaneous squamous cell carcinoma, the average Pearson correlation coefficient between the actual values of all target genes obtained through sequencing and the predicted values of IGI-DL was 0.191, significantly higher than the three existing technologies: ST-Net, HisToGene, and Hist2ST. Figure 5 (as shown in subgraph b).
[0071] 3.4 The top 5 genes with the best prediction accuracy
[0072] Table 1 lists the top 5 genes with the best prediction accuracy of IGI-DL in various cancer types in this embodiment, and their Pearson correlation coefficients are much higher than the results predicted by existing technology models.
[0073] Table 1
[0074]
[0075]
[0076] For the internal validation set of colorectal cancer, the average Pearson correlation coefficient of the first 5 genes in IGI-DL reached 0.374, an improvement of 0.113 compared to the best existing technology model (ST-Net) in this scenario. For the external validation set of colorectal cancer, the average Pearson correlation coefficient of the first 5 genes in IGI-DL was 0.424, an improvement of 0.172 compared to the best existing technology model (Hist2ST) in this scenario. For the internal validation set of breast cancer, the average Pearson correlation coefficient of the first 5 genes in IGI-DL reached 0.367, an improvement of 0.066 compared to the best existing technology model (ST-Net) in this scenario. For the external validation set of breast cancer, the average Pearson correlation coefficient of the first 5 genes in IGI-DL was 0.641, an improvement of 0.076 compared to the best existing technology model (ST-Net) in this scenario. For the internal validation set of cutaneous squamous cell carcinoma, the average Pearson correlation coefficient of the first 5 genes in IGI-DL reached 0.343, an improvement of 0.061 compared to the best existing technology model (Hist2ST) in this case. For the external validation set of cutaneous squamous cell carcinoma, the average Pearson correlation coefficient of the first 5 genes in IGI-DL was 0.436, an improvement of 0.259 compared to the best existing technology model (ST-Net) in this case. The best existing technology model is not fixed on the internal validation set and external test set for different cancers, but the IGI-DL model in this invention consistently outperforms the best results of existing technology models, with an average improvement of 0.125 in the average correlation of the first 5 genes.
[0077] The proposed IGI-DL model, which combines convolutional and graph neural networks to predict gene spatial expression based on tumor pathology images, has achieved excellent performance in three cancer types: colorectal cancer, breast cancer, and squamous cell carcinoma of the skin, significantly outperforming three existing models: ST-Net, HisToGene, and Hist2ST.
[0078] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for predicting gene space expression based on tumor pathology images, characterized by, The method includes: The pathological images and gene expression data from the spatial transcriptome were acquired and preprocessed, specifically as follows: The preprocessing of the pathological images includes: segmenting the hematoxylin-eosin stained tissue pathological images in the spatial transcriptome into small image blocks, segmenting the cell nucleus region in the image, extracting relevant features at the cell level, and constructing a cell nucleus map; The preprocessing of the gene expression data includes: selecting spatially variable genes using the SPARK-X method, and then selecting spatially variable genes expressed in more than k% of sequencing sites across all tissue sections as target genes to be predicted; transforming the original target gene count data to make it approximately follow a normal distribution, including: 1) adding 1 to each gene count data to construct a pseudo-count; 2) performing gene expression normalization counting for each sequencing site, dividing the pseudo-count of each gene by the total pseudo-count of all genes at that site; 3) scaling, multiplying all normalized count values by 1,000,000; 4) logarithmic transformation; Convolutional neural networks and graph neural networks were used to extract tissue texture information and cell structure features, respectively. A gene spatial expression prediction model based on deep learning model was constructed, and the model was trained using preprocessed pathological images and gene expression data. Gene spatial expression prediction is performed using a trained gene spatial expression prediction model. The method employs convolutional neural networks and graph neural networks to extract tissue texture information and cell structure features, respectively, and constructs a gene spatial expression prediction model based on a deep learning model, specifically as follows: The gene spatial expression prediction model includes two input branches and one fused output branch. The input branches include an image branch and a graph branch; wherein: The image branch contains a deep convolutional neural network with 18 ResNet convolutional layers to extract tissue texture information from pathological image blocks of 200×200 pixels, and applies average pooling to output 512-dimensional features. The graph branch uses a four-layer graph isomorphic network to aggregate neighbor node information in the Nuclei-Graph, and uses global pooling of maximum and average values to calculate the feature representation of each graph isomorphic network GIN layer; The fusion output branch, through a two-layer fully connected neural network, fuses the texture features extracted by the image branch and the geometric features extracted by the graph branch to output the spatial gene expression level of the target to be predicted.
2. The method of claim 1, wherein the method is based on a tumor pathology image. The process of segmenting hematoxylin-eosin stained histopathological images from spatial transcriptomics into small image blocks, dividing the cell nucleus region within the images, extracting relevant features at the cellular level, and constructing a cell nucleus map specifically involves: Based on the coordinates of different sequencing sites in the spatial transcriptome, the entire hematoxylin-eosin stained pathological image was divided into small image blocks with a size of 200×200 pixels and a resolution of 0.5 / pixel, and then the colors were normalized. A pre-trained HoVer-Net model was used for cell nucleus region segmentation and cell type classification. One-hot encoding is used to map the classification label of each segmentation kernel; After extracting and standardizing the features of each nuclear region, multidimensional nuclear features are obtained; the features include morphology, Fourier shape descriptor, intensity, gradient and Haralick features.
3. The gene spatial expression prediction method based on tumor pathology images according to claim 2, characterized in that, For each image patch, a nucleus map is constructed to represent the geometric topology between cells in the tissue. The nucleus serves as a node in the map, and the node features are multidimensional nucleus features of each nucleus.
4. The method of claim 3, wherein the method is based on a tumor pathology image. If the distance between two cell nuclei is less than 20 μm, then add an edge between the two nodes.
5. The method of claim 1, wherein the method is based on a tumor pathology image. The loss function used in training the gene spatial expression prediction model based on deep learning is mean squared error, and the Adam optimizer is used for optimization.
6. The method of claim 5, wherein the method is based on a tumor pathology image. The gene spatial expression prediction model based on deep learning is trained and optimized using a weight decay and early stopping strategy.