Construction method and device of tumor virtual three-dimensional transcriptome, equipment and storage medium

Abstract

Embodiments of the present application disclose a method, device and equipment for constructing a tumor virtual three-dimensional transcriptome and a storage medium. The method comprises: inputting a first slice tissue image containing tumor tissue into a first neural network model to obtain first gene expression data of a spatial transcriptome of the first slice tissue; performing cell type annotation on the first gene expression data to obtain second gene expression data; inputting the second gene expression data into a second neural network model to obtain third gene expression data of a spatial transcriptome of a second slice tissue, the first slice tissue being adjacent to the second slice tissue in the tumor tissue; the third gene expression data being labeled with label information of a cell type; and constructing a virtual three-dimensional transcriptome of the tumor tissue according to the second gene expression data and the third gene expression data. The present application constructs a tumor virtual three-dimensional transcriptome based on a digital image processing technology, and provides a possibility for exploring disease occurrence mechanisms and researching treatment methods.

Description

Technical Field

[0001] This invention relates to the field of medical auxiliary technology, and in particular to a method, apparatus, device and storage medium for constructing a virtual three-dimensional transcriptome of a tumor. Background Technology

[0002] Tumor pathological and histological evaluation helps identify cell-related characteristics, including cell type, tumor invasion pattern and grade, cell viability, and the presence and quantity of other cell populations (such as inflammatory cells and fibroblasts). It also allows for immunohistochemical analysis and target protein expression level analysis. Changes in gene expression are associated with the development and progression of many tumors, and tumor-related genetic features help elucidate the mechanisms of tumor development and progression, determine treatment methods, and predict prognosis. Furthermore, changes in gene expression are also related to cell type, proliferation, and differentiation status, and these cell-related characteristics can be assessed through histopathological examination.

[0003] One study used deep learning combined with transcriptome data from TCGA (The Cancer Genome Atlas) to predict the expression of 28 tumor RNA-Seq samples from WSI (Whole Slide Image), ultimately successfully predicting the expression of an average of 3627 genes. However, ordinary transcriptome data cannot obtain transcriptome levels across different cells or different tumor sites. Although different cells share the same genome, their gene expression patterns can be drastically different, and even the same type of cells in different locations within a tumor can exhibit different gene expression patterns.

[0004] To address cellular and spatial heterogeneity, existing technologies include single-cell sequencing and spatial transcriptome sequencing. Single-cell sequencing enables the analysis and comparison of gene expression patterns in a large number of individual cells within tissues and organs, playing a crucial role in discovering novel cell types and laying the foundation for creating comprehensive cellular atlases across different species. However, single-cell sequencing requires mechanical and enzymatic dissociation to obtain single-cell suspensions, disrupting the original tissue structure and resulting in the loss of cell location information. Spatial transcriptome sequencing, on the other hand, can obtain transcriptome-level data at each location, facilitating research on how cells communicate with their tissue environment, but its ability to differentiate cell types is inferior to that of single-cell sequencing.

[0005] Therefore, how to combine single-cell sequencing with spatial transcriptomics to spatially locate different single-cell subpopulations and effectively obtain the transcriptome level of each cell type at each location in tumor tissue is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] This invention provides a method, apparatus, device, and storage medium for constructing a virtual three-dimensional transcriptome of a tumor. By combining single-cell sequencing with spatial transcriptomics to construct a virtual three-dimensional transcriptome of a tumor, it is helpful to understand the pathogenesis of tumors. While reducing time and cost, it can provide a reliable basis for clinical diagnosis, treatment, and prognostic decisions for tumor patients.

[0007] In a first aspect, embodiments of the present invention provide a method for constructing a virtual three-dimensional transcriptome of a tumor, comprising:

[0008] The first tissue slice image containing tumor tissue is input into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first tissue slice; wherein, the tumor tissue includes a first tissue slice and a second tissue slice, and the first tissue slice is adjacent to the second tissue slice in the tumor tissue;

[0009] Based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain the second gene expression data;

[0010] The second gene expression data is input into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second tissue slice; wherein the third gene expression data is labeled with the cell type tag information;

[0011] A virtual three-dimensional transcriptome of the tumor tissue is constructed based on the expression data of the second gene and the expression data of the third gene.

[0012] Secondly, embodiments of the present invention provide an apparatus for constructing a virtual three-dimensional transcriptome of a tumor, comprising:

[0013] The first input unit is used to input a first slice image containing tumor tissue into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first slice tissue; wherein, the tumor tissue includes a first slice tissue and a second slice tissue, and the first slice tissue is adjacent to the second slice tissue in the tumor tissue;

[0014] The first annotation unit is used to annotate the first gene expression data with cell type based on single-cell sequencing to obtain the second gene expression data.

[0015] The second input unit is used to input the second gene expression data into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second slice tissue; wherein the third gene expression data is labeled with the cell type tag information;

[0016] The first building unit is used to construct a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data.

[0017] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for constructing a virtual three-dimensional transcriptome of a tumor as described in the first aspect above.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the method for constructing a virtual three-dimensional transcriptome of a tumor as described in the first aspect.

[0019] This invention provides a method, apparatus, device, and storage medium for constructing a virtual three-dimensional transcriptome of a tumor. Based on digital image processing and machine learning technologies, this method directly correlates specific molecular features with pathological morphological patterns to rapidly acquire tumor spatial transcriptome information, enabling prediction of tumor spatial transcriptome levels in pathological images. Simultaneously, by combining single-cell sequencing results to construct a virtual three-dimensional transcriptome of the tumor, it is expected to elucidate the interactions between tumor cells and specific cell types, map the characteristics of the tumor microenvironment, and contribute to understanding the pathogenesis of tumors. Furthermore, it aims to provide a reliable basis for clinical diagnosis, treatment, and prognostic decisions for cancer patients while reducing time and cost. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a sub-process of the method for constructing a virtual three-dimensional transcriptome of a tumor provided in an embodiment of the present invention;

[0023] Figure 3 This is another flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0024] Figure 4A This is a schematic diagram of a pathological tissue image before preprocessing, provided in an embodiment of the present invention.

[0025] Figure 4B This is a denoised image of a pathological tissue provided in an embodiment of the present invention.

[0026] Figure 4C This is a diagram showing the effect of background segmentation of a pathological tissue image provided in an embodiment of the present invention.

[0027] Figure 5 This is a schematic diagram of the process for generating expression data of a normal transcriptome provided in an embodiment of the present invention;

[0028] Figure 6 This is another flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the network structure of ResNet50 provided in an embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the structure of the global average pooling layer provided in an embodiment of the present invention;

[0031] Figure 9 This is another flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0032] Figure 10 This is another flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0033] Figure 11 This is a schematic diagram of the process for generating expression data of the ninth gene provided in an embodiment of the present invention;

[0034] Figure 12 This is another flowchart illustrating the method for constructing a tumor virtual three-dimensional transcriptome provided in an embodiment of the present invention;

[0035] Figure 13 A schematic block diagram of a device for constructing a virtual three-dimensional transcriptome of a tumor provided in an embodiment of the present invention;

[0036] Figure 14 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0039] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0040] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0041] Please see Figure 1 , Figure 1 This is a flowchart illustrating the method for constructing a virtual three-dimensional transcriptome of a tumor according to an embodiment of the present invention. The method for constructing a virtual three-dimensional transcriptome of a tumor according to this embodiment is applied to a terminal device, and the method is executed through application software installed on the terminal device. The terminal device is a terminal device with internet access capability, such as a desktop computer, laptop computer, tablet computer, or mobile phone.

[0042] The method for constructing a virtual three-dimensional transcriptome of a tumor is described in detail below.

[0043] like Figure 1 As shown, the method includes the following steps S110 to S140.

[0044] S110. Input the first slice image containing tumor tissue into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first slice tissue; wherein, the tumor tissue includes a first slice tissue and a second slice tissue, and the first slice tissue is adjacent to the second slice tissue in the tumor tissue.

[0045] Specifically, the first tissue slice is a portion of the tumor tissue after multiple slices. The first neural network model is a pre-trained system capable of generating the first gene expression data of the spatial transcriptome of the corresponding tissue slice from the tissue slice image. The first gene expression data is the gene expression data of the spatial transcriptome of the first tissue slice predicted by the first neural network model based on the first tissue slice image. After the first tissue slice image is processed by digital image processing techniques in the first neural network model, the first gene expression data of the spatial transcriptome of the first tissue slice can be obtained. The first neural network model can employ a ResNet50 network structure. The trained ResNet50 network structure can extract features from the tissue slice image and then predict the expression data of the spatial transcriptome of the tissue slice based on the extracted feature information.

[0046] In another embodiment, such as Figure 2 As shown, the training method for the first neural network model includes steps S210 and S220.

[0047] S210. The first neural network model is trained based on the fourth gene expression data of the ordinary transcriptome of the tumor tissue sample and the pathological tissue image to obtain the first neural network model after the first stage of training; wherein, the fourth gene expression data is the data after the transcriptome sequencing of the tumor tissue sample.

[0048] Specifically, the fourth gene expression data refers to the gene expression data obtained by processing tumor tissue samples using whole transcriptome sequencing technology (RNA-Seq). This fourth gene expression data serves as the sample label for training the first neural network model in the first training phase. In the first training phase, the fourth gene expression data and pathological images are simultaneously input into the first neural network model, thus enabling the first stage of training. After completing the first stage of training, the first neural network model can predict the gene expression data of the ordinary transcriptome of the tumor tissue sample.

[0049] In another embodiment, such as Figure 3 As shown, step S210 includes steps S211, S212, S213, S214 and S215.

[0050] S211. Preprocess the pathological tissue image to obtain multiple small block images of the pathological tissue image;

[0051] S212. Perform feature extraction on each of the small image blocks to obtain the feature information of each of the small image blocks;

[0052] S213. Generate the expression data of the fifth gene of the ordinary transcriptome of the tumor tissue sample based on the aforementioned feature information;

[0053] S214. Generate the mean square error of the first neural network model based on the expression data of the fourth gene and the expression data of the fifth gene;

[0054] S215. Update the network parameters of the first neural network model according to the mean square error to obtain the first neural network model after the first stage of training.

[0055] Specifically, such as Figure 4A As shown, the preprocessing of pathological tissue images includes removing noise such as handwriting from the image, and separating the foreground and background of the tissue. Since the original pathological images are usually very large, in order to speed up the processing, the preprocessing is first performed on the pathological thumbnail. Then, the foreground part of the thumbnail is used as a mask, enlarged to the size of the original image, and applied to the original pathological image. The pathological tissue image is then divided into multiple small images, thereby enabling the extraction of the foreground of the pathological tissue image.

[0056] In some embodiments, preprocessing of pathological tissue images may include image scaling, image denoising, background segmentation, enlarging the foreground thumbnail to its original size and removing noise and background, and cropping into multiple small image blocks.

[0057] The pathological tissue images were selected as 20x magnified WSI (Whole Slide Image) images, ranging from 19,920 to 198,220 pixels with an average of 101,688 pixels; the height ranged from 13,347 to 256,256 pixels with an average of 73,154 pixels. To accelerate processing, the pathological images were reduced by a factor of 32. Specifically, from left to right and top to bottom along the top left corner of the pathological tissue image, each 32x32 pixel block corresponded to one pixel of the reduced image. The RGB color values ​​of these pixels corresponded to the average color values ​​in the RGB space of the 32x32 pixel block. Furthermore, at the edges of the original image where the width or height was less than 32 pixels, only the corresponding width or height pixel block was used.

[0058] The primary purpose of image denoising after image scaling is to remove handwriting from pathological tissue images. This can be achieved by using multiple threshold filters across the RGB color channels. For example, to remove green handwriting, a threshold can be applied where the red channel value is greater than 150, the green channel value is less than 160, and the blue channel value is less than 140, denoted as (R>150, G<160, B<140). Combining multiple filters, such as (R>70, G<110, B<110) or (R>195, G<220, B<210), can effectively remove handwriting. The resulting image with handwriting removed is shown below. Figure 4B As shown.

[0059] It should be noted that when removing handwriting from pathological tissue images, the threshold of the RGB channel can be set according to the color and depth of the handwriting to achieve the removal of handwriting from pathological tissue images. The specific removal method can be selected according to the actual application, and this application does not make specific limitations.

[0060] In this embodiment, background segmentation mainly involves separating the tissue portion from the background portion of the pathological tissue image. After removing the handwriting from the image, the pathological tissue image is converted to grayscale, then color inversion is performed. The background of the pathological tissue image is set to dark, and the foreground to light. The Otsu algorithm is then used to determine the optimal threshold for background segmentation. Simultaneously, a truncation method is used to separate the foreground portion of the pathological tissue image. Finally, the foreground mask of the grayscale image is applied to the RGB image to obtain the desired result. Figure 4C The effect shown.

[0061] Specifically, enlarging the foreground thumbnail of a pathological tissue image to its original size can be achieved using a bilinear interpolation algorithm. After enlarging the foreground thumbnail to its original size, noise and background in the foreground need to be removed. This can be done by using the enlarged foreground image as a mask for the original image, with each pixel in the enlarged foreground image corresponding to a pixel in the original image. Then, the pixels in the original image corresponding to the non-foreground portion of the foreground thumbnail are set to RGB(0,0,0), thus removing noise and background from the foreground.

[0062] Meanwhile, after the foreground thumbnail is enlarged to its original size and noise and background are removed, it needs to be cropped into multiple small images. Specifically, it can be cropped into small images of 256×256μm (512×512 pixels). Then, the Monte Carlo algorithm can be used to select small images with more than 50% tissue for training.

[0063] It should be noted that the maximum number of small image patches selected is 8000. The percentage of tumor tissue in a small image patch is calculated by dividing the total number of foreground pixels in the small image patch by the total number of pixels in the small image patch. The formula is as follows:

[0064]

[0065] Where Ratio is a percentage, P ij Let be the pixel at position i*j.

[0066] In addition, since pathological tissue images are usually generated after sectioning, and pathological sections are usually stained with hematoxylin-eosin, the staining depth of hematoxylin-eosin will vary for different cells. In order to reduce the impact of color differences on the subsequent fitting model, small images need to be color normalized before feature extraction.

[0067] This embodiment uses the Macenko normalization method from Python's Staintools package to normalize the color of all small image patches. The specific steps are as follows: First, specify the small image patch to which the target color needs to be converted. Then, use the `Read_Image` method of Staintools to read the image. Next, use the `StainNormalizer` method of Staintools to define a normalization operation instance, and use the FIT method to fit the small image patch. Finally, apply the normalization to other small image patches.

[0068] In some embodiments, after feature extraction is completed from small image patches, the expression data of the fifth gene in the common transcriptome of tumor tissue samples can be predicted using a multi-layer neural network. Figure 5 In the illustrated embodiment, after feature extraction, the output feature information of each small image patch is a one-dimensional vector with dimension (2048,). The feature information of each small image patch is input into a multi-layer neural network. First, in the Dense layer, it is processed sequentially through the activation function layer (BN), the ReLU function layer, and the Dropout layer in the form of a (2048, 1024) multi-dimensional vector to generate a multi-dimensional vector with dimension (1024, 512). Then, it is processed again in the Dense layer sequentially through the activation function layer (BN layer), the ReLU function layer, and the Dropout layer. Thus, the expression data of the fifth gene of the ordinary transcriptome of the tumor tissue sample can be obtained, which is output in the form of (512, N_GenePred). Among them, the expression data output by each small image patch in the fifth gene expression data is (N_GenePred,).

[0069] In this embodiment, the fifth gene expression data is the gene expression data predicted by a multi-layer neural network. After the fifth gene expression data is predicted by the multi-layer neural network, the mean square error of the first neural network model can be generated based on the fourth gene expression data. Then, the network parameters of the first neural network model can be updated based on the mean square error to perform the first stage of training on the first neural network model.

[0070] The mean square error function is:

[0071]

[0072] Where m is the number of input image patches, Y represents the expression data of the fourth gene, used to predict the expression data of the fifth gene.

[0073] In addition, during the training of the first neural network model, the cross-entropy loss function can be used to adjust the network parameters of the first neural network model, thereby avoiding the problem of decreased learning rate caused by the mean squared error loss function.

[0074] The cross-entropy loss function is:

[0075]

[0076] Where m is the number of input image patches, y represents the expression data of the fifth gene, and y represents the expression data of the fourth gene.

[0077] In another embodiment, such as Figure 6 As shown, step S212 includes steps S2121 and S2122.

[0078] S2121. Perform a convolution operation on each of the small image blocks to obtain the convolution features of each of the small image blocks;

[0079] S2122. Perform pooling operation on the convolutional features to obtain feature information for each small image block.

[0080] In some embodiments, each small image patch can be used for feature extraction using a ResNet50 network, such as Figure 7 As shown, the ResNet50 network performs feature extraction on small image patches in five stages: STAGE0, STAGE1, STAGE2, STAGE3, and STAGE4. Here, (3, 224, 224) refers to the number of channels, height, and width of the input INPUT, i.e., (C, H, W).

[0081] The first layer in STAGE0 includes three sequential operations: CONV, BN, and RELU. CONV is short for Convolution, 7×7 refers to the kernel size, 64 refers to the number of kernels (i.e., the number of output channels of this convolutional layer), and / 2 refers to the stride of the kernels being 2. BN is short for Batch Normalization, which is the commonly referred to BN layer. RELU refers to the RELU activation function. The second layer in STAGE0 is MAXPOOL, which is the max pooling layer. Its kernel size is 3×3 and its stride is 2. (64, 56, 56) are the number of output channels, height, and width of STAGE0. Here, 64 is equal to the number of kernels in the first convolutional layer of STAGE0, and 56 is equal to 224 / 2 / 2 (a stride of 2 will halve the input size). In Stage 0, the input of shape (3, 224, 224) passes through a convolutional layer, a BN layer, a ReLU activation function, and a MaxPooling layer to obtain an output of shape (64, 56, 56).

[0082] In Stage 1, the input shape is (64, 56, 56), and the output shape is (64, 56, 56). BTNK is an abbreviation for BottleNeck, and the two types of BTNK correspond to two cases: the number of input and output channels are the same (BTNK2), and the number of input and output channels are different (BTNK1).

[0083] BTNK1 has four variable parameters C, W, C1, and S. Compared to BTNK2, BTNK1 has one more convolutional layer on the right, which can be denoted as a function G(x). BTNK1 addresses the case where the number of channels in the input x and the output F(x) are different. It is this added convolutional layer that transforms x into G(x), thus matching the difference in the dimensions of the input and output. Since G(x) and F(x) have the same number of channels, they can be summed as G(x) + F(x).

[0084] BTNK2 has two variable parameters, C and W, which are c and W in the shape (C,W,W) of the input. Let the input with shape (C,W,W) be x, and let the three convolutional blocks on the left side of BTNK2 (as well as the associated BN and ReLU) be functions F(x). The sum of the two, F(x)+x, is then passed through a ReLU activation function to obtain the output of BTNK2. The shape of this output is still (C,W,W), which is the case mentioned above where the number of channels of the input x and the output F(x) are the same.

[0085] In addition, after feature extraction using the ResNet50 network for each small image patch, it is necessary to use methods such as... Figure 8The Global Average Pooling (GAP) layer structure shown is used to perform pooling operations, thereby obtaining the feature information of each small image patch.

[0086] S220. The first neural network model trained in the first stage is trained according to the preset first sample to obtain the first neural network model trained in the second stage; wherein, the first sample includes the tissue slice image of the tumor tissue sample and the gene expression data after spatial transcriptome sequencing of the corresponding tissue slice.

[0087] Specifically, after the first stage of training, the first neural network model can only predict gene expression data from the general transcriptome, not from the spatial transcriptome. Therefore, a second stage of training is needed to enable the first neural network model to predict gene expression data from the spatial transcriptome. The first sample pair includes tissue images of tumor tissue slices and the expression data obtained after spatial transcriptome sequencing of the corresponding tissue slices. That is, a tissue slice and its corresponding expression data are considered a sample pair. The expression data obtained after spatial transcriptome sequencing of the corresponding tissue slices can be obtained by performing spatial transcriptome sequencing on sequentially sliced ​​tumor tissue.

[0088] S120. Based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain the second gene expression data.

[0089] Single-cell sequencing is a high-throughput experimental technique that uses RNA sequencing to quantify the gene expression profile of a specific cell population at the single-cell level. After single-cell isolation, RNA extraction, reverse transcription, library construction, and sequencing, the gene expression profile of multiple cells can be obtained through data analysis. Conventional transcriptomics uses samples composed of a mixture of cells for sequencing, therefore it can only estimate the average gene expression level in the cell population, without considering the heterogeneity of gene expression in individual cells within the sample. It cannot analyze heterogeneous systems in early developmental tissues or complex tissues (such as brain tissue). Single-cell sequencing technology uses a single cell as the object, uniformly amplifying the genetic material of a single cell, labeling and constructing a library, and then sequencing it. Finally, it performs data analysis on the genome or transcriptome of a single cell. Its technical principles mainly include three aspects: single-cell isolation, amplification sequencing, and data analysis.

[0090] In this embodiment, the second gene expression data can be obtained by labeling the first gene expression data with cell types using data generated from single-cell sequencing of tumor tissue. The second gene expression data is gene expression data labeled with cell types.

[0091] In another embodiment, such as Figure 9 As shown, step S120 includes steps S121 and S122.

[0092] S121. Obtain the cell type of each single cell after single-cell sequencing of the tumor tissue, the expression data of the sixth gene corresponding to each single cell, and the variance of the change in the expression data of the sixth gene.

[0093] S122. The first gene expression data is labeled according to the cell type, the sixth gene expression data, and the variance of the variation to obtain the second gene expression data.

[0094] In this embodiment, after single-cell sequencing of the tumor tissue, the cell types and corresponding gene expression data of each cell in the tumor tissue can be obtained. i Gene j Expression) and the variance of the expressed data (Cell) i Gene j Variance), through first gene expression data Gene i Expression and corresponding gene expression data (Cell) i Gene j The expression data is compared one by one. If the difference value of each gene is within the range of the variance of the cellular gene expression data, then the spatial transcriptome spot is determined. k For this type of cell, the formula is:

[0095]

[0096] S130. Input the second gene expression data into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second slice tissue; wherein, the third gene expression data is labeled with the cell type tag information.

[0097] Specifically, the second neural network model is a trained system capable of predicting the expression data of the third gene in the spatial transcriptome of a second tissue slice adjacent to the first tissue slice. This third gene expression data includes cell type labeling information.

[0098] It is understandable that the second neural network model can predict any slice tissue adjacent to the first slice tissue. The second slice tissue can be located above or below the first slice tissue.

[0099] In addition, the second neural network model can output gene expression data of the tissue adjacent to the first tissue slice, and can also output gene expression data of the two tissue slice groups at the same time. The specific choice can be made according to the actual application, and this embodiment does not make specific limitations.

[0100] In another embodiment, such as Figure 10 As shown, the training method for the second neural network model includes steps S310 and S320.

[0101] S310. Obtain a second sample pair; wherein, the second sample pair includes the seventh gene expression data of the spatial transcriptome of the third tissue slice and the eighth gene expression data of the spatial transcriptome of the fourth tissue slice, the seventh gene expression data and the eighth gene expression data are both labeled with the cell type tag information, the third tissue slice and the fourth tissue slice are adjacent, and the eighth gene expression data is a sample tag;

[0102] S320. Train the second neural network model based on the second sample to obtain the trained second neural network model.

[0103] Specifically, the second neural network model can adopt a three-layer feedforward neural network structure, which includes an input layer, a hidden layer, and an output layer. Figure 11 In the illustrated embodiment, after cell type labeling, the gene expression data of each small image patch can be represented as a vector of (N_GenePred+1, ), where N_GenePred represents the gene expression data of the spatial transcriptome, and 1 represents the cell type annotation. After inputting multiple vectors of this form into the second neural network model, they are processed sequentially in the Dense layer as a multidimensional vector of (N_GenePred+1, 512) through an activation function layer (BN layer), a ReLU function layer, and a Dropout layer. This yields the gene expression data of the spatial transcriptome of the tissue slice, which is output in the form of (512, N_GenePred+1).

[0104] S140. Construct a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data.

[0105] Specifically, the virtual three-dimensional transcriptome consists of gene expression data from the spatial transcriptome and a map labeled with cell types. Since the second and third gene expression data are both labeled with cell types, after predicting the gene expression data with cell labels in the first and second tissue sections, they can be stacked layer by layer to construct the virtual three-dimensional transcriptome of the tumor tissue.

[0106] In another embodiment, such as Figure 12 As shown, step S140 includes steps S141 and S142.

[0107] S141. Input the second gene expression data into a pre-trained third neural network model to obtain the ninth gene expression data of the spatial transcriptome of the third slice tissue; wherein, the ninth gene expression data is labeled with the cell type tag information;

[0108] S142. Construct the virtual three-dimensional transcriptome based on the second gene expression data, the third gene expression data, and the ninth gene expression data.

[0109] Specifically, the tumor tissue is serially sliced, including not only the first and second slices but also a third slice. The third slice is adjacent to the first slice but not to the second slice. In this embodiment, the first neural network model can only predict gene expression data from one slice adjacent to the first slice. Therefore, a third neural network model needs to be constructed to predict gene expression data from the other adjacent slice, thus enabling the construction of a virtual three-dimensional transcriptome of the tumor tissue. The expression data of the ninth gene is also labeled with cell type information.

[0110] It should be noted that the third neural network model can have the same network structure as the second neural network model. Furthermore, this application can construct a virtual three-dimensional transcriptome of the tumor tissue using only one slice of tissue image, or it can use all slices of tissue image from the tumor tissue. Specifically, when using only one slice of tissue image to construct the virtual three-dimensional transcriptome, after predicting the gene expression data of adjacent slices, the gene expression data of adjacent slices can be predicted again iteratively based on the predicted gene expression data, thereby achieving the construction of the virtual three-dimensional transcriptome of the tumor tissue.

[0111] In addition, after constructing a virtual three-dimensional transcriptome of tumor tissue, the virtual three-dimensional transcriptome can be corrected using gene expression data of the spatial transcriptome of tumor tissue and the corresponding cell type, thereby making the constructed virtual three-dimensional transcriptome more accurate.

[0112] In the method for constructing a virtual three-dimensional transcriptome of tumors provided in this invention, a first tissue slice image containing tumor tissue is input into a pre-trained first neural network model to obtain first gene expression data of the spatial transcriptome of the first tissue slice. The tumor tissue includes a first tissue slice and a second tissue slice, with the first tissue slice adjacent to the second tissue slice within the tumor tissue. Based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain second gene expression data. The second gene expression data is input into a pre-trained second neural network model to obtain third gene expression data of the spatial transcriptome of the second tissue slice. The third gene expression data is labeled with cell type information. A virtual three-dimensional transcriptome of the tumor tissue is constructed based on the second gene expression data and the third gene expression data. This invention not only achieves rapid quantification of tumor molecular feature expression, reducing the time and cost of molecular detection, but also enables automatic quantitative assessment of the tumor spatial transcriptome, which can assist pathologists in better tumor diagnosis and molecular subtyping, treatment decisions, and prognostic prediction. Furthermore, it also enables the prediction of three-dimensional transcriptomes, providing possibilities for exploring disease development mechanisms and researching treatment methods.

[0113] This invention also provides a device 100 for constructing a virtual three-dimensional transcriptome of a tumor, which is used to perform any of the aforementioned methods for constructing a virtual three-dimensional transcriptome of a tumor.

[0114] Specifically, please refer to Figure 13 , Figure 13 This is a schematic block diagram of the tumor virtual three-dimensional transcriptome construction device 100 provided in an embodiment of the present invention.

[0115] like Figure 13 As shown, the tumor virtual three-dimensional transcriptome construction device 100 includes: a first input unit 110, a first annotation unit 120, a second input unit 130, and a first construction unit 140.

[0116] The first input unit 110 is used to input a first slice image containing tumor tissue into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first slice tissue; wherein, the tumor tissue includes a first slice tissue and a second slice tissue, and the first slice tissue is adjacent to the second slice tissue in the tumor tissue.

[0117] In other embodiments of the invention, the apparatus for constructing a tumor virtual three-dimensional transcriptome further includes: a first training unit and a second training unit.

[0118] The first training unit is used to train the first neural network model based on the fourth gene expression data of the ordinary transcriptome of the tumor tissue sample and the pathological tissue image to obtain the first neural network model after the first stage of training; wherein, the fourth gene expression data is the data after transcriptome sequencing of the tumor tissue sample; the second training unit is used to train the first neural network model after the first stage of training based on a preset first sample to obtain the first neural network model after the second stage of training; wherein, the first sample includes the gene expression data after spatial transcriptome sequencing of the tissue slice image of the tumor tissue sample and the corresponding tissue slice.

[0119] In other embodiments of the invention, the first training unit includes: a preprocessing unit, a feature extraction unit, a first generation unit, a second generation unit, and an update unit.

[0120] A preprocessing unit is used to preprocess the pathological tissue image to obtain multiple small patch images of the pathological tissue image; a feature extraction unit is used to extract features from each of the small patch images to obtain feature information of each of the small patch images; a first generation unit is used to generate the expression data of the fifth gene of the ordinary transcriptome of the tumor tissue sample based on the feature information; a second generation unit is used to generate the mean squared error of the first neural network model based on the expression data of the fourth gene and the expression data of the fifth gene; and an update unit is used to update the network parameters of the first neural network model based on the mean squared error to obtain the first neural network model after the first stage of training.

[0121] In other embodiments of the invention, the feature extraction unit includes a convolution unit and a pooling unit.

[0122] A convolution unit is used to perform a convolution operation on each of the small image blocks to obtain the convolution features of each small image block; a pooling unit is used to perform a pooling operation on the convolution features to obtain the feature information of each small image block.

[0123] The first annotation unit 120 is used to annotate the first gene expression data with cell type based on single-cell sequencing to obtain the second gene expression data.

[0124] In other embodiments of the invention, the first annotation unit 120 includes: a first acquisition unit and a second annotation unit.

[0125] The first acquisition unit is used to acquire the cell type of each single cell after single-cell sequencing of the tumor tissue, the expression data of the sixth gene corresponding to each single cell, and the variance of the change of the sixth gene expression data; the second annotation unit is used to annotate the first gene expression data according to the cell type, the sixth gene expression data, and the variance of the change to obtain the second gene expression data.

[0126] The second input unit 130 is used to input the second gene expression data into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second slice tissue; wherein the third gene expression data is labeled with the cell type tag information.

[0127] In other embodiments of the invention, the apparatus for constructing a virtual three-dimensional transcriptome of a tumor further includes a second acquisition unit and a third training unit.

[0128] The second acquisition unit is used to acquire a second sample pair; wherein the second sample pair includes the expression data of the seventh gene of the spatial transcriptome of the third tissue slice and the expression data of the eighth gene of the spatial transcriptome of the fourth tissue slice, the expression data of the seventh gene and the expression data of the eighth gene are both labeled with the cell type tag information, the third tissue slice and the fourth tissue slice are adjacent, and the expression data of the eighth gene is a sample tag; the third training unit is used to train the second neural network model according to the second sample pair to obtain the trained second neural network model.

[0129] Construction unit 140 is used to construct a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data.

[0130] In other embodiments of the invention, the first building unit 140 includes: a third input unit and a second building unit.

[0131] The third input unit is used to input the second gene expression data into a pre-trained third neural network model to obtain the ninth gene expression data of the spatial transcriptome of the third slice tissue; wherein the ninth gene expression data is labeled with the cell type tag information; the second construction unit is used to construct the virtual three-dimensional transcriptome based on the second gene expression data, the third gene expression data and the ninth gene expression data.

[0132] The tumor virtual three-dimensional transcriptome construction apparatus 100 provided in this embodiment of the invention is used to perform the above-mentioned inputting a first tissue slice image containing tumor tissue into a pre-trained first neural network model to obtain first gene expression data of the spatial transcriptome of the first tissue slice; wherein, the tumor tissue includes a first tissue slice and a second tissue slice, and the first tissue slice is adjacent to the second tissue slice in the tumor tissue; based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain second gene expression data; the second gene expression data is input into a pre-trained second neural network model to obtain third gene expression data of the spatial transcriptome of the second tissue slice; wherein, the third gene expression data is labeled with the cell type tag information; and a virtual three-dimensional transcriptome of the tumor tissue is constructed based on the second gene expression data and the third gene expression data.

[0133] Please see Figure 14 , Figure 14 This is a schematic block diagram of a computer device provided in an embodiment of the present invention.

[0134] See Figure 14 The device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501, wherein the memory may include a storage medium 503 and internal memory 504.

[0135] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it enables the processor 502 to execute a method for constructing a virtual three-dimensional transcriptome of a tumor.

[0136] The processor 502 provides computing and control capabilities to support the operation of the entire device 500.

[0137] The internal memory 504 provides an environment for the execution of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for constructing a virtual three-dimensional transcriptome of a tumor.

[0138] This network interface 505 is used for network communication, such as providing data transmission. Those skilled in the art will understand that... Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the device 500 to which the present invention is applied. The specific device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following functions: inputting a first tissue slice image containing tumor tissue into a pre-trained first neural network model to obtain first gene expression data of the spatial transcriptome of the first tissue slice; wherein the tumor tissue includes a first tissue slice and a second tissue slice, and the first tissue slice is adjacent to the second tissue slice in the tumor tissue; based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain second gene expression data; inputting the second gene expression data into a pre-trained second neural network model to obtain third gene expression data of the spatial transcriptome of the second tissue slice; wherein the third gene expression data is labeled with the cell type tag information; and constructing a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data.

[0140] Those skilled in the art will understand that Figure 14 The embodiments of device 500 shown do not constitute a limitation on the specific configuration of device 500. In other embodiments, device 500 may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, in some embodiments, device 500 may include only a memory and processor 502. In such embodiments, the structure and function of the memory and processor 502 are similar to those shown. Figure 14 The embodiments shown are consistent and will not be repeated here.

[0141] It should be understood that, in this embodiment of the invention, the processor 502 may be a Central Processing Unit (CPU), or it may be other general-purpose processors 502, digital signal processors 502 (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor 502 may be a microprocessor 502, or it may be any conventional processor 502, etc.

[0142] In another embodiment of the present invention, a computer storage medium is provided. This storage medium may be a non-volatile computer-readable storage medium or a volatile storage medium. The storage medium stores a computer program 5032, wherein when executed by a processor 502, the computer program 5032 performs the following steps: inputting a first tissue slice image containing tumor tissue into a pre-trained first neural network model to obtain first gene expression data of the spatial transcriptome of the first tissue slice; wherein the tumor tissue includes a first tissue slice and a second tissue slice, the first tissue slice being adjacent to the second tissue slice in the tumor tissue; based on single-cell sequencing, labeling the first gene expression data with cell types to obtain second gene expression data; inputting the second gene expression data into a pre-trained second neural network model to obtain third gene expression data of the spatial transcriptome of the second tissue slice; wherein the third gene expression data is labeled with the cell type tag information; and constructing a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data.

[0143] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0144] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0146] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device 500 (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0148] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for constructing a virtual three-dimensional transcriptome of a tumor, characterized in that, include: The first tissue slice image containing tumor tissue is input into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first tissue slice; wherein, the tumor tissue includes a first tissue slice and a second tissue slice, and the first tissue slice is adjacent to the second tissue slice in the tumor tissue; Based on single-cell sequencing, cell type labeling is performed on the first gene expression data to obtain the second gene expression data; The second gene expression data is input into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second tissue slice; wherein the third gene expression data is labeled with the cell type tag information; A virtual three-dimensional transcriptome of the tumor tissue was constructed based on the second gene expression data and the third gene expression data. The training method for the first neural network model includes: The first neural network model is trained based on the expression data of the fourth gene in the ordinary transcriptome of the tumor tissue sample and pathological tissue images to obtain the first neural network model after the first stage of training; wherein, the expression data of the fourth gene is the data after transcriptome sequencing of the tumor tissue sample; The first neural network model trained in the first stage is trained according to a preset first sample to obtain the first neural network model trained in the second stage; wherein, the first sample includes tissue images of the tumor tissue sample and gene expression data after spatial transcriptome sequencing of the corresponding tissue slices; The training methods for the second neural network model include: Obtain a second sample pair; wherein, the second sample pair includes the expression data of the seventh gene of the spatial transcriptome of the third tissue slice and the expression data of the eighth gene of the spatial transcriptome of the fourth tissue slice, wherein the expression data of the seventh gene and the expression data of the eighth gene are both labeled with the cell type tag information, the third tissue slice and the fourth tissue slice are adjacent, and the expression data of the eighth gene is a sample tag; The second neural network model is trained based on the second sample to obtain the trained second neural network model. The tumor tissue also includes a third slice of tissue, which is adjacent to the first slice of tissue but not adjacent to the second slice of tissue in the tumor tissue. The construction of the virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data includes: The second gene expression data is input into a pre-trained third neural network model to obtain the ninth gene expression data of the spatial transcriptome of the third tissue slice; wherein the ninth gene expression data is labeled with the cell type tag information; The virtual three-dimensional transcriptome is constructed based on the expression data of the second gene, the expression data of the third gene, and the expression data of the ninth gene.

2. The method for constructing a virtual three-dimensional transcriptome of a tumor according to claim 1, characterized in that, The process of training the first neural network model based on the fourth gene expression data of the ordinary transcriptome of tumor tissue samples and pathological tissue images to obtain the first neural network model after the first stage of training includes: The pathological tissue image is preprocessed to obtain multiple small patch images of the pathological tissue image; Feature extraction is performed on each of the small image blocks to obtain the feature information of each small image block; Generate the expression data of the fifth gene in the ordinary transcriptome of the tumor tissue sample based on the aforementioned feature information; The mean squared error of the first neural network model is generated based on the expression data of the fourth gene and the expression data of the fifth gene; The network parameters of the first neural network model are updated based on the mean square error to obtain the first neural network model after the first stage of training.

3. The method for constructing a virtual three-dimensional transcriptome of a tumor according to claim 2, characterized in that, The step of extracting features from each of the small image patches to obtain feature information for each small image patch includes: Perform a convolution operation on each of the small image blocks to obtain the convolutional features of each small image block; Pooling is performed on the convolutional features to obtain the feature information of each small image block.

4. The method for constructing a virtual three-dimensional transcriptome of a tumor according to claim 1, characterized in that, The process of using single-cell sequencing to annotate the first gene expression data by cell type to obtain the second gene expression data includes: The cell type of each single cell in the tumor tissue after single-cell sequencing, the expression data of the sixth gene corresponding to each single cell, and the variance of the change in the expression data of the sixth gene were obtained. The first gene expression data is labeled according to the cell type, the sixth gene expression data, and the variance of variation to obtain the second gene expression data.

5. A device for constructing a virtual three-dimensional transcriptome of a tumor, characterized in that, include: The first input unit is used to input a first slice image containing tumor tissue into a pre-trained first neural network model to obtain the first gene expression data of the spatial transcriptome of the first slice tissue; wherein, the tumor tissue includes a first slice tissue and a second slice tissue, and the first slice tissue is adjacent to the second slice tissue in the tumor tissue; The first annotation unit is used to annotate the first gene expression data with cell type based on single-cell sequencing to obtain the second gene expression data. The second input unit is used to input the second gene expression data into a pre-trained second neural network model to obtain the third gene expression data of the spatial transcriptome of the second slice tissue; wherein the third gene expression data is labeled with the cell type tag information; The first construction unit is used to construct a virtual three-dimensional transcriptome of the tumor tissue based on the second gene expression data and the third gene expression data; The device for constructing a virtual three-dimensional transcriptome of a tumor further includes: a first training unit and a second training unit; the first training unit is used to train the first neural network model based on the fourth gene expression data of the ordinary transcriptome of a tumor tissue sample and pathological tissue images to obtain the first neural network model after a first stage of training; wherein, the fourth gene expression data is the data after transcriptome sequencing of the tumor tissue sample; the second training unit is used to train the first neural network model after the first stage of training based on a preset first sample to obtain the first neural network model after a second stage of training; wherein, the first sample includes gene expression data after spatial transcriptome sequencing of the tissue slice images of the tumor tissue sample and the corresponding tissue slices; The device for constructing a virtual three-dimensional transcriptome of a tumor further includes: a second acquisition unit and a third training unit; the second acquisition unit is used to acquire a second sample pair; wherein, the second sample pair includes the expression data of the seventh gene of the spatial transcriptome of the third tissue slice and the expression data of the eighth gene of the spatial transcriptome of the fourth tissue slice, the expression data of the seventh gene and the expression data of the eighth gene are both labeled with the cell type tag information, the third tissue slice and the fourth tissue slice are adjacent, and the expression data of the eighth gene is a sample tag; the third training unit is used to train the second neural network model according to the second sample pair to obtain the trained second neural network model; The first construction unit includes: a third input unit and a second construction unit; the third input unit is used to input the second gene expression data into a pre-trained third neural network model to obtain the ninth gene expression data of the spatial transcriptome of the third slice tissue; wherein the ninth gene expression data is labeled with the cell type tag information; the second construction unit is used to construct the virtual three-dimensional transcriptome based on the second gene expression data, the third gene expression data and the ninth gene expression data.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for constructing a tumor virtual three-dimensional transcriptome as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method for constructing a virtual three-dimensional transcriptome of a tumor as described in any one of claims 1 to 4.

Images