A method for quantifying tumor stroma in pathological sections

Abstract

The application discloses a pathological section tumor stroma ratio quantification method, and steps are as follows: (1) a weakly supervised neural network training process based on an image block: firstly, block a full section image scanned by a digital scanner, then assign a unique label, i.e., tumor or non-tumor, to each image block, and finally, train a neural network added with a two-dimensional random inactivation layer using cross-entropy loss and translation invariance loss; (2) a tumor stroma ratio quantification algorithm based on a neural network: generate a class activation mapping graph using a previously trained model to identify a section, calculate a stroma region using a morphological algorithm, and further calculate a tumor stroma ratio. The application is suitable for making an accurate quantitative judgment on the tumor state of a patient in a clinic, and compared with existing methods, the application has the characteristics of high repeatability, low execution cost and high calculation accuracy, can help a pathologist to quantize a prognosis factor difficult to manually calculate, and significantly reduces the workload of the pathologist.

Description

Technical Field

[0001] This invention relates to clinical medicine, and in particular to a method for quantifying the stroma-to-tumor ratio in pathological sections of tumors. Background Technology

[0002] Studies have shown that many factors influence the survival prognosis of cancer patients. The 8th edition of the AJCC staging system added two prognostic parameters: depth of invasion and extracapsular lymph node invasion. Furthermore, histopathological parameters of various malignant tumors (such as neural invasion, worst-case invasion pattern, and tumor budding) have been used in numerous studies. Histopathological morphology studies have also demonstrated that grading systems based on the tumor invasion front have certain predictive value for cancer prognosis. In addition, research indicates that the composition of the tumor stroma significantly influences carcinogenesis and malignant phenotypes at multiple stages of tumor development and progression. Since the tumor stroma is primarily composed of non-malignant cells in the tumor microenvironment (including cancer-associated fibroblasts, innate and adaptive immune cells, microvascular cells, extracellular matrix, and some other tissue components), all these cells lie between malignant cells and normal host tissue. The complex interactions between tumor cells and various cell and stroma components in the tumor microenvironment play a crucial role in the occurrence, progression, invasion, and metastasis of cancer. The ratio of tumor cells to tumor-associated stroma in tumor tissue is defined as the tumor-stromal ratio (TSR), which has been shown to have independent prognostic significance in multiple cancer studies. Currently, clinical TSR assessment methods rely on visual evaluation by pathologists under a microscope on HE-stained slides. Furthermore, there is a lack of universal methods and classification cutoff values ​​for TSR assessment across different cancer types, and their effectiveness and reliability require further validation. Therefore, there is an urgent clinical need to establish an objective and standardized method for quantifying TSR. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide a method for quantifying the tumor-to-stromal ratio in pathological sections, thereby enabling precise quantitative measurement of the tumor-to-stromal ratio in the sections. This helps pathologists quantify prognostic factors that are difficult to calculate manually, significantly reducing the workload of pathologists, and potentially improving the prediction of clinical prognosis and treatment outcomes for patients.

[0004] Technical solution: The present invention provides a method for quantifying the stroma-to-tumor ratio in pathological sections, comprising the following steps:

[0005] (1) Training process of weakly supervised neural network based on image patch: First, the digitally scanned full slice image is divided into blocks, then each image block is assigned a unique label, namely tumor or non-tumor. Finally, cross-entropy loss and translation invariant loss are used to train the neural network with added two-dimensional random deactivation layer.

[0006] (2) Neural network-based tumor stroma ratio quantification algorithm: The previously trained model is used to generate a class activation map to identify the slices, and the stroma region is calculated using a morphological algorithm. The tumor stroma ratio is then calculated.

[0007] The specific steps (1) are as follows:

[0008] (1.1) Input the specified convolutional neural network structure M, and the set of digital scan full slices. Number of training phases Learning rate Regularization coefficient Batch size ;

[0009] (1.2) For the neural network M, a two-dimensional dropout layer is added after the last feature map and before the global pooling layer;

[0010] (1.3) For sets Each sample in the dataset is divided into blocks according to a grid of 256*256 pixels, resulting in a fixed number of image blocks for each sample. The dataset is obtained from all the image blocks of all samples. ;

[0011] (1.4) For For each image patch x, assign a label y, where y=1 indicates that x contains a tumor, and y=0 indicates that x does not contain a tumor. Labeling all image patches yields the training set. {X, Y}, where X is a set of image patches and Y is a set of markers;

[0012] (1.5) From the training sample set Randomly select data corresponding to the batch size of this stage. ;

[0013] (1.6) For For each sample in the original slice, an image patch overlapping with that sample is cropped from the region near the original slice, resulting in a patch with the same shape as the original slice. Corresponding batch data ;

[0014] (1.7) Using a neural network M to compute batch data and Feature map ;

[0015] (1.8) Calculate batch data The probability that each sample contains a tumor ;

[0016] (1.9) Calculate the loss function for this batch, which is the sum of the cross-entropy loss and the shift-invariant regularization loss, where the shift-invariant loss is... and The feature difference of the overlapping region ; , ;

[0017] (1.10) Update the weights of M using the gradient of the loss function. = ;

[0018] (1.11) If the number of training stages has reached the preset number of stages S, then terminate the training; otherwise, return to step (1.5).

[0019] Step (2) specifically involves:

[0020] (2.1) Input the trained convolutional neural network M, and the full digit scan slice to be computed. ;

[0021] (2.2) Remove the global pooling layer in M;

[0022] (2.3) Divide S into blocks, dividing each sample into several image blocks of fixed size, converting the image blocks from RGB mode to HSV mode, discarding image blocks whose average value of S channels is less than the threshold, and forming a set from the remaining image blocks. ;

[0023] (2.4) Calculated using model M The probability map of each image patch is converted into a 0-1 segmentation mask with a threshold of 0.5.

[0024] (2.5) will The segmentation masks for each image patch are arranged according to their coordinates in S, resulting in the tumor segmentation mask for the whole slice S. ;

[0025] (2.6) will Pixels with a value of 1 are defined as tumor-stromal regions;

[0026] (2.7) Traverse all pixels. For each pixel in the tumor-stromal region that is adjacent to a non-tumor-stromal region, add all pixels within a fixed radius of the surrounding pixels to the tumor-stromal region. Then traverse all pixels again. For each pixel in the tumor-stromal region that is adjacent to a non-tumor-stromal region, delete it from the tumor-stromal region.

[0027] (2.8) Count the total number of pixels in all tumor-stromal regions. The total number of pixels with a value of 1 in the segmentation mask ;

[0028] (2.9) Calculate the tumor-stromal ratio of whole slice S. .

[0029] A system for quantifying the tumor-to-stromal ratio in pathological sections, the system employing the aforementioned method for quantifying the tumor-to-stromal ratio in pathological sections, comprising the following modules:

[0030] Weakly supervised neural network training module: used to divide digitally scanned full-slice images into blocks, then assign a unique label to each image block, i.e., tumor or non-tumor, and finally train a neural network with added two-dimensional random deactivation layers using cross-entropy loss and translation invariant loss.

[0031] The tumor-to-stromal ratio quantification algorithm module is used to generate class activation maps using a previously trained model to identify slices, calculate the stromal region using morphological algorithms, and further calculate the tumor-to-stromal ratio.

[0032] A computer storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for quantifying the tumor stroma ratio in pathological sections.

[0033] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for quantifying the tumor stroma ratio in pathological sections.

[0034] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0035] 1. This invention can be applied to make accurate quantitative judgments on the tumor status of patients in clinical practice. Compared with existing methods, this invention has the characteristics of high repeatability, low execution cost and high calculation accuracy. It can help pathologists quantify prognostic factors that are difficult to calculate manually and significantly reduce the workload of pathologists.

[0036] 2. This invention has the potential to improve the prediction of patients' clinical prognosis and treatment outcomes. Attached Figure Description

[0037] Figure 1 (a) in the diagram is a flowchart of the training process for a weakly supervised neural network based on image patches; Figure 1 (b) in the diagram is a flowchart of the tumor stroma ratio quantification algorithm based on neural networks. Detailed Implementation

[0038] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0039] The method for quantifying the tumor stroma-to-pathological section ratio provided by this invention can be applied in clinical practice to make accurate quantitative judgments on the tumor status of patients and in academic research to make more precise measurements of tumor development. It is suitable for scenarios with a large number of samples in the dataset to be classified and sufficient computing resources. Figure 1 As shown, taking clinical practice as an example, the specific workflow of the tumor stroma ratio quantification method for pathological sections described in this invention in clinical application is as follows:

[0040] (1) The training process of a weakly supervised neural network based on image patches is as follows: Figure 1 As shown in (a): First, the digitally scanned full-slice image is divided into blocks, then each image block is assigned a unique label, i.e., tumor or non-tumor. Finally, a neural network with added two-dimensional random deactivation layers is trained using cross-entropy loss and translation-invariant loss.

[0041] (1.1) Input the specified convolutional neural network structure M and a set of full digital scan slices. Number of training phases Learning rate Regularization coefficient Batch size .

[0042] (1.2) For the neural network M, the following modification should be made: After the last feature map and before the global pooling layer, add a two-dimensional dropout layer, which sets any pixel on the feature map to zero with probability p.

[0043] (1.3) Create a dataset, targeting the set Each sample in the dataset is divided into several image patches of size 256*256 pixels using a grid of 256*256 pixels. The dataset is obtained from all the image patches of all samples. .

[0044] (1.4) For Each image patch x in the dataset is labeled y by the pathologist, where y=1 indicates that x contains a tumor, and y=0 indicates that x does not contain a tumor. Labeling all image patches yields the training set. {X, Y}, where X is a set of image patches and Y is a set of tags.

[0045] (1.5) After preparing the model and data, begin the training process: from the training sample set Randomly select data corresponding to the batch size of this stage. .

[0046] (1.6) For For each sample in the original slice, a patch of image that overlaps with the sample is cropped from the region near the sample, resulting in a patch of image that overlaps with the original sample. Corresponding batch data .

[0047] (1.7) Using a neural network M to compute batch data and Feature map .

[0048] (1.8) Calculate batch data The probability that each sample contains a tumor .

[0049] (1.9) Calculate the loss function for this batch, which is the sum of the cross-entropy loss and the shift-invariant regularization loss, where the shift-invariant loss is... and The feature difference of the overlapping region. ; , .

[0050] (1.10) Finally, the weights of M are updated using the gradient of the loss function. = .

[0051] (1.11) If the number of training stages has reached the preset number of stages S, then terminate the training; otherwise, return to step (1.5) to continue the iteration.

[0052] (2) A neural network-based algorithm for quantifying the tumor stroma ratio, such as Figure 1 As shown in (b): the previously trained model is used to generate a class activation map to identify the slices, the stromal region is calculated using a morphological algorithm, and the tumor stroma ratio is further calculated.

[0053] (2.1) In actual clinical use, input the whole digital scan slices of the clinical patient to be calculated. .

[0054] (2.2) The convolutional neural network M trained using the training process proposed in this invention. Remove the global pooling layer in M.

[0055] (2.3) Divide S into blocks according to a grid of size 256*256 pixels. Divide each sample into several image blocks of size 256*256 pixels. Convert the image blocks from RGB mode to HSV mode. Discard image blocks whose average value of S channels is less than 15. The remaining image blocks form a set. .

[0056] (2.4) Calculated using model M The 8x8 probability map of each image patch is converted into a 0-1 segmentation mask with a threshold of 0.5.

[0057] (2.5) will The segmentation masks for each image patch are arranged according to their coordinates in S, resulting in the tumor segmentation mask for the whole slice S. .

[0058] (2.6) will All pixels with a value of 1 are considered as tumor-stromal regions.

[0059] (2.7) Traverse all pixels. For each pixel in the tumor-stromal region that is adjacent to a non-tumor-stromal region, add all pixels within a radius of 57 pixels around it to the tumor-stromal region. Then traverse all pixels again. For each pixel in the tumor-stromal region that is adjacent to a non-tumor-stromal region, delete it from the tumor-stromal region.

[0060] (2.8) Count the total number of pixels in all tumor-stromal regions. The total number of pixels with a value of 1 in the segmentation mask .

[0061] (2.9) Calculate the tumor-stromal ratio of whole slice S. .

[0062] The method described in this invention was experimentally tested on actual clinical data from pathological sections. The experiments demonstrated the role of multi-dimensional quantitative TSR in predicting the prognosis of OSCC patients. The results show that artificial neural network models can help pathologists quantify prognostic factors that are difficult to calculate manually, significantly reducing their workload. Therefore, it has the potential to improve the prediction of clinical prognosis and treatment outcomes for patients.

Claims

1. A method of quantifying the tumor stroma ratio of a pathology section, characterized by, The method comprises the following steps: (1) a weakly supervised neural network training process based on image blocks: firstly, block the whole slide image scanned by a digital scanner, then assign a unique label, i.e. tumor or non-tumor, to each image block, and finally train a neural network with a two-dimensional random dropout layer using cross-entropy loss and translation-invariant loss; (2) a tumor stroma ratio quantification algorithm based on a neural network: generate a class activation map using the previously trained model to identify the slice, calculate the stroma region using a morphological algorithm, and further calculate the tumor stroma ratio; Step (2) specifically comprises: (2.1) inputting a trained neural network M, a digital scanned whole slide to be computed ; (2.2) removing the global pooling layer in the neural network M; (2.3) dividing S into blocks, dividing each sample into a number of image blocks of fixed size, converting the image blocks from RGB mode to HSV mode, discarding image blocks whose average value in the S channel is less than a threshold, forming a set from the remaining image blocks ; (2.4) using the model M to calculate a probability map for each image patch, and converting the probability map into a 0-1 segmentation mask using 0.5 as a threshold; (2.5) to The segmentation mask of each image block is arranged according to the coordinates in S, and the tumor segmentation mask of the whole slice S is obtained ; (2.6) defining as tumor-stroma regions all pixels with a value of 1; all pixels with a value of 1 are defined as tumor-stroma regions; (2.7) traversing all pixel points, adding all pixel points within a fixed radius around each tumor-stroma region pixel point adjacent to the non-tumor-stroma region to the tumor-stroma region, and then traversing all pixel points again, deleting each tumor-stroma region pixel point adjacent to the non-tumor-stroma region from the tumor-stroma region; (2.8) Count the total number of pixels in all tumor-stromal regions , count the total number of pixels in the segmentation mask with value 1 ; (2.9) Calculate the tumor stroma ratio for the whole slice S .

2. The method of quantifying the stromal content of a tumor in a pathology section according to claim 1, characterized in that, The step (1) specifically comprises: (1.1) input a specified neural network M, a digital scanned whole slide set , a number of training epochs N, a learning rate , a regularization term coefficient , a batch size ; (1.2) for the neural network M, add a two-dimensional random dropout layer after the last feature map and before the global pooling layer; (1.3) For each sample in the set , tile according to a grid of size 256*256 pixels, segmenting each sample into fixed-size image tiles, obtaining a dataset from all image tiles of all samples; (1.4) For For each image patch x, assign a label y, where y=1 indicates that x contains a tumor, and y=0 indicates that x does not contain a tumor. Labeling all image patches yields the training set. {X, Y}, where X is a set of image patches and Y is a set of markers; (1.5) randomly picking the batch data corresponding to the stage from the training sample set ;​ (1.6) For each sample in the batch data , crop an image patch that overlaps with the sample from the region near the sample in the original slice, resulting in the corresponding batch data ;​ (1.7) computing batch data using neural network M and feature maps ; (1.8) computing batch data the probability that each sample contains a tumor ; (1.9) computing the loss function for the batch of data and the loss function is the sum of a cross-entropy loss and a translation invariance regularizing loss, where the translation invariance regularizing loss is and a feature difference of overlapping regions of ; , ; (1.10) updating the weights of the neural network M using the gradient of the loss function ;​ (1.11) if the number of training stages has reached the pre-set number of stages N, terminate the training, otherwise return to step (1.5).

3. A system for quantifying tumor stroma ratio in a pathology section, said system employing a method for quantifying tumor stroma ratio in a pathology section according to any one of claims 1-2, characterized in that, The method comprises the following modules: a weakly supervised neural network training module: for blocking the whole slide image scanned by a digital scanner, then assigning a unique label, i.e. tumor or non-tumor, to each image block, and finally training a neural network with a two-dimensional random dropout layer using cross-entropy loss and translation-invariant loss; a tumor stroma ratio quantification algorithm module: for generating a class activation map using the previously trained model to identify the slice, calculating the stroma region using a morphological algorithm, and further calculating the tumor stroma ratio; The tumor stroma ratio quantification algorithm module specifically comprises: input the trained neural network M, the digital scan full slice to be computed ; removing the global pooling layer in the neural network M; S is divided into blocks, each sample is divided into several image blocks of fixed size, the image blocks are converted from RGB mode to HSV mode, image blocks with average value of S channel less than a threshold are discarded, and a set consisting of all the remaining image blocks is obtained ; The probability map for each image block is calculated using the model M, and the probability map is converted to a 0-1 segmentation mask using 0.5 as the threshold. The probability map for each image block is calculated using the model M, and the probability map is converted to a 0-1 segmentation mask using 0.5 as the threshold. Will The segmentation masks for each image patch are arranged according to their coordinates in S, resulting in the tumor segmentation mask for the whole slice S. ; Define all pixels with values of 1 in the center as tumor-interstitial region. all pixels with values of 1 in the center as tumor-interstitial region; traversing all pixel points, adding all pixel points within a fixed radius around each tumor-stroma region pixel point adjacent to the non-tumor-stroma region to the tumor-stroma region, and then traversing all pixel points again, deleting each tumor-stroma region pixel point adjacent to the non-tumor-stroma region from the tumor-stroma region; counting the total number of pixels in all tumor-stromal regions counting the total number of pixels in all tumor-stromal regions ; calculating the tumor stroma ratio for the whole slide S .

4. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement a pathological slice tumor stroma ratio quantification method according to any one of claims 1-2.

5. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement a pathological slice tumor stroma ratio quantification method according to any one of claims 1-2.

Images