Detection method of rapid pathological tumor images based on TNM multi-level

Abstract

This invention discloses a rapid pathological tumor image detection method based on TNM multi-level probabilistic maps, comprising the following steps: acquiring WSI images and preprocessing them; converting the preprocessed images into multi-level probabilistic maps (PMAPs); dividing the target tumor region using a tumor threshold; acquiring multi-level images based on the PMAP map; and performing image detection using the TNM-NET model to obtain TNM grading results and tumor region detection results. This invention uses PMAP for probabilistic map navigation, guiding the system to quickly locate suspected pathological regions. It uses the correspondence between pathological coordinates, levels, and locations for result mapping, greatly saving computational resources and eliminating the need for large-scale full-image scanning for further detection. Furthermore, it uses TNM-Net combined with the characteristics of WSI pathological data and the characteristics of TNM for different TNM tumor stages to design a deep network, which not only utilizes the features of images at multiple resolutions but also greatly enhances the model's extraction and segmentation capabilities, resulting in higher efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for detecting pathological tumor images based on TNM multi-level rapid imaging. Background Technology

[0002] Pathological examination is a crucial part of clinical diagnosis. The basic medical task of pathological examination is to obtain disease diagnosis results by examining pathological images to guide patient treatment. Due to the numerous and complex environments and technologies involved in pathological examination, results are usually saved as WSI digital whole slide images. Analysis of pathological images requires pathologists to analyze them. At high resolutions, pathologists need to locate lesions within images of billions of pixels, a time-consuming, labor-intensive process with low accuracy and a high risk of misidentification. Even with current computer-aided pathological imaging techniques, this level of image processing either misses pathological information at low resolutions or consumes enormous computational resources at high resolutions, with no guarantee of accurate results.

[0003] The TNM staging system differs for each type of tumor; therefore, the meanings of the letters and numbers in the TNM staging system vary depending on the type of tumor. Once the T, N, and M components are determined in the TNM staging system, the overall stage can be derived, such as stage I, II, III, IV, etc. Sometimes, a combination of letters may be used for further subdivision, such as IIa or IIIb. Stage I tumors generally have a relatively good prognosis as they are at a relatively early stage. Higher stages indicate greater tumor progression.

[0004] In the TNM staging system: T ("T" stands for "Tumor") refers to the primary tumor site, and is designated T1 through T4 as the tumor size and extent of involvement of adjacent tissues increase. N ("N" stands for "Node") refers to the involvement of regional lymph nodes. No lymph nodes are involved, designated N0. As the extent and degree of lymph node involvement increase, it is designated N1 through N3. M ("M" stands for "Metastasis") refers to distant metastasis (usually hematogenous metastasis). No distant metastasis is designated M0, and distant metastasis is designated M1. Based on this, a combination of these three TNM indicators (grouping) is used to classify specific stages. These include seven stages: Ia, Ib, IIa, IIb, IIIa, IIIb, and IV.

[0005] Current pathology testing methods do not fully utilize the information extraction capabilities of deep learning models for suspected tumor areas. Most solutions only go as far as the pathological region is extracted, leaving a significant amount of subsequent discrimination and grading work to be done subjectively by physicians. Furthermore, most methods employ full-image scanning at a specific resolution, making it difficult to balance efficiency and accuracy. For example, at low resolution, the detection speed is fast, but the accuracy is very poor; or at high resolution, the detection accuracy is improved, but the detection speed is extremely slow. Moreover, no invention has directly incorporated the TNM table for tumor region staging.

[0006] For example, Chinese patent CN201810495014.3 discloses a method and apparatus for classifying thyroid tumor pathological tissue slice images. The method involves acquiring a set of original images of classified thyroid tumor pathological tissue slices; automatically extracting several images containing cells from each original image as a sub-image set; using all or part of the sub-image set as a training set; constructing a preliminary convolutional neural network model; training the preliminary convolutional neural network model with the training set to obtain a mature convolutional neural network model; and using the mature convolutional neural network model to classify the thyroid tumor pathological tissue slice images to be classified. However, this method only considers full-image scanning at a certain resolution, resulting in low detection accuracy. Summary of the Invention

[0007] This invention primarily addresses the problem that existing technologies for pathological image detection cannot simultaneously achieve both efficiency and accuracy. It provides a rapid pathological tumor image detection method based on TNM multi-level hierarchical structure for tumor region detection. By fully integrating the concepts of WSI multi-level and TNM staging, a TNM-NET deep network is constructed. This method balances efficiency and accuracy while leveraging multi-level hierarchical structure to improve results. Finally, it provides auxiliary evaluation information from TNM, significantly enhancing the auxiliary intelligence of the pathological detection system and greatly facilitating doctors.

[0008] The above-mentioned technical problems of the present invention are mainly solved by the following technical solution: a method for detecting pathological tumor images based on TNM multi-level rapid images, comprising the following steps: acquiring WSI images and preprocessing the WSI images; converting the preprocessed images into multi-level probability maps (PMAPs); dividing the target tumor region using a tumor threshold; acquiring multi-level images based on the PMAP map; and performing image detection using the TNM-NET model to obtain TNM grading results and tumor region detection results. Whole slide images (WSI) are used as input image data, with six resolution levels (0-5). Probabilistic map navigation (PMAP) is used to guide the system to quickly locate suspected pathological areas. TNM-Net is used to combine the characteristics of WSI pathological data and the characteristics of TNM for different TNM tumor stages to design a deep network. This not only utilizes the features of images at multiple resolution levels but also greatly enhances the model's extraction and segmentation capabilities. A TNM-NET deep network is proposed as a model for TNM tumor staging and extraction of suspected tumor areas. By using PMAP for pathological area navigation and TNM-NET for pathological stage detection, rapid detection of pathological areas is achieved, and image detection at different resolutions is possible with higher accuracy.

[0009] Preferably, the preprocessing includes image cropping and image enhancement, removing some obvious slide noise.

[0010] Preferably, the multi-level probabilistic map (PMAP) conversion method includes the following steps: inputting an image; encoding it into image features through the image feature encoding part; converting the image features into probability image values ​​of the target area; decoding it through the probability decoding part; and outputting the multi-level probabilistic map (PMAP). Through PMAP conversion, the computational load is significantly reduced; even at the highest resolution Level-5, the overall computation time does not exceed 1 second, thus improving computational efficiency.

[0011] As a preferred method, the target tumor region is divided as follows: setting a threshold to process PMAP into an image mask; using dilation to process image boundaries; and using connected component detection to divide the tumor region.

[0012] Preferably, the TNM-NET model is designed with a 60-layer structure, with each layer corresponding to one layer of image data in WSI. Images at different resolution levels retain relevant image details within each layer, and when fused with features from the downsampling process, more details of the pathological images can be extracted, improving accuracy.

[0013] Preferably, the TNM-NET model performs image detection including convolution operations, transposed convolution operations, data stitching, and image scaling.

[0014] Preferably, the formula for calculating image boundaries using dilation is:

[0015]

[0016] Where A is the image data, B is the convolution template or convolution kernel, and x represents the image coordinates.

[0017] The beneficial effects of this invention are as follows: Using PMAP for probabilistic map navigation guides the system to quickly locate suspected pathological areas. The mapping of results using pathological coordinates, hierarchical relationships, and locations significantly saves computational resources, eliminating the need for large-scale full-image scanning for further detection. Furthermore, the use of TNM-Net, combined with the characteristics of WSI pathological data and the different TNM tumor staging features, designs a deep network that not only utilizes the features of images at multiple resolutions but also greatly enhances the model's segmentation capabilities. At different resolutions, it provides targeted feature detection, accurately delineating pathological tumor regions while simultaneously staging the pathology based on TNM, offering doctors a reference. This results in higher efficiency and accuracy in pathological detection. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of the detection method according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a WSI image according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of PMAP conversion according to an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the TNM-NET model according to an embodiment of the present invention. Detailed Implementation

[0022] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the invention.

[0024] Example: A method for detecting rapid pathological tumor images based on TNM multi-level images, such as... Figure 1 As shown, the process includes the following steps: First, the input WSI image is preprocessed: 1. Tissue images are selected for examination; 2. Noise in the image is removed; 3. The image is cropped and normalized; The lowest resolution image in WSI is extracted and converted to 512x512 resolution through boundary filling; The PMAP network is used to detect the suspected tumor area in the image; The tumor area is divided using a tumor threshold; Then, the tumor area bounding box is converted into probability maps of various levels according to the WSI grading standard, and the images of the relevant levels are obtained; The images of each level are sequentially input into TNM-NET, and the tumor segmentation is extracted after detection by the deep network. Based on the features of different levels of TNM-NET, the corresponding TNM classification of the tumor in the image is detected; Finally, the TNM staging assessment result of the target image is given.

[0025] Figure 2 This is a schematic diagram of whole slide images (WSI) used as input image data. Level (0-5) represents pathological images at different resolution levels.

[0026] Image preprocessing includes tissue image cropping and image enhancement, removing obvious slide noise and preserving the appropriate mapping relationship between the tissue image and its original location. Where L represents the Lth level of WSI. This represents the original image pixels. For each pixel in the mapped image, 'Location' is the location mapping function. L (x,y)=(2 L ×x,2 L ×y),2 L The scaling factor is the corresponding resolution level, and x and y are the image coordinates.

[0027] PMAP (Probability Map) has the following process structure: Figure 3As shown, the input image is encoded into image features by the image feature encoding part, then converted into target region probability image values ​​according to the model. These values ​​are then decoded by the probability decoding part, finally outputting an image probability map (PMAP). The input image has dimensions W×H×C. After processing by a three-layer convolutional encoding part with a stride of 2, the image features are encoded. These features are then bridged by probabilistic features and input to the probability decoding part, where they are decoded by a three-layer deconvolutional decoding part with a stride of 2. The final output is a probability map with dimensions W×H×1.

[0028] Probabilistic map navigation significantly reduces the computational load of scanning the entire map, thus speeding up the process. As shown in Table 1, after PMAP uses probabilistic map navigation, the computational load is greatly reduced; even at the highest resolution Level-5, the overall computation time does not exceed 1 second.

[0029] Table 1 Computation time for images of different layers

[0030] type resolution Full scan PMAP Level-0 computational complexity 1528x3422 5228816 52288 Level-0 calculation time - 0.97s 0.01s Level-5 computational complexity 97792x219648 21,479,817,216 214,798,172 Level-5 calculation time - 34s 0.31s

[0031] As can be seen from the table, the computation time for the lowest resolution Level-0 is 0.01s, and the computation time for the highest resolution Level-5 is 0.31s, which greatly improves work efficiency.

[0032] The segmentation of the target tumor region mainly involves dividing the pixel regions in the PMAP image with a high probability of being suspected tumors. A suspected tumor threshold Threshold(PMAP) of 0.8 is set, and the PMAP is processed into an image mask. Use dilation to handle boundaries. (This formula represents using B to dilate image A, where B is a convolution template or kernel. The convolution calculation is performed between template B and image A, scanning each pixel in the image. An AND operation is performed between the template element and the binary image element; if both are 0, the target pixel is 0; otherwise, it is 1.) Finally, connected component detection (four-connected regions refer to the four adjacent positions (up, down, left, right) of a corresponding pixel location; if a pixel with a value of 1 has a neighboring pixel with a value of 1, then these two pixels belong to the same connected component) is used to divide the region into a suitable region: Tumor(n) = *(x,y), (x,y)∈Tumor. n Tumor n ∈Tumor}, and finally divided into 16x16 pixel partitions. Tumor is a collection of all suspected tumor areas in the image, where A 16x16 pixel graphic block.

[0033] Through RefL (Tumor(n,m)) maps the coordinates of the suspected lesion area back to different levels of the image, where ref L (x, y) = (2 L ×x,2 L ×y). Level-Input-n serves as the image feature input for the TNM-NET model. For example... Figure 4 and Figure 2 As shown, Level (0-5) corresponds to images at different levels in the WSI pathological image data. The TNM-NET model's network structure is designed with six layers to correspond to different levels of WSI data. Images at different resolution levels retain relevant image details within each level. Fusion with features from the downsampling process allows for the extraction of more details from the pathological images, which is significant for analyzing images of suspected pathological areas. The TNM-NET model includes Down-Sample, Up-Sample, Layer-Connect, Convolution, and scaling / decoding operations. The Down-Sample operation performs a 2x2 convolution on the data, reducing the data size and extracting important features from different levels. The Up-Sample operation performs a 2x2 transposed convolution on the data, increasing the data size while restoring local features. The Layer-Connect operation concatenates two sets of data along a specified dimension. It also requires convolution with a 1x1 kernel and concatenation with the sampled data. The Convolution operation performs convolution on the feature data to generate the object detection results. The scaling and decoding operations scale the detection results to the same size as the original image, decode the one-hot encoding, and finally output the detection results including Output-segment, Output-T, Output-N, Output-M, and Output-TNM.

[0034] The model's final output is the detection result for a suspected tumor region, which requires the use of hierarchical-location-based inverse mapping. in The detected contour coordinates are mapped back to the initial layer size. Ultimately, the entire model returns the contour segment coordinates of the pathology, the TNM rating, and the T, N, M evaluation values. Finally, the image labeling and evaluation results are combined to assist doctors in making diagnoses.

[0035] This invention employs Probabilistic Mapping (PMAP) for probabilistic map navigation, guiding the system to quickly locate suspected pathological areas. It uses pathological coordinates, hierarchical relationships, and location correspondences for result mapping, significantly saving computational resources and eliminating the need for large-scale full-image scanning for further detection. Furthermore, it utilizes TNM-Net, combining the characteristics of WSI pathological data with the specific features of TNM for different TNM tumor stages, to design a deep network that not only leverages the features of images at multiple resolutions but also greatly enhances the model's extraction and segmentation capabilities. This invention is the first to propose the TNM-NET deep network for extracting TNM tumor stages and suspected tumor regions.

[0036] This invention enables pathological examination images to directly generate results based on the TNM tumor staging system, delineating the target tumor region and providing TNM-based T, N, and M references and TNM staging assessment. This not only saves time examining pathology slides but also conserves significant computational resources. It provides targeted feature detection at different resolutions, accurately delineating the pathological tumor region while simultaneously staging the pathology based on TNM, offering doctors a reference and improving the efficiency and accuracy of pathological examinations.

[0037] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims.

Claims

1. A detection method based on TNM multi-level fast pathological tumor image, characterized in that, The method comprises the following steps: Obtaining a WSI image and preprocessing the WSI image; the WSI image is a pathological image at a Level (0-5) resolution level, and the preprocessing retains a suitable tissue image and an image position mapping relationship; Converting the preprocessed image into a multi-level probability map PMAP; Dividing the PMAP map using a tumor threshold to obtain a target tumor region, performing coordinate mapping on the target tumor region, and obtaining a multi-layer image of the target tumor region; Using a TNM-NET model to perform image detection to obtain a TNM grading result and a tumor region detection result; the TNM-NET model is designed to have a six-layer structure, and each layer inputs corresponding layer image data of the target tumor region at a Level (0-5) resolution level.

2. The method according to claim 1, wherein the preprocessing comprises image cropping and image enhancement. The conversion method of the multi-level probability map PMAP comprises the following steps: 3.The TNM multi-level quick pathological tumor image-based detection method according to claim 1 or 2, characterized in that, Inputting an image; Converting the image into an image feature through image feature coding; Converting the image feature into a target region probability image value; Decoding through a probability decoding part; Outputting a multi-level probability map PMAP.

4. The method according to claim 3, wherein the method for dividing the target tumor region comprises: Setting a threshold to process the PMAP into an image mask; Processing the image boundary in an inflation manner; Dividing the tumor region in a connected domain detection manner.

5. The method according to claim 1, wherein different resolution level images retain relevant image details in the level and fuse the features in a down-sampling process to extract more details of the pathological image; the down-sampling performs a step-2 convolution operation on the data to reduce the data size and extract important features at different levels.

6. The method according to claim 1, wherein the image detection by the TNM-NET model comprises convolution operation, transposed convolution operation, data splicing and image scaling.

7. The method according to claim 4, wherein the calculation formula for processing the image boundary in an inflation manner is: wherein A is image data, B is a convolution template or a convolution kernel, and x represents an image coordinate. ​ ​ ​ ​ ​

Images