Apparatus and method for detecting tumor using pathology slide on basis of artificial intelligence

Abstract

According to the present disclosure, disclosed are an apparatus and a method for detecting a tumor using a pathology slide on the basis of artificial intelligence. The apparatus comprises: a data processing unit configured to acquire a thumbnail image from pathology slide data, separate a tumor region and a normal region from the thumbnail image, and generate image patches for the tumor region and the normal region; a training unit configured to train a patch classification model using a dataset configured on the basis of the image patches of the tumor region and the image patches of the normal region; an inference unit configured to evaluate a tumor probability of each patch by inputting the image patches of the input pathology slide data generated by the data processing unit into the trained patch classification model; and a result generation unit configured to generate a visualized tumor detection result for the input pathology slide data on the basis of the evaluated tumor probability of each patch.

Description

Device and method for detecting tumors using pathology slides based on artificial intelligence

[0001] The present disclosure relates to a tumor detection technique, and more specifically, to an apparatus and method for detecting tumors using pathology slides based on artificial intelligence.

[0002] Pathological tissue slides refer to the results of a series of chemical processing steps taken from a patient during the course of medical examination and testing to convert them into tissue for microscopic observation. The field of computer pathology is developing, which allows these pathological slides to be digitized for analysis and diagnosis on a computer.

[0003] With the advancement of digital pathology interpretation technology, various companies are providing software related to pathology slide readers; however, its utilization remains low due to the high cost of equipment and subscription fees. Furthermore, while attempts are being made to apply artificial intelligence technology to pathology image interpretation, most require high-magnification images, which presents a limitation as it does not align with the low-magnification environments found in clinical settings.

[0004] Accordingly, there is a need to develop a solution that enables rapid and highly accurate tumor diagnosis using pathology slide data commonly used in clinical settings.

[0005] The present disclosure aims to solve these problems by providing an apparatus and method for detecting tumors using pathology slides based on artificial intelligence.

[0006] According to one embodiment of the present disclosure, an apparatus for detecting tumors using pathology slides based on artificial intelligence may be provided. The apparatus may include: a data processing unit configured to acquire a thumbnail image from pathology slide data, separate a tumor region and a normal region from the thumbnail image, and generate image patches for the tumor region and the normal region; a learning unit configured to train a patch classification model using a dataset configured based on the image patches of the tumor region and the image patches of the normal region; an inference unit configured to input the image patches of the input pathology slide data generated by the data processing unit into the trained patch classification model to evaluate the tumor probability of each patch; and a result generation unit configured to generate a visualized tumor detection result for the input pathology slide data based on the evaluated tumor probability of each patch.

[0007] In addition, the data processing unit may be configured to annotate the tumor areas of the thumbnail image and generate an XML file containing coordinate data of each tumor area.

[0008] In addition, the data processing unit may be configured to convert the thumbnail image from an RGB color space to an HSV (Hue, Saturation, Value) color space and to apply an OTSU binarization algorithm to the value of the HSV color space to divide the tissue region and the background region.

[0009] In addition, the data processing unit may be configured to separate the tumor region and the normal region from the thumbnail image by generating a tissue mask for the tissue region, generating a tumor mask from the tissue mask using the XML file, and generating a normal mask by removing the tumor mask from the tissue mask.

[0010] Additionally, the data processing unit may be configured to generate image patches of the tumor area and image patches of the normal area according to the overlap ratio between the adjusted image patches or the ratio of the image patches of the adjusted tumor area and the normal area.

[0011] In addition, the patch classification model may include a Vision Transformer-based Swin V2 Tiny model.

[0012] Additionally, the above dataset may be composed of K datasets of the image patches for K-fold cross-validation and provided to the training unit. The training unit may be configured to train K independent patch classification models using the K datasets.

[0013] In addition, the learning unit may be configured to apply data augmentation to the dataset and provide it as a patch classification model, and to apply a modified classification layer to the patch classification model.

[0014] In addition, the inference unit may be configured to input the patches generated by applying a sliding window algorithm to the image patches generated by the data processing unit for the input pathology slide data into the K learned patch classification models, and to determine the average tumor probability of each patch by ensembling the predictions from each patch classification model.

[0015] In addition, the result generation unit may be configured to generate a tumor suspicion mask representing a tumor suspicion area based on the patch tumor probability evaluated by the inference unit and a preset tumor probability threshold, apply small area filtering to the tumor suspicion mask, and apply a contour algorithm to the input pathology slide data using the small area filtered tumor suspicion mask to generate a pathology slide image with tumor contours displayed as a tumor detection result.

[0016] In addition, the result generation unit may be configured to generate multiple tumor detection results according to each of the multiple thresholds for the evaluated patch tumor probability by applying a tumor probability threshold adjustment algorithm.

[0017] According to one embodiment of the present disclosure, a method for detecting tumors using pathology slides based on artificial intelligence executable by a computing device may be provided. The method may include the steps of: acquiring a thumbnail image from pathology slide data; separating a tumor region and a normal region from the thumbnail image; generating image patches for the tumor region and the normal region; training a patch classification model using a dataset configured based on the image patches of the tumor region and the image patches of the normal region; inputting the image patches generated from the input pathology slide data into the trained patch classification model to evaluate the tumor probability of each patch; and generating a visualized tumor detection result for the input pathology slide data based on the evaluated tumor probability of each patch.

[0018] According to one embodiment of the present disclosure, a computer program stored on a computer-readable medium may be provided, comprising computer-executable instructions for executing a method for detecting a tumor using a pathology slide based on artificial intelligence.

[0019] According to the present disclosure, an artificial intelligence model-based solution can be provided that can detect high-purity tumor regions even from low-magnification pathology slide data in clinical settings and infer whether a tumor exists.

[0020] In addition, according to the present disclosure, a pathology image analysis solution can be implemented by defining a data processing algorithm specialized for pathology slide data, designing an artificial intelligence model that ensures robust performance even with low-magnification tissue pathology image data, and providing GUI-based visualized tumor detection results applicable in clinical settings.

[0021] FIG. 1 is an exemplary block diagram illustrating a computing device for detecting tumors using pathology slides based on artificial intelligence according to one embodiment of the present disclosure.

[0022] FIG. 2 is an exemplary block diagram showing functional modules of a computing device of FIG. 1 according to one embodiment of the present disclosure.

[0023] FIG. 3 is an exemplary diagram illustrating the process of obtaining a thumbnail image and annotating from pathology slide data according to one embodiment of the present disclosure.

[0024] FIG. 4 is an exemplary drawing illustrating the process of generating a tumor mask and a normal mask according to one embodiment of the present disclosure.

[0025] FIG. 5 is an exemplary drawing illustrating the process of generating image patches for a tumor region and a normal region according to one embodiment of the present disclosure.

[0026] FIG. 6 is an exemplary drawing illustrating the process of training a patch classification model according to one embodiment of the present disclosure.

[0027] Figure 7 is an exemplary figure showing the 5-fold cross-validation performance in the model of Figure 6.

[0028] FIG. 8 is an exemplary diagram illustrating a process of inferring tumor probability using a learned patch classification model according to one embodiment of the present disclosure.

[0029] FIG. 9 is an exemplary drawing illustrating a process of generating a thumbnail image showing a suspected tumor area based on an inferred patch tumor probability according to one embodiment of the present disclosure.

[0030] FIG. 10 is an exemplary drawing showing a visualized tumor detection result reflecting the tumor detection image of FIG. 9 according to one embodiment of the present disclosure.

[0031] FIG. 11 is an exemplary drawing showing a plurality of tumor detection results according to the adjustment of a tumor probability threshold in accordance with one embodiment of the present disclosure.

[0032] FIG. 12 is an exemplary flowchart illustrating a method for detecting tumors using pathology slides based on artificial intelligence according to one embodiment of the present disclosure.

[0033] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. First, it should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present invention, if it is determined that a detailed description of related known components or functions could obscure the essence of the invention, such detailed description is omitted.

[0034] Various aspects of the present invention are described below. It should be understood that the inventions presented herein may be embodied in a wide variety of forms, and that any specific structure, function, or all thereof presented herein are merely illustrative. Based on the inventions presented herein, those skilled in the art will understand that any one aspect presented herein may be embodied independently of any other aspects, and that two or more such aspects may be combined in various ways. For example, an apparatus may be embodied or a method may be practiced using any number of aspects described herein. Furthermore, such an apparatus may be embodied or such a method may be practiced using structures, functions, or structures and functions other than those described herein, in addition to or other than these aspects.

[0035] The present disclosure may be applied to digital pathology solutions operated in hospitals, medical institutions, etc., but is not limited thereto, and may be used in applications that generate image patches from various types of images and provide information based on learning and inference results.

[0036] FIG. 1 is an exemplary block diagram illustrating a computing device for detecting tumors using pathology slides based on artificial intelligence according to one embodiment of the present disclosure.

[0037] As illustrated in FIG. 1, the computing device (100) may include a processor (110), a storage medium (120), a memory (130), and a network interface (140), which may be connected to each other via a system bus (150).

[0038] An operating system (OS) (122) and a computer program (124) may be installed on the storage medium (120). The storage medium (120) may be a data storage device such as a hard disk, an SSD (Solid State Drive), etc., capable of storing computer programs and related data. The operating system (122) may be operating system software such as Windows, iOS, Linux, etc., for operating the computing device (100). The computer program (124) may include functional modules for detecting tumors using pathology slides based on artificial intelligence according to the present disclosure, as well as computer-executable commands for this purpose. Additionally, the computer program (124) may be loaded into memory (130) so that it can be executed by the processor (110). When the computer-executable commands of the computer program (124) are executed by the processor (110), they may cause the processor (110) to perform the method for detecting tumors using pathology slides based on artificial intelligence according to the present disclosure. The processor (110) may be configured to provide computing and control capabilities to support the execution of the entire computing device (100). The processor (110) may be a data processing device such as a CPU (Central Processing Unit), MPU (Microprocessor Unit), AP (Application Processor), etc., and may be composed of one processor or multiple processors. If composed of multiple processors, the processors (110) may operate as parallel processing processors. The network interface (140) may provide an interface that can communicate data by connecting to an external device (e.g., a database storing pathology slide data, a display device, another wired or wireless communication device connectable via a network, etc.).

[0039] FIG. 2 is an exemplary block diagram showing functional modules of a computing device of FIG. 1 according to one embodiment of the present disclosure.

[0040] As illustrated in FIG. 2, the computing device (100) may include: a data processing unit (210) configured to acquire a thumbnail image from pathology slide data (i.e., raw data), separate a tumor region and a normal region from the thumbnail image, and generate image patches for the tumor region and the normal region; a learning unit (220) configured to train a patch classification model using a dataset configured based on the image patches of the tumor region and the image patches of the normal region; an inference unit (230) configured to input the image patches of the input pathology slide data (i.e., input data subject to tumor detection) generated by the data processing unit into the trained patch classification model to evaluate the tumor probability of each patch; and a result generation unit (240) configured to generate a visualized tumor detection result for the input pathology slide data and a tumor detection result report based thereon based on the evaluated tumor probability of each patch.

[0041] FIG. 3 is an exemplary diagram illustrating the process of obtaining a thumbnail image and annotating from pathology slide data according to one embodiment of the present disclosure.

[0042] As illustrated in FIG. 3, pathology slide data (i.e., pathology slide image) (310) can be acquired through a scanner for the digitization of pathology slides. This pathology slide data can typically be acquired as a low-magnification (e.g., X10 to X40) pathology slide image in a clinical setting. The data processing unit (210) can select a resolution level suitable for the purpose of analysis from the pathology slide data (310), remove unnecessary margins, and acquire a thumbnail image (320) of a size suitable for analysis. This data preprocessing process can contribute to increasing analysis efficiency and reducing the processing burden of large-capacity WSI data (i.e., the entire pathology slide image).

[0043] The data processing unit (210) can annotate (330) the tumor area of ​​the thumbnail image (320) and generate an XML file (340) containing coordinate data of each tumor area. Specifically, the data processing unit (210) can define the boundary line of the tumor area in the thumbnail image (320) using, for example, an open source library such as ASAP, and can generate an XML file (340) containing these tumor coordinates. Since the coordinate data of each tumor area is recorded in the XML file (340), the tumor location recorded in the XML file (340) can be utilized in the analysis process described later.

[0044] FIG. 4 is an exemplary drawing illustrating the process of generating a tumor mask and a normal mask according to one embodiment of the present disclosure.

[0045] As illustrated in FIG. 4, the data processing unit (210) can separate the tumor region and the normal region from the thumbnail image. To this end, the data processing unit (210) can convert the thumbnail image (320) from the RGB (Red, Green, Blue) color space to the HSV (Hue, Saturation, Value) color space. The value extracted from the converted HSV color space can be used to distinguish between the cell tissue and the background. Since the HSV color space separates and expresses hue, saturation, and value, it is easy for image analysis because value and color information can be processed independently. When only value is extracted from the HSV color space, only the brightness information of the image can be used, allowing the cell tissue and the background to be clearly distinguished.

[0046] The data processing unit (210) can divide the tissue region and the background region by applying an automated binarization algorithm, such as the OTSU algorithm, to the brightness of the HSV color space, and can generate a tissue mask (410) for the divided tissue region. The OTSU binarization algorithm can classify pixels into two classes (foreground and background) by automatically determining an optimal threshold value based on the histgram of the image. Through this, the OTSU binarization algorithm can effectively divide the tissue and the background and obtain accurate binarization results in an automated manner.

[0047] The data processing unit (210) can extract the tumor area within the tissue mask (410) using an XML file (340) containing the tumor location, and can generate a tumor mask (420) from it. The data processing unit (210) can generate a normal mask (430) by removing the tumor mask (420) from the tissue mask (410). By these tumor mask (420) and normal mask (430), the tumor area and the normal area can be clearly separated from the thumbnail image (320).

[0048] FIG. 5 is an exemplary drawing illustrating the process of generating image patches for a tumor region and a normal region according to one embodiment of the present disclosure.

[0049] As illustrated in FIG. 5, the data processing unit (210) can generate image patches for the tumor region and image patches (510) for the normal region through a patch generation algorithm using the generated tumor mask (420) and normal mask (430).

[0050] A patch generation algorithm can be configured to control factors for generating image patches. These factors may include patch size, patch downsampling rate, overlap rate between patches, proportion of tumor patches, proportion of normal patches, etc., and may be defined as exemplified in Table 1.

[0051]

[0052] For example, the data processing unit (210) can generate image patches (510) according to the ratio of image patches of adjusted tumor areas and the ratio of image patches of adjusted normal areas. Through this, diversity of the training data can be secured by appropriately adjusting the ratio of tumor areas and the ratio of normal areas. Additionally, for example, the data processing unit (210) can generate image patches (510) according to the overlap ratio between adjusted image patches. Through this, the overlap ratio between patches can be adjusted for data augmentation as described later, thereby contributing to the optimization of model performance.

[0053] The data processing unit (210) can construct a dataset based on the generated image patches of the tumor area and the image patches of the normal area, and this dataset can be provided to the learning unit (220) as a learning dataset (520) for learning a patch classification model.

[0054] In one embodiment, the training dataset (520) may be composed of K datasets of image patches for K-fold cross-validation and provided to the training unit (520), and the training unit (220) may train K independent patch classification models using these K datasets as described below.

[0055] For convenience of explanation, the following example is given with K=5 (i.e., 5-fold cross-validation). In this case, the training dataset (520) may consist of five datasets (i.e., folds) composed of training image patches and test image patches. As illustrated in FIG. 5, for 5-fold cross-validation, the training dataset (520) can be configured to repeat the process of using each fold as a test dataset once and the remaining folds as a training dataset. This allows for more reliable evaluation of the model's performance, reduces data bias, and improves generalization performance.

[0056] In an exemplary implementation, the training dataset consisted of a total of 410,000 image patches, of which 330,000 (80%) were used for training and 80,000 (20%) were used for validation. To address class imbalance, the non-tumor / tumor patch ratio was set to 1:1 for training, and 20,000 unseen patches were used for evaluation to verify the model's generalization performance.

[0057] FIG. 6 is an exemplary drawing illustrating the process of training a patch classification model according to one embodiment of the present disclosure.

[0058] As illustrated in FIG. 6, the learning unit (220) can train an artificial intelligence (AI) model (610) for patch classification using a learning dataset (520). In one embodiment, this patch classification model (610) may apply a Vision Transformer-based Swin V2 Tiny model. This Swin V2 Tiny model is suitable for pathology image analysis as it can effectively learn scaling and global context information, and has the advantage of being able to effectively process the complex structure and various patterns of pathology slide images. In this exemplary implementation, the model learning parameter may be patch_size [3,192,192], and the hyperparameters may be set to batch size 800, epoch 500, and initial learning rate 1e-04, and CrossEntropyLoss may be used as the loss function. CrossEntropyLoss is advantageous for improving the model's prediction performance by effectively minimizing the difference between the predicted probability for each class and the actual class in multi-class classification problems. Meanwhile, the learning model of the learning unit (220) is not limited to the Swin V2 Tiny model, and depending on the implementation example, a convolutional neural network (CNN) or other Transformer-based artificial intelligence model capable of performing feature extraction and classification may be applied.

[0059] Additionally, as illustrated in FIG. 6, the learning unit (220) may apply data augmentation (620) to the learning dataset (520) and provide it as a patch classification model (610). For example, the learning unit (220) may apply rotation and inversion, brightness and contrast, color change, image quality, intensity normalization, tensor transformation, etc., as data augmentation. Through such data augmentation, various forms of learning data can be generated to contribute to improving the generalization performance of the model.

[0060] Additionally, as illustrated in FIG. 6, the learning unit (220) may be configured to apply a modified classification layer (630) to the patch classification model (610). For example, the modified classification layer may include Linear, ReLU, BatchNorm1d, Dropout, etc., as illustrated in Table 2.

[0061]

[0062] By modifying these classification layers, features can be extracted through dimensionality reduction, complex tumor patterns can be learned by introducing non-linearity, learning stabilization and speed improvement can be achieved through batch normalization, and overfitting prevention and generalization performance can be improved. As a result, the patch classification model (610) can classify tumors and normal tissues in pathology slide images more accurately and can be designed to have a custom layer optimized to better reflect specific pathological features. Accordingly, the patch classification model (610) according to the present disclosure can be implemented to infer DCIS (ductal carcinoma in situ), IDC (invasive ductal carcinoma), ILC (invasive lobular carcinoma), inflammation, necrosis, fat, etc.

[0063] The learning unit (220) can improve the accuracy of class classification (640) (e.g., normal tissue (0) and tumor tissue (1)) through the learned patch classification model (610). Additionally, using the learning dataset (520) for 5-fold cross-validation as described above, the learning unit (220) can train five independent patch classification models (650) (i.e., Fold 1, Fold 2, Fold 3, Fold 4, Fold 5).

[0064] Figure 7 is an exemplary figure showing the 5-fold cross-validation performance in the model of Figure 6.

[0065] As shown in Fig. 7, when the model of Fig. 6 was trained through 5-fold cross-validation, the accuracy (i.e., F1 score) at each fold was recorded as 0.9495, 0.9506, 0.9524, 0.9515, and 0.9525 for folds 0 to 5, respectively. In addition, the average performance of the model achieved an F1 score of 0.9513, which is greater than or equal to 0.93.

[0066] FIG. 8 is an exemplary diagram illustrating a process of inferring tumor probability using a learned patch classification model according to one embodiment of the present disclosure.

[0067] As illustrated in FIG. 8, input pathology slide data (i.e., input data to be detected for tumor detection) (810) can be processed into image patches according to a data processing algorithm (820) performed by a data processing unit (210) (as described in relation to FIG. 5). An inference unit (230) can generate overlapping patches (i.e., preprocessed patches) (840) by applying a sliding window algorithm to the image patches generated from the input pathology slide data (810). This sliding window algorithm sequentially scans the image in areas of a certain size, allowing each patch to partially overlap. This method preserves detailed information and contextual information of high-resolution images and minimizes information loss in boundary areas between patches. Additionally, this method enables the detection of tumor areas without omission by analyzing the entire image thoroughly.

[0068] The inference unit (230) can input these patches (840) in parallel to five independent tumor patch classification models (850) that have been pre-trained in the learning unit (220) for 5-fold cross-validation. The inference unit (230) can combine the prediction results from each patch classification model (fold 1, fold 2, fold 3, fold 4, fold 5) and apply an ensemble technique to calculate the average tumor probability of each patch. This ensemble approach can reduce overfitting of individual models, improve the stability of predictions, and increase the generalization ability of the entire system. The inference unit (230) can evaluate the average patch tumor probability calculated for each patch. If such probability exceeds a preset tumor probability threshold, the inference unit (230) can obtain a thumbnail image (860) that reflects the suspected tumor area by displaying the coordinates of the patch on a thumbnail of the original image (i.e., input pathology slide data) (810).

[0069] FIG. 9 is an exemplary drawing illustrating a process of generating a thumbnail image showing a suspected tumor area based on an inferred patch tumor probability according to one embodiment of the present disclosure.

[0070] As illustrated in FIG. 9, the result generation unit (240) may be configured to generate a pathology slide image (930) with a tumor contour marked as a tumor detection result. To this end, the result generation unit (240) may apply a tumor region contour generation algorithm to more accurately identify and visualize a suspected tumor region on a thumbnail image (860) inferred by the inference unit (230). This algorithm may receive coordinate information of individual patches marked on the thumbnail image (860) as input and convert it into a clinically significant tumor region contour. Through this process, the initial prediction of the model can be refined and visual information that is clearer and easier to interpret can be provided to the pathologist. This tumor contour generation process may include three steps as described below.

[0071] The result generation unit (240) can generate a tumor suspicion mask (910) representing a tumor suspicion area based on the patch tumor probability evaluated by the inference unit (230) and a preset tumor probability threshold. Specifically, the result generation unit (240) can generate a binary mask by receiving coordinate information of individual patches displayed in a thumbnail image (860) as input. In this process, an initial tumor area can be estimated based on the location information of patches classified as tumor suspicion areas. The binary mask can visualize the approximate shape and location of the tumor by representing the tumor suspicion area in white (1) and the non-tumor area in black (0). This serves as the basis for subsequent processing steps and can provide a more stable initial tumor area by spatially integrating the prediction results of the ensemble model (850).

[0072] Next, the result generation unit (240) can apply small area filtering to the suspected tumor mask (920). This filtering may include average area filtering (filterUsingMean) and clustering-based filtering (filterUsingKMeans). Average area filtering can remove small components based on the average area of ​​connected components and can effectively remove most small areas (noise). Clustering-based filtering can identify and remove groups of small components by considering not only the size but also the distribution of connected components. By combining these two methods, it is possible to effectively respond to false positives of various sizes and patterns, and to improve the accuracy and reliability of subsequent tumor area contour generation by removing fine noise while preserving the main tumor area.

[0073] Next, the result generation unit (240) can generate a pathology slide image (930) with tumor contours by applying a contour algorithm to the input pathology slide data using a small-region filtered tumor suspicion mask. This contour algorithm may sequentially apply various image processing techniques to increase the accuracy and visibility of the tumor contours, and these processing techniques are exemplified in Table 3.

[0074]

[0075] As exemplified in Table 3, the result generation unit (240) can first increase the computational processing efficiency of the pathology slide image through image resizing, fill and refine small holes within the tumor area through morphological expansion operations, remove noise and smooth the contours through blurring, then reduce the area previously expanded by morphological expansion and refine the boundaries through morphological erosion operations, and finally remove the noise area through connected component filtering.

[0076] Through this data post-processing process, the result generation unit (240) can automatically generate and provide clinically useful visualizations that allow a pathologist to easily identify the characteristics of the tumor.

[0077]

[0078] * FIG. 10 is an exemplary drawing showing a visualized tumor detection result reflecting the tumor detection image of FIG. 9 according to one embodiment of the present disclosure.

[0079] The result generation unit (240) can provide a visualized tumor detection result including detailed information such as the size, shape, and location of the tumor, file list, log information, etc., along with a pathology slide image (1030) showing a tumor contour that can be generated as described in FIG. 9. In addition, so that pathologists can visually analyze the tumor without separate high-performance equipment, the software providing these tumor detection results can be implemented to operate on various GPUs that take into account the hardware environment commonly used in clinical settings (e.g., PC environment).

[0080] FIG. 11 is an exemplary drawing showing a plurality of tumor detection results according to the adjustment of a tumor probability threshold in accordance with one embodiment of the present disclosure.

[0081] The result generation unit (240) may be configured to generate and provide multiple tumor detection results according to each of the multiple thresholds for the patch tumor probability evaluated by applying a tumor probability threshold adjustment algorithm. In the embodiment according to FIG. 11, the result generation unit (240) may adjust the tumor probability thresholds so that tumor inference results with tumor probabilities of 50%, 70%, and 90% or higher, respectively, can be viewed, and may provide tumor detection results (1110, 1120, 1130) according to the respective reliability (i.e., Soft, Moderate, Hard). In addition, the result generation unit (240) may provide additional information (1140), such as the maximum length (Tmax), size (Size), and area (Area) of the inferred tumor according to each reliability, along with the corresponding tumor detection results. Additionally, the tumor inference reliability is not limited to the aforementioned numerical value, and the result generation unit (240) may be configured to provide a tumor detection result based on a reliability (i.e., threshold) adjusted according to user settings, or to provide a tumor detection result by adjusting the reliability to an optimal threshold determined based on the probability distribution of the inferred tumor region.

[0082] FIG. 12 is an exemplary flowchart illustrating a method for detecting a tumor using a pathology slide based on artificial intelligence, executable by a computing device (100) according to one embodiment of the present disclosure.

[0083] As illustrated in FIG. 12, a computing device (100) can acquire a thumbnail image from pathology slide data (1210). The computing device (100) can separate tumor regions and normal regions from the acquired thumbnail image (1220). The computing device (100) can generate image patches for the separated tumor regions and normal regions (1230). The computing device (100) can train a patch classification model using a dataset configured based on the image patches of the tumor regions and the image patches of the normal regions (1240). The computing device (100) can input the image patches generated from the input pathology slide data into the trained patch classification model to evaluate the tumor probability of each patch (1250). The computing device (100) can generate a visualized tumor detection result for the input pathology slide data based on the evaluated tumor probability of each patch (1260).

[0084] It should be understood that any particular order or hierarchy of steps in any of the presented processes is an example of exemplary approaches. It should be understood that, based on design priorities, any particular order or hierarchy of steps in the processes may be rearranged within the scope of the invention. The appended method claims provide elements of various steps in an exemplary order, but do not imply limitation to the particular order or hierarchy presented.

[0085] As used herein, terms such as “component,” “unit (or part),” “module,” “system,” etc., may refer to computer-related entities, hardware, firmware, software, combinations of software and hardware, or executions of software. For example, a component may be a process, processor, object, execution thread, program, and / or computer executed on a processor, but is not limited thereto. For example, both an application executed on a computing device and the computing device itself may be a component. One or more components may reside within a processor and / or execution thread, and a component may be localized within a single computer or distributed among two or more computers. Additionally, these components may be executed from various computer-readable media having various data structures stored therein.

[0086] The description of the presented embodiments is provided so that any person skilled in the art may use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present invention. Thus, the present invention is not limited to the embodiments presented herein, but should be interpreted in the broadest possible scope consistent with the principles and novel features presented herein.

[0087] (Explanation of symbols)

[0088] 100: Computing device

[0089] 110: Processor

[0090] 120: Storage media

[0091] 122: Operating System

[0092] 124: Computer program

[0093] 130: Memory

[0094] 140: Network Interface

[0095] 150: System bus

[0096] 210: Data Processing Department

[0097] 220: Learning Department

[0098] 230: Inference section

[0099] 240: Result generation section

Claims

1. A device for detecting tumors using pathology slides based on artificial intelligence, A data processing unit configured to acquire a thumbnail image from pathology slide data, separate a tumor region and a normal region from the thumbnail image, and generate image patches for the tumor region and the normal region; A learning unit configured to train a patch classification model using a dataset configured based on image patches of the tumor region and image patches of the normal region; An inference unit configured to evaluate the tumor probability of each patch by inputting image patches of input pathology slide data generated by the data processing unit into the learned patch classification model; and A result generation unit configured to generate visualized tumor detection results for the input pathology slide data based on the tumor probability of each evaluated patch, comprising device.

2. In Paragraph 1, The above data processing unit is, Configured to annotate the tumor regions of the above thumbnail image and generate an XML file containing coordinate data for each tumor region, device.

3. In Paragraph 2, The above data processing unit is, The above thumbnail image is configured to convert from the RGB color space to the HSV (Hue, Saturation, Value) color space, and to segment tissue and background regions by applying the OTSU binarization algorithm to the value of the HSV color space. device.

4. In Paragraph 3, The above data processing unit is, A method configured to separate the tumor region and the normal region from the thumbnail image by generating a tissue mask for the tissue region, generating a tumor mask from the tissue mask using the XML file, and generating a normal mask by removing the tumor mask from the tissue mask. device.

5. In Paragraph 1, The above data processing unit is, Configured to generate image patches of the tumor region and image patches of the normal region according to the overlap ratio between adjusted image patches or the ratio of adjusted image patches of the tumor region and the normal region, device.

6. In Paragraph 1, The above patch classification model includes a Vision Transformer-based Swin V2 Tiny model, device.

7. In Paragraph 1, The above dataset is composed of K datasets of the image patches for K-fold cross-validation and is provided to the learning unit, and The above learning unit is configured to train K independent patch classification models using the above K datasets, device.

8. In Paragraph 1, The above learning unit is, Configured to apply data augmentation to the above dataset and provide it as the above patch classification model, and to apply a modified classification layer to the above patch classification model, device.

9. In Paragraph 7, The above inference unit, A method configured to determine the average tumor probability of each patch by applying a sliding window algorithm to image patches generated by the data processing unit for input pathology slide data, inputting the generated patches into the K trained patch classification models, and ensembling the predictions from each patch classification model. device.

10. In Paragraph 1, The above result generating unit is, A method configured to generate a tumor suspicion mask indicating a suspected tumor region based on a patch tumor probability evaluated in the inference unit and a preset tumor probability threshold, apply small area filtering to the tumor suspicion mask, and apply a contour algorithm to the input pathology slide data using the small area filtered tumor suspicion mask to generate a pathology slide image with tumor contours displayed as a tumor detection result. device.

11. In Paragraph 10, The above result generating unit is, Configured to generate multiple tumor detection results according to each of multiple thresholds for the evaluated patch tumor probability by applying a tumor probability threshold adjustment algorithm, device.

12. A method for detecting tumors using pathology slides based on artificial intelligence executable by a computing device, A step of acquiring a thumbnail image from pathology slide data; A step of separating the tumor region and the normal region from the above thumbnail image; A step of generating image patches for the tumor region and the normal region; A step of training a patch classification model using a dataset configured based on image patches of the tumor region and image patches of the normal region; A step of inputting image patches generated from input pathology slide data into the learned patch classification model to evaluate the tumor probability of each patch; and A step comprising generating visualized tumor detection results for the input pathology slide data based on the tumor probability of each evaluated patch, method.

13. In Paragraph 12, The step of acquiring the above thumbnail image includes the step of commenting out the tumor region of the above thumbnail image and generating an XML file containing coordinate data of each tumor region, and The step of separating the tumor region and the normal region is, A step of converting the above thumbnail image from an RGB color space to an HSV color space, and applying an OTSU binarization algorithm to the brightness of the HSV color space to segment tissue regions and background regions; and A method comprising the steps of generating a tissue mask for the tissue region, generating a tumor mask from the tissue mask using the XML file, and generating a normal mask by removing the tumor mask from the tissue mask. method.

14. In Paragraph 13, The step of generating the above image patches includes the step of generating image patches of the tumor area and image patches of the normal area according to the overlap ratio between the adjusted image patches or the ratio of the image patches of the adjusted tumor area and the normal area. method.

15. In Paragraph 12, The above patch classification model includes the Vision Transformer-based Swin V2 Tiny model, method.

16. In Paragraph 12, The above dataset consists of K datasets of the image patches for K-fold cross-validation, and The step of training the patch classification model includes the step of training K independent patch classification models using the K datasets. method.

17. In Paragraph 12, The step of training the above patch classification model is, The step of applying data augmentation to the above dataset and providing it as the patch classification model; and A method comprising the step of applying a modified classification layer to the above-mentioned patch classification model, method.

18. In Paragraph 16, The step of evaluating the tumor probability of each of the above patches is, A step of applying a sliding window algorithm to image patches generated from input pathology slide data; A step of inputting patches generated according to the above sliding window algorithm into the above-mentioned K patch classification models; and A step comprising determining the average tumor probability of each patch by ensembling predictions from each patch classification model, method.

19. In Paragraph 12, The step of generating the above tumor detection result is, A step of generating a tumor suspicion mask indicating a tumor suspicion area based on an evaluated patch tumor probability and a preset tumor probability threshold; Step of applying small-region filtering to the above-mentioned tumor suspicion mask; and A method comprising the step of generating a pathology slide image with tumor contours displayed as a tumor detection result by applying a contour algorithm to the input pathology slide data using the above-mentioned small-region filtered tumor suspicion mask. method.

20. In Paragraph 19, The step of generating the above tumor detection result is, A method further comprising the step of generating multiple tumor detection results according to each of the multiple thresholds for the evaluated patch tumor probability by applying a tumor probability threshold adjustment algorithm. method.

21. A computer program stored on a computer-readable medium comprising computer-executable instructions for executing a method according to any one of claims 12 through 20.

Images