Pleural fluid cytological detection assisted device and method thereof

Abstract

According to the present embodiments, an apparatus and method are provided for extracting a plurality of tile images from a cytopathology slide image of pleural fluid and classifying at least one class among the presence of cancer and the type of cancer from a cytopathology slide image of any pleural fluid using a prediction model that performs annotation-based learning on the cytopathology slide image or the tile image.

Description

Technology Field

[0001] The embodiments of the present disclosure relate to a pleural fluid cell pathology examination support device and a method that support pleural fluid cell pathology examination using artificial neural network image analysis technology. Background Technology

[0002] Currently, pleural fluid examination typically relies on cytopathology as the basic screening method, which involves smearing the fluid onto a slide and examining it under a microscope. However, reports indicate that the sensitivity of cytopathology is very low because it becomes difficult to distinguish malignant cells when reactive mesothelial cells become activated in environments where the pleura is irritated, such as by inflammation. These false negative results lead to missed opportunities for timely treatment or failure to properly diagnose recurrence, resulting in patients losing treatment opportunities, poor prognoses, and contributing to increased medical costs.

[0003] Currently, other screening tests and tumor markers utilizing molecular pathology techniques are being developed to replace cytopathology tests; however, they are not widely used as replacements because the costs are very high and their sensitivity and accuracy do not meet expectations.

[0004] Recently, artificial neural network image analysis technology has advanced significantly and is demonstrating promising results when applied to the fields of classification, detection, and measurement using various digital pathology images. Applying this to the analysis of pleural fluid cytopathology specimens is expected to enable more accurate early diagnosis through a simple, non-invasive, and very low-cost examination.

[0005] Applying this to pleural fluid cytopathology is expected to enable more accurate early diagnosis through a simple, low-cost, non-invasive test. Prior art literature

[65535] Korean Patent Publication No. 10-2023-0023568 (Publication Date: Feb. 17, 2023): Method and apparatus for outputting information related to pathology slide images Korean Patent Publication No. 10-2023-0011895 (Publication Date: Jan. 25, 2023): Method and system for analyzing pathology images The problem to be solved

[0006] The present embodiments provide a pleural fluid cytopathology support device and a method thereof that enable more accurate pleural fluid cytopathology examination through a simple, non-invasive examination at a very low cost. means of solving the problem

[0007] The present embodiments provide an apparatus and a method for extracting a plurality of tile images from pleural fluid cell pathology slide images of body fluids of pleural fluid, and classifying at least one class among the presence of cancer and the type of cancer from any plurality of pleural fluid cell pathology slide images using a prediction model that performs annotation-based learning on the pleural fluid cell pathology slide images or tile images.

[0008] In one aspect, a pleural fluid cytopathology examination support device according to one embodiment includes a preprocessing unit that extracts a plurality of tile images from a pleural fluid cytopathology slide image of a pleural fluid body fluid, and a classification unit that classifies at least one class among the presence of cancer and the type of cancer in any pleural fluid cytopathology slide image using a prediction model that performs annotation-based learning using a pleural fluid cytopathology slide image or a plurality of tile images.

[0009] In another aspect, a method for supporting cytopathology examination according to another embodiment includes an extraction step of extracting a plurality of tile images from a cytopathology slide image of pleural fluid, and a classification step of classifying at least one of the presence of cancer and the type of cancer in any cytopathology slide image of pleural fluid using a prediction model that has undergone annotation-based learning using the cytopathology slide image or the plurality of tile images. Effects of the invention

[0010] The cell pathology examination support device and method according to the embodiments can provide high accuracy and efficiency, thereby greatly assisting in the diagnosis and treatment of pathology examinations.

[0011] The cell pathology examination support device and method according to the present embodiments can examine more accurate cell pathology with a simple, non-invasive examination at a very low cost. Brief explanation of the drawing

[0012] FIG. 1 is a block diagram of a cytopathology examination support device according to one embodiment. FIG. 2 is a conceptual diagram of WSI (Whole Slide Image) image extraction according to the present embodiment. Figure 3 illustrates the processes of generating multiple tile images from an original slide image. Figure 4 is a schematic diagram of the three-dimensional phase difference in a cell pathology slide according to the present embodiment. FIG. 5 is a diagram illustrating Z-stacking for overcoming three-dimensional phase difference according to the present embodiment. FIG. 6 is a diagram illustrating an example of a supervised learning method according to the present embodiment. FIG. 7 is a flowchart of a method (200) for supporting cell pathology examination according to another embodiment. FIG. 8 is a configuration diagram of a computing system according to embodiments of the present invention. FIG. 9 is a configuration diagram of a client-server computer system according to embodiments of the present invention. Specific details for implementing the invention

[0013] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the embodiments, if it is determined that a detailed description of related known components or functions may obscure the essence of the technical concept, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless otherwise specified.

[0014] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0015] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0016] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0017] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0018] The embodiments are described in detail below with reference to the drawings.

[0019] FIG. 1 is a block diagram of a cytopathology examination support device according to one embodiment.

[0020] Referring to FIG. 1, a cytopathology examination support device (100) according to one embodiment extracts a plurality of tile images (20) from a cytopathology slide image (10) of pleural fluid and classifies at least one class of whether there is cancer and the type of cancer from a cytopathology slide image (10) of any pleural fluid using a prediction model (122) that performs annotation-based learning on the cytopathology slide image (10) or the tile images (20).

[0021] A cytopathology examination support device (100) according to one embodiment includes a preprocessing unit (110) for preprocessing a cytopathology slide image (10) of pleural fluid and a classification unit (120) for classifying one class from a cytopathology slide image (10) of any pleural fluid using a prediction model (122) that has undergone annotation-based learning.

[0022] The preprocessing unit (110) extracts a plurality of tile images (20) from a cellular pathology slide image (10) of the pleural fluid.

[0023] The preprocessing unit (110) uses image processing techniques such as image segmentation and fusion techniques to extract tile images (20) from the cellular pathology slide images (10) of the pleural fluid, and adjusts the size and resolution of the tile images to enable efficient learning and prediction.

[0024] The classification unit (120) classifies at least one class of cancer and the type of cancer in a cytopathology slide image (10) of any pleural fluid using a prediction model (122) that has performed annotation-based learning using a cytopathology slide image (10) or two or more tile images (20).

[0025] At this time, the cell pathology slide image (10) may be a whole slide image (WSI (Whole Slide Image)) as an example of a slide image of pleural fluid.

[0026] Figure 2 is a conceptual diagram for extracting a whole slide image (WSI).

[0027] As illustrated in FIG. 2, a full slide image is extracted by smearing the pleural fluid onto a glass slide and photographing or scanning it. The extracted full slide image may be an unprocessed original slide image.

[0028] FIG. 3 illustrates the processes of generating multiple tile images from an original slide image. The processes of FIG. 3 include Z-stacking or focus stacking processes and color normalization processes.

[0029] Referring to FIG. 3, the cell pathology slide image (10) is obtained by using Z-stacking or focus stacking techniques from an original slide image (12) that is photographed or scanned by smearing the pleural fluid onto a glass slide.

[0030] Specifically, a cell pathology slide image (10) can be obtained by synthesizing images (14) focused at different phases from an original slide image (12) into a single image (16) through secondary post-processing using Z-stacking or focus stacking techniques.

[0031] For example, the whole slide image of the cytopathology (WSI) has a three-dimensional structure within the slide due to the characteristics of the cell specimen, as shown in FIG. 4. Therefore, to observe the cell nucleus, nucleolus, cytoplasm, etc., the whole slide image of the cytopathology (WSI) may require scanning with two or more images (14) focused at different phases at a high magnification, for example, 40x.

[0032] For example, to overcome three-dimensional phase differences, two or more images (14) focused at different phases, for example, 5 to 20, can be acquired and stored and displayed, or they can be combined into one image (16) through secondary post-processing.

[0033] As illustrated in FIG. 5, for example, images focused at five different phases (z=0 to z=4) can be obtained and all used, or images focused at different phases (z=0 to z=4) can be synthesized into one image (16) through secondary post-processing such as averaging, maximizing, minimizing, or applying a focus-stacking algorithm to obtain a cell pathology slide image (10).

[0034] Next, as illustrated in FIG. 3, through research on standardization technology of scanned digital images, a color-normalized image (18) can be obtained that matches the color of the staining, which may appear different due to various staining conditions. Additionally, the cytopathology slide image (10) can be corrected using image processing technology for various artifacts that may occur during the slide preparation process, such as tissue detachment, crushing, air bubbles, dust, foreign substances, etc.

[0035] The cell pathology slide image (10) may be any one of the original slide image (12), images (14) focused at different phases from the original slide image (12), a single image (16) synthesized through secondary post-processing, and a color normalized image (18).

[0036] Additionally, the cytopathology slide image (10) may be a slide image obtained by omitting some of the processes described with reference to FIG. 4. For example, the cytopathology slide image (10) may be a single image (16) synthesized through secondary post-processing without color normalization. The cytopathology slide image (10) may be an image (18) obtained by color normalizing the original slide image (12) without applying images (14) focused at different phases from the original slide image (12) and a single image (16) synthesized through secondary post-processing.

[0037] Furthermore, the extracted lesion region, for example, a cancer region, can be cropped to a specific size and extracted into multiple tile images or structured patch data suitable for training. Additionally, class annotation information assigned to the cytopathology slide image can be assigned to all tile images or patch data extracted from the corresponding cytopathology slide image.

[0038] Meanwhile, multiple tile images may be smaller in size than the cell pathology slide image.

[0039] For this reason, cytopathology images including the aforementioned cytopathology slide images (10) or tile images (20) can be stored as files having a capacity 5 to 10 times that of general histopathology images. For example, while histopathology images are on average 1 Gb, cytopathology images can be on average 10 Gb.

[0040] The preprocessing unit (110) can generate a plurality of tile images (20) based on a sliding window algorithm. That is, the preprocessing unit (110) can generate a plurality of tile images by repeating the process of extracting a tile image that overlaps with the sliding window on a cell pathology slide image, then moving the position of the sliding window, and then extracting the tile image again.

[0041] For example, multiple tile images may be RGB images having an R (Red) channel, a B (Blue) channel, and a G (Green) channel.

[0042] When the classification unit (120) uses a prediction model (122) that has undergone annotation-based learning using a cytopathology slide image (10) or two or more tile images (20), the annotation-based learning allows the prediction model (122) to learn accurately by having an expert directly annotate the extracted lesion area, for example, the cancer area.

[0043] A prediction model (122) that has undergone annotation-based learning can be trained by adding one or more of the following annotations to the cytopathology slide image (10) or multiple tile images (20) used for learning: a partial annotation (32) that indicates a cancer area in the form of a line, a bounding box annotation (34) that indicates a cancer area in the form of a box, and an image-level label (36) that indicates the entire image. The form of the annotation is not limited to a line or a bounding box and can be diverse. For example, the form of the annotation can be a line, an ellipse, a bracket, etc.

[0044] In other words, the prediction model (122) that has undergone annotation-based learning can learn through a cell pathology slide image (10) or a plurality of tile images (20) to which annotations indicating cancer regions have been added.

[0045] This prediction model (122) may be a cytopathology slide-based Neoplasm prediction model. For example, the prediction model may be developed as a weakly-supervised learning model that can predict square tile-level results using slide-level labels and a square tile detection algorithm in which tissue is present in the whole cytopathology slide image (WSI).

[0046] Specifically, a loss function known to work well for classification model training can be applied to the model training. Additionally, model training can be performed based on annotations regarding whether the object is cancerous on a slide-by-slide basis. As previously mentioned, the annotations may be partial annotations (32), bounding box annotations (34), and image-level labels (36).

[0047] This prediction model (122) proceeds with a data collection step, a data preprocessing step as described above, a model training step that performs annotation-based learning or weakly-supervised learning, and a model validation step that validates the trained model. When an arbitrary cytopathology slide image is input, the model analyzes the arbitrary cytopathology slide image and can classify at least one class of whether it is cancer and the type of cancer. The prediction model (122) can also identify the location of the cancer using an image classification algorithm.

[0048] The presence of cancer can be classified as either positive or negative. The types of cancer may include lung and breast cancer occurring in the pleural effusion.

[0049] This prediction model (122) can distinguish only whether or not there is cancer, or distinguish the type of cancer along with whether or not there is cancer.

[0050] For example, the prediction model (122) may distinguish only whether or not there is pleural fluid, or distinguish the type of cancer along with whether or not there is cancer. Specifically, the prediction model (122) may be configured as a model for distinguishing between lung cancer and breast cancer depending on the type of cancer.

[0051] The prediction model (122) may be a model that distinguishes each cancer. For example, the prediction model (122) may be a model that distinguishes whether or not it is lung cancer. The prediction model (122) may be a model that distinguishes whether or not it is breast cancer.

[0052] This prediction model (122) defines whether or not there is cancer or the type of cancer by classifying them into respective classes, and when a request is made to classify a class for a slide image (10) of a cellular pathology of any pleural fluid, it can classify that class as a result of annotation-based learning.

[0053] Additionally, the prediction model (122) can be generated using an ensemble learning method. While a single prediction model (122) may classify the presence of cancer and the type of cancer as described above, there may also exist prediction models (122) for thoracotylosis learned for each type of cancer, and for each prediction model (122) for thoracotylosis learned for each type of cancer, it may determine whether it corresponds to the corresponding cancer and classify the presence of cancer and the type of cancer by combining the prediction results of each prediction model (122).

[0054] For example, regarding a cytopathology slide image (10) of a random pleural fluid, a prediction model (122) trained as lung cancer can classify it as lung cancer. A prediction model (122) trained as breast cancer can classify it as not corresponding to breast cancer. In this case, the classification unit (120) can classify the cytopathology slide image (10) of a random pleural fluid as lung cancer but not corresponding to breast cancer.

[0055] As described above, the prediction model (122) proceeds with a data collection step, a data preprocessing step, a model training step that performs annotation-based learning or weakly-supervised learning, and a model validation step that validates the trained model, and these processes and results are described exemplarily below.

[0056] For example, the prediction model (122) can define model-specific optimization parameters for improving accuracy. Additionally, the prediction model (122) can define parameters for algorithm comparison and performance optimization suitable for medical data characteristics.

[0057] For example, the key optimization parameters can be shown as in Table 1.

[0058]

[0059] As another example, the prediction model (122) can apply a data learning algorithm and perform data learning. Specifically, it can build a server for image learning and create a quality result report for the entire dataset.

[0060] For example, the quality result ports for the entire dataset may appear as shown in Table 2.

[0061]

[0062] In addition, the prediction model (122) can use a CNN (Convolutional Neural Network) algorithm as an algorithm for learning image data. Specifically, the CNN algorithm is receiving attention as one of the two major pillars of deep learning models along with the RNN (Recurrent Neural Network), and is basically based on the structure proposed by Yann LeCun in 1989.

[0063] For example, the CNN algorithm can be applied as the AlexNet algorithm and can be composed of a conv layer, a max-pooling layer, five dropout layers, three fully connected layers, and a nonlinearity function (ReLU, batch stochastic gradient descent).

[0064] In addition, CNN algorithms can utilize the GoogleNet algorithm. While simply stacking convolutional layers with a single convolutional filter applied deep is an option, individual layers can be expanded thickly by introducing various types of filters or pooling within a single layer.

[0065] For example, the scale of data construction can be shown as in Table 3.

[0066]

[0067] For example, the performance of a data learning algorithm can be calculated based on the summary table of AI data utilization model development, the validation environment, and the learning conditions.

[0068] For example, the development summary table of the prediction model (122) can be shown as Table 4.

[0069]

[0070] For example, the validation environment and learning conditions may be shown as in Table 5.

[0071]

[0072] Performance is evaluated on the test data set among the established data sets, and there should be no overlap between the test data set and the basic data set. In addition, as described above, the prediction model (122) can be based on annotations regarding whether it is a slide-unit Neoplasm.

[0073] Accuracy can be defined as a cutoff criterion positive / negative prediction score compared with the reference standard result of each cancer type prediction model (122). A 2x2 table can be created based on the defined result. The created table can be shown as Table 6.

[0074]

[0075] Here, accuracy can refer to the sum of true positives and true negatives. Additionally, the total sample size accuracy can range from 0 to 100%, and a value closer to 100% indicates ideal diagnostic performance.

[0076] As another example, acquired digital cytopathology slide images can be basically divided into training / validation / test data in a ratio of approximately 8:1:1 considering the quantitative distribution by class, and can be configured in a final ratio of 8:1:3 considering the importance of the test data set.

[0077] According to the embodiments, a classification model can be provided that distinguishes four types of cancer, including ovarian cancer, colorectal cancer, gastric cancer, and pancreatic cancer, occurring in pleural fluid by utilizing digitally scanned images of glass slides for pleural fluid cytopathology examination. When applied to the analysis of pleural fluid cytopathology specimens according to the classification model, a classification model can be provided that enables more accurate early diagnosis through a simple, non-invasive examination at a very low cost.

[0078] FIG. 7 is a flowchart of a method (200) for supporting cell pathology examination according to another embodiment.

[0079] Referring to FIG. 7, a cytopathology examination support method (200) according to another embodiment includes an extraction step (S210) for extracting a plurality of tile images from a cytopathology slide image of pleural fluid, and a classification step (S220) for classifying at least one class among the presence of cancer and the type of cancer from a cytopathology slide image of any pleural fluid using a prediction model that performs annotation-based learning using the cytopathology slide image or the plurality of tile images.

[0080] As described above with reference to FIGS. 2 to 5, the cytopathology slide image (10) can be obtained by photographing the original slide image scanned by smearing it onto a glass slide of pleural fluid or by using Z-stacking or focus stacking techniques.

[0081] Additionally, the cell pathology slide image (10) can be synthesized into a single image (16) through secondary post-processing of images (14) focused at different phases from the original slide image (12) using Z-stacking or focus stacking techniques.

[0082] In the preprocessing step (S210), multiple tile images can be generated based on a sliding window algorithm.

[0083] As described above, the prediction model (122) that has undergone annotation-based learning can be trained by adding one or more of the following annotations to the cytopathology slide image or a plurality of tile images used for learning: a partial annotation (32) that indicates the cancer area in a line shape, a bounding box annotation (34) that indicates the cancer area in a box shape, and an image level label (36) that indicates the entire image.

[0084] The presence of cancer can be classified as either positive or negative. The types of cancer may include lung and breast cancer occurring in the pleural effusion.

[0085] This prediction model (122) can distinguish only whether or not there is cancer, or distinguish the type of cancer along with whether or not there is cancer.

[0086] For example, the prediction model (122) may distinguish only whether or not there is pleural fluid, or distinguish the type of cancer along with whether or not there is cancer. Specifically, the prediction model (122) may be configured as a model for distinguishing between lung cancer and breast cancer depending on the type of cancer.

[0087] The prediction model (122) may be a model that distinguishes each cancer. For example, the prediction model (122) may be a model that distinguishes whether or not it is lung cancer. The prediction model (122) may be a model that distinguishes whether or not it is breast cancer.

[0088] This prediction model (122) defines whether or not there is cancer or the type of cancer by classifying them into respective classes, and when a request is made to classify a class for a slide image (10) of a cellular pathology of any pleural fluid, it can classify that class as a result of annotation-based learning.

[0089] Additionally, the prediction model (122) can be generated using an ensemble learning method. While a single prediction model (122) may classify the presence of cancer and the type of cancer as described above, there may also exist prediction models (122) for thoracotylosis learned for each type of cancer, and for each prediction model (122) for thoracotylosis learned for each type of cancer, it may determine whether it corresponds to the corresponding cancer and classify the presence of cancer and the type of cancer by combining the prediction results of each prediction model (122).

[0090] For example, regarding a cytopathology slide image (10) of a random pleural fluid, a prediction model (122) trained as lung cancer can classify it as lung cancer. A prediction model (122) trained as breast cancer can classify it as not corresponding to breast cancer. In this case, in the classification step (S220), regarding a cytopathology slide image (10) of a random pleural fluid, it can be classified as lung cancer but not corresponding to breast cancer.

[0091] The cell pathology examination support method (200) according to another embodiment described with reference to FIG. 7 may apply the contents described in the cell pathology examination device (100) according to one embodiment described with reference to FIG. 1 to 6 in the same or similar way.

[0092] The cell pathology examination support device and method according to the present embodiments provide high accuracy and efficiency, which can greatly assist in the diagnosis and treatment of pathology examinations.

[0093] FIG. 8 is a configuration diagram of a computing system (300) according to embodiments of the present invention.

[0094] Referring to FIG. 8, the computing system (300) may include memory (310) and a processor (320).

[0095] The memory (310) can store the cell pathology slide image (10) and a plurality of tile images (20), but may also store them separately in a separate large-capacity storage server, etc. The memory (310) may be a volatile memory (e.g., SRAM, DRAM) or a non-volatile memory (e.g., NAND Flash).

[0096] The processor (320) can extract a plurality of tile images from a cytopathology slide image of pleural fluid and classify at least one class of cancer and the type of cancer from any cytopathology slide image of pleural fluid using a prediction model that performs annotation-based learning using the cytopathology slide image or the plurality of tile images.

[0097] The memory (310) stores a prediction model (122) that has undergone annotation-based learning. When a task is requested to classify at least one class of cancer and the type of cancer in a cytopathology slide image of any pleural fluid, the processor (320) executes the prediction model that has undergone annotation-based learning stored in the memory (310) to classify at least one class of cancer and the type of cancer in the cytopathology slide image and outputs the result.

[0098] A computing system according to embodiments of the present invention may include a computer device (300) including memory (310) and a processor (320) and a server (400) including memory (410) and a processor (420). The computer device (300) and the server (400) may be connected via a network via a wired or wireless connection.

[0099] The memory (410) of the server (400) can store the prediction model (122) that has undergone the aforementioned annotation-based learning.

[0100] When a query is made to classify at least one class of cancer and the type of cancer in a cytopathology slide image of any pleural fluid, the processor (320) of the computer device (300) extracts a plurality of tile images from the cytopathology slide image of the pleural fluid. The memory (310) of the computer device (300) can store the aforementioned cytopathology slide image (10) and the plurality of tile images (20).

[0101] The processor (320) of the computer device (300) can transmit the cell pathology slide image (10) and a plurality of tile images (20) stored in memory (310) and this request (query) to the server (400).

[0102] The processor (420) of the server (400) can classify at least one class of cancer and the type of cancer in a cell pathology slide image of any pleural fluid using a prediction model that has performed annotation-based learning on a received cell pathology slide image or a plurality of tile images, and transmit the result to a computer device (300).

[0103] Various possible examples of the computer system described with reference to FIGS. 8 and FIGS. 9 are described below.

[0104] The cell pathology examination support device (200) may be configured as a computing system (300) as shown in FIG. 8, or as a GPU server equipped with storage for storing scan files (WSI images), a GPU processor, and general memory, but the present invention is not limited thereto.

[0105] The aforementioned cytopathology examination support device (100) may be implemented by a computing device comprising at least some of a processor, memory, a user input device, and a presentation device. Memory is a medium for storing computer-readable software, applications, program modules, routines, instructions, and / or data, etc., which are coded to perform a specific task when executed by a processor. The processor may read and execute computer-readable software, applications, program modules, routines, instructions, and / or data, etc., stored in memory. A user input device may be a means for a user to input commands to the processor to execute a specific task or to input data necessary for the execution of a specific task. The user input device may include a physical or virtual keyboard or keypad, key buttons, a mouse, a joystick, a trackball, a touch-sensitive input means, or a microphone, etc. A presentation device may include a display, a printer, a speaker, or a vibration device, etc.

[0106] A computing device may include various devices such as smartphones, tablets, laptops, desktops, servers, and clients. A computing device may be a single stand-alone device, or it may include multiple computing devices operating in a distributed environment composed of multiple computing devices that cooperate with each other through a communication network.

[0107] In addition, the aforementioned cytopathology examination support device (100) may be executed by a computing device having a processor and a memory that stores computer-readable software, applications, program modules, routines, instructions, and / or data structures, etc., coded to perform a cytopathology examination support method (200) utilizing a deep learning model when executed by the processor.

[0108] The embodiments described above may be implemented through various means. For example, the embodiments may be implemented by hardware, firmware, software, or a combination thereof.

[0109] In the case of implementation by hardware, the method (200) for supporting cell pathology examination using a deep learning model according to the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), processors, controllers, microcontrollers, or microprocessors.

[0110] For example, the cell pathology examination support method (200) according to the embodiments may be implemented using an artificial intelligence semiconductor device in which the neurons and synapses of a deep neural network are implemented with semiconductor devices. In this case, the semiconductor devices may be currently used semiconductor devices, such as SRAM, DRAM, NAND, etc., or next-generation semiconductor devices, such as RRAM, STT MRAM, PRAM, etc., or a combination thereof.

[0111] When implementing the cell pathology examination support method (200) according to the embodiments using an artificial intelligence semiconductor device, the result (weight) of training a deep learning model with software may be transferred to a synapse mimic device arranged in an array, or training may be performed on the artificial intelligence semiconductor device.

[0112] In the case of implementation by firmware or software, the cytopathology examination support method (200) according to the embodiments may be implemented in the form of a device, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory unit and executed by a processor. The memory unit may be located inside or outside the processor and may exchange data with the processor by various means already known.

[0113] Additionally, terms such as "system," "processor," "controller," "component," "module," "interface," "model," or "unit" described above may generally refer to computer-related entities, hardware, combinations of hardware and software, software, or running software. For example, the aforementioned components may be, but are not limited to, processes driven by a processor, processors, controllers, control processors, objects, execution threads, programs, and / or computers. For example, both the application running on the controller or processor and the controller or processor may be components. One or more components may reside within a process and / or execution thread, and the components may be located on a single device (e.g., a system, a computing device, etc.) or distributed across two or more devices.

[0114] Meanwhile, another embodiment provides a computer program stored on a computer recording medium that performs the aforementioned method for supporting a cytopathology examination (200). Additionally, another embodiment provides a computer-readable recording medium that records a program for implementing the aforementioned method for analyzing cytopathology slide images.

[0115] The program recorded on the recording medium can execute the aforementioned steps by being read, installed, and executed on a computer.

[0116] In this way, in order for a computer to read a program recorded on a recording medium and execute functions implemented in the program, the aforementioned program may include code encoded in computer languages ​​such as C, C++, JAVA, and machine language, which can be read by the computer's processor (CPU) through the computer's device interface.

[0117] Such code may include functional code related to functions that define the aforementioned functions, and may also include control code related to execution procedures necessary for a computer processor to execute the aforementioned functions according to a predetermined procedure.

[0118] In addition, this code may further include memory reference-related code regarding where (address) in the computer's internal or external memory additional information or media required for the computer's processor to execute the aforementioned functions should be referenced.

[0119] In addition, if the computer processor needs to communicate with any other computer or server located remotely in order to execute the aforementioned functions, the code may further include communication-related code regarding how the computer processor should communicate with any other computer or server located remotely using the computer's communication module, and what information or media should be transmitted or received during communication.

[0120] A computer-readable recording medium that has recorded a program as described above includes, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage device, etc., and may also include one implemented in the form of a carrier wave (for example, transmission via the Internet).

[0121] In addition, computer-readable recording media are distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner.

[0122] Furthermore, the functional program for implementing the present invention, and the related code and code segments, etc., may be easily inferred or modified by programmers skilled in the art to which the present invention belongs, taking into account the system environment of a computer that reads a recording medium and executes the program.

[0123] The cytopathology examination support method (200) described through FIG. 7 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0124] The above-described cytopathology examination support method (200) may be executed by an application basically installed on a terminal (which may include a program included in a platform or operating system, etc., basically installed on the terminal), or by an application (i.e., a program) directly installed by a user on a master terminal through an application providing server, such as an application store server, an application, or a web server related to the service. In this sense, the above-described cytopathology examination support method (200) may be implemented as an application (i.e., a program) that is basically installed on the terminal or directly installed by a user, and may be recorded on a computer-readable recording medium such as a terminal.

[0125] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0126] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

[0127] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the technical concept. Furthermore, since these embodiments are intended to explain, not limit, the scope of the technical concept is not limited by these embodiments. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present disclosure.

Claims

Claim 1 A cytopathology examination support device comprising: a preprocessing unit that synthesizes a plurality of images focused at different phases through Z-stacking or focus stacking techniques into a single image from a cytopathology slide image taken by smearing or scanning a pleural fluid onto a glass slide, and extracts a plurality of tile images based on a sliding window algorithm from the synthesized image; and a classification unit that classifies the presence of cancer and at least one type of lung cancer or breast cancer in any cytopathology slide image of pleural fluid using a prediction model learned based on at least one annotation among a line-shaped partial annotation, a box-shaped bounding box annotation, and an image-level label representing the entire image, using the cytopathology slide image or the plurality of tile images. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 A method for supporting cytopathology examination using a cytopathology examination support device comprising a preprocessing unit and a classification unit, wherein the preprocessing unit synthesizes a plurality of images focused at different phases through Z-stacking or focus stacking techniques into a single image through secondary post-processing from a cytopathology slide image taken by smearing or scanning a pleural fluid onto a glass slide, and extracts a plurality of tile images based on a sliding window algorithm from the synthesized image; and the classification unit classifies the presence of cancer and at least one type of lung cancer or breast cancer in an arbitrary cytopathology slide image of pleural fluid using a prediction model learned based on at least one annotation among a line-shaped partial annotation, a box-shaped bounding box annotation, and an image-level label representing the entire image, using the cytopathology slide image or the plurality of tile images. Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 A memory storing a prediction model, a plurality of tile images extracted from a pleural fluid cytopathology slide image and a cytopathology slide image of the pleural fluid; wherein the prediction model is a prediction model that has undergone annotation-based learning to classify at least one class among the presence of cancer and the type of cancer in an arbitrary cytopathology slide image of the pleural fluid using the cytopathology slide image or the plurality of tile images; and, from a cytopathology slide image taken by smearing or scanning the pleural fluid onto a glass slide, a plurality of images focused at different phases through Z-stacking or focus stacking techniques are synthesized into a single image through secondary post-processing, and a plurality of tile images are extracted from the synthesized image based on a sliding window algorithm; A computer device comprising a processor for classifying the presence of cancer and at least one type of lung cancer or breast cancer in any cytopathology slide image of pleural fluid using a prediction model learned based on at least one annotation among a line-shaped partial annotation, a box-shaped bounding box annotation, and an image-level label representing the entire image, using the cytopathology slide image or the plurality of tile images.

Images