A system for automatic tumor detection and classification

Abstract

Certain aspects of the disclosure provide techniques for automatically detecting and classifying tumor regions in tissue slides. The method generally includes obtaining a digitized tissue slide from a tissue slide database, and determining the type of tissue shown in the digitized tissue slide based on output from a tissue classification module. The method further includes determining a region of interest (ROI) of the digitized tissue slide based on output from a tumor classification model for tissue type, and generating a classification slide that displays the ROI of the digitized tissue slide and an estimated diameter of the ROI. The method further includes displaying the classification slide and a user interface (UI) element on an image display unit, thereby enabling a pathologist to enter input related to the classification slide.

Description

Technical Field

[0001] Various aspects of the present disclosure relate to machine learning, and in particular to image classification using machine learning pattern recognition. Background Art

[0002] Currently, the process of diagnosing a specific slide containing potentially cancerous tissue is both time-consuming and laborious for the pathologist who performs the diagnosis. In the current process, the pathologist must manually analyze many images of the tissue slide. Each individual slide is typically scanned at multiple magnification and zoom levels. As a result, a slide representing one tissue can take a significant amount of time to determine if a given tissue includes a tumor. Additionally, if the pathologist detects a tumor, even more time is required to analyze the tumor. In total, the identification and analysis of a single tumor typically takes a pathologist at least two and a half hours of their time. And, many times, the pathologist is unsure of their diagnosis and may send the slide to a second pathologist, which at least doubles the time required for a complete analysis.

[0003] In some cases, a large amount of tissue needs to be analyzed by a relatively scarce qualified pathologist, which means that the pathologist's time is very valuable. In such cases, it may take several days to hand over the slide to the pathologist before the analysis begins. In order to speed up the analysis of tissue slides, analysis via machine learning can be implemented. However, existing machine learning systems cannot effectively identify and classify the tumors displayed in tissue slides. In addition, existing systems cannot properly annotate or mark tissue slides to help pathologists diagnose. Therefore, existing machine learning systems cannot effectively save time for pathologists or can only save very little time. Therefore, it is necessary to avoid the problems of existing systems for methods and systems for automatic detection and classification. Summary of the invention

[0004] Certain embodiments provide a method for automatically detecting and classifying tumor regions in a tissue slide. The method generally includes obtaining a digitized tissue slide from a tissue slide database, and determining the type of tissue shown in the digitized tissue slide based on output from a tissue classification module. The method further includes: determining a region of interest (ROI) of the digitized tissue slide based on output from a tumor classification model for the tissue type, wherein the ROI corresponds to a segment of the digitized tissue slide determined to be a tumor by the tumor classification model; and generating a classification slide that displays the ROI of the digitized tissue slide and an estimated diameter of the ROI. The method further includes displaying the classification slide and a user interface (UI) element on an image display unit, thereby enabling a pathologist to enter input related to the classification slide.

[0005] Another embodiment provides a method for automatically detecting and classifying tumor regions in a tissue slide by a tumor classification module. The method generally includes receiving a digitized tissue slide from a pathologist assistance system, and generating a binary mask, wherein the binary mask indicates a region of interest (ROI) in the digitized tissue slide. The method further includes determining the diameter of the ROI based on an ellipse fitted to match the ROI, and classifying the ROI into a tumor category based on the diameter of the ROI, wherein the tumor category indicates the stage of the tumor in the tumor category. The method further includes: generating a classified slide based on the ROI, wherein the classified slide displays each ROI as color-coded based on the tumor category associated with each ROI; and transmitting the classified slide to the pathologist assistance system.

[0006] The following description and the associated drawings set forth in detail certain illustrative features of the one or more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings depict certain aspects of one or more implementations, and therefore should not be considered to limit the scope of the disclosure.

[0008] Figure 1 An exemplary computing environment for automatically detecting and classifying tumors is depicted.

[0009] Figure 2 An exemplary user interface for presenting classified slides is depicted.

[0010] Figure 3 is a flow chart of an exemplary method for automatically detecting and classifying tumor regions in a tissue slide.

[0011] Figure 4 is a flow chart of an exemplary method for detecting and classifying tumor regions by a tumor classification module.

[0012] Figure 5 is a block diagram of an exemplary computing device for automatically detecting and classifying tumors.

[0013] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION

[0014] Figure 1 An exemplary computing environment 100 for automatically detecting and classifying tumors is depicted. As shown, the computing environment 100 includes a pathologist assistance system 110, a slide source 130, and an output display device 140. Although shown as separate entities, in other examples, the functions of the pathologist assistance system 110, the slide source 130, and the output display device 140 can be performed by a single computing device or a distributed computing system. In other embodiments, the pathologist assistance system 110, the slide source 130, and the output display device 140 can be connected via a network such as, for example, a local area network (LAN), a wide area network (WAN), or the Internet.

[0015] The pathologist assistance system 110 is a computing device including at least a processor and a memory, and the computing device is capable of executing software resident in the memory through the processor. The pathologist assistance system 110 includes a tissue classification module 120, a tumor classification module 122, a classification slide 124, and a pathologist input 126. The tissue classification module 120 is a software object that includes a machine learning model using graphic recognition, which can be used to identify the type of tissue displayed in a digital slide. The digital slide is obtained from a slide source 130. The slide source 130 is a variety of digital slide collection and storage tools available to the pathologist assistance system 110. The slide source 130 includes a microscope 132 and a slide database 134. The microscope 132 is an electron microscope capable of creating digital images of tissue slides at various resolutions and zoom levels, which can be beneficial to the diagnostic process. The microscope 132 can be any suitable electron microscope, such as an optical microscope, a fluorescent microscope, an electron microscope, or a probe microscope, and can utilize various spectral imaging techniques, such as multispectral imaging or hyperspectral imaging. The digital images created by the microscope 132 are stored in a slide database 134. The slide database 134 is a storage database that includes a plurality of digital slide images organized by slide based on a unique ID (such as a specific tissue). All tissue slides from a given tissue at all sizes, resolutions, and zoom levels can be accessed from the slide database 134.

[0016] The tissue classification module 120 includes a tissue classification model trained using labeled images of human tissue. In other instances, the tissue classification module 120 can be used to classify other types of tissue, such as animal tissue. In such instances, the tissue classification module 120 can be trained using labeled images of animal tissue. The purpose of the tissue classification model is to identify the type of tissue displayed in the slide, rather than to identify any specific tumor area displayed in the slide. Once the tissue classification model identifies the type of tissue displayed in a given slide, the tissue classification module 120 verifies that there is a tumor classification model for that type of tissue and passes the given slide to the tumor classification module 122.

[0017] The tumor classification module 122 is a software object that includes multiple machine learning classification models that can be used to detect and classify tumors within a specific type of tissue. Tumor analysis can be performed on many different types of tissue, and the analysis may be different for each type of tissue. For example, the analysis of breast tissue may be different from the analysis of pancreatic tissue, and so on. Therefore, a different tumor classification model can be used to classify each tissue type for which a tumor analysis is performed. In this example, the tumor classification module 122 includes at least a tumor classification model for the tissue type identified by the tissue classification module 120.

[0018] Each tumor classification model of the tumor classification module 122 is a machine learning model that can identify a region of a digital slide image that corresponds to a tumor in a tissue. For example, the tumor classification model can be a convolutional neural network (CNN) model. The CNN model is trained using a machine learning algorithm modeled after human brain neurons.

[0019] In order to train a machine learning model such as a CNN model, training data is supplied to the machine learning model to produce an output. The machine learning model typically has a set of initialization parameters that affect how the machine learning model processes data. The training data is typically classified with a ground truth value (e.g., a known value). The output is typically evaluated relative to the ground truth value. The results of the evaluation can be used to adjust or update the parameters of the machine learning model so that the machine learning model produces more accurate outputs during subsequent runs. Once fully trained, the machine learning model accepts input in the same form as the data used to train the machine learning model and produces output based on the input. A CNN model for classifying visual images is trained using training data consisting of labeled images. In this example, the training data can be a digital image slide previously diagnosed by a qualified pathologist.

[0020] The tumor classification module 122 analyzes the slide to detect any potential tumor areas displayed in the slide. Potential tumor areas are called regions of interest (ROIs). Typically, the process of tumor classification by the tumor classification module 122 begins with creating a binary mask for identifying ROIs using a tumor classification model for the tissue type identified by the tissue classification module 120. The binary mask filters the image to only display continuous cancer cell areas in the slide. These continuous areas correspond to ROIs. Once this information is received, the tumor classification module 122 then determines the size of each ROI. The tumor classification module 122 then classifies the ROI into various tumor categories corresponding to the tumor stage based on size. The tumor classification module 122 then color codes the ROI based on the associated tumor stage. The result of these processes by the tumor classification module 122 is a classification slide 124, which is a modified version of the slide analyzed to display the color-coded ROI.

[0021] Once the classified slide 124 is generated, the pathologist assistance system 110 displays the classified slide 124 via an output display device 140. The output display device 140 is a device that includes at least a display that can be used to present screens or other information to a user of the output display device 140. The output display device 140 can be a computing device that is separate from the pathologist assistance system 110, or can be a screen, monitor, or display that acts as an output device for the pathologist assistance system 110.

[0022] Once displayed, the pathologist is able to determine whether the identification and classification performed by the tumor classification module 122 is correct. If so, the pathologist assistance system 110 stores the classified slide 124. If the pathologist determines that the classified slide 124 is incorrect, user interface (UI) elements provided to the pathologist via the output display device 140 enable the pathologist to input corrections and modifications to the detection, sizing, or staging of the ROIs illustrated in the classified slide 124. In such cases, the pathologist assistance system 110 uses these corrections and modifications to update the tumor classification model used to identify the ROI and improve subsequent output from the tumor classification model.

[0023] Figure 2 An exemplary user interface 200 is depicted for presenting a classification slide 210. The classification slide 210 shows three ROIs labeled 0001-0003. These numbers correspond to the tumor IDs illustrated for each ROI at box 220. Box 230 illustrates the type or tumor category of each ROI corresponding to the determined stage of each ROI.

[0024] The ROI is usually staged based on the size of the ROI. In this example, the size of each ROI is determined by the diameter of the ROI. In this example, the diameter is measured as the distance between the farthest points from each other in the ROI. For example, the tumor classification model can place an ellipse on the ROI until the ellipse is in the smallest possible state while still covering the entire ROI. The major axis of the ellipse can then be used as the diameter of the ROI. The diameter of the ROI of the classification slide 210 is illustrated at box 240.

[0025] Typically, the user interface 200 is an example of presenting a classified slide 210 to a pathologist using a pathologist assistance system. In some embodiments, the user interface 200 may provide additional UI elements to enable the pathologist to make corrections to the classified slide 210. For example, if the pathologist believes that the classified slide 210 has improperly determined the size of the ROI, the pathologist may be able to redraw the line illustrated around the ROI. The pathologist may also be able to indicate that the ROI is completely incorrect, or draw a line around an area of ​​the classified slide 210 where the ROI should be displayed but is not displayed. Depending on the output device used, the pathologist can use various input devices to correct or modify the classified slide 210. For example, a device with touch screen functionality (such as a smartphone or tablet) can implement touch-sensitive controls, while a device with a mouse and keyboard can alternatively provide mouse and keyboard controls for input.

[0026] Figure 3 is a flow chart of an exemplary method 300 for automatically detecting and classifying tumor regions in a tissue slide. The method 300 may be implemented by a pathologist assistance system such as Figure 1 The method 300 is performed by the pathologist assistance system 110 of the present invention. The method 300 begins at 310, where the pathologist assistance system obtains a digitized tissue slide from a tissue slide database. As described above, the digitized issued slides can be stored in a database, and many different slides can be made for a single tissue sample at different sizes and zoom levels. All such slides of the tissue sample may need to be analyzed to perform an accurate diagnosis of the tissue, so although the method 300 describes the analysis of a single digitized tissue slide, the method 300 can be performed multiple times to complete a complete diagnosis. Once the slide is received, the pathologist assistance system provides the digitized tissue slide as input to the tissue classification module.

[0027] At 320, the pathologist assistance system determines the type of tissue illustrated in the digitized tissue slide based on the output from the tissue classification module. The tissue classification module includes a tissue classification model, which is used in this instance to identify human anatomical structures, and is therefore trained on labeled images of human anatomical structures. In some embodiments of method 300, the tissue classification module uses graphic recognition to determine the portion of the human anatomical structure associated with the digitized tissue slide. In other instances, the tissue classification module can be used to identify animal anatomical structures instead. The tissue classification model can identify the type of tissue illustrated in the digitized tissue slide. The tissue classification module can then verify that the pathologist assistance system includes a tumor classification for the identified tissue type. If so, the tissue classification module passes the digitized tissue slide to the tumor classification module together with an explanation of the tissue type illustrated in the digitized tissue slide.

[0028] At 330, the pathologist assistance system determines the ROI of the digitized tissue slide based on the output from the tumor classification model regarding the tissue type. In some embodiments of method 300, the tumor classification model is a convolutional neural network. The ROI corresponds to a segment of the digitized tissue slide that is determined to be a tumor by the tumor classification model. The tumor classification model uses a binary mask to identify the ROI within the digitized tissue slide. The binary mask is configured to display only cancer cells within the digitized tissue slide. When only cancer cells are displayed, the tumor classification module can perform various operations on the display of only cancer cells to estimate the size and stage of the tumor displayed in the ROI. Typically, the ROI corresponds to an area that is considered to be a tumor by the tumor classification model, but further diagnosis by a qualified pathologist may be helpful in confirming whether the ROI displays a tumor.

[0029] At 340, the pathologist assistance system generates a classification slide that displays the ROI of the digitized tissue slide and the estimated diameter of the ROI. As discussed, the tumor classification module can perform various operations to estimate the size (diameter) of the ROI. The pathologist assistance system can use this information to generate a classification slide that highlights the ROI within the classification slide and displays the estimated size of the ROI. An example of such a classification slide is Figure 2 Classification slide 210.

[0030] At 350, the pathologist assistance system displays the classified slide and user interface (UI) elements on the image display unit, thereby enabling the pathologist to enter input related to the classified slide. In general, the pathologist can use the displayed information to assist in diagnosing the tissue illustrated in the digitized tissue slide. Figure 2 The user interface 200 of is an example of such a display. In some embodiments of the method 300, the UI element allows the pathologist to redefine the ROI or change the classification of the ROI, such as by redrawing the line of the ROI or indicating the ROI as completely non-tumorous.

[0031] At 360, the pathologist assistance system optionally receives input from the pathologist related to the classified slide. In some embodiments of the method 300, the input related to the classified slide is a correction to the classified slide as described above. However, in other embodiments of the method 300, the input related to the classified slide is a validation of the ROI determined by the tumor classification model.

[0032] At 370, the pathologist assistance system updates the classified slide based on the input received from the pathologist. If the input related to the classified slide is a correction to the classified slide, the correction to the classified slide can be used to update the tumor classification model. Generally, updating the tumor classification model can involve changing parameters of the machine learning to change the data processing operation of the machine learning model. Updating the tumor classification model can also involve retraining the tumor classification model with the correction to the classified slide included in the training data.

[0033] If the input related to the classified slide is a correction to the classified slide, the method 300 may further include applying the correction to the classified slide and storing the classified slide in a classified slide storage device. Such storage of the classified slide may be subsequently used to retrain or modify the tumor classification model used to identify the ROI.

[0034] Figure 4 is a flow chart of an example method 400 for detecting and classifying tumor regions using a tumor classification model. The method 400 may be performed by a tumor classification module such as Figure 1 The method 400 begins at 410, where the tumor classification module receives a digitized tissue slide from a pathologist assistance system. The digitized tissue slide can be obtained from a digitized tissue slide database. The digitized tissue slide may have been previously analyzed by a tissue classification model and may therefore include a description of the tissue illustrated in the digitized tissue slide. Some embodiments of the method 400 further include receiving from the pathologist assistance system a description of biomarkers used to stain the digitized tissue slide, and selecting a tumor classification model for analyzing the digitized tissue slide based on the biomarkers used. Typically, a biomarker is a reagent used to prepare a tissue slide for analysis. Different reagents can be used to enable analysis of different features of the displayed tissue sample. The tumor classification module can use the identified biomarkers to select a tumor classification model for both the tissue type and the biomarkers used.

[0035] At 420, the tumor classification module generates a binary mask, wherein the binary mask indicates the ROI in the digitized tissue slide. Typically, binary masks can be used in computerized graphics to separate certain visual images. In this case, based on the output from the tumor classification model, the tumor classification module creates a binary mask that separates the tumor cells illustrated in the digitized tissue slide by filtering out non-tumor cells. The result is an image that includes only continuous tumor cell regions (i.e., ROIs).

[0036] At 430, the tumor classification module determines the diameter of the ROI based on the ellipse that is fitted to match the ROI. The stage of the tumor can be established by using the size of the tumor (in this case, the diameter). In this example, the tumor classification module can fit the ellipse to the ROI to estimate the diameter. Typically, the tumor classification module generates an ellipse and can resize, rotate, rescale, or otherwise transform the ellipse to fit the ROI. The ellipse has two axes, namely a major axis and a minor axis. At the moment when the ellipse completely covers the ROI, the tumor classification module determines the major axis of the ellipse and uses the length of the major axis as an estimate of the diameter of the ROI.

[0037] At 440, the tumor classification module classifies the ROI into a tumor class based on the diameter of the ROI, wherein the tumor class indicates the stage of the tumor within the tumor class. As discussed, the size of the tumor can be used to determine the stage of the tumor. Using the estimated diameter determined at 430, the tumor classification module can also estimate the stage of the tumor and thus classify the ROI into a tumor class corresponding to the stage. Some embodiments of method 400 further include segmenting the ROI based on size.

[0038] At 450, the tumor classification module generates a classification slide based on the ROI, wherein the classification slide displays the ROI as color-coded based on the tumor category associated with the ROI. For example, a ROI determined to be a stage I tumor can be red, while a ROI determined to be a stage II tumor can be yellow, and so on. Color coding the ROI can assist pathologists in using the classification slide by making it easier to distinguish the ROI. In some embodiments of method 400, the classification slide displays each ROI as an overlay on the classification slide. In order to generate the classification slide, the tumor classification module can combine an image that includes only the above-mentioned continuous tumor cell area with the original digital slide image. The color coding effect can be achieved by outlining only the tumor cell image with certain colors. In addition, an overlay effect can be achieved by coloring only the tumor cell image.

[0039] At 460, the tumor classification module transmits the classified slide to the pathologist assistance system. After transmission, the pathologist assistance system can provide the classified slide to the pathologist to assist in diagnosing the digitized tissue slide. Some embodiments of method 400 further include receiving a correction to the classified slide made by the pathologist from the pathologist assistance system, and updating the tumor classification model based on the correction to the classified slide.

[0040] Figure 55 is a block diagram of an exemplary computing device 500 for automatic detection and classification of tumors. As shown, the computing device 500 includes, but is not limited to, a central processing unit (CPU) 502; one or more input / output (I / O) device interfaces 504, which can allow various I / O devices 514 (e.g., keyboard, display, mouse device, pen input, etc.) to be connected to the computing device 500; a network interface 506; a memory 508; a storage device 510; and an interconnect 512.

[0041] CPU 502 can retrieve and execute programmed instructions stored in memory 508. Similarly, CPU 502 can retrieve and store application data residing in memory 508. Interconnect 512 transmits programmed instructions and application data between CPU 502, I / O device interface 504, network interface 506, memory 508, and storage device 510. CPU 502 is included to represent a single CPU, multiple CPUs, a single CPU with multiple processing cores, etc. I / O device interface 504 can provide an interface for capturing data from one or more input devices (such as a keyboard, mouse, touch screen, etc.) integrated into or connected to computing device 500. One such I / O device may be a Figure 1 140. For example, memory 508 may represent random access memory (RAM), and storage device 510 may be a solid state drive. Although illustrated as a single unit, storage device 510 may be a combination of fixed and / or removable storage devices, such as fixed drives, removable memory cards, network attached storage devices (NAS), or cloud-based storage devices.

[0042] As shown, the memory 508 includes a tissue classification module 522, a tumor classification module 524, and a pathologist input 526. The tissue classification module 522 and the tumor classification module 524 are software objects that execute based on instructions stored in the storage device 510. Such instructions can be executed by the CPU 502. The pathologist input 526 is data that temporarily resides in the memory 508.

[0043] As shown, the storage device 510 includes a slide database 532, training data 534, and a classification slide 536. The slide database 532, training data 534, and classification slide 536 can be used by software executed outside the memory 508 to perform a method for automatically detecting and classifying tumors in digitized tissue slides. In particular, the tissue classification module 522 can use the tissue slides obtained from the slide database 532 to identify the type of tissue in the slide. Based on the tissue type, the tumor classification module 524 can generate a classification slide 536. The classification slide 536 can be displayed to the pathologist user via the I / O device interface 504 and the I / O device 514, and the pathologist input 526 can be received in return. Subsequently, the pathologist input 526 can be used to update the tumor classification module 524. The training data 534 may have been previously used to train the machine learning models of the tissue classification module 522 and the tumor classification module 524.

[0044] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein do not limit the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. For example, the functions and arrangements of the elements discussed may be changed without departing from the scope of the present disclosure. Various examples may appropriately omit, replace, or add various programs or components. For example, the described method may be performed in an order different from that described, and individual steps may be added, omitted, or combined. Moreover, the features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods that use other structures, functions, or structures and functions that supplement or replace the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the claims.

[0045] As used herein, the term "determining" encompasses a variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database, or another data structure), ascertaining, etc. Also, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Also, "determining" may include resolving, selecting, choosing, establishing, etc.

[0046] The method disclosed herein includes one or more steps or actions for implementing the method. Without departing from the scope of the claims, the method steps and / or actions can be interchangeable with each other. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions can be modified without departing from the scope of the claims. In addition, the various operations of the above method can be performed by any suitable component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. Typically, in the case where operations are illustrated in the accompanying drawings, these operations may have corresponding matching means plus functional components using similar numbers.

[0047] The various illustrative logic blocks, modules, and circuits described in conjunction with the present disclosure may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, separate hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be a graphics processing unit (GPU), such as those commonly used in computer graphics and image processing. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0048] The processing system may be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus may include any number of interconnecting buses and bridges. The bus may connect various circuits including a processor, a machine-readable medium, and an input / output device. A user interface (e.g., a keypad, a display, a mouse, a joystick, etc.) may also be connected to the bus. The bus may also connect various other circuits, such as timing sources, peripheral devices, voltage regulators, power management circuits, and other circuit elements that are well known in the art and will not be described in detail. The processor may be implemented using one or more general and / or special purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems capable of executing software. Those skilled in the art will determine how to best implement the described functions of the processing system based on the specific application and the overall design constraints imposed on the overall system.

[0049] If implemented in software, the function can be stored or transmitted as one or more instructions or codes on a computer-readable medium. Software should be broadly interpreted as representing instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or other things. Computer-readable media include computer storage media and communication media, such as any medium that facilitates the transfer of computer programs from one place to another. The processor may be responsible for managing the bus and general processing, including executing software modules stored on a computer-readable storage medium. A computer-readable storage medium may be coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Alternatively, the storage medium may be integrated with the processor. For example, a computer-readable medium may include a transmission line, a carrier modulated by data, and / or a computer-readable storage medium with instructions stored thereon separated from a wireless node, all of which may be accessed by the processor via a bus interface. Alternatively or additionally, a computer-readable medium or any portion thereof may be integrated into a processor, such as a case with a cache and / or a general register file. Examples of machine-readable storage media may include, for example, RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, magnetic disks, optical disks, hard drives, or any other suitable storage media, or any combination thereof. The machine-readable medium may be embodied in a computer program product.

[0050] A software module may include a single instruction or multiple instructions, and may be distributed over several different code segments, between different programs, and over multiple storage media. The computer-readable medium may include multiple software modules. The software module includes instructions that, when executed by a device such as a processor, cause a processing system to perform various functions. A software module may include a sending module and a receiving module. Each software module may reside in a single storage device, or may be distributed over multiple storage devices. For example, when a triggering event occurs, a software module may be loaded from a hard drive into a RAM. During the execution of a software module, a processor may load some instructions into a cache to increase access speed. Subsequently, one or more cache lines may be loaded into a general register file for execution by a processor. When referring to the functionality of a software module, it is understood that such functionality is implemented by a processor when executing instructions from the software module.

[0051] The attached claims are not intended to be limited to the embodiments shown herein, but to conform to the full scope consistent with the language of the claims. In the claims, unless otherwise specified, referring to an element in the singular is not intended to mean "one and only one", but "one or more". Unless otherwise specifically stated, the term "some" refers to one or more. Claim elements should not be interpreted according to the provisions of 35 U.S.C. § 112 (f), unless the element is explicitly stated using the term "member for...", or in the case of a method claim, the element is stated using the term "step for...". All structural and functional equivalents of the elements throughout the various aspects described in this disclosure that are known or will be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Moreover, whether or not the content disclosed herein is explicitly described in the claims, it is not intended that such disclosures be dedicated to the public.

Claims

1. A method for automatically detecting and classifying tumor regions in a tissue slide, the method comprising: digitized tissue slides were obtained from the tissue slide database; determining a type of tissue depicted in the digitized tissue slide based on output from a tissue classification module; determining a region of interest (ROI) of the digitized tissue slide based on an output from the tumor classification model for the tissue type, wherein the ROI corresponds to a segment of the digitized tissue slide determined to be a tumor by the tumor classification model; generating a classification slide illustrating the ROI of the digitized tissue slide and an estimated diameter of the ROI; as well as The classified slide and a user interface (UI) element are displayed on an image display unit, thereby enabling a pathologist to enter input related to the classified slide.

2. The method of claim 1, further comprising: receiving input from the pathologist regarding the classified slides; as well as The classified slide is updated based on the input received from the pathologist.

3. The method of claim 2, wherein the input related to the classified slide is a correction to the classified slide.

4. The method of claim 3, wherein the correction to the classified slide is used to update the tumor classification model.

5. The method of claim 3, further comprising: applying the correction to the classification slide; as well as The classified slides are stored in a classified slide storage device.

6. A system comprising: processor; as well as a memory comprising instructions that, when executed by the processor, cause the system to perform a method for automatically detecting and classifying tumor regions in a tissue slide, the method comprising: digitized tissue slides were obtained from the tissue slide database; determining a type of tissue depicted in the digitized tissue slide based on output from a tissue classification module; determining a region of interest (ROI) of the digitized tissue slide based on an output from the tumor classification model for the tissue type, wherein the ROI corresponds to a segment of the digitized tissue slide determined to be a tumor by the tumor classification model; generating a classification slide illustrating the ROI of the digitized tissue slide and an estimated diameter of the ROI; and The classified slide and a user interface (UI) element are displayed on an image display unit, thereby enabling a pathologist to enter input related to the classified slide.

7. The system of claim 6, wherein the method further comprises: receiving input from the pathologist regarding the classified slides; as well as The classified slide is updated based on the input received from the pathologist.

8. The system of claim 7, wherein the input related to the classified slide is a correction to the classified slide.

9. The system of claim 8, wherein the correction to the classified slide is used to update the tumor classification model.

10. The system of claim 6, wherein the UI element allows the pathologist to redefine the ROI or change the classification of the ROI.

11. A method for automatically detecting and classifying tumor regions in a tissue slide by a tumor classification module, the method comprising: receiving digitized tissue slides from the Pathologist Assistant system; generating a binary mask, wherein the binary mask indicates a region of interest (ROI) in the digitized tissue slide; determining a diameter of the ROI based on an ellipse fitted to match the ROI; classifying the ROI into a tumor class based on the diameter of the ROI, wherein the tumor class indicates a stage of a tumor in the tumor class; generating a classification slide based on the ROIs, wherein the classification slide displays each ROI as color-coded based on the tumor class associated with each ROI; as well as The classified slides are transferred to the pathologist assistance system.

12. The method of claim 11, further comprising: The ROI is segmented based on size.

13. The method of claim 11, wherein the classification slide displays the ROI as an overlay.

14. The method of claim 11, further comprising: receiving from the pathologist assistance system corrections made by a pathologist to the classified slide; as well as Based on the correction of the classification slide, a tumor classification model used to identify the ROI is updated.

15. The method of claim 11, further comprising: receiving from the pathologist assistance system a description of a biomarker used to stain the digitized tissue slide; as well as A tumor classification model used to analyze the digitized tissue slide is selected based on the biomarkers used.

16. The method of claim 2, wherein the input related to the classification slide is a validation of the ROI determined by the tumor classification model.

17. The method of claim 1, wherein the pathologist uses the classified slide to diagnose the tissue depicted in the digitized tissue slide.

18. The method of claim 1, wherein the UI element allows the pathologist to redefine the ROI or change the classification of the ROI.

19. The method of claim 1, wherein the tumor classification model is a convolutional neural network.

20. The method of claim 1, wherein the tissue classification module uses pattern recognition to determine a portion of a human anatomy associated with the digitized tissue slide.

Images