Method and device for providing information associated with immune phenotype for pathological slide image

Abstract

To provide a method for providing information associated with an immune phenotype for a pathological slide image carried out by at least one computer device.SOLUTION: The method can include: a step of acquiring information associated with an immune phenotype for one or more regions of interest in a pathological slide image; a step of generating an image indicating the information associated with the immune phenotype on the basis of the information associated with the immune phenotype for the one or more regions of interest; and a step of outputting an image indicating the information associated with the immune phenotype.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure provides immunophenotypic and associated information for pathology slide images. The present invention relates to a method and apparatus for detecting one or more regions of interest in a pathology slide image. The image shows the information related to the immune phenotype of the target organism. The present invention relates to a method and apparatus for generating and outputting an image. [Background technology]

[0002] Recently, as a third-generation anticancer drug for cancer treatment, immunosuppressants that utilize the immune system in the patient's body have been developed. There is growing interest in anti-cancer drugs. Anti-cancer drugs are drugs that prevent cancer cells from evading the body's immune system. Any drug that prevents or allows immune cells to better recognize and attack cancer cells It acts through the immune system of the human body, so it has almost no side effects of anti-cancer drugs. It can prolong the survival time of cancer patients more than other anti-cancer drugs. However, such immunotherapy is not effective for all cancer patients. To predict the effectiveness of immunotherapy for current cancer patients, it is necessary to predict the responsiveness of immunotherapy to anticancer drugs. It is important to do so.

[0003] On the other hand, users (e.g., doctors and patients) can use the system to predict the response of immune anticancer drugs. It can provide immune response information generated through pathology slide images of patient tissues. According to the conventional technology, users (e.g., doctors and patients) are not able to know the reactivity of immune anticancer drugs. To predict the pathology, we performed immunoreaction analysis for each of multiple patches included in the pathology slide image. In this case, the user can provide pathological information (e.g., immune cell expression information). Intuitively recognize immune response information for each of the numerous patches included in the image It may be difficult to predict the reactivity of immune anticancer drugs among multiple patches. Even for patches that are qualitatively unnecessary, immune response information may be generated. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an immunological imaging method for pathology slide images to solve the above problems. A method and apparatus for providing information associated with a current model is provided. [Means for solving the problem]

[0005] The present disclosure relates to a method, an apparatus (system), or a computer-readable medium storing instructions. The present invention may be embodied in various ways, including as a storage medium or a computer program.

[0006] Pathology processing performed by at least one computer device according to an embodiment of the present disclosure. A method for providing immunophenotypic and related information on slide images is proposed. Immunophenotyping of one or more regions of interest in the image pe) and obtaining information associated with the immune table for one or more regions of interest. Image showing immunophenotyping and associated information based on phenotyping and associated information and outputting an image showing information associated with the immunophenotype. Includes the top.

[0007] In one embodiment of the present disclosure, the one or more regions of interest are The determination is based on the detection results of one or more target items.

[0008] In one embodiment of the present disclosure, the one or more regions of interest are located in a pathology slide image. The target item includes at least a portion of the area that satisfies a condition associated with the target item.

[0009] In one embodiment of the present disclosure, the one or more regions of interest are The detection results of one or more target items or pathology slide images are input to the region of interest extraction model. The region of interest extraction model is the region output by extracting the region of interest from the reference pathology slide image. The detection results of one or more target items for the page or the reference pathology slide image are input. By doing so, the system is trained to output a reference region of interest.

[0010] In one embodiment of the present disclosure, the obtaining step comprises immunophenotyping one or more regions of interest. The generating step includes obtaining a corresponding immunophenotype of one or more regions of interest. and generating an image containing a visual label indicating the immunoreactivity, immunophenotyping, This includes at least one of immune deficiency or immunosuppression.

[0011] In one embodiment of the present disclosure, the acquiring step includes one or more images for one or more regions of interest. The generating step includes obtaining an immunophenotypic score above, wherein the generating step comprises obtaining one or more immunophenotypic scores. generating an image comprising visual indicators corresponding to the phenotypic scores, The above immunophenotype scores are based on the score for immune activity, the score for immune exclusion, or the score for immune deficiency. The score for the deficiency is at least one of:

[0012] In one embodiment of the present disclosure, the acquiring step includes one or more images for one or more regions of interest. obtaining features associated with the immunophenotypes described above. The steps of generating and generating the immunophenotypes correspond to features associated with one or more immunophenotypes. generating an image comprising a visual label, the image being associated with one or more immunophenotypes; The selected features are at least a subset of statistical values ​​or vectors associated with the immunophenotype. Also includes one.

[0013] In one embodiment of the present disclosure, the outputting step includes outputting one or more pathology slide images. and outputting an image including both the region of interest and the visual indicator.

[0014] In one embodiment of the present disclosure, the outputting step includes outputting one or more pathology slide images. overlaying an image containing visual landmarks in the region of interest of the image; Includes flops.

[0015] In one embodiment of the present disclosure, detecting one or more items of interest from a pathology slide image. obtaining a result and generating an image showing the detection of the one or more target items. and outputting an image showing the detection of one or more target items. Further includes:

[0016] The immunophenotyping and association of the pathology slide images described above according to one embodiment of the present disclosure. computer-readable code for executing a method for providing the specified information A computer program stored on a recording medium is provided.

[0017] According to one embodiment of the present disclosure, a computer device includes one or more instructions. memory for storing instructions and for executing one or more of the stored instructions This allows for correlation of immunophenotypes to one or more regions of interest in the pathology slide image. and obtaining associated immunophenotype information for one or more regions of interest. Based on the results, an image showing information associated with the immunophenotype is generated, and and a processor configured to output an image indicative of the associated information. [Effects of the Invention]

[0018] According to some embodiments of the present disclosure, an image visually representing information associated with an immunophenotype is provided. By providing the user with a page, the user can easily view the immunophenotype and associated information for each region. In addition, visual indicators are provided to indicate information associated with the immunophenotype. , by providing an overlay on the corresponding region of interest in the pathology slide image. The user can determine which region of the pathology slide image the information about the visual marker is located in. You can see at a glance whether it applies to you.

[0019] According to some embodiments of the present disclosure, immunophenotyping and / or immunohistochemistry of pathology slide images is performed. Alternatively, it is possible to determine the region of interest that needs to be analyzed to determine whether or not an immunological anticancer drug is responsive. That is, the information processing system and / or the user terminal can process the entire pathology slide image. Instead, only the region of interest is processed (e.g., immunophenotyping and / or or immunophenotype score calculation, etc., reducing computer resources and processing costs. This can minimize the

[0020] According to some embodiments of the present disclosure, immunophenotyping and / or immunotherapy response determination In determining the target gene, only meaningful regions are processed (e.g., immunophenotyping and / or immunorecognition). By performing calculations such as the current score, more accurate results can be provided.

[0021] The effects of the present disclosure are not limited to these, and other effects not mentioned are within the scope of the claims. A person having ordinary skill in the art to which the present disclosure pertains (hereinafter, "ordinary engineer") This should be clearly understood as [Brief explanation of the drawings]

[0022] Embodiments of the present disclosure will be described with reference to the accompanying drawings, in which like reference numerals refer to: indicates similar elements, but is not limited to these. [Figure 1] FIG. 1 is an exemplary configuration diagram showing a system in which an information processing system according to an embodiment of the present disclosure provides information associated with an immunophenotype for a pathology slide image. [Figure 2] 1 is a block diagram showing an internal configuration of an information processing system according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a block diagram illustrating an internal configuration of a user terminal according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart illustrating a method for providing information associated with an immunophenotype for a pathology slide image according to one embodiment of the present disclosure. [Figure 5] FIG. 10 illustrates an example of determining one or more regions of interest within a pathology slide image according to one embodiment of the present disclosure. [Figure 6] FIG. 10 illustrates an example of generating immunophenotyping results according to an embodiment of the present disclosure. [Figure 7] FIG. 10 is a diagram showing an example of outputting an immunophenotyping result according to an embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram showing an example of outputting an immunophenotyping result according to another embodiment of the present disclosure. [Figure 9]FIG. 1 illustrates an example of an artificial neural network model according to an embodiment of the present disclosure. [Figure 10] FIG. 1 is a block diagram of an exemplary computer device (e.g., a user terminal) that provides information associated with immunophenotypes for pathology slide images according to one embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram illustrating an example of outputting a target item detection result according to an embodiment of the present disclosure. [Figure 12] 10A and 10B are diagrams illustrating an example of outputting a target item detection result and an immune phenotype determination result according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, known functions and A detailed description of the configuration will be omitted.

[0024] In the accompanying drawings, the same or corresponding components are given the same reference numerals. In the following description of the embodiments, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is understood that such component may be included in a particular implementation. It should not be intended to be an example that is not included.

[0025] The advantages and features of the disclosed embodiments and the manner in which they are achieved will be described below with reference to the accompanying drawings. However, the present disclosure is not limited to the implementations disclosed below. However, the present disclosure is not limited to the above examples and may be embodied in many different forms. and to enable one of ordinary skill in the art to accurately appreciate the scope of the invention. It is only provided to

[0026] A brief explanation of terms used in this specification and a detailed description of the disclosed embodiments will be provided. The terms used herein are currently used as broadly as possible while still considering their function in this disclosure. The general terms used have been selected based on the intentions of engineers engaged in the relevant field or on legal precedents. In particular cases, the term may be arbitrarily selected by the applicant. These meanings will be explained in detail in the description of the invention. The terms used in this disclosure are not simply names of terms, but rather are used to describe the meanings that the terms have and the meanings that the terms have in this disclosure. should be defined based on the overall content of

[0027] In this specification, unless otherwise clearly specified in the context, the singular expression includes the plural expression. Throughout the specification, certain parts may be used interchangeably with certain structures. When a component is referred to as "including," this does not include other components unless specifically stated to the contrary. This means that other components may be included instead of the above.

[0028] Furthermore, the terms "module" and "part" used in the specification do not refer to software or hardware. A "module" or "part" refers to a hardware component that performs a certain function. However, "module" or "part" does not mean limited to software or hardware. A "module" or "unit" is configured to reside on an addressable storage medium. One or more processors may be configured to regenerate. For example, a "module" or a "unit" may refer to a software component, an object, or a Components such as directional software components, class components, task components, and Processes, functions, attributes, procedures, subroutines, segments of program code, Drivers, firmware, microcode, circuits, data, databases, data structures The components may include at least one of a structure, a table, an array, or a variable. A "module" or "unit" is a unit that provides functionality within a smaller number of components and "module" or "unit" or "unit" with additional components can be further separated into

[0029] According to one embodiment of the present disclosure, a "module" or a "unit" may be embodied by a processor and a memory. "Processor" can be a general-purpose processor, a central processing unit (CPU), a microprocessor, processors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some circumstances, "processor" may be used for specific purposes. Application-specific integrated circuits (ASIC), programmable logic devices (PLD), field It can also be called a programmable gate array (FPGA). For example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, A combination of one or more microprocessors coupled with a DSP core, or any other It can also refer to a combination of processing devices such as a combination of configurations such as "Memory" is to be interpreted broadly to include any electronic component capable of storing electronic information. "Memory" refers to RAM (Random Access Memory), R OM (Read Only Memory), NVRAM (Non-Volatile Random Access Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Program mable Read-Only Memory), EEPROM (Electrica lly Erasable Programmable Reda-Only Memo ry), flash memory, magnetic or optical data storage devices, registers, etc. Various types of processor-readable media may be used. If you can read / write information or write information to the memory, the memory is a The memory integrated into the processor is said to be in electronic communication with the processor. It is in this state.

[0030] In this disclosure, a "system" includes at least one of a server device and a cloud device. For example, a system may include, but is not limited to, one or more servers. In another example, the system may consist of one or more cloud devices. In another example, the system can be configured to use both the server device and the cloud device. It can also be operated as

[0031] In this disclosure, "target data" refers to any data that can be used to train a machine learning model. It can refer to data or data items, for example, data showing an image, sound or audio. This includes, but is not limited to, data indicating voice characteristics. The data consist of the entire pathology slide image and / or a small portion of the data contained in the pathology slide image. Although the description is given using at least one patch (or region), it is not limited to this and may be applied to other machines. Any data that can be used to train a learning model can be the target data. The data is tagged with label information through annotation work. do.

[0032] In this disclosure, a "pathological slide image" refers to a tissue or the like removed from a human body, taken under a microscope. Photographing pathology slides that have been fixed and stained through a series of chemical processes for viewing in For example, a pathology slide image is a digital image taken with a microscope. It can refer to cells, tissues and / or stem cells in the human body. It can also contain information about the structure of the tissue. A user image can contain one or more patches, and one or more patches can have annotations. Label information (e.g., information about immunophenotype) is tagged by the scanning process. "Pathology slide images" refer to H&E stained tissue slides and / or IHC stained tissue slides. These may include, but are not limited to, histological slides containing various staining schemes (e.g., CISH (Chromogenic in situ hybridization) FISH (Fluorescent in situ hybridization) Multiplex IHC and other applied tissue slides or unstained ned) tissue slides. Another example is "pathology slide images." is a patient tissue slide generated for predicting immune anti-cancer drug response, Includes tissue slides of patients before cancer drug treatment and / or tissue slides of patients after immunotherapy. It can be done.

[0033] In this disclosure, a "biomarker" is a biomarker that indicates a normal or pathological state, a process of response to a drug, or a specific function of a particular cell. It can be defined as an index that can be measured objectively, such as the degree of , immunotherapy and biomarkers may include, but are not limited to, PD-L1. TMB (Tumor Mutation Burden) value, MSI (Micro satellite instability value, HRD (Homologous Recombination Deficiency), CD3, CD8, CD68, FOXP3, CD20, CD4, CD45, CD163, and other immune cell-associated proteins The present invention can include a variety of biomarkers.

[0034] In this disclosure, a "patch" may refer to a small region within a pathology slide image. For example, patches are generated for pathology slide images using segmentation. Regions corresponding to semantic objects extracted by performing a In another example, the patch may include analyzing a pathology slide image. The combination of the label information generated by the do.

[0035] In this disclosure, a "region of interest" is at least the area within a pathology slide image that is the subject of analysis. For example, a region of interest may be a region of interest in a pathology slide image. It can refer to at least a portion of the area in which the item of interest is contained. The region is at least one of the patches generated by dividing the pathology slide image. It can also be called a part.

[0036] In this disclosure, a "machine learning model" and / or an "artificial neural network model" refers to a given It can contain any model used to infer an answer to an input. According to one embodiment, the machine learning model comprises an input layer, a plurality of hidden layers, and The artificial neural network model may include a neural network model including a neural network layer and an output layer, where each layer includes multiple nodes. For example, the machine learning model can be generated using pathology slide images and / or pathology slides. It can learn to infer label information for at least one patch contained in the id. At this time, the label information generated by the annotation work is used to train the machine learning model. In addition, machine learning models are used to analyze the data of multiple nodes included in the machine learning model. where the weights are associated with the machine learning model. It may contain any parameters attached to it.

[0037] In this disclosure, "learning" refers to the process of learning a machine using at least one patch and label information. It can refer to any process of changing the weights associated with a machine learning model. According to an embodiment, the learning is performed by a machine learning model using at least one patch and label information. The model is propagated in forward and backward directions one or more times. Backward propagation is used to associate the model with machine learning. This may refer to the process of changing or updating the weights.

[0038] In this disclosure, "label information" refers to data This is the correct answer information for the sample, and is information obtained as a result of the annotation work. Label or label information is often used interchangeably with terms such as annotation and tag in the art. In this disclosure, "annotation" means , annotation work and / or annotations determined by performing annotation work In this disclosure, "anonymous" may be referred to as "anonymous information" (e.g., label information). "Annotation information" refers to information for and / or in annotation work. The generated information (eg, label information) may be referred to as "label information."

[0039] In this disclosure, a "target item" is a target item to be detected in a pathology slide image. These may refer to data / information, image regions, objects, etc. According to the regulations, the target items are used for the diagnosis, treatment and prevention of diseases (e.g., cancer), pathology, etc. It can contain the object to be detected from the slide image. For example, The "system" can include cell-based target items and area-based target items.

[0040] In the present disclosure, "each of a plurality of A's" and / or "each of a plurality of A's" means that the plurality of A's are included in the plurality of A's. It refers to all the components included in the A, or to some of the components included in multiple A. For example, each of the multiple regions of interest can be a region of interest that includes all regions of interest contained in the multiple regions of interest. It can refer to each of the regions of interest, or each of the regions of interest included in multiple regions of interest. .

[0041] In this disclosure, "instructions" are functionally It is one or more instructions combined together and is a component of a computer program. At the same time, it can refer to something that is executed by a processor.

[0042] In this disclosure, a "user" can refer to a person who uses a user terminal. For example, a user may include an annotator who performs annotation work. As another example, a user may provide information associated with an immune phenotype and / or immune antigen. Prediction of response to cancer drugs (e.g., prediction of whether a patient will respond to an immunotherapy anti-cancer drug) The user may be a doctor or a patient to whom the results are provided. Conversely, a user terminal can refer to a user. The terms user equipment and user terminal may be used interchangeably herein.

[0043] FIG. 1 shows an example of a pathology slide image processed by an information processing system 100 according to an embodiment of the present disclosure. Provides information related to the immune phenotype of the patient FIG. 1 is an exemplary block diagram showing a system for detecting immunizations for pathology slide images. The system for providing information associated with an epiphenotype includes an information processing system 100, a user The information processing system 10 may include a terminal 110 and a storage system 120. 0 is connected to and capable of communicating with the user terminal 110 and the storage system 120. Although one user terminal 110 is shown in FIG. 1, the present invention is not limited to this. A plurality of user terminals 110 are configured to be connected to and communicate with the information processing system 100. In addition, in FIG. 1, the information processing system 100 is shown as a single computer device. However, the information processing system 100 is not limited to this, and may be implemented via multiple computer devices. In addition, in FIG. 1, the storage system can be configured to perform distributed processing of information and / or data. Although the system 120 is shown as a single device, it is not limited to this and may be configured as multiple storage devices. It can be configured as a cloud-supporting system. In addition, Figure 1 provides information related to immunophenotypes for pathology slide images. Each component of the system is a functional element that is divided into multiple components. The elements may be embodied in a manner that integrates them with one another in a real physical environment.

[0044] The information processing system 100 and the user terminal 110 perform immune analysis on a pathology slide image. Any computer device used to generate and provide information associated with a phenotype. Here, the computer device is any kind of device equipped with computer functions. It can refer to a device such as a notebook, desktop, or laptop. This can be, but is not limited to, a laptop, server, cloud system, etc. It is not something that can be done.

[0045] The information processing system 100 can receive pathology slide images. The system 100 receives pathology slide images from the storage system 120 and / or the user terminal 110. The information processing system 100 can receive immunophenotyping data for pathology slide images. and provide it to the user 130 via the user terminal 110. In one embodiment, the information processing system 100 detects one determining the above regions of interest and associating the immunophenotype with one or more regions of interest; wherein the information associated with the immunophenotype can be generated by immunophenotyping one or more regions of interest. phenotype, immunophenotype score of one or more regions of interest, and immunophenotype of one or more regions of interest; It may include at least one associated feature.

[0046] In one embodiment, the information processing system 100 performs one or more image processing operations on a pathology slide image. Based on the detection of the above items of interest, one or more regions of interest can be determined. The information processing system 100 is configured to process pathology slide images associated with one or more target items. One or more regions of interest can be determined that include at least a portion of the region that satisfies the attached condition. Additionally or alternatively, the information processing system 100 may perform one or more The target item detection results (e.g., pathology slide images containing target item detection results) ) is input to the region of interest extraction model, and the output region is treated as one or more regions of interest. Here, the region of interest extraction model is determined by one of the reference pathology slide images. The above target item detection results (e.g., reference pathology slides containing target item detection results) The model is trained to output a reference region of interest (ROI) based on the input image. can.

[0047] The user terminal 110 receives one or more pathology slide images from the information processing system 100. For example, one or more The information associated with the immunophenotype for the region of interest may be used to identify immunophenotypes for one or more regions of interest. Phenotype (e.g., immune inflamed, immune eliminated) at least one of: immune deficiency (e excluded) or immune desert Additionally or alternatively, immunophenotyping for one or more regions of interest may be performed. The associated information may include one or more immunophenotype scores (e.g., For example, the immunophenotype score may be a score for immune activity, a score for immune exclusion, or an immune deficiency. Additionally or alternatively, one or more The information associated with the immunophenotype for a region of interest may be used to identify a single immunophenotype for one or more regions of interest. one or more immunophenotype-associated features (e.g., immunophenotype-associated features) The data may include at least one of a statistical value or a vector.

[0048] The user terminal 110 then associates the immunophenotype with one or more regions of interest. Based on the information obtained, an image can be generated that shows information associated with the immunophenotype. In an embodiment, the user terminal 110 may generate visual images corresponding to the immunophenotype of one or more regions of interest. In another embodiment, the user terminal 110 may generate an image containing one or more An image can be generated containing visual indicators corresponding to the above immunophenotypic scores. Visual indicators include color (e.g., color, brightness, saturation, etc.), text, images, marks, , graphics, etc.

[0049] The user terminal 110 displays an image showing information associated with the generated immunophenotype. In one embodiment, the user terminal 110 can input one or more images of the pathology slide. The image containing the region of interest and visual markers can be output together, i.e., a pathology slide. The image, including one or more regions of interest and visual indicators in the image, is transmitted to a user terminal 110. In another embodiment, the user terminal The end 110 is an image including a visual indicator for one or more regions of interest in a pathology slide image. The image can be overlaid with a visual cue. One or more regions of interest in the pathology slide image are stored in a device associated with the user terminal 110. This allows the user 130 (e.g., a doctor or a patient) to provides an image showing information associated with the immunophenotype via a user terminal 110. can.

[0050] The storage system 120 stores information associated with immunophenotypes for pathology slide images. Pathology slide images associated with the patient, machine learning models to provide information It is a device or cloud system that stores and manages various data associated with the data. In order to efficiently manage the data, the storage system 120 uses a database to store various data. Here, various data can be stored and managed in any way associated with the machine learning model. It can include data such as files of the target data, meta information of the target data, Label information on the target data, which is the result of the annotation work, This may include, but is not limited to, data, machine learning models (e.g., artificial neural network models), etc. In FIG. 1, the information processing system 100 and the storage system 120 are separate systems. However, it is not limited to this and may be configured as an integrated system. It can be done.

[0051] According to some embodiments of the present disclosure, a user 130 may be provided with a visual representation of information associated with an immunophenotype. By providing visual images, the user 130 can associate the immunophenotype with each region. Furthermore, according to some embodiments of the present disclosure, the pathology slide can be displayed. Among the images, for immunophenotyping and / or determining whether or not the patient will respond to immunotherapy, The area of ​​interest that needs to be analyzed can be determined. Alternatively, the user terminal 110 may display not the entire pathology slide image but only the relevant portion excluding unnecessary areas. Processing (e.g., immunophenotyping and / or immunophenotypic scoring) of cardiac regions only ) can be performed, minimizing computer resources and processing costs. Only regions that are meaningful for phenotyping and / or determining whether or not an immunotherapy will be effective are included. By performing processing (e.g., immunophenotyping and / or immunophenotypic scoring) , which can provide more accurate prediction results.

[0052] FIG. 2 is a block diagram showing the internal configuration of an information processing system 100 according to an embodiment of the present disclosure. To provide immunophenotypic and correlated information for pathology slide images. The information processing system 100 then associates the immunophenotype with the pathology slide image. According to one embodiment, as shown in the figure, an information processing system 100 includes: , including a target item detection unit 210, a region of interest determination unit 220, and an immunophenotyping unit 230. In FIG. 2, each component of the information processing system 100 is functionally divided. It refers to the functional elements that are integrated together in a real physical environment. It can be embodied in various forms.

[0053] The target item detection unit 210 detects a pathology slide image (e.g., an H&E-stained pathology slide image). slide images, IHC stained pathology slide images, etc.) One or more items of interest can be detected within a pathology slide image. The target item detection unit 210 uses an artificial neural network target item detection model to One or more target items can be detected within a slide image. The target item detection model extracts one or more reference target items from the reference pathology slide image. For example, the target item detection unit 210 may be a model trained to detect , cell-based target items and / or area-based target items in the pathology slide image That is, the target item detection unit 210 can detect items from a pathology slide image. Within the tumor cells, lymphocytes, Macrophages, dendritic cells , fibroblast, endothelial cell ), blood vessels, cancer stroma, cancer epithelium, cancer area, Normal area (e.g., normal lymphatic structure) ph node architecture area) as target items. do.

[0054] The region of interest determination unit 220 determines one or more regions of interest within the pathology slide image. Here, the region of interest is one or more items of interest within the pathology slide image. For example, the region of interest determination unit 220 may include a region where a pathological image is detected. Among the multiple patches that make up an image, patches that contain one or more target items are called regions of interest. In one embodiment, the region of interest determiner 220 determines the region of interest from the pathology slide image. Based on the detection of one or more target items on the page, one or more regions of interest can be determined. For example, the region of interest determining unit 220 may determine one or more target regions in the pathology slide image. One or more regions of interest that contain at least a portion of the area that satisfies the condition associated with the item. Additionally or alternatively, the region of interest determiner 220 may determine a region of interest for the pathology slide image. The detection results of one or more target items and / or pathology slide images are used to extract regions of interest. By inputting the model, the output regions can be determined as one or more regions of interest.

[0055] The immunophenotyping unit 230 determines the immunophenotype of one or more regions of interest in the pathology slide image. In one embodiment, the immunophenotyping unit 230 can generate information associated with the type. Immunophenotyping one or more regions of interest based on the detection results for one or more target items For example, the immunophenotyping unit 230 can perform detection for one or more target items. Based on the results, the immunophenotype of one or more regions of interest may be determined to be immunoreactive or immunoexcludable. In another embodiment, the immunophenotyping unit 23 can determine whether the patient has an immune deficiency or not. 0 immunizes one or more regions of interest based on the detection results for one or more target items. For example, the immunophenotyping unit 230 may determine the phenotype score for one or more target items. Based on the results of the detection of the immune system, scores for immune activity and immune exclusion of one or more regions of interest are calculated. A score for immune deficiency and / or a score for immune deficiency can be calculated. The phenotyping unit 230 determines the probability that the immunophenotype of one or more regions of interest is immunoreactive. a score indicating the probability of immune exclusion and / or a score indicating the probability of immune deficiency A score can be calculated.

[0056] In yet another embodiment, the immunophenotyping unit 230 performs a single immunophenotyping analysis for one or more regions of interest. One or more immune phenotypes can be associated with features, where one or more immune phenotypes can be associated with features. A phenotype-associated feature is a statistical value or vector associated with an immunophenotype. For example, the antibody may be associated with one or more immunophenotypes. The features extracted are correlated with immune phenotypes output from artificial neural network and machine learning models. The immune system may include a score value assigned to one or more regions of interest. It may include a score value output in the process of determining the current type. The features associated with the immunophenotype are determined by a threshold (or cutoff) for the immunophenotype. Density value, number, various statistical values ​​of immune cells corresponding to the cut-off It may include a vector value indicating the distribution.

[0057] In yet another example, the features associated with one or more immune phenotypes may be immune cells or Cancer cells and specific cells (e.g., cancer cells, immune cells, fibroblasts, lymphocytes) cell, plasma cell, macrophage, endothelial relative relationships between cells (e.g., histogram vectors that take into account direction and distance) vectors or graph representations) and relative statistics (e.g., specific cell counts vs. immune cell counts). It can contain scalar values, vector values, etc., including ratios of The features associated with one or more immunophenotypes may be associated with specific regions (cancer regions, cancer stroma, etc.). area, Tertiary lymphoid structure, Normal re gion, Necrosis, Fat, Blood vessel, High end othelial venule, lymphatic vessel, nerve etc. Statistics of immune cells or cancer cells in the tumor (e.g., ratio of cancer stroma area to immune cell count) or a scalar value containing a distribution (e.g., a histogram vector or a graph representation vector). and vector values.

[0058] In yet another example, features associated with one or more immunophenotypes may be biomarkers. Positive / negative cells and identification based on the expression level of CAR Cells (e.g. cancer cells, immune cells, Fibroblasts, Lymphocytes, pl asma cell, Macrophage, Endothelial cell, etc.) Relative relationships between (e.g., histogram vectors or graph representations taking into account direction and distance) vectors) and relative statistics (e.g., ratio of specific cell counts to immune cell counts). As another example, the value may include one or more immune The features associated with the phenotype were identified by specific regions (cancer region, cancer stroma region, Terti ary lymphoid structure, Normal region, Nec. rosis, Fat, Blood vessel, High endothelial Biomarkers in the venule, lymphatic vessel, nerve, etc. Statistics of positive / negative cells by CAR expression level (e.g., ratio of cancer stroma area to immune cell count) vectors containing distributions (e.g., histogram vectors, graph representation vectors, etc.) It can include color values, vector values, etc.

[0059] In FIG. 2, the information processing system 100 includes a target item detection unit 210, a region of interest determination unit 212, and a target item detection unit 214. The immunophenotyping unit 220 and the immunophenotyping unit 230 are included, but are not limited to these, and some components may be omitted. In one embodiment, the information processing system 100 includes an The immunological anticancer drug response prediction unit (not shown) may further be included. Based on the information associated with the immunophenotype, it is possible to predict whether a patient will respond to an immunotherapy. In another embodiment, the information processing system 100 may include an output unit ( The output unit further includes a detection result for one or more target items, The immunophenotype of the region of interest above, the predicted result of whether the patient will respond to an immunotherapy or not, or At least one of the densities of immune cells within one or more regions of interest can be output.

[0060] FIG. 3 is a block diagram illustrating the internal configuration of a user terminal 110 according to an embodiment of the present disclosure. According to one embodiment, as shown in the figure, the user terminal 110 includes an image generator 310 and The user terminal 110 may include an image output unit 320. In FIG. The element indicates a functional element that is functionally divided, and multiple components are placed in the actual physical environment. These can be embodied in a form that is integrated with each other.

[0061] The image generation unit 310 performs an immunodetection of one or more regions of interest in the pathology slide image. For example, the image generation unit 310 may include an information processing unit. Receive information associated with the immunophenotype for one or more regions of interest generated by the system. Additionally or alternatively, the user terminal 110 may generate an immune table for one or more regions of interest. By generating information associated with the model, the image generator 310 may generate one or more images of interest. Additionally or alternatively, information relating to the immunophenotype for the region can be obtained. The message generator 310 generates one or more messages stored in a device inside and / or outside the user terminal 110. Information associated with the immunophenotype for the region of interest above can be received.

[0062] The image generation unit 310 generates information associated with the immunophenotype for one or more regions of interest. Based on the information, an image can be generated that shows information associated with the immunophenotype. In the case of acquiring an immunophenotype of one or more regions of interest, the image generating unit 310 An image can be generated that includes visual indicators corresponding to the immunophenotype of one or more regions of interest. In another embodiment, receiving one or more immunophenotype scores for one or more regions of interest. If so, the image generator 310 generates a visual indicator corresponding to one or more immunophenotype scores. You can generate an image that contains

[0063] In yet another embodiment, the image generation unit 310 may associate one or more immunophenotypes. An image containing visual indicators corresponding to the identified features (e.g., feature values) For example, a heatmap of biomarker expression rates can be generated. A classification map ( It is possible to generate images containing classification maps. The genotype map is based on the TPS (Tumor Profile System), which is information associated with PD-L1 expression. portion score) and / or CPS (Combined Proportion Score) It may include a map visualizing the results of classifying the data into categories based on a specific threshold. can.

[0064] The image output unit 320 displays an image showing information associated with the immunophenotype. In one embodiment, the image output unit 320 can output a pathology slide. and displaying the image, including one or more regions of interest and visual indicators, on a display device. Alternatively, the image output unit 320 can output one or more images of the pathology slide. An image containing visual markers can be overlaid on the region of interest above.

[0065] In FIG. 3, the image generating unit 310 is shown as being included in the user terminal 110. The image generating unit 310 can communicate with the user terminal 110 via wired and / or wireless communication. The information processing system 100 may also be included in any external device capable of processing the information. According to this configuration, the image output unit 320 of the user terminal 110 receives an image generated from an external device. The generated image is received by the user terminal 110 via wired and / or wireless communication. It can be displayed on a display device connected by a line.

[0066] 2 and 3, the target item detection unit 210, the region of interest determination unit 220, the immune table The current type determination unit 230, the image generation unit 310 and the image output unit 320 are included in the information processing system. Although the above description is based on the example in which the system 100 and the user terminal 110 are separately executed, the present invention is not limited to this. These components may also be implemented in a single device. Such components may be included in any number of devices (e.g., the information processing system 100 and the user The processing may be distributed among various terminals (e.g., the user terminal 110) in any combination.

[0067] FIG. 4 shows an example of immunophenotyping of a pathology slide image according to one embodiment of the present disclosure. 4 is a flowchart illustrating a method 400 for providing pathological information. The method 400 for providing information associated with an immunophenotype for a slide image includes: processor (e.g., at least one processor and / or information processing system of a user terminal) The processing can be performed by at least one processor in the system. The method 400 for providing information associated with an epidemiological phenotype includes: a processor processing a pathology slide image; By obtaining information associated with the immunophenotype for one or more regions of interest in the page, Here, immunophenotypes for one or more regions of interest can be determined (S410). The information associated with the type may include an immunophenotype (e.g., immunoreactivity) for one or more regions of interest. , immune exclusion, or immune deficiency) and / or immunity to one or more areas of interest. Phenotypic scores (e.g., scores for immune activity, scores for immune exclusion, or immune deficiency) Additionally or alternatively, one or more The information associated with the immunophenotype for a region of interest may be used to identify a single immunophenotype for one or more regions of interest. one or more immunophenotype-associated features (e.g., immunophenotype-associated features) It can contain statistical values ​​or vectors.

[0068] In one embodiment, the one or more regions of interest are one or more images of a pathology slide image. The determination can be based on the detection of the item of interest. For example, one or more regions of interest can be determined based on the pathology. At least one ride image that meets the conditions associated with one or more target items In another example, one or more regions of interest may be included in a pathology slide image. The results of one or more target item detections and / or pathology slide images are related. The region of interest can be input to the heart region extraction model and output. The region extraction model detects one or more target items in a reference pathology slide image. (e.g., a reference pathology slide image containing the target item detection results) and / or a reference pathology A model trained to take a slide image as input and output a reference region of interest. This can apply to:

[0069] The processor performs a step of generating a phenotype based on information associated with the immunophenotype for one or more regions of interest. An image showing information associated with the immunophenotype can be generated (S420). In an example, the processor generates visual indicators corresponding to the immunophenotype of one or more regions of interest. In another embodiment, the processor can generate an image including one or more immunophenotypes. An image can be generated that includes a visual indicator corresponding to the score. An image can be generated that includes a visual indicator corresponding to the score for immunoactivity. The processor then generates an image containing visual indicators corresponding to scores for immune exclusion. In another example, the processor may generate a visual response corresponding to a score for an immune deficiency. In yet another embodiment, the processor may generate an image including one or more indicia. Generate images containing visual labels corresponding to features associated with the immunophenotype of For example, the processor may generate a set of feature values ​​associated with one or more immunophenotypes. An image can be generated that includes visual indicators corresponding to the

[0070] The processor can then output an image showing information associated with the immunophenotype. In one embodiment, the processor detects one or more of the following in the pathology slide image: An image containing the region of interest and the visual markers can be output together. The processor generates an image containing visual markers for one or more regions of interest in a pathology slide image. can be overlayed.

[0071] In one embodiment, the processor extracts one or more items of interest from the pathology slide image. The detection results can be obtained and an image can be generated that shows the detection results for one or more target items. The processor can then output an image indicating the detection of one or more target items. .

[0072] FIG. 5 illustrates one or more related images within a pathology slide image 510 according to one embodiment of the present disclosure. Example of determining cardiac regions 522_1, 522_2, 522_3, 522_4, 524, and 526 1 is a diagram showing a user ( For example, doctors and researchers) may use patient tissue (e.g., tissue immediately before treatment or tissue after immunotherapy with anti-cancer drugs). The system can acquire a pathology slide (e.g., post-treatment tissue) and generate one or more pathology slide images. The researchers performed H&E staining on the acquired patient tissues and prepared H&E-stained tissue slides. Digitization by a scanner can generate pathology slide images. The user then performs IHC staining on the acquired patient tissue and analyzes the IHC-stained tissue samples. By digitizing the slides using a scanner, pathology slide images can be generated.

[0073] The region of interest determination unit 220 of the information processing system determines one or more regions in the pathology slide image. The region of interest can be determined in the following manner: a circle, a square, a rectangular rectangle, a polygon, and The region may have various shapes such as a contour, a line, a square, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a circle ... line, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a rectangle, a circle, a line, a rectangle, a rectangle, a rectangle In the present invention, the region of interest determining unit 220 determines one or more target items in the pathology slide image. Based on the detection result of the image, one or more regions of interest can be determined. For example, the region of interest determination unit 2 20 is a diagram showing the pathology slide image divided into N grids (where N is any natural number). Multiple patches (e.g., 1 mm) are generated by 2 Of the patches of this size, One or more items of interest (e.g., items associated with cancer and / or immune cells) The detected patches can be determined as regions of interest.

[0074] In one embodiment, the region of interest determiner 220 determines one or more regions of interest in the pathology slide image. one or more items that contain at least a partial area that satisfies the conditions associated with the target item For example, the region of interest determining unit 220 can determine a region of interest in a pathology slide image. In this case, target items (e.g., tumor cells, immune cells, cancer area ), cancer stroma, etc.) are greater than or equal to the standard value in number and / or area Additionally or alternatively, the region of interest determiner 220 may In the pathology slide image, the ratio and density of the target item are above the standard value. The region of interest can be determined as the region of interest, where the target item is the area to be immunophenotyped. The reference values ​​can correspond to the cells and / or regions detected for the purpose of statistical significance. Targeted interventions designed to define a clinically meaningful and / or meaningful immunophenotype A numerical value can be assigned to the item.

[0075] At this time, the region of interest determining unit 220 determines a region of any size in the pathology slide image. For example, the size of the region of interest can be determined by adding one or more target items to the region of interest. The size of the area of ​​interest can be dynamically determined to meet the criteria associated with the system. The area of ​​interest is not fixedly determined in advance, but is determined by the area of ​​interest determining unit 220 based on the number, size, ratio and The area where the numerical values ​​of the density and / or the like are equal to or greater than the reference value is determined as the area of ​​interest. Alternatively, the region of interest determining unit 220 may determine the size of the region of interest dynamically by One or more regions of interest can be defined to have a predetermined value.

[0076] In another embodiment, the region of interest determiner 220 determines one or more regions of interest for the pathology slide image. The detection results of the above target items and / or pathology slide images are input into the region of interest extraction model. By inputting the region, the output region can be determined as one or more regions of interest. The region extraction model detects one or more target items in a reference pathology slide image. The results and / or reference pathology slide images are input, and the reference region of interest is output. A machine learning model trained in this way (e.g. For example, neural networks, CNNs, SVMs, etc. In any case, the extent of the region of interest can be determined dynamically and / or statically.

[0077] In one embodiment, when multiple regions of interest are determined for a pathology slide image: The region of interest determination unit 220 determines whether at least some of the regions of interest overlap each other. For example, the region of interest determiner 220 may determine multiple regions of interest so that they overlap. When determining a first region of interest and a second region of interest for a slide image, the first region of interest is At least a part of the cardiac region and at least a part of the second region of interest overlap each other. In another embodiment, the area for one pathology slide image may be When determining multiple regions of interest, the region of interest determining unit 220 determines at least one of the multiple regions of interest. However, multiple regions of interest can be determined so that some regions of interest do not overlap each other. .

[0078] As shown in the figure, the region of interest determining unit 220 determines the region of interest in a pathology slide image (e.g., Pathology slide image (510) including detection results for the system is received, and One or more regions of interest 522_1, 522_2, 522_3, 522_4, 52 For example, the region of interest determiner 220 can determine the area of ​​interest associated with the target item. A specific width (e.g., 1 mm as the default width) that meets the specified conditions 2 ) area 522_1, 522_2, 522_3, and 522_4 can be determined as regions of interest. The region determining unit 220 determines the region 524 that satisfies the condition associated with the target item as a region of interest. The size of the region of interest can be determined dynamically. The unit 220 defines an oval region 526 that satisfies the conditions associated with the item of interest as a region of interest. can be determined as:

[0079] FIG. 6 illustrates an example of generating an immunophenotyping result 620 according to one embodiment of the present disclosure. In one embodiment, the immunophenotyping unit 230 performs one or more immunophenotyping tests in one or more regions of interest. The detection results of the target items (e.g., items associated with cancer and / or immune cells) Based on the results, immunophenotyping results 620 for one or more regions of interest can be generated. The phenotype determination unit 230 determines the phenotype based on the detection result of one or more target items in one or more regions of interest. Based on this, the immunophenotype of the region of interest is determined as at least one of immune activity, immune exclusion, or immune deficiency. It can be determined as follows.

[0080] In another embodiment, the immunophenotyping unit 230 may perform one or more immunophenotyping tests in one or more regions of interest. Calculate an immunophenotype score for one or more regions of interest based on the detection results of the target items. For example, the immunophenotyping unit 230 may detect one or more target molecules in one or more regions of interest. Based on the results of the item detection, the score for immune activity in the area of ​​interest and the score for immune exclusion are calculated. At least one of a score for immune deficiency or a score for immune deficiency can be calculated. The phenotyping unit 230 determines the scores for immune activity, immune exclusion, and the like of one or more regions of interest. and determining whether the area of ​​interest is immune-deficient based on at least one of the scores for the area of ​​interest or the score for the immune deficiency. For example, if the score for immune activity in a particular region of interest is above a threshold, a phenotype can be determined. In this case, the immunophenotype determination unit 230 can determine the immunophenotype of the region of interest as immune activity. Cut.

[0081] In one embodiment, the immunophenotyping unit 230 detects immunophenotypes within one or more regions of interest. At least one of the number, distribution, or density of cells is calculated, and the calculated number, distribution, or density of immune cells is analyzed. immunophenotyping and / or immunophenotyping of one or more regions of interest based on at least one of the density For example, the immunophenotyping unit 230 may determine a phenotyping score for a region of interest within one or more regions of interest. Lymphocytes in the cancer area The density of lymphocytes in cancer stroma and the number of immune cells in cancer stroma The density of immune cells in the cancer area or the stroma area was calculated. The immunophenotype of one or more regions of interest can be determined based on at least one of the following: density of immune cells; Additionally or alternatively, the immunophenotyping unit 230 may determine the presence or absence of a specific region within the cancerous region. Based on the number of immune cells detected, the immunophenotype of one or more regions of interest can be determined as immune activation, immune exclusion, or can be determined as a type of immune deficiency.

[0082] For example, the immunophenotyping unit 230 may determine whether the density of immune cells in the cancerous region is equal to or greater than a first threshold density. In some cases, the immunophenotype of the first region of interest 612 can be determined as immunoreactive. The phenotype determination unit 230 determines whether the density of immune cells in the cancerous region is less than a first threshold density and at the same time, If the density of immune cells in the cancer stroma is equal to or greater than a second threshold density, the second region of interest 614 The immunophenotype determination unit 230 can determine the immunophenotype as immune exclusion. The density of immune cells is less than the first threshold density, and at the same time, the density of immune cells in the cancer stroma is greater than the first threshold density. If the density is less than the threshold density of 2, the immunophenotype of the third region of interest 616 can be determined as immunodeficient. Here, the first threshold density is a threshold value for each of a plurality of regions of interest in a plurality of pathology slide images. In this case, the distribution of immune cell density within the cancer area can be determined. The threshold density of 2 is set to the value of the cancer stroma density in each of the plurality of regions of interest in the plurality of pathology slide images. This can be determined based on the distribution of immune cell density within the roma.

[0083] In another embodiment, the immunophenotyping unit 230 uses an artificial neural network immunophenotyping model. Entering features for each of one or more regions of interest allows you to Each immunophenotype and / or immunophenotype score can be determined. The current classification model is based on the reference region of interest, which is input as a feature. was trained to determine the immune phenotype of the subject as one of immune activity, immune exclusion, or immune deficiency. Also, where the feature for each of the one or more regions of interest is Statistical features (e.g., : density or number of specific target items in a region of interest), for one or more target items Geometric features (e.g., features containing relative position information between specific target items, etc.) image features (e.g., interest features) corresponding to each of the one or more regions of interest. Features extracted from multiple pixels contained in the region, image corresponding to the region of interest Additionally or alternatively, each of one or more regions of interest may include a vector, etc. The features for each are for one or more target items in each of one or more regions of interest. statistical features for one or more target items, geometric features for one or more target items, or Among the image features corresponding to each of the above regions of interest, two or more features are connected. The feature may include a plurality of interconnected features.

[0084] As shown, the immunophenotyping unit 230 detects one or more regions of interest 612, 614, 616, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 16 to generate immunophenotyping results 620, where one or more regions of interest 612, 614, and 616 may contain target item detection results for the region of interest. Therefore, the immunophenotyping unit 230 can determine the region of interest including the target item detection result. For example, the immunophenotyping unit 230 can generate an immunophenotyping result 620. an immunological table comprising the immunophenotype of the region of interest and / or an immunophenotype score for the region of interest; The immunophenotyping result 620 thus generated can be generated. One or more regions of interest 612, 614, 616 may be provided to a user terminal.

[0085] FIG. 7 is a diagram showing an example of outputting an immunophenotyping result according to one embodiment of the present disclosure. an information processing system (e.g., at least one processor of the information processing system) The user can then view and display information associated with the immunophenotype for one or more regions of interest in the image. By providing the information to the terminal, the user terminal can output the received information through the output device. where the immunophenotype for one or more regions of interest can be associated with the The obtained information may include an immunophenotype score for one or more regions of interest. The terminal (e.g., at least one processor of the user terminal) may An image showing the information associated with the corresponding immunophenotype can be output.

[0086] To this end, the user terminal may acquire one or more immunophenotype scores for one or more regions of interest. For example, the user terminal may generate an image containing visual indicators corresponding to one or more In a region corresponding to the region of interest, one or more immunophenotype scores for the region of interest An image can be generated containing visual indicators corresponding to one or more immunophenotypic sequences. The core is a score for immune activity, a score for immune exclusion, or a score for immune deficiency. The visual indicator may include at least one of the following: It can include color, brightness, saturation, etc.), text, images, marks, shapes, etc.

[0087] For example, for a score for immune activity (i.e., immune activity score), the user terminal The higher the immunoreactivity score, the higher the saturation of the color. It is possible to generate an image containing colors with lower saturation in areas that correspond to the area. Then, for the score for immune activity, the user terminal determines whether the immune activity score is in the first score zone. a first visual indicator in a region corresponding to the region of interest falling between the two, and an immunoreactivity score; A second visual indicator is placed in a region corresponding to the region of interest that falls within the second score interval. and a second immunoreactivity score in a region corresponding to the region of interest that falls within a third score interval. An image can be generated that includes three visual indicators, where the first visual indicator, the second visual indicator, The visual indicator and the third visual indicator can be different from each other.

[0088] In one embodiment, the user terminal is configured to detect one or more regions of interest in a pathology slide image and For example, the user terminal may output a live image containing visual indicators as described above. of generated images (e.g., images containing visual markers) and pathology slide images. At least a portion of the area (for example, the area corresponding to the generated image) is displayed on a display device. In another embodiment, the user terminal can simultaneously display one of the pathology slide images. An image containing visual markers can be overlaid on one or more regions of interest. The terminal may then transmit the generated image (e.g., an image including a visual indicator) to: At least a portion of the pathology slide image (e.g., a region corresponding to the generated image) It can be displayed on a display device overlapping the image area.

[0089] For example, as shown in the figure, the user terminal may Regions of interest whose immunoreactivity scores fall within the second score interval are colored white. The regions corresponding to the regions of interest whose immunoreactivity scores fall within the third score interval are colored light gray. The area corresponding to the region of interest is displayed in dark grey, and the area outside the region of interest is displayed in black. The user terminal can then generate an image 710 containing the visual indicators and the pathology. The slide image is then processed in parallel with the corresponding area 720 of the generated image 710. They can be arranged and displayed together on the user interface.

[0090] Additionally or alternatively, the user terminal may include, as information associated with the immunophenotype, tal Tissue Region" (e.g., tissue region in a pathology slide image) , "Cancer Region" (e.g., cancer region in a pathology slide image), "A Analyzable Region (e.g., region of interest in a pathology slide image), “Immune Phenotype Proportion” (e.g. The user interface can further display text, numbers, graphs, etc. for the Additionally or alternatively, the area of ​​the pathology slide image that is displayed on the display device may be A region may contain one or more target item detection results for that region, i.e. When outputting an image of at least a part of the pathology slide image, the user terminal , an image of at least a portion of an area showing detection results for one or more target items; can be output by a display device.

[0091] FIG. 8 is a diagram showing an example of outputting an immunophenotyping result according to another embodiment of the present disclosure. In one embodiment, a user terminal (e.g., at least one processor of the user terminal) The immunophenotype associated information for one or more regions of interest that is obtained may be used to identify one or more related For example, the user terminal may include one or more regions of interest. An image showing the information associated with the corresponding immunophenotype can be output.

[0092] To this end, the user terminal may display visual indicators corresponding to the immunophenotype of one or more regions of interest. For example, the user terminal may generate an image including regions corresponding to one or more regions of interest. In the region of interest, an image can be generated containing visual markers corresponding to the immunophenotype of the region of interest. That is, a first observation is made in a region corresponding to a region of interest in which the immunophenotype is immunoreactive. a second target in a region corresponding to a region of interest containing a visual label and whose immunophenotype is immunoexclusion; A third in a region corresponding to a region of interest that contains a visual marker and is immunophenotypically immunodeficient An image can be generated that includes a visual label for each immunophenotype. Color (e.g., color, brightness, saturation, etc.), text, and images that allow you to distinguish between types For example, the first visual indicator indicative of immune activity may include a mark, a graphic, or the like. is red, a second visual marker indicating immune exclusion is green, and a third visual marker indicating immune deficiency Additionally or alternatively, the first visual indicator indicative of immune activity may be a circular marker. The second visual sign of immune exclusion is a triangle, and the third visual sign of immune deficiency is a Knowledge can be marked with an X.

[0093] In one embodiment, the user terminal is configured to detect one or more regions of interest in a pathology slide image and For example, the user terminal may output a live image containing visual indicators as described above. The generated image (e.g., an image containing visual markers) is then compared to the pathology slide image. a display with at least a portion of the area (e.g., the area corresponding to the generated image) In another embodiment, the user terminal can display one of the pathology slide images. An image containing visual cues can be overlaid on these areas of interest. Finally, the generated images (e.g., images containing visual markers) are analyzed using the At least a portion of the processed slide image (e.g., a region corresponding to the generated image) ) can be displayed on a display device by overlapping it transparently, semi-transparently or opaquely.

[0094] For example, the user terminal may be configured to generate a region of interest with an immunophenotype of immunoreactive, as shown in the figure. The diagonal labeling in the region corresponds to a region of interest where the immunophenotype is immune exclusion. The region includes a vertical line label and corresponds to a region of interest where the immunophenotype is immunodeficient. The user terminal can then generate an image containing the visual indicator. The image containing the data is overlaid onto the corresponding area of ​​the pathology slide image. The pathology slide 810 is displayed on the user interface. At least a portion of the image (e.g., an area corresponding to the image containing a visual indicator) and / or the entire pathology slide image) 820 separately (e.g., as a minimap) They can be displayed together on the user interface (in a tab format).

[0095] Additionally or alternatively, the user terminal may store information associated with the immunophenotype, such as "AN ALYSIS SUMMARY”, “Biomaker Findings”, “Sco re" (e.g., immunoreactivity score), "Cutoff" (e.g., immunophenotyping) "Total Tissue Region", "Cance r Region”, “Analyzable Region”, “Immune P "henotype Proportion", "Tumor Infiltrating" Lymphocyte Density (e.g., density of immune cells in the cancer area, cancer Users can view text, numbers, and graphs for various parameters (e.g., immune cell density within the stromal region). Additionally or alternatively, the pathology slide images can be displayed on the interface. The area displayed on the display device is a result of detecting one or more target items in the area. That is, an image of at least a partial region of the pathology slide image can be included. When outputting, the user terminal displays the detection results for one or more target items. An image of at least a portion of the area can be output by a display device.

[0096] FIG. 9 is a diagram illustrating an example of an artificial neural network model 900 according to an embodiment of the present disclosure. The network model 900 is an example of a machine learning model. rning) technology and cognitive science, embodied in the structure of biological neural networks A statistical learning algorithm or a structure that implements the algorithm.

[0097] According to one embodiment, the artificial neural network model 900 is based on the synaptic network model 902, similar to a biological neural network. Nodes, which are artificial neurons that form a network by coupling, are called synapses. Iteratively adjust the weights of the inputs to find the error between the normal output and the inferred output for a specific input. By learning to reduce the number of problems, we can demonstrate that the machine learning model has the ability to solve problems. For example, the artificial neural network model 900 can be used in artificial intelligence such as machine learning and deep learning. This includes any probabilistic models, neural network models, etc. used in neural learning methods. can be done.

[0098] According to one embodiment, the artificial neural network model 900 performs the following steps from an input pathology slide image: The method may include an artificial neural network model configured to detect one or more items of interest. Additionally or alternatively, the artificial neural network model 900 may be configured to generate a pathology slide image. The method may include an artificial neural network model configured to determine one or more regions of interest from the do.

[0099] The artificial neural network model 900 is a multi-layered model consisting of multiple nodes and connections between them. Implemented as a perceptron (MLP: multilayer perceptron) The artificial neural network model 900 according to this embodiment can be used with various artificial neural network models including MLP. As shown in FIG. 9, an artificial neural network model 900 can be implemented using one of the following structures: 9, an input layer 920 that receives an input signal or data 910 from the outside, and a an output layer 940 that outputs an output signal or data 950; and a signal between the input layer 920 and the output layer 940. , which receives signals from the input layer 920, extracts characteristics, and transmits them to the output layer 940. (where n is a positive integer) hidden layers 930_1 to 930_n. The output layer 940 receives signals from the hidden layers 930_1 to 930_n and outputs them to the outside.

[0100] The learning method of the artificial neural network model 900 involves solving problems in response to the input of a teacher signal (correct answer). Supervised learning method to learn to optimize and unsupervised learning (Unsupervised Learning) which does not require a teacher signal. In one embodiment, the information processing system includes: The artificial neural network model 900 is trained and / or supervised to detect one or more target items. can be learned by unsupervised learning. For example, the information processing system can Using the slide image and label information for one or more reference items, the pathologist An artificial neural network model 90 is configured to detect one or more target items from the slide image. 0 can be learned through supervised learning.

[0101] In another embodiment, the information processing system extracts one or more images of interest from the pathology slide image. The artificial neural network model 900 is trained by supervised and / or unsupervised learning to determine the region. For example, the information processing system can learn from reference pathology slide images and and / or the detection results of one or more target items against reference pathology slide images (e.g. , a reference pathology slide image containing the target item detection results and a reference region of interest The signal information is used to determine one or more regions of interest from a pathology slide image. The neural network model 900 can be trained by supervised learning. and / or the reference region of interest is a region of interest in the pathology slide image and / or the reference pathology slide image. That is, it includes at least a portion of the area that satisfies the conditions associated with one or more target items. This can be done.

[0102] The artificial neural network model 900 thus trained is stored in the memory (not shown) of the information processing system. Pathology slide images received from the communication module and / or memory can be stored in the Detect one or more target items in a pathology slide image in response to an input. Additionally or alternatively, the artificial neural network model 900 may perform a The method detects one or more target items and / or inputs to a pathology slide image. Optionally, one or more regions of interest can be determined from the pathology slide image.

[0103] According to one embodiment, the input variables of the artificial neural network model for detecting the target item include one or more Pathology slide images (e.g., H&E stained pathology slide images, IHC stained For example, the input layer 9 of the artificial neural network model 900 may be The input variables input to 20 are one or more pathology slide images as one vector data. It can be an image vector 910 composed of elements. The output variables output by the output layer 940 of the neural network model 900 are obtained from the pathology slide image. The resulting vector 950 represents or characterizes one or more detected target items. That is, the output layer 940 of the artificial neural network model 900 obtains the following from the pathology slide image: It is designed to output a vector that represents or characterizes one or more detected target items. In the present disclosure, the output variables of the artificial neural network model 900 are as described above. It refers to one or more target items detected from a pathology slide image, not limited to the type of Furthermore, the output layer of the artificial neural network model 900 may include any information / data. 940 indicates the reliability and / or accuracy of the output target item detection result, etc. Can be configured to output a vector.

[0104] In another embodiment, a machine learning model, i.e., an artificial neural network model, is used to determine the region of interest. The input variables of the rule 900 are the detection results of one or more target items on the pathology slide image. results (e.g., detection data for items of interest in pathology slide images) and / or pathology For example, the input to the input layer 920 of the artificial neural network model 900 may be a processed slide image. The input variables are the detection results of one or more target items on the pathology slide images and Image vector data consisting of a pathology slide image and / or a pathology slide image as a single vector data element. Detecting one or more items of interest in a pathology slide image. Depending on the results and / or inputs to the pathology slide images, the artificial neural network model 900 The output variables output by the output layer 940 represent or characterize one or more regions of interest. In this disclosure, the output variables of the artificial neural network model 900 can be: Including, but not limited to, the types described above, any information / data that indicates one or more areas of interest It is possible.

[0105] In this way, the input layer 920 and the output layer 940 of the artificial neural network model 900 are provided with a plurality of input variables. The numbers and corresponding output variables are matched, and the input layer 920, hidden layer 930_1 and By adjusting the synaptic values ​​between the nodes included in the output layer 930_n and the output layer 940, It can learn to extract the correct output corresponding to a specific input. This allows us to understand the characteristics hidden in the input variables of the artificial neural network model 900 and to generate a model based on the input variables. The artificial neural network model 90 is used to reduce the error between the calculated output variables and the target output. The synaptic values ​​(or weights) between nodes with 0 can be adjusted. Using the network model 900, the target item is selected according to the input pathology slide image. Additionally or alternatively, an artificial neural network model 900 can be used to generate inputs. The pathology slide image and / or one or more target addresses for the pathology slide image Depending on the item detection results (e.g., pathology slide images containing target item detection results), One or more regions of interest can be output.

[0106] FIG. 10 shows an example of a method for associating immunophenotypes with pathology slide images according to one embodiment of the present disclosure. 10 shows the configuration of an exemplary computer device (e.g., user terminal) 1000 that provides the requested information. As shown in the figure, the computer system 1000 includes one or more processors 10 10, bus 1030, communication interface 1040, and processor 1010. A memory 1020 into which a computer program 1060 is loaded and a computer The computer program 1060 may include a storage module 1050 for storing the computer program 1060. However, FIG. 10 shows only the components associated with the embodiment of the present disclosure. A person skilled in the art of the present disclosure would understand that, in addition to the components shown in FIG. It will be appreciated that other general purpose components and the like may also be included.

[0107] The processor 1010 controls the overall operation of each component of the computer device 1000 . The processor 1010 includes a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Cont) GPU (Graphic Processing Unit), GPU (Graphic Processing Unit) ) or any other type of processor known in the art. 010 includes at least one application for performing the method according to the embodiments of the present disclosure. The computing device 1000 can perform operations on one or more applications or programs. The device may include a processor.

[0108] The memory 1020 may store various data, instructions, and / or information. , from the storage module 1050 to perform the methods / operations according to various embodiments of the present disclosure. One or more computer programs 1060 can be loaded from the memory 1020. M, but the scope of the present disclosure is not limited thereto. It's not that.

[0109] The bus 1030 may provide communication between the components of the computer device 1000. The bus 1030 is an address bus, a data bus, and a control bus. .

[0110] The communication interface 1040 is used for wired or wired internet communication of the computer device 1000. The communication interface 1040 can also support various communication methods other than internet communication. For this purpose, the communication interface 1040 can support various communication methods. It may include any communication module known in the art.

[0111] The storage module 1050 stores one or more computer programs 1060 non-temporarily. The storage module 1050 is a ROM (read-only memory) , EPROM(erasable programmable read-only m EEPROM (electrically erasable PROM) ), non-volatile memory such as flash memory, hard disk, removable disk or any other form of computer-readable recording medium known in the art. Cut.

[0112] When loaded into memory 1020, the computer program 1060 executes the program. 1010 includes one or more interfaces that perform operations / methods according to various embodiments of the present disclosure. It can include instructions. That is, the processor 1010 can perform the operations / methods according to various embodiments of the present disclosure by executing one or more instructions.

[0113] For example, the computer program 1060 can include one or more instructions to perform operations such as obtaining information associated with the immunophenotype for one or more regions of interest in a pathological slide image, generating an image showing the information associated with the immunophenotype based on the information associated with the immunophenotype for one or more regions of interest, and outputting an image showing the information associated with the immunophenotype. In such a case, a reaction prediction system for an immune anticancer agent according to some embodiments of the present disclosure can be implemented via the computer device 1000. embodiments can be implemented.

[0114] FIG. 11 is a diagram showing an example of outputting a detection result of a target item according to an embodiment of the present disclosure. The processor (for example, at least one processor of the user terminal) can obtain the detection result of one or more target items from the pathological slide image. For example, the information processing system can use a target item detection model to detect one or more target items from the pathological slide image and provide the target item detection result to the user terminal. Here, the target item detection model can include a model learned to detect one or more reference target items from a reference pathological slide image.

[0115] The processor generates an image showing the detection result of one or more target items, and the generated ​​​​​​​​​​​​​​​​​​An image showing the detection of one or more target items may be output. The processor detects one or more items of interest from the pathology slide image based on the results. , an image can be generated and output that includes visual markers that distinguish each target item. For example, the processor may determine the items of interest that can be considered when determining the immunophenotype, such as , cancer area, cancerous tissue area, blood vessels, cancer cells, immune cells, positive / negative by biomarker expression level Additionally or alternatively, an image can be generated that includes visual indicators for each of the cells (e.g., negative cells). Then, the processor performs area-based target item detection based on one or more target item detection results. Segmentation map for a specific structure Contours for target items with structures (e.g., blood vessels, etc.) Indicates the center point of the target item in cell units or the shape of the target item in cell units Images can be generated that include contours, etc.

[0116] In one embodiment, the processor comprises: one or more regions of interest in the pathology slide image; and outputting an image showing the detection of one or more target items for one or more regions of interest. For example, the processor may process the generated image (e.g., a visual target) as described above. and at least a partial area of ​​the pathology slide image (e.g., the generated In another example, the processor may output a pathology slide along with the image and corresponding regions. An image showing the detection of one or more target items in one or more regions of interest in the image. It is possible to overlay a page (e.g., an image containing visual indicators) on the The terminal may then transmit the generated image (e.g., an image including a visual indicator) to: At least a portion of the pathology slide image (e.g., a region corresponding to the generated image) The image data can be displayed on a display device connected to the user terminal.

[0117] For example, the processor may select the area other than the target item (background) as shown in the figure. The first color indicates the cancerous epithelial area, the second color indicates the cancerous stromal area, and the third color indicates the cancerous stromal area. a fourth color indicating immune cells (e.g., immune cell centers) and a fourth color indicating cancer cells (e.g., cancer A fifth color indicating the center of the cell is displayed in the corresponding area of ​​the pathology slide image. The image 1110 is displayed on the user interface. 0 is a pathology slide image showing the detection result of one or more target items. The first image corresponds to the image overlaid (or merged) with the corresponding area of ​​the first image. The color, the second color, the third color, the fourth color, and / or the fifth color are distinct from one another. It can correspond to different colors.

[0118] Additionally or alternatively, the user terminal may store information associated with each subject's immunophenotype. Visual indicators corresponding to the items (e.g., color information corresponding to each target item) ), text, numbers, indicators, graphs, etc. for "ANALYSIS SUMMARY" Additionally or alternatively, the user terminal may display a pathology slide. The user selects the target item and displays a visual indicator in the corresponding area of ​​the image. In Figure 11, the user interface can be selected and output. Color is used as an example, but not limited to, a visual cue to convey information. There is no.

[0119] FIG. 12 shows a target item detection result and an immunophenotype determination result according to an embodiment of the present disclosure. 1 illustrates an example of a processor (e.g., at least one processor in a user terminal) The immunophenotype associated with one or more regions of interest (e.g., immunophenotype) The processor can output an image showing one or more target items (a map). In one embodiment, the processor may output an image showing the detection result of the system. information associated with the immunophenotype for the region of interest above and / or one or more items of interest; The image showing the detection result of the smear is overlaid onto the corresponding area of ​​the pathology slide image. It can be output in a ray (or merged).

[0120] For example, the processor may select immune desert as shown in the figure. The first color indicates immune exclusion, the second color indicates immune exclusion, A third color indicates immune inflamed tissue, and a fourth color indicates cancerous epithelial areas. a third color indicating cancer stroma regions; a fourth color indicating immune cells (e.g., immune cell foci); The sixth color and the seventh color indicating the cancer cells (for example, the center point of the cancer cells) are added to the pathology slide image. The image 1210 displayed in the corresponding area of ​​the page is displayed on the user interface. Here, the image 1210 is associated with an immunophenotype for one or more regions of interest. An image showing the information received (e.g., an immunophenotype map) and one or more target items The image showing the detection results is overlaid onto the corresponding area of ​​the pathology slide image. It can be a combination of the first color, second color, and third color. The fourth color, the fifth color, the sixth color, and / or the seventh color are different and distinguishable from one another. This can correspond to the color.

[0121] Additionally or alternatively, the processor may select at least a portion of the pathology slide image (e.g., For example, images containing visual landmarks and corresponding regions and / or entire pathology slide images. The image 1220 of the body is displayed separately (for example, in the form of a minimap) on the user interface. Additionally or alternatively, the processor may display information associated with the immunophenotype. For information, visual signs corresponding to each target item (e.g., color information), text, numbers, and graphs for the ANALYSIS SUMMARY Additionally or alternatively, pathology slides can be displayed on the user interface. The target item and / or immunorepresentation display a visual indicator in a corresponding region of the image. A user interface can be output that allows the user to select information associated with a type. 12, information associated with the immunophenotype and / or visual indicators indicating the item of interest. Color is used as an example of, but is not limited to, the color.

[0122] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications of the present disclosure will be apparent to those of ordinary skill in the art and are within the scope of the present disclosure as defined herein. The general principles and the like described herein may be adapted to various modifications without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure should not be construed as being limited to the examples and the like described herein. The broadest scope consistent with the principles and novel features disclosed in this application is not intended to be limiting. is intended to be granted.

[0123] Exemplary embodiments are presently disclosed in the context of one or more stand-alone computer systems. While reference may be made to utilizing the subject matter in a variety of forms, the subject matter is not so limited. Works with any computing environment, such as a network or distributed computing environment Additionally, aspects of the presently disclosed subject matter may be embodied in multiple processing chips. Storage may be implemented on or across multiple devices. Such devices can be PCs, networks, It may also include a client and a handheld device.

[0124] Although the present disclosure has been described herein in connection with some embodiments, the present invention pertains to Various modifications within the scope of the present disclosure that would be understood by one of ordinary skill in the art are possible. It will be appreciated that variations and modifications are possible and that such variations and modifications are within the spirit and scope of the present invention. It must be understood that the invention falls within the scope of the appended claims.

Claims

1. 1. A computer device comprising: a memory storing one or more instructions; A computer device comprising: a processor configured to execute the one or more stored instructions to detect immune cells and cancer-related items in a pathology slide image, determine an immune phenotype for each of one or more regions of interest in the pathology slide image based on the detection results, and output information related to the immune phenotype for each of the one or more regions of interest overlaid on the pathology slide image.

2. the cancer-related items include at least one of a cancer region and a cancer stromal region; The computer device of claim 1 , wherein the one or more regions of interest are determined based on at least one of a detection result for the cancer region or a detection result for the cancer stroma region.

3. the one or more regions of interest are determined from a plurality of patches obtained by dividing the pathology slide image into equal-sized patches; The information associated with the immunophenotype may include: at least one of a plurality of classes representative of the immune environment of each of said one or more regions of interest; At least one of the scores for each of the plurality of classes; or The computer device of claim 1 , further comprising at least one feature indicative of the immune environment of each of the one or more regions of interest.

4. the immune phenotype comprises at least one of immune activity, immune exclusion, and immune deficiency; The computer device of claim 1 , wherein each of the immune phenotypes is indicative of the immune environment of the region of interest.

5. the outputting generates one or more visual indicators corresponding to the immunophenotype of each of the one or more regions of interest; The computing device of claim 1 , further comprising outputting the generated one or more visual indicators by overlaying them on the pathology slide image.

6. 1. A method for providing immunophenotype-related information for a pathology slide image, performed by at least one computing device, comprising: detecting immune cells and cancer-associated items within the pathology slide image; determining an immunophenotype for each of one or more regions of interest within the pathology slide image based on the detection results; A method for providing information related to immunophenotypes for a pathology slide image, comprising a step of overlaying and outputting information related to immunophenotypes for each of the one or more regions of interest on the pathology slide image.

7. the cancer-related items include at least one of a cancer region and a cancer stromal region; The method for providing information related to an immunophenotype for a pathology slide image according to claim 6, wherein the one or more regions of interest are determined based on at least one of detection results for the cancer region or detection results for the cancer stroma region.

8. The method for providing information related to an immune phenotype for a pathology slide image described in claim 7, wherein the one or more regions of interest are regions that satisfy conditions associated with at least one of the immune cells, the cancer region, and the cancer stroma region.

9. the one or more regions of interest are determined from a plurality of patches obtained by dividing the pathology slide image into equal-sized patches; The information associated with the immunophenotype may include: at least one of a plurality of classes representative of the immune environment of each of said one or more regions of interest; At least one of the scores for each of the plurality of classes; or The method for providing information related to an immunophenotype for a pathology slide image of claim 6, including at least one feature indicative of the immune environment of each of the one or more regions of interest.

10. the immune phenotype comprises at least one of immune activity, immune exclusion, and immune deficiency; The method for providing information associated with immunophenotypes for pathology slide images according to claim 6 , wherein each of the immunophenotypes indicates the immune environment of the region of interest.

11. The outputting step includes: generating one or more visual indicators corresponding to the immunophenotype of each of the one or more regions of interest; The method for providing information related to an immunophenotype for a pathology slide image of claim 6, further comprising the step of overlaying the generated one or more visual markers on the pathology slide image and outputting the same.

12. the immune phenotype is determined based on at least one of a plurality of classes, each class representing the immune environment of the region of interest; The image including the one or more visual indicators may include: a first visual indicator corresponding to a first class in a region corresponding to a first region of interest having an immunophenotype of the first class among the plurality of classes; The method of claim 11, further comprising: a second visual indicator corresponding to a second class in a region corresponding to a second region of interest having an immunophenotype of a second class among the plurality of classes.

13. The method of claim 12 , wherein the first visual indicator and the second visual indicator are displayed visually different from each other.

14. calculating an immune activity score based on information associated with the immune phenotype for each of the one or more regions of interest; The method for providing information related to an immune phenotype for a pathology slide image described in claim 6, further comprising a step of outputting a prediction result regarding whether a patient associated with the pathology slide image will respond to an immune anti-cancer drug based on the immune activity score.

15. The determining step includes: calculating at least one of the number, distribution, and density of the immune cells for each of the one or more regions of interest; The method for providing information related to an immunophenotype for a pathology slide image of claim 7, further comprising: determining an immunophenotype for each of the one or more regions of interest based on the calculated results.

16. The calculating step 16. The method for providing information related to an immune phenotype for a pathology slide image of claim 15, comprising the step of calculating, for each of the one or more regions of interest, at least one of the density of the immune cells in the cancer region within the region of interest and the density of the immune cells in the cancer stroma region within the region of interest.

17. The method for providing information related to an immune phenotype for a pathology slide image described in claim 6, further comprising a step of outputting at least one of the area of ​​the total tissue area in the pathology slide image, the area of ​​the total cancer area in the pathology slide image, the area of ​​the total region of interest, a graph showing the proportion of the immune phenotype, an immune activity score calculated based on information related to the immune phenotype for each of the one or more regions of interest, or cutoff information used to predict whether a patient associated with the pathology slide image will respond to an immune anticancer drug.

18. the detecting step includes detecting tumor cells, the immune cells, the cancer region, and the cancer stroma region in the pathology slide image; The method further includes a step of overlaying the detection result on the pathology slide image and outputting the result; The method for providing information related to an immunophenotype for a pathology slide image according to claim 7, wherein the detection results include detection results regarding at least one of the tumor cells, the immune cells, the cancer region, or the cancer stroma region.

19. The method for providing information related to an immunophenotype for a pathology slide image according to claim 18, wherein the tumor cells, the immune cells, the cancer region, and the cancer stroma region are displayed visually differently from each other on the pathology slide image.

20. A computer program recorded on a computer-readable recording medium for executing the method for providing information related to an immunophenotype for a pathology slide image according to any one of claims 6 to 19.

Images