Method and system for predicting response to immunological anti-cancer drugs

Abstract

A method for predicting response to an immunological anti-cancer agent is provided, the method being performed by at least one computer device. [Solution] This method includes the steps of receiving a first pathology slide image, detecting one or more target items within the first pathology slide image, determining at least one of the immunophenotype or information associated with the immunophenotype of at least a portion of the region within the first pathology slide image based on the detection results for the one or more target items, and generating a prediction result regarding whether a patient associated with the first pathology slide image will respond to an immunological anticancer drug based on the immunophenotype or information associated with the immunophenotype of at least a portion of the region within the first pathology slide image.

Description

[Technical Field]

[0001] The present disclosure relates to a method and system for predicting response to an immune anti-cancer drug, and more particularly to a method and system for generating a prediction result as to whether a patient associated with a pathology slide image will respond to an immune anti-cancer drug based on at least one of the immune phenotype of at least a portion of the region in the pathology slide image or information associated with the immune phenotype. [Background technology]

[0002] Recently, there has been growing interest in immunotherapy, a third-generation anticancer drug that utilizes the patient's immune system. An immunotherapy can refer to any drug that prevents cancer cells from evading the body's immune system or enables immune cells to recognize and attack cancer cells. Because it acts through the body's immune system, it has minimal side effects and can extend the survival time of cancer patients compared to other anticancer drugs. However, such immunotherapy is not effective for all cancer patients. Therefore, predicting the reactivity of immunotherapy is important to predict the effectiveness of immunotherapy in current cancer patients.

[0003] Meanwhile, PD-L1 expression can be used as a biomarker to predict the response to immunotherapy anticancer drugs. Pretreatment tissues from patients are obtained and stained using immunohistochemistry (IHC). PD-L1 expression levels are then measured manually. Patients with a certain level of expression can be predicted to be responsive to immunotherapy anticancer drugs. Conventional methods involve manually calculating PD-L1 expression, which is subjective and difficult to quantify objectively. Furthermore, while there are many factors involved in predicting response to immunotherapy anticancer drugs, relying solely on PD-L1 expression may result in reduced accuracy. This is because, even when PD-L1 is expressed, immune cells must be present in the vicinity of cancer cells to demonstrate a response to immunotherapy anticancer drugs. Furthermore, current methods for quantifying PD-L1 expression alone make it difficult to determine the spatial distribution of immune cells associated with the antitumor effect of immunotherapy anticancer drugs. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method and system for predicting response to an immunological anti-cancer drug to solve the above-mentioned problems. [Means for solving the problem]

[0005] The present disclosure may be embodied in numerous ways, including as a method, an apparatus (system), a computer-readable storage medium storing instructions, or a computer program.

[0006] A method for predicting a response to an immune anti-cancer drug, performed by at least one computer device according to one embodiment of the present disclosure, includes the steps of receiving a first pathology slide image, detecting one or more target items in the first pathology slide image, determining an immune phenotype or information associated with the immune phenotype of at least a portion of the area in the first pathology slide image based on the detection results for the one or more target items, and generating a prediction result regarding whether a patient associated with the first pathology slide image will respond to the immune anti-cancer drug based on the immune phenotype or information associated with the immune phenotype of at least a portion of the area in the first pathology slide image.

[0007] In one embodiment of the present disclosure, the detecting step includes detecting one or more target items in the first pathology slide image using an artificial neural network target item detection model, wherein the artificial neural network target item detection model is trained to detect one or more reference target items from a reference pathology slide image.

[0008] In one embodiment of the present disclosure, at least a portion of the region within the first pathology slide image includes one or more target items, the one or more target items including items associated with cancer and immune cells, and the determining step includes calculating at least one of the number, distribution, or density of immune cells within the items associated with cancer within at least a portion of the region within the first pathology slide image, and determining at least one of the immune phenotype or information associated with the immune phenotype of at least a portion of the region within the first pathology slide image based on the calculated at least one of the number, distribution, or density of immune cells.

[0009] In one embodiment of the present disclosure, the items associated with cancer include a cancer area and a cancer stroma, the calculating step includes a step of calculating the density of immune cells in the cancer area within at least a portion of the first pathology slide image, and a step of calculating the density of immune cells in the cancer stroma within at least a portion of the first pathology slide image, and the determining step includes a step of determining at least one of the immune phenotype or information associated with the immune phenotype of at least a portion of the first pathology slide image based on at least one of the density of immune cells in the cancer area or the density of immune cells in the cancer stroma.

[0010] In one embodiment of the present disclosure, if the density of immune cells in the cancerous region is equal to or greater than a first threshold density, the immune phenotype of at least a portion of the region in the first pathology slide image is determined as immune inflamed; if the density of immune cells in the cancerous region is less than the first threshold density and at the same time the density of immune cells in the cancer stroma is equal to or greater than a second threshold density, the immune phenotype of at least a portion of the region in the first pathology slide image is determined as immune excluded; and if the density of immune cells in the cancerous region is less than the first threshold density and at the same time the density of immune cells in the cancer stroma is less than the second threshold density, the immune phenotype of at least a portion of the region in the first pathology slide image is determined as immune desert.

[0011] In one embodiment of the present disclosure, a first threshold density is determined based on the distribution of immune cell density within cancerous regions in each of a plurality of regions of interest in a plurality of pathology slide images, and a second threshold density is determined based on the distribution of immune cell density within cancer stroma in each of a plurality of regions of interest in a plurality of pathology slide images.

[0012] In one embodiment of the present disclosure, the determining step further includes determining the immune phenotype of at least a portion of the region in the first pathology slide image as one of immune activation, immune exclusion, or immune deficiency based on the number of immune cells contained in a specific region within the cancerous region.

[0013] In one embodiment of the present disclosure, the determining step includes a step of determining the immunophenotype or information associated with the immunophenotype of each of at least a portion of the region in the first pathology slide image by inputting features for each of at least a portion of the region in the first pathology slide image or at least a portion of the region in the first pathology slide image into an artificial neural network immunophenotyping classification model, and the artificial neural network immunophenotyping classification model is trained to determine at least one of the immunophenotypes or information associated with the immunophenotype of at least a portion of the region in the reference pathology slide image by inputting features for at least a portion of the region in the reference pathology slide image or at least a portion of the region in the reference pathology slide image.

[0014] In one embodiment of the present disclosure, the features for at least a portion of the region within the first pathology slide image include at least one of statistical features for one or more target items in at least a portion of the region within the first pathology slide image, geometric features for one or more target items, or image features corresponding to at least a portion of the region within the first pathology slide image.

[0015] In one embodiment of the present disclosure, at least a portion of the region within the first pathology slide image includes a plurality of regions of interest, and the immunophenotype of at least a portion of the region within the first pathology slide image includes each of the immunophenotypes of the plurality of regions of interest, and the generating step includes a step of determining the immunophenotype most frequently contained within the entire region of the first pathology slide image based on the immunophenotype of each of the plurality of regions of interest, and a step of generating a prediction result regarding whether or not the patient will respond to an immunological anticancer drug based on the immunophenotype most frequently contained within the entire region of the first pathology slide image.

[0016] In one embodiment of the present disclosure, the generating step includes a step of generating an immune phenotype map for at least a portion of the first pathology slide image using the immune phenotype of at least a portion of the first pathology slide image, and a step of generating a prediction result as to whether or not the patient will respond to the immune anticancer drug by inputting the generated immune phenotype map into an immune anticancer drug response prediction model, wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype map.

[0017] In one embodiment of the present disclosure, the generating step includes a step of generating an immune phenotype feature map for at least a portion of the area in the first pathology slide image using information associated with the immune phenotype of at least a portion of the area in the first pathology slide image, and a step of generating a prediction result as to whether or not the patient will respond to the immune anticancer drug by inputting the generated immune phenotype feature map into an immune anticancer drug response prediction model, wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype feature map.

[0018] In one embodiment of the present disclosure, the method further includes a step of obtaining information regarding the expression of biomarkers from a second pathology slide image associated with the patient, and the generating step includes a step of generating a prediction result regarding whether the patient will respond to an immunological anticancer drug based on at least one of the immunophenotype or information associated with the immunophenotype of at least a portion of the region in the first pathology slide image and the information regarding the expression of biomarkers.

[0019] In one embodiment of the present disclosure, the biomarker is PD-L1.

[0020] In one embodiment of the present disclosure, the information about PD-L1 expression includes at least one of a Tumor Proportion Score (TPS) value or a Combined Proportion Score (CPS) value.

[0021] In one embodiment of the present disclosure, the obtaining step includes receiving a second pathology slide image; and generating information about PD-L1 expression by inputting the second pathology slide image into an artificial neural network expression information generation model, wherein the artificial neural network expression information generation model is trained to generate reference information about PD-L1 expression by inputting the reference pathology slide image.

[0022] In one embodiment of the present disclosure, the step of generating information about PD-L1 expression by inputting the second pathology slide image into the artificial neural network expression information generation model includes the step of using the artificial neural network expression information generation model to detect at least one of the location of tumor cells, the location of lymphocytes, the location of macrophages, or the presence or absence of PD-L1 expression in at least a portion of the second pathology slide image, thereby generating information about PD-L1 expression.

[0023] In one embodiment of the present disclosure, the method further includes outputting at least one of the detection results for one or more target items, the immune phenotype of at least a portion of the area in the first pathology slide image, information associated with the immune phenotype, a prediction result as to whether the patient will respond to an immune anticancer drug, or the density of immune cells in at least a portion of the area in the first pathology slide image.

[0024] In one embodiment of the present disclosure, the method further includes a step of outputting information regarding at least one immune anticancer drug that is suitable for the patient from among the multiple immune anticancer drugs based on the prediction result regarding whether the patient will respond to the immune anticancer drug.

[0025] A computer program stored on a computer-readable recording medium is provided for executing the method for predicting a response to an immunological anti-cancer drug according to one embodiment of the present disclosure.

[0026] An information processing system according to one embodiment of the present disclosure includes a memory storing one or more instructions; and a processor configured to execute the one or more stored instructions to receive a first pathology slide image, detect one or more target items in the first pathology slide image, determine an immunophenotype or information associated with the immunophenotype of at least a portion of the region in the first pathology slide image based on the detection results for the one or more target items, and generate a prediction result regarding whether a patient associated with the first pathology slide image will respond to an immunological anticancer drug based on the immunophenotype or information associated with the immunophenotype of at least a portion of the region in the first pathology slide image. [Effects of the Invention]

[0027] According to some embodiments of the present disclosure, it is possible to predict whether a patient will respond to an immunological anticancer drug by using at least one of an immunophenotype determined from a pathology slide image and information associated with the immunophenotype, i.e., by objectively analyzing the immune environment surrounding cancer cells, it is possible to improve the prediction rate of whether a patient will respond to an immunological anticancer drug.

[0028] According to some embodiments of the present disclosure, PD-L1 expression can be objectively quantified, and the quantified PD-L1 expression can be used to predict whether a patient will respond to an immunotherapy. By using not only information on PD-L1 expression but also immunophenotypes determined from pathology slide images and / or information associated with the immunophenotypes, the accuracy of predicting whether a patient will respond to an immunotherapy can be further improved.

[0029] According to some embodiments of the present disclosure, an artificial neural network model can be used to determine immunophenotypes and / or information associated with immunophenotypes, and to obtain information regarding PD-L1 expression, which can be more accurate and faster than conventional techniques.

[0030] According to some embodiments of the present disclosure, a user may be provided with visually and intuitively generated results from the process of predicting response to an immunological anticancer drug. A report organizing the results generated from the process of predicting response to an immunological anticancer drug may also be provided to the user. Furthermore, a recommendation may be made to the user of an immunological anticancer drug and / or a combination of immunological anticancer drugs that is optimal for the patient from among multiple immunological anticancer drugs, based on the results generated from the process of predicting response to an immunological anticancer drug.

[0031] The effects of the present disclosure are not limited to these, and other effects not mentioned should be clearly understood by a person having ordinary knowledge in the technical field to which the present disclosure pertains (hereinafter referred to as "ordinary engineer") from the description of the claims. [Brief explanation of the drawings]

[0032] BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Embodiments of the present disclosure will now be described, without limitation, with reference to the accompanying drawings, in which like reference numerals indicate like elements and in which:

[0014] FIG. [Figure 1] 1 is an exemplary configuration diagram showing a system in which an information processing system according to an embodiment of the present disclosure provides response prediction results to an immunological anticancer drug. [Figure 2] 1 is a block diagram showing an internal configuration of an information processing system according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart illustrating a method for predicting response to an immune anti-cancer drug according to one embodiment of the present disclosure. [Figure 4] 10A-10C show examples of detecting target items in a pathology slide image, determining a region of interest, and determining the immunophenotype of the region of interest according to one embodiment of the present disclosure. [Figure 5] FIG. 10 illustrates an example of generating immunophenotyping results according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram showing an example of generating a response prediction result for an immunological anticancer drug according to an embodiment of the present disclosure. [Figure 7] FIG. 10 is a diagram showing the results of an analysis of the predictive ability of information on PD-L1 expression and / or immunophenotype information regarding the response to an immunotherapy anticancer drug in accordance with one embodiment of the present disclosure. [Figure 8] 1 is a graph showing the correlation between immunophenotype and information regarding the expression of CD3-positive cells, CD8-positive cells, FOXP3-positive cells, and CD68-positive cells according to one embodiment of the present disclosure. [Figure 9] 1 is a graph showing the performance of various methods for predicting whether a patient will respond to an immunotherapy anti-cancer drug, according to one embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing an example of outputting a result generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to an embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 12]FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 13] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 14] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 15] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 16] FIG. 1 illustrates an example of an artificial neural network model according to an embodiment of the present disclosure. [Figure 17] FIG. 1 is a diagram illustrating an exemplary system configuration for providing response prediction results to an immune anticancer drug according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, detailed descriptions of functions and configurations known in the art will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0034] In the accompanying drawings, the same or corresponding components are denoted by the same reference numerals. In addition, in the following description of the embodiments, repeated descriptions of the same or corresponding components may be omitted. However, the omission of a description of a component does not mean that such a component is not included in a certain embodiment.

[0035] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the following examples in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, and may be embodied in various different forms. However, the present embodiments are provided solely for the purpose of completeness of the disclosure and to enable those skilled in the art to accurately recognize the scope of the invention.

[0036] The terms used in this specification will be briefly explained, and the disclosed embodiments will be specifically described. The terms used in this specification are currently commonly used and generally used terms, taking into consideration the function of the present disclosure. However, these terms may change depending on the intentions of engineers in the relevant field, legal precedents, the emergence of new technologies, etc. In addition, in specific cases, the applicant may arbitrarily select terms, and their meanings will be described in detail in the description of the invention. Therefore, the terms used in this disclosure should be defined based on the meanings of the terms and the overall content of the present disclosure, rather than simply by the names of the terms.

[0037] In this specification, unless otherwise clearly specified in the context, singular expressions can include plural expressions, and plural expressions can include singular expressions. Throughout this specification, when a part "comprises" a certain element, this does not mean that other elements are excluded, and that other elements can also be included, unless otherwise specified.

[0038] Additionally, the terms "module" and "module" as used herein refer to software or hardware components, each of which performs a specific function. However, the terms "module" and "module" are not limited to software or hardware. A "module" or "module" may reside on an addressable storage medium or execute on one or more processors. Thus, by way of example, a "module" or "module" may include at least one of a software component, an object-oriented software component, a component such as a class component or task component, a process, a function, an attribute, a procedure, a subroutine, a program code segment, a driver, firmware, microcode, a circuit, data, a database, a data structure, a table, an array, or a variable. The components and "modules" or "modules" may be combined into fewer components and "modules" or "modules," or the functionality provided therein may be further separated into additional components and "modules" or "modules."

[0039] According to one embodiment of the present disclosure, a "module" or "unit" may be embodied with a processor and memory. "Processor" should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, "processor" may also refer to an application-specific semiconductor (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), etc. "Processor" may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such configuration. Also, "memory" should be broadly interpreted to include any electronic component capable of storing electronic information. "Memory" can refer to various types of processor-readable media, such as RAM (Random Access Memory), ROM (Read Only Memory), NVRAM (Non-Volatile Random Access Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic or optical data storage devices, registers, etc. Memory is said to be in electronic communication with a processor when the processor can read / write information from or write information to the memory. Memory that is integrated into a processor is in electronic communication with the processor.

[0040] In the present disclosure, a "system" may include at least one of a server device and a cloud device, but is not limited to this. For example, a system may be composed of one or more server devices. As another example, a system may be composed of one or more cloud devices. As yet another example, a system may be operated by both a server device and a cloud device.

[0041] In the present disclosure, "target data" may refer to any data or data item that can be used to train a machine learning model, including, but not limited to, data representing an image, data representing audio or audio features, etc. In the present disclosure, the target data is described as an entire pathology slide image and / or at least one patch included in the pathology slide image, but is not limited thereto, and any data that can be used to train a machine learning model may be the target data. In addition, the target data is tagged with label information through annotation work.

[0042] In this disclosure, a "pathology slide image" refers to a captured image of a pathology slide in which tissue, etc., removed from a human body has been fixed and stained through a series of chemical processes for microscopic observation. For example, a pathology slide image may refer to a digital image captured with a microscope and may include information about cells, tissues, and / or structures within the human body. A pathology slide image may also include one or more patches, and one or more patches may be tagged with label information (e.g., information about immunophenotypes) through annotation work. For example, a "pathology slide image" may include, but is not limited to, H&E-stained tissue slides and / or IHC-stained tissue slides, and may also include tissue slides stained with various staining methods (e.g., chromogenic in situ hybridization (CISH), fluorescent in situ hybridization (FISH), multiplex IHC, etc.) or unstained tissue slides. As another example, a "pathology slide image" may be a patient tissue slide generated to predict immune anticancer drug response, and may include a patient tissue slide before immune anticancer drug treatment and / or a patient tissue slide after immune anticancer drug treatment.

[0043] In the present disclosure, a "patch" may refer to a small region within a pathology slide image. For example, a patch may include a region corresponding to a semantic object extracted by performing segmentation on the pathology slide image. As another example, a patch may refer to a combination of pixels associated with label information generated by analyzing the pathology slide image.

[0044] In the present disclosure, "at least a portion of a region of a pathology slide image" may refer to at least a portion of a region of a pathology slide image that is to be analyzed. For example, "at least a portion of a region of a pathology slide image" may refer to at least a portion of a region of the pathology slide image that includes a target item. As another example, "at least a portion of a region of a pathology slide image" may refer to at least a portion of multiple patches generated by dividing the pathology slide image. Furthermore, "at least a portion of a region of a pathology slide image" may refer to the entire region or a portion of all regions (or all patches) that make up the pathology slide image. In the present disclosure, "at least a portion of a region of a pathology slide image" may be referred to as a region of interest, and conversely, a region of interest may refer to at least a portion of a pathology slide image.

[0045] In the present disclosure, the terms "machine learning model" and / or "artificial neural network model" may include any model used to infer an answer to a given input. According to one embodiment, the machine learning model may include an artificial neural network model including an input layer, multiple hidden layers, and an output layer. Here, each layer may include multiple nodes. For example, the machine learning model may be trained to infer label information for at least one patch included in a pathology slide image and / or a pathology slide. In this case, the label information generated by the annotation operation is used to train the machine learning model. In addition, the machine learning model may include weights associated with multiple nodes included in the machine learning model. Here, the weights may include any parameters associated with the machine learning model.

[0046] In this disclosure, "training" may refer to any process of modifying weights associated with a machine learning model using at least one patch and label information. According to one embodiment, training may refer to a process of modifying or updating weights associated with a machine learning model using at least one patch and label information through one or more rounds of forward propagation and backward propagation.

[0047] In the present disclosure, "label information" refers to correct answer information for a data sample, and is information obtained as a result of an annotation operation. The terms "label" and "label information" may be used interchangeably with terms such as "annotation" and "tag" in the art. In the present disclosure, "annotation" may refer to an annotation operation and / or annotation information (e.g., label information, etc.) determined by performing the annotation operation. In the present disclosure, "annotation information" may refer to information for the annotation operation and / or information (e.g., label information) generated by the annotation operation.

[0048] In the present disclosure, a "target item" may refer to data / information, image regions, objects, etc. that are to be detected in a pathology slide image. According to one embodiment, a target item may include an object to be detected from a pathology slide image for the diagnosis, treatment, and prevention of a disease (e.g., cancer). For example, a "target item" may include a cell-based target item and a region-based target item.

[0049] In the present disclosure, the "immunophenotype of at least a region of a pathology slide" can be determined based on at least one of the number, distribution, and density of immune cells in at least a region of the pathology slide. Such immune phenotype can be displayed in various classification systems, for example, immune inflamed, immune excluded, and immune desert.

[0050] In the present disclosure, "information associated with an immune phenotype" may include any information that represents or characterizes an immune phenotype. According to one embodiment, the information associated with an immune phenotype may include features of the immune phenotype. Here, the features of the immune phenotype may include score values ​​(class-specific scores or density values ​​of a classifier) ​​corresponding to a class corresponding to the immune phenotype (e.g., immune activity, immune exclusion, immune deficiency) and / or various vectors associated with the immune phenotype, such as features input to the classifier. For example, information associated with an immune phenotype may include: 1) a score value associated with the immune phenotype output from an artificial neural network or machine learning model; 2) density values, numbers, and various statistical values ​​of immune cells applied to a threshold (or cut-off) for the immune phenotype, or vector values ​​expressing the distribution of immune cells; 3) scalar values ​​or vector values ​​including the relative relationships (e.g., histogram vectors or graph representation vectors considering direction and distance) between immune cells or cancer cells and other cell types (cancer cells, immune cells, fibroblasts, lymphocytes, plasma cells, macrophages, endothelial cells, etc.) and relative statistical values ​​(e.g., the ratio of the number of immune cells to the number of other cells); and 4) types of immune cells and cancer cells and surrounding regions (e.g., cancer region, cancer stroma region, tertiary lymphoid structure, normal region, necrosis, fat, blood vessel, high endothelial venule, lymphatic It can include scalar or vector values ​​including statistical values ​​by vessel, nerve, etc. (e.g., ratio of cancer stroma area to immune cell count), and distributions (e.g., histogram vectors, graph representation vectors, etc.).

[0051] In the present disclosure, "each of a plurality of A's" and / or "each of a plurality of A's" can refer to each of all the components included in the plurality of A's, or each of some of the components included in the plurality of A's. For example, each of a plurality of regions of interest can refer to each of all the regions of interest included in the plurality of regions of interest, or each of some of the regions of interest included in the plurality of regions of interest.

[0052] In this disclosure, "instructions" may refer to one or more instruction words that are combined based on functionality, are components of a computer program, and are executed by a processor.

[0053] In the present disclosure, a "user" may refer to a person who uses a user terminal. For example, a user may include an annotator who performs annotation work. As another example, a user may include a doctor or patient who is provided with a response prediction result for an immunosuppressant (e.g., a prediction result as to whether a patient will respond to an immunosuppressant). Furthermore, a user may refer to a user terminal, and conversely, a user terminal may refer to a user. That is, the terms "user" and "user terminal" may be used interchangeably in this specification.

[0054] FIG. 1 is an exemplary configuration diagram illustrating a system in which an information processing system 100 provides a response prediction result to an immunological anticancer drug according to an embodiment of the present disclosure. As shown in the figure, the system for providing a response prediction result to an immunological anticancer drug (e.g., a prediction result as to whether a patient will respond to the immunological anticancer drug) may include an information processing system 100, a user terminal 110, and a storage system 120. Here, the information processing system 100 may be configured to be connected to and communicate with each of the user terminal 110 and the storage system 120. While FIG. 1 illustrates a single user terminal 110, the present invention is not limited thereto, and multiple user terminals 110 may be connected to and communicate with the information processing system 100. Also, while FIG. 1 illustrates the information processing system 100 as a single computer device, the present invention is not limited thereto, and the information processing system 100 may be configured to perform distributed processing of information and / or data via multiple computer devices. Also, while FIG. 1 illustrates the storage system 120 as a single device, the present invention is not limited thereto, and the information processing system 100 may be configured as multiple storage devices or as a cloud-based system. In addition, in Figure 1, each component of the system that provides response prediction results for immune anticancer drugs represents a functionally divided functional element, and multiple components can be embodied in a form that is integrated with each other in an actual physical environment.

[0055] The information processing system 100 and the user terminal 110 are any computer devices used to generate and provide a predicted result of a response to an immunosuppressant. Here, the computer device may refer to any type of device equipped with computer functions, such as, but not limited to, a notebook, desktop, laptop, server, cloud system, etc.

[0056] The information processing system 100 may receive a first pathology slide image. For example, the information processing system 100 may receive the first pathology slide image from the storage system 120. As another example, the information processing system 100 may receive the first pathology slide image from the user terminal 110. The information processing system 100 may be configured to use the received first pathology slide image to generate a prediction result regarding whether a patient associated with the first pathology slide image will respond to an immunotherapy anticancer drug.

[0057] In one embodiment, the information processing system 100 can detect one or more target items in a first pathology slide image. For example, the information processing system 100 can detect one or more target items in the first pathology slide image using an artificial neural network target item detection model. Here, the artificial neural network target item detection model can correspond to a model trained to detect one or more reference target items from a reference pathology slide image. Here, the one or more target items can include items associated with cancer and immune cells. Also, the items associated with cancer can include a cancer area and cancer stroma. In this specification, the term "cancer area" can be used interchangeably with the term "cancer epithelium."

[0058] The information processing system 100 can determine at least one of the immune phenotype or information associated with the immune phenotype of at least a portion of a region in a pathology slide image based on the detection results for one or more target items. In one embodiment, the information processing system 100 calculates at least one of the number, distribution, or density of immune cells in an item associated with cancer within at least a portion of a region in the pathology slide image, and determines at least one of the immune phenotype or information associated with the immune phenotype of at least a portion of the region in the pathology slide image based on the calculated number, distribution, or density of immune cells. For example, the information processing system 100 can calculate the density of immune cells in a cancerous region and the density of immune cells in a cancer stroma within at least a portion of the region in the pathology slide image, and determine at least one of the immune phenotype or information associated with the immune phenotype of at least a portion of the region in the pathology slide image based on at least one of the density of immune cells in the cancerous region and the density of immune cells in the cancer stroma.

[0059] In this case, if the density of immune cells in the cancerous region is equal to or greater than a first threshold density, the immune phenotype of at least a portion of the region in the pathology slide image can be determined as immune inflamed. Alternatively, if the density of immune cells in the cancerous region is less than the first threshold density and the density of immune cells in the cancer stroma is equal to or greater than a second threshold density, the immune phenotype of at least a portion of the region in the pathology slide image can be determined as immune excluded. Alternatively, if the density of immune cells in the cancerous region is less than the first threshold density and the density of immune cells in the cancer stroma is less than a second threshold density, the immune phenotype of at least a portion of the region in the pathology slide image can be determined as immune desert. Here, the first threshold density is determined based on the distribution of immune cell densities in the cancerous region in each of a plurality of regions of interest in a plurality of pathology slide images, and the second threshold density is determined based on the distribution of immune cell densities in the cancer stroma in each of a plurality of regions of interest in a plurality of pathology slide images. Additionally or alternatively, the information processing system 100 can determine the immune phenotype of one or more regions of interest as one of immune activation, immune exclusion, or immune deficiency based on the number of immune cells contained within a particular region within the cancerous region.

[0060] In another embodiment, the information processing system 100 may determine at least one of an immunophenotype or information associated with the immunophenotype of at least a portion of the pathology slide image by inputting features for at least a portion of the pathology slide image or at least a portion of the pathology slide image into the artificial neural network immunophenotyping model. Here, the artificial neural network immunophenotyping model may correspond to a model trained to determine at least one of an immunophenotype or information associated with the immunophenotype of at least a portion of the pathology slide image by inputting features for at least a portion of the pathology slide image or at least a portion of the pathology slide image. Here, the features for at least a portion of the pathology slide image may include at least one of statistical features for one or more target items in at least a portion of the pathology slide image, geometric features for one or more target items, or image features corresponding to at least a portion of the pathology slide image. Furthermore, the at least a portion of the pathology slide image input into the artificial neural network immunophenotyping model may include, but is not limited to, at least a portion of the H&E stained image, at least a portion of the IHC stained image, at least a portion of the Multiplex IHC stained image, etc.

[0061] The information processing system 100 can generate a prediction result as to whether a patient associated with a first pathology slide image will respond to an immunological anti-cancer drug based on at least one of the immunophenotype of at least a partial region in the pathology slide image or information associated with the immunophenotype. In one embodiment, the information processing system 100 can determine the immunophenotype most frequently contained within the entire region of the first pathology slide image (i.e., a representative immunophenotype) based on the immunophenotype for each of multiple regions of interest in the first pathology slide image. The information processing system 100 can generate a prediction result as to whether a patient will respond to an immunological anti-cancer drug based on the immunophenotype most frequently contained within the entire region of the first pathology slide image thus determined.

[0062] In another embodiment, the information processing system 100 generates an immune phenotype map for at least a portion of the pathology slide image using the immune phenotype of at least a portion of the pathology slide image, and inputs the generated immune phenotype map into an immune anticancer drug response prediction model to generate a prediction result regarding whether or not a patient will respond to the immune anticancer drug. Here, the immune anticancer drug response prediction model may include a statistical model and / or an artificial neural network model trained to generate a reference prediction result by inputting a reference immune phenotype map.

[0063] In another embodiment, the information processing system 100 can acquire information about biomarker expression from a second pathology slide image associated with a patient, and generate a prediction result regarding whether the patient will respond to an immune-mediated anticancer drug based on at least one of the immunophenotype and information associated with the immunophenotype of at least a portion of the first pathology slide image and the information about the biomarker expression. Here, the biomarker may be, but is not limited to, PD-L1, and can include various biomarkers associated with immune cells, such as CD3, CD8, CD68, FOXP3, CD20, CD4, CD45, and CD163. Here, the information about PD-L1 expression can include at least one of a tumor proportion score (TPS) or a combined proportion score (CPS). For example, the information processing system 100 can receive the second pathology slide image and input the second pathology slide image into an artificial neural network expression information generation model to generate information about biomarker expression. For example, the information processing system 100 can generate information related to biomarker expression by detecting at least one of the positions of cells (tumor cells, lymphocytes, macrophages, etc.) present in at least a partial region of the second pathology slide image, whether or not the cells express a biomarker, the number of biomarker-positive cells, and a score for the amount of biomarker-positive cells using the artificial neural network expression information generation model. Here, the artificial neural network expression information generation model can correspond to a model trained to generate reference information related to biomarker expression by inputting a reference pathology slide image.

[0064] For example, the information processing system 100 can receive a second pathology slide image and input the second pathology slide image into the artificial neural network expression information generation model to generate information about PD-L1 expression. For example, the information processing system 100 can use the artificial neural network expression information generation model to detect at least one of the location of tumor cells, the location of lymphocytes, the location of macrophages, and the presence or absence of PD-L1 expression in at least a portion of the second pathology slide image, thereby generating information about PD-L1 expression. Here, the artificial neural network expression information generation model can correspond to a model trained to generate reference information about PD-L1 expression by inputting a reference pathology slide image.

[0065] The information processing system 100 can output at least one of the detection results for one or more target items, the immune phenotype of at least a partial region in the pathology slide image, information associated with the immune phenotype, a prediction result as to whether a patient will respond to an immunological anticancer drug, or the density of immune cells in at least a partial region in the pathology slide image via the user terminal 110. That is, a result generated in the process of predicting a response to an immunological anticancer drug is provided to a user 130 (e.g., a doctor or a patient) via the user terminal 110. Additionally or alternatively, the information processing system 100 can output information regarding at least one immunological anticancer drug that is suitable for the patient from among multiple immunological anticancer drugs, based on the prediction result as to whether the patient will respond to the immunological anticancer drug, via the user terminal 110.

[0066] The storage system 120 is a device or cloud system that stores and manages pathology slide images associated with a target patient and various data associated with a machine learning model to provide response prediction results for an immunotherapy anticancer drug. To efficiently manage data, the storage system 120 may store and manage various data using a database. Here, the various data may include any data associated with a machine learning model, such as, but not limited to, a file of the target data, meta information of the target data, label information related to the target data resulting from the annotation work, data related to the annotation work, and a machine learning model (e.g., an artificial neural network model). While the information processing system 100 and the storage system 120 are shown as separate systems in FIG. 1 , they are not limited thereto and may be integrated into a single system.

[0067] Since information on the overall distribution of immune cells (e.g., the extent to which immune cells have infiltrated into cancer cells) plays an important role in predicting response to immune-mediated anti-cancer drugs, information on the distribution of various immune cells in H&E-stained pathology slide images (i.e., H&E-stained tissue slide images) can be used to predict response to immune-mediated anti-cancer drugs. According to some embodiments of the present disclosure, the immune environment surrounding cancer cells can be objectively analyzed to improve the prediction rate of whether a patient will respond to an immune-mediated anti-cancer drug. Furthermore, PD-L1 expression can be objectively quantified, and the quantified PD-L1 expression can be used to predict whether a patient will respond to an immune-mediated anti-cancer drug.

[0068] 2 is a block diagram showing the internal configuration of an information processing system 100 according to an embodiment of the present disclosure. According to one embodiment, as shown in the figure, the information processing system 100 may include a target item detection unit 210, a region of interest determination unit 220, an immunophenotype determination unit 230, and an immunoanticancer drug response prediction unit 240. In FIG. 2, each component of the information processing system 100 represents a functionally separated functional element, and multiple components may be embodied in a form in which they are integrated with each other in an actual physical environment.

[0069] The target item detection unit 210 receives a pathology slide image (e.g., a first pathology slide image stained with H&E, a second pathology slide image stained with IHC, etc.) and detects one or more target items in the received first pathology slide image. In one embodiment, the target item detection unit 210 detects one or more target items in the first pathology slide image using an artificial neural network target item detection model. For example, the target item detection unit 210 can detect, as target items, tumor cells, lymphocytes, macrophages, dendritic cells, fibroblasts, endothelial cells, blood vessels, cancer stroma, cancer epithelium, cancer areas, normal areas (e.g., normal lymph node architecture areas), etc. in the first pathology slide image.

[0070] The region of interest determination unit 220 can determine one or more regions of interest within the first pathology slide image. Here, the region of interest can include a region in which one or more target items are detected within the pathology slide image. For example, the region of interest determination unit 220 can determine, as the region of interest, a patch including one or more target items among multiple patches constituting the first pathology slide image. In the present disclosure, the region of interest determination unit 220 is illustrated as being included in the information processing system 100, but this is not limited thereto. The information processing system 100 can process at least a portion of the first pathology slide image without determining a region of interest, and the immunophenotype determination unit 230 and the immunoanticancer drug response prediction unit 240 can process at least a portion of the first pathology slide image.

[0071] The immunophenotyping unit 230 can determine at least one of the immunophenotypes or information associated with the immunophenotypes of at least a portion of the region (e.g., one or more regions of interest) in the first pathology slide image based on the detection results for one or more target items. In one embodiment, the immunophenotyping unit 230 can calculate at least one of the number, distribution, or density of immune cells in an item associated with cancer in at least a portion of the region in the first pathology slide image, and determine at least one of the immunophenotypes or information associated with the immunophenotypes of at least a portion of the region in the first pathology slide image based on the calculated number, distribution, or density of immune cells. For example, the immunophenotyping unit 230 can calculate at least one of the density of immune cells in a cancer region or the density of immune cells in a cancer stroma in one or more regions of interest, and determine the immunophenotypes and / or information associated with the immunophenotypes of the one or more regions of interest based on the calculated number, distribution, or density of immune cells. Additionally, the immunophenotyping unit 230 can determine the immunophenotype of one or more regions of interest as one of immune activation, immune exclusion, or immune deficiency based on the number of immune cells contained within a particular region within the cancerous region.

[0072] In one embodiment, the immunophenotyping unit 230 can determine the immunophenotype and / or information associated with the immunophenotype of at least a portion of the first pathology slide image by inputting at least one of features for at least a portion of the first pathology slide image or at least a portion of the first pathology slide image into an artificial neural network immunophenotyping model. Here, the features for at least a portion of the first pathology slide image can include at least one of statistical features for one or more target items in at least a portion of the first pathology slide image, geometric features for one or more target items, or image features corresponding to at least a portion of the first pathology slide image. Additionally or alternatively, the features for at least a portion of the first pathology slide image can include a feature that combines two or more of the statistical features, geometric features, and image features described above.

[0073] The immune anticancer drug response prediction unit 240 can generate a prediction result as to whether a patient associated with the first pathology slide image will respond to an immune anticancer drug based on the immune phenotype of at least a portion of the region in the first pathology slide image and / or information associated with the immune phenotype.

[0074] In one embodiment, the immunological anticancer drug response prediction unit 240 determines the most prevalent immunophenotype within the entire area of ​​the first pathology slide image based on the immunophenotypes of each of the multiple regions of interest in the first pathology slide image, and generates a prediction result as to whether the patient will respond to the immunological anticancer drug based on the most prevalent immunophenotype within the entire area of ​​the first pathology slide image. In another embodiment, the immunological anticancer drug response prediction unit 240 generates an immunological phenotype map for at least a portion of the area in the first pathology slide image using the immunological phenotypes of at least a portion of the area in the first pathology slide image, and inputs the generated immunological phenotype map into an immunological anticancer drug response prediction model to generate a prediction result as to whether the patient will respond to the immunological anticancer drug.

[0075] Here, the immunophenotype map can refer to a collection of regions of interest generated by classifying at least a portion of a region in a first pathology slide image into one of three immunophenotypes. For example, the immunophenotype map can refer to a map in which the immunophenotypes of each of a plurality of regions of interest are displayed in a pathology slide image. Additionally or alternatively, the immunophenotype map can include an immune phenotype feature map that further includes information associated with the three immune phenotypes (e.g., scores for the immune phenotypes output from the artificial neural network immunophenotyping model, immune cell density values ​​applied to a threshold (or cut-off) for the immune phenotype, etc.).

[0076] In another embodiment, the immunotherapy anticancer drug response prediction unit 240 can acquire information on the expression of biomarkers (e.g., PD-L1) from a second pathology slide image associated with the patient (e.g., an IHC-stained pathology slide image of a region corresponding to the first pathology slide image). The immunotherapy anticancer drug response prediction unit 240 can generate a prediction result on whether the patient will respond to the immunotherapy anticancer drug based on the immunophenotype or at least one of information associated with the immunophenotype of at least a portion of the region in the first pathology slide image and information on the expression of biomarkers. The information on the expression of biomarkers can be generated by inputting the second pathology slide image into an artificial neural network expression information generation model.

[0077] 2, the information processing system 100 includes a target item detection unit 210, a region of interest determination unit 220, an immunophenotype determination unit 230, and an immunosuppressant response prediction unit 240. However, the present invention is not limited to this, and some components may be omitted or other components may be added. In one embodiment, the information processing system 100 further includes an output unit (not shown), which can output at least one of the detection results for one or more target items, the immunophenotype of at least a portion of the first pathology slide image, information associated with the immunophenotype, a prediction result as to whether the patient will respond to the immunosuppressant, or the density of immune cells in one or more regions of interest. For example, the output unit can output information regarding at least one immunosuppressant suitable for the patient from among multiple immunosuppressants.

[0078] 3 is a flowchart illustrating a method 300 for predicting a response to an immunological anti-cancer drug according to one embodiment of the present disclosure. In one embodiment, the method 300 for predicting a response to an immunological anti-cancer drug may be performed by a processor (e.g., at least one processor of an information processing system). The method 300 for predicting a response to an immunological anti-cancer drug may begin by the processor receiving a first pathology slide image (S310). The processor may detect one or more items of interest within the first pathology slide image (S320). For example, the processor may detect one or more items of interest within the first pathology slide image using an artificial neural network object item detection model.

[0079] The processor can then determine at least one of an immunophenotype or information associated with the immunophenotype of at least a portion of the first pathology slide image based on the detection results for the one or more target items (S330). Here, the at least a portion of the first pathology slide image can include one or more target items, and the one or more target items can include items associated with cancer and immune cells. For example, the processor can calculate at least one of the number, distribution, or density of immune cells in the items associated with cancer within at least a portion of the first pathology slide image, and determine the immunophenotype and / or information associated with the immunophenotype of at least a portion of the first pathology slide image based on the calculated number, distribution, or density of immune cells.

[0080] The processor can generate a prediction result as to whether a patient associated with the first pathology slide image will respond to an immunological anti-cancer drug based on at least one of the immunophenotype of at least a portion of the first pathology slide image or information associated with the immunophenotype (S340). In one embodiment, the processor can generate a prediction result as to whether a patient will respond to an immunological anti-cancer drug based on the immunophenotype most frequently contained within the entire region of the first pathology slide image. In another embodiment, the processor can generate a prediction result as to whether a patient will respond to an immunological anti-cancer drug by inputting an immunophenotype map for at least a portion of the first pathology slide image into an immunological anti-cancer drug response prediction model. In yet another embodiment, the processor can generate an immunophenotype feature map for at least a portion of the first pathology slide image using information associated with the immunophenotype of at least a portion of the first pathology slide image, and input the generated immunophenotype feature map into an immunological anti-cancer drug response prediction model to generate a prediction result as to whether a patient will respond to an immunological anti-cancer drug.

[0081] In another embodiment, the processor may acquire information about biomarker expression from a second pathology slide image associated with the patient, and generate a prediction result regarding whether the patient will respond to an immunological anticancer drug based on the immunophenotype and biomarker expression information of at least a portion of the first pathology slide image. To this end, the processor may receive the second pathology slide image associated with the patient and input the second pathology slide image into an artificial neural network expression information generation model to generate information about biomarker expression. Here, the second pathology slide image may correspond to a pathology slide image for a region corresponding to the first pathology slide image.

[0082] FIG. 4 illustrates an example of detecting an item of interest in a pathology slide image, determining a region of interest, and determining the immunophenotype of the region of interest according to one embodiment of the present disclosure. To predict whether a patient will respond to an immunotherapy, a user (e.g., a doctor or researcher) can acquire a patient's tissue (e.g., tissue immediately before treatment) and generate one or more pathology slide images. For example, the user can generate a pathology slide image (e.g., a first pathology slide image) by staining the acquired patient's tissue with H&E and digitizing the H&E-stained tissue slide using a scanner. As another example, the user can generate a pathology slide image (e.g., a second pathology slide image) by staining the acquired patient's tissue with IHC and digitizing the IHC-stained tissue slide using a scanner.

[0083] A processor (e.g., at least one processor of an information processing system) may detect various target items from a pathology slide image (e.g., a digitized H&E whole slide image) using an artificial neural network target item detection model (S410). Here, the artificial neural network target item detection model may correspond to a model trained to detect one or more reference target items from a reference pathology slide image. For example, the processor may detect tumor cells, lymphocytes, macrophages, dendritic cells, fibroblasts, endothelial cells, etc. as cell-based target items. Additionally or alternatively, the processor may detect cancer stroma, cancer epithelium, cancer areas, normal areas (e.g., areas containing normal lymph node architecture), etc. as region-based target items.

[0084] In one embodiment, the processor can detect cell-based target items in a pathology slide image using an artificial neural network model trained to detect cell-based target items. Additionally or alternatively, the processor can detect area-based target items in a pathology slide image using an artificial neural network model trained to detect area-based target items. That is, the artificial neural network model for detecting cell-based target items and the artificial neural network model for detecting area-based target items can be separate models. Alternatively, the processor can detect cell-based target items and / or area-based target items in a pathology slide image using an artificial neural network model trained to simultaneously detect cell-based target items and area-based target items. That is, both cell-based and area-based target items can be detected using a single artificial neural network model.

[0085] The cell-based target item detection results and the region-based target item detection results may be mutually complementary. In one embodiment, the processor may estimate a cancerous region (i.e., a region-based target item detection result) within a pathology slide image based on the detection results for tumor cells (i.e., a cell-based target item detection result), or may correct or modify the estimated cancerous region. Conversely, the processor may estimate a tumor cell (i.e., a cell-based target item detection result) within a pathology slide image based on the detection results for cancerous regions (i.e., a region-based target item detection result), or may correct or modify the estimated tumor cell. That is, although there is a difference between cancerous regions and tumor cells, in that a cancerous region corresponds to a region-based target item and a tumor cell corresponds to a cell-based target item, a region including tumor cells ultimately corresponds to a cancerous region, and therefore the cell-based target item detection results and the region-based target item detection results may be mutually complementary.

[0086] For example, when detecting target items in regions, detection errors may occur, such as missing detailed cell-level target items (i.e., tumor cells), but the processor can complement the detection results based on whether cells in a region detected as a cancerous region were actually detected as tumor cells. Conversely, the processor can complement the detection results based on whether a region detected as a tumor cell was actually detected as a cancerous region. In this way, the processor can complement the strengths and weaknesses of the cell-level target item detection results and the region-level target item detection results, thereby minimizing errors in the detection results.

[0087] The processor can detect cancerous regions (e.g., cancer stroma), cancer stroma, and / or cancer-associated items, including tumor cells, and immune cells, within a pathology slide image (e.g., an H&E-stained whole slide image) (S410). The processor can then determine one or more regions of interest within the pathology slide image (S420). For example, the processor can determine a plurality of patches (e.g., 1 mm ) generated by dividing the pathology slide image into N grids (where N is any natural number). 2 At least some of the patches (e.g., patches in which items and / or immune cells associated with cancer are detected) can be determined as regions of interest (e.g., at least a portion of a region within a pathology slide image).

[0088] The processor may determine an immunophenotype of one or more regions of interest based on the detection results of cancer-associated items and / or immune cells in the one or more regions of interest (S430). Here, the one or more regions of interest may include a plurality of pixels, and the detection results of cancer-associated items and / or immune cells in the one or more regions of interest may include a predicted value for each of the cancer-associated items and / or immune cells in each of the plurality of pixels included in the one or more regions of interest. In one embodiment, the processor may calculate at least one of the number, distribution, or density of immune cells in the cancer-associated items in the one or more regions of interest, and determine the immunophenotype of the one or more regions of interest based on at least one of the calculated number, distribution, or density of immune cells. For example, the processor may calculate the density of immune cells in the cancer region (lymphocytes in tumor area) and the density of immune cells in the cancer stroma (lymphocytes in stroma area) in the one or more regions of interest, and determine the immunophenotype of the one or more regions of interest based on at least one of the density of immune cells in the cancer region or the density of immune cells in the cancer stroma. Additionally, the processor may determine the immune phenotype of one or more regions of interest as one of immune activation, immune exclusion, or immune deficiency based on the number of immune cells contained within a particular region within the cancerous region.

[0089] As shown in Figure 4, if the density of immune cells in the cancer region is equal to or greater than a first threshold density, the immunophenotype of one or more regions of interest can be determined as immune inflamed. Alternatively, if the density of immune cells in the cancer region is less than the first threshold density while the density of immune cells in the cancer stroma is equal to or greater than a second threshold density, the immunophenotype of one or more regions of interest can be determined as immune excluded. Alternatively, if the density of immune cells in the cancer region is less than the first threshold density while the density of immune cells in the cancer stroma is less than a second threshold density, the immunophenotype of one or more regions of interest can be determined as immune desert.

[0090] Here, the first threshold density can be determined based on the distribution of immune cell densities in cancerous regions in each of multiple regions of interest in multiple pathology slide images. Similarly, the second threshold density can be determined based on the distribution of immune cell densities in cancer stroma in each of multiple regions of interest in multiple pathology slide images. For example, the first threshold density can correspond to the top X% (where X is a number between 0 and 100) density values ​​of the distribution of immune cell densities in cancerous regions in each of multiple regions of interest in multiple pathology slide images. Similarly, the second threshold density can correspond to the top Y% (where Y is a number between 0 and 100) density values ​​of the distribution of immune cell densities in cancer stroma in each of multiple regions of interest in multiple pathology slide images. The values ​​of X and / or Y can be determined by biological criteria or any other criteria according to the prior art.

[0091] The first threshold density and the second threshold density may be the same or different. Furthermore, the first threshold density or the second threshold density may each include two or more threshold densities. In one embodiment, the first threshold density or the second threshold density may include a threshold density for an upper region and a threshold density for a lower region. For example, if the density of immune cells in the cancerous region is equal to or greater than the first threshold density for the upper region, this may correspond to a case where the density of immune cells in the cancerous region is equal to or greater than the first threshold density. Furthermore, if the density of immune cells in the cancerous region is less than the first threshold density for the lower region, this may correspond to a case where the density of immune cells in the cancerous region is less than the first threshold density.

[0092] 4 illustrates generating an immunophenotype of the region of interest (S430), but is not limited thereto. Information associated with the immunophenotype can also be generated. To generate information associated with the immunophenotype, the processor can use the factors used to determine the immunophenotype, such as those described above. For example, such factors can include at least one of the number, distribution, or density of immune cells within the item associated with cancer.

[0093] 5 is a diagram illustrating an example of generating an immunophenotyping result 540 according to an embodiment of the present disclosure. In one embodiment, a processor (e.g., at least one processor of an information processing system) can determine the immunophenotype of at least a portion of a first pathology slide image by inputting features for at least a portion of a first pathology slide image into an artificial neural network immunophenotyping model 530. Here, the artificial neural network immunophenotyping model 530 can correspond to a classifier that is trained to determine the immunophenotype of at least a portion of a first pathology slide image as one of immune activation, immune exclusion, or immune deficiency by inputting features for at least a portion of a first pathology slide image.

[0094] Here, the features for at least a portion of the region in the first pathology slide image may include statistical features for one or more target items in at least a portion of the region in the first pathology slide image (e.g., density or number of specific target items in at least a portion of the region in the first pathology slide image), geometric features for one or more target items (e.g., features including relative position information between specific target items, etc.), and / or image features corresponding to at least a portion of the region in the first pathology slide image (e.g., features extracted from a plurality of pixels included in at least a portion of the region in the first pathology slide image, image vectors corresponding to at least a portion of the region in the first pathology slide image), etc. Additionally or alternatively, the features for at least a portion of the region in the first pathology slide image may include features combining two or more of the statistical features for one or more target items in at least a portion of the region in the first pathology slide image, geometric features for one or more target items, or image features corresponding to at least a portion of the region in the first pathology slide image.

[0095] The immunophenotyping unit 230 of the information processing system can receive input of at least a partial region 510 in a first pathology slide image and determine the immunophenotype of the at least a partial region 510 in the first pathology slide image. Here, the at least a partial region 510 in the first pathology slide image input to the immunophenotyping unit 230 can include a detection result for a target item in the at least a partial region 510 in the first pathology slide image. As shown in the figure, the feature extraction unit 520 of the immunophenotyping unit 230 can receive the at least a partial region 510 in the first pathology slide image, extract features for each of the at least a partial region 510 in the first pathology slide image, and input the features to the artificial neural network immunophenotyping model 530. As a result, the artificial neural network immunophenotyping model 530 can determine the immunophenotype of each of the at least a partial region 510 in the first pathology slide image and output an immunophenotyping determination result 540.

[0096] 5 illustrates that the feature extraction unit 520 is included in the immunophenotyping unit 230, and that the immunophenotyping unit 230 receives at least a partial region 510 in the first pathology slide image, but this is not limiting. For example, the immunophenotyping unit 230 can receive features directly for each of the at least a partial region 510 in the first pathology slide image and input them into the artificial neural network immunophenotyping model 530 to output an immunophenotyping result 540.

[0097] While FIG. 5 illustrates the artificial neural network immunophenotyping model 530 receiving input of features for at least a portion of a region in a first pathology slide image, the artificial neural network immunophenotyping model 530 is not limited thereto and may be configured to receive and process input of at least a portion of a region in a first pathology slide image. Additionally or alternatively, the artificial neural network immunophenotyping model 530 may be configured to output not only the immunophenotyping result 540 but also at least one piece of information associated with the immunophenotype. In such a case, the artificial neural network immunophenotyping model 530 may be trained to determine the immunophenotype or at least one piece of information associated with the immunophenotype of at least a portion of a region in a reference pathology slide image by receiving input of features for at least a portion of a region in a reference pathology slide image or at least a portion of a region in a reference pathology slide image.

[0098] FIG. 6 illustrates an example of generating a response prediction result 630 to an immunological anti-cancer drug according to one embodiment of the present disclosure. The processor can determine an immunophenotype for each of a plurality of regions of interest in a pathology slide image (e.g., a first pathology slide image). The processor can then generate a prediction result as to whether a patient associated with the pathology slide image will respond to the immunological anti-cancer drug based on the immunophenotype of each of the plurality of regions of interest (i.e., the immunophenotype for each of the plurality of regions of interest). For example, the processor can generate a prediction result as to whether a patient associated with the pathology slide image will respond to the immunological anti-cancer drug based on the distribution of immunophenotypes, etc. in the pathology slide image.

[0099] In one embodiment, the processor can determine the most prevalent immunophenotype within the entire region of the pathology slide image based on the immunophenotypes of each of the multiple regions of interest. The processor can then generate a prediction result for whether a patient will respond to an immunotherapy based on the most prevalent immunophenotype within the entire region of the pathology slide image. For example, if the most prevalent immunophenotype within the entire region of the pathology slide image is immunoactive, the processor can predict that the patient associated with the pathology slide image will respond to an immunotherapy (i.e., the patient will be a responder). Alternatively, if the most prevalent immunophenotype within the entire region of the pathology slide image is immunoexclusion or immunodeficiency, the processor can predict that the patient associated with the pathology slide image will not respond to an immunotherapy (i.e., the patient will be a non-responder).

[0100] In another embodiment, the processor can generate an immunophenotype map 610 for at least a portion of a region in the first pathology slide image using the immunophenotype of at least a portion of the region in the first pathology slide image. For example, the processor can generate the immunophenotype map using a set generated by classifying multiple regions of interest based on the immunophenotype of at least a portion of the region in the first pathology slide image (e.g., each of multiple regions of interest). Here, the immunophenotype map can include an immunophenotype feature map, which can include not only the immunophenotype of the region of interest but also information associated therewith. For example, the processor can generate an immunophenotype feature map including information regarding the immunophenotype of at least a portion of the region in the first pathology slide image, an immunophenotype score (e.g., a score output from an artificial neural network immunophenotyping model), and the number, distribution, and density of immune cells in an item associated with cancer (e.g., the density of immune cells in a cancerous region or the density of immune cells in a cancer stroma).

[0101] As shown in the figure, the processor then inputs the generated immunophenotype map 610 into an immunological anticancer drug response prediction model 620, thereby generating a prediction result 630 regarding whether a patient will respond to the immunological anticancer drug. For example, by inputting the immunophenotype map 610 into the immunological anticancer drug response prediction model 620, the processor can classify a patient associated with the pathology slide image as a responder or a non-responder. In this case, the immunological anticancer drug response prediction model 620 can ultimately predict the patient's responsiveness by taking into account spatial information and / or positional information between immunophenotypes. For example, the immunological anticancer drug response prediction model 620 can aggregate at least one global feature of the pathology slide image based on the immunophenotype map. To this end, the immunological anticancer drug response prediction model 620 can be configured to form a graph or use various pooling methods (e.g., RNN, CNN, simple average pooling, sum pooling, max pooling, bag-of-words / VLAD / Fisher, etc.) based on the immunological phenotype map.

[0102] Here, the immunological anticancer drug response prediction model 620 may correspond to a statistical model and / or an artificial neural network model trained to generate a prediction result regarding whether or not a patient will respond to the reference immunological anticancer drug when a reference immunophenotype map is input. For example, a user (e.g., a doctor) may perform annotation work based on the actual treatment results of multiple patients to determine a label for each patient's pathology slide image as a responder or a non-responder. The processor may then train the immunological anticancer drug response prediction model 620 using each patient's pathology slide image (or the immunophenotype map for each patient's pathology slide image) and the label. As another example, the processor may train the immunological anticancer drug response prediction model 620 to classify an immunological phenotype graph as a responder or a non-responder when an immunological phenotype graph is input. To this end, the processor may convert and generate an immunological phenotype graph that represents or characterizes the immunological phenotype corresponding to a region of interest in the immunological phenotype map.

[0103] In another embodiment, the processor can generate an immunophenotype feature map for at least a portion of the first pathology slide image using information associated with the immunophenotype of at least a portion of the first pathology slide image. The processor can then input the generated immunophenotype feature map into an immunotherapy anticancer drug response prediction model 620 to generate a prediction result regarding whether a patient will respond to the immunotherapy anticancer drug. In this case, the immunotherapy anticancer drug response prediction model 620 can be trained to generate a reference prediction result by inputting a reference immunophenotype feature map.

[0104] FIG. 7 illustrates a graph showing the results of analyzing the predictive performance of biomarker expression information and / or immunophenotype information on immune-mediated anticancer drug response, according to one embodiment of the present disclosure. A processor (e.g., at least one processor in an information processing system) can acquire biomarker (e.g., PD-L1) expression information from pathology slide images associated with a patient (e.g., a first pathology slide image and a corresponding second pathology slide image). The processor can then generate a prediction result for whether the patient will respond to the immune-mediated anticancer drug based on the immunophenotype and PD-L1 expression information of at least a portion of the first pathology slide image. The PD-L1 expression information can include at least one of a tumor proportion score (TPS) or a combined proportion score (CPS). The processor can also acquire additional patient data, such as the patient's tumor mutation burden (TMB) score, microsatellite instability (MSI) score, and homologous recombination deficiency (HRD) score, to predict the patient's response to the immune-mediated anticancer drug.

[0105] In one embodiment, the processor receives a second pathology slide image (e.g., an IHC-stained pathology slide image) and inputs the second pathology slide image into the artificial neural network expression information generation model to generate information about PD-L1 expression. For example, the processor may use the artificial neural network expression information generation model to detect at least one of the location of tumor cells, lymphocytes, and macrophages, or the presence or absence of PD-L1 expression in at least a portion of the first pathology slide image included in the second pathology slide image, thereby generating information about PD-L1 expression. Here, the artificial neural network expression information generation model may correspond to a model trained to generate reference information about PD-L1 expression when a reference pathology slide image is input. In another embodiment, the processor may receive information about PD-L1 expression calculated directly by a user (e.g., a pathologist) from a pathology slide image (e.g., an IHC-stained pathology slide image).

[0106] For example, among the information regarding PD-L1 expression, TPS can be calculated using the following formula 1.

[0107]

number

[0108] Based on the calculated TPS value and the TPS reference value (cut-off value), patients can be classified into one or more groups. For example, for non-small cell lung cancer, patients with a TPS of <1% can be classified as a "PD-L1 non-expressing group," patients with a TPS of 1%≦TPS≦49% can be classified as a "PD-L1 expressing group," and patients with a TPS of ≥50% can be classified as a "high PD-L1 expressing group." The TPS reference values ​​are not limited to the above values ​​(1%, 49%, 50%) and can vary depending on the type of cancer, the type of PD-L1 antibody, etc.

[0109] As another example, the CPS, which is information about PD-L1 expression, can be calculated using the following formula 2. In this case, the maximum upper limit of the CPS is 100, and if the CPS calculated using formula 2 is greater than 100, the CPS can be determined to be 100.

[0110]

number

[0111] Based on the calculated CPS value and the reference value (cut-off value) of CPS, patients can be classified into one or more groups. For example, using 10% as the reference value (cut-off value) of CPS, patients can be classified into groups with CPS ≥ 10% and CPS < 10%, but this is not limiting, and the reference value (cut-off value) of CPS can vary depending on the type of cancer, the type of PD-L1 antibody, etc.

[0112] Table 710 shown compares the response of patients whose immunophenotype (i.e., representative immunophenotype) is classified as immune-active (inflamed in Table 710) with that of patients whose immunophenotype is classified as immune-inactive (not-inflamed in Table 710). The results show that the overall response rate (ORR) and median progression-free survival (mPFS) are improved in patients whose PD-L1 TPS is 1% to 49% and in patients whose TPS is 50% or higher. Therefore, the processor can use information on both the immunophenotype and PD-L1 expression to generate meaningful predictions of whether a patient will respond to an immune-mediated anticancer drug. In Table 710, N represents the number of patients, CI represents the confidence interval, and HR represents the hazard ratio.

[0113] Graph 720 also shows receiver operating characteristic (ROC) curves for specificity-sensitivity when predicting response to an immunotherapy (i.e., whether a patient will respond to an immunotherapy) using PD-L1 TPS (i.e., "PD-L1 TPS"), immunophenotyping (i.e., "H&E"), TMB (i.e., "TMB"), and all of the above information (i.e., "Ensemble of three"). According to graph 720, the AUROC (Area Unser ROC curve) for TMB was calculated to be 0.6404, the AUROC for PD-L1 TPS was calculated to be 0.7705, the AUROC for H&E was calculated to be 0.7719, and the AUROC for Ensemble of three was calculated to be 0.8874. In other words, it was demonstrated that the best performance was achieved when PD-L1 expression information (i.e., PD-L1 TPS and TMB) and immunophenotype (i.e., H&E) were combined to predict response to immunotherapy.

[0114] 8 is a graph showing the correlation between immunophenotypes and information on the expression of CD3-positive cells, CD8-positive cells, FOXP3-positive cells, and CD68-positive cells according to one embodiment of the present disclosure. A processor (e.g., at least one processor of an information processing system) can calculate an inflamed score (IS) based on the immunophenotype of each of a plurality of regions of interest in the pathology slide image. Here, the IS is calculated based on the immunophenotype of a region of interest in the pathology slide image that is determined to be immunoactive and that includes a tumor (e.g., a 1 mm thick region constituting the pathology slide image). 2 For example, the activity score can be calculated using the following formula:

[0115]

number

[0116] Using a cutoff activity score of 20%, the overall survival time after immunotherapy for patients with 10 cancers (N = 1,013) was compared. The results demonstrated that patients with activity scores above the cutoff value had a longer overall survival time after immunotherapy for not only lung cancer but also melanoma and head and neck cancer. Furthermore, whether lung cancer was included (N = 519) or excluded (N = 494), patients with higher activity scores tended to have a more favorable prognosis for immunotherapy.

[0117] In this study, for pathological verification of the immunophenotype, H&E stained pathological slide images of the patients were collected at 1 mm 2 The pathology slide image was divided into multiple patches, and the immunophenotype was determined based on the tumor-infiltrating lymphocytes (TILs) in the cancerous region and the cancer stroma TILs in each patch. Based on the immunophenotype of each of the multiple patches in the pathology slide image, a representative immunophenotype (i.e., the most prevalent immunophenotype in the pathology slide image) was determined. For example, if the activity score was 33.3% or higher, the representative immunophenotype of the pathology slide image could be determined as immunoactive.

[0118] Next, multiplex IHC staining was performed on non-small cell lung cancer (NSCLC) tissue treated with immunotherapy, and staining for the biomarkers CD3, CD8, CD20, CD68, FOXP3, CK, and DAPI was performed. The normalized cell count was calculated by dividing the total number of positive cells by the number of DAPI-positive cells in the pathology slide image. For example, the normalized CD3-positive cell count was calculated using the following formula:

[0119]

number

[0120] Graphs 810, 820, 830, and 840 show correlations between immunophenotypes determined from H&E-stained pathology slide images (i.e., representative immunophenotypes) and biomarker expression confirmed by multiplex IHC analysis, as described above. According to the first graph 810 and the second graph 820, normalized CD3-positive cells and CD8-positive cells, which play a role in anti-tumor activity, are highly abundant in pathology slide images corresponding to immune activity compared to other immunophenotypes (CD3: fold change (FC) = 1.57, P = 0.0182) (CD8: fold change (FC) = 1.24, P = 0.0697).

[0121] Conversely, according to the third graph 830, FOXP3-positive cells, which are associated with immunosuppressive activity, are abundant in pathology slide images corresponding to immune exclusion (FC=1.26, P=0.0656). Also, according to the fourth graph 840, CD68-positive cells are abundant in pathology slide images corresponding to immune deficiency (FC=1.76, P=0.00467). This allows the processor to generate meaningful predictions of whether a patient will respond to an immunotherapy anticancer drug based on the immune phenotype determined from the H&E-stained pathology slide images.

[0122] 9 shows graphs 910 and 920 illustrating the performance of various methods for predicting whether a patient will respond to an immunological anti-cancer drug according to one embodiment of the present disclosure. To generate a prediction result regarding whether a patient will respond to an immunological anti-cancer drug, a processor (e.g., at least one processor of an information processing system) can determine an immunophenotype for a first pathology slide image (e.g., an H&E-stained pathology slide image) of the patient according to the above-described embodiments. Here, the immunophenotype for the pathology slide image can refer to the immunophenotype of one or more regions of interest within the pathology slide image, the immunophenotype most frequently contained within the pathology slide image (i.e., a representative immunophenotype), an immunophenotype map, etc.

[0123] Additionally or alternatively, to generate a prediction result regarding whether a patient will respond to an immunotherapy, the processor can obtain information regarding PD-L1 expression (e.g., TPS or CPS) from a second pathology slide image (e.g., an IHC-stained pathology slide image) associated with the patient. For example, the information regarding PD-L1 expression can correspond to information calculated directly by a user. As another example, the information regarding PD-L1 expression can correspond to information generated by the processor from the second pathology slide image of the patient using an artificial neural network expression information generation model.

[0124] In one embodiment, the processor receives a second pathology slide image and inputs the second pathology slide image into the artificial neural network expression information generation model to generate information about PD-L1 expression. Additionally or alternatively, the processor can use the artificial neural network expression information generation model to detect at least one of the location of tumor cells, lymphocytes, and macrophages, or the presence or absence of PD-L1 expression in at least a portion of the first pathology slide image included in the second pathology slide image, thereby generating information about PD-L1 expression. Here, the second pathology slide image can include a pathology slide image of the same region of the same patient as the first pathology slide image. For example, the processor can remove an in-house control tissue region from the IHC-stained pathology slide image, divide the patient's tissue region into one or more regions of interest, and use the artificial neural network expression information generation model to detect the locations of tumor cells, lymphocytes, and macrophages and the presence or absence of PD-L1 expression (e.g., PD-L1 negative, PD-L1 positive) from the ROIs to calculate the TPS and / or CPS.

[0125] Here, the artificial neural network expression information generation model may be a model trained to generate reference information regarding PD-L1 expression by inputting reference pathology slide images. To generate / train the artificial neural network expression information generation model, the processor may receive a plurality of training pathology slide images and label information (or annotation information) regarding the plurality of training pathology slide images. Here, the label information regarding the training pathology slide images may be generated by a user performing an annotation task.

[0126] The illustrated ROC curve graph 910 shows ROC curves for specificity-sensitivity when predicting response to an immunotherapy anticancer drug using a TPS calculated using an artificial neural network expression information generation model (i.e., "AI PD-L1 TPS"), a TPS calculated manually (i.e., "Human PD-L1 TPS"), and a TMB (i.e., "Tumor Mutation Burden"). In the ROC curve graph 910, the AUROC for the AI ​​PD-L1 TPS was calculated to be 0.784, the AUROC for the Human PD-L1 TPS was calculated to be 0.764, and the AUROC for the TMB was calculated to be 0.710. This demonstrates that the AI ​​PD-L1 TPS has the best performance in terms of specificity-sensitivity when predicting response to an immunotherapy anticancer drug.

[0127] In one embodiment, the processor may generate a prediction result regarding whether a patient will respond to an immunotherapy based on information regarding the immunophenotype and PD-L1 expression of at least a portion of a region in a first pathology slide image. Specifically, when predicting whether a patient will respond to an immunotherapy, using both the immunophenotype and information regarding PD-L1 expression provides better performance than using either the immunophenotype determined through an H&E-stained pathology slide image (e.g., a first pathology slide image) or information regarding PD-L1 expression obtained from an IHC-stained pathology slide image (e.g., a second pathology slide image) separately. Here, the immunophenotype determined through an H&E-stained pathology slide image may be referred to as a representative immunophenotype of the pathology slide image, an immunophenotype of one or more regions of interest, an immunophenotype map, etc.

[0128] According to the illustrated Comparison of Accuracy Rates of AI-powered Biomarkers graph 920, when predicting response to immune-based anticancer drugs, the accuracy of using a PD-L1 TPS calculated directly by a human ("Human PD-L1 TPS") is approximately 0.68, the accuracy of using TMB is approximately 0.69, the accuracy of using immunophenotypes ("AI H&E") is approximately 0.715, and the accuracy of using a TPS calculated using an artificial neural network expression information generation model ("AI PD-L1 TPS") is approximately 0.72. Additionally, when Human PD-L1 TPS and AI PD-L1 TPS were used together ("Human PD-L1 TPS + AI PD-L1 TPS"), the accuracy was approximately 0.715, when TMB and AI PD-L1 TPS were used together ("TMB + AI PD-L1 TPS"), the accuracy was approximately 0.72, and when AI H&E and AI PD-L1 TPS were used together ("AI H&E + AI PD-L1 TPS"), the accuracy was approximately 0.77. This demonstrates that AI H&E + AI PD-L1 TPS has the best accuracy when predicting response to immune-mediated anti-cancer drugs.

[0129] FIG. 10 is a diagram illustrating an example of outputting results generated during a process for predicting whether or not a subject will respond to an immunotherapy anticancer drug, according to an embodiment of the present disclosure. In one embodiment, a processor (e.g., at least one processor of an information processing system terminal and / or at least one processor of a user terminal) can output at least one of the following: detection results for one or more target items; an immunophenotype of at least a portion of a first pathology slide image; information associated with the immunophenotype; a prediction result for whether or not a subject will respond to an immunotherapy anticancer drug; or the density of immune cells within at least a portion of a first pathology slide image. The information processing system (e.g., at least one processor of the information processing system) provides the results generated during the process for predicting whether or not a subject will respond to an immunotherapy anticancer drug to the user terminal, allowing the user terminal to output the received results. For example, the user terminal can display the results, etc., on a display via a user interface, as shown in the figure.

[0130] In one embodiment, the user terminal can output a target item detection result 1010 for at least a partial region in the pathology slide image and / or the first pathology slide image. For example, the user terminal can output a pathology slide image including label information for each target item. The user terminal can output a pathology slide image in which target items in units of regions are displayed using a mask, and in which target items in units of cells are displayed using the center point or bounding box of a cell nucleus. In another embodiment, the user terminal can visualize and output an immunophenotype map (or immunophenotype feature map) for the entire pathology slide image or at least a partial region in the first pathology slide image using a representation method such as a minimap, a heatmap, and / or a label map. In yet another embodiment, the user terminal can visualize and output a response / non-response score map using a representation method such as a hit map and / or a label map based on the immune phenotype, activity score, and response / non-response score (respond score and / or non-respond score) as a prediction result of the response or non-response of an immune anticancer drug to at least a portion of the first pathology slide image.

[0131] In another embodiment, the user terminal may output the density of immune cells by region for the entire pathology slide image and / or a partial region. For example, the user terminal may output a numerical value or a bar graph for the density of immune cells by region for the entire pathology slide image and / or a partial region. In another embodiment, the user terminal may output the distribution of the patient's immune phenotype represented in the form of a circle plot. As shown in the figure, the user terminal may output an analysis result 1020 including a bar graph for the density of immune cells by region and a pie chart for the distribution of the patient's immune phenotype.

[0132] The user terminal can receive results generated in the process of predicting whether or not a patient will respond to an immunosuppressant from the information processing system and output the received results. This allows the user to visually and intuitively recognize the results generated in the process of predicting whether or not a patient will respond to an immunosuppressant. In FIG. 10, the results generated in the prediction process are visually output through a user interface operated on the user terminal, but the results can be provided to the user in various ways, without being limited thereto.

[0133] 11 to 15 are diagrams illustrating examples of outputting results generated in a process for predicting whether a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. In one embodiment, an information processing system (e.g., at least one processor of the information processing system) can generate and provide reports 1100, 1200, 1300, 1400, and 1500 including results generated in a process for predicting whether a patient will respond to an immunosuppressant. The generated reports 1100, 1200, 1300, 1400, and 1500 can be provided as files, data, text, images, and the like in a format that can be output via a user terminal and / or an output device. Here, the reports 1100, 1200, 1300, 1400, and 1500 can include at least one of the results, etc., described in FIG. 10.

[0134] In one embodiment, the information processing system can generate a report including a score (e.g., a score between 0 and 1) for the patient's final responder / non-responder status (e.g., a result indicating the probability that the patient is a responder and / or a non-responder). Additionally or alternatively, the information processing system can generate a report including information on cut-off values ​​for determining responder / non-responder status. Additionally or alternatively, the information processing system can generate a report including the distribution of immunophenotypes and / or TIL density (e.g., Min, Max, Avg values) within a pathology slide image. For example, the information processing system can generate a report including the distribution of TIL density and Min, Max, and Avg values ​​for each region of interest classified into three immunophenotypes. Additionally or alternatively, the information processing system can generate a report including an immunophenotype map in which regions of interest within a pathology slide image are classified into three immunophenotypes.

[0135] The information processing system can perform some embodiments of the present disclosure on pathology images (e.g., pathology slide images) taken before and / or after immunotherapy with an anticancer drug to identify acquired resistance mechanisms and provide treatment strategies tailored to each resistance mechanism. For example, the processor can predict treatment outcomes for each immunotherapy and / or other immunotherapy administered to the patient by performing analysis using input data such as pathology images of a patient who underwent immunotherapy with an anticancer drug and the type of therapeutic agent administered to the patient. In one embodiment, the information processing system can output information regarding at least one immunotherapy that is suitable for the patient among multiple immunotherapy agents based on the prediction result regarding whether the patient will respond to the immunotherapy. For example, if the patient is determined to be a responder, the information processing system can generate a report including immunotherapy products and / or product combinations that are likely to result in a response.

[0136] As shown in reports 1100, 1200, 1300, 1400, and 1500, results generated during the process of predicting response to an immunotherapy anticancer drug can be provided to a user in the form of a document containing text and / or images. For example, as shown in FIG. 11, report 1100 can include a pathology slide image, patient information, basic information, and / or predicted results (responder / non-responder status). As shown in FIG. 12, report 1200 can include a graph showing the immunophenotype ratio, numerical values, and information on TIL density (e.g., density distribution, etc.). As shown in FIG. 13, report 1300 can include statistical results (e.g., TCGA PAN-CARCINOMA STATISTICS) and / or graphs showing analysis results, clinical notes, etc. As shown in FIG. 14, report 1400 can include information on references (e.g., academic references), etc. As shown in FIG. 15, the report 1500 may include results generated in the prediction process (e.g., immunophenotype map images, FEATURE STATISTICS, etc.) and / or information used in the prediction process.

[0137] In FIGS. 11 to 15, the results generated in the prediction process are output in the form of a report, but the results are not limited to this and can be provided to the user in various ways.

[0138] 16 is a diagram illustrating an example of an artificial neural network model 1600 according to an embodiment of the present disclosure. The artificial neural network model 1600 is an example of a machine learning model, which is a statistical learning algorithm implemented based on the structure of a biological neural network in machine learning technology and cognitive science, or a structure for executing the algorithm.

[0139] According to one embodiment, the artificial neural network model 1600 may represent a machine learning model having problem-solving capabilities, in which nodes, which are artificial neurons forming a network through synaptic connections like a biological neural network, repeatedly adjust synaptic weights to learn to reduce the error between a normal output and an inferred output corresponding to a specific input. For example, the artificial neural network model 1600 may include any probability model, neural network model, etc. used in artificial intelligence learning methods such as machine learning and deep learning.

[0140] According to one embodiment, the artificial neural network model 1600 may include an artificial neural network model configured to detect one or more items of interest from an input pathology slide image. Additionally or alternatively, the artificial neural network model 1600 may include an artificial neural network model configured to determine at least one of an immunophenotype or information associated with an immunophenotype of at least a portion of a first pathology slide image based on features for at least a portion of a first pathology slide image or at least a portion of a first pathology slide image. Additionally or alternatively, the artificial neural network model 1600 may include an artificial neural network model configured to generate a prediction result for whether a patient will respond to an immunotherapy or anti-cancer drug based on an input immunophenotype feature map. Additionally or alternatively, the artificial neural network model 1600 may include an artificial neural network model configured to generate a prediction result for whether a patient will respond to an immunotherapy or anti-cancer drug based on an input immunophenotype feature map.

[0141] Additionally or alternatively, the artificial neural network model 1600 may include an artificial neural network model configured to generate information regarding biomarker expression (e.g., PD-L1 expression) from input pathology slide images.

[0142] The artificial neural network model 1600 is embodied as a multilayer perceptron (MLP) composed of multiple nodes and connections between them. The artificial neural network model 1600 according to this embodiment may be embodied using one of various artificial neural network model structures, including an MLP. As shown in FIG. 16, the artificial neural network model 1600 includes an input layer 1620 that receives an input signal or data 1610 from the outside, an output layer 1640 that outputs an output signal or data 1650 corresponding to the input data, and n hidden layers 1630_1 through 1630_n (where n is a positive integer) positioned between the input layer 1620 and the output layer 1640. The hidden layers 1630_1 through 1630_n receive signals from the hidden layers 1630_1 through 1630_n and output the signals to the outside.

[0143] The training method of the artificial neural network model 1600 includes a supervised learning method, in which the model learns to optimize problem solving in response to the input of a supervised signal (correct answer), and an unsupervised learning method, in which no supervised signal is required. In one embodiment, the information processing system can train the artificial neural network model 1600 by supervised learning and / or unsupervised learning to detect one or more target items from a pathology slide image. For example, the information processing system can train the artificial neural network model 1600 by supervised learning to detect one or more target items from a pathology slide image using label information for a reference pathology slide image and one or more reference target items. In another embodiment, the information processing system can train the artificial neural network model 1600 by supervised learning and / or unsupervised learning to determine at least one of an immunophenotype or information associated with the immunophenotype of at least a portion of a first pathology slide image based on features for at least a portion of a first pathology slide image or at least a portion of a first pathology slide image. For example, the information processing system can train the artificial neural network model 1600 through supervised learning to determine the immunophenotype or at least one of the information associated with the immunophenotype of at least a portion of the region in the reference pathology slide image based on the features for at least a portion of the region in the reference pathology slide image or at least one of the information associated with the immunophenotype of at least a portion of the region in the reference pathology slide image, using label information relating to the features for at least a portion of the region in the reference pathology slide image and at least one of the immunophenotypes or information associated with the immunophenotype of at least a portion of the region in the reference pathology slide image.

[0144] In yet another embodiment, the information processing system can train the artificial neural network model 1600 by supervised learning and / or unsupervised learning to generate a prediction result for whether a patient will respond to an immunological anti-cancer drug based on the immunophenotype map or the immunophenotype feature map. For example, the information processing system can train the artificial neural network model 1600 by supervised learning, using a reference immunophenotype map (or reference immunophenotype feature map) and label information related to the reference prediction result, to generate a prediction result for whether a patient will respond to an immunological anti-cancer drug based on the immunophenotype map (or immunophenotype feature map). In yet another embodiment, the information processing system can train the artificial neural network model 1600 by supervised learning and / or unsupervised learning to generate information related to PD-L1 expression from pathology slide images. For example, the information processing system can train the artificial neural network model 1600 by supervised learning, using label information related to the reference pathology slide images and reference information related to PD-L1 expression, to generate information related to PD-L1 expression from pathology slide images.

[0145] The artificial neural network model 1600 thus trained can be stored in the information processing system's memory (not shown) and can detect one or more items of interest within a pathology slide image in response to inputs for the pathology slide image received from the communication module and / or memory. Additionally or alternatively, the artificial neural network model 1600 can determine an immunophenotype or information associated with an immunophenotype of at least a portion of a first pathology slide image in response to inputs for features for each of at least a portion of a first pathology slide image or at least a portion of a first pathology slide image. Additionally or alternatively, the artificial neural network model 1600 can generate a prediction of whether a patient will respond to an immunological anti-cancer drug in response to inputs for the immunophenotype map or immunophenotype feature map. Additionally or alternatively, the artificial neural network model 1600 can generate information regarding biomarker expression in response to inputs for the pathology slide image received from the communication module and / or memory.

[0146] According to one embodiment, the input variables of an artificial neural network model for detecting items of interest and / or generating information about biomarker expression (e.g., PD-L1 expression) may be one or more pathology slide images (e.g., H&E-stained pathology slide images, IHC-stained pathology slide images). For example, the input variable input to input layer 1620 of artificial neural network model 1600 may be image vector 1610, which is a vector data element that represents one or more pathology slide images. In response to the image input, output variable output by output layer 1640 of artificial neural network model 1600 may be vector 1650 that represents or characterizes one or more items of interest detected from the pathology slide images. Additionally or alternatively, the output variable output by output layer 1640 of artificial neural network model 1600 may be vector 1650 that represents or characterizes information about biomarker expression generated from the pathology slide images. That is, the output layer 1640 of the artificial neural network model 1600 can be configured to output a vector representing or characterizing information regarding the expression of one or more target items detected from the pathology slide image and / or biomarkers generated from the pathology slide image. In the present disclosure, the output variables of the artificial neural network model 1600 are not limited to the types described above, but may include any information / data indicating information regarding the expression of one or more target items detected from the pathology slide image and / or biomarkers generated from the pathology slide image.

[0147] In another embodiment, the input variables of the machine learning model, i.e., artificial neural network model 1600, which determines the immunophenotype of at least a portion of a region in a reference pathology slide image, may be features for at least a portion of a region in a first pathology slide image or at least a portion of a region in the first pathology slide image. For example, the input variables input to the input layer 1620 of the artificial neural network model 1600 may be a numerical vector 1610, which is a vector data element that represents the features for at least a portion of a region in the first pathology slide image or at least a portion of a region in the first pathology slide image. In response to such input, the output variables output by the output layer 1640 of the artificial neural network model 1600 may be a vector 1650 that represents or characterizes the immunophenotype or information associated with the immunophenotype of each of the at least a portion of a region in the first pathology slide image. In the present disclosure, the output variables of the artificial neural network model 1600 are not limited to the types described above and may include any information / data that represents or characterizes the immunophenotype of each of the at least a portion of a region in the first pathology slide image. In another embodiment, the input variables of the machine learning model, i.e., artificial neural network model 1600, which generates a prediction result on whether a patient will respond to an immunological anti-cancer drug, may be an immunophenotype map or an immunophenotype feature map. For example, the input variables input to the input layer 1620 of the artificial neural network model 1600 may be a numerical vector 1610 and / or image data in which the immunophenotype map or the immunophenotype feature map is configured as a single vector data element. In response to the input of the immunophenotype map or the immunophenotype feature map, the output variables output from the output layer 1640 of the artificial neural network model 1600 may be a vector 1650 that represents or characterizes the prediction result on whether a patient will respond to an immunological anti-cancer drug. In the present disclosure, the output variables of the artificial neural network model 1600 are not limited to the types described above and may include any information / data that indicates the prediction result on whether a patient will respond to an immunological anti-cancer drug. Additionally, the output layer 1640 of the artificial neural network model 1600 can be configured to output a vector indicating the confidence and / or accuracy of the output prediction result or the like.

[0148] In this way, the input layer 1620 and output layer 1640 of the artificial neural network model 1600 are matched with a plurality of input variables and a plurality of corresponding output variables, and the synaptic values ​​between the nodes included in the input layer 1620, hidden layers 1630_1 through 1630_n, and output layer 1640 are adjusted, thereby learning to extract a normal output corresponding to a specific input. Through this learning process, characteristics hidden in the input variables of the artificial neural network model 1600 can be identified, and the synaptic values ​​(or weights) between the nodes of the artificial neural network model 1600 can be adjusted to reduce the error between the output variables calculated based on the input variables and the target output. The artificial neural network model 1600 trained in this way can be used to output target item detection results and / or information regarding PD-L1 expression in response to an input pathology slide image. Additionally or alternatively, at least one of an immunophenotype or information associated with an immunophenotype of each of one or more regions of interest can be output in response to features for at least a portion of the region in one or more input reference pathology slide images or at least a portion of the region in the reference pathology slide image using the artificial neural network model 1600. Additionally or alternatively, the artificial neural network model 1600 can be used to output a prediction result as to whether a patient will respond to an immunotherapy anti-cancer drug in response to the input immunophenotype map.

[0149] 17 is a block diagram of an exemplary system 100 for providing response prediction results to an immunological anticancer drug according to one embodiment of the present disclosure. As shown in the figure, the information processing system 100 may include one or more processors 1710, a bus 1730, a communication interface 1740, a memory 1720 for loading a computer program 1760 executed by the processor 1710, and a storage module 1750 for storing the computer program 1760. However, FIG. 17 shows only components associated with the embodiment of the present disclosure. Therefore, a person of ordinary skill in the art to which the present disclosure pertains will understand that other general-purpose components may be included in addition to the components shown in FIG. 17.

[0150] The processor 1710 controls the overall operation of each component of the information processing system 100. The processor 1710 may include a central processing unit (CPU), a microprocessor unit (MPU), a microcontroller unit (MCU), a graphic processing unit (GPU), or any other type of processor known in the art. The processor 1710 may also perform calculations for at least one application or program for executing a method according to an embodiment of the present disclosure. The information processing system 100 may include one or more processors.

[0151] The memory 1720 may store various data, instructions, and / or information. The memory 1720 may load one or more computer programs 1760 from the storage module 1750 to perform the methods / operations according to various embodiments of the present disclosure. The memory 1720 may be embodied as a volatile memory such as RAM, although the scope of the present disclosure is not limited in this respect.

[0152] The bus 1730 can provide a communication function between components of the information processing system 100. The bus 1730 can be implemented as various types of buses such as an address bus, a data bus, and a control bus.

[0153] The communication interface 1740 can support wired and wireless Internet communication for the information processing system 100. The communication interface 1740 can also support various communication methods other than Internet communication. To this end, the communication interface 1740 can be configured to include a communication module known in the art.

[0154] The storage module 1750 can non-temporarily store one or more computer programs 1760. The storage module 1750 can include non-volatile memory such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a hard disk, a removable disk, or any other form of computer-readable storage medium known in the art.

[0155] The computer program 1760 may include one or more instructions that, when loaded into the memory 1720, cause the processor 1710 to perform operations / methods according to various embodiments of the present disclosure. That is, the processor 1710 can perform operations / methods according to various embodiments of the present disclosure by executing the one or more instructions.

[0156] For example, the computer program 1760 may include one or more instructions for performing operations such as receiving a first pathology slide image, detecting one or more target items in the first pathology slide image, determining an immunophenotype or at least one of information associated with the immunophenotype of at least a portion of the first pathology slide image based on the detection results for the one or more target items, and generating a prediction result regarding whether a patient associated with the first pathology slide image will respond to an immunotherapy anticancer drug based on the immunophenotype or at least one of information associated with the immunophenotype of at least a portion of the first pathology slide image. In this case, a system for predicting a response to an immunotherapy anticancer drug according to some embodiments of the present disclosure may be embodied through the information processing system 100.

[0157] The previous description of the present disclosure is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications of the present disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to various modifications without departing from the spirit or scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the examples described herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0158] Although example embodiments may be referenced as utilizing aspects of the presently disclosed subject matter in the context of one or more stand-alone computer systems, the present subject matter is not so limited and may be implemented in connection with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the presently disclosed subject matter may be implemented on or across multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include PCs, network servers, and handheld devices.

[0159] Although the present disclosure has been described in connection with some embodiments herein, it will be understood that various modifications and changes that would be understood by those of ordinary skill in the art to which the present invention pertains can be made without departing from the scope of the present disclosure, and such modifications and changes should be understood to fall within the scope of the claims appended hereto.

Claims

1. 1. A method for predicting a response to an immunological anti-cancer drug, performed by at least one computer device, comprising: receiving a first pathology slide image; detecting one or more items of interest within the first pathology slide image; determining at least one of an immunophenotype or information associated with an immunophenotype of each of a plurality of regions of interest within the first pathology slide image based on the detection results for the one or more items of interest; and generating a prediction result as to whether a patient associated with the first pathology slide image will respond to an immunological anti-cancer drug based on at least one of the immunophenotypes of each of a plurality of regions of interest in the first pathology slide image or information associated with the immunophenotypes; The detecting step includes: detecting immune cells as cell-level target items in the first pathology slide image using an artificial neural network target item detection model; and detecting cancer regions and cancer stroma as region-based target items in the first pathology slide image using an artificial neural network target item detection model; The determining step includes:

10. A method for predicting a response to an immune anti-cancer drug, comprising determining an immune phenotype or at least one of information associated with the immune phenotype of each of the plurality of regions of interest based on at least one of the density of immune cells in the cancer region or the density of immune cells in the cancer stroma.

2. A method for predicting response to an immune anticancer drug as described in claim 1, wherein the artificial neural network target item detection model is trained to detect one or more reference target items from a reference pathology slide image.

3. each of the plurality of regions of interest includes the one or more items of interest, the one or more items of interest including an item associated with cancer and an immune cell; The determining step includes: calculating at least one of the number, distribution, or density of the immune cells within an item associated with the cancer within each of the plurality of regions of interest; The method for predicting a response to an immune anticancer drug described in claim 1, comprising a step of determining at least one of the immune phenotypes of each of the plurality of regions of interest or information associated with the immune phenotypes based on at least one of the calculated number, distribution, or density of immune cells.

4. the items associated with the cancer include the cancer region and the cancer stroma; The calculating step calculating a density of the immune cells in the cancer region within each of the plurality of regions of interest; and calculating a density of the immune cells in the cancer stroma within each of the plurality of regions of interest.

5. If the density of the immune cells in the cancer region in each one of the plurality of regions of interest is equal to or greater than a first threshold density, the immune phenotype of each one of the plurality of regions of interest is determined as immune active; If the density of the immune cells in the cancer region of one of the plurality of respective regions of interest is less than the first threshold density and simultaneously the density of the immune cells in the cancer stroma is equal to or greater than a second threshold density, the immune phenotype of the one of the plurality of respective regions of interest is determined as immune exclusion; 5. The method of claim 4, wherein if the density of the immune cells in the cancer region in each of the plurality of regions of interest is less than the first threshold density and at the same time the density of the immune cells in the cancer stroma is less than the second threshold density, the immune phenotype of each of the plurality of regions of interest is determined as immune deficient.

6. the first threshold density is determined based on a distribution of the density of the immune cells in the cancer region in each of a plurality of regions of interest in a plurality of pathology slide images; The method for predicting a response to an immune anticancer drug according to claim 5, wherein the second threshold density is determined based on the distribution of the density of the immune cells within the cancer stroma in each of a plurality of regions of interest within the plurality of pathology slide images.

7. The items associated with the cancer include the cancer region and the cancer stroma; The determining step includes: The method for predicting a response to an immune anticancer drug according to claim 3, further comprising determining the immune phenotype of each of the plurality of regions of interest as one of immune activity, immune exclusion, or immune deficiency based on the number of immune cells contained within a specific region within the cancer region.

8. The determining step includes: determining at least one of an immunophenotype or information associated with an immunophenotype for each of the plurality of regions of interest by inputting features for each of the plurality of regions of interest or each of the plurality of regions of interest into an artificial neural network immunophenotyping model; The method for predicting response to an immune anticancer drug described in claim 1, wherein the artificial neural network immune phenotyping classification model is trained to determine at least one of the immune phenotypes of each of the multiple regions of interest in the reference pathology slide image or information associated with the immune phenotypes by inputting features for each of the multiple regions of interest in the reference pathology slide image or each of the multiple regions of interest in the reference pathology slide image.

9. The method for predicting a response to an immune anticancer drug described in claim 8, wherein the features for each of the multiple regions of interest in the first pathology slide image include at least one of statistical features for the one or more target items of each of the multiple regions of interest in the first pathology slide image, geometric features for the one or more target items, or image features corresponding to each of the multiple regions of interest in the first pathology slide image.

10. The generating step comprises: determining a most abundant immunophenotype within the entire area of ​​the first pathology slide image based on the immunophenotype of each of the plurality of regions of interest; The method for predicting response to an immune anti-cancer drug according to claim 1, further comprising: generating a prediction result as to whether the patient will respond to the immune anti-cancer drug based on the immune phenotype most frequently contained within the entire area of ​​the first pathology slide image.

11. The generating step includes: generating an immunophenotype map for the first pathology slide image using the immunophenotypes of each of the plurality of regions of interest; and generating a prediction result as to whether the patient will respond to the immune anticancer drug by inputting the generated immune phenotype map into an immune anticancer drug response prediction model, The method for predicting a response to an immune anticancer drug according to claim 1 , wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype map.

12. The generating step includes: generating an immunophenotype feature map for the first pathology slide image using information associated with the immunophenotype of each of the plurality of regions of interest; and generating a prediction result as to whether the patient will respond to the immune anticancer drug by inputting the generated immune phenotype feature map into an immune anticancer drug response prediction model; The method for predicting a response to an immune anticancer drug according to claim 1 , wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype feature map.

13. obtaining information regarding biomarker expression from a second pathology slide image associated with the patient; The generating step includes: The method for predicting a response to an immune anti-cancer drug according to claim 1, comprising a step of generating a prediction result as to whether the patient will respond to the immune anti-cancer drug based on at least one of the immune phenotypes of each of the plurality of regions of interest or information associated with the immune phenotypes, and information regarding the expression of the biomarkers.

14. the biomarker is PD-L1; The obtaining step includes: receiving the second pathology slide image; and generating information about the expression of PD-L1 by inputting the second pathological slide image into an artificial neural network expression information generation model; The method for predicting a response to an immunological anticancer drug according to claim 13, wherein the artificial neural network expression information generation model is trained to generate reference information regarding PD-L1 expression by inputting reference pathology slide images.

15. The method for predicting a response to an immune anti-cancer drug according to claim 1, further comprising the step of outputting at least one of the detection results for the one or more target items, the immune phenotype of each of the plurality of regions of interest, information associated with the immune phenotype, a prediction result as to whether the patient will respond to the immune anti-cancer drug, or the density of immune cells within each of the plurality of regions of interest.

16. The method for predicting a response to an immune anticancer drug according to claim 1, further comprising a step of outputting information regarding at least one immune anticancer drug that is suitable for the patient from among a plurality of immune anticancer drugs, based on a prediction result as to whether the patient will respond to the immune anticancer drug.

17. A method for predicting response to an immune anticancer drug as described in claim 1, wherein the step of determining at least one of the immune phenotypes or information associated with the immune phenotype further includes a step of determining the multiple regions of interest by dividing the first pathology slide image.

18. A method for predicting response to an immune anticancer drug as described in claim 1, wherein the step of generating the prediction result includes a step of generating a prediction result as to whether the patient will respond to the immune anticancer drug based on the ratio of areas of interest determined to be immune active among the plurality of areas of interest.

19. A computer program stored on a computer-readable recording medium for executing the method for predicting a response to an immunological anticancer agent according to any one of claims 1 to 18.

20. An information processing system, a memory for storing one or more instructions; Executing the one or more stored instructions, a processor configured to receive a first pathology slide image, detect one or more items of interest in the first pathology slide image, determine an immunophenotype or information associated with the immunophenotype for each of a plurality of regions of interest in the first pathology slide image based on the detection results for the one or more items of interest, and generate a prediction result for whether or not a patient associated with the first pathology slide image will respond to an immunological anti-cancer drug based on the immunophenotype or information associated with the immunophenotype for each of the plurality of regions of interest, The processor: detecting immune cells as cell-level target items in the first pathology slide image using an artificial neural network target item detection model; Detecting cancer areas and cancer stroma as region-based target items in the first pathology slide image using an artificial neural network target item detection model; The information processing system is further configured to determine at least one of an immune phenotype or information associated with the immune phenotype of each of the plurality of regions of interest based on at least one of the density of the immune cells in the cancer region or the density of the immune cells in the cancer stroma.

21. The information processing system of claim 20, wherein the artificial neural network target item detection model is trained to detect one or more reference target items from a reference pathology slide image.

22. each of the plurality of regions of interest includes the one or more items of interest, the one or more items of interest including an item associated with cancer and an immune cell; The processor:

21. The information processing system of claim 20, further configured to calculate, within each of the plurality of regions of interest, at least one of the number, distribution, or density of the immune cells in the item associated with the cancer, and determine an immune phenotype or at least one of information associated with the immune phenotype for each of the plurality of regions of interest based on the calculated at least one of the number, distribution, or density of the immune cells.

23. the items associated with the cancer include the cancer region and the cancer stroma; The processor:

23. The information processing system of claim 22, further configured to calculate, within each of the plurality of regions of interest, a density of the immune cells within the cancer region, and to calculate, within each of the plurality of regions of interest, a density of the immune cells within the cancer stroma.

24. The method of claim 24, wherein if the density of the immune cells in the cancer region in each one of the plurality of regions of interest is equal to or greater than a first threshold density, the immune phenotype of each one of the plurality of regions of interest is determined as immune active; If the density of the immune cells in the cancer region of one of the plurality of respective regions of interest is less than the first threshold density and simultaneously the density of the immune cells in the cancer stroma is equal to or greater than a second threshold density, the immune phenotype of the one of the plurality of respective regions of interest is determined as immune exclusion; 24. The information processing system of claim 23, wherein if the density of the immune cells in the cancer region in each of the plurality of regions of interest is less than the first threshold density and at the same time the density of the immune cells in the cancer stroma is less than the second threshold density, the immune phenotype of each of the plurality of regions of interest is determined as immune deficient.

25. The items associated with the cancer include the cancer region and the cancer stroma; The processor: The information processing system of claim 22, further configured to determine the immune phenotype of each of the plurality of regions of interest as one of immune activation, immune exclusion, or immune deficiency based on the number of immune cells contained within a specific region within the cancer region.

26. The processor, The information processing system of claim 20, further configured to determine the most prevalent immunophenotype within the entire area of ​​the first pathology slide image based on the immunophenotype of each of the multiple regions of interest, and to generate a prediction result regarding whether the patient will respond to an immunological anticancer drug based on the most prevalent immunophenotype within the entire area of ​​the first pathology slide image.

27. The processor: generating an immunophenotype map for the first pathology slide image using the immunophenotypes of each of the plurality of regions of interest; The method is further configured to input the generated immune phenotype map into an immune anticancer drug response prediction model to generate a prediction result on whether the patient will respond to the immune anticancer drug; The immune anticancer drug response prediction model is configured to obtain a reference predictor by inputting a reference immune phenotype map.

21. The information processing system of claim 20, trained to generate a measurement.

28. The processor: The method is further configured to generate an immunophenotype feature map for the first pathology slide image using information associated with each immunophenotype of the plurality of regions of interest, and input the generated immunophenotype feature map into an immunological anticancer drug response prediction model to generate a prediction result as to whether the patient will respond to the immunological anticancer drug; The information processing system according to claim 20 , wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype feature map.

29. The processor: The information processing system of claim 20, further configured to acquire information regarding expression of biomarkers from a second pathology slide image associated with the patient, and generate a prediction result regarding whether the patient will respond to an immunological anticancer drug based on the immunophenotype of each of the plurality of regions of interest and the information regarding the expression of the biomarkers.

30. the biomarker is PD-L1; The processor: receiving the second pathology slide image and inputting the second pathology slide image into an artificial neural network expression information generation model to generate information about the expression of PD-L1; The information processing system of claim 29, wherein the artificial neural network expression information generation model is trained to generate reference information regarding PD-L1 expression by inputting a reference pathology slide image.

31. The processor: The information processing system of claim 20, further configured to output at least one of the detection results for the one or more target items, the immune phenotype of each of the plurality of regions of interest, information associated with the immune phenotype, a prediction result for whether the patient will respond to an immune anti-cancer drug, or the density of immune cells within each of the plurality of regions of interest.

32. The processor: Based on the predicted results of whether the patient will respond to the immunotherapy, and outputting information about at least one immunological anticancer drug suitable for the patient.

21. The information processing system of claim 20, further configured with:

Images