Method and system for predicting response to immune anticancer agent

Abstract

To provide a method for predicting a response to an immune anticancer agent, the method being executed by at least one computer device.SOLUTION: The method includes the steps of: receiving a first pathological slide image; detecting at least one target item in the first pathological slide image; determining, on the basis of the detection result for the at least one target item, at least one of an immune phenotype in at least a partial region of the first pathological slide image and information associated with the immune phenotype; and generating, on the basis of the immune phenotype in at least the partial region of the first pathological slide image or the information associated with the immune phenotype, a prediction result as to whether a patient associated with the first pathological slide image responds to an immune anticancer agent.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to methods and systems for predicting response to immune anti-cancer drugs, and more particularly The immunophenotype or based on at least one of the information associated with the immunophenotype, on the pathology slide image. and a method for generating a predictive result as to whether a patient associated with the and systems. [Background technology]

[0002] Recently, as a third-generation anticancer drug for cancer treatment, immunotherapy that utilizes the immune system in the patient's body has been developed. There is growing interest in anti-cancer drugs. Immuno-anti-cancer drugs are drugs that help cancer cells evade the immune system of the human body. or to allow immune cells to better recognize and attack cancer cells. It acts through the immune system of the human body, so it has almost no side effects like anti-cancer drugs. It can also prolong the survival time of cancer patients compared to other anti-cancer drugs. However, such immunotherapy is not effective for all cancer patients. To predict the effectiveness of immunotherapy for cancer patients, we need to predict the responsiveness of immunotherapy to anticancer drugs. It is important to do so.

[0003] On the other hand, PD is a biomarker for predicting the response to immunotherapy. The expression rate of -L1 can be confirmed by obtaining tissue from patients before treatment and performing IHC (immunoassay). Histochemistry (Histology) staining was performed, and then PD-L was directly extracted from the stained tissue. After counting the expression level of 1, patients with a certain level of expression will be evaluated for the effectiveness of immune anti-cancer drugs. It can be predicted that there will be some results using such conventional techniques. The problem is that objective quantification is difficult due to subjective factors. There are also various factors for predicting response to immunotherapy. When predicting PD-L1 expression as a single factor, the accuracy is reduced. This is because, even when PD-L1 is expressed, immune cells may be present around cancer cells. Without PD-L, immune responses to anti-cancer drugs are unlikely to occur. Quantification of the expression of 1 alone does not reveal the presence of immune cells associated with the antitumor effect of immunotherapy. It can be difficult to grasp the distribution of these variables. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method for predicting immune responses to anticancer drugs to solve the above-mentioned problems. Provide laws and systems. [Means for solving the problem]

[0005] The present disclosure relates to a method, an apparatus (system), or a computer-readable medium storing instructions. The present invention may be embodied in various ways, including as a storage medium or a computer program.

[0006] An immune anti-cancer method performed by at least one computer device according to one embodiment of the present disclosure The method for predicting a response to an agent includes receiving a first pathology slide image; detecting one or more items of interest within the first pathology slide image; based on the detection results for the one or more target items, associated with at least some regional immune phenotype or immune phenotype determining information and immunofluorescence of at least a portion of the region within the first pathology slide image; Based on the information associated with the phenotype or immunophenotype, a first pathology slide image and generating a prediction result as to whether the associated patient will respond to the immunotherapy; Includes:

[0007] In one embodiment of the present disclosure, the detecting step comprises using an artificial neural network target item detection model. detecting one or more items of interest in a first pathology slide image using The method includes: detecting an artificial neural network target item detection model from a reference pathology slide image; It is trained to detect one or more reference target items.

[0008] In one embodiment of the present disclosure, at least a portion of the first pathology slide image is One or more target items include an item associated with cancer and and immune cells, and the determining step includes determining at least one of the first pathology slide image and the second pathology slide image. In some areas, the number, distribution, or density of immune cells in items associated with cancer is low. and calculating at least one of the number, distribution or density of the calculated immune cells. and determining an immunophenotype of at least a portion of the first pathology slide image based on one of the following: and determining at least one piece of information associated with the immunophenotype.

[0009] In one embodiment of the present disclosure, the items associated with cancer include cancer areas and and cancer stroma, and the calculating step is performed by using a first pathology slide image. calculating the density of immune cells in the cancerous region in at least a portion of the image; and detecting an immunological abnormality in the cancer stroma in at least a partial region of the first pathology slide image. and calculating the density of immune cells in the cancerous region, wherein the determining step includes: a first pathological slice based on at least one of the density of immune cells in the cancer stroma or the density of immune cells in the cancer stroma; and (b) determining an immunophenotype or information associated with an immunophenotype of at least a portion of the region in the image. The method includes determining at least one of the following:

[0010] In one embodiment of the present disclosure, when the density of immune cells in the cancerous region is equal to or greater than a first threshold density, The immunophenotype of at least a portion of the region in the first pathology slide image is determined as immunoreactivity (immunoactivity). The cancer is determined as inflamed, and the density of immune cells in the cancer area is less than a first threshold density. At the same time, if the density of immune cells in the cancer stroma is equal to or greater than the second threshold density, the first pathological slurries are detected. The immunophenotype of at least some areas within the id image is immune excluded. and determining whether the density of immune cells in the cancerous region is less than a first threshold density and simultaneously determining whether cancer stroma is present. If the density of immune cells in the image is less than a second threshold density, The immunophenotype of at least some regions is determined to be immune desert.

[0011] In one embodiment of the present disclosure, the first threshold density is a plurality of threshold densities in the plurality of pathology slide images. In each of the regions of interest, a second The threshold density is a density of cancer stroma in each of a plurality of regions of interest in a plurality of pathology slide images. It is determined based on the distribution of immune cell density within the spleen.

[0012] In one embodiment of the present disclosure, the determining step is performed by determining the amount of immune cells contained within a specific region within the cancerous region. and determining an immunophenotype of at least a portion of the region in the first pathology slide image based on the number of immunoglobulin cells. The method further includes determining the type as one of immune activity, immune exclusion, or immune deficiency.

[0013] In one embodiment of the present disclosure, the determining step comprises determining the , features for each of at least a portion of the area in the first pathology slide image ure) or by inputting at least a partial area in the first pathology slide image, Each immunophenotype or immunophenotype-related immunophenotype of at least a portion of the area in the pathology slide image determining associated information, and the artificial neural network immunophenotyping model Feature for at least a portion of the area in the pathology slide image or reference pathology slide By inputting at least a partial region in the image, a small region in the reference pathology slide image can be identified. At least one of the immunophenotype of at least a part of the region or information related to the immunophenotype Learned to make decisions.

[0014] In one embodiment of the present disclosure, for at least a portion of the first pathology slide image, The feature is one or more pairs of at least a portion of the first pathology slide image. statistical features for the target item, geometric features for one or more target items feature or an image field corresponding to at least a portion of the first pathology slide image. The feature includes at least one of the following:

[0015] In one embodiment of the present disclosure, at least a portion of the first pathology slide image is and immunophenotyping at least a portion of the first pathology slide image, the immunophenotyping comprising a plurality of regions of interest. includes an immunophenotype of each of the plurality of regions of interest, and the generating step comprises: Based on the individual immunophenotypes, the most abundant tumors within the entire area of ​​the first pathology slide image were identified. determining the immunophenotype contained within the entire area of ​​the first pathology slide image; Based on the most prevalent immunophenotype in the study, patients were predicted to respond to immunotherapy. and generating a prediction result.

[0016] In one embodiment of the present disclosure, the generating step comprises: and identifying at least one region of the first pathology slide image using an immunophenotype of at least a portion of the region. A step of generating an immune phenotype map for a partial region and a model for predicting immune responses to anticancer drugs. By inputting the generated immunophenotype map, it is possible to predict whether a patient will respond to an immunotherapy or not. and generating a prediction result based on the reference immune expression. Given a type map, it is trained to generate reference prediction results.

[0017] In one embodiment of the present disclosure, the generating step comprises: The first pathology slide image is then analyzed using information associated with the immunophenotype of at least some of the regions. generating an immunophenotypic feature map for at least a portion of the image; By inputting the generated immune phenotype feature map into the anticancer drug response prediction model, the patient's immune status can be predicted. and generating a prediction result as to whether or not the patient will respond to the immuno-anticancer drug. The adaptive prediction model generates a reference prediction result by inputting a reference immunophenotype feature map. It is learned to do so.

[0018] In one embodiment of the present disclosure, a second pathology slide image associated with the patient is used to generate a backlog of the patient's pathology data. The generating step further includes obtaining information regarding expression of biomarkers. and determining an immunophenotype or immunophenotype-related immunophenotype of at least a portion of the region in the first pathology slide image. based on at least one of the linked information and information regarding expression of the biomarkers; The method includes generating a predictive result as to whether the patient will respond to the immunotherapy.

[0019] In one embodiment of the present disclosure, the biomarker is PD-L1.

[0020] In one embodiment of the present disclosure, information regarding PD-L1 expression is obtained from TPS (Tumor Propagation Sites). It includes at least one of a Combined Proportion Score (CPS) value or a Combined Proportion Score (CPS) value.

[0021] In one embodiment of the present disclosure, the acquiring step includes receiving a second pathology slide image. and inputting the second pathology slide image into the artificial neural network expression information generation model. and generating information about PD-L1 expression by analyzing the expression of PD-L1 in an artificial neural network. The information generation model uses reference pathology slide images as input to generate PD-L1 expression data. It is trained to generate reference information about

[0022] In one embodiment of the present disclosure, the artificial neural network expression information generation model is provided with a second pathological slide image. The step of generating information about PD-L1 expression by inputting a message is performed by an artificial neurotransmitter. and generating a second pathology slide image using the retinal expression information generation model to generate a second pathology slide image. The location of tumor cells, lymphocytes, macrophages, or the presence or absence of PD-L1 expression generating information about PD-L1 expression by detecting at least one of the include.

[0023] In one embodiment of the present disclosure, the detection results for one or more target items, a first pathology Immunophenotype of at least a portion of the region in the slide image, and information associated with the immunophenotype. , a predicted result as to whether the patient will respond to the immunotherapy or a first pathology slide image and outputting at least one of the density of immune cells in at least a portion of the area. Included.

[0024] In one embodiment of the present disclosure, based on the predicted results of whether a patient will respond to an immunosuppressant, Based on this, at least one immunological anticancer agent that is compatible with the patient is selected from among the plurality of immunological anticancer agents. The step of outputting the information is further included.

[0025] According to one embodiment of the present disclosure, the method for predicting a response to the aforementioned immunological anti-cancer drug is performed by a computer. A computer program stored on a computer-readable recording medium for execution by a computer A program will be provided.

[0026] An information processing system according to an embodiment of the present disclosure includes one or more instructions. instructions) and the execution of one or more of the stored instructions. Receive a first pathology slide image, and within the first pathology slide image, Detecting the above target items, and based on the detection results for one or more target items, Immunophenotype or immunophenotype-associated immunophenotype of at least a portion of a region in one pathology slide image and determining the immunophenotype of at least a portion of the first pathology slide image. and associating the first pathology slide image with information associated with the type or immunophenotype. The method is configured to generate a predictive result as to whether an assigned patient will respond to an immunotherapy anti-cancer drug. and a processor. [Effects of the Invention]

[0027] According to some embodiments of the present disclosure, immunophenotypes or At least one of the information associated with the immunophenotype is used to determine whether a patient responds to an immunotherapy. In other words, by objectively analyzing the immune environment around cancer cells, This can improve the prediction rate of whether a patient will respond to an immunotherapy anti-cancer drug.

[0028] According to some embodiments of the present disclosure, the expression rate of PD-L1 can be objectively quantified, and the quantification The expression rate of PD-L1 can be used to predict whether a patient will respond to immunotherapy. In addition to this information on PD-L1 expression, pathology slide images were also used to determine The immunophenotype and / or information associated with the immunophenotype may be used to identify patients. This can further improve the accuracy of predictions as to whether a patient will respond to an immunotherapy.

[0029] According to some embodiments of the present disclosure, an artificial neural network model is used to analyze immune phenotypes and / or immune responses. Determine information associated with immune phenotype and obtain information on PD-L1 expression This allows for more accurate and faster processing than conventional technology.

[0030] According to some embodiments of the present disclosure, a user can view the results generated during the immune anticancer drug response prediction process. The results can be presented visually and intuitively. In addition, the user can predict the response of immune anti-cancer drugs. A report summarizing the results generated during the process can be provided. Based on the results generated in the anticancer drug response prediction process, we select the most suitable anticancer drug for the patient from among multiple immune anticancer drugs. Optimal immunotherapy and / or combinations of immunotherapy can be recommended.

[0031] The effects of the present disclosure are not limited to these, and other effects not mentioned are within the scope of the claims. A person having ordinary skill in the art to which this disclosure pertains (hereinafter referred to as an "ordinary engineer") ) should be clearly understood. [Brief explanation of the drawings]

[0032] Embodiments of the present disclosure will be described with reference to the accompanying drawings, in which like reference numerals refer to: indicates similar elements, but is not limited to these. [Figure 1] 1 is an exemplary configuration diagram showing a system in which an information processing system according to an embodiment of the present disclosure provides response prediction results to an immunological anticancer drug. [Figure 2] 1 is a block diagram showing an internal configuration of an information processing system according to an embodiment of the present disclosure. [Figure 3] 1 is a flowchart illustrating a method for predicting response to an immune anti-cancer drug according to one embodiment of the present disclosure. [Figure 4] 1A and 1B are diagrams illustrating an example of detecting a target item in a pathology slide image, determining a region of interest, and determining the immunophenotype of the region of interest according to one embodiment of the present disclosure. [Figure 5] FIG. 10 illustrates an example of generating immunophenotyping results according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram showing an example of generating a response prediction result for an immunological anticancer drug according to an embodiment of the present disclosure. [Figure 7]FIG. 10 is a diagram showing the results of an analysis of the predictive ability of information on PD-L1 expression and / or immunophenotype information regarding the response to an immunotherapy anticancer drug in accordance with one embodiment of the present disclosure. [Figure 8] 1 is a graph showing the correlation between immunophenotype and information regarding the expression of CD3-positive cells, CD8-positive cells, FOXP3-positive cells, and CD68-positive cells according to one embodiment of the present disclosure. [Figure 9] 1 is a graph showing the performance of various methods for predicting whether a patient will respond to an immunotherapy anti-cancer drug, according to one embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing an example of outputting a result generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to an embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 12] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 13] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 14] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 15] FIG. 10 is a diagram showing an example of outputting results generated in a prediction process for whether or not a patient will respond to an immunosuppressant according to another embodiment of the present disclosure. [Figure 16] FIG. 1 illustrates an example of an artificial neural network model according to an embodiment of the present disclosure. [Figure 17] FIG. 1 is a diagram illustrating an exemplary system configuration for providing response prediction results to an immune anticancer drug according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, the following description will be omitted. A detailed description of the functions and configurations will be omitted.

[0034] In the accompanying drawings, the same or corresponding components are given the same reference numerals. In the following description of the embodiments, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is understood that such component may be included in a particular implementation. It should not be intended to be an example that is not included.

[0035] The advantages and features of the disclosed embodiments and the manner in which they are achieved will be described below with reference to the accompanying drawings. However, the present disclosure is not limited to the implementations disclosed below. However, the present disclosure is not limited to the above examples and may be embodied in many different forms. and to enable one of ordinary skill in the art to accurately appreciate the scope of the invention. It is only provided to

[0036] A brief explanation of terms used in this specification and a detailed description of the disclosed embodiments will be provided. The terms used herein are currently used as broadly as possible while still considering their function in this disclosure. The general terms used have been selected based on the intentions of engineers engaged in the relevant field or on legal precedents. In particular cases, the term may be arbitrarily selected by the applicant. These meanings will be explained in detail in the description of the invention. The terms used in this disclosure are not simply names of terms, but rather are used to describe the meanings that the terms have and the meanings that the terms have in this disclosure. should be defined based on the overall content of

[0037] In this specification, unless otherwise clearly specified in the context, the singular expression includes the plural expression. Throughout the specification, certain parts may be used interchangeably with certain structures. When a component is referred to as "including," this does not include other components unless specifically stated to the contrary. This means that other components may be included instead of the above.

[0038] Furthermore, the terms "module" and "part" used in the specification do not refer to software or hardware. A "module" or "part" refers to a hardware component that performs a certain function. However, "module" or "part" does not mean limited to software or hardware. A "module" or "unit" is configured to reside on an addressable storage medium. One or more processors may be configured to regenerate. For example, a "module" or a "unit" may refer to a software component, an object, or a Components such as directional software components, class components and task components, process functions, attributes, procedures, subroutines, segments of program code, drivers Bars, firmware, microcode, circuits, data, databases, data structures, A module can contain at least one of a table, an array, or a variable. A "module" or "part" is a system in which the functionality provided is divided into a smaller number of components and "modules." or are combined into a "module" or "unit" or are subject to additional components and "modules" or "units." It can be separated into

[0039] According to one embodiment of the present disclosure, a "module" or a "unit" may be embodied by a processor and a memory. "Processor" can be a general-purpose processor, a central processing unit (CPU), a microprocessor, processors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some circumstances, "processor" may be used for specific purposes. Application-specific integrated circuits (ASIC), programmable logic devices (PLD), field It can also be called a programmable gate array (FPGA). For example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, A combination of one or more microprocessors coupled with a DSP core, or any other It can also refer to a combination of processing devices such as a combination of configurations such as "Memory" is to be interpreted broadly to include any electronic component capable of storing electronic information. "Memory" can be RAM (Random Access Memory), ROM (Read Only Memory), ry), NVRAM (Non-Volatile Random Access Memory), PROM (Programmable Re ad-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPR OM (Electrically Erasable Programmable Read-Only Memory), flash memory, Processor-readable media such as magnetic or optical data storage devices, registers, etc. It can be called various types. The processor reads / reads information from memory. The memory is said to be in electronic communication with the processor if the memory can store information. Memory integrated into a processor is in electronic communication with the processor.

[0040] In this disclosure, a "system" includes at least one of a server device and a cloud device. For example, a system may include, but is not limited to, one or more servers. In another example, the system may consist of one or more cloud devices. In another example, the system can be configured to use both the server device and the cloud device. It can also be operated as

[0041] In this disclosure, "target data" refers to any data that can be used to train a machine learning model. It can refer to data or data items, for example, data showing an image, sound or audio. This disclosure includes, but is not limited to, data indicating voice characteristics. The entire pathology slide image and / or a small portion of the pathology slide image are used as data. Although the explanation is given using at least one patch, it is not limited to this and can be applied to learning machine learning models. Any data that can be used for learning can be the target data. Label information is tagged using annotation.

[0042] In this disclosure, a "pathological slide image" refers to a tissue or the like removed from a human body, taken under a microscope. Photographs of pathology slides that have been fixed and stained through a series of chemical treatments for viewing in the For example, a pathology slide image is a digital image taken with a microscope. It can refer to cells, tissues and / or structures in the human body. The pathology slide image can also contain information about the structure of the specimen. It can contain more than one patch, and one or more patches can be labeled by annotation. The pathology slide is tagged with information about the pathology slide (e.g., immunophenotype information). Images are H&E stained tissue slides and / or IHC stained tissue slides. These include, but are not limited to, various staining methods (e.g., CISH (Chromo In situ Hybridization)). genic in situ hybridization), FISH (Fluorescent in situ hybridization), M Tissue slides on which ULTIPLEX IHC has been applied or unstained Histology slides may also be included. As another example, a "pathology slide image" may include an immunological A patient tissue slide generated for predicting cancer drug response, showing the patient's condition before immunotherapy with anti-cancer drugs. The patient's tissue slides may include those from the original patient and / or those from the patient after immunotherapy. Cut.

[0043] In this disclosure, a "patch" may refer to a small region within a pathology slide image. For example, patches are generated for pathology slide images using segmentation. It is possible to include regions corresponding to semantic objects extracted by performing In another example, patches can be generated by analyzing pathology slide images. The combination of the label information and the associated pixel can be referred to as a pixel.

[0044] In the present disclosure, "at least a part of the area of ​​the pathology slide image" refers to the area of ​​the pathology slide image. For example, a pathology slide can be used to refer to at least a portion of an image that is to be analyzed. At least a portion of the image is a portion of the pathology slide image that contains the target item. As another example, a pathology slide image may be used. At least a part of the region is a plurality of regions generated by dividing a pathology slide image. At least a part of the patch can be referred to as "pathology slide image." "At least a part of the area" refers to all areas (or all parts) that make up the pathology slide image. In the present disclosure, the term "pathological slide image" can refer to the whole or a part of the above. "At least a portion of a page" can be referred to as a region of interest, and conversely, The region can refer to at least a portion of a pathology slide image.

[0045] In this disclosure, a "machine learning model" and / or an "artificial neural network model" refers to a given It can include any model used to infer an answer to an input. According to one embodiment, the machine learning model includes an input layer, a plurality of hidden layers, and an output layer. The model may include an artificial neural network model including a layer, where each layer may include multiple nodes. For example, the machine learning model can analyze pathology slide images and / or the data contained in the pathology slides. The model can learn to infer label information for at least one patch. The label information generated by the annotation process is useful for training machine learning models. In addition, a machine learning model is associated with multiple nodes included in the machine learning model. The weights may include weights assigned to the machine learning model. Any parameter may be included.

[0046] In this disclosure, "learning" refers to the process of learning a machine using at least one patch and label information. It can refer to any process of changing the weights associated with a machine learning model. According to an embodiment, the learning is performed by a machine learning model using at least one patch and label information. The model is propagated in one or more forward and backward propagations. The process of changing or updating weights associated with a machine learning model through It can be called.

[0047] In this disclosure, "label information" refers to the correct answer information of a data sample. Labels or labels are information obtained as a result of annotation work. Information may be used interchangeably with terms such as annotation and tag in the art. In this disclosure, "annotation" refers to the act of annotation and / or or annotation information determined by performing annotation work (e.g., labels) In this disclosure, "annotation information" may refer to annotation information. Information for the annotation work and / or information generated by the annotation work (e.g., It can be called "bell information."

[0048] In this disclosure, a "target item" is a target item to be detected in a pathology slide image. These may refer to data / information, image regions, objects, etc. According to the regulations, the target items are used for the diagnosis, treatment and prevention of diseases (e.g., cancer), pathology, etc. It can contain the object to be detected from the slide image. For example, The "system" can include cell-based target items and area-based target items.

[0049] In the present disclosure, the "immunophenotype of at least a portion of a pathology slide" refers to the immunophenotype of at least a portion of a pathology slide. The determination is based on at least one of the number, distribution, and density of immune cells within at least a portion of the area. Such immunophenotypes can be expressed by a variety of classification systems, for example, immunophenotypes. The phenotypes are classified as immune inflamed, immune excluded, and immune deficient. is shown as immune desert.

[0050] In this disclosure, "information associated with an immunophenotype" refers to information that describes or characterizes an immunophenotype. According to one embodiment, the immunophenotype may be associated with any information. The acquired information may include immunophenotypic features, where the immunophenotypic features are Scores are assigned to classes corresponding to immune phenotypes (e.g., immune activation, immune exclusion, immune deficiency). Score value (class-specific score or density value of the classifier) ​​and / or the value input to the classifier It includes various vectors associated with immune phenotypes, such as features. For example, information associated with immune phenotypes can be used to: 1) develop artificial neural networks and mechanistic models; 2) the score value associated with the immune phenotype output from the training model; Density, number, and various statistical values ​​of immune cells applied to thresholds (or cut-offs) 3) immune cells or cancer cells and other cells, such as vector values ​​that represent the distribution of immune cells Types of cysts (cancer cells, immune cells, fibroblasts, lymphocytes, plas ma cells, macrophages, endothelial cells, etc.) Relative relationships (e.g., histogram vectors and graph representation vectors that consider direction and distance) ) or scalar values ​​including relative statistics (e.g., ratio of immune cells to other cells). 4) the types of immune cells and cancer cells and surrounding areas (e.g., cancer area, cancer stroma, etc.) area, Tertiary lymphoid structure, Normal re gion, Necrosis, Fat, Blood vessel, High end othelial venule, lymphatic vessel, nerve etc. ) statistics (e.g., ratio of cancer stroma area to immune cell count) and distribution (e.g., histogram It can contain scalar or vector values, including vectors and graph representations. can.

[0051] In the present disclosure, "each of a plurality of A's" and / or "each of a plurality of A's" means that the plurality of A's are included in the plurality of A's. It refers to all the components included in the A, or to some of the components included in multiple A. For example, each of the multiple regions of interest can be a region of interest that includes all regions of interest contained in the multiple regions of interest. It can refer to each of the regions of interest, or each of the regions of interest included in multiple regions of interest. .

[0052] In this disclosure, "instructions" are combined based on function. One or more instructions combined together that are components of a computer program and It can refer to something that is executed by a processor.

[0053] In this disclosure, a "user" can refer to a person who uses a user terminal. For example, a user may include an annotator who performs annotation tasks. As another example, the user may wish to view predicted responses to immune-mediated anti-cancer drugs (e.g., if the patient is receiving immune-mediated anti-cancer drugs). This includes doctors and patients who are provided with the predicted results of whether or not they will respond to the drug. Also, a user can refer to a user terminal, and conversely, a user terminal can refer to a user. That is, the terms user and user terminal may be used interchangeably herein.

[0054] FIG. 1 shows a graph showing a response prediction for an immunological anticancer drug in an information processing system 100 according to an embodiment of the present disclosure. 1 is an exemplary block diagram showing a system for providing measurement results. provide response predictions (e.g., whether a patient will respond to an immunotherapy or anti-cancer drug) The system includes an information processing system 100, a user terminal 110, and a storage system 120. Here, the information processing system 100 includes a user terminal 110 and a storage system. 1, one unit can be configured to be connected to and able to communicate with each of the systems 120. However, the present invention is not limited to this, and multiple user terminals 110 may be used for information processing. 1, the information processing system 100 may be configured to communicate with the Although the system 100 is shown as a single computer device, it is not limited to this. The system 100 is a distributed processing system for information and / or data across multiple computer devices. In addition, in FIG. 1, the storage system 120 is shown as a single device. However, it is not limited to this, and may be configured as multiple storage devices or support a cloud. In addition, Figure 1 shows a system for predicting the response to an immunological anticancer drug. Each component of the system that provides the measurement results represents a functionally distinct functional element. In this way, multiple components can be embodied in a form that is integrated with each other in an actual physical environment. .

[0055] The information processing system 100 and the user terminal 110 predict the response to the immunological anticancer drug. Any computer device that can be used to generate and provide a A computer device can refer to any type of device equipped with computer functionality, For example, notebooks, desktops, laptops, servers, It can be a cloud system, but is not limited to this.

[0056] The information processing system 100 can receive a first pathology slide image. The processing system 100 can receive a first pathology slide image from the storage system 120. As another example, the information processing system 100 receives a first pathological slide from the user terminal 110. The information processing system 100 can receive the first pathology slide image. Using the image, the patient's response to the immunotherapy drug was correlated with the first pathology slide image. The system can be configured to generate a prediction result on whether or not to perform the operation.

[0057] In one embodiment, the information processing system 100 detects in the first pathology slide image: For example, the information processing system 100 may include an artificial neural network. Utilizing a network object detection model to detect one or more Here, the artificial neural network target item detection model is A model trained to detect one or more reference items from slide images. Wherein the one or more target items are cancer-associated items and immune The cancer-related items may include cancer areas. and cancer stroma. The term "cancer area" can be confused with "cancer epithelium."

[0058] The information processing system 100 determines the disease based on the detection results for one or more target items. Immunophenotype or immunophenotype of at least a portion of the area in the slide image At least one piece of information associated with the phenotype can be determined. The processing system 100 detects cancer-related features in at least a partial region of the pathology slide image. Calculating at least one of the number, distribution, or density of immune cells in the linked items; Based on at least one of the number, distribution, or density of immune cells detected in the pathology slide image, At least one of the immunophenotypes of at least a part of the region in the For example, the information processing system 100 can determine one of the following: The density of immune cells in the cancer area and the density of immune cells in the cancer stroma are The density of immune cells in the cancer region or the density of immune cells in the cancer stroma is calculated. Based on one of the above, immunophenotype or immunorepresentation of at least a portion of the region in the pathology slide image is determined. At least one of the pieces of information associated with the present model can be determined.

[0059] At this time, if the density of immune cells in the cancer region is equal to or greater than the first threshold density, the pathology slide is The immunophenotype of at least some regions within the image is determined as immune inflamed. In addition, the density of immune cells in the cancer region is less than the first threshold density, and at the same time, the cancer streptococcus is If the density of immune cells in the lesion is equal to or greater than a second threshold density, the density of immune cells in the pathology slide image is The immunophenotype of at least some areas can be determined as immune excluded. The density of immune cells in the cancer region is less than the first threshold density, and at the same time, the density of immune cells in the cancer stroma is If the density of cells is less than a second threshold density, at least a portion of the area in the pathology slide image is The immune phenotype of the immune system can be determined as immune desert. In each of a plurality of regions of interest in a plurality of pathological slide images, the method detects immune cells in a cancer region. The second threshold density is determined based on the distribution of cell densities in the multiple pathology slide images. In each of the multiple regions of interest, a density of immune cells within the cancer stroma is determined based on the distribution of the density of immune cells. Additionally or alternatively, the information processing system 100 may be included within a specific region within the cancer region. Based on the number of immune cells detected, the immunophenotype of one or more regions of interest can be determined as immune active, immune excluded, or It can be determined as an immune deficiency.

[0060] In another embodiment, the information processing system 100 is configured to use an artificial neural network immunophenotyping model. , a feature or pathology for at least a portion of the area in the pathology slide image; By inputting at least a partial area in the slide image, the pathology slide image At least one of an immunophenotype of at least a portion of the region or information associated with the immunophenotype Here, the artificial neural network immunophenotyping model is based on the reference pathology slide image. or at least a portion of the feature in the reference pathology slide image. By inputting even a partial region, the immunophenotype of at least a partial region in the pathology slide image can be determined. trained to determine at least one piece of information associated with a phenotype or immunophenotype Here, the function for at least a part of the area in the pathology slide image can be used as a model. The feature is applied to one or more target items in at least a portion of the pathology slide image. statistical features for one or more target items, geometric features for one or more target items, or At least one image feature corresponding to at least a portion of the pathology slide image is identified. Also, the pathological sequence input to the artificial neural network immunophenotyping model can be At least a portion of the area in the slide image is at least one area in the H&E stained image. Partial area, at least a partial area in the IHC stained image, Multiplex IH C. At least a portion of the stained image may include, but is not limited to, It is not something that can be done.

[0061] The information processing system 100 detects an immunorepresentation of at least a portion of a region in a pathology slide image. a first pathology slice based on at least one of information associated with the type or immunophenotype; Generates predictions for whether a patient will respond to an immunotherapy drug based on the associated image. In one embodiment, the information processing system 100 performs the following steps: Based on the immunophenotype for each of the plurality of regions of interest, an entire image of the first pathology slide is The most abundant immunophenotype (i.e., representative immunophenotype) in the body region can be determined. The information processing system 100 then performs a full scan of the first pathological slide image thus determined. Patient response to immunotherapy based on the most prevalent immunophenotype in the body region It is possible to generate a prediction result as to whether or not

[0062] In another embodiment, the information processing system 100 may include at least one The immunophenotype of at least a portion of the area in the pathology slide image is used to identify the area. We will generate an immune phenotype map and use it as a model for predicting immune responses to anticancer drugs. By inputting the generated immunophenotype map, it is possible to predict whether a patient will respond to an immunotherapy or not. Here, the immune anticancer drug response prediction model can generate prediction results that correspond to the reference immune phenotype map. A statistical model is trained to generate a reference prediction result based on the input data. cal model) and / or artificial neural network models.

[0063] In yet another embodiment, the information processing system 100 may include a second disease associated with the patient. Information on biomarker expression was obtained from the pathology slide images and analyzed using the first pathology slide. and an immunophenotype of at least a portion of the region in the image and information associated with the immunophenotype. and determining whether the patient is a candidate for an immunotherapy based on information regarding the expression of at least one biomarker. A predictive outcome for response or non-response can be generated, where the biomarker is PD-L1. These may include, but are not limited to, CD3, CD8, CD68, FOXP3, CD20, C In addition to D4, CD45, and CD163, various biomarkers associated with immune cells Here, the information on PD-L1 expression can be included in the TPS (Tumor Proportion It can include at least one of the following: For example, the information processing system 100 may receive the second pathology slide image and By inputting the second pathology slide image into the neural network expression information generation model, For example, the information processing system 100 can generate information about the expression of the neural network. and generating a second pathology slide image using the retinal expression information generation model to generate a second pathology slide image. the location of cells (tumor cells, lymphocytes, macrophages, etc.) present in the The presence or absence of biomarker expression, the number of biomarker-positive cells, and the amount of biomarker-positive cells By detecting at least one of the scores, information about the expression of biomarkers is generated. Here, the artificial neural network expression information generation model is based on the input of a reference pathology slide image. The model is trained to generate reference information about biomarker expression. This can be applied to the following cases.

[0064] For example, the information processing system 100 may receive a second pathology slide image and generate an artificial neural By inputting the second pathology slide image into the omentum expression information generation model, the expression of PD-L1 was estimated. For example, the information processing system 100 can generate artificial neural network expression information. The generative model is used to identify tumor cells in at least a portion of the second pathology slide image. the location of lymphocytes, the location of macrophages, or the presence or absence of PD-L1 expression By detecting one, information about PD-L1 expression can be generated. The omentum expression information generation model uses reference pathology slide images as input to generate PD-L1 expression information. This can refer to a model that has been trained to generate reference information about expression.

[0065] The information processing system 100 receives a request for one or more target items via a user terminal 110. the detection results, the immunophenotype of at least a portion of the region in the pathology slide image, The associated information, the predicted outcome of whether the patient will respond to the immunotherapy or the pathology slides At least one of the density of immune cells in at least a partial region in the image can be output. That is, the user 130 (e.g., a doctor or a patient) is provided with a The results generated during the prediction process are provided via the user terminal 110. Additionally or alternatively, In other words, the information processing system 100 predicts whether a patient will respond to an immunosuppressant. Based on this, at least one immunological anticancer drug that is compatible with the patient is selected from among multiple immunological anticancer drugs. The information can be output via the user terminal 110.

[0066] The storage system 120 stores the target patient's data to provide a response prediction result to an immunological anti-cancer drug. and associated pathology slide images, various data associated with machine learning models. It is a device or cloud system that stores and manages data. The storage system 120 can store and manage various data using a database. The various data may include any data associated with the machine learning model, such as For example, the target data file, the meta information of the target data, and the target data that is the annotation work result. Label information about the data, data about annotation work, machine learning models (e.g. These may include, but are not limited to, artificial neural network models. Although the information processing system 100 and the storage system 120 are shown as separate systems, However, the present invention is not limited to this, and the present invention may be configured as an integrated system.

[0067] Information on the overall distribution of immune cells (e.g., how many immune cells are present inside cancer cells) Because these factors (such as the state of invasion by the immune system) play an important role in predicting the response to anticancer drugs, In predicting response to immunotherapy, H&E stained pathology slide images ( i.e., information on the distribution of various immune cells in H&E-stained tissue slide images. According to some embodiments of the present disclosure, the immune environment surrounding cancer cells can be objectively analyzed. By analyzing the data, it is possible to improve the predictability of whether a patient will respond to an immunotherapy. The expression rate of PD-L1 can be objectively quantified, and the quantified expression rate of PD-L1 can be used to This can predict whether a patient will respond to an immunotherapy or anti-cancer drug.

[0068] FIG. 2 is a block diagram showing the internal configuration of an information processing system 100 according to an embodiment of the present disclosure. According to one embodiment, as shown in the figure, the information processing system 100 A detection unit 210, a region of interest determination unit 220, an immunophenotyping unit 230, and a reaction prediction unit for an immunological anticancer drug 2, each component of the information processing system 100 may include a measurement unit 240. , which indicates functionally divided functional elements, and multiple components are used in the actual physical environment. These can be embodied in a form that is integrated with each other.

[0069] The target item detection unit 210 detects a pathology slide image (for example, a first H&E stained slide) Receive a pathology slide image of the first slide, a second pathology slide image of the IHC stained slide, etc. and detecting one or more items of interest within the received first pathology slide image. In one embodiment, the target item detection unit 210 uses an artificial neural network target item detection model. The method can be used to detect one or more items of interest in a first pathology slide image. For example, the target item detection unit 210 may detect, in the first pathology slide image, Tumor cells, lymphocytes, macrophages, and dendritic cells dendritic cell, fibroblast, endothelial cell , blood vessel, cancer stroma, cancer epithelium , cancer area, normal area (e.g., normal lymphatic structure) It can detect target items such as lymph node architecture regions.

[0070] The region of interest determination unit 220 determines one or more regions of interest in the first pathology slide image. Here, the region of interest is defined as one or more target areas within the pathology slide image. For example, the region of interest determiner 220 may determine a first region of interest. Among the multiple patches that make up the pathology slide image, patches that contain one or more target items In the present disclosure, the region of interest determining unit 220 can determine the region of interest as an information processing system. Although shown as being included in the information processing system 100, it is not limited thereto. 0 is used to determine the region of interest without determining the immunophenotype determination unit 230 and the immunoanticancer drug response prediction unit. 240 can process at least a portion of the area within the first pathology slide image.

[0071] The immunophenotyping unit 230 determines the first immunophenotype based on the detection results for one or more target items. Immunofluorescence of at least a portion of a region (e.g., one or more regions of interest) within one pathology slide image At least one of information associated with an immunophenotype or an immunophenotype can be determined. In the method, the immunophenotyping unit 230 detects at least a portion of the first pathology slide image. Within the area, a decrease in the number, distribution, or density of immune cells within the item associated with cancer and calculating at least one of the number, distribution, and density of immune cells based on the calculated number, distribution, or density of immune cells. and determining an immunophenotype or immunophenotype-related immunophenotype of at least a portion of the region in the first pathology slide image. For example, the immunophenotyping unit 230 may determine at least one of the associated information. Within one or more regions of interest, the density of immune cells within the cancerous region or the density of immune cells within the cancer stroma and based on this, immunophenotyping and Further, the immunophenotyping unit 23 can determine information associated with the immunophenotype and / or the immunophenotype. 0 is based on the number of immune cells contained within a specific area within the cancerous area, and The immune phenotype can be determined as one of immune activity, immune exclusion, or immune deficiency.

[0072] In one embodiment, the immunophenotyping unit 230 may include an artificial neural network immunophenotyping model: A feature or a first pathology for at least a portion of the area in the first pathology slide image is At least one of at least a portion of the slide image is input to identify a first disease. Immunophenotyping and / or immunophenotyping-associated immunophenotypes of at least a portion of a region within a slide image The information obtained by detecting at least a portion of the first pathology slide image can be determined. The feature for the region is one or more of at least a portion of the region in the first pathology slide image. Statistical features for the above target items, geometry for one or more target items an image corresponding to a characteristic feature or at least a portion of the first pathology slide image; Additionally or alternatively, the first pathology may include at least one of The features for at least some regions in the slide image are determined by the statistical filtering described above. Two or more features of the feature, geometric feature and image feature are fused. The feature may include:

[0073] The immunological anticancer drug response prediction unit 240 predicts at least a part of the region in the first pathological slide image. A first pathology slide is generated based on the immunophenotype of the area and / or information associated with the immunophenotype. The results of the study correlated with the image of the patient and predicted whether or not the patient would respond to an immunotherapy. It can be achieved.

[0074] In one embodiment, the immunological anticancer drug response prediction unit 240 is Based on the immunophenotype of each of the plurality of regions of interest, the entire region of the first pathology slide image is The most abundant immunophenotype within the entire area of ​​the first pathology slide image was determined. Based on the most prevalent immunophenotype in the region, patients will likely respond to immunotherapy. In another embodiment, the immune anticancer drug response prediction unit 240 using an immunophenotype of at least a portion of the first pathology slide image to identify a first generating an immunophenotype map for at least a portion of the region within the pathology slide image; By inputting the generated immune phenotype map into the anticancer drug response prediction model, it is possible to predict the patient's immune response. It is possible to generate a prediction result as to whether or not a patient will respond to cancer drugs.

[0075] Here, the immunophenotype map is a map showing at least a part of the area in the first pathology slide image. To refer to a collection of regions of interest generated by classifying them into one of three immunophenotypes For example, an immunophenotype map can be created by comparing the immunophenotype of each of multiple regions of interest with the pathology sequence. Additionally or alternatively, immunophenotypes may be referred to as maps displayed on a single image. The phenotype map contains information associated with the three immunophenotypes (e.g., artificial neural network immunophenotyping). The score for the immune phenotype output from the class model, the threshold (or cutoff) for the immune phenotype, immunophenotypic features, such as immune cell density values ​​applied to the cut-off It may include an immune phenotype feature map.

[0076] In yet another embodiment, the immunotherapy response predictor 240 may be configured to predict the number of patients associated with the patient. 2. Pathology slide image (e.g., IH of the corresponding area in the first pathology slide image) C) Expression of biomarkers (e.g., PD-L1) from stained pathology slide images At this time, the immune anticancer drug response prediction unit 240 can obtain information about the first pathology. The immunophenotype of at least a portion of the region in the image or information associated with the immunophenotype and based on at least one of the information and information regarding expression of biomarkers, the patient is A predictive outcome can be generated for whether a patient will respond to a drug, where the expression of biomarkers is The information to be generated is obtained by inputting the second pathology slide image into the artificial neural network expression information generation model. It can be generated by:

[0077] In FIG. 2, the information processing system 100 includes a target item detection unit 210, a region of interest determination unit 212, and a target item detection unit 214. The immunological phenotyping unit 220, the immunophenotyping unit 230, and the immunological anticancer drug response prediction unit 240 are However, the present invention is not limited to the above, and some components may be omitted or other components may be added. The information processing system 100 further includes an output unit (not shown), which outputs one or more A detection result for the target item, at least a portion of the area in the first pathology slide image. Immunophenotype, information associated with the immunophenotype, and whether a patient will respond to immunotherapy and outputting at least one of a predicted result of the immune cell density in one or more regions of interest. For example, the output unit may select at least one immunosuppressant suitable for the patient from among a plurality of immunosuppressants. Information about cancer drugs can be output.

[0078] FIG. 3 illustrates a method 300 for predicting response to an immunological anti-cancer drug according to one embodiment of the present disclosure. 1 is a flowchart. In one embodiment, a method for predicting response to an immunological anti-cancer drug 30 0 is performed by a processor (e.g., at least one processor of an information processing system). The method 300 for predicting a response to an immune anti-cancer drug can be performed by a processor. The process can start by receiving a ride image (S310). , one or more items of interest can be detected in the first pathology slide image (S32 For example, the processor may utilize an artificial neural network object detection model to detect a first disease. One or more target items can be detected within the slide image.

[0079] The processor then generates a first Immunophenotype or immunophenotype-related data for at least a portion of a region in a pathology slide image At least one of the pieces of information obtained from the first pathology slide image can be determined (S330). At least a portion of the page may contain one or more target items, and one or more target Target items can include items associated with cancer and immune cells. The processor detects cancer-associated regions in at least a portion of the first pathology slide image. Calculate at least one of the number, distribution, or density of immune cells in the tagged item, and and determining whether the first pathology slide image is a marker based on at least one of the number, distribution, or density of the detected immune cells. and / or information associated with the immunophenotype of at least a portion of the region within the page. can be decided.

[0080] The processor determines an immunophenotype or A first pathology slide image is generated based on at least one of the information associated with the immunophenotype. It can generate predictive results for whether a patient associated with a given page will respond to an immunotherapy. In one embodiment, the processor performs a step of extracting the entire area of ​​the first pathology slide image (S340). Based on the most prevalent immunophenotype in the region, patients will likely respond to immunotherapy. In another embodiment, the processor may generate a predicted outcome for the first pathology slide. The immunophenotype map for at least a portion of the region in the image is used as an immunoprediction model for anticancer drug responses. By inputting data into the database, a prediction can be generated as to whether a patient will respond to an immunotherapy. In yet another embodiment, the processor is configured to: Using information related to the immunophenotype of some regions, the first pathology slide image and generating an immunophenotype feature map for at least a portion of the region within the target region, and By inputting the generated immune phenotype feature map into Dell, patients can respond to immune-mediated anti-cancer drugs. It is possible to generate a prediction result as to whether or not

[0081] In yet another embodiment, the processor may further include: Obtain information about biomarker expression from the first pathology slide image. Based on information about the immunophenotype and biomarker expression of at least a portion of the It can generate predictive results on whether a patient will respond to an immunotherapy. The processor receives a second pathology slide image associated with the patient and generates an artificial neural network expression information. By inputting the second pathology slide image into the information generation model, the expression of biomarkers can be predicted. Here, the second pathology slide image can generate information related to the first pathology slide image. The image can correspond to a pathology slide image for the corresponding area.

[0082] FIG. 4 illustrates a method for detecting an item of interest in a pathology slide image according to an embodiment of the present disclosure. FIG. 1 shows an example of determining the region of interest and determining the immunophenotype of the region of interest. To predict whether a patient will respond to anti-cancer drugs, users (e.g., doctors, researchers, etc.) The patient's tissue (e.g., immediately prior to treatment) is acquired and one or more pathology slide images are taken. For example, a user can perform H&E staining on acquired patient tissue and generate H&E images. Pathological slide images can be obtained by digitizing the &E stained tissue slides using a scanner. As another example, a user can generate an image (e.g., a first pathology slide image). The acquired patient tissue was stained with IHC and the IHC-stained tissue slides were scanned. By digitizing the pathology slide image (e.g., a second pathology slide) image).

[0083] The processor (e.g., at least one processor of an information processing system) may include an artificial neural network. Using the network object detection model, pathology slide images (e.g., digitized Various target items can be detected from H&E tissue images (whole slide images) (S41 0), where the artificial neural network target item detection model is based on a set of reference pathology slide images. This can apply to a model trained to detect one or more reference target items. The processor targets cell-level items such as tumor cells and lymphocytes. hocyte, macrophage, dendritic cell, fibrocyte ( fibroblasts, endothelial cells, etc. may be detected. Additionally or alternatively, The processor selects cancer stroma, cancer epithelium, etc. as target items in the area unit. (cancer epithelium), cancer area, normal area (e.g. It is possible to detect areas containing normal lymph node architecture.

[0084] In one embodiment, the processor is trained to detect cellular items of interest. Using the artificial neural network model, we identify cell-level target items in pathology slide images. Additionally or alternatively, the processor may detect area-based items of interest. We use an artificial neural network model trained to identify regions in pathology slide images. It can detect target items in units of cells. The neural network model and the artificial neural network model for detecting area-based target items are separate models. Alternatively, the processor may process cell-based and area-based target items. Pathological screening was performed using an artificial neural network model trained to simultaneously detect multiple target items. Cell-based target items and / or area-based target items in the ride image That is, by using one artificial neural network model, the target item and the cell unit can be detected. It can detect all target items by area.

[0085] The cell-based and region-based target item detection results complement each other. In one embodiment, the processor may process the detection results for tumor cells (all In other words, based on the results of cell-level target item detection, The cancer area (i.e., area-wise target item detection result) is estimated by using the Conversely, the processor may modify or change the detection results for cancerous regions. Based on the results (i.e., region-wise target item detection results), In this case, tumor cells (i.e., cell-level target item detection results) are estimated or estimated. In other words, the cancer area is a target item in the area unit. The difference is that tumor cells are classified as cell-level target items, but tumor cells are classified as cell-level target items. The included area is ultimately a cancer area, so the target item detection results for each cell and each area are The target item detection results can be complemented with each other.

[0086] For example, when detecting area-based target items, detailed cell-based target items ( In other words, it causes detection errors such as missing tumor cells, but the processor The difference depends on whether the cells in the area detected as cancerous are actually detected as tumor cells. Conversely, areas detected as tumor cells can be detected as cancerous areas. In this way, the processor can complement the detection result by detecting whether or not the cell is detected. Using the results of target item detection in each area and the results of target item detection in each region, This can complement the location and minimize errors in the detection results.

[0087] The processor processes pathology slide images (e.g., H&E stained whole slide images). Within the tumor, a cancer region (e.g., a cancer matrix), a cancer containing cancer stroma and / or tumor cells, Associated items and immune cells can be detected (S410). One or more regions of interest can be determined within the pathology slide image (S420). For example, the processor may divide a pathology slide image into N grids (where N is any natural number). ) into multiple patches (e.g., 1 mm 2 A patch of the size ) at least some patches (e.g., items associated with cancer and / or immunizations) The patch in which the immune cells were detected is then defined as a region of interest (e.g., at least a portion of the pathology slide image). area).

[0088] The processor may select items and / or immune cells associated with cancer in one or more regions of interest. Based on the cell detection results, the immunophenotype of one or more regions of interest can be determined (S430). Here, one or more regions of interest may include multiple pixels, and Cancer-associated items and / or immune cell detection results are included in one or more regions of interest. For each of the cancer-associated items and / or immune cells in each of the plurality of pixels, In one embodiment, the processor may include a predictor for one or more regions of interest. In the area, at least one of the number, distribution, or density of immune cells in the items associated with cancer was and calculating one of the number, distribution, and density of immune cells based on at least one of the calculated number, distribution, and density of immune cells. The immunophenotype of one or more regions of interest can be determined. For example, the processor can Within the area, the density of immune cells in the cancer area (lymphocytes in tumor area) and cancer stroma were measured. The density of immune cells (lymphocytes in stroma area) in the tumor was calculated, and the density of immune cells in the tumor area was calculated. and determining whether or not the tumor is in the region of interest based on at least one of the following: density of immune cells within the tumor stroma or the density of immune cells within the tumor stroma. The processor can then determine the immunophenotype of the region. The immune cell counts can be used to characterize the immune phenotype of one or more regions of interest as immune activation, immune exclusion, or It can be determined as an immune deficiency.

[0089] As shown in FIG. 4, when the density of immune cells in the cancer region is equal to or greater than a first threshold density, one or more The immunophenotype of the area of ​​interest above can be determined as immune inflamed. The density of immune cells in the region is less than a first threshold density, while the density of immune cells in the cancer stroma is If the density is equal to or greater than a second threshold density, the immunophenotype of one or more regions of interest is immunoexclusion. In addition, the density of immune cells in the cancer area is determined to be less than the first threshold density. and at the same time, if the density of immune cells in the cancer stroma is less than a second threshold density, one or more The immunophenotype of the above area of ​​interest can be determined as immune desert.

[0090] wherein the first threshold density is a threshold density for each of a plurality of regions of interest in a plurality of pathology slide images. The distribution of immune cell density within the cancer area can be determined. The second threshold density is a threshold density for determining whether a cancer stain is present in each of the plurality of regions of interest in the plurality of pathology slide images. The first threshold density can be determined based on the density distribution of immune cells within the stroma. For example, the first threshold density can be determined based on the density distribution of immune cells within the stroma. In each region of interest in the pathology slide image, the density of immune cells within the cancerous area was measured. The distribution can fall into the top X% (where X is a number between 0 and 100) of density values. Similarly, the second threshold density is set to a value that is equal to or less than the threshold density of the cancer stroma in each of the regions of interest of the plurality of pathology slide images. Top Y% of immune cell density distribution within the region (where Y is a number between 0 and 100) The values ​​of X and / or Y can correspond to the density values ​​of biological samples according to the prior art. It can be determined by a standard or any standard.

[0091] The first and second threshold densities may be the same or different. The first threshold density or the second threshold density may each include two or more threshold densities. In this case, the first threshold density or the second threshold density is a threshold density for the upper region and a threshold density for the lower region. For example, the density of immune cells in the cancer region may be within the first threshold density. , if the density of immune cells in the cancer region is equal to or greater than the threshold density for the upper region, In addition, the density of immune cells in the cancer region is within the first threshold density. , if the density of immune cells in the cancer region is less than the threshold density for the subregion, This may be the case where the amount is less than

[0092] FIG. 4 shows the generation of immunophenotyping of the region of interest (S430). Without being limited thereto, information associated with immunophenotypes can also be generated. The processor generates information associated with the immunophenotype used to determine the For example, such elements may include those associated with cancer. The information may include at least one of the number, distribution, or density of immune cells within the item.

[0093] FIG. 5 is a diagram illustrating an example of generating an immunophenotyping result 540 according to one embodiment of the present disclosure. In one embodiment, a processor (e.g., at least one processor in an information processing system) The artificial neural network immunophenotyping model 530 is configured to generate a first pathology slide image. inputting features for at least a portion of the first pathology slide image; The immunophenotype of at least a portion of the region can be determined. The model 530 calculates features for at least a portion of the region in the reference pathology slide image. By inputting the above information, the immunophenotype of at least a part of the region in the reference pathology slide image can be determined. The diversion device is trained to determine whether the patient is immune active, immune excluded, or immune deficient. Cut.

[0094] Here, the features for at least a portion of the region in the first pathology slide image are: for one or more target items in at least a portion of the first pathology slide image; Identifying statistical features (e.g., at least some regions) in the first pathology slide image. density or number of target items), geometric features for one or more target items (e.g., features containing relative position information between specific target items, etc.) and / or Image features (e.g., A plurality of pixels extracted from at least a portion of the pathology slide image of one the image corresponding to at least a portion of the first pathology slide image; Additionally or alternatively, the first pathology slide may include a The features for at least a portion of the image are Statistical features for one or more target items in at least a portion of the area, one or more Geometric features for the target item or at least Among the image features that correspond to a partial area, a frame that connects two or more features is It may include features.

[0095] The immunophenotyping unit 230 of the information processing system detects a small number of phenotypes in the first pathology slide image. and receiving input of at least a part of the area 510 in the first pathology slide image. The immunophenotype of the region 510 can be determined. At least a partial region 510 in the first pathology slide image is The image may include detection results for the target item in at least a portion 510 of the page. As shown in the figure, the feature extraction unit 520 of the immunophenotyping unit 230 extracts the first pathology receiving at least a portion 510 of the slide image; and extracting features for each of at least a portion of the region 510 in the image and generating an artificial neural network immune representation; This allows the artificial neural network immunophenotyping model 530 to be input. determine the immunophenotype of each of at least a portion 510 in the first pathology slide image. The immunophenotyping results 540 can be output.

[0096] In FIG. 5, the feature extraction unit 520 is included in the immunophenotyping unit 230, and the immunophenotyping The determining unit 230 receives at least a portion 510 of the first pathology slide image. However, the present invention is not limited to this. For example, the immunophenotyping unit 230 may be configured as follows: The features for each of at least a portion 510 in the pathology slide image are The received data is input to the artificial neural network immunophenotyping model 530 to perform immunophenotyping. The measurement result 540 can be output.

[0097] In FIG. 5, an artificial neural network immunophenotyping model 530 is used to identify the phenotypes in a first pathology slide image. Although it is stated that feature input for at least some areas is accepted, this is not limited to this. and receiving and processing an input of at least a partial area within the first pathology slide image. Additionally or alternatively, an artificial neural network immunophenotyping model may be used. The rule 530 not only provides the immunophenotyping result 540 but also information associated with the immunophenotype. In such a case, the artificial neural network immunophenotyping The class model 530 is a feature for at least a portion of the region in the reference pathology slide image. By inputting at least a partial area in the image of the reference pathology slide, Immunophenotype or immunophenotype-associated with at least a portion of a region within a slide image It can be trained to determine at least one of the pieces of information.

[0098] FIG. 6 is an example of generating a response prediction result 630 to an immunological anticancer drug according to one embodiment of the present disclosure. 1 is a diagram illustrating a processor for processing a pathology slide image (e.g., a first pathology slide image) The immunophenotype can be determined for each of multiple regions of interest within the image. The immunophenotype of each of the multiple regions of interest (i.e., the immunophenotype for each of the multiple regions of interest) Based on the immunophenotype, patients were correlated with pathology slide images to determine their response to immunotherapy. For example, the processor may generate a prediction result for whether or not a pathology slide image is to be used. Based on the distribution of immunophenotypes in the image, the patient associated with the pathology slide image It can generate predictive results for whether a patient will respond to an immunotherapy.

[0099] In one embodiment, the processor is configured to: The most abundant immunophenotype within the entire area of ​​the processed slide image can be determined. The processor then selects the most frequently included immune region within the entire area of ​​the pathology slide image. Based on the phenotype, a predictive outcome can be generated as to whether a patient will respond to an immunotherapy anti-cancer drug. For example, the most abundant immunophenotype in the entire area of ​​a pathology slide image is the immunophenotype If so, the processor determines whether the patient associated with the pathology slide image is immune. It can be predicted that the patient will respond to anticancer drugs (i.e., the patient will be a Responder). Unlike the immunophenotypes that were most prevalent within the entire area of ​​the pathology slide image, the immunophenotypes were If it is exclusion or immunodeficiency, the processor associates it with the pathology slide image. The patient does not respond to the immunotherapy (i.e., the patient is a non-responder). It can be predicted that:

[0100] In another embodiment, the processor is configured to: The immunophenotype of the region is used to identify a region of interest in at least a portion of the first pathology slide image. For example, the processor may generate an immunophenotype map 610 corresponding to a first pathology slide. Based on the immunophenotype of at least a portion of the image (e.g., each of a plurality of regions of interest), The immunophenotype map is generated using a set of regions of interest. wherein the immunophenotype map can include an immunophenotype feature map, The phenotypic feature map not only identifies the immunophenotype of the region of interest but also the associated information. For example, the processor may further include: In addition, the immunophenotype and immunophenotype score for some regions (e.g., artificial neural network immunophenotyping model) the number and distribution of immune cells in items associated with cancer, such as scores output by the , density (e.g., density of immune cells in the cancer area or density of immune cells in the cancer stroma, etc.) An immunophenotypic feature map containing information about the

[0101] Thereafter, as shown in the figure, the processor generates a model 620 for predicting the response to an immunological anti-cancer drug. By inputting the generated immunophenotype map 610, it is possible to determine whether a patient will respond to an immunotherapy or anticancer drug. For example, the processor may generate a prediction 630 for a response to an immunosuppressant. By inputting the immune phenotype map 610 into the predictive model 620, the pathological slide image Classify patients associated with a page as Responder or Non-responder In this case, the immune anticancer drug response prediction model 620 can predict the spatial information between immune phenotypes. Taking into account the immunological and / or location information, the patient's responsiveness can ultimately be predicted. The anticancer drug response prediction model 620 is based on the immunophenotype map and the pathology slides. It is possible to summarize at least one global feature of the image. For this purpose, the immune anticancer drug response prediction model 620 is based on the immune phenotype map. Based on this, graphs can be formed and various pooling methods (e.g., RNN, CNN, simple Net average pooling, sum pooling, max pooling , bag-of-words / VLAD / Fisher, etc.) It can be achieved.

[0102] Here, the immune anticancer drug response prediction model 620 is input with a reference immune phenotype map. By using the data, the system was trained to generate predictions for the response to the reference immune anticancer drug. This can be a statistical model and / or an artificial neural network model. For example, the actual treatment of multiple patients Based on the results, users (e.g., doctors) can perform annotation tasks to improve the quality of each patient's Responder or Non-responder pathology slide images The processor can then determine the label for each patient's pathology slide image (or each Using immunophenotype maps and labels for patient pathology slide images, immunoantibodies As another example, the processor may train a model 620 for predicting response to cancer drugs. By inputting the current graph, you can select Responder or Non-responder The immune anticancer drug response prediction model 620 can be trained to classify the The processor represents or characterizes the immunophenotype corresponding to the region of interest in the immunophenotype map. It can be converted into an immunophenotype graph and generated.

[0103] In another embodiment, the processor is configured to: The information associated with the immunophenotype of the region is used to identify the small area in the first pathology slide image. The processor may then generate an immunophenotypic feature map for at least a portion of the region. Input the generated immune phenotype feature map into the immune anticancer drug response prediction model 620. This allows for the generation of predictions as to whether a patient will respond to an immunotherapy. The immune anticancer drug response prediction model 620 receives a reference immune phenotype feature map as input. This allows the system to learn to generate reference prediction results.

[0104] FIG. 7 shows information on biomarker expression and / or immunophenotype according to one embodiment of the present disclosure. FIG. 10 is a diagram showing the results of analyzing the prediction performance of current information on the response to an immunological anticancer drug. (e.g., at least one processor of an information processing system) associated with the patient Pathology slide images (e.g., a first pathology slide image and a corresponding second pathology slide image) Information on biomarker (e.g., PD-L1) expression can be obtained from the PD-L1 image. At this time, the processor performs an immunodetection of at least a portion of the area in the first pathology slide image. Based on information on phenotype and PD-L1 expression, patients may or may not respond to immunotherapy. Here, information on PD-L1 expression can be used to generate predictive results for TPS ( At least one of the Tumor Proportion Score (CPS) or Combined Proportion Score (CPS) Further, the processor may include a TMB (Tumor Mutation Burden) of the patient. Numerical value, MSI (Microsatellite instability) numerical value, HRD (Homologous Recombination Obtain additional patient data, such as immune deficiency scores, to assess the patient's immune response to anticancer drugs. You can also predict whether it will happen or not.

[0105] In one embodiment, the processor may further include a second pathology slide image (e.g., an IHC stained slide image) The artificial neural network expression information generation model receives the second pathology slide image. By inputting the slide image, information on PD-L1 expression can be generated. For example, The processor utilizes an artificial neural network expression information generation model to generate a second pathology slide image. The location of tumor cells in at least a part of the first pathology slide image included in the Detect at least one of the following: lymphocyte location, macrophage location, or PD-L1 expression By doing this, information on the expression of PD-L1 can be generated. The generative model uses reference pathology slide images as input to generate PD-L1 expression data. In another embodiment, the prototyping may correspond to a model trained to generate reference information. The processor allows a user (e.g., a pathologist) to extract pathology slide images (e.g., IHC stained images). Receive information on PD-L1 expression calculated directly from the pathology slide images can.

[0106] For example, among the information on PD-L1 expression, TPS can be calculated using the following formula 1. Cut.

[0107]

number

[0108] Based on the calculated TPS value and the TPS standard value (cut-off value), Based on this, patients can be classified into one or more groups, e.g., for non-small cell lung cancer, TPS < 1% If TPS is 1% or less, patients are classified as "PD-L1 expression-free group," and if TPS is 1% or less, patients are classified as "PD-L1 expression-free group." Patients were classified as the "PD-L1 expression group," and patients were classified as the "PD-L1 expression group" if TPS ≥ 50%. The standard values ​​for TPS are the above values ​​(1%, 49%, 50%). It is not limited and may vary depending on the type of cancer, the type of PD-L1 antibody, etc.

[0109] As another example, the CPS of the PD-L1 expression information can be calculated by the following formula 2: In this case, the maximum upper limit of CPS is 100, and it is calculated by the formula 2. If the CPS is greater than 100, the CPS can be determined as 100.

[0110]

number

[0111] Based on the calculated CPS value and the standard value (cut-off value) of CPS, Based on this, patients can be classified into one or more groups. For example, 10% can be used as the reference value (cutoff) for CPS. The patients were classified into groups with CPS ≥ 10% and CPS < 10% as a cut-off value. However, the standard value of CPS may vary depending on the type of cancer, the type of PD-L1 antibody, etc. (Cutoff value) can be different.

[0112] The illustrated table 710 shows the immunophenotypes (i.e., representative immunophenotypes) classified as immunoactive. Patients classified as inflamed (Table 710) and patients classified as immunosuppressed (Table 710) As a result of comparing the response of immune anticancer drugs with those of non-inflamed patients, The patients with L1 TPS of 1% to 49% and those with TPS of 50% or more were objectively compared. Overall response rate (ORR) and median progression-free survival (MPFS) Therefore, the processor is Using information on both PD-L1 expression and PD-L1 expression, we can determine whether a patient will respond to immunotherapy. In table 710, N represents the number of patients, CI stands for confidence interval, and HR stands for hazard ratio. Represents o.

[0113] The graph 720 also shows whether the patient is responding to an immunosuppressant (i.e., whether the patient is responding to an immunosuppressant). When using the TPS of PD-L1 to predict whether a patient will respond to a cancer drug (i.e. , "PD-L1 TPS"), when immunophenotyping is utilized (i.e., "H&E"), T When using MB (i.e., "TMB"), when using all of the above information (i.e., Specificity of each of the "Ensemble of three" ROC (Receiver Operating Characteristic) curves for sensitivity According to the graph 720, the AUROC (Area Unser ROC curve) of TMB is The AUROC of PD-L1 TPS was calculated to be 0.6404, and the AUROC of PD-L1 TPS was calculated to be 0.7705. The AUROC for H&E was calculated to be 0.7719, and the Ensemble of three The AUROC was calculated to be 0.8874. PD-L1 TPS and TMB) and immunophenotyping (i.e., H&E) were all utilized. It has been shown that its performance is the best when used to predict whether or not a patient will respond to an immunotherapy anticancer drug. It was proven.

[0114] FIG. 8 shows an immunophenotype and a comparison of CD3-positive cells and CD8-positive cells according to one embodiment of the present disclosure. , showing a correlation between information on the expression of FOXP3-positive cells and CD68-positive cells. A processor (e.g., at least one processor in an information processing system) ) is a method for generating activity scores based on the immunophenotype of each of multiple regions of interest in a pathology slide image. The Inflamed Score (IS) can be calculated using the pathology slide image. Among multiple regions of interest, the immunophenotype was determined as immunoreactive, and the region of interest containing the tumor ( For example, a 1mm 2 is defined as the ratio of the area of ​​the For example, the activity score can be calculated using the following formula 3.

[0115]

number

[0116] The cut-off value of the activity score was set at 20%, and the number of patients (N(patients)) for 10 cancers was calculated. As a result of comparing the overall survival time after immunotherapy with anticancer drugs in patients with lung cancer (number of patients = 1,013), Even in melanoma and head and neck cancer, the activity score is within the standard range. It has been proven that patients who meet the above criteria have a longer overall survival time after immunotherapy with anticancer drugs. In addition, the activity scores were also obtained when lung cancer was included (N=519) or excluded (N=494). Patients with higher core scores tend to have a more favorable prognosis for immunotherapy with anticancer drugs.

[0117] In this study, H&E stained specimens of patients were used for pathological verification of the immunophenotype. Pathology slide image 1mm 2 Divide the image into multiple patches and measure the cancer area within each patch. Based on TILs from tumor-infiltrating lymphocytes (L) and cancer stroma The immunophenotype was determined as follows. Based on the phenotype, a representative immunophenotype (i.e., the most prevalent immunophenotype in the pathology slide images) was selected. For example, if the activity score was 33.3% or higher, the pathology slide was The representative immunophenotype of the image can be determined as immunoreactivity.

[0118] Next, we performed immunotherapy using Mul Tiplex IHC staining was performed to detect biomarkers CD3, CD8, CD20, and CD Staining for 68, FOXP3, CK, and DAPI was performed. Normalized The number of cells was calculated by dividing the total number of positive cells by the number of DAPI-positive cells in the pathology slide image. For example, the normalized number of CD3-positive cells can be calculated based on the following formula: It can be calculated as follows.

[0119]

number

[0120] The graphs 810, 820, 830, and 840 shown are H&E stained images, as previously described. Immunophenotype determined from pathology slide images (i.e., representative immunophenotype), and Correlations between biomarker expression confirmed by multiplex IHC analysis According to the first graph 810 and the second graph 820, it is clear that the compound plays an anti-tumor role. Normalized CD3- and CD8-positive cells show increased immunoreactivity compared to other immunophenotypes. It appears very abundantly in the pathology slide images corresponding to )=1.57, P=0.0182)(CD8:FC=1.24, P=0.0697).

[0121] On the other hand, according to the third graph 830, FOXP3-positive cells associated with immunosuppressive activity The cells appear abundantly in pathology slide images consistent with immune exclusion (FC = 1.26, P =0.0656). Also, according to the fourth graph 840, CD68-positive cells are immunodeficient. It appears abundantly in the pathology slide images corresponding to deficiency (FC=1.76, P=0.0046 7) This allows the processor to generate the results determined from the H&E stained pathology slide images. Based on the immunophenotype, meaningful predictions can be made as to whether a patient will respond to immunotherapy. can be generated.

[0122] FIG. 9 is a graph showing a prediction of whether a patient will respond to an immunotherapy anti-cancer drug according to one embodiment of the present disclosure. Graphs 910, 920 show the performance of various methods. a processor (e.g., an information processing system) to generate a predicted result for The processor (one processor) processes a first pathological slide image of a patient according to the above-described embodiment. Immunophenotyping can be performed on samples (e.g., H&E-stained pathology slide images). Here, the immunophenotype for the pathology slide image is determined by one of the The immunophenotypes of the above regions of interest and the immunophenotypes most frequently included in the pathology slide images ( i.e., a representative immunophenotype, an immunophenotype map, etc.

[0123] Additionally or alternatively, generating a predictive outcome for whether a patient will respond to an immunotherapy. To do this, the processor may generate a second pathology slide image (e.g., an I Information on PD-L1 expression (e.g., T) from HC-stained pathology slide images For example, information on PD-L1 expression can be obtained directly by the user. As another example, information on PD-L1 expression can be used to calculate The machine uses an artificial neural network expression information generation model to generate the second pathology slide image of the patient. This can apply to information generated from

[0124] In one embodiment, the processor receives the second pathology slide image and generates an artificial neural network. By inputting the second pathology slide image into the expression information generation model, the expression of PD-L1 was Additionally or alternatively, the processor may generate information relating to the artificial neural network. The synthesis model is used to identify the first pathology slide image contained in the second pathology slide image. The location of tumor cells, lymphocytes, macrophages, or other tissue in at least a portion of the area By detecting at least one of the PD-L1 expression status, Here, the second pathology slide image can generate information based on the first pathology slide image. This may include pathology slide images of the same area from the same patient as the patient. The processor extracts in-house control tissue from IHC-stained pathology slide images. The patient's tissue area is divided into one or more regions of interest, and an artificial Using the transmembrane expression information generation model, tumor cells, lymphocytes, and macrophages were identified from the ROI. Detect location and PD-L1 expression (e.g., PD-L1 negative, PD-L1 positive) This allows the calculation of TPS and / or CPS.

[0125] Here, the artificial neural network expression information generation model is based on the input of a reference pathology slide image. and a model trained to generate reference information about PD-L1 expression. To generate / train an artificial neural network expression information generation model, the processor uses multiple learning Label information (or annotation) for a pathology slide image and multiple training pathology slide images Here, label information about the training pathology slide images can be received. may be generated by a user performing an annotation task.

[0126] The illustrated ROC curve graph 910 shows the effectiveness of the ROC curve in predicting response to immunotherapy. When using the TPS calculated using the artificial neural network expression information generation model (i.e., AI PD-L1 TPS), when using TPS calculated directly by a human (i.e. , "Human PD-L1 TPS"), when using TMB (i.e., "Tum or Mutation Burden) ROC curves for the ROC curves (sitivity) are shown in the ROC curve graph 910. The AUROC of AI PD-L1 TPS was calculated to be 0.784, and the Human P The AUROC for D-L1 TPS was calculated to be 0.764, and the AUROC for TMB was 0.71 0. In other words, the AI PD-L1 TPS was the most effective for specificity-sensitivity It has been proven that it has excellent performance.

[0127] In one embodiment, the processor is configured to: Based on information on the immunophenotype and PD-L1 expression in the area, patients were assessed for their response to immunotherapy. Specifically, it can predict whether a patient will respond to an immunotherapy or anti-cancer drug. When predicting, H&E stained pathology slide images (e.g., first pathology slide images) are used. Immunophenotype determined through IHC stained pathology slide images Information about PD-L1 expression obtained from (e.g., second pathology slide images) The use of immunophenotype and PD-L1 expression information together rather than using either alone Here, the performance was excellent when H&E stained pathology slide images were used. The immunophenotypes obtained may include representative immunophenotypes of pathology slide images, immunophenotypes of one or more regions of interest, It may be referred to as a phenotype, an immunophenotype map, or the like.

[0128] Comparison of accuracy rates of AI-based biomarkers shown According to the I-powered Biomarker Graph 920, immunotherapy has been shown to be effective in predicting response to immunotherapy. When using PD-L1 TPS calculated directly by a human ("Human PD-L1" The accuracy of the TPS is approximately 0.68, while the accuracy of the TMB is approximately 0.6 9, and the accuracy of immunophenotyping ("AI H&E") is approximately 0.715. If applicable, when using the TPS calculated using the artificial neural network expression information generation model (" The accuracy of AI PD-L1 TPS is approximately 0.72. When using both D-L1 TPS and AI PD-L1 TPS ("Human P The accuracy of "D-L1 TPS + AI PD-L1 TPS" is approximately 0.715, and When using MB and AI PD-L1 TPS together (TMB+AI PD-L1 The accuracy of AI H&E and AI PD-L1 TPS is approximately 0.72. When used together ("AI H&E + AI PD-L1 TPS"), the accuracy is approximately 0. 77. In other words, AI H&E+A is a valuable tool for predicting the response to immunotherapy. PD-L1 TPS was proven to have the best accuracy. .

[0129] FIG. 10 shows the results of the prediction process for whether or not a patient will respond to an immunosuppressant according to one embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of outputting the generated results. At least one processor of the information processing system terminal and / or at least one processor of the user terminal The first pathology slide image is then processed by the first pathology slide image processor. immunophenotype of at least a portion of the image, information associated with the immunophenotype, immunoantibodies A predicted result of the response to the cancer drug or at least a part of the area in the first pathology slide image The information processing system (e.g., information processing system) can output at least one of the following: a prediction of whether or not the processor (at least one processor of the processing system) will respond to an immunological anti-cancer drug The results generated during the measurement process are provided to the user terminal, and the user terminal then For example, the user terminal can output the data via a user interface as shown in the figure. The results can be displayed on the screen.

[0130] In one embodiment, the user terminal receives the pathology slide image and / or the first pathology slide. The target item detection result 1010 for at least a partial area in the image can be output. For example, the user terminal may store a pathology slide containing label information for each of the target items. The user terminal can output a mask (ma) for the target item in the area unit. sk), and for cell-level items, the center point of the cell nucleus or the bounding box (bo In another embodiment, the user can output a pathology slide image displayed in a box. The user terminal scans the entire pathology slide image or at least a portion of the first pathology slide image. Minimaps such as immunophenotype maps (or immunophenotype feature maps) for a portion of the area are also available. , by representation methods such as heatmap and / or label map In another embodiment, the user terminal can visualize and output the first pathology slide image. As a result of predicting whether or not an immune anticancer drug will respond to at least a portion of the image, Based on the current type, activity score, and response score (response score and / or non-response score) Based on this, a response / non-response score map was generated as a hit map and / Or it can be visualized and output using a representation method such as a label map.

[0131] In yet another embodiment, the user terminal may receive all and / or a portion of the pathology slide image. The density of immune cells in each area can be output. For example, the user terminal can output the density of immune cells in each area on a pathology slide. Outputs numerical values ​​for the density of immune cells by region for the whole image and / or a part of the image. In another embodiment, the user can The terminal can output the distribution of the patient's immunophenotypes in the form of a circle plot. As shown in the figure, the user terminal displays a bar graph of the density of immune cells by region and the patient's The analysis results 1020 can be output, including a pie chart for the distribution of the subject's immunophenotype.

[0132] The user terminal uses the information processing system to predict whether or not the patient will respond to an immunotherapy drug. The generated results can be received and the received results can be output. This allows the user to The results generated during the process of predicting whether a patient will respond to cancer drugs can be visually and intuitively recognized. In Figure 10, the results generated in the prediction process are displayed on a user interface running on the user terminal. The results generated may be displayed in a variety of ways, including but not limited to visual output via a This can be provided to the user.

[0133] 11 to 15 show the results of the immunoassay for the anticancer drug according to another embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of outputting a result generated in a prediction process. The system (e.g., at least one processor of an information processing system) is configured to react with an immunological anti-cancer agent. A report 1100 containing the results generated during the prediction process as to whether or not the 200, 1300, 1400, 1500 can be generated and provided. Ports 1100, 1200, 1300, 1400, and 1500 are used by user terminals and / or It can be provided as a file, data, text, image, etc. that can be output through an input device. Here, reports 1100, 1200, 1300, 1400, and 1500 are shown in Figure 10. It may include at least one of the results described above.

[0134] In one embodiment, the information processing system is configured to: Score for the responder (e.g., a score between 0 and 1) (e.g., if the patient Results showing probability of being a ponder and / or probability of being a non-responder Additionally or alternatively, the information processing system may generate a report that includes the Responsibility Information on cut-off values ​​for determining responder / non-responder Additionally or alternatively, the information processing system may generate a report containing pathology information. Distribution of immunophenotype and / or TIL density within the image (e.g., Min, Max, A For example, the information processing system can generate a report containing three immunophenotypes. The distribution of TIL density and Min, Max for each of the regions of interest classified as Additionally or alternatively, the information processing system may generate a report including the average value of the disease. The immunophenotype map shows regions of interest within the slide image classified into three immunophenotypes. You can generate a report.

[0135] The information processing system may include pathological images (e.g., pathological slices) before and / or after immunotherapy. By implementing some embodiments of the present disclosure, it is possible to achieve acquired resistance against We can understand the mechanisms of resistance and provide treatment plans tailored to each resistance mechanism. For example, the processor may record pathological images of patients who have received immunotherapy and the results of the anti-cancer drug administered to the patients. By performing the analysis using input data such as the type of treatment administered to the patient, In one embodiment, the results of treatment with each of immunological anti-cancer drugs and / or other immunological anti-cancer drugs can be predicted. In this case, the information processing system performs a predictive analysis of whether a patient will respond to an immunotherapy or not. Based on this, at least one immunological anticancer agent that is compatible with the patient is selected from among the plurality of immunological anticancer agents. For example, the information processing system can output information when a patient is determined to be a Responder. If approved, a report will be sent containing the immunotherapy product and / or product combination with the highest likelihood of response. It can be generated.

[0136] As shown in the figures, the immunoantibodies The results generated during the cancer drug response prediction process are presented in text and / or images. For example, as shown in Figure 11, Report 11 00 is a pathology slide image, patient information, basic information and / or predicted results (Responder / Also, as shown in Figure 12, Report 12 00 provides graphs showing immunophenotypic ratios, numerical values, and information on TIL density (e.g., density As shown in FIG. 13, the report 1300 may include statistics. statistical results (e.g., TCGA PAN-CARCINOMA STATISTICS) and / or graphs showing analytical results , clinical notes, etc. As shown in FIG. 1400 may contain information about references (e.g., ACADEMIC REFERENCES). As shown in FIG. 15, the report 1500 displays the results (e.g., , immunophenotype map images, FEATURE STATISTICS, etc.) and / or used in the prediction process. It can include information such as:

[0137] In Figures 11 to 15, the results generated during the prediction process are output in the form of a report. However, the generated results can be provided to the user in a variety of ways, without being limited thereto.

[0138] FIG. 16 is a diagram illustrating an example of an artificial neural network model 1600 according to one embodiment of the present disclosure. The neural network model 1600 is an example of a machine learning model. ) In technology and cognitive science, statistical theory embodied in the structure of biological neural networks A learning algorithm or a structure that implements that algorithm.

[0139] According to one embodiment, the artificial neural network model 1600, like a biological neural network, uses synapses The nodes, which are artificial neurons that form a network by combining By repeatedly adjusting the weights, the error between the normal output and the inferred output corresponding to a specific input is By learning to reduce the number of iterations, we can demonstrate that machine learning models have problem-solving capabilities. For example, the artificial neural network model 1600 is based on artificial intelligence such as machine learning and deep learning. This includes any probabilistic models, neural network models, etc. used in neural learning methods. can be done.

[0140] According to one embodiment, the artificial neural network model 1600 generates a set of images from an input pathology slide image. The method may include an artificial neural network model configured to detect one or more items of interest from the Additionally or alternatively, the artificial neural network model 1600 may be configured to generate a first pathology slide as an input. a feature for at least a portion of the pathology slide image or a first pathology slide image; and determining whether at least a portion of the first pathology slide image is based on at least a portion of the first pathology slide image. determining at least one of the immunophenotype of the region or information associated with the immunophenotype; Additionally or alternatively, the artificial neural network model may include a constructed artificial neural network model. Based on the input immunophenotype map, the 1600 predicts whether a patient will respond to an immunotherapy drug. The method may include an artificial neural network model configured to generate a prediction result for whether or not Additionally or alternatively, the artificial neural network model 1600 may be configured to generate an input immunophenotype feature map. The method is configured to generate a prediction result as to whether a patient will respond to an immune anti-cancer drug based on the result. The neural network model may include an artificial neural network model.

[0141] Additionally or alternatively, the artificial neural network model 1600 may be configured to process input pathology slide images. Generate information on biomarker expression (e.g., PD-L1 expression) from It may include a constructed artificial neural network model.

[0142] The artificial neural network model 1600 is a multi-layered network consisting of nodes and connections between them. In this embodiment, the eigenvalue is embodied as a multilayer perceptron (MLP). Such an artificial neural network model 1600 may utilize one of a variety of artificial neural network model structures, including an MLP. As shown in FIG. 16, an artificial neural network model 1600 can be implemented using external input. an input layer 1620 that receives input signals or data 1610 and outputs signals corresponding to the input data; Or, an output layer 1640 that outputs data 1650, and a data structure between the input layer 1620 and the output layer 1640 It receives signals from the input layer 1620, extracts characteristics, and transmits them to the output layer 1640. The hidden layer 1630_1 to 1630_n consists of n hidden layers (where n is a positive integer). Here, the output layer 1640 receives signals from the hidden layers 1630_1 to 1630_n and outputs Output to the section.

[0143] The learning method of the artificial neural network model 1600 involves determining the solution to a problem in response to the input of a teacher signal (correct answer). There are two methods: supervised learning, which learns to optimize the decision, and supervised learning, which requires a teacher signal. In one embodiment, there is an unsupervised learning method that does not require supervision. The information processing system is configured to detect one or more items of interest from the pathology slide image. and training the artificial neural network model 1600 by supervised learning and / or unsupervised learning. For example, the information processing system may include a reference pathology slide image and one or more reference Extract one or more target items from a pathology slide image using label information about the target items. Training an artificial neural network model 1600 to detect items through supervised learning In another embodiment, the information processing system may a feature for at least a portion of the area or at least immunophenotyping at least a portion of the region in the first pathology slide image based on the portion of the region; or determining at least one of the pieces of information associated with the immunophenotype. The model 1600 can be trained by supervised and / or unsupervised learning. For example, The information processing system performs filtering on at least a portion of the region in the reference pathology slide image. Immunophenotyping or immunophenotyping of at least a portion of the area in the image of the reference pathology slide The label information relating to at least one of the types of information is used to identify the reference pathology slide. Feature or reference pathology slide image for at least a portion of the pathology slide image at least a portion of the region in the reference pathology slide image based on at least a portion of the region in the determining at least one of the immunophenotypes or information associated with the immunophenotypes of the The artificial neural network model 1600 can be trained by supervised learning.

[0144] In yet another embodiment, the information processing system is configured to generate an immunophenotype map or an immunophenotype signature. Based on the map, we are trying to generate a prediction result for whether a patient will respond to an immunotherapy. and training the artificial neural network model 1600 by supervised learning and / or unsupervised learning. For example, the information processing system may include a reference immunophenotype map (or a reference immunophenotype feature). Using the label information on the immunophenotype map and reference prediction results, an immunophenotype map (or immunophenotype map) is created. Prediction of whether a patient will respond to an immunotherapy anti-cancer drug based on the immunophenotype feature map The artificial neural network model 1600 can be trained by supervised learning to generate In still another embodiment, the information processing system extracts PD-L1 from the pathology slide image. The artificial neural network model 1600 is configured to perform supervised learning and / or For example, the information processing system can be trained by unsupervised learning. Using the label information for the slide image and reference information on PD-L1 expression, We developed an artificial neural network to generate information about PD-L1 expression from pathology slide images. The model 1600 can be trained by supervised learning.

[0145] The artificial neural network model 1600 thus trained is stored in the memory of the information processing system (see FIG. pathology slide images received from the communication module and / or memory can be stored in the Based on the input of the message, one or more target items are searched for in the pathology slide image. Additionally or alternatively, the artificial neural network model 1600 may generate a first pathology slide image. a feature or a first pathology slide for each of at least a portion of the area within the image; a first pathology slide image in response to an input for at least a partial region in the slide image; The immunophenotype or information associated with the immunophenotype of at least a portion of the region within the image can be determined. Additionally or alternatively, the artificial neural network model 1600 may be configured to generate an immunophenotype map or immunophenotype map. Depending on the input to the type feature map, a prediction result is made as to whether the patient will respond to the immunotherapy anti-cancer drug. Additionally or alternatively, the artificial neural network model 1600 may include a communication module and and / or a pathology slide image received from memory, Information about the expression of CAR can be generated.

[0146] According to one embodiment, an artificial neural network model and / or biomarkers for detecting items of interest are used. Input for an artificial neural network model that generates information on the expression of The variables are one or more pathology slide images (e.g., H&E-stained pathology slide images). For example, artificial neural network models can be used. The input variables input to the input layer 1620 of 1600 are one or more pathology slide images. It can be an image vector 1610 organized as a single vector data element. The output variables output by the output layer 1640 of the artificial neural network model 1600 are calculated in response to the input of the page. represents or characterizes one or more items of interest detected from a pathology slide image. Additionally or alternatively, the artificial neural network model 1600 may be The output variables output by the output layer 1640 are the biometric variables generated from the pathology slide images. It can be a vector 1650 that represents or characterizes information about the expression of a gene. That is, the output layer 1640 of the artificial neural network model 1600 detects the pathological features from the pathological slide image. Biomass generated from one or more target items and / or pathology slide images It can be configured to output vectors that represent or characterize information about the expression of the target gene. In this disclosure, the output variables of the artificial neural network model 1600 are of the type described above. One or more items of interest detected from the pathology slide image, including but not limited to: Any information that indicates the expression of biomarkers generated from pathology slide images. It may contain information / data.

[0147] In another embodiment, the immunophenotype of at least a portion of the region in the reference pathology slide image is The input variables of the machine learning model, i.e., the artificial neural network model 1600, which determines A feature or a first pathology slide for at least a partial region in the pathology slide image For example, the input of the artificial neural network model 1600 may be at least a portion of the image. The input variables input to the input layer 1620 are at least a portion of the first pathology slide image. A feature for the region or at least a portion of the region in the first pathology slide image It can be a numeric vector 1610, organized as vector data elements of In response to the force, the output variables output by the output layer 1640 of the artificial neural network model 1600 are Each immunophenotype or immunophenotype-related immunophenotype of at least a portion of the area in the pathology slide image It can be a vector 1650 that represents or characterizes the associated information. The output variables of the artificial neural network model 1600 are not limited to the types described above. Depicting or characterizing the immunophenotype of at least a portion of each of the regions within one pathology slide image. In another embodiment, the patient may be given an immunization A machine learning model that generates predictions on whether or not a patient will respond to anticancer drugs, i.e., artificial The input variables of the neural network model 1600 are the immunophenotype map or the immunophenotype feature map. For example, the input variables input to the input layer 1620 of the artificial neural network model 1600 are The immunophenotype map or immunophenotype feature map is configured as a single vector data element. It can be a numerical vector 1610 and / or image data. In response to the input to the phenotype feature map, the output layer 1640 of the artificial neural network model 1600 The output variables represent the predicted results of whether the patient will respond to the immunotherapy. In this disclosure, the artificial neural network model 16 The output variables of 00 are not limited to the types described above, but may include whether the patient responds to an immunotherapy or not. The artificial neural network may include any information / data that indicates the predicted outcome of the neural network. The output layer 1640 of the network model 1600 calculates the reliability and / or can be configured to output a vector indicating accuracy.

[0148] In this way, the input layer 1620 and the output layer 1640 of the artificial neural network model 1600 are The input variables and corresponding output variables are matched, and the input layer 1620, hidden layer 16 30_1 to 1630_n and the synapse values ​​between the nodes included in the output layer 1640 are adjusted. By doing so, it can learn to extract the correct output corresponding to a specific input. Through this learning process, the hidden characteristics of the input variables of the artificial neural network model 1600 can be grasped. , so that the error between the output variables calculated based on the input variables and the target output is reduced. The synaptic values ​​(or weights) between the nodes of the neural network model 1600 can be adjusted. Using the artificial neural network model 1600 trained in the input pathology slide image, In response, the target item detection results and / or information regarding PD-L1 expression can be output. Additionally or alternatively, the artificial neural network model 1600 may be used to generate one or more input references. Feature for at least a portion of the area in the pathology slide image or reference pathology slide and determining an immunophenotype or immunohistotype of each of the one or more regions of interest in response to at least a portion of the region within the image. At least one of the information associated with the immunophenotype can be output. Alternatively, an artificial neural network model 1600 may be used to generate a set of phenotypes based on the input immunophenotype map. This allows the system to output a prediction result as to whether or not a patient will respond to an immunotherapy anti-cancer drug.

[0149] FIG. 17 is an exemplary graph providing response prediction results for an immunological anti-cancer drug according to one embodiment of the present disclosure. As shown in the figure, the information processing system 100 includes one or more The processor 1710, the bus 1730, the communication interface 1740, the processor 171 172 to load a computer program 1760 executed by 17. The computer program 1760 may include a storage module 1750 for storing the computer program 1760. However, FIG. 17 shows only the components associated with the embodiment of the present disclosure. Therefore, a person skilled in the art to which the present disclosure pertains would understand the configuration shown in FIG. It will be appreciated that other general components may be included in addition to the elements.

[0150] The processor 1710 controls the overall operation of each component of the information processing system 100. The processor 1710 is a CPU (Central Processing Unit), MPU (Micro Processor Unit) unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit) or the like The processor 1710 may include any type of processor known in the art. , at least one application for performing a method according to an embodiment of the present disclosure, or The information processing system 100 includes one or more processors that can execute programs. It can be equipped with.

[0151] The memory 1720 may store various data, instructions, and / or information. , from the storage module 1750 to perform the methods / operations according to various embodiments of the present disclosure. One or more computer programs 1760 can be loaded from the memory 1720. M, but the scope of the present disclosure is not limited thereto. It's not that.

[0152] The bus 1730 can provide communication between components of the information processing system 100. 1730 is an address bus, a data bus, and a control bus. The bus may be implemented in various forms such as a WAN bus, ...

[0153] The communication interface 1740 is a communication interface for the information processing system 100. In addition, the communication interface 1740 can support various communication methods other than Internet communication. For this reason, the communication interface 1740 is currently The communication module may include any known communication module in the field.

[0154] The storage module 1750 stores one or more computer programs 1760 non-temporarily. The storage module 1750 can store data in a ROM (read-only memory), an EPROM (er asable programmable read-only memory), EEPROM (electrically erasable PROM) ), non-volatile memory such as flash memory, hard disk, removable disk or any other form of computer-readable recording medium known in the art. Cut.

[0155] The computer program 1760, when loaded into the memory 1720, 1710 includes one or more interfaces that allow the device to perform operations / methods according to various embodiments of the present disclosure. That is, the processor 1710 may include instructions. , executing one or more instructions to perform operations / operations according to various embodiments of the present disclosure. The method can be carried out.

[0156] For example, the computer program 1760 receives a first pathology slide image. an operation of detecting one or more items of interest within a first pathology slide image; based on the detection results for the one or more target items, At least one of an immunophenotype of at least a portion of the region or information associated with the immunophenotype determining an immunophenotype of at least a portion of the region within the first pathology slide image; A first pathology slide image is generated based on at least one of the information associated with the immunophenotype. A method for generating predictive results for whether a patient associated with a given page will respond to an immunotherapy. It may contain one or more instructions that cause the program to perform some operation or other function. In such a case, the information processing system 100 may be used to perform the immune anti-cancer treatment according to some embodiments of the present disclosure. A system for predicting response to an agent may be implemented.

[0157] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications of the present disclosure will be apparent to those of ordinary skill in the art and are within the scope of the present disclosure as defined herein. The general principles and the like described herein may be adapted to various modifications without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure should not be construed as being limited to the examples and the like described herein. The broadest scope consistent with the principles and novel features disclosed in this application is not intended to be limiting. is intended to be granted.

[0158] Exemplary embodiments are presently disclosed in the context of one or more stand-alone computer systems. While reference may be made to utilizing the subject matter in a variety of forms, the subject matter is not so limited. Works with any computing environment, such as a network or distributed computing environment Additionally, aspects of the presently disclosed subject matter may be embodied in multiple processing chips. Storage may be implemented on or across multiple devices. Such devices can be PCs, networks, It may also include a client and a handheld device.

[0159] Although the present disclosure has been described herein in connection with some embodiments, the present invention pertains to Various modifications within the scope of the present disclosure that would be understood by one of ordinary skill in the art are possible. It will be appreciated that variations and modifications are possible and that such variations and modifications are within the spirit and scope of the present invention. It must be understood that the invention falls within the scope of the appended claims.

Claims

1. 1. A method for predicting a response to an immunological anti-cancer drug, performed by at least one computer device, comprising: receiving a first pathology slide image; detecting one or more items of interest within the first pathology slide image, the one or more items of interest including immune cells and cancerous regions; determining at least one of an immunophenotype or information associated with an immunophenotype of a region of interest within the first pathology slide image based on the detection results for the one or more items of interest; and generating a prediction result as to whether a patient associated with the first pathology slide image will respond to an immunological anti-cancer drug based on at least one of the immunophenotype of the region of interest in the first pathology slide image or information associated with the immunophenotype; the detecting step includes detecting the immune cells and the cancerous region in the region of interest in the first pathology slide image using an artificial neural network object item detection model; A method for predicting response to an immune anti-cancer drug, wherein the determining step includes determining at least one of the immune phenotype of the region of interest or information associated with the immune phenotype based on at least one of the density, number, or distribution of the immune cells within the cancerous region.

2. 10. The method of claim 1, the one or more target items further include cancer stroma; The determining step includes: Calculating at least one of the number, distribution, and density of the immune cells within the cancer region or the cancer stroma within the region of interest in the first pathology slide image; and determining at least one of the immune phenotypes of the region of interest in the first pathology slide image or information associated with the immune phenotype based on at least one of the calculated number, distribution, or density of immune cells.

3. 3. The method of claim 2, The calculating step Calculating the density of the immune cells in the cancerous region within the region of interest in the first pathology slide image; calculating a density of the immune cells in the cancer stroma within the region of interest in the first pathology slide image; The determining step includes: A method for predicting a response to an immune anti-cancer drug, comprising determining at least one of an immune phenotype of the region of interest in the first pathology slide image or information associated with the immune phenotype based on at least one of the density of immune cells in the cancer region or the density of immune cells in the cancer stroma.

4. 10. The method of claim 1, The determining step includes: The method for predicting response to an immune anti-cancer drug further comprises determining the immune phenotype of the region of interest in the first pathology slide image as one of immune activity, immune exclusion, or immune deficiency based on at least one of the density, number, or distribution of the immune cells in the cancerous region.

5. 10. The method of claim 1, the region of interest in the first pathology slide image includes a plurality of regions of interest; the immunophenotype of the region of interest in the first pathology slide image includes an immunophenotype of each of the plurality of regions of interest; The generating step includes: A method for predicting response to an immunological anti-cancer drug, comprising a step of generating a prediction result as to whether the patient will respond to the immunological anti-cancer drug based on at least one of the most frequently contained immunological phenotype within the entire area of ​​the first pathology slide image, a region of interest in which the immunological phenotype is determined to be a specific immunological phenotype, and the distribution of the immunological phenotypes.

6. 10. The method of claim 1, The generating step includes: generating an immunophenotype map for the region of interest in the first pathology slide image using the immunophenotype of the region of interest in the first pathology slide image; and generating a prediction result as to whether the patient will respond to the immune anticancer drug by inputting the generated immune phenotype map into an immune anticancer drug response prediction model, A method for predicting a response to an immune anticancer drug, wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype map.

7. 10. The method of claim 1, The generating step includes: generating an immunophenotype feature map for the region of interest in the first pathology slide image using information associated with the immunophenotype of the region of interest in the first pathology slide image; and generating a prediction result as to whether the patient will respond to the immune anticancer drug by inputting the generated immune phenotype feature map into an immune anticancer drug response prediction model; The method for predicting a response to an immune anticancer drug, wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype feature map.

8. 10. The method of claim 1, obtaining information regarding biomarker expression from a second pathology slide image associated with the patient; The generating step includes: A method for predicting response to an immune anti-cancer drug, comprising a step of generating a prediction result as to whether the patient will respond to the immune anti-cancer drug based on at least one of the immune phenotype of the region of interest in the first pathology slide image or information associated with the immune phenotype, and information regarding the expression of the biomarker.

9. 10. The method of claim 1, The method for predicting response to an immune anti-cancer drug further includes outputting at least one of the detection results for the one or more target items, the immune phenotype of the region of interest in the first pathology slide image, information associated with the immune phenotype, a prediction result as to whether the patient will respond to the immune anti-cancer drug, or the density of immune cells in the region of interest in the first pathology slide image.

10. A computer program stored on a computer-readable recording medium for executing the method for predicting a response to an immunological anticancer agent according to any one of claims 1 to 9.

11. An information processing system, a memory for storing one or more instructions; a processor configured to receive a first pathology slide image by executing the one or more stored instructions; detect one or more target items in the first pathology slide image, the one or more target items including immune cells and cancer regions; determine an immunophenotype or information associated with the immunophenotype of a region of interest in the first pathology slide image based on detection results for the one or more target items; and generate a prediction result regarding whether a patient associated with the first pathology slide image will respond to an immune anti-cancer drug based on the immunophenotype or information associated with the immunophenotype of the region of interest in the first pathology slide image; The processor: Detecting the immune cells and the cancer region in the region of interest in the first pathology slide image using an artificial neural network object item detection model; The information processing system is further configured to determine an immunophenotype of the region of interest or at least one of information associated with the immunophenotype based on at least one of the density, number, or distribution of the immune cells within the cancerous region.

12. 12. The information processing system according to claim 11, the one or more target items further include cancer stroma; The processor: Calculating at least one of the number, distribution, and density of the immune cells in the cancer region or the cancer stroma within the region of interest in the first pathology slide image; The information processing system is further configured to determine at least one of an immunophenotype of the region of interest within the first pathology slide image or information associated with the immunophenotype based on at least one of the calculated number, distribution, or density of immune cells.

13. 13. The information processing system according to claim 12, The processor: The information processing system is further configured to calculate the density of the immune cells in the cancerous region within the region of interest within the first pathology slide image, calculate the density of the immune cells in the cancer stroma within the region of interest within the first pathology slide image, and determine the immune phenotype of the region of interest within the first pathology slide image as one of immune activation, immune exclusion, or immune deficiency based on at least one of the density of the immune cells in the cancerous region or the density of the immune cells in the cancer stroma.

14. 12. The information processing system according to claim 11, The processor: The information processing system is further configured to determine the immune phenotype of the region of interest in the first pathology slide image as one of immune activation, immune exclusion, or immune deficiency based on at least one of the density, number, or distribution of the immune cells within the cancerous region.

15. 12. The information processing system according to claim 11, the region of interest in the first pathology slide image includes a plurality of regions of interest; the immunophenotype of the region of interest in the first pathology slide image includes an immunophenotype of each of the plurality of regions of interest; The processor The information processing system is further configured to generate a prediction result as to whether the patient will respond to an immunological anticancer drug based on at least one of the most frequently contained immunophenotype within the entire area of ​​the first pathology slide image, the area of ​​interest in which the immunophenotype is determined to be a specific immunophenotype, and the distribution of the immunophenotypes.

16. 12. The information processing system according to claim 11, The processor: The method is further configured to generate an immunophenotype map for the region of interest in the first pathology slide image using the immunophenotype of the region of interest in the first pathology slide image, and input the generated immunophenotype map into an immunological anticancer drug response prediction model to generate a prediction result as to whether the patient will respond to the immunological anticancer drug, An information processing system, wherein the anticancer drug immune response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype map.

17. 12. The information processing system according to claim 11, The processor: The method is further configured to generate an immunophenotype feature map for the region of interest in the first pathology slide image using information associated with the immunophenotype of the region of interest in the first pathology slide image, and input the generated immunophenotype feature map into an immunological anticancer drug response prediction model to generate a prediction result as to whether the patient will respond to the immunological anticancer drug; An information processing system, wherein the immune anticancer drug response prediction model is trained to generate a reference prediction result by inputting a reference immune phenotype feature map.

18. 12. The information processing system according to claim 11, The processor: The information processing system is further configured to acquire information regarding biomarker expression from a second pathology slide image associated with the patient, and generate a prediction result regarding whether the patient will respond to an immune anti-cancer drug based on information regarding the immunophenotype and biomarker expression of the region of interest in the first pathology slide image.

19. 12. The information processing system according to claim 11, The processor: The information processing system is further configured to output at least one of the detection results for the one or more target items, the immune phenotype of the region of interest in the first pathology slide image, information associated with the immune phenotype, a prediction result as to whether the patient will respond to an immune anticancer drug, or the density of immune cells in the region of interest in the first pathology slide image.

Images