Method for training a machine learning model for analyzing immunohistochemical stained images and computing system for doing so

Abstract

A method and a computing system for performing such a determination on a pathological specimen image, such as analyzing the expression level of a biomarker, by analyzing a biological tissue slide image stained by an immunohistochemical staining method that stains tissue in which a specific biomarker is expressed, and a machine learning model used for analyzing the expression level of the biomarker in detail, are disclosed. According to one aspect of the present invention, a method and a system for generating learning data corresponding to an immunohistochemically stained image using a plurality of feature vectors calculated based on various staining intensity reference values, and training a machine learning model are provided.

Description

[Technical field]

[0001] The present invention relates to a method for training a machine learning model for analyzing immunohistochemical stained images, and a computing system for performing the same. More specifically, the present invention relates to a method for analyzing biological tissue slide images stained by an immunohistochemical staining method that stains tissues in which specific biomarkers are expressed, training a machine learning model used for detailed analysis of the expression level of the biomarker, and performing judgments on pathological specimen images, such as analyzing the expression level of the biomarker, using the trained machine learning model, and a computing system for performing the same. [Background technology]

[0002] As the average life expectancy of modern people increases, the incidence of serious diseases such as cancer also tends to increase, and measuring the expression of specific biomarkers in biological tissues is a very important factor in predicting the prognosis and determining treatment methods for patients with serious diseases such as cancer. Although next generation sequencing (NGS) and other methods have been attracting attention as methods for measuring the expression levels of biomarkers in biological tissues, the method mainly used in clinical settings at medical institutions is still visual observation of tissue slides stained with immunohistochemistry (IHC) staining.

[0003] Immunohistochemical staining allows visual confirmation of the expression level of a particular biomarker by staining tissue with a dye consisting of an antibody targeting a particular biomarker and a coloring agent such as diaminobenzidine (DAB).Hematoxylin is used as a control stain to show cells that do not express the biomarker.

[0004] Interpretation of immunohistochemically stained tissue is usually performed taking into consideration the intensity of staining and the proportion of stained tissue. In this case, the intensity of staining is usually classified into four stages, such as "almost no staining," "weak," "moderate," and "strong." The presence or absence of staining in tissue may be recognized as staining when the cell nucleus is partially stained by the biomarker, or when only the entire cell membrane is stained. Usually, the target area (e.g., cancer lesion) for measuring the expression level of the biomarker is determined, and the target for confirming the actual expression of the biomarker, such as cancer cells or immune cells, is selected for interpretation.

[0005] Meanwhile, the interpretation rules for immunohistochemically stained tissues vary depending on the target biomarker to be measured and the applied organ / disease. This is because the expression pattern differs for each biomarker, and the expression level of the same biomarker may differ depending on the applied organ / disease. In the case of a specific immunohistochemical staining method for measuring the expression levels of ER (estrogen receptor) and PR (progesterone receptor), a total score is used, which is calculated by adding up the proportion of stained tissue (proportion score) of 0 to 5 points and the staining intensity (intensity score) of 0 to 3 points. In the case of a specific immunohistochemical staining method for measuring the expression level of HER2 (human epidermal growth factor receptor 2), the score is determined as 0, 1+, 2+, or 3+ based on the proportion of cancer cells determined to be stained, taking into account the staining intensity and degree of staining of the cell membrane. If the expression rate of PD-L1 in tumor-infiltrating immune cells is 1% or more, it may be determined that the cells are expressed, and in this case, the cell membrane of the immune cells must be stained at an intensity above the appropriate level.

[0006] As described above, it is very difficult to interpret immunohistochemically stained tissues considering the staining intensity and the proportion of stained tissue for the following reasons, and the agreement between interpreters is poor, and the reproducibility between individual interpreters is also poor. First, the criteria for classifying the staining intensity are unclear, and in particular, the boundaries between weak, moderate, and other intensities are often unclear. Therefore, interpretation results may vary from day to day depending on the condition of the interpreter. Second, it is often difficult to determine whether or not tissue is stained. In particular, when determining whether or not staining is present considering the staining intensity, it is not easy to determine whether a cell having a cell membrane that is partly strongly stained and other parts that are moderately or weakly stained is stained.

[0007] Meanwhile, with the development of image analysis technology, attempts are being made to apply machine learning technology to various tasks in the medical field for practical use. In the field of pathological diagnosis, instead of the conventional method of pathologists examining and interpreting stained tissue slides at high magnification through an optical microscope, a method of converting tissue slides into high-resolution digital images through a digital slide scanner and then having pathologists interpret the images on a computer monitor is gradually being put to practical use, and products are also appearing that allow pathologists to refer to the results of image analysis using machine learning technology to make faster and more accurate diagnoses. A method of diagnosing cancer by applying image analysis deep learning technology to whole-slide images (WSI), which are images generated by scanning the entire body tissue slide, has already reached a practical level, and there are products that have been approved as medical devices both domestically and internationally for some carcinomas such as prostate cancer. Such products perform functions such as analyzing WSI generated from stained slides to determine the presence or absence of cancerous lesions or visualizing the location of detected lesions.

[0008] In addition, with the recent advances in digital pathology and the rapid development of computer-based image analysis technologies such as video processing, machine learning, and deep learning, software that uses computer analysis of scanned digital images of immunohistochemically stained slides to automatically measure biomarker expression levels has emerged and is continually improving. Automatic measurement of biomarker expression levels using software can solve the problems of inter-interpreter agreement and individual interpreter reproducibility.

[0009] However, since the software outputs results according to interpretation rules developed for an environment where humans measure and interpret through an optical microscope, detailed analysis of the expression level of biomarkers is not possible. For example, the software can classify the staining intensity in much more detail than "almost no staining," "weak," "moderate," and "strong," as well as quantitatively determine the staining level of each cell, but such analysis is not possible with the current system. Therefore, in order to more precisely determine the treatment method for patients based on the expression level of biomarkers, a new interpretation method that allows detailed analysis of the expression level of biomarkers is needed. In addition, a technical idea is required that applies this to the learning of a machine learning model to learn a machine learning model that can be used to analyze the expression level of biomarkers in detail, and thereby make judgments on pathology specimen images, such as analyzing the expression level of biomarkers. Summary of the Invention [Problem to be solved by the invention]

[0010] The technical problem to be solved by the present invention is to provide a method and system capable of analyzing the expression level of a biomarker in detail by analyzing an immunohistochemically stained tissue slide using machine learning, and also to provide a method and system capable of learning a machine learning model used for detailed analysis of the expression level of a biomarker, thereby making judgments on various pathological specimen images, such as analyzing the expression level of a biomarker. [Means for solving the problem]

[0011] According to one aspect of the present invention, there is provided a method for training a machine learning model, the method comprising: a step of a training system for the machine learning model generating a training dataset including M pieces of individual training data (wherein M is a natural number equal to or greater than 2); and a step of the training system for the machine learning model training the machine learning model based on the training dataset, the step of generating the training dataset including the M pieces of individual training data includes a step of generating m-th training data included in the training dataset for all integers m in the range of 1≦m≦M, the step of generating the m-th training data including the m-th training data including the m-th training data, the m-th immunohistochemically stained image including an area corresponding to immunohistochemically stained tissue stained by an immunohistochemistry (IHC) staining method for staining a predetermined target biomarker, a step of calculating a staining intensity by immunohistochemical staining for each pixel of the m-th immunohistochemically stained image, and a step of calculating a staining intensity by immunohistochemical staining for each pixel of the m-th immunohistochemically stained image for all integers n in the range of 1≦n≦N (wherein N is an integer equal to or greater than 2). The present invention provides a method for training a machine learning model, comprising: generating an nth feature vector of the mth immunohistochemical stained image based on the staining intensity for each cell and a predetermined nth staining intensity reference value; and generating the mth training data based on a first staining intensity reference value to an Nth staining intensity reference value and the first feature vector of the mth immunohistochemical stained image to the Nth feature vector, wherein the step of generating the nth feature vector of the mth immunohistochemical stained image based on the staining intensity for each pixel of the mth immunohistochemical stained image and the predetermined nth staining intensity reference value comprises: comparing the staining intensity for each pixel of the mth immunohistochemical stained image with the nth staining intensity reference value to generate an nth binarized image corresponding to the mth immunohistochemical stained image (here, the nth binarized image is an image partitioned into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity); and generating the nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image.

[0012] In one embodiment, for all integers i, 1≦i≦(N−1), the i-th staining intensity reference value may be less than the (i+1)-th staining intensity reference value.

[0013] In one embodiment, the step of obtaining the mth immunohistochemical stained image may include the steps of obtaining an mth original pathology image by scanning a pathology slide stained with a staining agent obtained by mixing an immunohistochemical staining reagent and a control staining reagent, and separating an immunohistochemically stained portion stained with the immunohistochemical staining reagent and a control stained portion stained with the control staining reagent from the mth original pathology image, and generating an mth immunohistochemically stained image corresponding to the mth original pathology image, and an mth control stained image corresponding to the mth original pathology image.

[0014] In one embodiment, the step of generating an nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image includes a step of generating an nth feature vector of the mth immunohistochemical stained image, in which each of at least one calculated value calculated based on the nth binarized image corresponding to the mth immunohistochemical stained image and the mth control stained image is a component of a vector, and the at least one calculated value may include at least one of the percentage of stained cell tissue, the percentage of cells whose cell membranes are stained, and the percentage of cells whose cell nuclei are stained.

[0015] In one embodiment, the step of generating the mth learning data based on the first staining intensity reference value to the Nth staining intensity reference value and the first feature vector to the Nth feature vector of the mth immunohistochemical stained image may include a step of generating the mth learning data composed of pairs of the first staining intensity reference value and the first feature vector to pairs of the Nth staining intensity reference value and the Nth feature vector, and labeled with an expression score of HER2 (human epidermal growth factor receptor 2).

[0016] In one embodiment, the step of generating the mth training data based on the first staining intensity reference value to the Nth staining intensity reference value and the first feature vector to the Nth feature vector of the mth immunohistochemical stained image includes a step of generating the mth training data composed of a pair of the (N-1)th staining intensity reference value and the (N-1)th feature vector difference value from a pair of the first staining intensity reference value and the first feature vector difference value, and labeled with an expression score of ER (estrogen receptor) or PR (progesterone receptor), and for all integers i such that 1≦i≦(N-1), the ith feature vector difference value may be a difference value between the (i+1)th feature vector and the ith feature vector.

[0017] According to another aspect of the present invention, there is provided a method for providing a judgment result for a predetermined pathological specimen to be judged through a machine learning model trained by the above-mentioned machine learning model training method, the method comprising the steps of: acquiring an immunohistochemical stained image of a subject to be judged by a computing system (wherein the immunohistochemical stained image of the subject to be judged includes an area corresponding to the immunohistochemically stained tissue of the pathological specimen to be judged stained by the immunohistochemical staining method); calculating a staining intensity by immunohistochemical staining for each pixel of the immunohistochemical stained image of the subject to be judged by the computing system; generating an n-th feature vector of the immunohistochemical stained image of the subject to be judged based on the staining intensity for each pixel of the immunohistochemical stained image of the subject to be judged and the n-th staining intensity reference value for all integers n of 1≦n≦N; and calculating a first feature vector of the immunohistochemical stained image of the subject to be judged based on the ... The method includes the steps of: generating input data based on feature vector 1 to N-th feature vector; and outputting, by the computing system, a judgment result for the pathology specimen to be judged, judged by the machine learning model based on the input data, wherein the step of generating the n-th feature vector of the immunohistochemical stained image to be judged based on the staining intensity for each pixel of the immunohistochemical stained image to be judged and the n-th staining intensity reference value includes the steps of: comparing the staining intensity for each pixel of the immunohistochemical stained image to be judged with the n-th staining intensity reference value to generate an n-th binarized image corresponding to the immunohistochemical stained image to be judged (here, the n-th binarized image is an image divided into areas having a staining intensity greater than the n-th staining intensity reference value and areas not having such a staining intensity); and generating the n-th feature vector of the immunohistochemical stained image to be judged based on the n-th binarized image corresponding to the immunohistochemical stained image to be judged.

[0018] According to another aspect of the invention there is provided a computer program recorded on a medium for being installed on a data processing device and for carrying out the method as described above.

[0019] According to another aspect of the present invention, there is provided a computer readable recording medium having recorded thereon a computer program for performing the method described above.

[0020] According to another aspect of the present invention, there is provided a system for training a machine learning model, the system including a processor and a memory for storing a computer program, the computer program, when executed by the processor, causing the computing system to execute a method for training a machine learning model, the method for training the machine learning model comprising: a step of the machine learning model training system generating a training dataset including M pieces of individual training data (wherein M is a natural number equal to or greater than 2); and a step of the machine learning model training system training the machine learning model based on the training dataset, the step of generating the training dataset including the M pieces of individual training data comprising: generating m-th training data included in the training dataset for all integers m, where 1≦m≦M; and the step of generating the m-th training data comprising: acquiring an m-th immunohistochemical stained image (wherein the m-th immunohistochemical stained image is subjected to immunohistochemistry (immunohistochemistry) for staining a predetermined target biomarker;the mth immunohistochemically stained image includes a region corresponding to an immunohistochemically stained tissue stained by an IHC (intracellular immunosorbent) staining method, and a step of calculating a staining intensity by immunohistochemical staining for each pixel of the mth immunohistochemically stained image; for all integers n of 1≦n≦N (where N is an integer of 2 or more), a step of generating an nth feature vector of the mth immunohistochemically stained image based on the staining intensity for each pixel of the mth immunohistochemically stained image and a predetermined nth staining intensity reference value; and a step of generating the mth learning data based on a first staining intensity reference value to an Nth staining intensity reference value and a first feature vector to an Nth feature vector of the mth immunohistochemically stained image, A learning system for a machine learning model is provided, the step of generating an nth feature vector of the mth immunohistochemical stained image based on the staining intensity for each cell and a predetermined nth staining intensity reference value includes a step of comparing the staining intensity for each pixel of the mth immunohistochemical stained image with the nth staining intensity reference value to generate an nth binarized image corresponding to the mth immunohistochemical stained image (here, the nth binarized image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity), and a step of generating an nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image;

[0021] According to another aspect of the present invention, there is provided a system for providing a judgment result for a pathological specimen, the system including a processor and a memory for storing a computer program, the computer program, when executed by the processor, causes the system to execute a method for providing a judgment result for a predetermined pathological specimen to be judged through a machine learning model trained by the above-mentioned machine learning model learning method, the method for providing the judgment result comprising the steps of: acquiring an immunohistochemically stained image of a subject to be judged (wherein the immunohistochemically stained image of the subject to be judged includes an area corresponding to an immunohistochemically stained tissue of the pathological specimen to be judged stained by the immunohistochemical staining method); calculating a staining intensity by immunohistochemical staining for each pixel of the immunohistochemically stained image of the subject to be judged; generating an n-th feature vector of the immunohistochemically stained image of the subject to be judged for all integers n of 1≦n≦N based on the staining intensity for each pixel of the immunohistochemically stained image of the subject to be judged and the n-th staining intensity reference value; The method includes a step of a judgment result providing system generating input data based on a first staining intensity reference value to an Nth staining intensity reference value and a first feature vector to an Nth feature vector of the immunohistochemical stained image to be judged, and a step of the judgment result providing system outputting a judgment result for the pathological specimen to be judged by the machine learning model based on the input data, wherein the step of generating the nth feature vector of the immunohistochemical stained image to be judged based on the staining intensity for each pixel of the immunohistochemical stained image to be judged and the nth staining intensity reference value includes a step of comparing the staining intensity for each pixel of the immunohistochemical stained image to be judged with the nth staining intensity reference value to generate an nth binarized image corresponding to the immunohistochemical stained image to be judged (here, the nth binarized image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity), and a step of generating the nth feature vector of the immunohistochemical stained image to be judged based on the nth binarized image corresponding to the immunohistochemical stained image to be judged. Effect of the Invention

[0022] According to the technical idea of ​​the present invention, it is possible to provide a method and system capable of analyzing the expression level of a biomarker in detail by analyzing an immunohistochemically stained tissue slide using deep learning, i.e., a method and system capable of learning a machine learning model used for detailed analysis of the expression level of a biomarker, thereby analyzing the expression level of a biomarker, and making judgments on various pathological specimen images can be provided.

[0023] The method for training a machine learning model according to the technical idea of ​​the present invention is characterized in that multiple different staining intensity reference values ​​are applied to a single immunohistochemical stained image, and training data is constructed from the values ​​derived in this way, thereby having the effect of constructing a machine learning model that can perform detailed analysis of the expression levels of biomarkers. [Brief description of the drawings]

[0024] In order to more fully understand the drawings referred to in the detailed description of the present invention, a brief description of each of the drawings is provided. [Figure 1] 1 is a diagram illustrating an environment in which a method for learning a machine learning model and a method for providing a judgment result for a pathological specimen according to the technical concept of the present invention are performed. [Diagram 2] 1 is a flowchart illustrating a method for learning a machine learning model according to an embodiment of the present invention. [Figure 3a] FIG. 13 shows a pathological image stained with a staining agent obtained by mixing an immunohistochemical staining reagent and a control staining reagent. [Figure 3b] FIG. 3B is a diagram showing an immunohistochemical staining image in which the colors of the immunohistochemical staining in FIG. 3A are extracted. [Figure 4] FIG. 3 is a diagram showing a more specific example of S140 in FIG. 2. [Diagram 5] FIG. 13 is a diagram showing the results of a binarized image changing as the staining intensity reference value increases. [Figure 6]1 is a flowchart illustrating an example of a method for providing a determination result for a pathological specimen according to an embodiment of the present invention. [Figure 7] FIG. 1 is a diagram illustrating a schematic configuration of a machine learning model learning system according to an embodiment of the present invention. [Figure 8] 1 is a diagram showing a schematic configuration of a judgment result providing system according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Since the present invention can be modified in various ways and can have various embodiments, specific embodiments are shown in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to the specific embodiments, and it should be understood that the present invention includes all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. In describing the present invention, if a detailed description of related publicly known technology is deemed to obscure the gist of the present invention, the detailed description will be omitted.

[0026] Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms first and second do not indicate a particular order, but are used only to distinguish one component from another.

[0027] The terms used in this application are merely used to describe specific embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless a clearly different meaning is present in the context.

[0028] In this specification, terms such as "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof set forth in the specification, but are to be understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0029] In addition, in this specification, when a certain component "transmits" data to another component, it means that the component can transmit the data directly to the other component, or can transmit the data to the other component via at least one other component. Conversely, when a certain component "transmits" data to another component, it means that the data is transmitted from the component to the other component without passing through any other component.

[0030] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings, in which the same reference numerals in the various drawings refer to the same elements.

[0031] FIG. 1 is a diagram illustrating an environment in which a method for learning a machine learning model and a method for providing a determination result for a pathological specimen according to the technical concept of the present invention are performed.

[0032] Referring to FIG. 1, a method for learning a machine learning model according to one embodiment of the present invention can be performed by a machine learning model learning system 100 (hereinafter also referred to as a "learning system"), and a method for providing a judgment result for a pathology specimen according to one embodiment of the present invention can be performed by a system for providing a judgment result for a pathology specimen 200 (hereinafter also referred to as a "judgment result providing system").

[0033] The learning system 100 is capable of learning a machine learning model 300 that receives input of data in a predetermined format derived from the staining intensity of tissue (e.g., cell membranes, cell nuclei, etc.) stained by immunohistochemical staining, and provides the expression level of a predetermined biomarker, detection of lesions contained in a pathological specimen, output of diagnostic information for the pathological specimen, or prognosis information and / or response information for a treatment method, and the judgment result providing system 200 is capable of making various judgments on the target specimen (e.g., judgment of the expression level of a predetermined biomarker, detection of lesions, presence or absence of a disease, prognosis, judgment on a treatment method, etc.) using the trained machine learning model 300.

[0034] The learning system 100 and / or the judgment result providing system 200 may be a computing system, which is a data processing device having computing power for realizing the technical ideas of the present invention, and may generally include not only a server, which is a data processing device accessible by a client via a network, but also a computer device such as a personal computer or a mobile terminal.

[0035] The learning system 100 and / or the judgment result providing system 200 may be realized as a single physical device, but an average expert in the technical field of the present invention can easily deduce that multiple physical devices can be organically combined as necessary to realize the learning system 100 and / or the judgment result providing system 200 according to the technical idea of ​​the present invention.

[0036] The training system 100 can train the machine learning model 300 based on training data generated from a plurality of immunohistochemical stained images.

[0037] The immunohistochemical staining image may be an image or a part thereof obtained by staining a slide of a pathological specimen by a predetermined immunohistochemical staining method. The pathological specimen may be a biopsy taken from various organs of the human body, or a biological tissue excised by surgery. On the other hand, the immunohistochemical staining image may be a tissue slide image obtained by staining a slide of a pathological specimen with an immunohistochemical staining reagent, or a tissue slide image obtained by staining with a staining agent in which an immunohistochemical staining reagent and a control staining reagent are mixed. In this case, the immunohistochemical staining reagent may be a staining agent in the form of an antibody targeting a specific biomarker bound to a coloring agent such as diaminobenzidine (DAB), and the control staining reagent may be a hematoxylin staining reagent (hereinafter referred to as "H reagent").

[0038] Meanwhile, the immunohistochemical stained image may be a digital slide image of a stained pathological specimen or a part of a digital slide image. The slide of the pathological specimen may be a part of a slice of the pathological specimen. The digital slide image of the pathological specimen may be generated by slicing the pathological specimen to produce a glass slide, staining it with a predetermined staining agent, and digitizing it. That is, the immunohistochemical stained image may be a pathological slide image produced by producing a slide of the pathological specimen and staining it with a predetermined staining agent (e.g., an immunohistochemical staining agent, or a staining agent obtained by mixing an immunohistochemical staining agent and a control staining agent), or an image obtained by dividing the stained pathological slide image into a predetermined size.

[0039] The learning system 100 can generate individual learning data using digital pathology images of pathology samples, and input the individual learning data to the input layer of the machine learning model 300 to train the machine learning model 300. The process by which the learning system 100 generates learning data will be described later.

[0040] The machine learning model 300 may be, but is not limited to, an artificial neural network or a deep neural network, and may include various models that can be trained through training data. For example, the machine learning model 300 may be a support vector machine, a random forest, a gradient-boosting tree, or an ensemble model thereof.

[0041] The learning of the machine learning model may be supervised learning. For example, when the machine learning model 300 is a deep neural network, the learning of the machine learning model 300 may be performed by back propagation, which updates the weights of the machine learning model 300 so that a loss value, which is the difference between a predicted value output by the machine learning model to which the learning data is input, and an actual value labeled in the learning data, is minimized. In this case, a method such as gradient descent may be used. The details of the learning process of the machine learning model are widely known at the time of filing of the present invention, and therefore further detailed description will be omitted.

[0042] The machine learning model 300 may be a machine learning model trained to output a probability value for the expression level of a specific biomarker. For example, the machine learning model 300 may output a value indicating the expression level of HER2 (human epidermal growth factor receptor 2), ER (estrogen receptor) and / or PR (progesterone receptor). However, the technical idea of ​​the present invention is not limited thereto, and a numerical value indicating a judgment result for a target specimen (e.g., the possibility of disease occurrence), i.e., a probability value, may be output, and the output value may change according to a label tagged to the learning data. Depending on the embodiment, the machine learning model 300 may be a machine learning model trained to output a probability value for the occurrence or non-occurrence of a disease, a treatment method, or the effect of a specific treatment method for a specific disease.

[0043] In one embodiment, the machine learning model 300 may be a patch-level clashfication neural network. The patch-level clashfication neural network may be a neural network that receives an image in units of a patch and outputs a value for classifying the corresponding patch. In another embodiment, the machine learning model 300 may be a pixel-level clashfication neural network. The pixel-level clashfication neural network may be a neural network that outputs a value for classifying the corresponding pixel for each pixel included in the image.

[0044] In this specification, an artificial neural network is a machine learning model artificially constructed based on the operating principle of human neurons, and may include a multi-layer perceptron model, and may refer to a collection of information representing a set of design items that define an artificial neural network. In one embodiment, the machine learning model 300 may be or may include a convolutional neural network.

[0045] Meanwhile, the trained machine learning model 300 may be stored in the judgment result providing system 200, and the judgment result providing system 200 can make a judgment on a predetermined diagnostic target sample using the trained machine learning model.

[0046] As shown in Fig. 1, the learning system 100 and / or the judgment result providing system 200 may be realized in the form of a subsystem of a given parent system 10. The parent system 10 may be a server. The server 10 means a data processing device having a computing capability for realizing the technical idea of ​​the present invention, and an average expert in the technical field of the present invention can easily deduce that a server can generally be defined as not only a data processing device accessible by a client via a network, but also any device capable of performing a specific service, such as a personal computer or a mobile terminal.

[0047] Alternatively, depending on the embodiment, the learning system 100 and the judgment result providing system 200 may be realized in a form separate from each other.

[0048] FIG. 2 is a flowchart illustrating a method for learning a machine learning model according to an embodiment of the present invention.

[0049] 2, the learning system 100 can generate a learning data set including M individual learning data (where M is a natural number equal to or greater than 2). To this end, the learning system 100 can generate the mth learning data included in the learning data set for all integers m, where 1≦m≦M (S100).

[0050] To generate training data, the training system 100 can acquire an mth immunohistochemically stained image (S110), where the mth immunohistochemically stained image may include a region corresponding to an immunohistochemically stained tissue stained by an immunohistochemistry (IHC) staining method for staining a predetermined target biomarker.

[0051] In one embodiment, the mth immunohistochemical stained image may be a pathology slide image (or patch) stained with diaminobenzidine (DAB) reagent.

[0052] In an embodiment, the learning system 100 can obtain an mth original pathology image by scanning a pathology slide stained with a staining agent that is a mixture of an immunohistochemical staining reagent (e.g., DAB reagent) and a control staining reagent (e.g., H reagent), and can separate from the mth original pathology image an immunohistochemical stained portion stained with the immunohistochemical staining reagent and a control stained portion stained with the control staining reagent, and generate an mth immunohistochemical stained image corresponding to the mth original pathology image, and an mth control stained image corresponding to the mth original pathology image.

[0053] There are various methods for separating immunohistochemically stained areas from control stained areas from an original pathological image stained with a mixture of an immunohistochemical staining reagent (e.g., DAB reagent) and a control staining reagent (e.g., H reagent), and one example is a method using color deconvolution (Quantification of histochemical staining by color deconvolution; Anal Quant Cytol Histol 23: 291-299, 2001.). In this method, the signal intensity for each channel in the color space representing the original pathological image is converted into optical density, and then the optical density is converted into staining intensity by a predetermined relational equation (which is determined by experiment), and the immunohistochemically stained areas and control stained areas are separated depending on whether the staining intensity is greater or smaller than a certain level.

[0054] In one embodiment, the learning system 100 can receive the mth immunohistochemical stained image or the mth original pathology image corresponding to a specified pathological specimen from an external terminal, and in another embodiment, the learning system 100 can also acquire the mth immunohistochemical stained image or the mth original pathology image corresponding to a pathological specimen from a memory device that pre-stores the mth immunohistochemical stained image or the mth original pathology image corresponding to a pathological specimen.

[0055] FIG. 3a shows a pathology image stained with a staining agent that is a mixture of an immunohistochemical staining reagent and a control staining reagent, and FIG. 3b shows an immunohistochemical stained image in which the immunohistochemical staining colors in FIG. 3a have been extracted.

[0056] Meanwhile, the mth immunohistochemical stained image or the mth original pathology image may be labeled with predetermined information, and the machine learning model 300 may be trained by a supervised learning method. The labeled information may vary depending on the purpose of the machine learning model 300. For example, the information labeled on the mth immunohistochemical stained image or the mth original pathology image may be the expression level of HER2, the proportion score and / or the staining intensity score of the immunohistochemical stained portion, or the expression level score of ER and PR, but the technical idea of ​​the present invention is not limited thereto. Depending on the embodiment, the present invention may be used for generating a new interpretation rule of the expression level of a biomarker that has not existed in the past. In particular, according to a method of learning so that the correlation between the output value obtained using linear regression, logistic regression, and other machine learning models and the actual treatment effect is high, it is also possible to newly discover an interpretation rule of the expression level of a biomarker that is very effective in determining the corresponding treatment method.

[0057] Meanwhile, the learning system 100 can calculate the staining intensity by immunohistochemical staining for each pixel of the mth immunohistochemical stained image (S120). In one embodiment, the learning system 100 can calculate the staining intensity of each pixel converted from the optical density determined by the above-mentioned color deconvolution method as the staining intensity of each pixel of the mth immunohistochemical stained image. Alternatively, the learning system 100 can convert the mth immunohistochemical stained image into a black and white image and calculate the staining intensity of each pixel of the mth immunohistochemical stained image using the brightness of the converted black and white image.

[0058] Thereafter, the learning system 100 can generate N mutually different feature vectors (S130). More specifically, for all integers n where 1 ≦ n ≦ N (where N is an integer of 2 or more), based on the staining intensity for each pixel of the m-th immunohistochemical staining image and a predetermined n-th staining intensity reference value, the learning system 100 can generate the n-th feature vector of the m-th immunohistochemical staining image (S140). At this time, for all integers i where 1 ≦ i ≦ (N - 1), the i-th staining intensity reference value may be smaller than the (i + 1)-th staining intensity reference value. At this time, the staining intensity reference values are not the same as each other, and when 1 ≦ i < j ≦ N, the reference value t_i < t_j can be satisfied. That is, the N staining intensity reference values may be values that gradually increase. On the other hand, the staining intensity reference values may be determined in consideration of the distribution of the staining intensity values measured by the constructed data by constructing the image data to include various staining intensities, or may be determined by other methods.

[0059] On the other hand, the above vector generation method can be realized by applying one or more staining intensity reference values to the m-th immunohistochemical staining image, determining that the portion having a staining intensity exceeding the corresponding reference value is stained, and generating a binarized image corresponding to each reference value. The specific method is shown in FIG. 4. FIG. 4 is a diagram showing an example of the S140 step in FIG. 2 in more detail.

[0060] Referring to FIG. 4, the learning system 100 can compare the staining intensity for each pixel of the m-th immunohistochemical staining image with the n-th staining intensity reference value to generate the n-th binarized image corresponding to the m-th immunohistochemical staining image (S141). At this time, the n-th binarized image is an image divided into a region having a staining intensity greater than the n-th staining intensity reference value and a region not having such staining intensity. For example, the learning system 100 can generate a binarized image by assigning a value of 1 to the pixels having a staining intensity greater than the n-th staining intensity reference value and assigning a value of 0 to the pixels having a staining intensity smaller than the n-th staining intensity reference value.

[0061] Fig. 5 shows the results of the binarized image changing as the staining intensity reference value increases. Fig. 5(a) shows an immunohistochemical stained image, and Fig. 5(b) to Fig. 5(h) show binarized images corresponding to the first staining intensity reference value to the seventh staining intensity reference value, respectively, and it can be seen that the binarized image changes as the staining intensity reference value gradually increases.

[0062] 4, the learning system 100 can generate an nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image (S142). The feature vector may be composed of one or more components. The feature vector being generated based on the binarized image means that one of the main elements for determining the feature vector is the binarized image, and it goes without saying that other elements besides the binarized image can also be used in the process of generating the feature vector.

[0063] In one embodiment, the learning system 100 can generate an nth feature vector of the mth immunohistochemical stained image, where each of at least one calculated value calculated based on the nth binarized image corresponding to the mth immunohistochemical stained image and the mth control stained image is a component of the vector. In this case, the at least one calculated value may include at least one of the percentage of stained cell tissue, the percentage of cells whose cell membranes are stained, and the percentage of cells whose cell nuclei are stained, and the percentage of cells whose cell membranes are stained may include the percentage of cells whose cell membranes are completely stained and the percentage of cells whose cell membranes are partially stained.

[0064] For example, the learning system 100 may calculate the ratio of the area of ​​pixels having a value of "1" in the nth binarized image (i.e., pixels having a staining intensity greater than the nth staining intensity reference value) to the area of ​​pixels stained in the mth control stained image, thereby calculating the ratio of stained cellular tissue, and may determine this as one of the components of the nth feature vector. Alternatively, the learning system 100 may determine a closed region determined by pixels having a value of "1" in the nth binarized image (i.e., pixels having a staining intensity greater than the nth staining intensity reference value), and calculate the ratio of the area occupied by the closed region to the area of ​​pixels stained in the mth control stained image, thereby calculating the ratio of the area occupied by the closed region to the area of ​​pixels stained in the mth control stained image, thereby calculating the ratio of the area occupied by the closed region to the area of ​​pixels stained in the mth control stained image, and may determine this as one of the components of the nth feature vector. Alternatively, the learning system 100 may determine closed regions determined by pixels having a value of "1" in the nth binarized image (i.e., pixels having a staining intensity greater than the nth staining intensity reference value), and may calculate the proportion of cells whose cell membranes are completely stained based on the area of ​​cells surrounded by a border of each closed region in which the proportion of pixels having a value of "1" in the nth binarized image is equal to or greater than a certain size, and determine this as one of the components of the nth feature vector. In addition, the components of the feature vector may be determined in various ways.

[0065] Referring again to FIG. 2, the learning system 100 can generate mth learning data based on the first staining intensity reference value to the Nth staining intensity reference value and the first feature vector to the Nth feature vector of the mth immunohistochemical stained image (S150).

[0066] In one embodiment, the mth training data may be composed of a pair of the first staining intensity reference value and the first feature vector to a pair of the Nth staining intensity reference value and the Nth feature vector. For example, the mth training data may be composed of a pair of the first staining intensity reference value and the first feature vector to a pair of the Nth staining intensity reference value and the Nth feature vector.<t_1、F_1> ,<t_2、F_2> , …,<t_N、F_N> ], where t_i is the ith staining intensity reference value and F_i is the ith feature vector of the mth immunohistochemical stained image.

[0067] Alternatively, in another embodiment, the m-th training data may be composed of a pair of the first staining intensity reference value and the first feature vector difference value to a pair of the (N-1)-th staining intensity reference value and the (N-1)-th feature vector difference value. For example, the m-th training data may be composed of a pair of [<t_1、F_2-F1> ,<t_2、F_3-F_2> , …,<t_(N-1)、F_N-F_(N-1)> ] may also be used.

[0068] On the other hand, each training data may be labeled with a value indicating an interpretation rule of the expression level of the biomarker. The value labeled with each training data may be information previously tagged to the mth immunohistochemical stained image or the mth immunohistochemical stained image or the mth original pathology image, or information derived from such information.

[0069] For example, each learning data may be labeled with an expression level score of HER2, expression scores of ER (estrogen receptor), PR (progesterone receptor), proportion score of stained tissue, and / or intensity score. However, the above-mentioned labels are merely examples, and on the other hand, when there is diagnostic information, prognostic information, and / or response information to a specific treatment method for a pathological specimen corresponding to the m-th learning data, the learning system 100 may set this as a label of the learning data. In addition, various values ​​expressing a new expression level interpretation rule of a biomarker that has not existed before may be labeled. For example, values ​​such as treatment effect and life expectancy after surgery may be labeled, and a predetermined value designed to increase the correlation between the output value obtained using linear regression, logistic regression, and other machine learning models and the actual treatment effect may be labeled.

[0070] When a learning data set including M individual learning data is generated by the above method, the learning system 100 can input the generated learning data set to the input layer of the machine learning model 300 to learn the machine learning model 300 (S160 in FIG. 2). Machine learning processes such as deep learning techniques in which learning data tagged with correct answer information is input to the machine learning model 300 and the machine learning model 300 is trained by methods such as gradient descent and backpropagation are widely known, and therefore detailed explanations will be omitted.

[0071] As described above, the method for training a machine learning model according to the technical idea of ​​the present invention is characterized in that a plurality of different staining intensity reference values ​​are applied to a single immunohistochemical stained image, and the training data is composed of the values ​​derived thereby, thereby having the effect of constructing a machine learning model capable of analyzing the expression levels of biomarkers in detail.

[0072] As described above, the method of constructing learning data and training a machine learning model can be applied in various ways. In one embodiment, the learning method includes extracting a feature vector F_i including the proportion of cancer cells whose cell membranes are completely stained for a plurality of different staining intensity reference values ​​t_i, and generating the feature vector F_i.<t_i、F_i> A machine learning model is trained that receives values ​​based on the above and outputs a HER2 expression level score, and the trained model can be used to determine the HER2 expression level.

[0073] Alternatively, a feature vector F_i including the proportion of cells whose nuclei are stained for a plurality of different staining intensity reference values ​​t_i is extracted and generated.<t_i、F_i> ] based on [(t_i, F_(i+1)-F_i)], which can be inducible from [(t_i, F_(i+1)-F_i)], is used as input to train a machine learning model that outputs a proportion score and intensity score, and the trained model can be used to actually determine the proportion score and intensity score.

[0074] In addition, the method can be used to generate new interpretation rules for the expression levels of biomarkers that have not existed before. In particular, by using a method of learning to increase the correlation between the output value obtained using linear regression, logistic regression, and other machine learning models and the actual treatment effect, it is possible to newly discover interpretation rules for the expression levels of biomarkers that are very effective in determining the corresponding treatment method.

[0075] 6 is a flowchart showing an example of a method for providing a determination result for a pathological specimen according to an embodiment of the present invention. The method for providing a determination result for a pathological specimen according to FIG. 6 may be performed by a determination result providing system 200, and the determination result providing system 200 may store a machine learning model 300 trained by the learning system 100.

[0076] 6, the determination result providing system 200 can obtain a determination target immunohistochemical stained image of a predetermined determination target pathology specimen (S210). At this time, the determination target immunohistochemical stained image may include an area corresponding to the immunohistochemical stained tissue of the determination target pathology specimen stained by an immunohistochemical staining method.

[0077] The judgment result providing system 200 can calculate the staining intensity by immunohistochemical staining for each pixel of the immunohistochemical stained image to be judged (S220), can generate the first to Nth feature vectors from the immunohistochemical stained image to be judged (S230, S240), and can generate input data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image (S250). The process of generating input data corresponding to the immunohistochemical image of the specimen to be judged is very similar to the process described above, and therefore a separate description will be omitted.

[0078] The judgment result providing system 200 inputs input data to the machine learning model 300 and outputs the results output by the machine learning model, or can output a judgment result for the pathological specimen to be judged based on the results output by the machine learning model (S260).

[0079] FIG. 7 is a diagram showing a schematic configuration of a machine learning model learning system 100 according to one embodiment of the present invention, and FIG. 8 is a diagram showing a schematic configuration of a judgment result providing system 200 according to one embodiment of the present invention.

[0080] The machine learning model learning system 100 and the judgment result providing system 200 may refer to a logical configuration having hardware resources and / or software necessary to realize the technical idea of ​​the present invention, but do not necessarily refer to one physical component or device. In other words, the machine learning model learning system 100 and the judgment result providing system 200 may refer to a logical combination of hardware and / or software provided to realize the technical idea of ​​the present invention, and may be realized as a set of logical configurations for realizing the technical idea of ​​the present invention by installing them in devices separated from each other and performing their respective functions, if necessary. In addition, the machine learning model learning system 100 and the judgment result providing system 200 may refer to a set of configurations realized separately for each function or role for realizing the technical idea of ​​the present invention. Each configuration of the machine learning model learning system 100 and the judgment result providing system 200 may be located in different physical devices or in the same physical device. In addition, depending on the embodiment, the software and / or hardware constituting each of the components of the machine learning model learning system 100 and the judgment result providing system 200 may also be located in different physical devices, and the components located in the different physical devices may be organically combined with each other to realize each of the above modules.

[0081] In addition, the term "module" in this specification may refer to a functional and structural combination of hardware for carrying out the technical idea of ​​the present invention and software for driving the hardware. For example, a module may refer to a logical unit of a certain code and a hardware resource for carrying out the certain code, and it is easy for an average expert in the technical field of the present invention to deduce that a module does not necessarily refer to physically connected code or one type of hardware.

[0082] 7, the machine learning model learning system 100 may include a storage module 110, an acquisition module 120, a generation module 130, and a learning module 140. Depending on the embodiment of the present invention, some of the above-mentioned components may not be essential components for realizing the present invention, and it goes without saying that the machine learning model learning system 100 may include more components than the above components depending on the embodiment. For example, the machine learning model learning system 100 may further include a communication module (not shown) for communicating with an external device and a control module (not shown) for controlling the components and resources of the machine learning model learning system 100.

[0083] The storage module 110 may store the machine learning model 300 to be trained.

[0084] The acquisition module 120 can acquire M immunohistochemical stained images including areas corresponding to immunohistochemically stained tissue stained by an immunohistochemical staining method for staining a predetermined target biomarker. In one embodiment, the acquisition module can acquire an original pathology image obtained by scanning a pathology slide stained with a staining agent obtained by mixing an immunohistochemical staining reagent and a control staining reagent, and separate an immunohistochemically stained portion stained with the immunohistochemical staining reagent and a control stained portion stained with the control staining reagent from the original pathology image to generate an immunohistochemically stained image corresponding to the original pathology image and a control stained image corresponding to the original pathology image.

[0085] The generation module 130 can generate individual training data based on each immunohistochemical stained image and a predetermined staining intensity reference value, and can configure a training data set including a plurality of individual training data. The individual training data generated by the generation module 130 may be a vector representing the expression level of a biomarker.

[0086] The training module 140 can train the machine learning model 300 based on the training dataset.

[0087] 8, the judgment result providing system 200 may include a storage module 210, an acquisition module 220, a generation module 230, and a judgment module 240. Depending on the embodiment of the present invention, some of the above-mentioned components may not be essential components for realizing the present invention, and it goes without saying that the judgment result providing system 200 may include more components than the above components depending on the embodiment. For example, the judgment result providing system 200 may further include a communication module (not shown) for communicating with an external device, and a control module (not shown) for controlling the components and resources of the judgment result providing system 200.

[0088] The storage module 210 can store the trained machine learning model 300.

[0089] The acquisition module 220 can acquire a subject immunohistochemical staining image.

[0090] The generation module 230 can generate input data based on the subject immunohistochemical stained image.

[0091] The judgment module 240 inputs input data to the machine learning model 300 and can make a judgment on the specimen to be judged based on the predicted value output from the machine learning model 300.

[0092] Meanwhile, according to an embodiment, the machine learning model learning system 100 and the judgment result providing system 200 may include a processor and a memory for storing a program executed by the processor. The processor may include a single-core CPU or a multi-core CPU. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory by the processor and other components may be controlled by a memory controller.

[0093] Meanwhile, the method according to the embodiment of the present invention may be realized in the form of a computer readable program instruction and stored in a computer readable recording medium, and the control program and the target program according to the embodiment of the present invention may also be stored in a computer readable recording medium. The computer readable recording medium includes any kind of recording device in which data readable by a computer system is stored.

[0094] The program instructions contained on the recording media may be those specially designed and constructed for the purposes of the present invention, or they may be of the type well known and available to those having skill in the software arts.

[0095] Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Also, computer readable recording media can be distributed among computer systems connected via a network to store and execute computer readable code in a distributed manner.

[0096] Examples of program instructions include not only machine code produced by a compiler, but also high level language codes that may be executed by a device for processing information electronically, such as a computer, using an interpreter or the like.

[0097] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0098] The above description of the present invention is for illustrative purposes only, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting. For example, each component described as a single component can be implemented in a distributed form, and similarly, each component described as a distributed form can be implemented in a combined form.

[0099] The scope of the present invention is indicated by the claims set forth below rather than by the above detailed description, and all modifications and variations derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included within the scope of the present invention. [Industrial Applicability]

[0100] The present invention can be used in a method, and a computing system, for training a machine learning model for analyzing immunohistochemical stained images.

Claims

1. 1. A method for training a machine learning model, comprising: A step in which a learning system of a machine learning model generates a learning dataset including M pieces of individual learning data (where M is a natural number equal to or greater than 2); A learning system for the machine learning model learns the machine learning model based on the training dataset; Including, the step of generating a training data set including the M individual training data includes a step of generating m-th training data included in the training data set for all integers m, where 1≦m≦M; The step of generating m training data includes: acquiring an mth immunohistochemical stained image, wherein the mth immunohistochemical stained image includes an area corresponding to an immunohistochemical stained tissue stained by an immunohistochemistry (IHC) staining method for staining a predetermined target biomarker; Calculating the staining intensity by immunohistochemical staining for each pixel of the mth immunohistochemical stained image; generating an n-th feature vector of the m-th immunohistochemical stained image based on the staining intensity for each pixel of the m-th immunohistochemical stained image and a predetermined n-th staining intensity reference value, for all integers n of 1≦n≦N (where N is an integer equal to or greater than 2); generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image; The step of generating an n-th feature vector of the m-th immunohistochemical stained image based on the staining intensity for each pixel of the m-th immunohistochemical stained image and a predetermined n-th staining intensity reference value, a step of comparing the staining intensity for each pixel of the mth immunohistochemical stained image with the nth staining intensity reference value to generate an nth binary image corresponding to the mth immunohistochemical stained image (here, the nth binary image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity); A method for training a machine learning model, comprising: generating an nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image.

2. The learning method for a machine learning model according to claim 1, characterized in that for all integers i, 1≦i≦(N−1), the i-th staining intensity reference value is smaller than the (i+1)-th staining intensity reference value.

3. The step of acquiring an mth immunohistochemical stained image includes: Obtaining an mth original pathology image by scanning a pathology slide stained with a staining agent obtained by mixing an immunohistochemical staining reagent and a control staining reagent; and separating an immunohistochemically stained portion stained with the immunohistochemical staining reagent and a control stained portion stained with the control staining reagent from the mth original pathology image, thereby generating an mth immunohistochemically stained image corresponding to the mth original pathology image, and an mth control stained image corresponding to the mth original pathology image.

4. generating an n-th feature vector of the m-th immunohistochemical stained image based on the n-th binarized image corresponding to the m-th immunohistochemical stained image, generating an nth feature vector of the mth immunohistochemical stained image, each of which is a component of a vector, the nth calculated value being calculated based on the nth binarized image corresponding to the mth immunohistochemical stained image and the mth control stained image; The at least one calculated value is The method for training a machine learning model according to claim 3 , comprising at least one of a percentage of stained cell tissue, a percentage of cells whose cell membranes are stained, and a percentage of cells whose cell nuclei are stained.

5. generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image, 2. The method for training a machine learning model according to claim 1, further comprising: generating the mth training data, the mth training data being composed of pairs of a first staining intensity reference value and a first feature vector to pairs of an Nth staining intensity reference value and an Nth feature vector, and labeled with an expression score of human epidermal growth factor receptor 2 (HER2).

6. generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image, generating the mth learning data, which is composed of a pair of a first staining intensity reference value and a first feature vector difference value to a pair of an (N-1)th staining intensity reference value and an (N-1)th feature vector difference value, and is labeled with an expression score of an ER (estrogen receptor) or a PR (progesterone receptor); 3. The machine learning model learning method of claim 2, wherein for all integers i, 1≦i≦(N−1), the i-th feature vector difference value is the difference value between the (i+1)-th feature vector and the i-th feature vector.

7. A method for providing a judgment result for a predetermined judgment target pathological specimen through a machine learning model trained by the machine learning model training method according to claim 1, A computing system acquires a target immunohistochemical stained image, the target immunohistochemical stained image including an area corresponding to the immunohistochemical stained tissue of the target pathological specimen stained by the immunohistochemical staining method; The computing system calculates the staining intensity of the immunohistochemical stain for each pixel of the immunohistochemical stained image to be judged; The computing system generates an n-th feature vector of the immunohistochemical stained image to be judged based on the staining intensity of each pixel of the immunohistochemical stained image to be judged and the n-th staining intensity reference value for all integers n, where 1≦n≦N; The computing system generates input data based on a first staining intensity reference value to an Nth staining intensity reference value and a first feature vector to an Nth feature vector of the immunohistochemical stained image to be judged; The computing system outputs a judgment result for the pathological specimen to be judged, the judgment result being judged by the machine learning model based on the input data; Including, The step of generating an n-th feature vector of the immunohistochemical stained image to be judged based on the staining intensity for each pixel of the immunohistochemical stained image to be judged and the n-th staining intensity reference value, a step of comparing the staining intensity for each pixel of the immunohistochemical stained image to be judged with the nth staining intensity reference value to generate an nth binary image corresponding to the immunohistochemical stained image to be judged (here, the nth binary image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity); generating an nth feature vector of the subject immunohistochemical stained image based on the nth binarized image corresponding to the subject immunohistochemical stained image.

8. A computer program recorded on a medium for being installed in a data processing device and for carrying out the method according to any one of claims 1 to 7.

9. A computer-readable recording medium having a computer program recorded thereon for carrying out the method according to any one of claims 1 to 7.

10. A machine learning model training system, comprising: A processor; a memory for storing a computer program; Including, The computer program, when executed by the processor, causes the computing system to perform a method for training a machine learning model, The learning method of the machine learning model includes: A step in which a learning system of the machine learning model generates a learning dataset including M pieces of individual learning data (where M is a natural number equal to or greater than 2); The machine learning model training system includes a step of training the machine learning model based on the training dataset; The step of generating a training data set including the M individual training data includes a step of generating m-th training data included in the training data set for all integers m, where 1≦m≦M; The step of generating m training data includes: acquiring an mth immunohistochemical stained image, wherein the mth immunohistochemical stained image includes an area corresponding to an immunohistochemical stained tissue stained by an immunohistochemistry (IHC) staining method for staining a predetermined target biomarker; Calculating the staining intensity by immunohistochemical staining for each pixel of the mth immunohistochemical stained image; generating an n-th feature vector of the m-th immunohistochemical stained image based on the staining intensity for each pixel of the m-th immunohistochemical stained image and a predetermined n-th staining intensity reference value, for all integers n of 1≦n≦N (where N is an integer equal to or greater than 2); generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image; The step of generating an n-th feature vector of the m-th immunohistochemical stained image based on the staining intensity for each pixel of the m-th immunohistochemical stained image and a predetermined n-th staining intensity reference value, a step of comparing the staining intensity for each pixel of the mth immunohistochemical stained image with the nth staining intensity reference value to generate an nth binary image corresponding to the mth immunohistochemical stained image (here, the nth binary image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity); and generating an nth feature vector of the mth immunohistochemical stained image based on the nth binarized image corresponding to the mth immunohistochemical stained image.

11. The machine learning model learning system according to claim 10, wherein for all integers i, 1≦i≦(N−1), the i-th staining intensity reference value is smaller than the (i+1)-th staining intensity reference value.

12. The step of acquiring an mth immunohistochemical staining image includes: Obtaining an mth original pathology image by scanning a pathology slide stained with a mixed staining agent of an immunohistochemical staining reagent and a control staining reagent; and separating an immunohistochemically stained portion stained with the immunohistochemical staining reagent and a control stained portion stained with the control staining reagent from the mth original pathology image, to generate an mth immunohistochemically stained image corresponding to the mth original pathology image, and an mth control stained image corresponding to the mth original pathology image.

13. generating an n-th feature vector of the m-th immunohistochemical stained image based on the n-th binarized image corresponding to the m-th immunohistochemical stained image, generating an nth feature vector of the mth immunohistochemical stained image, each of which is a component of a vector, the nth calculated value being calculated based on the nth binarized image corresponding to the mth immunohistochemical stained image and the mth control stained image; The at least one calculated value is The system for training a machine learning model of claim 12 , comprising at least one of a percentage of stained cell tissue, a percentage of cells with stained cell membranes, and a percentage of cells with stained cell nuclei.

14. generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image, 11. The learning system for a machine learning model according to claim 10, further comprising a step of generating the mth training data, the mth training data being composed of pairs of a first staining intensity reference value and a first feature vector to pairs of an Nth staining intensity reference value and an Nth feature vector, and labeled with an expression score of human epidermal growth factor receptor 2 (HER2).

15. generating the mth learning data based on the first to Nth staining intensity reference values ​​and the first to Nth feature vectors of the mth immunohistochemical stained image, generating the mth learning data, which is composed of a pair of a first staining intensity reference value and a first feature vector difference value to a pair of an (N-1)th staining intensity reference value and an (N-1)th feature vector difference value, and is labeled with an expression score of an ER (estrogen receptor) or a PR (progesterone receptor); 12. The machine learning model learning system of claim 11, wherein for all integers i, 1≦i≦(N−1), the i-th feature vector difference value is the difference value between the (i+1)-th feature vector and the i-th feature vector.

16. A system for providing judgment results for pathology samples, comprising: A processor; a memory for storing a computer program; Including, When the computer program is executed by the processor, the computer program causes the judgment result providing system to execute a method for providing a judgment result for a predetermined judgment target pathological specimen through a machine learning model trained by the machine learning model training method according to claim 1, The method for providing the judgment result includes: The judgment result providing system acquires an immunohistochemically stained image of a judgment target (wherein the immunohistochemically stained image of the judgment target includes an area corresponding to the immunohistochemically stained tissue of the judgment target pathological specimen stained by the immunohistochemical staining method); a step of the judgment result providing system calculating a staining intensity by immunohistochemical staining for each pixel of the judgment target immunohistochemical stained image; the judgment result providing system generates an n-th feature vector of the judgment target immunohistochemical stained image based on the staining intensity of each pixel of the judgment target immunohistochemical stained image and the n-th staining intensity reference value for all integers n of 1≦n≦N; The judgment result providing system generates input data based on the first staining intensity reference value to the Nth staining intensity reference value and the first feature vector to the Nth feature vector of the judgment target immunohistochemical stained image; The judgment result providing system outputs a judgment result for the pathological specimen to be judged, the judgment result being judged by the machine learning model based on the input data, The step of generating an n-th feature vector of the immunohistochemical stained image to be judged based on the staining intensity for each pixel of the immunohistochemical stained image to be judged and the n-th staining intensity reference value, a step of comparing the staining intensity for each pixel of the immunohistochemical stained image to be judged with the nth staining intensity reference value to generate an nth binary image corresponding to the immunohistochemical stained image to be judged (here, the nth binary image is an image divided into an area having a staining intensity greater than the nth staining intensity reference value and an area not having such a staining intensity); generating an nth feature vector for the subject immunohistochemical stained image based on the nth binarized image corresponding to the subject immunohistochemical stained image.

Images