Apparatus and method for determining possibility of cancer recurrence

Abstract

To provide a new device which can easily determine a possibility of recurrence after treating a cancer.SOLUTION: A determination device 1 comprises: an image acquisition unit 11 which acquires a slide image of a cancer tissue; an extraction unit 12 which extracts a plurality of partial images from the slide image; a generation unit 13 which determines which of a cancer cell region and interstitial region corresponds to the partial image and generates a feature vector; a clustering unit 14 which performs clustering to a corresponding cluster on the basis of the type and the feature vector; a combination pattern decision unit 15 which decides a combination pattern of a cluster group indicating in what combination each cluster of the plurality of partial images extracted from the slide image corresponds to a plurality of cluster groups obtained by grouping the clusters in a combination between the type and the presence / absence of recurrence; a determination unit 16 which determines a recurrence possibility of a cancer tissue on the basis of the pattern; and an output unit 17 which outputs a determination result of the recurrence possibility of a cancer tissue.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a recurrence determination device and a recurrence determination method for determining the possibility of cancer recurrence. [Background technology]

[0002] There are various known cancer treatment methods, but in clinical practice, it is necessary to take into account the possibility of recurrence as well as the patient's wishes and QOL. However, since there are various differences between patients in terms of the extent of cancer, physical condition, etc., it is very difficult to make treatment decisions that take into account the possibility of recurrence. Summary of the Invention [Problem to be solved by the invention]

[0003] Therefore, an object of the present invention is to provide a new device and method that can easily determine the possibility of recurrence of cancer after treatment. [Means for solving the problem]

[0004] In order to achieve the above object, the present invention provides a method for determining the possibility of cancer recurrence, comprising: The apparatus includes an image acquisition unit, an extraction unit, a generation unit, a clustering unit, a combination pattern determination unit, a classification unit, a judgment unit, and an output unit, The image acquisition unit acquires a slide image of a cancer tissue of a subject, The extraction unit extracts a plurality of partial images from the slide image, the generating unit uses a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and generates a feature vector; the clustering unit clusters each of the partial images into a corresponding cluster based on the type and the feature vector using a second model; the combination pattern determination unit determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence; The determination unit determines a possibility of recurrence of the cancer tissue of the slide image based on a combination pattern of the cluster groups of the slide image by a third model; The output unit outputs a result of a determination of a possibility of recurrence of the cancer tissue in the slide image, The first model is The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector; The second model is a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal regions into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data; The third model is a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data; The combination pattern of the cluster group of the cancer patients is For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is designated as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is designated as a cluster without recurrence; For each of the learning slide images of the plurality of cancer patients, the clusters of the plurality of partial images contained therein are determined as a pattern indicating in what combination they correspond to a cluster group of recurrence in the cancer cell region, a cluster group of no recurrence in the cancer cell region, a cluster group of recurrence in the stromal region, and a cluster group of no recurrence in the stromal region. It is characterized by:

[0005] The method for determining the possibility of cancer recurrence of the present invention comprises the steps of: The method includes an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a judgment step, and an output step, and these steps are executed by a computer; The image acquisition step includes acquiring a slide image of a cancer tissue of a subject, The extracting step extracts the slide image into a plurality of partial images, the generating step includes using a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and to generate a feature vector; the clustering step includes clustering each of the partial images into a corresponding cluster based on the type and the feature vector using a second model; The combination pattern determination step determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence, The determination step includes: A third model is used to determine the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster group of the slide image; The output step outputs a result of the assessment of the possibility of recurrence of the cancer tissue in the slide image, The first model is The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector; The second model is a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal regions into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data; The third model is a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data; The combination pattern of the cluster group of the cancer patients is For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is designated as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is designated as a cluster without recurrence; The method is characterized in that, for each of the learning slide images of the multiple cancer patients, the clusters of the multiple partial images contained therein are determined as a pattern indicating in what combination they fall into a cluster group with recurrence in the cancer cell region, a cluster group with no recurrence in the cancer cell region, a cluster group with recurrence in the stromal region, and a cluster group with no recurrence in the stromal region.

[0006] The program of the present invention is characterized by causing a computer to execute the recurrence determination method of the present invention for determining the possibility of cancer recurrence.

[0007] The recording medium of the present invention is a computer-readable recording medium having the program of the present invention recorded thereon. Effect of the Invention

[0008] According to the present invention, for example, by using a model generated by learning using a partial image of a cancer cell region and a partial image of a stromal region derived from a slide image of a cancer patient who has been given adjuvant treatment after surgery and has been diagnosed with the presence or absence of recurrence, it is possible to more easily determine the possibility of recurrence of a subject after adjuvant treatment. Therefore, according to the present invention, it is possible to select whether or not further adjuvant treatment is required after surgery and the type of adjuvant treatment, taking into account the possibility of recurrence of a cancer patient after surgery. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of a recurrence determination device according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the recurrence determination device of the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the recurrence determination device of the present invention, for example, in the type information, the number of types of the cancer cell regions is 2 to 20, and the number of types of the stromal regions is 2 to 10.

[0011] In the recurrence determination device of the present invention, for example, in the type information, the types of the cancer cell regions are a cancer infiltrated region and a cancer main region, and the stromal regions are a stromal mature region and a stromal immature region.

[0012] In the recurrence determination device of the present invention, the cluster groups include, for example, a cluster group with recurrence in wet cancer regions, a cluster group with no recurrence in wet cancer regions, a cluster group with recurrence in main cancer regions, a cluster group with no recurrence in main cancer regions, a cluster group with recurrence in mature interstitial regions, a cluster group with no recurrence in mature interstitial regions, a cluster group with recurrence in immature interstitial regions, and a cluster group with no recurrence in immature interstitial regions.

[0013] In the recurrence determination device of the present invention, for example, the clustering section has 100 to 1000 clusters.

[0014] In the recurrence assessment device of the present invention, for example, the third model is a model set including a model with adjuvant treatment generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have undergone any treatment after surgery, and a model without adjuvant treatment generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have not undergone any adjuvant treatment, The determination unit is using the model set as the third model; The likelihood of recurrence is determined using the model with the adjuvant treatment, and the likelihood of recurrence is determined using the model without the adjuvant treatment.

[0015] In the recurrence determination method of the present invention, for example, the determination unit makes a comprehensive determination based on the determination result of the possibility of recurrence using the model with supplemental treatment and the determination result of the possibility of recurrence using the model without supplemental treatment.

[0016] In the recurrence determination method of the present invention, for example, the type information includes 2 to 20 types of the cancer cell regions and 2 to 10 types of the stromal regions.

[0017] In the recurrence determination method of the present invention, for example, the type information indicates that the types of the cancer cell regions are a cancer infiltrated region and a cancer main region, and the types of the stromal regions are a stromal mature region and a stromal immature region.

[0018] In the recurrence determination method of the present invention, the cluster groups include, for example, a cluster group with recurrence in wet cancer regions, a cluster group with no recurrence in wet cancer regions, a cluster group with recurrence in main cancer regions, a cluster group with no recurrence in main cancer regions, a cluster group with recurrence in mature interstitial regions, a cluster group with no recurrence in mature interstitial regions, a cluster group with recurrence in immature interstitial regions, and a cluster group with no recurrence in immature interstitial regions.

[0019] In the recurrence determination method of the present invention, for example, the number of clusters in the clustering step is 100 to 1,000.

[0020] In the recurrence assessment method of the present invention, for example, the third model is a third model set including a model with adjuvant treatment generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have undergone any treatment after surgery, and a model without adjuvant treatment generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients among the plurality of cancer patients who have not undergone any adjuvant treatment, The determination step includes: using the third model set as the third model; The likelihood of recurrence is determined using the model with the adjuvant treatment, and the likelihood of recurrence is determined using the model without the adjuvant treatment.

[0021] In the recurrence assessment method of the present invention, for example, the assessment step comprises making a comprehensive assessment based on the assessment result of the possibility of recurrence using the model with adjuvant treatment and the assessment result of the possibility of recurrence using the model without adjuvant treatment.

[0022] According to the present invention, for example, the possibility of recurrence in a cancer patient when the patient is treated (specifically, adjuvant treatment after surgery) can be determined. The type of cancer that is the subject of the present invention is not particularly limited, and includes any cancer from which a pathological image can be obtained, such as digestive system cancer, respiratory system cancer, reproductive system cancer, skin cancer, and nervous system cancer. The means of the adjuvant treatment is not particularly limited, and includes, for example, drug therapy (chemotherapy), radiation therapy, immunotherapy, transplantation, etc.

[0023] In the present invention, the target cancer is not particularly limited, but in determining recurrence, for example, it is preferable that the type of cancer of the subject and the type of cancer of the model set are matched. In addition, since the present invention can determine the possibility of recurrence after adjuvant treatment performed after surgery, it is preferable to use a model set generated using learning data related to adjuvant treatment that matches the expected method of adjuvant treatment.

[0024] In the present invention, the first model, the second model and the third model are collectively referred to as a model set.

[0025] The embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiments. In each of the drawings, the same parts are given the same reference numerals. The explanations of each embodiment can be mutually incorporated unless otherwise specified. The configurations of each embodiment can be combined unless otherwise specified.

[0026] [Embodiment 1] An example of a determination device for determining the possibility of cancer recurrence (hereinafter also referred to as a recurrence determination device) and a determination method (hereinafter also referred to as a recurrence determination method) of the present invention will be described with reference to the drawings. In the present invention, determination can be read as estimation.

[0027] 1 is a block diagram showing an example of the configuration of the recurrence determination device 1 of this embodiment. The recurrence determination device 1 includes an image acquisition unit 11, an extraction unit 12, a generation unit 13, a clustering unit 14, a combination pattern determination unit 15, a determination unit 16, and an output unit 17. The recurrence determination device 1 further includes, for example, a model set 19. The recurrence determination device 1 further includes, for example, a storage unit 18, and the model set 19 is stored in the storage unit 18. The model set 19 includes a first model 191, a second model 192, and a third model 193. The recurrence determination device 1 is also referred to as, for example, a recurrence determination system.

[0028] 2 illustrates an example of a block diagram of the hardware configuration of the recurrence determination device 1. The recurrence determination device 1 includes, for example, a CPU (Central Processing Unit) 101, a memory 102, a bus 103, an input device 104, a display unit 105, a communication device 106, a storage device 107, and an imaging device 109. The components of the recurrence determination device 1 are connected to each other via the bus 103 by their respective interfaces (I / F).

[0029] The CPU 101 is a processor that is responsible for the overall control of the recurrence determination device 1, and is not limited to a CPU and may be another processor. In the recurrence determination device 1, the CPU 101 executes, for example, the program of the present invention and other programs, and also reads and writes various types of information.

[0030] The recurrence determination device 1 can be connected to a communication network by, for example, a communication device 106 connected to a bus 103, and can also be connected to an external device via the communication network. The external device is not particularly limited, and examples thereof include a terminal, a PC (personal computer), and a server. The terminal is, for example, a smartphone, a mobile phone, and a tablet. The connection method between the recurrence determination device 1 and the external device is not particularly limited, and may be, for example, a wired connection or a wireless connection. The wired connection may be, for example, a cord connection or a cable connection for using a communication network. The wireless connection may be, for example, a connection using a communication network or a connection using wireless communication. The communication network is not particularly limited, and for example, a known communication network may be used, and examples thereof include an Internet line, a telephone line, a LAN (Local Area Network), and WiFi (Wireless Fidelity).

[0031] The memory 102 includes, for example, a main memory, and the main memory is also called a primary storage device. When the CPU 101 performs processing, the memory 102 reads various operation programs 108, such as the program of the present invention, stored in, for example, an auxiliary storage device described later, and the CPU 101 receives data from the memory 102 and executes the program 108. Also, for example, the memory 102 reads a model set 19 (first model 191, second model 192, third model 193) in the present invention stored in, for example, an auxiliary storage device described later, and the CPU 101 receives data from the memory 102 and executes the program 108 using the model set 19. The main memory is, for example, a RAM (random access memory). The memory 102 further includes, for example, a ROM (read only memory).

[0032] The storage device 107 includes, for example, the above-mentioned storage unit 18, and is also called an auxiliary storage device in contrast to the main memory (primary storage device). The storage device 107 includes, for example, a storage medium and a drive for reading and writing data from and to the storage medium. The storage medium is not particularly limited, and may be, for example, an internal or external type, and may be, for example, a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, memory card, etc., and the drive is not particularly limited. The storage device 107 may be, for example, a hard disk drive (HDD) in which a storage medium and a drive are integrated. The storage device 107 stores, for example, the program 108 and the model set 19, as described above. Various information and slide images input to the recurrence determination device 1, information obtained in the recurrence determination device 1, etc. may be stored in the storage device 107 (storage unit 18), for example.

[0033] The recurrence determination device 1 may further include, for example, an input device 104. The input device 104 is, for example, a scanner, a touch panel, a keyboard, etc. The recurrence determination device 1 may input, for example, subject information, slide images, etc. through the input device 104, or may input them through the input device 104 via the communication device 106.

[0034] The recurrence determination device 1 may further include, for example, an imaging device 109, which may capture an image of, for example, a slide of cancer tissue and store the obtained slide image in the storage unit 18. The imaging device 109 is not particularly limited and may be, for example, a camera such as a CCD camera.

[0035] The recurrence determination device 1 may have, for example, a display unit 105. The display unit 105 is, for example, a display, and specific examples include an LED display and a liquid crystal display. The recurrence determination device 1 may, for example, display the obtained determination result on the display unit 105 via the output unit 17, or may output the result from the output unit 16 to an external terminal or the like via the communication device 106.

[0036] The recurrence determination device 1 will now be described in further detail.

[0037] In the recurrence assessment device 1, the image acquisition unit 11 acquires a slide image of a cancer tissue of a subject. The subject is, for example, a cancer patient, and the slide image can be acquired, for example, by imaging a cancer tissue excised by a surgical operation of the cancer patient.

[0038] The type of the slide image is not particularly limited, and is, for example, an image of a cancer tissue slice that has been subjected to a staining process such as immunostaining, etc. The actual size of the slice shown in one slide image is, for example, 20 to 40 mm in length × 20 to 40 mm in width.

[0039] The slide image of the subject may be linked to information about the subject and stored in the storage unit 18. The information about the subject may include, for example, identification information such as a name, cancer morbidity information including the type and stage of cancer, and treatment information.

[0040] The extraction unit 12 extracts a plurality of partial images from the slide image. The extraction of the partial images is also referred to as, for example, cutting out the partial images. The number of partial images obtained from one slide is not particularly limited, and may be, for example, 1000 to 40,000. The method of extracting the partial images from the slide image is not particularly limited, and may be performed, for example, as follows. That is, in the recurrence determination device 1, the slide image is displayed on the display unit 105. Then, the operator selects a partial area from the slide image on the display unit 105 and inputs the selection via the input device 104, whereby the partial image can be extracted. The partial area can be selected, for example, by designating an area with a pointer, designating coordinates, or the like on the slide image displayed on the display unit 105.

[0041] In addition, in the recurrence determination device 1, the extraction unit 12 may automatically extract a partial image, for example. The automatic extraction of a partial image can be performed, for example, as follows. That is, for example, a virtual magnification (for example, 5x, 10x, 20x, 40x) is specified based on the microscope magnification when the slide image was captured. As a specific example, for example, if pixels are acquired every other pixel from an image file of the slide image captured at a magnification of 40x, an image captured at a virtual magnification of 20x can be acquired. Then, the size of the partial image (for example, a square with one side of 128, 256, 512 pixels) is specified, and the partial image is cut out in a tile shape from the slide image. The determination as to whether the partial image contains tissue can be performed, for example, by extracting pixels with a saturation equal to or greater than a predetermined threshold value for each pixel value of the partial image, and determining the proportion of the pixels (for example, 30% or more) of the pixels to the total number of pixels of one partial image to determine whether the partial image contains tissue.

[0042] The generation unit 13 determines whether each of the partial images corresponds to a cancer cell region or a stromal region and generates a feature vector using a first model 191. According to the first model 191, by providing the partial image to the model, the type of the object captured in the partial image (i.e., type of whether it is a cancer cell region or a stromal region, etc.) can be estimated, and a feature vector of the partial image can be generated.

[0043] The clustering unit 14 clusters each of the partial images into a corresponding cluster based on the type and the feature vector using the second model 192. According to the second model 192, by providing the type of the partial image and its feature vector to the model, it is possible to determine which of a plurality of clusters the partial image belongs to.

[0044] The combination pattern determination unit 15 determines a combination pattern of cluster groups indicating what combination each of the clusters of the multiple partial images extracted from the slide image corresponds to for the multiple cluster groups obtained by grouping the clusters according to the combination of the type and the presence or absence of recurrence. As described later, the multiple clusters classified by clustering can be further classified into multiple groups (cluster groups) according to the combination of the type and the recurrence information on the presence or absence of recurrence. Therefore, it is possible to determine the combination pattern of the cluster group according to what combination each of the clusters of the multiple partial images derived from the slide image of the subject corresponds to the cluster group.

[0045] The determination unit 16 determines the possibility of recurrence of the cancer tissue of the slide image based on the combination pattern of the cluster groups of the slide image by the third model 193. According to the third model 193, it is possible to determine whether the subject has a possibility of recurrence or not, based on the combination pattern of the slide image of the subject.

[0046] The output unit 17 outputs the determination result of the possibility of recurrence of the cancer tissue in the slide image of the subject.

[0047] The recurrence determination method of the present invention includes, for example, an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a determination step, and an output step, as described above, and these steps are executed by a computer. The recurrence determination method of the present invention can be executed, for example, by using the recurrence determination device of the present invention. For example, unless otherwise specified, the description of the recurrence determination device of the present invention can be used for the recurrence determination method of the present invention.

[0048] The image acquiring step acquires a slide image of the cancer tissue of the subject, and this step can be executed by, for example, the image acquiring unit 11 of the recurrence determination device 1.

[0049] The extraction step is to extract the slide image into a plurality of partial images, and this step can be executed by, for example, the extraction unit 12 of the recurrence determination device 1.

[0050] The generating step determines whether each of the partial images corresponds to a cancer cell region or a stromal region, and generates a feature vector, using the first model. This step can be executed by the generating unit 13 of the recurrence determination device 1, for example, using the first model 191.

[0051] The clustering step is to cluster each of the partial images into a corresponding cluster based on the type and the feature vector by the second model. This step can be executed by the clustering unit 14 of the recurrence determination device using the second model 192, for example.

[0052] The combination pattern determination step determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to the combination of the type and the presence or absence of recurrence. This step can be executed, for example, by the combination pattern determination unit 15 of the recurrence determination device 1.

[0053] The determination step determines the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster groups of the slide image by a third model. This step can be executed by the determination unit 16 of the recurrence determination device 1 using the third model 193, for example.

[0054] The output step outputs a determination result of the possibility of recurrence of the cancer tissue in the slide image, and this step can be executed by, for example, the output unit 17 of the recurrence determination device 1.

[0055] [Embodiment 2] The model set in the recurrence determination device and recurrence determination method of the present invention is the model set of the present invention. The model set of the present invention can be applied to the descriptions in other embodiments including the first embodiment. Hereinafter, the model set of the present invention will be described by way of an example, but the present invention is not limited to this example.

[0056] The model set of the present invention is a model set for the recurrence determination device of the present invention or a model set for the recurrence determination method of the present invention, and includes the first model, the second model, and the third model. Each model is a learning model, and the form thereof is not particularly limited, and is, for example, a support vector machine.

[0057] The first model is The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector; The second model is a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal regions into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data; The third model is a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data; The combination pattern of the cluster group of the cancer patients is For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is designated as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is designated as a cluster without recurrence; For each of the multiple learning slide images of the cancer patients, a pattern is determined that indicates in what combination each of the clusters of the multiple partial images contained therein corresponds to a cluster group with recurrence in the cancer cell region, a cluster group with no recurrence in the cancer cell region, a cluster group with recurrence in the stromal region, and a cluster group with no recurrence in the stromal region.

[0058] As an example of the generation of a model set of the present invention, the model set 19 including the first model 191, the second model 192, and the third model 193 in the first embodiment will be described.

[0059] The model set may be generated without distinguishing the type of cancer (the type of tissue in which the cancer occurs), but it is preferable to generate the model set according to the type of cancer (for example, each type of cancer, such as colon cancer, pancreatic cancer, lung cancer, etc.) because this leads to a more reliable judgment. In the following examples, the target cancer is assumed to be a "specific cancer X", but the present invention is not limited to this example.

[0060] To generate the model set 19, information on cancer patients who have undergone auxiliary treatment after surgery and have been clinically diagnosed as having recurrence as a result is used. The information includes, for example, slide images of cancer tissues removed by surgery; morbidity information such as the type and stage of cancer; the presence or absence of auxiliary treatment, the content of auxiliary treatment, and progress information after auxiliary treatment; and identification information such as an ID that identifies the patient, which are linked to each other. The cancer patients in the description of the model set 19 are referred to as learning cancer patients in order to distinguish them from subjects who are to be judged for the possibility of recurrence. In addition, in the present invention, subjects whose cancer has not recurred due to auxiliary treatment are also referred to as cancer patients for convenience in terms of their history of cancer.

[0061] As described above, the first model 191 is a model for determining whether a region corresponds to a cancer cell region or a stromal region, and for generating a feature vector. The first model 191 can be generated, for example, as follows.

[0062] First, as the learning slide images, slide images of cancer tissues before the adjuvant treatment of a cancer patient who has undergone adjuvant treatment after surgery for cancer X and has been clinically diagnosed as having recurrence as a result are used. The number of slide images is not particularly limited, but from the viewpoint of generating a more reliable model, for example, the number of cancer patients for learning is preferably 400 or more.

[0063] The type of the slide image is not particularly limited, and is, for example, an image of a cancer tissue slice that has been subjected to a staining process such as immunostaining, etc. The actual size of the slice shown in one slide image is, for example, 20 to 40 mm in length × 20 to 40 mm in width.

[0064] Next, a plurality of partial images are extracted from the slide image. The extraction of the partial images is also referred to as, for example, cutting out the partial images. The number of partial images obtained from one slide is not particularly limited, and is, for example, 1000 to 40,000. The method of extracting partial images from the slide image is not particularly limited, and for example, based on pathology, a part of the slide image showing a cancer cell region and a part of the slide image showing a stromal region can be selected and extracted. The partial images extracted from the slide image are also referred to as, for example, ROI (Region of interest) images. The partial images are, for example, tagged (annotated) and used for learning.

[0065] Then, the partial image and type information of the partial image (i.e., information on whether it is a cancer cell region or a stromal region) are used as learning data, and a first model 19 can be generated by performing learning. According to the first model 19, even if it is unclear whether a partial image is a cancer cell region or a stromal region, for example, the type can be determined, and a feature vector of the partial image can be generated.

[0066] The type information is not limited to, for example, two types, a cancer cell region and a stromal region, and the cancer cell region may be further classified, for example, into 2 to 20 types, and the stromal region may be further classified, for example, into 2 to 10 types. As a specific example, the cancer cell region may be classified, for example, into two types, an invasive part (also called an advanced part) region and a main part region, and the stromal region may be classified, for example, into two types, a mature region and an immature region.

[0067] As described above, the second model 192 is a clustering model that classifies cancer cell regions into a plurality of clusters and stromal regions into a plurality of clusters. The second model 192 can be generated, for example, as follows.

[0068] The second model 192 can be generated by performing learning using, as learning data, the type information of the partial image and the feature vector of the partial image linked thereto, which were also used to generate the first model 191. According to the second model 192, as described later, a plurality of partial images extracted from a slide image of a subject can be classified into corresponding clusters.

[0069] In the learning, for example, the number of clusters to be classified can be set arbitrarily. As a specific example, for example, in the case of rough classification, the number of clusters may be about 10 to 50, in the case of medium classification, the number of clusters may be about 50 to 100, and in the case of detailed classification, the number of clusters may be about 100 to 1000.

[0070] As described above, the third model 193 is a model for determining the presence or absence of a recurrence possibility. The third model 193 can be generated, for example, as follows.

[0071] First, the second model 192 is used to perform clustering for each partial image extracted from the learning slide image of the learning cancer patient. Since the presence or absence of recurrence of the learning cancer patient is known, for each cluster, the partial images derived from the learning cancer patient who recurred after supplementary treatment and the partial images derived from the learning cancer patient who did not recur after supplementary treatment are counted for the partial images contained therein. As a result, for each cluster, the count number of partial images derived from the learning cancer patient with recurrence and the count number of partial images derived from the learning cancer patient with recurrence are obtained. Then, for each cluster, the count number of recurrence and the count number of no recurrence are compared, and, for example, if the count number of recurrence is significantly larger than the other, the cluster is determined as a cluster with recurrence, and if the count number of no recurrence is significantly larger than the other, the cluster is determined as a cluster without recurrence. The degree of significance is not particularly limited and can be set arbitrarily, and may be set as a difference in number (e.g., 50 or more times larger, etc.) or as a ratio to the other (e.g., 1.5 times larger, etc.). Also, clusters that do not show significant differences between the two may be excluded, for example.

[0072] Next, a combination pattern of cluster groups is determined for each of the plurality of learning slide images of cancer patients.

[0073] Here, as an example, a case will be described in which the type information includes four types, for example, an invaded area of ​​cancer cells, a main area of ​​cancer cells, a mature area of ​​the stroma, and an immature area of ​​the stroma. A plurality of clusters classified by clustering based on a combination of the type information and the presence or absence of recurrence are classified into eight types of groups as shown in Table 1 below. Suppose that the plurality of partial images from the learning slide image of the learning cancer patient A are cluster Nos. 1, 15, 19, and 27. In this case, the combination pattern of the cluster groups of the learning slide image is group 1 / group 4 / group 5 / group 7.

[0074] [Table 1]

[0075] Such a combination pattern is determined for each of the slide images of the plurality of learning cancer patients. Then, the combination patterns and the recurrence information (information on the presence or absence of recurrence) linked thereto for the plurality of learning cancer patients are used as learning data, and a third model 193 can be generated by performing learning. According to the third model 193, as described below, it is possible to determine the possibility of recurrence from the combination pattern of the cluster group of the slide image of the subject.

[0076] As exemplified in the first embodiment, in the combination pattern step using the combination pattern determination unit 15, a combination pattern of cluster groups is determined for the slide images of the subject. In this case, taking the example of Table 1, if the clusters of the multiple partial images derived from the slide images of the subject are cluster Nos. 1, 15, 19, and 27, the combination pattern of the cluster groups of the slide images can be determined to be the combination pattern of group 1 / group 4 / group 5 / group 7.

[0077] (Variation 1) The third model may be, for example, a third model (3M) generated using learning data from all learning patients, regardless of whether or not any auxiliary treatment was performed, as described above, or may be a set of two third models generated by dividing the learning data from the learning patients into two types and using them for learning, as in this modified example. That is, it may be a set of a third model with auxiliary treatment (3M(+)) generated using learning data from learning patients who have received any auxiliary treatment, and a third model without auxiliary treatment (3M(-)) generated using learning data from learning patients who have not received any auxiliary treatment. In the recurrence determination device or recurrence determination method of the present invention, by using a third model set including such two types of third models, it is possible to determine, for example, the presence or absence of recurrence when any auxiliary treatment is performed, and the presence or absence of recurrence when any auxiliary treatment is not performed. This makes it possible to determine, for example, whether an optional auxiliary treatment is suitable for the subject from the viewpoint of recurrence. The optional auxiliary treatment is not particularly limited and can be arbitrarily selected from the treatments described above.

[0078] In this modification, chemotherapy is taken as an example of an optional auxiliary treatment, but the present invention is not limited thereto. That is, in cancer treatment, the presence or absence of recurrence may differ depending on whether chemotherapy is adopted or not performed as an auxiliary treatment method for each patient. Therefore, if the two types of third models generated in this modification, that is, the third model with chemotherapy (3M(+)) and the third model without chemotherapy (3M(-)), are used in the recurrence determination device or recurrence determination method of the present invention, two determination results, recurrence (Re) (+) / no recurrence (-), can be obtained from each of the third models, as shown in Table 2 below. Then, from these two determination results, a more comprehensive determination result can be obtained. Therefore, by using this modification, it is possible to further determine the necessity of auxiliary treatment for the subject and select the method thereof, taking into account the possibility of recurrence.

[0079] [Table 2]

[0080] (Variation 2) In this modification 2, for example, the third model set (3M(+), 3M(-)) of modification 1 may be used in combination with the third model (3M) generated using the learning data derived from all the learning patients described above. In this case, three judgment results of recurrence (Re) (+) / no recurrence (-) can be obtained from the third model with treatment (3M(+)), the third model without treatment (3M(-)), and the third model of all data (3M). Then, from these three judgment results, a more comprehensive judgment result can be obtained. Therefore, by using this modification, it is possible to select auxiliary treatment for the subject, further taking into account the possibility of recurrence.

[0081] In this case, a comprehensive judgment result can be obtained from the results of each third model set (3M(+), 3M(-)) in variant example 1 and the results of the third model (3M) of the entire data, for example, as shown in Table 3 below.

[0082] [Table 3]

[0083] (Variation 3) As shown in Table 1, the clusters are classified into a plurality of cluster groups according to the combination of the type of partial image derived from the learning slide image and the presence or absence of recurrence. In this case, all the clusters in each cluster group may be adopted as constituent clusters, or only the clusters ranked high may be adopted as constituent clusters. The ranking of the clusters may be, for example, a ranking in order of the significant difference in the count number. That is, in the case of a cluster with recurrence, it is assumed that the count number with recurrence is significantly larger than the count number without recurrence, and the clusters can be ranked in order of the magnitude of the count number. Then, only the clusters showing a relatively larger count number than the count number without recurrence can be selected as clusters constituting the cluster group with recurrence. The same is true for the cluster group without recurrence.

[0084] [Embodiment 3] The program of the present invention is a program capable of executing the recurrence determination method of the present invention on a computer. The program of the present invention can cause a processor of a computer to execute the recurrence determination method of the present invention. Alternatively, the program of this embodiment may be recorded, for example, in a computer-readable recording medium. The recording medium is not particularly limited, and examples thereof include the storage media described above. EXAMPLES

[0085] [Example 1] The model set of the present invention was generated under the following conditions using slide images of colon cancer tissues surgically removed from colon cancer patients.

[0086] Total number of patients: 400 Number of patients who relapsed: 140 Number of patients without recurrence: 260 Number of clusters for each type of partial image: 1000 Cluster group composition clusters: Clusters that fall into the top 20% of the ranking Kernel of learning model (SVM): rbf

[0087] The model set of the comparative example and the model set of the example were generated using the following partial images obtained from slide images of cancer patients as learning data. Comparative Example 1A: Partial image of the infiltrated area of ​​cancer Comparative Example 1B: Partial image of the main area of ​​cancer Comparative Example 1C: Partial images of the infiltrated area and the main area of ​​cancer Comparative Example 2A: Partial image of the mature region of the stroma Comparative Example 2B: Partial image of immature areas of the stroma Comparative Example 2C: Partial images of mature stroma and immature stroma Example 1: Partial images of the infiltrated area of ​​cancer, partial images of the main area of ​​cancer, and partial images of the mature area of ​​the stroma

[0088] Using each generated model set, the slide images of the patients used in the learning data were re-evaluated by computer to determine the accuracy rate for recurrence / no recurrence. Additionally, using each of the same model sets, slide images before adjuvant treatment of new subjects who had been diagnosed with recurrence after adjuvant treatment were evaluated to determine the accuracy rate for recurrence / no recurrence. The closer the accuracy rate (Accuracy) is to 1, the higher the accuracy. These results are shown in Table 4 below.

[0089] As shown in Table 4 below, according to the embodiment, the learning data showed a high rate of correct answers, and further, the subjects also showed a similarly high rate of correct answers.

[0090] [Table 4]

[0091] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Industrial Applicability]

[0092] As described above, according to the present invention, for example, by using a model generated by learning using partial images of a cancer cell region and a partial image of a stromal region derived from a slide image of a cancer patient who has been given adjuvant treatment after surgery and has been diagnosed with the presence or absence of recurrence, it is possible to more easily determine the possibility of recurrence of a subject after adjuvant treatment. Therefore, according to the present invention, it is possible to determine whether further adjuvant treatment is necessary after surgery and to select an adjuvant treatment, taking into account the possibility of recurrence of a cancer patient after surgery.

Claims

1. The apparatus includes an image acquisition unit, an extraction unit, a generation unit, a clustering unit, a combination pattern determination unit, a classification unit, a judgment unit, and an output unit, The image acquisition unit acquires a slide image of a cancer tissue of a subject, The extraction unit extracts a plurality of partial images from the slide image, the generating unit uses a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and generates a feature vector; the clustering unit clusters each of the partial images into a corresponding cluster based on the type and the feature vector using a second model; the combination pattern determination unit determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence; The determination unit determines a possibility of recurrence of the cancer tissue in the slide image based on a combination pattern of the cluster groups of the slide image by a third model; The output unit outputs a result of a determination of a possibility of recurrence of the cancer tissue in the slide image, The first model is The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector; The second model is a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal region into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data; The third model is a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data; The combination pattern of the cluster group of the cancer patients is For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence; A pattern is determined that indicates in what combination each cluster of the plurality of partial images included in each learning slide image of the plurality of cancer patients corresponds to a cluster group of recurrence in the cancer cell region, a cluster group of no recurrence in the cancer cell region, a cluster group of recurrence in the stromal region, and a cluster group of no recurrence in the stromal region; A recurrence determination device for determining the possibility of cancer recurrence, wherein the type information indicates that the types of the cancer cell regions are 2 to 20 types and the types of the stromal regions are 2 to 10 types.

2. The recurrence determination device according to claim 1 , wherein in the type information, the types of the cancer cell regions are a cancer infiltration region and a cancer main region, and the types of the stromal regions are a stromal mature region and a stromal immature region.

3. The recurrence determination device of claim 2, wherein the cluster groups include a cluster group with recurrence in invasive cancer regions, a cluster group with no recurrence in invasive cancer regions, a cluster group with recurrence in main cancer regions, a cluster group with no recurrence in main cancer regions, a cluster group with recurrence in mature interstitial regions, a cluster group with no recurrence in mature interstitial regions, a cluster group with recurrence in immature interstitial regions, and a cluster group with no recurrence in immature interstitial regions.

4. 4. The recurrence determination device according to claim 1, wherein the number of clusters in the clustering section is 100 to 1000.

5. The third model is a model set including a model with adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have received any adjuvant treatment after surgery among the plurality of cancer patients, and a model without adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have not received any adjuvant treatment among the plurality of cancer patients, The determination unit is using the model set as the third model; The recurrence determination device according to claim 1 , further comprising: a determination of the possibility of recurrence based on the model with the auxiliary treatment; and a determination of the possibility of recurrence based on the model without the auxiliary treatment.

6. The recurrence determination device according to claim 5 , wherein the determination unit performs a comprehensive determination based on a result of the determination of the possibility of recurrence based on the model with the auxiliary treatment and a result of the determination of the possibility of recurrence based on the model without the auxiliary treatment.

7. The method includes an image acquisition step, an extraction step, a generation step, a clustering step, a combination pattern determination step, a classification step, a judgment step, and an output step, and these steps are executed by a computer; The image acquisition step includes acquiring a slide image of a cancer tissue of a subject, The extracting step extracts the slide image into a plurality of partial images, the generating step includes using a first model to determine whether each of the partial images corresponds to a cancer cell region or a stromal region, and to generate a feature vector; the clustering step includes clustering each of the partial images into a corresponding cluster based on the type and the feature vector using a second model; The combination pattern determination step determines a combination pattern of cluster groups that indicates what combination each of the clusters of the plurality of partial images extracted from the slide image corresponds to for a plurality of cluster groups obtained by grouping clusters according to a combination of the type and the presence or absence of recurrence, The determination step includes: A third model is used to determine the possibility of recurrence of the cancer tissue in the slide image based on the combination pattern of the cluster group of the slide image; The output step outputs a result of the assessment of the possibility of recurrence of the cancer tissue in the slide image, The first model is The model uses, as learning data, partial images extracted from learning slide images of cancer tissues of a plurality of cancer patients, and type information indicating whether the partial images correspond to a cancer cell region or a stromal region, and generates, through learning, a model that determines whether a region corresponds to a cancer cell region or a stromal region, and generates a feature vector; The second model is a clustering model that classifies the cancer cell regions into a plurality of clusters and the stromal region into a plurality of clusters, the clustering model being generated by learning using type information of the partial images and feature vectors of the partial images as learning data; The third model is a model for determining the presence or absence of recurrence, the model being generated by learning using recurrence information regarding the presence or absence of recurrence after adjuvant treatment following surgery and combination patterns of cluster groups for the plurality of cancer patients as learning data; The combination pattern of the cluster group of the cancer patients is For each cluster obtained by the second model, the partial images derived from cancer patients who have recurred after adjuvant treatment following surgery and the partial images derived from cancer patients who have not recurred after adjuvant treatment following surgery are counted, and if the count number of cases with recurrence is significantly large, the cluster is determined as a cluster with recurrence, and if the count number of cases without recurrence is significantly large, the cluster is determined as a cluster without recurrence; A pattern is determined that indicates in what combination each cluster of the plurality of partial images included in each learning slide image of the plurality of cancer patients corresponds to a cluster group of recurrence in the cancer cell region, a cluster group of no recurrence in the cancer cell region, a cluster group of recurrence in the stromal region, and a cluster group of no recurrence in the stromal region; In the type information, the types of the cancer cell regions are 2 to 20 types, and the types of the stromal regions are 2 to 10 types. A method for determining the possibility of cancer recurrence, comprising:

8. The method of determining recurrence according to claim 7 , wherein in the type information, the types of the cancer cell regions are a cancer infiltration region and a cancer main region, and the types of the stromal regions are a stromal mature region and a stromal immature region.

9. The method for determining recurrence according to claim 8, wherein the cluster groups include a cluster group with recurrence in invasive cancer regions, a cluster group with no recurrence in invasive cancer regions, a cluster group with recurrence in main cancer regions, a cluster group with no recurrence in main cancer regions, a cluster group with recurrence in mature interstitial regions, a cluster group with no recurrence in mature interstitial regions, a cluster group with recurrence in immature interstitial regions, and a cluster group with no recurrence in immature interstitial regions.

10. The recurrence determination method according to any one of claims 7 to 9, wherein the number of clusters in the clustering step is 100 to 1000.

11. The third model is a third model set including a model with adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have received any adjuvant treatment after surgery among the plurality of cancer patients, and a model without adjuvant treatment, which is generated by learning using, as the learning data, recurrence information on the presence or absence of recurrence after adjuvant treatment and a combination pattern of cluster groups for cancer patients who have not received any adjuvant treatment among the plurality of cancer patients, The determination step includes: using the third model set as the third model; The method for determining recurrence according to claim 7 , further comprising: determining the likelihood of recurrence using a model with adjuvant treatment; and determining the likelihood of recurrence using a model without adjuvant treatment.

12. The recurrence determination method according to claim 11 , wherein the determination step comprises making a comprehensive determination based on a result of the determination of the possibility of recurrence using the model with the supplemental treatment and a result of the determination of the possibility of recurrence using the model without the supplemental treatment.

13. A program for causing a computer to execute the method for determining the possibility of cancer recurrence according to any one of claims 7 to 12.

14. A computer-readable recording medium having the program according to claim 13 recorded thereon.

Images