Diagnostic imaging support apparatus, diagnostic imaging support method, and diagnostic imaging support program

Abstract

To predict the presence or absence of a molecular biological marker of a brain tumor based on an image of the brain tumor with higher accuracy.SOLUTION: An image diagnosis assistance apparatus 1 includes a first acquisition unit that acquires an image of a head by MRI, an image processing unit 11 that extracts a region including information on a lesion site and a periphery of the lesion site from the acquired image of the head and generates a first image having the extracted region, an extraction unit 12 that extracts a predetermined image feature amount from the first image, a classification unit 14 that classifies presence or absence of a molecular biological marker related to the lesion site using a classifier constructed in advance based on the image feature amount extracted by the extraction unit 12, and a presentation unit 15 that presents assistance information on whether or not the lesion site included in the image of the head has the molecular biological marker based on a classification result by the classification unit 14.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image diagnosis support device, an image diagnosis support method, and an image diagnosis support program. [Background technology]

[0002] In recent years, there has been progress in the development of image diagnostic support technology that uses computers to analyze medical images such as MRI (Magnetic Resonance Imaging) and CT (Computed Tomography) to help diagnose the presence, type, and stage of disease.

[0003] In recent years, machine learning techniques, such as deep learning, have been utilized in the field of image recognition, and machine learning techniques are also being used in the analysis of medical images.

[0004] For example, Patent Document 1 discloses a technology that uses deep learning technology to predict future brain tomography images of a subject based on the brain tomography images of the same subject. Also, Patent Document 2 discloses a technology that uses hyperspectral images to automatically segment brain tissue in a captured image using a machine learning model to determine whether the brain tissue is tumor or normal tissue.

[0005] Furthermore, Patent Document 3 discloses a technology for determining the type of lesion for each of a plurality of cross-sectional images using a learning model that has learned the type of lesion for brain tumor images, using information indicating the type of brain tumor and a plurality of cross-sectional images obtained from a single patient.

[0006] Furthermore, Patent Document 4 discloses a technique for performing whole tumor segmentation and multi-class tumor segmentation of the brain using machine learning.

[0007] Meanwhile, with recent advances in molecular biology and molecular genetics, it is expected that new pathological conditions will be understood, diagnosed, and even treated using molecular biological and molecular genetic techniques. Molecular genetic analysis has also progressed in the field of brain tumors, and a new molecular genetic pathological classification (WHO pathological classification) was introduced in 2016 (see Non-Patent Document 1).

[0008] Primary brain tumors occur in approximately 20 people per 100,000 people per year, and due to the unique nature of the brain, many of these tumors remain fatal. Gliomas are a typical malignant brain tumor that develops in an infiltrative manner within the brain, accounting for one-quarter to one-third of all primary brain tumors.

[0009] In particular, in recent years, it has been shown that genetic abnormalities in tumors can predict treatment response and life prognosis in gliomas. In addition to the traditional classification based on tissue morphology, the 2016 WHO pathological classification introduced an integrated diagnosis that combines morphological and molecular diagnosis based on the presence or absence of representative molecular biological markers (gene and chromosomal abnormalities).

[0010] Among molecular biological markers, 1p / 19q co-deletion, which is the simultaneous deletion of the short arm of chromosome 1 and the long arm of chromosome 19, and the presence or absence of methylation in the promoter region of the MGMT gene are particularly important as markers for predicting tumor chemotherapy responsiveness (see Non-Patent Document 1).

[0011] However, the treatment of gliomas is still almost uniformly performed by surgical removal of as much of the brain tumor as possible, followed by chemoradiotherapy. That is, first, as much of the brain tumor as safely possible is removed by surgery, and after surgery, the need for radiation and chemotherapy is determined based on an integrated pathological diagnosis, but most medical facilities internationally use a combination of radiation and anticancer drugs.

[0012] There are three reasons why the above-mentioned uniform treatment is being carried out. First, in the case of gliomas, treatment guidelines such as the necessity and amount of radiation therapy and chemotherapy are based on the pathological diagnosis of tumor tissue obtained by surgery. Second, in the case of gliomas, the rate of surgical removal correlates with life prognosis (see Non-Patent Documents 2 and 3). Third, while biopsy and tissue diagnosis are possible with many cancers of other organs using endoscopic techniques and local anesthesia, tissue diagnosis of brain tumors, including gliomas, requires a more invasive procedure called craniotomy.

[0013] If craniotomy is performed when a glioma is suspected, it must satisfy the first and second reasons above, and craniotomy for biopsy is usually not permitted. Therefore, neurosurgeons are forced to remove as much of the brain tumor as possible if they perform a craniotomy. Furthermore, removing a larger area of ​​the brain tumor carries a higher risk of functional impairment. Furthermore, despite the critical importance of maintaining brain function in the brain, neoadjuvant chemotherapy, which is used for cancers of other organs, is not used for brain tumors. Therefore, as a first step in glioma treatment, extensive removal of the brain tumor, which entails risks, is the current situation.

[0014] Preoperative chemotherapy is a treatment strategy in which tissue is collected using an endoscope or other device, and after confirmation by pathological diagnosis, chemotherapy is first administered to reduce the extent of cancer progression (down-staging), and then surgery is performed to remove the cancer.

[0015] On the other hand, in recent years, for example, it has been shown that there is a certain correlation between imaging findings such as CT and MRI, the site of onset, and the tissue type (integrated pathological diagnosis) for gliomas (see Non-Patent Documents 4 to 7). [Prior art documents] [Patent documents]

[0016] [Patent Document 1] Special Announcement No. 2019-211307 [License 2] Special Announcement No. 2019-537180 [License 3] Special Announcement No. 2020-010804 [License 4] Special Announcement No. 2019-523065 [Non-licensed literature]

[0017] [Non-licensed Document 1] Louis DN, Ohgaki H, Wiestler OD, et al. World Health Organization Histological Classification of Tumours of the Central Nervous System. Revised 4th ed., Lyon, France: IARC; 2016. [Non-licensed Document 2] Smith JS, Chang EF, Lamborn KR, et al. Role of extent of resection in the long-term outcome of low-grade hemispheric gliomas. J Clin Oncol. 26(8):1338-45, 2008 [Non-licensed Document 3] Sanai N, Polley MY, McDermott MW, et al. An extent of resection threshold for newly diagnosed glioblastomas. J Neurosurg. 115(1):3-8, 2011 [Non-licensed Document 4] Zhou H, Vallieres M, Bai HX, et al. MRI features predict survival and molecular markers in diffuse lower grade gliomas. Neuro-Oncology 19:862-870, 2017 [Non-patent document 5] Kanazawa T, Fujiwara H, Takahashi H, et al. Imaging scoring systems for preoperative molecular diagnoses of lower-grade gliomas Neurosurg Rev. 42(2):433-441, 2019 [Non-patent document 6] Kanazawa T, Minami Y, Takahashi H, et al. Magnetic resonance imaging texture analyzes in lower-grade gliomas with a commercially available software: correlation of apparent diffusion coefficient and T2 skewness with 1p / 19q codeletion Neurosurg Rev. 2019 Aug 11 [Non-Patent Document 7] Fukuma R, Yanagisawa T, Kinoshita M, et al. Prediction of IDH and TERT promoter mutations in low-grade glioma from magnetic resonance images using a convolutional neural network. Sci Rep. 9(1):20311, 2019 [Non-patent document 8] Lasocki, Arian & Gaillard, Frank & Gorelik, Alexandra & Gonzales, M.. (2018). MRI Features Can Predict 1p / 19q Status in Intracranial Gliomas. American Journal of Neuroradiology. 39. 10.3174 / ajnr.A5572. Summary of the Invention [Problem to be solved by the invention]

[0018] However, the medical image analysis techniques described in Patent Documents 1 to 3 are not intended to predict the presence or absence of molecular biological markers such as gene mutations in brain tumors. Furthermore, when using conventional techniques to predict pathological diagnoses or gene mutations in brain tumors such as gliomas based on image findings, it is difficult to achieve specificity, sensitivity, objectivity, and versatility, and therefore these techniques have not yet been put to clinical use.

[0019] As described above, it has been difficult to predict with high accuracy the presence or absence of molecular biological markers for brain tumors based on head images using conventional techniques.

[0020] The present invention has been made to solve the above-mentioned problems, and aims to predict with higher accuracy the presence or absence of molecular biological markers for brain tumors based on head images. [Means for solving the problem]

[0021] In order to solve the above-mentioned problems, the image diagnosis support device of the present invention comprises an acquisition unit that acquires a head image by MRI, an image processing unit that extracts an area containing information about the lesion site and the edge of the lesion site from the acquired head image and generates a first image having the extracted area, an extraction unit that extracts predetermined image features from the first image, a classification unit that classifies the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image features extracted by the extraction unit, and a presentation unit that presents support information regarding whether the lesion site included in the head image has the molecular biological marker based on the classification result by the classification unit.

[0022] In order to solve the above-mentioned problems, the image diagnosis support device of the present invention comprises an acquisition unit that acquires a head image by MRI; an image processing unit that extracts an area containing information about the lesion site and the margins of the lesion site from the acquired head image and generates a first image having the extracted area; an extraction unit that extracts predetermined image features from the first image and selects, from the extracted image features, image features that satisfy a condition set based on an importance indicating the degree of relevance with a molecular biological marker; a classification unit that classifies the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image features selected by the extraction unit; and a presentation unit that presents support information regarding whether the lesion site included in the head image has the molecular biological marker based on the classification result by the classification unit.

[0023] Furthermore, the image diagnosis support device according to the present invention may further include a selection unit that selects image features that satisfy set conditions from the image features extracted by the extraction unit, the set conditions including conditions that are set based on importance indicating the degree of relevance of the image features extracted by the extraction unit with the molecular biological marker, and the classification unit may classify the presence or absence of the molecular biological marker in the lesion area with respect to the image features selected by the selection unit.

[0024] In the image diagnosis support device according to the present invention, the importance may be calculated using a learning model that has previously learned the degree of relevance of image features to the molecular biological markers.

[0025] In addition, in the image diagnosis support device of the present invention, the importance may be obtained by learning the learning model taking into account the presence or absence of calcification in the lesion area in a head CT image corresponding to the head image.

[0026] In addition, in the image diagnosis support device of the present invention, the importance may be determined by learning the learning model taking into account the age of the subject related to the head image and the localization information of the lesion area in the head.

[0027] In the image diagnosis support device according to the present invention, the learning model may include any one of a random forest, a gradient boosting tree, and a LASSO model.

[0028] In addition, in the image diagnosis support device of the present invention, the image processing unit may use a FLAIR image of the head image taken by MRI to extract an area including the lesion site from the head image, and generate the first image.

[0029] In the image diagnosis support device according to the present invention, the extraction unit may extract the image feature amount by radiomics analysis of the first image.

[0030] In the image diagnosis support device according to the present invention, the classifier may include any one of a support vector machine, a random forest, and a gradient boosting tree.

[0031] In the image diagnosis support device according to the present invention, the lesion site may indicate glioma, and the molecular biological marker of the lesion site may be co-deletion of chromosomes 1p / 19q.

[0032] In addition, in the image diagnosis support device of the present invention, the presentation unit may display on a display screen the support information regarding whether or not the lesion area included in the head image, the first image, and the head image based on the classification result has the molecular biological marker.

[0033] Furthermore, in order to solve the above-mentioned problems, the image diagnosis support method according to the present invention comprises a first step of acquiring a head image by MRI; a second step of extracting an area from the acquired head image that includes information on the lesion site and the borders of the lesion site and generating a first image having the extracted area; a third step of extracting predetermined image features from the first image; a fourth step of classifying the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image features extracted in the third step; and a fifth step of presenting support information regarding whether the lesion site included in the head image has the molecular biological marker based on the classification result in the fourth step.

[0034] Furthermore, the image diagnosis support method according to the present invention may further comprise a sixth step of selecting image features that satisfy set conditions from the image features extracted in the third step, wherein the set conditions include a condition set based on importance indicating the degree of relevance of the image features extracted in the third step with the molecular biological marker, and the importance is determined by a learning model that has previously learned the degree of relevance of the image features with the molecular biological marker, and the fourth step may classify the presence or absence of the molecular biological marker in the lesion area for the image features selected in the sixth step.

[0035] Furthermore, in order to solve the above-mentioned problems, the image diagnosis support program of the present invention causes a computer to execute the following steps: a first step of acquiring a head image by MRI; a second step of extracting an area containing information about the lesion site and the edge of the lesion site from the acquired head image and generating a first image having the extracted area; a third step of extracting predetermined image features from the first image; a fourth step of classifying the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image features extracted in the third step; and a fifth step of presenting support information regarding whether the lesion site included in the head image contains the molecular biological marker based on the classification result in the fourth step.

[0036] Furthermore, the image diagnosis support program according to the present invention may further include a sixth step of selecting image features that satisfy set conditions from the image features extracted in the third step, the set conditions including a condition set based on an importance indicating a degree of relevance of the image features extracted in the third step with the molecular biological marker, the importance being determined by a learning model that has previously learned the degree of relevance of the image features with the molecular biological marker, and the fourth step may classify the presence or absence of the molecular biological marker in the lesion area with respect to the image features selected in the sixth step. [Effects of the Invention]

[0037] According to the present invention, a region containing information about the lesion site and its margins is extracted from a head image, a first image is generated containing the extracted region, and a pre-constructed classifier is used to classify the presence or absence of a molecular biological marker related to the lesion site based on predetermined image features extracted from the first image, thereby enabling more accurate prediction of the presence or absence of a molecular biological marker for brain tumors based on a head image. [Brief explanation of the drawings]

[0038] [Figure 1]FIG. 1 is a block diagram showing an outline of the configuration of an image diagnosis support apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of an image diagnosis support system including an image diagnosis support apparatus according to this embodiment. [Figure 3] FIG. 3 is a block diagram showing the configuration of an image diagnosis support apparatus according to this embodiment. [Figure 4] FIG. 4 is a block diagram showing the configuration of the learning device according to this embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the hardware configuration of the image diagnosis support apparatus according to this embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the hardware configuration of the learning device according to this embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of the operation of the image diagnosis support apparatus according to this embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of the operation of the learning device according to this embodiment. [Figure 9] FIG. 9 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment. [Figure 10] FIG. 10 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment, showing the prediction accuracy of 1p / 19q codeletion based on age and the presence or absence of calcification when a group of cases from specific multiple institutions is targeted. [Figure 11] FIG. 11 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment, showing the prediction accuracy of 1p / 19q codeletion based on age, the presence or absence of calcification, and tumor localization when a group of cases from specific multiple institutions is targeted. [Figure 12] FIG. 12 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment. [Figure 13] FIG. 13 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment. [Figure 14]FIG. 14 is a diagram for explaining the effect of the image diagnosis support device according to this embodiment, and shows the prediction accuracy of 1p / 19q codeletion when machine learning technology is used in addition to age, presence or absence of calcification, and tumor localization when a group of cases from a specific number of institutions is targeted. DETAILED DESCRIPTION OF THE INVENTION

[0039] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail with reference to FIGS.

[0040] [Summary of the Invention] First, an overview of an image diagnosis support device 1 according to an embodiment of the present invention will be described.

[0041] The image diagnosis support device 1 according to this embodiment is a device that supports image diagnosis by predicting the molecular biological characteristics of brain tumors from preoperative head images using machine learning technology. Based on the molecular biological characteristics of brain tumors predicted by the image diagnosis support device 1, image diagnosis support is possible, such as surgical strategies, including the extent of tumor removal before craniotomy, and more appropriate design of overall treatment strategies. In other words, while brain tumor treatment has traditionally been based on the pathological diagnosis of the removed tumor, for glioma grades 2 and 3, the image diagnosis support device 1 makes it possible to design treatment strategies based on image diagnosis.

[0042] As mentioned above, three molecular biological markers are known to be important factors related to the treatment sensitivity and prognosis of glioma: 1p / 19q codeletion, the presence or absence of promoter methylation of the DNA repair enzyme MGMT gene, and the presence or absence of IDH1 / 2 gene mutations. Gliomas with 1p / 19q codeletion are particularly sensitive to chemotherapy, and the presence or absence of this chromosomal abnormality is extremely important when determining treatment strategies.

[0043] The image diagnosis support device 1 according to this embodiment focuses on the presence or absence of 1p / 19q codeletion among these three important molecular biological markers. That is, the image diagnosis support device 1 according to this embodiment predicts whether or not the lesion site has 1p / 19q codeletion based on an image including a site suspected of glioma.

[0044] By being able to predict with higher accuracy and specificity the presence or absence of 1p / 19q codeletion in gliomas based solely on imaging findings, rather than relying on the conventional pathological diagnosis, it will be possible to predict favorable chemotherapy response and support more appropriate glioma treatment, including the indication for preoperative chemotherapy.

[0045] To enable prediction of the presence or absence of molecular biological markers based on such imaging findings, the image diagnosis support device 1 according to this embodiment employs an algorithm based on machine learning. More specifically, the image diagnosis support device 1 predicts the presence or absence of 1p / 19q codeletion for unknown image data of a head including a lesion site related to glioma using a trained machine learning model constructed using training data. The training data used is a dataset consisting of head images including a lesion site related to glioma and training labels assigned to the images indicating the presence or absence of 1p / 19q codeletion.

[0046] 1, for example, the image diagnosis support device 1 according to this embodiment performs radiomics analysis of MRI images of the head and extracts high-dimensional image features related to the texture, such as the homogeneity and texture of the lesion, as well as the size and shape. Such image features are also called radiomics features, and are information that is recognized to be related to clinical information of the lesion, such as glioma.

[0047] Furthermore, the region of interest set in the image for extracting image features is an area that broadly includes information about the tumor margin. This is based on the novel finding that information about the tumor margin, i.e., the area including the boundary between the lesion and normal areas, is important information for predicting the presence or absence of molecular biological markers. Image features are extracted from a mask image containing this region of interest.

[0048] The extracted image features are then classified by a classifier that has learned to determine whether or not a sample has 1p / 19q co-deletion based on the image features of the training data. As the classifier, a machine learning model applicable to classification problems, such as a support vector machine (SVM), random forest, or gradient boosting, can be used. In the embodiment of the present invention, an SVM is used as an example.

[0049] Among the image features extracted by radiomics analysis, specific types of image features that satisfy pre-defined conditions are input to the SVM. More specifically, specific types and numbers of image features that satisfy the pre-defined conditions are selected based on the importance of the image features that indicate the high correlation between the image features and 1p / 19q codeletion, and input to the trained SVM. The importance is acquired, for example, through a pre-training process performed on an external server.

[0050] The learning model that learns the importance is trained based on image features extracted from the mask images of the training data, as well as information known to be more relevant in predicting the presence or absence of 1p / 19q co-deletion. For example, the model is trained using information on the head images of the target patient, including age and tumor location, as input data.

[0051] Furthermore, based on the classification results of the classifier, information to assist in image diagnosis is presented regarding whether or not the glioma lesion site contained in the head image has 1p / 19q codeletion.

[0052] For example, by predicting whether a glioma has 1p / 19q codeletion from brain images taken before craniotomy, it becomes possible to develop specific treatment strategies, as shown in 1) to 3) below. 1) In the initial surgery, areas near functionally important brain regions, such as the motor and language regions, are not removed to avoid risk, and a deliberate partial resection is performed. This is followed by chemotherapy to shrink the tumor, and then the tumor near the functional region is removed. 2) No surgery is performed, i.e., chemotherapy is performed first without a tissue diagnosis, and the tumor is then removed after shrinking. 3) Intentional placement of an intracerebral sustained-release anticancer drug is performed. The realization of such glioma treatment strategies is expected to reduce the risks of surgery and adverse drug events, reduce the area of ​​brain resection, increase the resection rate, and improve functional and life prognosis.

[0053] [Embodiment Mode] 2 is a block diagram showing an outline of the configuration of an image diagnosis support system including an image diagnosis support device 1 according to an embodiment of the present invention. The image diagnosis support system predicts whether or not a brain tumor contained in an MRI head image has a molecular biological marker, and presents the prediction result as support information for image diagnosis.

[0054] 2, the image diagnosis support system includes, for example, a plurality of image diagnosis support devices 1 and a learning device 2. The image diagnosis support devices 1 and the learning devices 2 are connected via a network NW such as the Internet.

[0055] Based on training data of MRI head images, the learning device 2 learns whether the lesion site of a glioma contained in the image has 1p / 19q codeletion or not, and constructs a first model M1 and a second model M2 (described below). The first model M1 and the second model M2 are sometimes collectively referred to as the learning models.

[0056] The image diagnosis support device 1 reads the learning model constructed by the learning device 2 and predicts the presence or absence of 1p / 19q codeletion in a lesion area included in an MRI head image based on the image. The image diagnosis support device 1 can acquire the learning model from the learning device 2 via a network NW, for example. Alternatively, the image diagnosis support device 1 can acquire the learning model using a portable semiconductor memory such as a USB memory.

[0057] For example, the image diagnosis support device 1 is installed in a medical institution such as a hospital, and predicts whether or not the lesion site related to glioma has 1p / 19q co-deletion from an MRI image of a patient's head, and presents the prediction result as support information for image diagnosis.

[0058] The image diagnosis support system can also be configured such that the learning device 2 performs re-learning based on the prediction results of the MRI image of the head related to a new case in the image diagnosis support device 1, and updates the learning model.

[0059] [Function block of diagnostic imaging system] Next, the functional configuration of the image diagnosis support device 1 will be described with reference to the block diagram of FIG.

[0060] The image diagnosis support device 1 includes a first acquisition unit 10 (acquisition unit), an image processing unit 11, an extraction unit 12, a selection unit 13, a classification unit 14, a presentation unit 15, and a memory 16.

[0061] The first acquisition unit 10 acquires MRI images of the head. More specifically, the first acquisition unit 10 acquires three types of image sets: T2-weighted images, T1CE images, and FLAIR images of the patient's head captured by MRI. For example, the first acquisition unit 10 can acquire multiple T2-weighted images, T1CE images, and FLAIR images including a lesion site suspected of being a glioma. These multiple images are, for example, MRI images obtained by scanning the entire brain from the base to the parietal region. The first acquisition unit 10 may also acquire CT images of the head of the patient from which the MRI images were acquired.

[0062] In T2-weighted images, not only fatty tissue but also water, liquid components, and cysts appear white, and tumors generally appear slightly white. In T1CE images, fatty tissue appears mainly white, and water, liquid components, and cysts appear black. In T1CE images, tumors generally appear black, and areas with abundant blood flow appear white due to the contrast enhancement effect. In addition, FLAIR (fluid-attenuated inversion-recovery) images are T2-weighted images that suppress the signal from water, and cerebrospinal fluid appears black, or low signal.

[0063] The image processing unit 11 extracts, from the head image acquired by the first acquisition unit 10, a region including information on the lesion site related to glioma and the margin of the lesion site as a region of interest, and generates a mask image (first image) including the extracted region. The "margin of the lesion site" refers to a portion including the boundary between the lesion site and a normal region. More specifically, the image processing unit 11 can use the FLAIR image acquired by the first acquisition unit 10 to extract an abnormal region including the lesion site on the FLAIR image and generate a mask image. As another example, the image processing unit 11 can set, as a region of interest, the lesion site related to glioma, its margin, and a region extending further outward from the margin by approximately 5 mm, and generate a mask image.

[0064] The image processing unit 11 may perform known preprocessing, image size adjustment, etc. on the MRI image acquired by the first acquisition unit 10. Furthermore, the mask image generated by the image processing unit 11 can be appropriately adjusted or corrected in response to an external input operation.

[0065] The extraction unit 12 extracts image features from the mask image of the region of interest generated by the image processing unit 11. Specifically, the extraction unit 12 performs radiomics analysis on the mask image to extract predetermined types of image features. Radiomics analysis is a technique for comprehensively analyzing a large number of image features obtained from a large-scale medical image database. Image features quantified from images by radiomics analysis, such as the size, shape, density, signal intensity, and texture (e.g., heterogeneity) of lesions contained in the image, are information that exhibit some correlation with clinical evaluation of the lesion. The extraction unit 12 can extract a number of image features according to the settings using, but is not limited to, the open-source radiomics analysis software PyRadiomics. For example, image features may be extracted using a deep learning model, such as a convolutional neural network (VGG19).

[0066] The selection unit 13 selects image features that satisfy set conditions from the image features extracted by the extraction unit 12 and passes them to the classification unit 14. The set conditions are conditions that are set in advance based on the importance of the image features, which indicate the degree of relevance of each of the image features extracted by the extraction unit 12 to 1p / 19q co-deletion. The importance is a value that can be calculated and ranked for each image feature. The importance is also a value acquired by learning the first model M1 performed by the external learning device 2. The importance learning process will be described later. The set conditions are stored in the selection condition 16C of the memory 16.

[0067] For example, the selection unit 13 selects a specific type and number of image features from among the information that is correlated with 1p / 19q codeletion in addition to the 93 image features extracted by the extraction unit 12, and inputs these to the trained second model M2. Specifically, the selection unit 13 may select the patient's age and tumor location information related to the head image to be analyzed as information that is correlated with 1p / 19q codeletion.

[0068] Tumor location information is a value indicating a predetermined anatomical location of the tumor, such as the frontal lobe, occipital lobe, temporal lobe, or deep region. Tumor location is known to be related to the tumor's genetic background. For example, brain tissue can be classified into the following 10 regions: "frontal: 1," "temporal: 2," "parietal: 3," "occipital: 4," "insular: 5," "diencephalon: 6," "corpus callosum: 7," "brainstem: 8," "cerebellum: 9," and "multicentric: 10," and a value from 1 to 10 assigned to each region can be used. If the tumor spans multiple regions, the value of the primary lesion can be used. Furthermore, the location classification can be determined by the examiner, a physician who actually uses the image diagnosis support device 1.

[0069] The selection unit 13 can further select, according to the set conditions, information indicating whether or not the lesion site of a glioma in a head CT image is accompanied by calcification as information that is recognized to be correlated with 1p / 19q codeletion. The presence or absence of tumor calcification in a CT image is information known to be correlated with 1p / 19q codeletion. In addition, information on the margin of the lesion site of a tumor is considered to be correlated with the presence or absence of molecular biological markers such as 1p / 19q codeletion.

[0070] The patient's age, tumor location information, and information on the presence or absence of calcification in the CT image are linked to identification information, such as the ID of the patient associated with the head image to be analyzed, and are stored in advance in additional data 16B of memory 16. Note that the feature amounts stored in additional data 16B may also be referred to as image feature amounts.

[0071] For example, the selection unit 13 can select, according to a set condition, 15 or more specific types of image features having high importance from the extracted 93 image features. Alternatively, the selection unit 13 can select, according to a set condition, 15 or more image features from the patient's age, tumor location information, and information on the presence or absence of calcification in the CT image in addition to the 93 image features.

[0072] The classification unit 14 classifies whether or not the glioma lesion site has 1p / 19q co-deletion based on the image features selected by the selection unit 13. The classification unit 14 performs classification using a second model M2, which is a classifier that has been trained and constructed in advance to determine whether or not the image features have 1p / 19q co-deletion. The second model M2 can be a linear SVM or a random forest. More specifically, the classification unit 14 inputs, for example, 15 or 17 specific types of image features selected based on importance, and calculates a trained linear SVM or random forest to classify whether or not the image features have 1p / 19q co-deletion. In addition to linear SVM and random forest, gradient boosting or the like may also be used as the second model M2.

[0073] The presentation unit 15 presents support information regarding whether or not a lesion site related to glioma contained in a head image has 1p / 19q co-deletion based on the classification results by the classification unit 14. For example, the presentation unit 15 performs threshold processing on the classification results obtained as predicted probability values ​​using a threshold set in consideration of prevalence, etc., to determine whether or not the lesion site has 1p / 19q co-deletion, and can present this as support information for image diagnosis. Alternatively, the presentation unit 15 can present support information as an either / or choice regarding whether or not the lesion site has 1p / 19q co-deletion based on the value of the classification result.

[0074] The presentation unit 15 can display, for example, support information indicating whether or not the lesion site has 1p / 19q co-deletion together with text data, a head image, etc. on the display device 108 described below. Note that the presentation unit 15 can present support information in other forms, such as audio, in addition to visual information, as long as the presentation method is capable of supporting image diagnosis.

[0075] The memory 16 has areas for storing an inference program 16A, additional data 16B, selection conditions 16C, and a second model M2.

[0076] The inference program 16A is a program for each functional unit of the image diagnosis support device 1, including the extraction unit 12, the selection unit 13, and the classification unit 14, to predict whether or not a lesion site related to glioma contained in an image of a patient's head has 1p / 19q co-deletion, using the importance obtained from the pre-constructed second model M2 and the pre-constructed first model M1.

[0077] The additional data 16B stores the patient's age, tumor location information, and the presence or absence of calcification in the CT image of the lesion site, which may be selected when the selection unit 13 selects inputs to the trained second model M2 according to the conditions set. The selection conditions 16C stores set conditions used by the selection unit 13. The second model M2 is a trained classifier used by the classification unit 14.

[0078] [Learning device functional blocks] Next, the configuration of the learning device 2 included in the image-aided diagnosis system will be described with reference to the block diagram of FIG.

[0079] 3, and performs a learning process for a first model M1 that learns the importance for determining the selection conditions used by the selection unit 13, and a learning process for a second model M2 that is a classifier used by the classification unit 14. The learning device 2 includes, for example, a second acquisition unit 20, an image processing unit 21, a setting unit 22, a learning unit 23, and a memory 24.

[0080] The second acquisition unit 20 acquires training data from the training data DB 205, which includes MRI images of the head and training labels indicating the presence or absence of 1p / 19q codeletion. Specifically, the training data is a set of three types of images, namely, T2-weighted images, T1CE images, and FlAIR images of the head, labeled with the presence or absence of 1p / 19q codeletion. For example, the training images used as training data can be 159 cases of glioma available from the Cancer Imaging Archive (TCIA) or, for example, 100 cases of other available primary glioma cases. The second acquisition unit 20 can also acquire CT images of the head.

[0081] The image processing unit 21 extracts a region containing information about the glioma lesion and its margin as a region of interest from the training image included in the training data acquired by the second acquisition unit 20, and generates a mask image having the extracted region. More specifically, the image processing unit 21 can use the FLAIR image acquired by the second acquisition unit 20 to extract an abnormal region containing the lesion on the FLAIR image and generate a mask image. As another example, the image processing unit 21 can set the glioma lesion, its margin, and a region extending outward from the margin up to about 5 mm as a region of interest and generate a mask image. Note that the mask image generated by the image processing unit 21 can be adjusted or corrected as needed in response to external input.

[0082] The setting unit 22 sets a learning model, such as setting hyperparameters in the learning process executed by the learning unit 23, in response to an input received from outside. For example, the setting unit 22 sets the configuration of a first model M1 that learns importance for determining selection conditions used by the selection unit 13. The setting unit 22 sets, for example, a gradient boosting tree as the learning model of the first model M1. Note that the first model M1 set by the setting unit 22 may be other learning models besides gradient boosting, such as a random forest or a LASSO model.

[0083] Furthermore, when gradient boosting is adopted as the first model M1, the setting unit 22 sets the structure of the gradient boosting decision tree and the input nodes in the gradient boosting tree. For example, the setting unit 22 sets the number of image features to be input to gradient boosting, and specifies additional input data to be used together with the image features, such as age, tumor localization information, and the presence or absence of calcification in CT images.

[0084] Furthermore, when a linear SVM is employed as a classifier for classifying the presence or absence of 1p / 19q co-deletion used in the classification unit 14, the setting unit 22 specifies a linear kernel as the kernel function of the linear SVM. Furthermore, the setting unit 22 sets, for example, the number of divisions for cross-validation of the linear SVM. Regarding the number of input data for the linear SVM, the setting unit 22 can set a condition for selecting, for example, the top 17 specific image features in order of importance based on the importance obtained by learning the first model M1. The set condition is stored in the selection condition 24C. The setting unit 22 may use classifiers other than SVM, such as random forests and gradient boosting.

[0085] The setting unit 22 also specifies and sets the number of image features to be extracted by radiomics analysis of the head image captured by MRI. For example, the setting unit 22 can set the number of image features to be extracted to 93.

[0086] The learning unit 23 uses learning images in accordance with a learning program 24A stored in the memory 24 to learn a first model M1 that learns the importance for determining the selection conditions used by the selection unit 13 when selecting image features, and a second model M2 for classifying the presence or absence of 1p / 19q co-deletion used by the classification unit 14, and constructs the trained first model M1 and second model M2. The following describes an example in which gradient boosting is used as the first model M1 and SVM is used as the second model M2.

[0087] More specifically, the learning unit 23 performs radiomics analysis on mask images of the training images included in the training data, extracting, for example, 93 image features. The learning unit 23 also performs learning by gradient boosting based on the image features of the extracted training images, age, tumor localization information, and, depending on the settings, the presence or absence of calcification in the CT image of the tumor, to learn the importance indicating the degree of association with 1p / 19q codeletion.

[0088] Here, the learning of the first model M1 by the learning unit 23, i.e., learning of importance, will be described in more detail. The learning unit 23 inputs image features extracted from training images by radiomics analysis to gradient boosting set as the first model M1. In addition to the extracted image features, the learning unit 23 can also provide information that is recognized to be correlated with 1p / 19q codeletion as input to gradient boosting to learn importance.

[0089] For example, information correlated with 1p / 19q co-deletion can be obtained from the patient's age and tumor location information related to the head training images. The tumor location information is a value indicating the predetermined anatomical location of the tumor and is known to be related to the genetic background of the tumor.

[0090] Furthermore, information indicating whether or not the lesion site of glioma in a head CT image is accompanied by calcification can be further used as information that is recognized to be correlated with 1p / 19q codeletion. Head CT images containing information on the margins of tumor lesion sites are thought to be related to the presence or absence of molecular biological markers such as 1p / 19q codeletion. Based on such knowledge, the first model M1 may be trained taking into account the presence or absence of calcification in the lesion site of glioma in a CT image.

[0091] In this way, the learning unit 23 trains the first model M1 to learn the importance of the association between image features and 1p / 19q codeletion using gradient boosting or the like. The trained first model M1 outputs the importance expressed as a value obtained by ranking the 93 image features extracted from the training images by radiomics analysis, as well as age, tumor localization information, and the presence or absence of calcification in the CT image in order of importance. The setting unit 22 sets selection conditions 24C for the trained importance.

[0092] The learning unit 23 reads out from the selection conditions 24C of the memory 24 the conditions set by the setting unit 22 based on the importance obtained by learning the first model M1. For example, the selection conditions 24C specify that the top 15 or 17 specific types of image features are selected in order of importance. The selected 15 or 17 specific types of image features, including, for example, age, tumor localization information, and the presence or absence of calcification in the CT image, are used as input to the second model M2. The learning unit 23 selects image features that satisfy the set conditions from the image features extracted by the radiomics analysis and the age, tumor localization information, and the presence or absence of calcification in the CT image included in the additional data 24B.

[0093] The learning unit 23 trains the second model M2 by providing image features that satisfy set conditions as input to the second model M2 based on the importance obtained in training the first model M1. Specifically, the learning unit 23 trains a linear SVM that classifies image features as having or not having 1p / 19q co-deletion using teacher labels that indicate the presence or absence of 1p / 19q co-deletion and are assigned to training images related to the image features, thereby constructing a trained linear SVM.

[0094] The memory 24 stores a learning program 24A, additional data 24B, selection conditions 24C, a first model M1, and a second model M2.

[0095] The learning program 24A is a program for predicting whether a lesion site related to glioma has 1p / 19q co-deletion using training data consisting of training images related to head images taken by MRI and training labels assigned to the images. The learning unit 23 learns a first model M1 and a second model M2 in accordance with the learning program 24A to construct a learning model, and stores the learned first model M1 and second model M2 in the memory 24.

[0096] The additional data 24B includes information such as age, tumor location information, and the presence or absence of calcification in the CT image of the tumor, which are provided as input to a first model M1 such as gradient boosting, which calculates the importance of image features extracted from the mask image and selects the image features.

[0097] The selection condition 24C stores a condition that is set based on the importance of the image feature amount acquired by learning the first model M1.

[0098] [Hardware configuration of image diagnosis support system] Next, an example of a hardware configuration for realizing the image diagnosis support device 1 according to this embodiment will be described with reference to the block diagram of FIG.

[0099] 5, the image diagnosis support device 1 can be realized by, for example, a computer including a processor 102, a main memory device 103, a communication interface (I / F) 104, an auxiliary memory device 106, and an input / output (I / O) 107, which are connected via a bus 101, and a program that controls these hardware resources. The image diagnosis support device 1 is also connected to an external MRI device 105 via the bus 101. The image diagnosis support device 1 can also include a display device 108 connected via the bus 101.

[0100] The main memory device 103 pre-stores programs for the processor 102 to perform various controls and calculations. The processor 102 and the main memory device 103 implement the functions of the image diagnosis support device 1, such as the first acquisition unit 10, image processing unit 11, extraction unit 12, selection unit 13, and classification unit 14 shown in FIG.

[0101] The communication I / F 104 is an interface circuit for connecting the image diagnosis support device 1 and various external electronic devices via a network. The communication I / F 104 realizes the first acquisition unit 10 and the presentation unit 15 shown in Fig. 2. Note that the communication I / F 104 may be configured to receive the additional data 16B and selection conditions 16C used by the selection unit 13 described in Fig. 2, and the second model M2 used by the classification unit 14 from an external terminal such as the learning device 2 via the network NW.

[0102] The MRI device 105 can capture T2 weighted images, T1CE images, and FLARI images according to the settings of the magnetic field and electromagnetic wave parameters. The T2 weighted images, T1CE images, and FLARI images of the patient's head captured by the MRI device 105 are used as input images to the image diagnosis support device 1. These head images are linked to, for example, the patient's ID information.

[0103] The auxiliary storage device 106 is composed of a readable / writable storage medium and a drive for reading and writing various information such as programs and data from and to the storage medium. The auxiliary storage device 106 can use a semiconductor memory such as a hard disk or flash memory as the storage medium.

[0104] The auxiliary storage device 106 realizes the memory 16 described in Fig. 2. The auxiliary storage device 106 has a program storage area for storing an inference program 16A (image diagnosis support program) executed by the image diagnosis support device 1. The auxiliary storage device 106 also has an area for storing a library for performing radiomics analysis of head images captured by the MRI device 105.

[0105] The auxiliary storage device 106 also has an area for storing selection conditions 16C used by the selection unit 13 when selecting image features according to conditions set based on the importance of the image features, and a trained second model M2 used by the classification unit 14 to classify the presence or absence of 1p / 19q co-deletion in image features. The auxiliary storage device 106 also has an area for storing additional data 16B used by the selection unit 13. The auxiliary storage device 106 also has an area for storing an image processing program 106A used by the image processing unit 11 to extract a region of interest from an image of the head and generate a mask image.

[0106] Furthermore, the auxiliary storage device 106 may have, for example, a backup area for backing up the above-mentioned data, programs, etc.

[0107] The input / output I / O 107 is configured by an I / O terminal for inputting signals from an external device and outputting signals to an external device.

[0108] The display device 108 is configured by an organic EL display, a liquid crystal display, etc. The display device 108 can also realize the presentation unit 15 described in FIG.

[0109] Here, the program stored in the program storage area of ​​the auxiliary storage device 106 may be a program that is processed in time series according to the order of the image diagnosis support method described in this specification, or may be a program that is processed in parallel, or at a required timing such as when called. Furthermore, the program may be processed by one computer, or may be processed in a distributed manner by multiple computers.

[0110] [Hardware configuration of the learning device] Next, an example of a hardware configuration for realizing the learning device 2 according to this embodiment will be described with reference to the block diagram of FIG.

[0111] 6, learning device 2 can be realized by, for example, a computer including processor 202, main memory device 203, communication interface (I / F) 204, auxiliary memory device 206, and input / output (I / O) 207, all connected via bus 201, and a program that controls these hardware resources. Learning device 2 is also connected to, for example, an external teacher data DB 205 via bus 201. Learning device 2 can also be equipped with display device 208 connected via bus 201.

[0112] The processor 202 and the main storage device 203 implement the functions of the learning device 2, such as the second acquisition unit 20, the image processing unit 21, the setting unit 22, and the learning unit 23 shown in FIG.

[0113] The auxiliary storage device 206 realizes the memory 24 described in Fig. 4. The auxiliary storage device 106 has a program storage area for storing the learning program 24A executed by the learning device 2. The auxiliary storage device 206 also has an area for storing a library for performing radiomics analysis of head images.

[0114] The auxiliary storage device 206 also has an area for storing a first model M1 constructed by the learning unit 23 through learning processing, and a second model M2, which is a classifier. The auxiliary storage device 106 also has an area for storing additional data 24B used in learning the first model M1. The auxiliary storage device 106 also has an area for storing selection conditions 24C set based on the importance acquired through learning the first model M1. The auxiliary storage device 106 also has an area for storing an image processing program 206A used by the image processing unit 21 to extract a region of interest from an image of the head and generate a mask image.

[0115] The input / output I / O 207 is configured by an I / O terminal for inputting signals from an external device and outputting signals to an external device, and an input device such as a keyboard, etc. The input / output I / O 207 realizes the setting unit 22 described in FIG.

[0116] [Operation of image diagnosis support device] Next, the operation of the image diagnosis support device 1 having the above-mentioned configuration will be described in detail using the flowchart in Fig. 7. The memory 16 stores selection conditions 16C and a second model M2 that are set based on a first model M1 constructed in advance through a learning process by an external learning device 2 or the like, and the following process is executed. In addition, the memory 16 stores in advance additional data 16B, including the age of the patient associated with the head image that is the target of image diagnosis support, the location information of the brain tumor to be analyzed, and the presence or absence of calcification in the CT image of the patient's head, in association with the patient's ID.

[0117] First, the first acquisition unit 10 acquires MRI images of the head (step S1). Specifically, the first acquisition unit 10 acquires three types of image sets: a T2 weighted image, a T1CE image, and a FLAIR image. The images acquired by the first acquisition unit 10 are associated with the patient's ID.

[0118] Next, image processing unit 11 extracts a region containing information about the lesion site and the margin of the lesion site related to the glioma as a region of interest from the head image acquired in step S1, and generates a mask image having the extracted region (step S2). For example, image processing unit 11 can use the FLAIR image acquired in step S1 to extract an abnormal region containing the lesion site on the FLAIR image to generate a mask image. Note that the mask image generated by image processing unit 11 can be adjusted or corrected in response to external input.

[0119] Next, the extraction unit 12 performs radiomics analysis on the mask image generated in step S2 and extracts image features from the mask image (step S3). For example, the extraction unit 12 extracts 93 image features from the mask image according to a pre-set value.

[0120] Next, the selection unit 13 reads the selection conditions 16C stored in the memory 16 and selects image features that satisfy the set conditions from the image features extracted in step S3, the age associated with the patient ID included in the additional data 16B stored in the memory 16, the tumor localization information, and the presence or absence of calcification in the CT scan (step S4). For example, the selection unit 13 can select 17 specific types of image features from the 93 image features extracted by the extraction unit 12 and input them to the second model M2. The set conditions are set in advance based on the learned importance.

[0121] Next, the classification unit 14 reads the trained second model M2 from the memory 16, provides the image features selected in step S4 as input, performs calculations on the trained second model M2, and classifies the input image features as having 1p / 19q co-deletion or not (step S5).

[0122] Then, based on the classification results obtained in step S5, the presentation unit 15 displays support information regarding the presence or absence of 1p / 19q co-deletion predicted for the lesion site related to glioma contained in the head image corresponding to the image features on the display device 108 (step S6).

[0123] [Learning device operation] Next, the operation of the image diagnosis support device 1 having the above-described configuration will be described in detail using the flowchart in Figure 8. It is assumed that the training data DB 205 stores training data to which training labels indicating the presence or absence of 1p / 19q codeletion are pre-assigned to MRI training images of the head. Furthermore, the memory 24 stores a first model M1 and a second model M2 before training. Furthermore, it is assumed that the memory 24 pre-stores additional data 24B associated with the training images, including the age of the patient corresponding to the training image, the location information of the brain tumor to be analyzed, and the presence or absence of calcification in the CT image of the patient's head.

[0124] First, the setting unit 22 sets various parameters for the learning process in accordance with an input received from the outside (step S20). For example, in setting the input node of the first model M1 that learns the importance of image features, the setting unit 22 sets the input node using some or all of the age information, tumor localization information, and the presence or absence of calcification in the CT image, which are included in the additional data 24B, in addition to the image features.

[0125] Next, the second acquisition unit 20 acquires training data from the training data DB 205 (step S21). The training data is a data set in which training labels indicating whether or not the glioma lesion site included in each of three types of image sets, namely, T2-weighted images, T1CE images, and FLAIR images of the head taken by MRI, are assigned. Identification information is also assigned to the training data.

[0126] Next, image processing unit 21 extracts, as regions of interest, regions of the glioma lesion and the margins of the lesion from the training images included in the training data in accordance with the settings made in step S20, and generates a mask image having the extracted regions (step S22). More specifically, image processing unit 21 can use a FLAIR image from the images acquired in step S21 to extract an abnormal region including the lesion on the FLAIR image to generate a mask image. Alternatively, image processing unit 21 can set, as regions of interest, the glioma lesion, its margins, and a region extending outward from the margins by approximately 5 mm, and generate a mask image.

[0127] Next, the learning unit 23 performs radiomics analysis on the mask image generated in step S22 and extracts image features from the mask image of the training data (step S23). For example, the learning unit 23 extracts 93 image features from the mask image in accordance with the prior settings made by the setting unit 22.

[0128] Next, using a first model M1 stored in memory 24, the learning unit 23 provides the image features extracted in step S23 and the age, tumor localization information, and CT calcification presence / absence values ​​associated with the learning images, which are included in the additional data 24B stored in memory 24, as inputs to the first model M1 (step S24). Next, the learning unit 23 causes the first model M1 to learn the importance of the image features (step S25). More specifically, the learning unit 23 causes the first model M1 to learn the importance of the association between the image features and 1p / 19q codeletion based on the extracted image features, age, tumor localization information, and the presence / absence of CT calcification, for example, by gradient boosting, random forest, LASSO regression, or the like.

[0129] Next, the learning unit 23 selects image features that satisfy conditions set based on the importance of the image features output from the first model M1 learned in step S25, provides the selected image features as input to the second model M2, and learns the second model M2 (step S26). For example, the learning unit 23 can select the top 15 or 17 image features in order of importance from the 93 image features extracted by the radiomics analysis in step S23. The conditions set based on the importance and used by the learning unit 23 are stored in the selection conditions 24C of the memory 24.

[0130] In addition, in step S26, when the learning unit 23 performs learning using linear SVM, random forest, gradient boosting, or the like set as the second model M2, it uses, for example, 15 or 17 image features selected from 93 image features as input and learns whether or not 1p / 19q co-deletion is present in the image features.

[0131] Next, the learning unit 23 verifies and tests the first model M1 and the second model M2 learned in steps S25 and S26 (step S27). For example, the setting unit 22 adjusts overfitting of the first model M1 and the second model M2 according to the results of the cross-validation, and adjusts the model settings, such as the parameters and input data of the first model M1 and the second model M2. For example, the learning unit 23 can divide the learning image data into 10 parts and perform cross-validation of the model. Furthermore, in step S27, the learning unit 23 tests a prediction model including the verified first model M1 and the second model M2, and can estimate the accuracy in actual operation.

[0132] Thereafter, the first model M1, the second model M2, and the selection condition 24C constructed by learning are stored in the memory 24 (step S28). The selection condition 24C and the learned second model M2 stored in the memory 24 are acquired by the image diagnosis support device 1 described above.

[0133] Here, the effects of the image diagnosis support device 1 according to this embodiment will be described with reference to Fig. 9 to Fig. 14. Fig. 9 is a vertical bar graph showing the importance of each image feature calculated by the trained first model M1.

[0134] The horizontal axis of FIG. 9 indicates each image feature, and the vertical axis indicates importance. An image feature with a larger value on the vertical axis is an image feature with a higher importance indicating a high degree of association with the presence or absence of 1p / 19q codeletion. Also, for example, image feature a shown in FIG. 9 indicates tumor calcification information, image feature b indicates age information ("age"), and image feature c indicates, among tumor localization information, whether a tumor is present in the frontal lobe, and it is shown that these image features have a higher importance. In this way, it can be seen that the first model M1 provided in the image diagnosis support device 1 according to this embodiment has learned the importance of the association between image features and 1p / 19q codeletion.

[0135] Here, for comparison with the prediction accuracy of the image diagnosis support device 1 according to the present embodiment, the prediction accuracy of the presence or absence of 1p / 19q codeletion according to a conventional example is shown in FIGS. 10 and 11. FIG. 10 shows the prediction accuracy of the presence or absence of 1p / 19q codeletion in a case group from a specific multi-institutional setting, taking into account age information and the presence or absence of CT calcification, using a prediction model based on numerical data described in Non-Patent Document 8. As shown in FIG. 10, the ROC (Receiver Operating Characteristic) curve is a curve with the false positive rate on the horizontal axis and the true positive rate on the vertical axis. The AUC (Area Under the Curve), which is the area under the ROC curve shown in FIG. 10, is 0.64.

[0136] On the other hand, Figure 11 shows the prediction accuracy of the presence or absence of 1p / 19q codeletion when tumor localization information is taken into account in addition to age information and the presence or absence of CT calcification in a case group from a specific multi-institutional setting, using a prediction model based on numerical data described in Non-Patent Document 8. The area under the ROC curve (AUC) value shown in Figure 11 is 0.70, which indicates an improvement in accuracy compared to the prediction accuracy obtained under the conditions shown in Figure 10.

[0137] In contrast, Figures 12 to 14 show the prediction accuracy of the presence or absence of 1p / 19q co-deletion obtained by the image diagnosis support device 1 according to this embodiment using machine learning technology. Figure 12 shows an example of the prediction accuracy of the presence or absence of 1p / 19q co-deletion obtained by the image diagnosis support device 1 equipped with the second model M2 and the trained first model M1, which were trained using labeled training data of 31 images with 1p / 19q co-deletion and 31 images without 1p / 19q co-deletion. The area under the ROC curve (AUC) shown in Figure 12 is 0.84. Note that the example in Figure 12 shows the ROC curve when 16 image features are used as input to the classifier.

[0138] Figure 13 shows an example of the prediction accuracy of the presence or absence of 1p / 19q co-deletion obtained by the image diagnosis support device 1 equipped with the trained second model M2 and the trained first model M1 that takes into account the presence or absence of CT calcification. In the example of Figure 13, learning was performed using labeled training data of 25 images with 1p / 19q co-deletion and 23 images without 1p / 19q co-deletion, and the AUC value, which indicates prediction accuracy, is 0.85. In this example, image features were selected using gradient boosting.

[0139] Figure 14 shows the prediction accuracy of the presence or absence of 1p / 19q codeletion obtained by the image diagnosis support device 1 equipped with the trained second model M2 and the trained first model M1 that takes into account age information, the presence or absence of CT calcification, and tumor localization information, using the same subjects (case groups from specific institutions) as in Figures 10 and 11. The area under the ROC curve (AUC) shown in Figure 14 is 0.81. It can be seen that the prediction accuracy is significantly improved compared to the prediction accuracy of the conventional example in Figure 11.

[0140] As such, while the prediction accuracy of the prediction model based on numerical data in the conventional example was 64% (Figure 10) and 70% (Figure 11), respectively, in all of the analyses in Figures 12 to 14, the prediction accuracy of the presence or absence of 1p / 19q co-deletion by the image diagnosis support device 1 of this embodiment, which is equipped with the trained first model M1 and second model M2, was shown to be 81% or more, a significant improvement in prediction accuracy.

[0141] As described above, the image diagnosis support device 1 according to this embodiment generates a mask image from an MRI head image, using a region of interest containing information about the margins of a brain tumor as a region of interest. Radiomics analysis of the mask image is then performed to extract image features. Furthermore, using a second model M2 that has learned whether the image features contain 1p / 19q co-deletion, the presence or absence of 1p / 19q co-deletion in the extracted image features is classified. Therefore, the presence or absence of molecular biological markers for brain tumors can be predicted based on the head image.

[0142] Furthermore, the image diagnosis support device 1 according to this embodiment classifies image features extracted by radiomics analysis using a classifier such as SVM, random forest, or gradient boosting to predict whether or not a lesion site has 1p / 19q co-deletion. Furthermore, the image features input to the classifier, such as SVM, are selected based on the importance indicating the degree of association with 1p / 19q co-deletion. This improves the accuracy of predicting the presence or absence of 1p / 19q co-deletion in gliomas, enabling image diagnosis support with sufficient specificity, sensitivity, objectivity, and versatility.

[0143] Furthermore, when specifying a region of interest from an MRI head image, the image diagnosis support device 1 according to this embodiment extracts an abnormal region including a glioma-related lesion on a FLAIR image, and targets a region including not only the region of the lesion but also information on the margins of the lesion. Because a mask image including such a region of interest is used as the analysis target, the correlation with 1p / 19q codeletion at the margins of the glioma-related lesion is taken into account, thereby improving the accuracy of predicting the presence or absence of 1p / 19q codeletion.

[0144] Furthermore, because the image diagnosis support device 1 according to this embodiment predicts the presence or absence of molecular biological markers for brain tumors based on head images, it is possible to formulate a treatment plan based on preoperative imaging findings rather than pathological diagnosis. In particular, it becomes possible to formulate a treatment strategy for brain tumors, including preoperative chemotherapy, which can avoid the risk of complications associated with aiming for maximum brain tumor removal, reduce the area of ​​brain resection, improve tumor removal rates, and even preserve radiation therapy. These improvements result in improved safety and therapeutic effectiveness.

[0145] On the other hand, even if a favorable response to chemotherapy is not predicted, the purpose of the initial surgery and the overall treatment strategy can be clarified preoperatively. Furthermore, by not placing intracerebral anticancer drugs, treatment with little effect and the possibility of adverse events can be avoided. In this way, by formulating and individualizing a treatment plan based on preoperative imaging findings, it becomes possible to preoperatively design an overall treatment strategy, including not only postoperative adjuvant therapy but also surgical resection strategies. This will also promote personalized treatment of brain tumors, which is expected to improve functional and life prognosis.

[0146] The above describes the embodiments of the image diagnosis support device, image diagnosis support method, and image diagnosis support program of the present invention, but the present invention is not limited to the described embodiments, and various modifications that a person skilled in the art can conceive are possible within the scope of the invention described in the claims.

[0147] For example, in the described embodiment, the extraction unit 12 extracts image features using radiomics analysis. However, as mentioned above, the extraction unit 12 may extract image features using a convolutional neural network such as VGG19. While the final layer of VGG19 captures the features of the entire image, layers prior to the final layer capture the features of each part of the image, which can be utilized. In this case, the process in which the learning unit 23 trains the first model M1 to learn the importance of the association between image features and 1p / 19q codeletion using gradient boosting or the like, i.e., the process in which the selection unit 13 selects image features that satisfy the set conditions from the image features extracted by the extraction unit 12 and passes them to the classification unit 14, may be omitted.

[0148] In the embodiment described above, the presence or absence of 1p / 19q co-deletion is predicted as a molecular biological marker for glioma. However, the molecular biological marker may be, for example, the presence or absence of methylation in the promoter region of the MGMT gene.

[0149] In the embodiment described above, the extraction unit 12 and the selection unit 13 are configured as separate functional blocks. However, the extraction unit 12 may be configured to include the functions of the selection unit 13. In this case, the extraction unit 12 extracts image features from the mask image of the region of interest generated by the image processing unit 11, selects image features that satisfy set conditions from the extracted image features, and passes the selected image features to the classification unit 14.

[0150] The image diagnosis support program according to the embodiment described above may be recorded on a computer-readable recording medium such as a hard disk or a flexible disk, and may be executed by being read from the recording medium by a computer. The program may also be stored on a transmission medium that can be distributed via a network such as the Internet.

[0151] It should be noted that the various functional blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented using a general purpose processor, a GPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of the above designed to achieve the functionality described above.

[0152] A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any such configuration of computing devices. [Explanation of symbols]

[0153] 1...image diagnosis support device, 2...learning device, 10...first acquisition unit, 11, 21...image processing unit, 12...extraction unit, 13...selection unit, 14...classification unit, 15...presentation unit, 16, 24...memory, 16A...inference program, M1...first model, M2...second model, 16B, 24B...additional data, 16C, 24C...selection conditions, 24A...learning program, 101, 201...bus, 102, 202...processor, 103, 203...main memory device, 104, 204...communication I / F, 105...MRI device, 106, 206...auxiliary memory device, 106A, 206A...image processing program, 107, 207...input / output I / O, 108, 208...display device, 205...teacher data DB, NW...network.

Claims

1. an acquisition unit that acquires MRI images of the head; an image processing unit that extracts an area including information about a lesion site and a periphery of the lesion site from the acquired image of the head, and generates a first image having the extracted area; an extraction unit that extracts a predetermined image feature from the first image; a classification unit that classifies the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image feature amount extracted by the extraction unit; a presentation unit that presents support information regarding whether or not the lesion site included in the head image has the molecular biological marker based on the classification result by the classification unit; An image diagnosis support device comprising:

2. an acquisition unit that acquires MRI images of the head; an image processing unit that extracts an area including information about a lesion site and a periphery of the lesion site from the acquired image of the head, and generates a first image having the extracted area; an extraction unit that extracts predetermined image features from the first image and selects, from the extracted image features, image features that satisfy a condition set based on an importance indicating a degree of relevance to a molecular biological marker; a classification unit that classifies the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image feature amount selected by the extraction unit; a presentation unit that presents support information regarding whether or not the lesion site included in the head image has the molecular biological marker based on the classification result by the classification unit; An image diagnosis support device comprising:

3. 2. The image diagnosis support device according to claim 1, Further, a selection unit is provided for selecting an image feature amount that satisfies a set condition from the image feature amounts extracted by the extraction unit, the set conditions include conditions set based on importance indicating a degree of relevance of the image feature extracted by the extraction unit to the molecular biological marker, The classification unit classifies the presence or absence of the molecular biological marker in the lesion site with respect to the image feature amount selected by the selection unit. An image diagnosis support device characterized by:

4. 4. The image diagnosis support device according to claim 2, The importance is determined by a learning model that has previously learned the degree of relevance of the image feature to the molecular biological marker. An image diagnosis support device characterized by:

5. 5. The image diagnosis support device according to claim 4, The importance is determined by learning the learning model taking into consideration the presence or absence of calcification in the lesion area in a head CT image corresponding to the head image. An image diagnosis support device characterized by:

6. 6. The image diagnosis support device according to claim 4, The importance is determined by learning the learning model taking into consideration the age of the subject related to the head image and localization information of the lesion site in the head. An image diagnosis support device characterized by:

7. 7. The image diagnosis support device according to claim 4, The learning model includes any of a random forest, a gradient boosted tree, and a LASSO model. An image diagnosis support device characterized by:

8. 8. The image diagnosis support device according to claim 1, The image processing unit extracts a region including the lesion site from the image of the head using a FLAIR image of the head obtained by MRI, and generates the first image. An image diagnosis support device characterized by:

9. 9. The image diagnosis support device according to claim 1, The extraction unit extracts the image feature amount by radiomics analysis of the first image. An image diagnosis support device characterized by:

10. 10. The image diagnosis support device according to claim 1, The classifier includes one of a support vector machine, a random forest, and a gradient boosting tree. An image diagnosis support device characterized by:

11. 11. The image diagnosis support device according to claim 1, the lesion site indicates a glioma, The molecular biological marker of the lesion is a co-deletion of chromosomes 1p / 19q. An image diagnosis support device characterized by:

12. The image diagnosis support device according to any one of claims 1 to 11, The presentation unit displays, on a display screen, the image of the head, the first image, and the support information regarding whether or not the lesion site included in the image of the head based on the classification result has the molecular biological marker. An image diagnosis support device characterized by:

13. a first step of acquiring an image of the head by MRI; a second step of extracting an area including information on the lesion and the edge of the lesion from the acquired image of the head, and generating a first image including the extracted area; a third step of extracting predetermined image features from the first image; a fourth step of classifying the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image feature amount extracted in the third step; a fifth step of presenting support information regarding whether the lesion site included in the head image has the molecular biological marker based on the classification result in the fourth step; An image diagnosis support method comprising:

14. The image diagnosis support method according to claim 13, Further, a sixth step of selecting image feature amounts that satisfy a set condition from the image feature amounts extracted in the third step is provided, the set conditions include conditions set based on importance indicating a degree of relevance of the image feature extracted in the third step to the molecular biological marker; the importance is determined by a learning model that has previously learned the degree of relevance of the image feature to the molecular biological marker; The fourth step classifies the presence or absence of the molecular biological marker in the lesion area with respect to the image feature amount selected in the sixth step.

1. An image diagnosis support method comprising:

15. On the computer, a first step of acquiring an image of the head by MRI; a second step of extracting an area including information on the lesion and the edge of the lesion from the acquired image of the head, and generating a first image including the extracted area; a third step of extracting predetermined image features from the first image; a fourth step of classifying the presence or absence of a molecular biological marker related to the lesion site using a pre-constructed classifier based on the image feature amount extracted in the third step; a fifth step of presenting support information regarding whether the lesion site included in the head image has the molecular biological marker based on the classification result in the fourth step; An image diagnosis support program for performing the above.

16. 16. The image diagnosis support program according to claim 15, Furthermore, a sixth step of selecting image feature quantities that satisfy a set condition from the image feature quantities extracted in the third step is executed, the set conditions include conditions set based on importance indicating a degree of relevance of the image feature extracted in the third step to the molecular biological marker; the importance is determined by a learning model that has previously learned the degree of relevance of the image feature to the molecular biological marker; The fourth step classifies the presence or absence of the molecular biological marker in the lesion area with respect to the image feature amount selected in the sixth step. An image diagnosis support program characterized by:

Images