DIAGNOSIS SUPPORT DEVICE, OPERATION METHOD OF DIAGNOSIS SUPPORT DEVICE, OPERATION PROGRAM OF DIAGNOSIS SUPPORT DEVICE, AND DEMENTIA DIAGNOSIS SUPPORT METHOD

Abstract

This diagnosis assistance device comprises a processor and a memory that is connected to or built into the processor, wherein the processor: obtains a medical image; extracts a plurality of anatomical regions of an organ from the medical image; inputs an image of the plurality of anatomical regions into a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions; causes a plurality of feature amounts for each of the plurality of anatomical regions to be output from the feature amount derivation models; inputs the plurality of feature amounts output for the respective anatomical regions into a disease findings derivation model; causes the disease findings derivation model to output a finding; derives a first contribution degree, which indicates the degree of contribution to the finding output, for each anatomical region; and presents the finding and the derivation results for the first contribution degree for each anatomical region.

Description

[Technical field]

[0001] The technology disclosed herein relates to a diagnostic support device, an operating method for the diagnostic support device, an operating program for the diagnostic support device, and a dementia diagnostic support method. [Background technology]

[0002] In diagnosing diseases, such as dementia typified by Alzheimer's disease, doctors refer to medical images such as head MRI (Magnetic Resonance Imaging) images. Doctors obtain findings of dementia by observing, for example, the degree of atrophy of the hippocampus, parahippocampal gyrus, amygdala, etc., the degree of vascular disorder in white matter, and the presence or absence of reduced blood flow metabolism in the frontal lobe, temporal lobe, and occipital lobe.

[0003] Patent No. 6483890 describes a diagnostic support device that uses a machine learning model to derive dementia findings from head MRI images and provide them to doctors. The diagnostic support device described in Patent No. 6483890 extracts multiple anatomical regions from head MRI images according to Brodmann's brain atlas, etc., and calculates a Z value that indicates the degree of atrophy of each anatomical region. The calculated Z value of each anatomical region is then input to a machine learning model, which outputs dementia findings. Summary of the Invention [Problem to be solved by the invention]

[0004] As mentioned above, in order to obtain findings of diseases such as dementia, it is necessary to thoroughly examine each anatomical region of an organ such as the brain from various viewpoints. However, in Patent No. 6483890, only one statistically obtained index value, the Z value, is used. Therefore, there is a limit to the accuracy of predicting findings of diseases obtained from such limited information.

[0005] Furthermore, even if a doctor were able to obtain disease findings with high predictive accuracy, he or she would be unable to judge the likelihood of the findings unless he or she knew the extent to which each anatomical region contributed to the output of the findings.

[0006] One embodiment of the technology of the present disclosure provides a diagnostic support device, an operating method of the diagnostic support device, an operating program of the diagnostic support device, and a dementia diagnostic support method, which are capable of obtaining more accurate findings of a disease and of knowing the extent to which each anatomical region of an organ contributed to the output findings. [Means for solving the problem]

[0007] The diagnosis support device of the present disclosure includes a processor and a memory connected to or built into the processor, and the processor acquires a medical image, extracts a plurality of anatomical regions of an organ from the medical image, inputs the images of the plurality of anatomical regions to a plurality of feature derivation models prepared for each of the plurality of anatomical regions, causes the feature derivation models to output a plurality of features for each of the plurality of anatomical regions, inputs the plurality of features output for each of the plurality of anatomical regions to a disease finding derivation model, causes the disease finding to be output from the disease finding derivation model, derives a first contribution degree indicating the degree of contribution to the output of the finding for each anatomical region, and presents the findings and the derivation result of the first contribution degree for each anatomical region.

[0008] The processor preferably presents the derivation results in descending order of the first contribution degree.

[0009] The processor preferably inputs disease-related information relating to the disease into the disease observation derivation model in addition to the plurality of features.

[0010] It is preferable that the disease-related information has a plurality of items, and the processor derives a second contribution representing the degree of contribution of each item to the output of the findings, and presents the derivation result of the second contribution for each item.

[0011] The feature derivation model preferably includes at least one of an autoencoder, a single-task convolutional neural network for class discrimination, and a multi-task convolutional neural network for class discrimination.

[0012] The processor preferably inputs an image of one anatomical region to a plurality of different feature derivation models and causes each of the plurality of feature derivation models to output a feature.

[0013] The disease finding derivation model is preferably constructed by any one of the following techniques: neural network, support vector machine, and boosting.

[0014] The processor preferably performs a normalization process to align the acquired medical image with a standard medical image prior to extraction of the anatomical region.

[0015] Preferably, the organ is the brain, and the disease is dementia. In this case, the plurality of anatomical regions preferably includes at least one of the hippocampus and the anterior temporal lobe. Also, the disease-related information preferably includes at least one of the volume of the anatomical region, the score of a dementia test, the test result of a genetic test, the test result of a cerebrospinal fluid test, and the test result of a blood test.

[0016] A method of operating the diagnosis support device disclosed herein includes acquiring a medical image, extracting a plurality of anatomical regions of an organ from the medical image, inputting the images of the plurality of anatomical regions into a plurality of feature derivation models prepared for each of the plurality of anatomical regions and outputting a plurality of features for each of the plurality of anatomical regions from the feature derivation models, inputting the plurality of features output for each of the plurality of anatomical regions into a disease finding derivation model and outputting disease findings from the disease finding derivation model, deriving a first contribution degree indicating the degree of contribution to the output of findings for each anatomical region, and presenting the findings and the derivation results of the first contribution degree for each anatomical region.

[0017] An operating program for the diagnosis support device disclosed herein causes a computer to execute processes including acquiring a medical image, extracting a plurality of anatomical regions of an organ from the medical image, inputting the images of the plurality of anatomical regions into a plurality of feature derivation models prepared for each of the plurality of anatomical regions and outputting a plurality of feature amounts for each of the plurality of anatomical regions from the feature derivation models, inputting the plurality of feature amounts output for each of the plurality of anatomical regions into a disease finding derivation model and outputting disease findings from the disease finding derivation model, deriving a first contribution degree indicating the degree of contribution to the output of findings for each anatomical region, and presenting the findings and the derivation results of the first contribution degree for each anatomical region.

[0018] The dementia diagnosis support method disclosed herein includes a computer having a processor and a memory connected to or built into the processor, which acquires a medical image showing the brain, extracts multiple anatomical regions of the brain from the medical image, inputs the images of the multiple anatomical regions to multiple feature derivation models prepared for each of the multiple anatomical regions, and outputs multiple feature amounts for each of the multiple anatomical regions from the feature derivation model, inputs the multiple feature amounts output for each of the multiple anatomical regions to a dementia finding derivation model, and outputs dementia findings from the dementia finding derivation model, deriving a first contribution degree indicating the degree of contribution to the output of findings for each anatomical region, and presents the findings and the derivation results of the first contribution degree for each anatomical region. Effect of the Invention

[0019] According to the technology of the present disclosure, it is possible to provide a diagnostic support device, an operating method for the diagnostic support device, an operating program for the diagnostic support device, and a dementia diagnostic support method, which are capable of obtaining more accurate findings of a disease and of knowing the extent to which each anatomical region of an organ contributed to the output of findings. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram illustrating a medical system including a diagnosis support device. [Diagram 2] FIG. 2 is a block diagram showing a computer constituting the diagnosis support apparatus. [Diagram 3] 2 is a block diagram showing a processing unit of a CPU of the diagnosis support device. FIG. [Figure 4] FIG. 13 is a diagram illustrating the process of a normalization unit. [Diagram 5] FIG. 13 is a diagram illustrating the process of an extraction unit. [Figure 6] FIG. 13 is a diagram illustrating a process performed by a feature amount derivation unit. [Figure 7] FIG. 13 is a diagram showing the processing of a dementia findings derivation unit. [Figure 8] FIG. 11 is a diagram showing first contribution degree information. [Figure 9] FIG. 2 is a diagram showing a first display screen. [Figure 10] FIG. 13 is a diagram showing a second display screen. [Figure 11] FIG. 13 is a diagram showing a third display screen. [Figure 12] FIG. 13 is a diagram showing switching of display of first contribution degree information. [Figure 13] FIG. 1 is a diagram illustrating the configuration of an autoencoder and the establishment of a feature derivation model. [Figure 14] FIG. 11 is an explanatory diagram of a convolution process. [Figure 15] FIG. 4 is a diagram showing a configuration of calculation data. [Figure 16] FIG. 11 is an explanatory diagram of a pooling process. [Figure 17] FIG. 1 is a diagram showing an overview of the processing in the learning phase of an autoencoder. [Figure 18] FIG. 13 is a diagram showing an overview of processing in the learning phase of the dementia findings derivation model. [Figure 19] 13 is a flowchart showing a processing procedure of the diagnosis support device. [Figure 20] 13 is a flowchart showing a processing procedure of the diagnosis support device. [Figure 21] FIG. 13 is a diagram showing another example of dementia finding information. [Figure 22] FIG. 13 is a diagram showing yet another example of dementia finding information. [Diagram 23] FIG. 13 is a diagram showing the processing of a dementia findings derivation unit in the second embodiment. [Figure 24]FIG. 13 is a diagram showing an overview of processing in a learning phase of the dementia findings derivation model according to the second embodiment. [Diagram 25] 13A to 13C are diagrams illustrating a process of a contribution degree derivation unit and second contribution degree information in the second embodiment. [Figure 26] FIG. 11 is a diagram showing a third display screen in the second embodiment. [Figure 27] FIG. 1 is a diagram showing the configuration of a single-task convolutional neural network for classifying data and the establishment of a feature extraction model. [Figure 28] FIG. 1 is a diagram showing an overview of processing in the learning phase of a single-task convolutional neural network for classifying classes. [Figure 29] FIG. 1 is a diagram showing the configuration of a convolutional neural network for multitask class discrimination and the establishment of a feature derivation model. [Diagram 30] FIG. 1 is a diagram showing an overview of the processing in the learning phase of a multitask convolutional neural network for classification. [Diagram 31] FIG. 13 is a diagram illustrating a process performed by a feature amount derivation unit according to the fifth embodiment. [Diagram 32] This figure shows the configuration of an autoencoder and a single-task convolutional neural network for classifying, and the formation of a feature extraction model. [Diagram 33] FIG. 2 is a diagram illustrating a detailed configuration of an output unit. [Diagram 34] FIG. 1 is a diagram showing an overview of the processing in the learning phase of an autoencoder and a single-task convolutional neural network for classification. [Diagram 35] 13 is a graph showing the evolution of weights given to autoencoder losses. [Diagram 36] FIG. 23 is a diagram showing the processing of a dementia findings derivation unit in the sixth embodiment. [Figure 37] 13 is a table showing a performance comparison between a method for predicting the progression of dementia described in a conventional document and a method for predicting the progression of dementia of the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] [First embodiment] 1, the medical system 2 includes an MRI apparatus 10, a PACS (Picture Archiving and Communication System) server 11, and a diagnostic support device 12. The MRI apparatus 10, the PACS server 11, and the diagnostic support device 12 are connected to a LAN (Local Area Network) 13 installed in a medical facility, and are capable of communicating with each other via the LAN 13.

[0022] The MRI device 10 captures an image of the head of patient P and outputs a head MRI image 15. The head MRI image 15 is voxel data representing the three-dimensional shape of the head of patient P. FIG. 1 shows a head MRI image 15S of a sagittal section. The MRI device 10 transmits the head MRI image 15 to a PACS server 11. The PACS server 11 stores and manages the head MRI image 15 from the MRI device 10. The head MRI image 15 is an example of a "medical image" according to the technology of the present disclosure.

[0023] The diagnostic support device 12 is, for example, a desktop personal computer, and includes a display 17 and an input device 18. The input device 18 is a keyboard, a mouse, a touch panel, a microphone, or the like. The doctor operates the input device 18 to transmit a request for delivery of the head MRI image 15 of the patient P to the PACS server 11. The PACS server 11 searches for the head MRI image 15 of the patient P for which delivery has been requested, and delivers it to the diagnostic support device 12. The diagnostic support device 12 displays the head MRI image 15 delivered from the PACS server 11 on the display 17. The doctor observes the brain of the patient P shown in the head MRI image 15, and diagnoses dementia for the patient P. The brain is an example of an "organ" according to the technology of the present disclosure, and dementia is an example of a "disease" according to the technology of the present disclosure. In addition, although only one MRI device 10 and one diagnostic support device 12 are depicted in FIG. 1, there may be multiple MRI devices 10 and multiple diagnostic support devices 12.

[0024] 2, the computer constituting the diagnosis support device 12 includes, in addition to the above-mentioned display 17 and input device 18, a storage 20, a memory 21, a CPU (Central Processing Unit) 22, and a communication unit 23. These are interconnected via a bus line 24. The CPU 22 is an example of a "processor" according to the technology of the present disclosure.

[0025] The storage 20 is a hard disk drive built into the computer constituting the diagnostic support device 12 or connected via a cable or network. Alternatively, the storage 20 is a disk array in which multiple hard disk drives are connected in series. The storage 20 stores control programs such as an operating system, various application programs, and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0026] The memory 21 is a work memory for the CPU 22 to execute processing. The CPU 22 loads a program stored in the storage 20 into the memory 21 and executes processing according to the program. In this way, the CPU 22 comprehensively controls each part of the computer. The communication unit 23 controls the transmission of various information to and from an external device such as the PACS server 11. The memory 21 may be built into the CPU 22.

[0027] As an example, as shown in Fig. 3, an operating program 30 is stored in the storage 20 of the diagnostic support device 12. The operating program 30 is an application program for causing a computer to function as the diagnostic support device 12. In other words, the operating program 30 is an example of an "operating program of a diagnostic support device" according to the technology of the present disclosure. The storage 20 also stores a head MRI image 15, a standard head MRI image 35, a segmentation model 36, a feature derivation model group 38 composed of a plurality of feature derivation models 37, and a dementia finding derivation model 39.

[0028] When the operating program 30 is started, the CPU 22 of the computer constituting the diagnostic support device 12, in cooperation with the memory 21, etc., functions as a read / write (hereinafter abbreviated as RW (Read Write)) control unit 45, a normalization unit 46, an extraction unit 47, a feature derivation unit 48, a dementia finding derivation unit 49, a contribution degree derivation unit 50, and a display control unit 51.

[0029] The RW control unit 45 controls the storage of various data in the storage 20 and the reading of various data in the storage 20. For example, the RW control unit 45 receives a head MRI image 15 from the PACS server 11, and stores the received head MRI image 15 in the storage 20. Note that although only one head MRI image 15 is stored in the storage 20 in FIG. 3, a plurality of head MRI images 15 may be stored in the storage 20.

[0030] The RW control unit 45 reads out head MRI image 15 of patient P specified by a doctor to diagnose dementia from storage 20, and outputs the read head MRI image 15 to the normalization unit 46 and the display control unit 51. By reading head MRI image 15 from storage 20, the RW control unit 45 acquires head MRI image 15.

[0031] Moreover, the RW control unit 45 reads out the standard head MRI image 35 from the storage 20, and outputs the read out standard head MRI image 35 to the normalization unit 46. The RW control unit 45 reads out the segmentation model 36 from the storage 20, and outputs the read out segmentation model 36 to the extraction unit 47. The RW control unit 45 reads out the feature derivation model group 38 from the storage 20, and outputs the read out feature derivation model group 38 to the feature derivation unit 48. Furthermore, the RW control unit 45 reads out the dementia finding derivation model 39 from the storage 20, and outputs the read out dementia finding derivation model 39 to the dementia finding derivation unit 49.

[0032] The normalization unit 46 performs normalization processing to match the head MRI image 15 with the standard head MRI image 35, and converts the head MRI image 15 into a normalized head MRI image 55. The normalization unit 46 outputs the normalized head MRI image 55 to the extraction unit 47.

[0033] The standard head MRI image 35 is a head MRI image that shows a brain with a standard shape, size, and density (pixel value). The standard head MRI image 35 is, for example, an image generated by averaging head MRI images 15 of multiple healthy subjects, or an image generated by computer graphics. The standard head MRI image 35 is an example of a "standard medical image" according to the technology of the present disclosure.

[0034] The extraction unit 47 inputs the normalized head MRI image 55 to the segmentation model 36. The segmentation model 36 is a machine learning model that performs so-called semantic segmentation, in which a label representing each anatomical region of the brain, such as the hippocampus, amygdala, and frontal lobe, is assigned to each pixel of the brain captured in the normalized head MRI image 55. The extraction unit 47 extracts images of multiple anatomical regions of the brain (hereinafter referred to as anatomical region images) 56 from the normalized head MRI image 55 based on the labels assigned by the segmentation model 36. The extraction unit 47 outputs an anatomical region image group 57 consisting of multiple anatomical region images 56 for each of the multiple anatomical regions to the feature amount derivation unit 48, the contribution degree derivation unit 50, and the display control unit 51.

[0035] One feature derivation model 37 is prepared for each of the multiple anatomical regions of the brain (see FIG. 6). The feature derivation unit 48 inputs an anatomical region image 56 to the corresponding feature derivation model 37. Then, the feature derivation unit 48 outputs a feature set 58 consisting of multiple types of feature amounts Z (see FIG. 6) from the feature derivation model 37. The feature derivation unit 48 outputs a feature set group 59 consisting of the multiple feature sets 58 corresponding to the multiple anatomical regions to the dementia findings derivation unit 49 and the contribution degree derivation unit 50.

[0036] The dementia finding derivation unit 49 inputs the feature set group 59 to the dementia finding derivation model 39. Then, the dementia finding derivation model 39 outputs dementia finding information 60 indicating dementia findings. The dementia finding derivation unit 49 outputs the dementia finding information 60 to the contribution degree derivation unit 50 and the display control unit 51. The dementia finding derivation model 39 is an example of a "disease finding derivation model" according to the technology of the present disclosure.

[0037] The contribution degree derivation unit 50 derives a first contribution degree representing the degree of contribution to the output of the dementia finding information 60 for each anatomical region, based on the anatomical region image group 57 from the extraction unit 47, the feature amount set group 59 from the feature amount derivation unit 48, and the dementia finding information 60 from the dementia finding derivation unit 49. The contribution degree derivation unit 50 derives the first contribution degree using, for example, a method based on Grad-CAM (Gradient-weighted Class Activation Mapping)++ described in the following document. <Daniel Omeiza, etc., Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models, arXiv:1908.01224, Aug 2019.>

[0038] The contribution degree derivation unit 50 generates first contribution degree information 61 from the derived first contribution degree. The first contribution degree information 61 is an example of a “derivation result of the first contribution degree” according to the technology of the present disclosure. The contribution degree derivation unit 50 outputs the first contribution degree information 61 to the display control unit 51.

[0039] The display control unit 51 controls the display of various screens on the display 17. The various screens include a first display screen 75 (see FIG. 9) for instructing analysis using the segmentation model 36, the feature derivation model 37, and the dementia findings derivation model 39, a second display screen 80 (see FIG. 10) for displaying the dementia findings information 60, and a third display screen 85 (see FIG. 11) for displaying the first contribution degree information 61.

[0040] 4 as an example, the normalization unit 46 performs a shape normalization process 65 and a density normalization process 66 on the head MRI image 15 as normalization processes. The shape normalization process 65 is, for example, a process of extracting landmarks serving as references for alignment from the head MRI image 15 and the standard head MRI image 35, and translating, rotating, and / or scaling the head MRI image 15 to match the standard head MRI image 35 so as to maximize the correlation between the landmarks of the head MRI image 15 and the landmarks of the standard head MRI image 35. The density normalization process 66 is, for example, a process of correcting the density histogram of the head MRI image 15 to match the density histogram of the standard head MRI image 35.

[0041] As an example, as shown in FIG. 5, the extraction unit 47 extracts an anatomical area image 56, such as an anatomical area image 56_1 of the hippocampus, an anatomical area image 56_2 of the parahippocampal gyrus, an anatomical area image 56_3 of the frontal lobe, an anatomical area image 56_4 of the anterior temporal lobe, an anatomical area image 56_5 of the occipital lobe, an anatomical area image 56_6 of the thalamus, an anatomical area image 56_7 of the hypothalamus, an anatomical area image 56_8 of the amygdala, an anatomical area image 56_9 of the pituitary gland, etc. In addition to these, the extraction unit 47 extracts an anatomical area image 56 of each anatomical area such as the mammillary body, the corpus callosum, the fornix, and the lateral ventricle. The anatomical areas such as the hippocampus, the frontal lobe, the anterior temporal lobe, and the amygdala are a pair on the left and right. Although not shown in the figure, for such a pair of anatomical areas on the left and right, an anatomical area image 56 of each of the left and right anatomical areas is extracted. For example, for the hippocampus, an anatomical area image 56_1 of the left hippocampus and an anatomical area image 56_1 of the right hippocampus are extracted. Among these anatomical areas, it is preferable to include at least one of the hippocampus and the anterior temporal lobe, and it is more preferable to include both the hippocampus and the anterior temporal lobe. The anterior temporal lobe means the front part of the temporal lobe. The extraction of the anatomical area by the extraction unit 47 using the segmentation model 36 is performed, for example, using the method described in the following document. <Patrick McClure, etc., Knowing What You Know in Brain Segmentation Using Bayesian Deep Neural Networks, Front. Neuroinform., 17 October 2019.>

[0042] 6, the feature derivation unit 48 inputs a hippocampal anatomical region image 56_1 to a hippocampal feature derivation model 37_1 and causes the hippocampal feature derivation model 37_1 to output a hippocampal feature set 58_1. The hippocampal feature set 58_1 is composed of a plurality of features Z1_1, Z2_1, . . . , ZN_1. N is the number of features, which may range from several tens to several hundreds of thousands, for example.

[0043] Similarly, the feature derivation unit 48 inputs the anatomical area image 56_2 of the parahippocampal gyrus to the feature derivation model 37_2 of the parahippocampal gyrus, inputs the anatomical area image 56_3 of the frontal lobe to the feature derivation model 37_3 of the frontal lobe, and inputs the anatomical area image 56_4 of the anterior temporal lobe to the feature derivation model 37_4 of the anterior temporal lobe. Then, the feature derivation model 37_2 of the parahippocampal gyrus outputs a feature set 58_2 of the parahippocampal gyrus, the feature derivation model 37_3 of the frontal lobe outputs a feature set 58_3 of the frontal lobe, and the feature derivation model 37_4 of the anterior temporal lobe outputs a feature set 58_4 of the anterior temporal lobe. The parahippocampal gyrus feature set 58_2 is composed of multiple features Z1_2, Z2_2, ..., ZN_2, the frontal lobe feature set 58_3 is composed of multiple features Z1_3, Z2_3, ..., ZN_3, and the anterior temporal lobe feature set 58_4 is composed of multiple features Z1_4, Z2_4, ..., ZN_4.

[0044] The feature derivation unit 48 also inputs the occipital lobe anatomical region image 56_5 to the occipital lobe feature derivation model 37_5, and inputs the thalamus anatomical region image 56_6 to the thalamus feature derivation model 37_6. Then, the occipital lobe feature derivation model 37_5 outputs the occipital lobe feature set 58_5, and the thalamus feature derivation model 37_6 outputs the thalamus feature set 58_6. The occipital lobe feature set 58_5 is composed of a plurality of features Z1_5, Z2_5, . . . , ZN_5, and the thalamus feature set 58_6 is composed of a plurality of features Z1_6, Z2_6, . . . , ZN_6. In this way, the plurality of anatomical region images 56 are input to the corresponding feature derivation models 37, and the plurality of feature sets 58 for each anatomical region image 56 are output from each feature derivation model 37. The number of feature amounts Z may be the same for each anatomical region, such as N in the example, or may be different for each anatomical region.

[0045] 7 as an example, the dementia finding derivation unit 49 inputs a feature set group 59 to the dementia finding derivation model 39. Then, the dementia finding derivation model 39 outputs, as dementia finding information 60, any one of normal (NC; Normal Control), mild cognitive impairment (MCI; Mild Cognitive Impairment), and Alzheimer's disease (AD; Alzheimer's Disease).

[0046] As an example, as shown in FIG. 8, the first contribution information 61 has a first contribution map group 71 consisting of a plurality of first contribution maps 70 for each of a plurality of anatomical regions, and a ranking table 72. The first contribution maps 70 include a first contribution map 70_1 of the hippocampus, a first contribution map 70_3 of the frontal lobe, and a first contribution map 70_5 of the occipital lobe. The first contribution map 70 is a so-called heat map of the first contribution, in which a color is assigned to each pixel of the anatomical region image 56 according to the level of the first contribution. Specifically, a warm color such as red or orange is assigned to a pixel having a relatively high first contribution, and a cool color such as purple or blue is assigned to a pixel having a relatively low first contribution.

[0047] Ranking table 72 is a table in which anatomical regions are arranged in descending order of the degree of first contribution. FIG. 8 shows an example in which the hippocampus is ranked 1st, the parahippocampal gyrus is ranked 2nd, the frontal lobe is ranked 3rd, the anterior temporal lobe is ranked 4th, and so on.

[0048] 9 shows an example of a first display screen 75 for instructing analysis using the segmentation model 36, the feature derivation model 37, and the dementia findings derivation model 39. The first display screen 75 displays head MRI images 15 of a patient P who is to be diagnosed with dementia. The head MRI images 15 are a head MRI image 15S of a sagittal section, a head MRI image 15A of an axial section, and a head MRI image 15C of a coronal section. A button group 76 for switching the display is provided below each of these head MRI images 15S, 15A, and 15C.

[0049] The first display screen 75 is provided with an analysis button 77. When the doctor wishes to perform an analysis using the segmentation model 36, the feature amount derivation model 37, and the dementia finding derivation model 39, the doctor selects the analysis button 77. This causes the CPU 22 to accept an instruction for analysis using the segmentation model 36, the feature amount derivation model 37, and the dementia finding derivation model 39.

[0050] Fig. 10 shows an example of a second display screen 80 that displays dementia findings information 60 obtained as a result of analysis using the segmentation model 36, the feature derivation model 37, and the dementia findings derivation model 39. A message 81 corresponding to the dementia findings information 60 is displayed on the second display screen 80. Fig. 10 shows an example in which the dementia findings information 60 is mild cognitive impairment (MCI), and the message 81 displayed is "suspected mild cognitive impairment".

[0051] The second display screen 80 is provided with a contribution degree derivation button 82. When the doctor wants to know the degree to which each anatomical region contributed to the output of the dementia findings information 60, he / she selects the contribution degree derivation button 82. This causes the CPU 22 to accept an instruction to derive the first contribution degree information 61. When the confirmation button 83 is selected, the display control unit 51 erases the display of the message 81 and returns the second display screen 80 to the first display screen 75.

[0052] FIG. 11 shows an example of the third display screen 85 displaying the first contribution degree information 61 output by the contribution degree derivation unit 50. In addition to the head MRI image 15 and the message 81 corresponding to the dementia findings information 60, the third display screen 85 displays a message 86 indicating the rank of the first contribution degree and the anatomical region of the rank, the anatomical region image 56, and the first contribution degree map 70. The anatomical region image 56 and the first contribution degree map 70 are displayed side by side. Below the anatomical region image 56 and the first contribution degree map 70, a button group 87 for switching the display is provided. When the confirmation button 88 is selected, the display control unit 51 erases the display of the message 86, the anatomical region image 56, and the first contribution degree map 70, and returns the third display screen 85 to the second display screen 80.

[0053] As an example, as shown in FIG. 12, when a button group 87 is operated, the display control unit 51 switches the display of a message 86, an anatomical area image 56, and a first contribution degree map 70. The display control unit 51 switches the display of the message 86, the anatomical area image 56, and the first contribution degree map 70 in descending order of the first contribution degree. FIG. 12 shows an example in which the display is switched according to the ranking table 72 shown in FIG. 8. Specifically, the display is switched in the following order: first, the anatomical area image 56_1 and the first contribution degree map 70_1 of the hippocampus in first place, then the anatomical area image 56_2 and the first contribution degree map 70_2 of the parahippocampal gyrus in second place, then the anatomical area image 56_3 and the first contribution degree map 70_3 of the frontal lobe in third place, and so on.

[0054] As an example, as shown in FIG. 13, a compression unit 91 of an autoencoder (hereinafter abbreviated as AE (Auto Encoder)) 90 is used for the feature derivation model 37. The AE 90 has a compression unit 91 and a restoration unit 92. An anatomical area image 56 is input to the compression unit 91. The compression unit 91 converts the anatomical area image 56 into a feature set 58. The compression unit 91 passes the feature set 58 to the restoration unit 92. The restoration unit 92 generates a restored image 93 of the anatomical area image 56 from the feature set 58.

[0055] The compression unit 91 converts the anatomical region image 56 into a feature set 58 by performing a convolution operation as shown in FIG. 14 as an example. Specifically, the compression unit 91 has a convolution layer 200 represented by "conv (short for convolution)". The convolution layer 200 applies, for example, a 3×3 filter 203 to target data 202 having a plurality of elements 201 arranged two-dimensionally. Then, the convolution layer 200 convolves an element value e of one target element 201I of the elements 201 with element values ​​a, b, c, d, f, g, h, and i of eight elements 201S adjacent to the target element 201I. The convolution layer 200 sequentially performs a convolution operation on each element 201 of the target data 202 while shifting the target element 201I by one element, and outputs an element value of an element 204 of the operation data 205. As a result, operation data 205 having a plurality of elements 204 arranged two-dimensionally like the target data 202 is obtained. The target data 202 inputted first to the convolution layer 200 is the anatomical region image 56, and thereafter, reduced operation data 205S (see FIG. 16) described later is inputted to the convolution layer 200 as the target data 202.

[0056] Assuming that the coefficients of filter 203 are r, s, t, u, v, w, x, y, and z, the element value k of element 204I corresponding to the element of interest 201I in operation data 205, which is the result of the convolution operation on element of interest 201I, can be obtained, for example, by calculating the following equation (1). k=az+by+cx+dw+ev+fu+gt+hs+ir...(1)

[0057] One piece of calculation data 205 is output for each filter 203. When multiple types of filters 203 are applied to one piece of target data 202, calculation data 205 is output for each filter 203. That is, as shown in FIG. 15 as an example, calculation data 205 is generated for the number of filters 203 applied to the target data 202. Furthermore, since the calculation data 205 has multiple elements 204 arranged two-dimensionally, it has a width and a height. The number of calculation data 205 is called the number of channels. FIG. 15 illustrates an example of four-channel calculation data 205 output by applying four filters 203 to the target data 202.

[0058] As an example, as shown in FIG. 16, the compression unit 91 has a pooling layer 210 represented by "pool" (short for pooling) in addition to the convolution layer 200. The pooling layer 210 obtains local statistics of element values ​​of elements 204 of the operation data 205, and generates reduced operation data 205S with the obtained statistics as element values. Here, the pooling layer 210 performs a maximum pooling process to obtain the maximum value of element values ​​in a block 211 of 2×2 elements as the local statistics. If the process is performed while shifting the block 211 by one element in the width direction and height direction, the reduced operation data 205S is reduced to 1 / 2 the size of the original operation data 205. FIG. 16 illustrates a case where b of the element values ​​a, b, e, and f in the block 211A, b of the element values ​​b, c, f, and g in the block 211B, and h of the element values ​​c, d, g, and h in the block 211C are maximum values. Alternatively, mean value pooling processing may be performed in which mean values, rather than maximum values, are obtained as local statistics.

[0059] The compression unit 91 outputs the final operation data 205 by repeating the convolution process by the convolution layer 200 and the pooling process by the pooling layer 210 multiple times. This final operation data 205 is the feature set 58, and the element value of each element 204 of the final operation data 205 is the feature Z. The feature Z thus obtained represents the shape and texture characteristics of the anatomical area, such as the degree of atrophy of the hippocampus, the degree of vascular disorder in the white matter, and the presence or absence of decreased blood flow metabolism in the frontal lobe, anterior temporal lobe, and occipital lobe. Note that, although the description is given in two dimensions for the sake of simplicity, each process is actually performed in three dimensions.

[0060] 17, in a learning phase before the compression unit 91 is converted into the feature derivation model 37, the AE90 receives the training anatomical area image 56L and learns it. The AE90 outputs a training reconstructed image 93L for the training anatomical area image 56L. A loss calculation is performed for the AE90 using a loss function based on the training anatomical area image 56L and the training reconstructed image 93L. Then, update settings for various coefficients of the AE90 (such as the coefficients of the filter 203) are performed according to the result of the loss calculation, and the AE90 is updated according to the update settings.

[0061] In the learning phase of the AE90, the above-mentioned series of processes, including input of the training anatomical area image 56L to the AE90, output of the training reconstructed image 93L from the AE90, loss calculation, update setting, and update of the AE90, are repeated while the training anatomical area image 56L is exchanged. The repetition of the above-mentioned series of processes is terminated when the restoration accuracy of the training reconstructed image 93L from the training anatomical area image 56L reaches a predetermined set level. The compression unit 91 of the AE90 whose restoration accuracy has thus reached the set level is stored in the storage 20 and used as the trained feature derivation model 37.

[0062] In FIG. 18 showing an example of an overview of processing in the learning phase of the dementia finding derivation model 39, the dementia finding derivation model 39 is constructed by any one of the following methods: neural network, support vector machine, and boosting. In the learning phase, the dementia finding derivation model 39 is given learning data 100 and learns. The learning data 100 is a combination of a learning feature set group 59L and correct dementia finding information 60CA corresponding to the learning feature set group 59L. The learning feature set group 59L is obtained by inputting an anatomical region image 56 of a certain head MRI image 15 to the feature derivation model 37. The correct dementia finding information 60CA is a result of an actual diagnosis by a doctor of the dementia findings for the head MRI image 15 from which the learning feature set group 59L was obtained.

[0063] In the learning phase, a group of training feature values ​​59L is input to the dementia finding derivation model 39. The dementia finding derivation model 39 outputs training dementia finding information 60L for the group of training feature values ​​59L. Based on this training dementia finding information 60L and correct dementia finding information 60CA, a loss calculation is performed for the dementia finding derivation model 39 using a loss function. Then, according to the result of the loss calculation, various coefficients of the dementia finding derivation model 39 are updated and set, and the dementia finding derivation model 39 is updated according to the update setting.

[0064] In the learning phase of the dementia finding derivation model 39, the above series of processes, including input of the learning feature set group 59L to the dementia finding derivation model 39, output of the learning dementia finding information 60L from the dementia finding derivation model 39, loss calculation, update setting, and update of the dementia finding derivation model 39, are repeatedly performed while exchanging the learning data 100. The repetition of the above series of processes is terminated when the prediction accuracy of the learning dementia finding information 60L with respect to the correct dementia finding information 60CA reaches a predetermined setting level. The dementia finding derivation model 39 whose prediction accuracy has thus reached the setting level is stored in the storage 20 and used by the dementia finding derivation unit 49 as a learned dementia finding derivation model.

[0065] Next, the operation of the above configuration will be described with reference to the flowcharts of Figures 19 and 20. First, when the operating program 30 is started in the diagnosis support device 12, the CPU 22 of the diagnosis support device 12 functions as a RW control unit 45, a normalization unit 46, an extraction unit 47, a feature amount derivation unit 48, a dementia finding derivation unit 49, a contribution degree derivation unit 50, and a display control unit 51, as shown in Figure 3.

[0066] 19, when an analysis button 77 is selected on the first display screen 75 shown in Fig. 9, the RW control unit 45 reads out the corresponding head MRI image 15 and the standard head MRI image 35 from the storage 20 (step ST100). The head MRI image 15 and the standard head MRI image 35 are output from the RW control unit 45 to the normalization unit 46.

[0067] 4, normalization unit 46 performs normalization processing (shape normalization processing 65 and intensity normalization processing 66) to match head MRI image 15 to standard head MRI image 35 (step ST110). As a result, head MRI image 15 is converted into normalized head MRI image 55. Normalized head MRI image 55 is output from normalization unit 46 to extraction unit 47.

[0068] 5, the extraction unit 47 uses the segmentation model 36 to extract a plurality of anatomical region images 56 from the normalized head MRI image 55 (step ST120). An anatomical region image group 57 consisting of the plurality of anatomical region images 56 is output from the extraction unit 47 to the feature amount derivation unit 48, the contribution degree derivation unit 50, and the display control unit 51.

[0069] 6, in the feature derivation unit 48, an anatomical region image 56 is input to a corresponding feature derivation model 37. As a result, a feature set 58 is output from the feature derivation model 37 (step ST130). A feature set group 59 consisting of the multiple feature sets 58 is output from the feature derivation unit 48 to the dementia findings derivation unit 49 and the contribution degree derivation unit 50.

[0070] 7, in the dementia finding derivation unit 49, the feature set group 59 is input to the dementia finding derivation model 39. As a result, the dementia finding information 60 is output from the dementia finding derivation model 39 (step ST140). The dementia finding information 60 is output from the dementia finding derivation unit 49 to the contribution degree derivation unit 50 and the display control unit 51.

[0071] Under the control of the display control unit 51, the second display screen 80 shown in Fig. 10 is displayed on the display 17 (step ST150). The doctor checks the dementia finding information 60 through the message 81 on the second display screen 80.

[0072] 20, when the contribution degree derivation button 82 is selected on the second display screen 80 and an instruction to derive the first contribution degree information 61 is received by the CPU 22 (YES in step ST200), the contribution degree derivation unit 50 derives the first contribution degree for each anatomical area based on the anatomical area image group 57, the feature amount set group 59, and the dementia finding information 60, and generates the first contribution degree information 61 shown in FIG. 8 from the first contribution degree (step ST210). The first contribution degree information 61 is output from the contribution degree derivation unit 50 to the display control unit 51.

[0073] Under the control of the display control unit 51, the third display screen 85 shown in FIG.

[0074] As described above, the CPU 22 of the diagnosis support device 12 includes the RW control unit 45, the extraction unit 47, the feature derivation unit 48, the dementia finding derivation unit 49, the contribution degree derivation unit 50, and the display control unit 51. The RW control unit 45 acquires the head MRI image 15 by reading the head MRI image 15 of the patient P to be diagnosed with dementia from the storage 20. The extraction unit 47 extracts anatomical region images 56 of a plurality of anatomical regions of the brain from the normalized head MRI image 55. The feature derivation unit 48 inputs the plurality of anatomical region images 56 to a plurality of feature derivation models 37 prepared for each of the plurality of anatomical regions, and causes the feature derivation model 37 to output a plurality of feature sets 58 for each of the plurality of anatomical regions. The dementia finding derivation unit 49 inputs a feature set group 59 composed of the plurality of feature sets 58 to the dementia finding derivation model 39, and causes the dementia finding derivation model 39 to output dementia finding information 60. The contribution degree derivation unit 50 derives, for each anatomical area, a first contribution degree indicating the degree of contribution to the output of the dementia finding information 60. The display control unit 51 presents the dementia finding information 60 and the first contribution degree information 61 to the doctor on the third display screen 85.

[0075] The number of feature values ​​Z is very large, for example, tens to hundreds of thousands. Therefore, the feature value Z does not represent a limited feature of the anatomical area like the Z value described in Patent No. 6483890, but represents a comprehensive feature of the anatomical area. In addition, the feature value Z is not a single value obtained statistically like the Z value described in Patent No. 6483890, but is obtained by inputting the anatomical area image 56 into the feature value derivation model 37. Therefore, the method of the present disclosure for deriving the dementia findings information 60 based on the feature value Z (a feature value set group 59 composed of a plurality of feature value sets 58) can improve the prediction accuracy of dementia findings compared to the method described in Patent No. 6483890, and makes it possible to obtain more accurate dementia findings.

[0076] Dementia, compared to other diseases such as cancer, does not easily show specific lesions in images that can be recognized by the naked eye. In addition, dementia affects the entire brain and is not localized. Due to this background, it has been difficult to obtain accurate dementia findings using a machine learning model from medical images such as head MRI images 15. However, according to the technology disclosed herein, the brain is divided into multiple anatomical regions, features are derived for each of the multiple anatomical regions, and the derived features are input into a single dementia findings derivation model 39. This makes it possible to achieve the goal of obtaining more accurate dementia findings, which was previously difficult.

[0077] Furthermore, the doctor can know the degree to which each anatomical region of the brain contributed to the output of the dementia findings information 60 by the third display screen 85. This allows the doctor to judge the plausibility of the dementia findings information 60.

[0078] In addition, a history of the ranking of the first contribution degree of each anatomical area may be stored and reflected in the learning of the dementia findings derivation model 39. Specifically, feature set 58 of an anatomical area having a relatively low ranking may be excluded from the learning feature set group 59L to be provided to the dementia findings derivation model 39.

[0079] As shown in FIG. 12, the display control unit 51 presents the derivation results in descending order of the first contribution degree. The doctor can see at a glance which anatomical region is contributing. The manner in which the derivation results are presented in descending order of the first contribution degree is not limited to the manner shown in FIG. 12 in which the display of the message 86, the anatomical region image 56, and the first contribution degree map 70 is switched in response to the operation of the button group 87. The message 86, the anatomical region image 56, and the first contribution degree map 70 of each rank may be displayed, for example, vertically arranged, with the higher the rank, the higher the position.

[0080] As shown in Fig. 13, the feature derivation model 37 is a repurposed compression unit 91 of the AE90. The AE90 is one of the neural network models frequently used in the field of machine learning and is generally very well known. For this reason, it can be relatively easily diverted into the feature derivation model 37.

[0081] As shown in Fig. 18, the dementia findings derivation model 39 is constructed by any one of the following techniques: neural network, support vector machine, and boosting. All of these techniques, neural network, support vector machine, and boosting, are generally very well known. For this reason, the dementia findings derivation model 39 can be constructed relatively easily.

[0082] 4, prior to extraction of the anatomical region, the normalization unit 46 performs a normalization process to align the head MRI image 15 with the standard head MRI image 35. Therefore, the individual differences of the patient P and the device differences of the MRI device 10 can be almost eliminated before the subsequent processes are performed, and as a result, the reliability of the dementia findings information 60 can be improved.

[0083] Dementia has become a social problem with the recent advent of an aging society. For this reason, this embodiment, which outputs dementia findings information 60 with the brain as the organ and dementia as the disease, can be said to be a form that matches the current social problem.

[0084] The hippocampus and anterior temporal lobe are anatomical regions that are particularly highly correlated with dementia, including Alzheimer's disease. Therefore, if at least one of the hippocampus and the anterior temporal lobe is included in multiple anatomical regions, as in this example, more accurate findings of dementia can be obtained.

[0085] The presentation form of the dementia finding information 60 and the first contribution degree information 61 is not limited to the third display screen 85. The dementia finding information 60 and the first contribution degree information 61 may be printed out on a paper medium, or the dementia finding information 60 and the first contribution degree information 61 may be sent to the doctor's mobile terminal as an attachment to an e-mail.

[0086] The dementia finding information is not limited to the contents (normal / mild cognitive impairment / Alzheimer's disease) exemplified in Fig. 7. For example, it may be whether the degree of progression of dementia of patient P one year later is fast or slow, as in dementia finding information 105 shown in Fig. 21. Or it may be the type of dementia, such as Alzheimer's disease, dementia with Lewy bodies, or vascular dementia, as in dementia finding information 108 shown in Fig. 22.

[0087] [Second embodiment] In the second embodiment shown in FIGS. 23 to 26, in addition to the multiple feature amounts Z, dementia-related information 111 relating to dementia is input to a dementia findings derivation model 112.

[0088] 23 as an example, the dementia finding derivation unit 110 of this embodiment inputs dementia-related information 111 relating to dementia to a dementia finding derivation model 112 in addition to a feature set group 59. Then, the dementia finding derivation model 112 outputs dementia finding information 113. Note that the dementia-related information 111 is an example of "disease-related information" according to the technology of the present disclosure.

[0089] The dementia-related information 111 is information on the patient P who is diagnosed with dementia. The dementia-related information 111 has a plurality of items. The items include, for example, the volume of the hippocampus. The items also include a Revised Hasegawa's Dementia Scale (HDS-R) score, a Mini-Mental State Examination (MMSE) score, a Rivermead Behavioural Memory Test (RBMT) score, a Clinical Dementia Rating (CDR), Activities of Daily Living (ADL), and an Alzheimer's Disease Assessment Scale-cognitive subscale (ADAS-Cog). The items also include the genotype of the ApoE gene, an amyloid beta measurement value, a tau protein measurement value, an apolipoprotein measurement value, a complement protein measurement value, and a transthyretin measurement value. The items also include the sex, age, and the like of the patient P. HDS-R score, MMSE score, RBMT score, CDR, ADL, ADAS-Cog, ApoE gene type, amyloid beta measurement values, tau protein measurement values, apolipoprotein measurement values, complement protein measurement values, transthyretin measurement values, patient P's gender, age, etc. are quoted from the electronic medical record system (not shown).

[0090] The volume of the hippocampus is, for example, the total number of pixels of the anatomical area image 56_1 of the hippocampus. The volume of the hippocampus is an example of the "volume of an anatomical area" according to the technology of the present disclosure. In addition to or instead of the volume of the hippocampus, the dementia-related information 111 may include the volumes of other anatomical areas such as the amygdala.

[0091] The HDS-R score, the MMSE score, the RBMT score, the CDR, the ADL, and the ADAS-Cog are examples of "dementia test scores" according to the technology of the present disclosure.

[0092] The genotype of the ApoE gene is a combination of two of the three ApoE genes, ε2, ε3, and ε4 (ε2 and ε3, ε3 and ε4, etc.). The risk of developing Alzheimer's disease in a genotype having one or two ε4s (ε2 and ε4, ε4 and ε4, etc.) is approximately 3 to 12 times higher than that in a genotype having no ε4 at all (ε2 and ε3, ε3 and ε3, etc.). The genotype of the ApoE gene is converted into a numerical value, such as 1 for the combination of ε2 and ε3, 2 for the combination of ε3 and ε3, and is input to the dementia findings derivation model 142. The genotype of the ApoE gene is an example of a "test result of a genetic test" according to the technology of the present disclosure.

[0093] The amyloid β measurement value and the tau protein measurement value are examples of the "test result of the cerebrospinal fluid test" according to the technology of the present disclosure. The apolipoprotein measurement value, the complement protein measurement value, and the transthyretin measurement value are examples of the "test result of the blood test" according to the technology of the present disclosure.

[0094] In FIG. 24 showing an example of an overview of the process in the learning phase of the dementia finding derivation model 112, the dementia finding derivation model 112 is trained by being given training data 118. The training data 118 is a set of the training feature set group 59L, the training dementia-related information 111L, and the correct dementia finding information 113CA corresponding to the training feature set group 59L and the training dementia-related information 111L. The training feature set group 59L is obtained by inputting the anatomical area image 56 of a certain head MRI image 15 to the feature derivation model 37. The training dementia-related information 111L is information on the patient P who is the subject of the head MRI image 15 from which the training feature set group 59L was obtained. The correct dementia finding information 113CA is the result of an actual diagnosis by a doctor of the dementia findings for the head MRI image 15 from which the training feature set group 59L was obtained, taking into account the training dementia-related information 111L.

[0095] In the learning phase, the learning feature set group 59L and the learning dementia-related information 111L are input to the dementia finding derivation model 112. The dementia finding derivation model 112 outputs the learning dementia finding information 113L for the learning feature set group 59L and the learning dementia-related information 111L. Based on this learning dementia finding information 113L and the correct dementia finding information 113CA, a loss calculation is performed for the dementia finding derivation model 112 using a loss function. Then, according to the result of the loss calculation, various coefficients of the dementia finding derivation model 112 are updated and the dementia finding derivation model 112 is updated according to the update setting.

[0096] In the learning phase of the dementia finding derivation model 112, the above series of processes, including input of the learning feature set group 59L and the learning dementia-related information 111L to the dementia finding derivation model 112, output of the learning dementia finding information 113L from the dementia finding derivation model 112, loss calculation, update setting, and update of the dementia finding derivation model 112, are repeatedly performed while exchanging the learning data 118. The repetition of the above series of processes is terminated when the prediction accuracy of the learning dementia finding information 113L with respect to the correct dementia finding information 113CA reaches a predetermined setting level. The dementia finding derivation model 112 whose prediction accuracy has thus reached the setting level is stored in the storage 20 and used in the learned dementia finding derivation model dementia finding derivation unit 110.

[0097] As an example, as shown in FIG. 25, the contribution degree derivation unit 120 of this embodiment generates the first contribution degree information 121 based on the anatomical area image group 57, the feature amount set group 59, and the dementia finding information 113, similarly to the first contribution degree information 61 of the above-mentioned first embodiment, and outputs the generated first contribution degree information 121 to the display control unit 123. In addition, the contribution degree derivation unit 120 derives a second contribution degree representing the degree of contribution to the output of the dementia finding information 113 for each item of the dementia-related information 111 based on the dementia-related information 111 and the dementia finding information 113. The contribution degree derivation unit 120 derives a numerical value of 10 levels, for example, from 1 to 10, as the second contribution degree. The contribution degree derivation unit 120 outputs the second contribution degree information 122, which is a summary of the derived second contribution degrees, to the display control unit 123. The second contribution degree information 122 is registered with the second contribution degree corresponding to each item of the dementia-related information 111. The second contribution degree information 122 is an example of a "derivation result of the second contribution degree" according to the technology of the present disclosure. Note that the derivation of the second contribution degree by the contribution degree derivation unit 120 uses, for example, the method described in the following document. <Scott M. Lundberg, etc., Explainable machine-learning predictions for the prevention of hypoxaemia during surgery, Nature Biomedical Engineering volume 2, pages749-760(2018)>

[0098] As an example, as shown in FIG. 26, the third display screen 130 of the present embodiment displays a head MRI image 15, a message 81 according to the dementia findings information 60, a message 86 indicating the rank of the first contribution degree and the anatomical area of ​​the rank, an anatomical area image 56, and a first contribution degree map 70, as well as a list 131 of each item of the dementia-related information 111. Each item of the list 131 is displayed according to the level of the second contribution degree, as shown by the shade of hatching. The display according to the level of the second contribution degree is, for example, displayed in a darker color as the second contribution degree is higher, and displayed in a lighter color as the second contribution degree is lower. FIG. 26 illustrates a case in which the second contribution degree of the hippocampus volume, HDS-R score, and ApoE gene genotype, etc. is relatively high, and the second contribution degree of the MMSE score, CDR, ADL, ADAS-Cog, etc. is relatively low. When the confirmation button 132 is selected, the display control unit 123 erases the display of the message 86, the anatomical area image 56, the first contribution map 70, and the list 131, and returns the third display screen 130 to the second display screen 80.

[0099] As described above, in the second embodiment, the dementia-related information 111 is input to the dementia findings derivation model 112. The dementia-related information 111 includes hippocampal volume, HDS-R score, MMSE score, CDR, ADL, ADAS-Cog, ApoE gene genotype, amyloid β measurement value, tau protein measurement value, apolipoprotein measurement value, complement protein measurement value, transthyretin measurement value, sex and age of patient P, and the like. Since the various dementia-related information 111 related to dementia, which is powerful information useful for prediction, is added, the prediction accuracy of dementia findings can be dramatically improved compared to the case where dementia findings are predicted using only the feature set group 59.

[0100] 25, the contribution degree derivation unit 120 derives a second contribution degree representing the degree of contribution to the output of the dementia finding information 113 for each item of the dementia-related information 111. Then, as shown in FIG. 26, the display control unit 123 presents the derivation result of the second contribution degree for each item on the third display screen 130. This allows the doctor to know the degree to which each item of the dementia-related information 111 contributed to the output of the dementia finding information 60. This allows the doctor to judge the plausibility of the dementia finding information 60 with more confidence.

[0101] The dementia-related information 111 may include at least one of the volume of an anatomical area, the score of a dementia test, the test results of a genetic test, the test results of a cerebrospinal fluid test, and the test results of a blood test. The dementia-related information 111 may include the medical history of the patient P, or whether or not the patient P has a relative who has developed dementia. As in the case of the first contribution degree, the history of the second contribution degree of each item may be stored and reflected in the learning of the dementia finding derivation model 112. Specifically, an item with a relatively low second contribution degree may be excluded from the learning dementia-related information 111L provided to the dementia finding derivation model 112.

[0102] [Third embodiment] In the third embodiment shown in FIGS. 27 and 28, a compression unit 141 of a single-task class discrimination convolutional neural network (hereinafter abbreviated as single-task CNN (Convolutional Neural Network)) 140 is used as a feature derivation model 145 instead of the compression unit 91 of the AE 90.

[0103] As an example, as shown in FIG. 27, a single-task CNN 140 has a compression unit 141 and an output unit 142. An anatomical area image 56 is input to the compression unit 141. The compression unit 141 converts the anatomical area image 56 into a feature set 143, similar to the compression unit 91. The compression unit 141 passes the feature set 143 to the output unit 142. The output unit 142 outputs one class 144 based on the feature set 143. In FIG. 27, the output unit 142 outputs a discrimination result that the subject has dementia or does not have dementia as the class 144. The compression unit 141 of this single-task CNN 140 is used as a feature derivation model 145.

[0104] 28, in a learning phase before the compression unit 141 is converted into a feature derivation model 145, the single-task CNN 140 is given learning data 148 and trained. The learning data 148 is a set of the training anatomical area image 56L and a correct answer class 144CA corresponding to the training anatomical area image 56L. The correct answer class 144CA is a result of a doctor's actual determination of whether or not dementia has developed for the head MRI image 15 from which the training anatomical area image 56L was obtained.

[0105] In the learning phase, the training anatomical area image 56L is input to the single-task CNN 140. The single-task CNN 140 outputs a training class 144L for the training anatomical area image 56L. A loss calculation is performed for the single-task CNN 140 based on the training class 144L and a correct answer class 144CA. Then, various coefficients of the single-task CNN 140 are updated according to the result of the loss calculation, and the single-task CNN 140 is updated according to the update setting.

[0106] In the learning phase of the single-task CNN 140, the above-mentioned series of processes, including input of the learning anatomical area image 56L to the single-task CNN 140, output of the learning class 144L from the single-task CNN 140, loss calculation, update setting, and update of the single-task CNN 140, are repeatedly performed while exchanging the learning data 148. The repetition of the above-mentioned series of processes is terminated when the prediction accuracy of the learning class 144L for the correct class 144CA reaches a predetermined set level. The compression unit 141 of the single-task CNN 140 whose prediction accuracy has thus reached the set level is stored in the storage 20 as a learned feature derivation model 145 and is used by the feature derivation unit 48.

[0107] In this way, in the third embodiment, the compression unit 141 of the single-task CNN 140 is used as the feature derivation model 145. The single-task CNN 140 is also one of the neural network models frequently used in the field of machine learning and is generally very well known. Therefore, it can be relatively easily diverted to the feature derivation model 145.

[0108] The class 144 may be, for example, the age of the patient P being under 75 years old or over 75 years old, or may be the age group of the patient P, such as in their 60s or 70s.

[0109] [Fourth embodiment] In the fourth embodiment shown in Figures 29 and 30, a compression unit 151 of a multi-task class discrimination CNN (hereinafter abbreviated as multi-task CNN) 150 is used as a feature derivation model 156 instead of the compression unit 91 of the AE 90 and the compression unit 141 of the single-task CNN 140.

[0110] As an example, as shown in FIG. 29, the multitask CNN 150 has a compression unit 151 and an output unit 152. The compression unit 151 receives an anatomical region image 56. The compression unit 151 converts the anatomical region image 56 into a feature set 153, similar to the compression units 91 and 141. The compression unit 151 transfers the feature set 153 to the output unit 152. The output unit 152 outputs two classes, a first class 154 and a second class 155, based on the feature set 153. In FIG. 29, the output unit 152 outputs a discrimination result that the patient has dementia or does not have dementia as the first class 154. Also, in FIG. 29, the output unit 152 outputs the age of the patient P as the second class 155. The compression unit 151 of the multitask CNN 150 is used as a feature derivation model 156.

[0111] As an example, as shown in FIG. 30, in a learning phase before converting the compression unit 151 to a feature derivation model 156, the multitask CNN 150 is given learning data 158 and trained. The learning data 158 is a set of the training anatomical area image 56L and a first correct class 154CA and a second correct class 155CA corresponding to the training anatomical area image 56L. The first correct class 154CA is a result of a doctor's actual determination of whether or not dementia has developed for the head MRI image 15 from which the training anatomical area image 56L was obtained. The second correct class 155CA is the actual age of the patient P who is the subject of the head MRI image 15 from which the training anatomical area image 56L was obtained.

[0112] In the learning phase, a training anatomical area image 56L is input to the multi-task CNN 150. The multi-task CNN 150 outputs a first training class 154L and a second training class 155L for the training anatomical area image 56L. A loss calculation is performed for the multi-task CNN 150 based on the first training class 154L and the second training class 155L, and a first correct class 154CA and a second correct class 155CA. Then, update settings of various coefficients of the multi-task CNN 150 are performed according to the result of the loss calculation, and the multi-task CNN 150 is updated according to the update settings.

[0113] In the learning phase of the multitasking CNN 150, a series of processes including input of the learning anatomical area image 56L to the multitasking CNN 150, output of the first learning class 154L and the second learning class 155L from the multitasking CNN 150, loss calculation, update setting, and update of the multitasking CNN 150 are repeatedly performed while exchanging the learning data 158. The repetition of the series of processes is terminated when the prediction accuracy of the first learning class 154L and the second learning class 155L for the first correct class 154CA and the second correct class 155CA reaches a predetermined set level. The compression unit 151 of the multitasking CNN 150 whose prediction accuracy has thus reached the set level is stored in the storage 20 as a learned feature derivation model 156 and is used by the feature derivation unit 48.

[0114] As described above, in the fourth embodiment, the compression unit 151 of the multi-task CNN 150 is used as the feature derivation model 156. Compared with the AE90 and the single-task CNN 140, the multi-task CNN 150 performs a more complicated process of outputting a plurality of classes (a first class 154 and a second class 155). Therefore, the feature set 153 output from the compression unit 151 is likely to more comprehensively represent the features of the anatomical area image 56. As a result, the accuracy of predicting dementia findings by the dementia findings derivation model 39 can be further improved.

[0115] The first class 154 may be, for example, a five-stage level of the degree of progression of dementia. The second class 155 may be a discrimination result of the age of the patient P. The multitask CNN 150 may output three or more classes.

[0116] [Fifth embodiment] In the fifth embodiment shown in FIG. 31, an anatomical region image 56 of one anatomical region is input to a plurality of different feature amount derivation models.

[0117] In Fig. 31, the feature derivation unit 160 of this embodiment inputs an anatomical region image 56 of one anatomical region to a first feature derivation model 161, a second feature derivation model 162, and a third feature derivation model 163. As a result, the feature derivation unit 160 outputs a first feature set 164 from the first feature derivation model 161, outputs a second feature set 165 from the second feature derivation model 162, and outputs a third feature set 166 from the third feature derivation model 163. The first feature derivation model 161 is a repurposed version of the compression unit 91 of the AE 90 of the first embodiment. The second feature derivation model 162 is a repurposed version of the compression unit 141 of the single task CNN 140 of the third embodiment. The third feature derivation model 163 is a repurposed version of the compression unit 151 of the multitask CNN 150 in the fourth embodiment.

[0118] As described above, in the fifth embodiment, the feature derivation unit 160 inputs the anatomical region image 56 of one anatomical region to the first feature derivation model 161, the second feature derivation model 162, and the third feature derivation model 163. Then, the first feature set 164, the second feature set 165, and the third feature set 166 are output from each of the models 161 to 163. Therefore, a wider variety of feature amounts Z can be obtained compared to the case where one type of feature derivation model 37 is used. As a result, the accuracy of predicting dementia findings by the dementia findings derivation model 39 can be further improved.

[0119] The different feature derivation models may be, for example, a combination of a first feature derivation model 161 obtained by reusing the compression unit 91 of the AE90 and a second feature derivation model 162 obtained by reusing the compression unit 141 of the single-task CNN 140. Alternatively, the different feature derivation models may be a combination of a second feature derivation model 162 obtained by reusing the compression unit 141 of the single-task CNN 140 and a third feature derivation model 163 obtained by reusing the compression unit 151 of the multi-task CNN 150. Furthermore, the different feature derivation models may be a combination of a second feature derivation model 162 obtained by reusing the compression unit 141 of the single-task CNN 140 that outputs whether or not dementia has developed as a class 144, and a second feature derivation model 162 obtained by reusing the compression unit 141 of the single-task CNN 140 that outputs the age of the patient P as the class 144.

[0120] [Sixth embodiment] In the sixth embodiment shown in FIGS. 32 to 37, a model combining an AE 250 and a single-task CNN 251 is used as a feature derivation model 252.

[0121] 32, the AE250 has a compression unit 253 and a restoration unit 254, similar to the AE90 of the first embodiment. The compression unit 253 receives an anatomical area image 56. The compression unit 253 converts the anatomical area image 56 into a feature set 255. The compression unit 253 passes the feature set 255 to the restoration unit 254. The restoration unit 254 generates a restored image 256 of the anatomical area image 56 from the feature set 255.

[0122] The single-task CNN 251 has a compression unit 253 and an output unit 257, similar to the single-task CNN 140 of the third embodiment. That is, the compression unit 253 is shared by the AE 250 and the single-task CNN 251. The compression unit 253 transfers the feature set 255 to the output unit 257. The output unit 257 outputs one class 258 based on the feature set 255. In FIG. 32, the output unit 257 outputs, as the class 258, a discrimination result that a patient P currently suffering from mild cognitive impairment will remain suffering from mild cognitive impairment after two years, or will progress to Alzheimer's disease after two years. In addition, the output unit 257 outputs an aggregate feature ZA that aggregates a plurality of feature values ​​Z constituting the feature set 255. The aggregate feature ZA is output for each anatomical region. In this embodiment, the aggregate feature ZA is input to a dementia findings derivation model 282 (see FIG. 36) instead of the feature set 255.

[0123] As an example, as shown in FIG. 33, the output unit 257 has a self-attention (hereinafter abbreviated as SA (Self-Attention)) mechanism layer 265, a global average pooling (hereinafter abbreviated as GAP (Global Average Pooling)) layer 266, a fully connected (hereinafter abbreviated as FC (Fully Connected)) layer 267, a softmax function (hereinafter abbreviated as SMF (SoftMax Function)) layer 268, and a principal component analysis (hereinafter abbreviated as PCA (Principal Component Analysis)) layer 269.

[0124] 14 on the feature set 255 while changing the coefficients of the filter 203 according to the element value of the element of interest 201I. Hereinafter, the convolution process performed by the SA mechanism layer 265 is referred to as SA convolution process. The SA mechanism layer 265 outputs the feature set 255 after the SA convolution process to the GAP layer 266.

[0125] The GAP layer 266 performs global average pooling on the feature set 255 after the SA convolution process. The global average pooling process is a process of calculating the average value of the feature Z for each channel of the feature set 255 (see FIG. 15 ). For example, if the number of channels of the feature set 255 is 512, the average value of the 512 features Z is calculated by the global average pooling process. The GAP layer 266 outputs the calculated average value of the feature Z to the FC layer 267 and the PCA layer 269.

[0126] The FC layer 267 converts the average value of the feature Z into a variable to be handled by the SMF of the SMF layer 268. The FC layer 267 has an input layer having units equal to the number of average values ​​of the feature Z (i.e., the number of channels of the feature set 255), and an output layer having units equal to the number of variables to be handled by the SMF. Each unit in the input layer and each unit in the output layer are fully connected to each other, and a weight is set for each. The average value of the feature Z is input to each unit in the input layer. The sum of the products of the average value of the feature Z and the weight set between each unit becomes the output value of each unit in the output layer. This output value is the variable to be handled by the SMF. The FC layer 267 outputs the variables to be handled by the SMF to the SMF layer 268. The SMF layer 268 outputs the class 258 by applying the variables to the SMF.

[0127] The PCA layer 269 performs PCA on the average values ​​of the feature amounts Z, and converts the average values ​​of the multiple feature amounts Z into a smaller number of aggregated feature amounts ZA. For example, the PCA layer 269 aggregates the average values ​​of 512 feature amounts Z into one aggregated feature amount ZA.

[0128] As an example, as shown in Fig. 34, in a learning phase, the AE250 is input with a learning anatomical area image 56L and learns. The AE250 outputs a learning reconstructed image 256L for the learning anatomical area image 56L. Based on the learning anatomical area image 56L and the learning reconstructed image 256L, a loss calculation is performed for the AE250 using a loss function. Then, update settings of various coefficients of the AE250 are performed according to the result of the loss calculation (hereinafter, referred to as loss L1), and the AE250 is updated according to the update settings.

[0129] In the learning phase of AE250, the above series of processes, including input of the training anatomical area image 56L to AE250, output of the training reconstructed image 256L from AE250, loss calculation, update setting, and updating of AE250, are repeated while the training anatomical area image 56L is replaced.

[0130] In the learning phase, the single-task CNN 251 is trained by being given training data 275. The training data 275 is a set of training anatomical area images 56L and correct answer classes 258CA corresponding to the training anatomical area images 56L. The correct answer classes 258CA indicate whether the patient P of the subject of the head MRI images 15 from which the training anatomical area images 56L were obtained actually remains suffering from mild cognitive impairment after two years or has progressed to Alzheimer's disease after two years.

[0131] In the learning phase, the training anatomical area image 56L is input to the single-task CNN 251. The single-task CNN 251 outputs a training class 258L for the training anatomical area image 56L. Based on the training class 258L and the correct answer class 258CA, a loss calculation is performed for the single-task CNN 251 using a cross-entropy function or the like. Then, various coefficients of the single-task CNN 251 are updated according to the result of the loss calculation (hereinafter, referred to as loss L2), and the single-task CNN 251 is updated according to the update setting.

[0132] In the training phase of the single-task CNN 251, the above-mentioned series of processes, including input of training anatomical area images 56L to the single-task CNN 251, output of training classes 258L from the single-task CNN 251, loss calculation, update setting, and updating of the single-task CNN 251, are repeated while the training data 275 is exchanged.

[0133] The update setting of the AE 250 and the update setting of the single-task CNN 251 are performed based on the total loss L shown in the following equation (2), where α is a weight. L = L1 × α + L2 × (1 - α) (2) That is, the total loss L is the weighted sum of the loss L1 of the AE 250 and the loss L2 of the single-task CNN 251.

[0134] As an example, as shown in Fig. 35, in the initial stage of the learning phase, the weight α is set to 1. When the weight α is set to 1, the total loss L = L1. Therefore, in this case, only the AE250 is learned, and the single-task CNN251 is not learned.

[0135] The weight α is gradually decreased from 1 as the learning progresses, and eventually becomes a fixed value (0.8 in FIG. 35). In this case, the learning of the AE250 and the learning of the single-task CNN251 are both performed with an intensity according to the weight α. In this way, the weight given to the loss L1 is greater than the weight given to the loss L2. In addition, the weight given to the loss L1 is gradually decreased from the maximum value of 1, and the weight given to the loss L2 is gradually increased from the minimum value of 0, and both are fixed values.

[0136] The learning of the AE250 and the single-task CNN251 is terminated when the restoration accuracy of the learning anatomical area image 56L to the learning restored image 256L by the AE250 reaches a predetermined set level, and the prediction accuracy of the learning class 258L for the correct class 258CA by the single-task CNN251 reaches a predetermined set level. The AE250 and the single-task CNN251 whose restoration accuracy and prediction accuracy have thus reached the set levels are stored in the storage 20 and used as the learned feature derivation model 252.

[0137] As an example, as shown in Fig. 36, the dementia finding derivation unit 280 of this embodiment inputs an aggregate feature group ZAG and dementia-related information 281 to a dementia finding derivation model 282. The aggregate feature group ZAG is composed of a plurality of aggregate features ZA output for each anatomical area. The dementia-related information 281 includes the sex, age, volume of the anatomical area, dementia test score, test results of a genetic test, test results of a cerebrospinal fluid test, test results of a blood test, etc. of a patient P to be diagnosed with dementia, similar to the dementia-related information 111 of the second embodiment.

[0138] The dementia finding derivation model 282 has a quantile normalization unit 283 and a linear discriminant analysis unit 284. The quantile normalization unit 283 receives the aggregate feature group ZAG and the dementia-related information 281. The quantile normalization unit 283 performs quantile normalization to convert the aggregate feature ZA constituting the aggregate feature group ZAG and each parameter of the dementia-related information 281 into data that follows a normal distribution in order to treat them on the same level. The linear discriminant analysis unit 284 performs linear discriminant analysis on the aggregate feature ZA after the quantile normalization process and each parameter of the dementia-related information 281, and outputs dementia finding information 285 as a result. The dementia finding information 285 is either that a patient P who currently has mild cognitive impairment will remain with mild cognitive impairment in two years, or that the patient will progress to Alzheimer's disease in two years. Note that the learning of the dementia findings derivation model 282 is the same as the learning of the dementia findings derivation model 112 shown in FIG. 24 except that the learning feature set group 59L is changed to the learning aggregate feature group ZAG, and therefore will not be illustrated or described.

[0139] In this way, in the sixth embodiment, the single-task CNN 251 that performs a main task such as outputting the class 258 and the AE 250 that has some commonality with the single-task CNN 251 and performs a subtask that is more general than the main task such as generating the restored image 256 are used as the feature derivation model 252. Then, the AE 250 and the single-task CNN 251 are trained simultaneously. For this reason, it is possible to output a more appropriate feature set 255 and aggregated feature ZA compared to a case where the AE 250 and the single-task CNN 251 are separately used, and as a result, it is possible to improve the prediction accuracy of the dementia finding information 285.

[0140] In the learning phase, the update setting is based on the total loss L, which is the weighted sum of the loss L1 of AE250 and the loss L2 of single-task CNN251. Therefore, by setting the weight α to an appropriate value, it is possible to focus on learning AE250, focus on learning single-task CNN251, or balance the learning of AE250 and single-task CNN251.

[0141] The weight given to the loss L1 is greater than the weight given to the loss L2. This allows the AE250 to always be trained in a focused manner. By always training the AE250 in a focused manner, the compression unit 253 can output a feature set 255 that better represents the shape and texture characteristics of the anatomical area, and as a result, the output unit 257 can output a more plausible aggregate feature ZA.

[0142] Also, the weight given to the loss L1 is gradually decreased from the maximum value, and the weight given to the loss L2 is gradually increased from the minimum value, and both are fixed after a predetermined number of learnings. This allows AE250 to be trained more intensively in the early learning stage. AE250 is responsible for the relatively simple subtask of generating the restored image 256. Therefore, by training AE250 more intensively in the early learning stage, it is possible to cause the compression unit 253 to output a feature set 255 that better represents the shape and texture features of the anatomical area in the early learning stage.

[0143] As an example, table 300 shown in Fig. 37 shows a performance comparison between Nos. 1 to 7 relating to the dementia progression prediction methods described in the following documents A, B, C, D, E, F, and G, and Nos. 8 and 9 relating to the dementia progression prediction method of this embodiment. Of the dementia progression prediction methods of this embodiment, No. 8 shows a case where only the aggregated feature group ZAG is input to the dementia findings derivation model 282, and the dementia-related information 281 is not input. In contrast, No. 9 shows a case where the aggregated feature group ZAG and the dementia-related information 281 are input to the dementia findings derivation model 282.

[0144] Literature A <Tam, A., Dansereau, C., Iturria-Medina, Y., Urchs, S., Orban, P., Sharmarke, H., Breitner, J., & Alzheimer’s Disease Neuroimaging Initiative., “A highly predictive signature of cognition and brain atrophy for progression to Alzheimer’s dementia.”, GigaScience, 8 (5), giz055 (2019).>

[0145] Literature B <Ledig, C., Schuh, A., Guerrero, R., Heckemann, R. A., & Rueckert, D., “Structural brain imaging in Alzheimer’s disease and mild cognitive impairment: biomarker analysis and shared morphometry database.”, Scientific reports, 8 (1), 11258 (2018).>

[0146] Document C <Lu, D., Popuri, K., Ding, G. W., Balachandar, R., & Beg, M. F., “Multimodal and multiscale deep neural networks for the early diagnosis of Alzheimer’s disease using structural MR and FDG-PET images.”, Scientific reports, 8 (1), 5697 (2018).>

[0147] Literature D <Basaia, S., Agosta, F., Wagner, L., Canu, E., Magnani, G., Santangelo, R., Filippi, M., Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks, NeuroImage: Clinical 21, 101645 (2019).>

[0148] Literature E <Nakagawa, T., Ishida, M., Naito, J., Nagai, A., Yamaguchi, S., Onoda, K., “Prediction of conversion to Alzheimer’s disease using deep survival analysis of MRI images”, Brain Communications, Vol. 2(1) (2020).>

[0149] Literature F <Lee, G., Nho, K., Kang, B., Sohn, K. A., & Kim, D., “Predicting Alzheimer’s disease progression using multi-modal deep learning approach.”, Scientific reports, 9 (1), 1952 (2019).>

[0150] Literature G <Goto, T., Wang, C., Li, Y., Tsuboshita, Y., Multi-modal deep learning for predicting progression of Alzheimer’s disease using bi-linear shake fusion, Proc. SPIE 11314, Medical Imaging (2020).>

[0151] The accuracy rates (Accuracy) of No. 8 and 9 are 0.84 and 0.90. In particular, the accuracy rate of No. 9, 0.90, is higher than the accuracy rates of No. 1 to 7. The AUC (Area Under the Curve) of No. 8 and 9 are 0.93 and 0.97. These values ​​are higher than No. 5 related to the dementia progression prediction method described in Document E. Therefore, it can be said that the dementia progression prediction method of this embodiment can predict the dementia progression with higher accuracy compared to the conventional dementia progression prediction methods described in Documents A to G.

[0152] The sensitivity of No. 8 and No. 9 is 0.85 and 0.91. These values ​​are higher than the sensitivity of any of No. 1 to No. 7. In particular, the sensitivity of No. 9, 0.91, is the highest among them. Therefore, it can be said that the method for predicting the progression of dementia of this embodiment can predict without missing that patient P, who currently has mild cognitive impairment, will progress to Alzheimer's disease after the prediction period, compared to the conventional methods for predicting the progression of dementia described in Documents A to G.

[0153] The specificity of No. 8 and 9 is 0.84 and 0.90. These values ​​are smaller than 0.97 of No. 1 related to the dementia progression prediction method described in document A, but are larger than other documents B, C, D, and F. Therefore, it can be said that the dementia progression prediction method of this embodiment can more accurately predict that patient P who currently has mild cognitive impairment will remain with mild cognitive impairment after the prediction period, compared to many other conventional dementia progression prediction methods.

[0154] In Table 300, ADNI in the learning image section is an abbreviation for "Alzheimer's Disease Neuroimaging Initiative." AIBL is an abbreviation for "Australian Imaging Biomarkers and Lifestyle Study of Ageing." J-ADNI is an abbreviation for "Japanese Alzheimer's Disease Neuroimaging Intiative." All of these refer to databases in which head MRI images 15 of patient P with Alzheimer's disease and the like are accumulated.

[0155] Instead of the single-task CNN 251, the multi-task CNN 150 of the fourth embodiment may be used.

[0156] The learning of AE90 shown in Fig. 17, the learning of dementia finding derivation model 39 shown in Fig. 18, the learning of dementia finding derivation model 112 shown in Fig. 24, the learning of single-task CNN 140 shown in Fig. 28, the learning of multi-task CNN 150 shown in Fig. 30, and the learning of AE250 and single-task CNN 251 shown in Fig. 34 may be performed in the diagnostic support device 12 or in a device other than the diagnostic support device 12. In addition, these learnings may be performed continuously after each model is stored in the storage 20 of the diagnostic support device 12.

[0157] The PACS server 11 may function as the diagnosis support device 12 .

[0158] The medical image is not limited to the illustrative head MRI image 15. It may be a PET (Positron Emission Tomography) image, a SPECT (Single Photon Emission Computed Tomography) image, a CT (Computed Tomography) image, an endoscopic image, an ultrasound image, or the like.

[0159] The organs are not limited to the brain, but may be the heart, lungs, liver, etc. In the case of the lungs, the right lungs S1 and S2 and the left lungs S1 and S2 are extracted as the anatomical regions. In the case of the liver, the right lobe, the left lobe, the gallbladder, etc. are extracted as the anatomical regions. In addition, the diseases are not limited to the dementia, but may be heart disease, pneumonia, liver dysfunction, etc.

[0160] In each of the above embodiments, for example, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the RW control unit 45, the normalization unit 46, the extraction unit 47, the feature derivation units 48 and 160, the dementia findings derivation units 49, 110, and 280, the contribution degree derivation units 50 and 120, and the display control units 51 and 123. As described above, the various processors include the CPU 22, which is a general-purpose processor that executes software (operation program 30) and functions as various processing units, as well as dedicated electric circuits that are processors having a circuit configuration designed specifically to execute specific processes, such as a programmable logic device (PLD) that is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0161] A single processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Also, multiple processing units may be composed of a single processor.

[0162] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System On Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0163] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0164] The technology of the present disclosure can be appropriately combined with the above-mentioned various embodiments and / or various modified examples. In addition, it is not limited to the above-mentioned embodiments, and various configurations can be adopted without departing from the gist of the technology. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program, in addition to a program.

[0165] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0166] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0167] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A processor; A memory connected to or embedded in the processor, The processor, Acquire medical images; Extracting a plurality of anatomical regions of an organ from the medical image; inputting the images of the plurality of anatomical regions into a plurality of feature derivation models prepared for each of the plurality of anatomical regions, and outputting a plurality of feature amounts for each of the plurality of anatomical regions from the feature derivation models; inputting the plurality of feature amounts output for each of the plurality of anatomical regions into a disease finding derivation model, and outputting disease findings from the disease finding derivation model; deriving a first contribution degree representing a degree of contribution of the findings to an output for each of the anatomical regions; presenting the findings and the derivation results of the first contribution for each anatomical region; Diagnostic aids.

2. The processor, The diagnosis support device according to claim 1 , wherein the derivation results are presented in descending order of the first contribution degree.

3. The processor, 3. The diagnosis support device according to claim 1, wherein disease-related information relating to the disease is input to the disease finding derivation model in addition to the plurality of feature amounts.

4. The disease-related information has a plurality of items, The processor, Derive a second contribution degree representing a degree of contribution of the findings to an output for each of the items; The diagnosis support device according to claim 3 , further comprising: a display that displays a result of deriving the second contribution degree for each of the items.

5. 5. The diagnostic support device according to claim 1, wherein the feature derivation model includes at least one of an autoencoder, a single-task convolutional neural network for classifying, and a multi-task convolutional neural network for classifying.

6. The processor, 6. The diagnosis support device according to claim 1, wherein an image of one of the anatomical regions is input to a plurality of different feature derivation models, and the feature is output from each of the plurality of feature derivation models.

7. 7. The diagnostic support device according to claim 1, wherein the disease finding derivation model is constructed by any one of a neural network, a support vector machine, and boosting.

8. The processor, 8. The diagnosis support device according to claim 1, further comprising: a normalization process for adjusting the acquired medical image to a standard medical image prior to the extraction of the anatomical region.

9. The diagnostic support device according to claim 1 , wherein the organ is a brain, and the disease is dementia.

10. The diagnostic support device according to claim 9 , wherein the plurality of anatomical regions includes at least one of the hippocampus and the anterior temporal lobe.

11. In the diagnosis support device according to claim 9 or 10, which cites claim 3 or 4, The disease-related information includes at least one of the volume of the anatomical area, a dementia test score, a genetic test result, a cerebrospinal fluid test result, and a blood test result.

12. Obtaining medical images; Extracting a plurality of anatomical regions of an organ from said medical image; inputting the images of the plurality of anatomical regions into a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and outputting a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models; inputting the plurality of feature amounts output for each of the plurality of anatomical regions into a disease finding derivation model, and outputting disease findings from the disease finding derivation model; Deriving a first contribution degree representing a degree of contribution of the findings to an output for each of the anatomical regions; and presenting the findings and the derivation result of the first contribution for each anatomical region; A method for operating a diagnostic support device comprising:

13. Obtaining medical images; Extracting a plurality of anatomical regions of an organ from said medical image; inputting the images of the plurality of anatomical regions into a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and outputting a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models; inputting the plurality of feature amounts output for each of the plurality of anatomical regions into a disease finding derivation model, and outputting disease findings from the disease finding derivation model; Deriving a first contribution degree representing a degree of contribution of the findings to an output for each of the anatomical regions; and presenting the findings and the derivation result of the first contribution for each anatomical region; An operating program for a diagnostic support device for causing a computer to execute a process including the steps of:

14. A computer including a processor and a memory connected to or embedded in the processor, Obtaining medical images of the brain; extracting a plurality of anatomical regions of the brain from the medical image; inputting the images of the plurality of anatomical regions into a plurality of feature amount derivation models prepared for each of the plurality of anatomical regions, and outputting a plurality of feature amounts for each of the plurality of anatomical regions from the feature amount derivation models; inputting the plurality of feature amounts output for each of the plurality of anatomical regions into a dementia findings derivation model, and outputting a dementia finding from the dementia findings derivation model; Deriving a first contribution degree representing a degree of contribution of the findings to an output for each of the anatomical regions; and presenting the findings and the derivation result of the first contribution for each anatomical region; A dementia diagnosis support method.

Images