System, control method and program

Abstract

Provided is a system for assisting in image evaluation for diagnostic imaging.　An assistance system (1) comprises: a first access unit (11) that can access a first image evaluation system for statistically evaluating a first type of medical image including at least a part of a subject's body as a target area; a second access unit (12) that can access a second image evaluation system for determining the subject's morbidity using a first model machine-trained to evaluate a first disease on the basis of the first type of medical image; and an assistance unit (30) that provides, through a common image evaluation environment (15), assistance in at least one of input of an evaluation target and output of an evaluation result to and from the first image evaluation system and the second image evaluation system.

Description

[Technical Field]

[0001] The present invention relates to a system for providing information to support image diagnosis. [Background technology]

[0002] The image processing device disclosed in Document 1 includes an input unit that inputs functional images of the subject's brain, an anatomical standardization unit that anatomically standardizes the functional images of the subject, an ROI candidate presentation unit that reads data of anatomical regions from a standard brain data storage unit that stores data of anatomical regions assigned to a standard brain and presents the data of the anatomical regions as ROI candidates, an ROI setting unit that accepts the selection of an anatomical region and sets an ROI on the anatomically standardized brain image of the subject based on the selected one or more anatomical regions, an evaluation value calculation unit that calculates an evaluation value based on pixel values ​​within the ROI, and a display unit that displays information related to the calculated evaluation value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-074343 Summary of the Invention [Problem to be solved by the invention]

[0004] Deep learning is expected to improve the accuracy of diagnostic aids using medical images. However, how the results and evaluations are calculated using deep learning remains a black box, making it difficult for medical professionals to interpret and explain the results. [Means for solving the problem]

[0005] One aspect of the present invention is a system including a first access unit accessible to a first image evaluation system that statistically evaluates a first type of medical image that includes at least a portion of a subject's body as a target region; a second access unit accessible to a second image evaluation system that determines the subject's disease state using a first model trained through machine learning to evaluate a first disease based on the first type of medical image; and a support unit that provides at least one of input of evaluation targets and output of evaluation results to the first and second image evaluation systems via a common image evaluation environment. This system enables evaluation of the machine-learned first model and evaluation of statistical processing via a common environment, thereby providing support to medical professionals and others in interpreting the machine-learned discrimination results of medical images. The support unit may use a common image evaluation environment that includes evaluation images standardized to images that can be input and output to the first and second image evaluation systems for the first type of medical image of the subject.

[0006] Another aspect of the present invention is a control method for an assistance system having the first access unit, the second access unit, and an assistance unit, the control method including at least one of the following steps: i) The support unit outputs a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system via a common image evaluation environment. ii) The support unit outputs the results of re-evaluating the first results regarding the evaluation of the medical images obtained from the first image evaluation system based on the disease state judgment of the second image evaluation system via a common image evaluation environment. iii) The support unit outputs, via a common image evaluation environment, a first area that is considered important based on a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second area that is considered important for determining the disease state of the second image evaluation system. iv) The support unit selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via a common image evaluation environment. v) The support unit selects an image area including a second area that is considered important for determining the disease state of the second image evaluation system as an evaluation target for the first image evaluation system via a common image evaluation environment. vi) The support unit controls the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment based on the reliability of the mapping of the medical image of the subject to the common image evaluation environment.

[0007] Another aspect of the present invention is a computer-implemented program for evaluating medical images. The program (program product) includes instructions for a computer to access a first image evaluation system that statistically evaluates a first type of medical image that includes at least a portion of a subject's body as a target region, access a second image evaluation system that determines the subject's disease state using a first model trained by machine learning to evaluate a first disease based on the first type of medical image, provide at least one of input of the evaluation target and output of the evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment, and execute at least one of steps i) to vi). The program may be provided recorded on a computer-readable recording medium. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an overview of an information provision system for supporting image diagnosis. [Figure 2] FIG. 1 is a diagram showing an overview of the processing of an evaluation support system. [Figure 3] Diagram showing an overview of the anatomical standardization process. [Figure 4] A diagram showing the standardization process. [Figure 5] FIG. 10 is a diagram showing an example of a region of interest when an anatomically standardized image is used as input for deep learning. [Figure 6] 1A-1C show different examples of anatomical standardization processing. [Figure 7] FIG. 10 is a diagram showing an example of input processing. [Figure 8] FIG. 10 is a diagram showing an example of input processing involving part extraction. [Figure 9] FIG. 10 is a diagram showing the attributes of subjects in an experimental example. [Figure 10] FIG. 1 shows the pulse sequence of an MRI scanner used in an experimental example. [Figure 11] FIG. 1 shows an example of the results of ROIs subjected to ANOVA analysis. [Figure 12] Graph showing ROIs for each group. [Figure 13] Graph showing the progression of atrophy in each group. [Figure 14] Diagram showing a deep learning model. [Figure 15] A graph showing changes in sensitivity and specificity. [Figure 16] FIG. 10 shows the reassessment of validation subjects. [Figure 17] Diagram showing ROI for deep learning models. [Figure 18] FIG. 10 is a diagram showing ROIs obtained by statistical processing. [Figure 19] A diagram showing the ROIs of the deep learning model and the ROIs of the statistical processing overlaid on each other. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 shows an overview of a system 1 that provides information for image diagnosis support. This system 1 provides information related to support for diagnosis using medical images that include the brain or a portion of the brain as a target region for image diagnosis or evaluation as part of the subject's body. The system 1 includes an image database 52 that stores brain images 53 of the subject as a first type of medical image that includes at least a portion of the subject's body as a target region; a first image evaluation system 60 that statistically evaluates the brain images 53; a second image evaluation system 70 that determines the subject's disease state using a first model trained by machine learning to evaluate a first disease based on the brain images 53; and an image evaluation support system 10. The image evaluation support system (support system) 10 has a first access unit (interface) 11 accessible to a first image evaluation system 60, a second access unit (interface) 12 accessible to a second image evaluation system 70, and a support unit 30 that provides at least one of inputs 62 and 72 of evaluation objects and outputs 63 and 73 of evaluation results to the first image evaluation system 60 and the second image evaluation system 70 via a common image evaluation environment.

[0010] Various types of tomography devices (modalities), such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), SPECT (Single Photon Emission Computed Tomography), and PET-CT, are known as devices for diagnosing the form and function of a subject's (test subject, examinee's) body or parts thereof, and these modality images (medical images) are used to diagnose various diseases. In particular, modality images (medical images) 53 that include the subject's brain as the target region for diagnosis or evaluation are used to obtain data related to the physical state of the subject's brain and are used to diagnose diseases such as dementia and Parkinson's disease.

[0011] Examples of types of medical images include CT and MRI, which can reflect highly accurate morphological information. MRI images include, for example, T1-weighted images, T2-weighted images, diffusion-weighted images, flare images, diffusion tensor images, QSM images, pseudo-PET images, and pseudo-SPECT images. QSM images (Quantitative Susceptibility Mapping) stand for quantitative susceptibility mapping. Other examples of types of medical images include PET and SPECT, which are generated by administering a radioactive drug into a subject's body via intravenous injection or the like and capturing radiation emitted from the drug within the body. Images using drugs allow doctors to understand not only the morphology of each part of the body but also how the administered drug is distributed within the body or the accumulation of substances in the body that react with the drug, thereby contributing to improved accuracy in diagnosing diseases. For example, a PET image can be captured using what is commonly known as Pittsburgh compound B as a PET radiopharmaceutical (tracer), and the degree of accumulation of amyloid beta protein in the brain can be measured based on the captured PET image, which can be useful for differential diagnosis or early diagnosis of Alzheimer's dementia. The pseudo-PET image is a term used to distinguish it from an actual PET image, and is an image that estimates the actual PET image. The pseudo-PET image may be generated based on, for example, an MRI image. Similarly, the pseudo-SPECT image is an image that estimates the actual SPECT image.

[0012] An example of a SPECT image is a SPECT scan using the radiopharmaceutical 123I-ioflupane, called DatSCAN (Dopamine transporter SCAN), which visualizes the distribution of dopamine transporters (DAT). The purposes of this imaging include early diagnosis of Parkinson's syndrome (PS) in Parkinson's disease (PD), diagnostic assistance for dementia with Lewy bodies (DLB), and determining the appropriate medication for striatal dopaminergic neuronal loss.

[0013] The system 1 includes a mapping system 55 that standardizes a first type of medical image 53 of a subject into an image (standardized image) 50 that can be input and output to a first image evaluation system 60 and a second image evaluation system 70. The assistance system 10 may include a third access unit (interface) 13 that can access the mapping system 55. An example of the standardized image 50 is an anatomical standardized image, and the mapping system 55 may function as an anatomical standardization processing unit. The mapping system 55 outputs a third result 56 regarding the evaluation of the standardization processing of the brain image 53, including the reliability of the processing at each voxel of the anatomical standardization. The assistance unit 30 may acquire the result 56 via the access unit 13. As described below, the assistance unit 30 includes a common image evaluation environment that provides an evaluation of the image based on the standardized image 50. The evaluation during mapping or the third result 56 regarding the evaluation can be output in the same environment as the mapping system 55 or can be used as input for processing in the assistance unit 30.

[0014] The assistance system 10 including the assistance unit 30 may provide users, such as medical professionals, with information for evaluating images, including standardized evaluation images, via a common image evaluation environment (common evaluation environment, user interface module, U / I module) 15. Users may access the assistance system 10 using access devices such as a display 16a and a touch panel 16b attached to the assistance system 10, or may access the assistance system 10 via the cloud (Internet) 17.

[0015] This system for providing information for image diagnosis support (image diagnosis support information providing system) 1 may be provided in a standalone configuration by a device (system) such as a server that has computer resources including a memory and a CPU, or a configuration area 8 including the image database 52, the first image evaluation system 60, and the second image evaluation system 70 may be provided via the cloud (Internet). Also, a configuration area 9 including the mapping system 55, storage of the standardized images 50, the input 62 and evaluation output 63 of the first image evaluation system 60, and the input 72 and evaluation output 73 of the second image evaluation system 70 may be provided via the cloud.

[0016] The first image evaluation system 60 for statistically evaluating medical images may be provided as a system equipped with computer resources, and may include a processor 61 for performing statistical processing and a database 65 storing libraries and programs for executing the statistical processing. The first image evaluation system 60 may output a first result 66 related to the evaluation of the medical image to be processed, i.e., the statistical evaluation. The evaluation result 66 may be output (displayed) based on the standardized image 50 by an evaluation output function (display unit) 63 of the evaluation system 60. The support unit 30 may acquire the result 66 via the access unit 11. The support unit 30 may use a common image evaluation environment (common evaluation environment) 15 that provides evaluation of the image based on the standardized image 50 to output the evaluation of the statistical processing or the first result 66 related to the evaluation in the same environment as the first image evaluation system 60, or use the result 66 as input for processing in the support unit 30. The first result 66 may include information about a first region (region of interest, ROI) that is considered important in the statistical processing.

[0017] The first image evaluation system 60 may include an input function (selection unit) 62 for selecting an image or a region in the image to be subjected to statistical processing based on the standardized image 50. Control of the analysis target (input control information) 67, including the selection of the region to be subjected to statistical processing, may be provided by the support unit 30. The support unit 30 can use the common evaluation environment 15 to input or select information to be subjected to statistical processing based on the standardized image 50 through the same environment as the first image evaluation system 60. Processing such as correcting each voxel data of the standardized processed image 50 for covariates such as age or sex (or various biomarker values) as further input may also be performed via the common evaluation environment 15.

[0018] The second image evaluation system 70 for determining a disease state based on medical images may be provided as a system equipped with computer resources, and may include a processor 71 for processing using a learning model, and a database 75 storing a first model 74 trained by machine learning to evaluate a disease state based on medical images, a library, and the like. The second image evaluation system 70 may output a second result 76 regarding the evaluation of the medical image to be processed, i.e., the determination of the disease state of the subject using the first model 74 trained by machine learning to evaluate a first disease based on medical images. The evaluation result 76 may be output (displayed) based on the standardized image 50 by an evaluation output function (display unit) 73 of the evaluation system 70. The support unit 30 may acquire the result 76 via the access unit 12. The support unit 30 may use a common evaluation environment 15 based on the standardized image 50 to output the evaluation regarding the disease state determination or the second result 76 regarding the evaluation via the same environment as the second image evaluation system 70, or use it as input for processing in the support unit 30.

[0019] In the case of dementia, a second image evaluation system 70 can be employed that uses a learning model 74 that has been machine-learned to differentiate between the causative diseases AD (Alzheimer's Disease) and DLB (Dementia with Lewy Bodies). The learning model 74 can be used to output a second result 76 regarding the subject's disease status. The second result 76 may include information on the presence or absence and progression of the first disease of interest, such as AD or DLB, as well as information on a second region (ROI) that is important for determining the disease status. Using a deep learning method such as GradCAM, it is possible to visualize the region of interest (ROI) used in deep learning differentiation. Using standardized images 50 via a common evaluation environment 15, the support unit 30 can evaluate the ROI in a shared environment with the second evaluation system 70.

[0020] The second image evaluation system 70 may include an input function (selection unit) 72 for selecting an image or a region in the image to be subjected to disease state assessment by the learning model 74 based on the standardized image 50. Control (input control information) 77 of the diagnostic target (differentiation target), including selection of the region to be processed, may be provided by the support unit 30. The support unit 30 can use the common evaluation environment 15 to input or select information to be subjected to the differentiation process of the learning model 74 based on the standardized image 50 in the same environment as the second image evaluation system 70. In this case, the common evaluation environment 15 may also be used to perform a process of correcting the covariates of age or sex (or various biomarker values) as input to each voxel data of the standardized processed image 50.

[0021] The assistance system 10 may be provided as a device equipped with computer resources such as a server accessible to the cloud, and may include a database 18 storing libraries required for various processes and a program 19 containing instructions for executing processes as the assistance system. Services to users using the common evaluation environment 15 may be provided as a cloud-based service (Software as a Service (SaaS)). As described above, the assistance system 10 provides the common evaluation environment 15 using the standardized processed image 50, and the assistance unit 30 can use the common evaluation environment 15 to seamlessly provide the user with the evaluation results 66 from the first image evaluation system 60 and the evaluation results 76 from the second image evaluation system 70 in a cross-referenceable state. The results 56 from the standardization process can also be provided to the user.

[0022] Deep learning is expected to improve the accuracy of diagnostic support using medical images. For neurodegenerative diseases such as Alzheimer's disease, a method is used to calculate regions of interest (ROIs) for each imaging modality (e.g., morphological MRI, blood flow SPECT, PET images) that statistically show significant differences in brain volume, blood flow (glucose metabolism), and the amount of disease-causing substances such as amyloid beta between healthy and diseased groups, and then evaluate values ​​such as Z-score and SUV within those regions. Many methods using deep learning for neurodegenerative diseases have been reported to improve diagnostic accuracy, but the way in which the diagnostic results and evaluations are calculated remains a black box, making them difficult for medical professionals to interpret and explain. There are two main reasons for this: 1) Neurodegenerative diseases require determining which part of the brain is altered. (For example, Alzheimer's disease may be suspected due to atrophy of the parahippocampal gyrus.) 2) The output values ​​of deep learning (including intermediate layers) are processed into information that cannot be interpreted by humans, such as brain volume or blood flow.

[0023] The support system 10 is capable of solving the two problems mentioned above for the results obtained using deep learning, and can provide interpretations and explanations of the results to medical professionals who handle the results of deep learning.

[0024] Regarding the above problem 1), a method using deep learning GradCAM etc. makes it possible to visualize the region of interest used in deep learning discrimination. Furthermore, if the input for discrimination by deep learning is a brain image mapped to an anatomical standard brain 50 provided in a common image environment 15 using the support system 10, it is possible to visually confirm the ROI and the statistically calculated ROI on the same image in comparison (by switching the display or overlaying).

[0025] Regarding issue 2), it is possible to specify the area of ​​interest for deep learning using a common image environment 15, perform statistical processing within that area, calculate values ​​for brain volume and blood flow, and present human-interpretable index values ​​for areas that are effective for differentiation.

[0026] This support system 10 can also serve as a tool function (system-side) that can be considered for other business purposes. For example, it can be applied to responses based on analysis results, allowing for the selection of intervention methods, and if possible, the selection of medication prescriptions and guidance on additional testing. In the case of DLB, it can provide support by recommending DatSCAN / MIBG myocardial scintigraphy and recommending testing institutions. By linking to a research paper database, it is also possible to provide support for research, diagnosis, and treatment by displaying links to related papers when the mouse is placed over the ROI region displayed in the common image environment 15.

[0027] In the assistance system 10, the assistance unit 30 can provide several functions using the common image environment 15. One function is an input assistance function (input assistance unit) 37, which can provide the following functions by supplying input control information 67 and 77 to the first image evaluation system 60 and / or the second image evaluation control system 70 via the assistance unit 30.

[0028] After the brain image 53 is subjected to anatomical standardization processing in the mapping system 55, when the anatomically standardized image 50 is used as input to the deep learning model 74 of the second image evaluation system 70, the second image evaluation system 70 is controlled to determine whether it is a brain-related disease (assuming a disease that can be diagnosed from a brain image) or to classify the type, progression, etc.

[0029] Each voxel data of the anatomically standardized processed image 50 is further corrected for covariates such as age or sex (or various biomarker values) as input, and then selected as input for the deep learning model 74.

[0030] When predicting the class of a deep learning model74, the model has the option to display a value between 0 and 1 for each class ultimately evaluated by the softmax function. Classes can be classified according to disease, for example, in the case of dementia, a brain disease, classes such as NC (Normal Control), AD, and DLB can be envisioned.

[0031] In the input support unit 37, if the first disease to be differentiated includes Alzheimer's disease (AD) and dementia with Lewy bodies (DLB), and if the first type medical image 53 is an MR image, the input control information 67 and 77 may be set to include, as the target region, at least one of the hippocampus, parahippocampal gyrus, dorsal brainstem, middle temporal pole, and basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala, etc.). If the first type medical image 53 is a SPECT image, the input control information 67 and 77 may be set to include, as the target region, at least one of the precuneus, occipital lobe, and dorsolateral prefrontal cortex.

[0032] The support unit 30 may include a function (individual / comparative evaluation unit) 31 that outputs a first result 66 regarding the evaluation of the medical image acquired from the first image evaluation system 60 and a second result 76 regarding the evaluation of the medical image of the subject acquired from the second image evaluation system 70 via a common image evaluation environment 15. Using the common image environment 15, the evaluation (first result) 66 obtained by statistically processing the medical image and the evaluation (second result) 76 predicted from the medical image by a deep learning model 74 may be output independently or in a comparable state, for example, in parallel or in a switched manner, via the standardized image 50.

[0033] The support unit 30 may include a function (re-statistical processing request unit) 32 for outputting the result of re-evaluating the first result 66 regarding the evaluation of the medical image acquired from the first image evaluation system 60 based on the disease state determination of the second image evaluation system 70 via the common image evaluation environment 15. Even if no region of interest (ROI) related to a disease is found in the result of the preceding statistical processing, it can be re-recognized as a disease ROI by re-evaluating the subtle differences in the statistical processing of the image of the subject determined to have the disease based on the disease state determination of the deep learning model 74.

[0034] The support unit 30 may include a function (overlay unit) 33 for overlaying a first region of interest (ROI) that is considered important in a first result 66 regarding the evaluation of a medical image obtained from the first image evaluation system 60 and a second region of interest (ROI) that is considered important for the disease state determination by the second image evaluation system 70 onto the common image evaluation environment 15. Information about the ROIs may be acquired as part of the information 66 and 76 regarding the evaluation results from each system 60 and 70. When the deep learning model 74 predicts a disease class, the regions that the model 74 considered important for the evaluation can be displayed on the anatomically standardized brain image 50. By overlaying the ROIs that the deep learning model 74 focused on for the disease state determination and the ROIs used in statistical processing on the standardized image 50, interpretations and explanations of the results of the deep learning model 74 can be provided to medical professionals handling the results.

[0035] The support unit 30 may include a function (model input selection unit) 34 for selecting an image region including a first region of interest (ROI) that is considered important by a first result 66 regarding the evaluation of the medical image acquired from the first image evaluation system 60 as an evaluation target for the second image evaluation system 70 via the common image evaluation environment 15. One or more anatomical regions may be further input as regions of interest (ROI) using the standardized image 50, and an image of the anatomically standardized processed image 50 filtered by the anatomical regions may be selected as an input for the deep learning model 74. When the anatomically standardized processed image 50 is used, anatomical regions are defined on the standardized brain coordinate system, and a "region of interest" that is a set of coordinates with the same name can be selected for the first image evaluation system 60 and the second image evaluation system 70.

[0036] The support unit 30 may include a function (statistical processing input selection unit) 35 for selecting an image region including a second region of interest (ROI) that is considered important for the second image evaluation system 70 to determine the disease state as an evaluation target for the first image evaluation system 60 via the common image evaluation environment 15. By having the deep learning 74 select an ROI that focuses on determining the disease state and perform statistical processing, it is possible to provide interpretations and explanations of the results of the deep learning 74 to medical professionals who handle them.

[0037] The support unit 30 may include a function (mapping evaluation unit) 36 that controls the output of a first result 66 regarding the evaluation of a medical image acquired from a first image evaluation system 60 or a second result 76 regarding the evaluation of a medical image of a subject acquired from a second image evaluation system 70 to a standardized image 50 using the common image evaluation environment 15 based on the reliability of mapping of the subject's medical image to a standardized image 50 in this example. When a deep learning model 74 using an anatomically standardized processed image 50 as input predicts a disease class, the model 74 can display the results of reliability correction of each voxel value of the region (ROI) used in the evaluation. Problems with the accuracy of mapping to the standard brain image 50 may result in problems with the reliability of analysis results using regions with low mapping accuracy as ROIs. By quantifying the mapping accuracy before analyzing the brain image (extracting ROIs and performing differentiation), an ROI filter can be applied to obtain an evaluation that does not use image regions with low reliability.

[0038] The assistance system 10 may further include a unit 20 that verifies the assessment of a first disease, such as AD or DLB, by the learning model (first model) 74 based on the output of these assessment results using the common image environment 15 of the assistance unit 30.

[0039] 2 is a flowchart showing an evaluation support method using the image evaluation support system 10. The evaluation support system 10 can be provided as an information processing device equipped with computer resources including a memory and a CPU, and this support method can be provided as a control method for the system 10 or as a program having instructions executable on a computer. The program (program product) may be provided by being recorded on a computer-readable recording medium, or may be provided in a downloadable form from the Internet, etc.

[0040] In step 81, a mapping system 55 maps a brain image 53 of a subject (user, examinee) to a standardized image (anatomical standardized image) 50. An overview of the anatomical standardization process is shown in Figure 3. The standardization process is also shown in Figure 4.

[0041] Anatomical standardization involves mapping an individual's functional image onto a standard template, either linearly or nonlinearly. Anatomical standardization has become a standard method for aligning the positions of brain regions between subjects when analyzing medical images, and even brain images. This technique was originally required for creating activation maps from PET imaging, but the basic idea is to set a standard brain and then linearly or nonlinearly align the input image to the standard brain.

[0042] For PET, fMRI, and MRI, one form of this processing is provided as a tool called VBM (Voxel-based morphometry), but it is also implemented in other tools such as 3D-SSP. It is now common to perform nonlinear transformations when analyzing MRI images, but originally a method based on diffeomorphic mapping called LDDMM was devised, and the DARTEL method, which was devised as an improved method due to the long calculation time, is often used.

[0043] In analyses after such anatomical standardization, it can be assumed that the data represent the same location on the brain, allowing comparisons of coordinate values ​​(volume values ​​in the case of MRI) between groups. Another advantage is that the analysis can absorb differences in imaging conditions, etc. Furthermore, the ability to refine the complexity of the information contained in the data to the granularity required for analysis offers the advantage of being a reasonable preprocessing step in situations such as clinical research where large amounts of data are involved and deep learning cannot be trained on all data processing processes.

[0044] In step 811 of Figure 3, horizontal axis adjustment (ACPC transformation) is performed on all images (Figure 4(a)). Furthermore, in step 812, the images of each subject are segmented into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) (Figure 4(b)). In step 813, the images segmented into gray matter images are subjected to DARTEL processing using a template created from a standard brain only (Figure 4(c)). In step 814, further standardization to MNI space is performed (Figure 4(d)). After a series of nonlinear registration steps to MNI space coordinates, the gray matter images may be further subjected to modulation processing to shape the volume information and smoothing processing using an 8-mm wide Gaussian kernel. Note that the smoothing width is not limited to 8 mm.

[0045] Deep learning allows for the display of regions of interest on input images for specific models during differential evaluation. On the other hand, if anatomical standardization is performed, the same coordinate space is used for statistical analysis, making it easy to draw side-by-side images.

[0046] Figure 5 shows an example of a visualization technique called SHAP, where anatomically standardized morphological MRI images were used as input for deep learning. For example, in addition to visualizing the regions heavily used in deep learning as mentioned above, we can also consider a method of extracting regions of interest from validation samples by using statistical analysis to improve accuracy (or sensitivity or specificity) when conducting accuracy comparison experiments with conventional methods for determining statistical cutoff values.

[0047] Returning to FIG. 2 , in step 82, a result 56 related to the reliability evaluation of mapping to the standardized image is obtained. FIG. 6 shows the process of performing anatomical standardization using a deep learning model in a mapping system 55. Instead of steps 811 to 814, in step 815, anatomical standardization is performed using a deep learning model. Deep learning may be applied after anatomical standardization, or it may be applied to the anatomical standardization process. Anatomical standardization may use LDDMM for nonlinear transformation, but this may result in long processing times. This process can be trained using a neural network, introducing a method that is faster and more accurate than DARTEL. Furthermore, in step 816, the reliability of the mapping may be calculated using a Bayesian neural network while also increasing the speed. This reliability evaluation result 56 may be used to correct the regions of interest (ROIs) in the first image evaluation system 60 and the second image evaluation system 70.

[0048] Returning to Figure 2, in step 83, the support unit 30 may use the common image environment 15 to provide input control support to the first image evaluation system 60 and the second image evaluation system 70. Attributes and biomarkers such as age and gender may be further input to the anatomically standardized processed image 50, and the corrected anatomically standardized processed image may be used for subsequent processing. Figure 7 shows the flow of processing when attributes and biomarkers are further input and used.

[0049] In the input / output support in step 83, the common image environment 15 may be used to provide input control support for the second image evaluation system 70. In the second image evaluation system 70, a choice may be made between directly determining the deep learning model and performing filtering by anatomical region before determining the deep learning model. Figure 8 shows the processing flow for region extraction.

[0050] Returning to Figure 2, in step 84, when the support unit 30 needs to obtain the evaluation results of the statistical processing, in step 85, it obtains the first evaluation results 66 regarding the statistical processing of the image of the first image evaluation system 60 via the first access unit (interface) 11 that is accessible to the first image evaluation system 60.

[0051] After anatomical standardization, statistical tests are used to compare groups and identify coordinate positions with significant differences. In clinical research, there are often differences in age range and gender between groups, so the coordinate values ​​are adjusted for these as covariates before analysis. Generalized linear models are often used. This processing can be performed on anatomically standardized data before inputting it into deep learning, which motivates the system to have an input format for attribute values ​​and biomarkers.

[0052] Statistical processing for evaluating medical images involves statistically comparing brain images of subjects with those of healthy individuals. A known method for evaluating brain atrophy using brain images is VBM (Voxel Based Morphometry), which processes brain images acquired by imaging the subject's head in units of voxels, which are three-dimensional pixels. A typical statistical processing technique is to generate a Z-score map.

[0053] Taking MR images as an example, the average and standard deviation are calculated for each voxel from the MR images of a normal case that has undergone brain morphology standardization processing, and the average and standard deviation images are created by substituting the values ​​of the data (normal standard brain) and the values ​​of the subject's image data (processed image) into the following formula, which calculates the Z score. z=(M(x,y,z)-I(x,y,z)) / SD(x,y,z) M and SD represent the mean and standard deviation images of a normal control brain, respectively, and I represents the processed image. Using the Z-score map, it is possible to quantitatively analyze which regions of the processed image have undergone changes compared to a normal control brain. For example, voxels with positive Z-score values ​​indicate areas of atrophy compared to a normal control brain, and the higher the value, the greater the statistical deviation. For example, a Z-score of 2 indicates a region that is more than two standard deviations from the mean, which is considered to have a statistically significant difference with a risk of approximately 5%. To quantitatively evaluate atrophy within a region, M, SD, and I are calculated for the region of interest, respectively, and the average of all positive Z-scores can be calculated.

[0054] Various statistical methods have been proposed, including comparing the volume or area of ​​each brain region and using a t-test with a general linear model (GLM). Using the so-called Pittsburgh compound B as a PET radiopharmaceutical (tracer), measuring the degree of amyloid beta protein accumulation in the brain based on PET images can be useful for the differential diagnosis or early diagnosis of Alzheimer's disease. These PET images can be used for statistical analysis to obtain the SUVR value (Standardized Uptake Value Ratio, or cerebellar ratio SUVR), which indicates the ratio of the sum of the amyloid beta protein accumulation in the cerebral gray matter (SUV) to the amyloid beta protein accumulation in the cerebellum (SUV). SUVR can be defined as follows:

[0055]

number

[0056] The numerator of this equation represents the sum of the SUVs of four cerebral gray matter regions, namely the cortical regions of the cerebral cortex (prefrontal cortex, anterior and posterior cingulate cortex, parietal lobe, and lateral temporal lobe), and the denominator represents the SUV of the cerebellum.

[0057] For statistical processing of DatSCAN using SPECT images, BR (Binding Ratio) can be used as an evaluation (index value), and is expressed by the following formula.

[0058]

number

[0059] In the formula, C is the average value of DAT within each region of interest, Cspecific represents the average value of the putamen and caudate nucleus within the brain, and Cnonspecific represents the average value of the occipital cortex within the brain.

[0060] Visualization of regions of interest (ROIs) that show significant differences based on statistical tests on anatomically standardized brain images is also performed in individual image evaluation systems, and tools such as SPM (Statistical Parametric Mapping) are well known.

[0061] As an example, we will describe a study in which 208 subjects participated (101 in the DLB group, 69 in the AD group, and 38 in the healthy control group). Each disease group was determined by two certified specialists from the Japan Dementia Society to have DLB or AD as the primary neurological disorder according to the DSM-5 diagnostic criteria. To exclude vascular disorders, subjects were recruited under the condition of excluding subjects with progressive and acute white matter lesions. This group excluded subjects from the healthy control group who were not diagnosed with dementia and were suspected of having a central nervous system disorder. This study was approved by the ethics committee and conducted in accordance with the guidelines of the participating institutions.

[0062] Figure 9 shows the demographics of the subjects. The DLB group consisted of 50 women and 51 men, with a mean and standard deviation of their ages being 73.25 ± 8.05 years. The AD group consisted of 36 women and 33 men, also with a mean and standard deviation of their ages being 71.58 ± 6.33 years. The NC group consisted of 28 women and 10 men, also with a mean and standard deviation of their ages being 71.03 ± 6.28 years. There were no significant differences in age or gender between any of the subject groups. All subjects underwent the MMSE test, and the scores for the DLB, AD, and NC groups were 22.21 ± 4.86, 21.32 ± 3.95, and 28.21 ± 1.26, respectively, indicating no significant differences between the DLB and AD groups.

[0063] The MRI data of the subjects were acquired using a total of 11 different scanners. The images were 3D T1-weighted images acquired using gapless sagittal imaging, and the pulse sequence of each MRI scanner is shown in Figure 10. Each MRI image was converted into an anatomical standardized image 50 using the processing described above.

[0064] Statistical analysis using SPM was performed on the three groups (NC, DLB, and AD) using ANOVA normalized for ICV (intracranial volume). The MNI (Montreal Neurological Institute) coordinates for which significant differences in gray matter volume were determined between groups are shown in Figure 11. The area names for the MNI coordinate positions are from the WFUPickatlas (Department of Radiology of Wake Forest University School of Medicine, Winston-Salem, North Carolina; fmri.wfubmc.edu).

[0065] Figure 12 shows the ROIs for the three groups after ICV normalization. Figure 12 shows the results of ICV normalization. Significant differences in gray matter volume were confirmed over a relatively wide area among the three groups, with the most significant difference occurring in the region extending from the parahippocampal gyrus to the brainstem.

[0066] The results of evaluating the effect within the ROI of each group after ICV normalization with 90% CI are shown in Figure 13. No significant difference was observed between the DLB group and the AD group.

[0067] Returning to Figure 2, in step 86, when the support unit 30 needs to obtain a second result 76 regarding the evaluation of the medical image from a second image evaluation system 70 that determines the subject's disease status using a first model (deep learning model) 74 that has been machine-learned to evaluate a first disease, such as AD or DLB, based on the medical image, in step 87, the support unit 30 obtains the second evaluation result 76 of the second image evaluation system 70 via a second access unit (interface) 13 that is accessible to the second image evaluation system 70.

[0068] A machine learning model based on medical image information (learning model) is used to differentiate diseases from a subject's medical images. Iizuka, Tomomichi et al., "Deep-learning-based imaging-classification identified cingulate island sign in dementia with Lewy bodies" (Scientific Reports 9.1 (2019): 1-9.), reported that an even higher accuracy of 89.32% was achieved in an experiment using a convolutional neural network on perfusion SPECT images, and that deep learning focused on blood flow findings in the occipital lobe, which had traditionally been used in image interpretation, for the purpose of differentiation. Litjens, Geert et al., "A survey on deep learning in medical image analysis." (Medical image analysis 42 (2017): 60-88.) and Wen, Junhao et al., "Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation." (CoRR abs / 1904.07773 (2019)) report that the application of recent deep learning techniques has demonstrated high accuracy in the diagnosis of AD.

[0069] However, no research has focused on whether deep learning can differentiate between DLB and AD, and there are no known studies that have applied deep learning to the differentiation of DLB ​​and AD using morphological MRI images. Therefore, as part of the above research, we investigated a differentiation approach using deep learning on morphological MRI images.

[0070] Figure 14 shows an overview of the model used. In this study, we adopted a ResNet-type neural network model adapted for 3D data. ResNet is a type of convolutional neural network model and, compared to conventional convolutional neural networks, has a model structure that prevents the loss of signal source features. Furthermore, by adding a mechanism called a "skip connection" to the convolutional layer, ResNet can adopt a mechanism that transmits the output of the convolution to the next layer together with the input of the current layer. This makes it possible to prevent information loss in the training data even when the model layers are deep, achieving high accuracy in many image classification tasks. In this study, we adopted an implementation using the PyTorch framework, and we previously tested network depths of 18, 34, 50, and 101 layers before settling on 34 layers. Training was performed using one NVIDIA Tesla K80 GPU, and optimization was performed using SGD (Stochastic Steepest Descent) with an inertia term.

[0071] Figure 15 shows the ROC plot of the model with the highest validation accuracy for each 5-fold, comparing the sensitivity and specificity while varying the threshold for disease detection in the softmax output layer. When balanced with the Youden Index, the sensitivity and specificity were 81.54 ± 10.43% and 76.77%, respectively, with an accuracy of 79.15 ± 5.22% (sensitivity and accuracy are 5-fold mean ± SD with fixed specificity). Although limited, this experiment confirmed that the deep learning model, using the same gray matter volume data, was capable of distinguishing between DLB and AD groups, where no significant difference could be confirmed using conventional SPM statistical tests.

[0072] The results of the preliminary validation experiment showed that the areas with significant differences in gray matter between the three groups (healthy, DLB, and AD) were the right middle temporal pole and left parahippocampal gyrus. There are precedents for detecting brainstem regions in ICV analysis, and other studies have confirmed significant differences between DLB and AD groups in the white matter dorsal to the brainstem. However, in our preliminary validation, the difference in the degree of atrophy of the gray matter, including the hippocampus and brainstem, between the DLB and AD groups was small compared to the intracranial volume.

[0073] The ICV analysis revealed a significant difference in the parahippocampal gyrus. Hippocampal atrophy is typically seen as a sign of AD progression, but in this study, atrophy was observed in both the DLB and AD groups, suggesting that atrophy was more advanced in the AD group than in the DLB group. Other studies have used a cascade approach, first assessing parahippocampal gyrus atrophy to differentiate between healthy and dementia groups, and then assessing atrophy in the dorsal brainstem to differentiate between DLB and AD. Our preliminary validation results suggest that using the parahippocampal gyrus may be more effective for differentiating between DLB and AD groups. Therefore, we performed a t-test between the two independent groups, DLB and AD, but no significant differences were detected in either the hippocampus or the dorsal brainstem. These findings suggest that the gray matter atrophy patterns between DLB and AD groups are subtler than those detected by conventional methods based on statistical significance.

[0074] In contrast, our experimental results using deep learning on VBM-based data demonstrated a consistent ability to differentiate between DLB and AD. For example, when using the deep learning model of this embodiment, we achieved a 35.79% improvement in sensitivity and a 15.70% improvement in accuracy compared to VBM-based methods. This demonstrates the feasibility of using a deep learning model like this one to differentiate between DLB and AD. Furthermore, in the VBM preprocessing step we employed, the deep learning model was able to assess minute volumetric differences more sensitively than statistical analysis under standard significance criteria. However, considering clinical applications, applying such very small differences to diagnosis carries risks. Therefore, comprehensive judgments, including test data other than morphological MRI, are required. Furthermore, it is necessary to further evaluate the generalization performance of this method by increasing the number of validation facilities.

[0075] For example, the atrophy patterns between the DLB and AD groups can be subtle, but the deep learning model demonstrates consistent discrimination performance. While these results suggest that deep learning may be able to handle subtle features that are effective for discrimination, it is important to analyze and evaluate the regions of interest that are effective for discrimination, for example by visualizing the features evaluated by deep learning. The support unit 30 provides a support environment for image evaluation in such cases.

[0076] Returning to Figure 2, in step 88a, when a request is made to display the evaluation results of each system 60 and 70, in step 88b, the display and comparison unit 31 of the support unit 30 provides the user with a first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 and a second result 76 regarding the evaluation of the subject's medical image obtained from the second image evaluation system 70 via the common image evaluation environment 15.

[0077] In step 89a, when a review of statistical processing is requested based on the discrimination result of the learning model 74, in step 89b, the re-statistical processing request unit 32 of the support unit 30 requests the first image evaluation system 60 via the input control information 67 to re-evaluate the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 based on the judgment of the disease state of the second image evaluation system 70, and outputs the result 66 via the common image evaluation environment 15.

[0078] For example, in the previous experiment, there were cases where the existing method gave incorrect results but the deep learning model 74 correctly identified five DLB cases. For these cases, a test was conducted using age and gender as covariates, including adjustments using a general linear model, and a control group with a predetermined or required number of healthy subjects. No significant differences were confirmed at p<0.05 using the FWE multiple comparison correction, but further testing without multiple comparison correction at p<0.001 revealed some significant differences.

[0079] Figure 16 shows the SPM statistical test results for five DLB-verified subjects, the areas with the lowest p-values, and the softmax output values ​​(0-1) that served as the basis for the deep learning model's 74 judgment for each subject. The gray matter areas with the highest p-values ​​were the putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, and amygdala for each subject, respectively. The striatum, like the putamen and caudate nucleus, is a region where dopaminergic neuronal degeneration is observed in dementia with Lewy bodies and Parkinson's disease. Atrophy around the hippocampus was observed in subjects (c) and (d). While atrophy around the hippocampus is often observed in AD cases, in these cases, the evaluation using the softmax function output values ​​of the proposed method provided relatively low support for DLB diagnosis, albeit with only slight differences. Atrophy was observed in the amygdala in subject (e), which is also a region where α-synuclein accumulation has been reported in cortical Lewy bodies. While existing methods define an ROI limited to the dorsal brainstem region and perform an evaluation, the method using learning model 74 is able to capture the necessary features better than existing methods in Lewy body dementia, where pathological mechanisms affect a wide range of areas, and this may have contributed to improved accuracy.

[0080] In the case of differentiating AD / DLB using morphological MRI images, it is necessary to capture very subtle changes between the AD / DLB groups, and statistical processing was unable to extract ROIs with significant differences. In contrast, the deep learning model 74 achieved a certain level of discrimination accuracy, and although the volume values ​​within the region of interest at this time were not significantly different, they can be considered an effective indicator for differentiating between groups. In particular, since VBM only leaves volume information, if the deep learning model 74 does not consider the interrelationships with distant areas, it can provide clinical significance as a UI that displays these subtle changes as indicators, which are discarded and not shown as results in conventional statistical analysis tools.

[0081] Returning to Figure 2, in step 90a, when there is a request to overlay the ROI of the statistical processing and the ROI of the deep learning model 74, in step 90b, the overlay display unit 33 of the support unit 30 outputs, via the common image evaluation environment 15, a first region (ROI) that is considered important by the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 and a second region (ROI) that is considered important for determining the disease state by the second image evaluation system 70.

[0082] Figure 17 shows an example of using GradCAM with a deep learning model 74 to output a region of interest (ROI) used in the deep learning model 74's discrimination, and displaying it on an anatomical standardized image 101. This screen 100 displays the GradCAM output using a sagittal section 102, a coronal section 103, and a horizontal section 104 of the anatomically standardized brain. Screen 100 also displays the volumes 105 of GM (gray matter), WM (white matter), TBV, and ICV, along with the average values ​​for healthy subjects (shown in parentheses). Clinical information 108 and DLB confidence score 106 of the subject are also displayed.

[0083] In this example, the only check item for "Select Region" is "Gray Matter," but multiple regions can be selected, not just gray matter. It may also be possible to select more specific regions within the gray matter.

[0084] Figure 18 shows an example of the results of statistical processing of brain images 53 from the same subject. In this display 110, regions of interest (ROIs) of Z-scores are shown using sagittal 112, coronal 113, and horizontal 114 slices of an anatomical standardized image 111.

[0085] FIG. 19 shows how a first evaluation result, including a region of interest in a statistical evaluation of a medical image of a first subject obtained from a first image evaluation system, and a second evaluation result, including a region of interest identified by a second image evaluation system when determining the disease state of the first subject, are overlaid and output in a common image evaluation environment. For example, the ROI of a deep learning model 74 obtained by GradCAM and the ROI of a Z-score are overlaid. In this overlay display 120, the ROIs of both models are overlaid on the sagittal section 122, coronal section 123, and horizontal section 124 of an anatomical standardized image 121. In this way, if the input for the differentiation by the deep learning model 74 is a brain image mapped to an anatomical standard brain, it becomes possible to visually compare the ROIs with conventional statistically calculated ROIs on the same image.

[0086] Returning to Figure 2, in step 91a, when processing of the learning model 74 is requested based on the ROI of statistical processing, in step 91b, the model input selection unit 34 of the support unit 30 selects an image area including a first area (ROI) that is considered important by the first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 via the common image evaluation environment 15, and provides it as an evaluation target for the second image evaluation system 70 via input control information 77.

[0087] In step 92a, if statistical processing is selected based on the ROI of the learning model 74, in step 92b, the statistical processing input selection unit 35 of the support unit 30 selects an image region including a second region (ROI) that is considered important for the second image evaluation system 70 to determine the disease state via the common image evaluation environment 15, and provides it as an evaluation target for the first image evaluation system 60 via input control information 67. The first image evaluation system 60 calculates values ​​of brain volume and blood flow within the region of interest of the deep learning model, and can present human-interpretable index values ​​for regions that are effective for differentiation. Statistical processing is not limited to Z scores, and may also include volume values, volume density values, blood flow rates, glucose metabolism rates, and accumulation amounts of tracer reactants.

[0088] If in step 93a it is requested to take into account an evaluation of the reliability of the mapping, in step 93b the mapping evaluation unit 36 ​​of the support unit 30 controls the output of a first result 66 regarding the evaluation of the medical image obtained from the first image evaluation system 60 or a second result 76 regarding the evaluation of the medical image of the subject obtained from the second image evaluation system 70 using the common image evaluation environment 15 based on the reliability of the mapping to the common image evaluation environment of the subject's medical image, in this example, the standardized image 50.

[0089] Once these image evaluation support programs are completed, in step 94, medical personnel may evaluate the discrimination results of the deep learning model 74 based on various information provided via the common image evaluation environment 15.

[0090] While this example primarily describes AD, DLB, and healthy individuals, the present invention is not limited to AD and DLB. The system, control method, and program of this embodiment can also be applied to brain disorders (including brain diseases). Brain disorders include dementia, as well as higher-level brain disorders such as attention disorders, memory disorders, executive dysfunction, social behavior disorders, aphasia, apraxia, and agnosia. Dementia includes AD (Alzheimer's Disease), DLB (Dementia with Lewy Bodies), and other degenerative dementias, such as frontotemporal dementia, progressive supranuclear palsy, corticobasal degeneration, and argyrophilic grain dementia. The state of brain damage includes various aspects related to the brain damage of the subject (examinee, patient, user), such as the presence or absence of brain damage, its progression, the presence and differentiation of causative diseases (causative illnesses) of brain damage such as dementia, and the progression of single or multiple causative illnesses. Brain diseases also include dementia (including AD, DLB, frontotemporal lobar degeneration (FTLD), normal pressure hydrocephalus (NPH), etc.), brain tumors, mental disorders (also known as mental illnesses, including schizophrenia, epilepsy, mood disorders, addiction disorders, and higher-level functional disorders), Parkinson's disease, Asperger's syndrome, attention-deficit hyperactivity disorder (ADHD), sleep disorders, childhood diseases, ischemic brain disorders, and mood disorders (including depression, etc.). Brain disorders also include brain-related diseases such as dementia and multiple sclerosis, and amyloid-β-related diseases include, for example, mild cognitive impairment (MCI), mild cognitive impairment due to Alzheimer's disease (MCI due to AD), prodromal Alzheimer's disease, the pre-symptomatic stage of Alzheimer's disease / preclinical AD, Parkinson's disease, multiple sclerosis, insomnia, sleep disorders, cognitive decline, cognitive impairment, and amyloid-positive / negative neurodegenerative diseases.

[0091] Although the present invention has been described above using an example in which the target region included in the medical image to be evaluated is the brain or a part of the brain, the target region is not limited to the brain and may be any other part of the subject's body.Furthermore, the disease to be evaluated is not limited to dementia and may be any disease related to any other part of the body as long as it is a disease that is the target of image diagnosis. [Explanation of symbols]

[0092] 1. Image diagnosis support information provision system 8, 9 Configuration range 10 Support System 11 First Access Unit 12 Second Access Unit 13 Access Units 15 Image evaluation environment (common evaluation environment) 16a Display 16b Touch Panel 17. Cloud 18 Databases 19 Programs 20 units 30 Support Units 31 Individual and Comparative Evaluation Units 32 Re-Statistical Processing Request Unit 33 Overlay display unit 34 Model Input Selection Unit 35 Statistical Processing Input Selection Unit 36 Mapping and Evaluation Unit 37 Input support function (input support unit) 50 standardized images 52 Image Database 53 Brain Imaging (Type 1 Medical Imaging) 55 Mapping System 56 Third Result 60 First Image Evaluation System 61 processors 62, 72 Input to be evaluated 63, 73 Output of evaluation results 65 databases 66 First Result 67, 77 Input control information 70 Second Image Evaluation System 71 processors 74 Deep Learning Model (Learning Model, First Model) 75 databases 76 Second Result

Claims

1. a first access unit accessible to a first image evaluation system for statistically evaluating a first type of medical image having as a region of interest at least a portion of a subject's body; a second access unit accessible to a second image evaluation system that determines the disease state of the subject using a first model trained by machine learning to evaluate a first disease based on the first type of medical image; a support unit that provides at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment; The support unit is one of the following: i) The support unit includes a unit that selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment. ii) The support unit includes a unit that selects an image area including a second area that is important for determining the disease state of the second image evaluation system as an evaluation target for the first image evaluation system via the common image evaluation environment. iii) The assistance unit includes a unit for controlling the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment based on the reliability of mapping of the medical image of the subject to the common image evaluation environment.

2. In claim 1, The support unit uses the common image evaluation environment, in which the first type of medical image of the subject includes evaluation images that are standardized to images that can be input and output to the first image evaluation system and the second image evaluation system.

3. In claim 1 or 2, The system includes a unit for outputting, via the common image evaluation environment, a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system.

4. In claim 3, The support unit includes a unit that outputs the first result and the second result in the common image evaluation environment in an overlaid manner.

5. In any one of claims 1 to 4, The system includes a unit for outputting, via the common image evaluation environment, a result of re-evaluating a first result regarding the evaluation of the medical image acquired from the first image evaluation system based on the judgment of the disease state of the second image evaluation system.

6. In any one of claims 1 to 5, The support unit includes a unit that outputs, via the common image evaluation environment, a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system and a second area that is considered important for determining the disease state of the second image evaluation system.

7. In claim 6, The support unit includes a unit that outputs the first area and the second area in the common image evaluation environment in an overlaid manner.

8. In any one of claims 1 to 7, The system, wherein the first disease includes dementia, the first type of medical image is an MR image, and the target region includes at least one of the parahippocampal gyrus, dorsal brainstem, and portions of the basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala) of the brain.

9. In any one of claims 1 to 7, the first disease comprises dementia; the first type of medical image is a SPECT image; The target region includes at least one of the precuneus, the occipital lobe, and the dorsolateral prefrontal cortex of the brain; system.

10. In any one of claims 1 to 9, The system further comprises a unit for verifying an assessment of the first disease according to the first model based on an output of the support unit.

11. A control method for an assistance system, comprising: The assistance system includes: a first access unit accessible to a first image evaluation system for statistically evaluating a first type of medical image having as a region of interest at least a portion of a subject's body; a second access unit accessible to a second image evaluation system that determines the disease state of the subject using a first model trained by machine learning to evaluate a first disease based on the first type of medical image; a support unit that provides at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment; The control method includes at least one of the following steps: i) The support unit selects an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment. ii) The support unit selects an image area including a second area that is important for the second image evaluation system to determine the disease state as an evaluation target for the first image evaluation system via the common image evaluation environment. iii) The assistance unit controls the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment based on the reliability of mapping of the medical image of the subject to the common image evaluation environment.

12. In claim 11, A control method, wherein the first disease includes dementia, the first type of medical image is an MR image, and the target region includes at least one of the hippocampus, parahippocampal gyrus, dorsal brainstem, middle temporal pole, and parts of the basal ganglia (putamen, caudate nucleus, entorhinal cortex, parahippocampal gyrus, amygdala) of the brain.

13. In claim 11, the first disease includes dementia, the first type of medical image is a SPECT image, and the region of interest includes at least one of the precuneus, the occipital lobe, and the dorsolateral prefrontal cortex of the brain; Control method.

14. In any one of claims 11 to 13, The control method further comprises verifying the assessment of the first disease by the first model based on the output of the assistance unit.

15. A program for evaluating medical images by a computer, comprising: accessing, by the computer, a first image evaluation system for statistically evaluating a first type of medical image having at least a portion of a subject's body as a region of interest; accessing a second image assessment system that determines the disease state of the subject using a first model trained on machine learning to assess a first disease based on the first type of medical image; providing at least one of input of an evaluation target and output of an evaluation result to the first image evaluation system and the second image evaluation system via a common image evaluation environment; A program having instructions for performing at least one of the following steps: i) Selecting an image area including a first area that is considered important by a first result regarding the evaluation of the medical image obtained from the first image evaluation system as an evaluation target for the second image evaluation system via the common image evaluation environment. ii) Selecting an image area including a second area that is considered important for determining the disease state by the second image evaluation system as an evaluation target for the first image evaluation system via the common image evaluation environment. iii) Based on the reliability of mapping of the medical image of the subject to the common image evaluation environment, controlling the output of a first result regarding the evaluation of the medical image obtained from the first image evaluation system or a second result regarding the evaluation of the medical image of the subject obtained from the second image evaluation system using the common image evaluation environment.

Images