Methods for operating diagnostic support devices, computer programs, and medical devices

Abstract

To provide a diagnosis support apparatus, a computer program, and a diagnosis support method capable of assisting a medical doctor in making a diagnosis.SOLUTION: A diagnosis support apparatus comprises an acquisition unit which acquires subject data about the brain of a subject; a prediction unit which predicts a brain disease of the subject based on the subject data; a specification unit which specifies a data item corresponding to the subject data contributing to a prediction result from the subject data; and an output unit which outputs the specified data item and prior knowledge about the brain disease in association with each other.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a diagnostic support device, a computer program, and Operation of medical devices a method.

Background Art

[0002] Regarding brain diseases including Alzheimer's disease, early diagnosis is required because it is effective to take appropriate measures at an early stage of the disease state. In the field of computer-aided diagnosis (CAD), there is a technology in which a diagnostic system analyzes based on a patient's image and clinical information. It is expected that a doctor can make a more accurate diagnosis by referring to the result output by the diagnostic system as a second opinion.

[0003] In recent years, diagnostic systems using machine learning have also been devised. Patent Document 1 discloses an artificial intelligence technology for predicting a situation regarding a matter at a time different from the imaging time based on a combination of image data and non-image data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in a diagnostic system using artificial intelligence technology, the internal processing is complex, and the intermediate judgment process and prediction algorithm are black-boxed. Therefore, even when a diagnostic result is output, the basis of the diagnostic result is often unclear, and it may be difficult for a doctor to refer to it when making a diagnosis.

[0006] The present invention has been made in view of the above circumstances, and relates to a diagnostic support device, a computer program, and that can assist a physician in making a diagnosis. Operation of medical devices The purpose is to provide a method. [Means for solving the problem]

[0007] The present invention includes several means for solving the above problems, but to give one example, the diagnostic support device comprises an acquisition unit that acquires subject data relating to the subject's brain, a prediction unit that predicts the subject's brain disease based on the subject data, an identification unit that identifies data items from the subject data that correspond to the subject data that contribute to the prediction result of the prediction unit, and an output unit that outputs the data items identified by the identification unit in association with prior knowledge relating to the brain disease. [Effects of the Invention]

[0008] According to the present invention, the basis for predicting brain diseases and prior knowledge are linked, which can be used as a reference for doctors' diagnoses and can support doctors' diagnoses. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the configuration of the diagnostic support device of this embodiment. [Figure 2] This figure shows an example of the configuration of an image processing function. [Figure 3] This figure shows an example of the configuration of the image feature calculation function of the first embodiment. [Figure 4] This figure shows an example of the configuration of the prediction function. [Figure 5] This figure shows an example of the structure of subject data. [Figure 6] This figure shows an example of how the contribution is calculated. [Figure 7] This figure shows an example of the structure of a prior knowledge database. [Figure 8] This figure shows an example of how to retrieve prior knowledge. [Figure 9] This figure shows the first example of how prediction results are displayed. [Figure 10] This is a diagram showing an example of the configuration of the image feature amount calculation function of the second embodiment. [Figure 11] This is a diagram showing an example of the configuration of the prediction basis calculation function of the second embodiment. [Figure 12] This is a diagram showing a second example of prediction result display. [Figure 13] This is a diagram showing an example of the configuration of the diagnostic support device of the present embodiment. [Figure 14] This is a diagram showing an example of the configuration of the diagnostic support system. [Figure 15] This is a diagram showing the procedure of the prediction process. [Figure 16] This is a diagram showing the procedure of the generation process of the learned prediction model. [Figure 17] This is a diagram showing other prediction tasks.

Mode for Carrying Out the Invention

[0010] (First Embodiment) Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a diagram showing an example of the configuration of the diagnostic support device 50 of this embodiment. The diagnostic support device 50 comprises a user interface unit 10, a processing unit 20, and a database unit 30. The user interface unit 10 comprises an image input function 11, a subject information input function 12, and a prediction result display function 13. The processing unit 20 comprises an image processing function 21, an image feature calculation function 22, a prediction function 23, a prediction basis calculation function 24, a prior knowledge matching function 25, and a learning processing function 26. The database unit 30 comprises an ROI (Region Of Interest) for image feature calculation, a control group database 32, trained model parameters 33, a prior knowledge database 34, and a brain atlas database 35. The database unit 30 may be incorporated into the diagnostic support device 50, or the database unit 30 may be provided outside the diagnostic support device 50 and accessed from the diagnostic support device 50. The diagnostic support device 50 can output prediction results for multiple types of prediction tasks in order to support brain-related diagnoses. Below, we will explain AD conversion as an example of a prediction task. AD conversion predicts the likelihood that a person with normal cognitive function or MCI (Mild Cognitive Impairment) will be converted to AD (Alzheimer's disease) after a certain number of years (for example, 1 year, 2 years, 3 years, 5 years, 10 years, etc.).

[0011] The image input function 11 has an interface function with an MRI (Magnetic Resonance Imaging) device or an image DB (not shown) and can acquire (receive) medical images related to the brain of a subject (including a patient). The medical image is, for example, an MRI image (also referred to as an MR image), but the medical image is not limited to an MRI image. For example, it may be a PET (Positron Emission Tomography) image that can be acquired from a PET device, a SPECT (single photon emission CT) image that can be acquired from a SPECT device, or a CT (computed tomography) image that can be acquired from a CT device. The MRI image includes not only the MRI image obtained by an MRI device (for example, a T1-weighted image, a T2-weighted image, a diffusion-weighted image, etc.), but also a processed image obtained by performing a predetermined operation on the MRI signal. Hereinafter, an MRI image will be described as an example of the medical image.

[0012] The subject information input function 12 has a function of inputting subject information and can acquire (receive) subject information from an external device. Details of the subject information will be described later.

[0013] The prediction result display function has a function of displaying the prediction result of the processing unit 20 on a display device (display unit) (not shown). Details of the prediction result will be described later.

[0014] FIG. 2 is a diagram showing an example of the configuration of the image processing function 21. The image processing function 21 includes an image reconstruction function 211, a tissue segmentation function 212, an anatomical normalization function 213, a smoothing function 214, and a density value correction function 215. Note that a part or all of the image processing function 21 may be omitted.

[0015] The image reconstruction function 211 performs image reconstruction on the subject's MRI images acquired from the image input function 11. Image reconstruction converts the subject's MRI image (3D image) into, for example, 100 to 200 T1-weighted images, each acquired in slices of a predetermined thickness to include the entire brain. During this process, the slice images are resampled so that the length of each side of the voxels in each slice image is equal. Subsequently, the subject's MRI image is spatially aligned with a standard brain image. Specifically, linear transformations (affine transformations), cropping, etc., are applied to the subject's MRI image to match the position, angle, and size of a standard brain image. This corrects for any displacement of the subject's head position during MRI imaging, improving accuracy when comparing with a standard brain image.

[0016] The tissue separation function 212 generates gray matter brain images and white matter brain images by extracting gray matter (GM) and white matter (WM) from the reconstructed MRI images. Since T1-weighted images contain three types of tissue: white matter which shows high signal values ​​corresponding to nerve fibers, gray matter which shows intermediate signal values ​​corresponding to nerve cells, and cerebrospinal fluid (CSF) which shows low signal values, the function focuses on the differences in these signal values ​​and processes them to extract gray matter, white matter, and cerebrospinal fluid, respectively.

[0017] The anatomical standardization function 213 performs anatomical standardization on the extracted gray matter brain images, white matter brain images, and cerebrospinal fluid images. Anatomical standardization involves aligning voxels to a standard brain image. In this embodiment, anatomical standardization is performed using DARTEL (diffeomorphic anatomical registration through exponentiated Lie algebra). DARTEL is an algorithm for performing nonlinear deformation using a large number of parameters.

[0018] The smoothing function 214 applies image smoothing to gray matter brain images and white matter brain images that have undergone anatomical standardization using Dartel, with the aim of improving the signal-to-noise ratio. By performing image smoothing in this way, individual differences that do not perfectly match during the anatomical standardization process can be reduced.

[0019] The density value correction function 215 performs density value correction to adjust the voxel values ​​of the entire brain to match the distribution of voxel values ​​in the image group of healthy individuals. The density-corrected gray matter brain images, white matter brain images, and cerebrospinal fluid images are output to the image feature calculation function 22.

[0020] Figure 3 shows an example of the configuration of the image feature calculation function 22 in the first embodiment. The image feature calculation function 22 includes atrophy score calculation function 221, ROI identification function 222, and atrophy degree calculation function 223. The image feature calculation function 22 calculates the atrophy degree (ROI atrophy degree) calculated by the atrophy degree calculation function 223 as an image feature.

[0021] The atrophy score calculation function 221 refers to MRI images of healthy individuals recorded in the control group database 32, compares the subject's MRI images with those of healthy individuals, and calculates an "atrophy score" indicating the degree of brain atrophy in the subject. In this embodiment, the statistical index "Z score" is used as the atrophy score. Specifically, the gray matter brain images and white matter brain images of the subject, which have been subjected to anatomical standardization, image smoothing, etc. by the image processing function 21, are statistically compared with a large number of MRI images of gray matter and white matter from healthy individuals recorded in the control group database 32, and the Z scores for gray matter and white matter are calculated for all voxels or voxels in specific regions of the MRI images.

[0022] The Z-score can be calculated using the following formula. Z score = (μ(x,y,z)-P(x,y,z)) / σ(x,y,z) μ represents the mean of voxel values ​​in the MRI image group of healthy individuals, σ represents the standard deviation of voxel values ​​in the MRI image group of healthy individuals, and P represents the voxel values ​​in the MRI image of the subject. (x,y,z) are the coordinate values ​​of the voxel. The Z score is a value obtained by scaling the difference between the voxel values ​​of the subject's image and the mean of the corresponding voxel values ​​in the healthy image group by the standard deviation, and indicates the degree of relative decrease in gray matter and white matter volume. By using the Z score, it is possible to quantitatively analyze which parts of the subject's MRI image are changing and what kind of changes are occurring compared to the healthy image group. For example, voxels with a positive Z score indicate areas of atrophy compared to the standard brain of the healthy group, and the larger the value, the greater the statistically significant deviation. For example, a Z score of "2" means that it is more than twice the standard deviation from the mean, and it is evaluated as a statistically significant difference at a significance level of approximately 5%. Note that the atrophy score is not limited to the Z score. An index that allows for the determination of the relative magnitudes of voxel values ​​between images of subjects and images of healthy individuals may be used as an atrophy score to indicate the degree of atrophy (e.g., a t-score).

[0023] The ROI identification function 222 identifies brain regions (regions of interest: ROIs) specific to each disease. The ROI identification function 222 can, for example, identify regions of interest associated with each disease based on statistical processing. Specifically, when identifying a region of interest corresponding to a certain disease, a two-sample t-test is performed to statistically test for significant differences between the two groups at the voxel level: a group of MRI images of subjects with the disease (disease image group) and a group of MRI images of subjects without the disease (e.g., healthy individuals) (non-disease image group). Voxels showing a significant difference are considered characteristic voxels in the disease, and the set of their coordinates is identified as the region of interest corresponding to the disease. There may be multiple regions of interest. Furthermore, regions of interest may be identified by considering both the significance level and empirical rules. In addition, the identification of regions of interest may be performed, for example, from disease images (or groups of disease images) alone. For example, for disease images (or groups of disease images), regions with large atrophy may be identified as regions of interest in correlation with the overall size of atrophy in the brain.

[0024] The ROI identification function 222 may read and use the region of interest recorded in the ROI 31 for image feature calculation. Alternatively, the ROI identification function 222 may refer to the brain atlas database 35 to identify the region of interest based on spatially segmented brain information (atlas). Examples of brain atlases that can be used include the AAL atlas (Automated Anatomical Labeling), Brodmann atlas, LPBA40 atlas (LONI Probabilistic Brain Atlas), and Talairach atlas.

[0025] The atrophy degree calculation function 223 calculates the atrophy degree for each region of interest identified by the ROI identification function 222. The atrophy degree can be, for example, the average value of positive Z-scores within the region of interest. For example, if the region of interest is the hippocampus, the average value of positive Z-scores within the hippocampus can be used as the hippocampal atrophy degree. Note that the atrophy degree is not limited to the "average value of positive Z-scores" within the region of interest; for example, a predetermined threshold for Z-scores may be set, and the atrophy degree may be the average value of Z-scores exceeding the threshold, or the average value of Z-scores, or the maximum value of Z-scores. Alternatively, the proportion of voxels with Z-scores exceeding the threshold to the total number of voxels within the region of interest may be used. Furthermore, without comparing with a control group, the sum or average of voxel values ​​within the region of interest may simply be used as the image feature. The atrophy degree (image feature) of the brain region calculated by the atrophy degree calculation function 223 is output to the prediction function 23.

[0026] Figure 4 shows an example of the configuration of the prediction function 23. The prediction function 23 comprises a scaling function 231 and a trained prediction model 232. The scaling function 231 functions as an acquisition unit that acquires subject data related to the subject's brain. The subject data includes image features and subject information. The subject data will be described below.

[0027] Figure 5 shows an example of the structure of subject data. Subject data is a collection of data with different units and attributes, and can be broadly classified into three categories: image features, image acquisition parameters, and subject information. Image features can be further classified into the degree of atrophy of brain regions, calculated by the image feature calculation function 22, and feature vectors, which will be described later. Details of feature vectors will be described later.

[0028] The subject data includes not only subject data at a single point in time, but also subject data at multiple point in time. For example, as a first example, subject data at two point in time, namely the current subject data and subject data from six months ago, can be input into the prediction function 23 to perform a prediction. As a second example, subject data at three point in time, namely the current subject data, subject data from six months ago, and subject data from one year ago, can be input into the prediction function 23 to perform a prediction. As a third example, subject data at the current subject data, subject data from a period Δt prior to the current subject data, and the period Δt can be input into the prediction function 23 to perform a prediction.

[0029] Image acquisition parameters include, for example, information about the model of the MRI device and the imaging protocol (e.g., magnetic field strength, sequence type, imaging parameters, etc.).

[0030] Subject information can be further classified into neuropsychological test information, clinical information, and biochemical test information.

[0031] Neuropsychological assessment information includes, for example, ADAS-cog (Alzheimer's Disease Assessment Scale), MMSE (Mini-Mental State Examination), CDR (Clinical Dementia Rating), FAQ (Functional Activity Questionnaire), GDS (Geriatric Depression Scale), and neuropsychological batteries (batteries combining several tests such as Logical Memory IA Immediate Recall, Logical Memory IIA Delayed Recall, WAIS-III, Clock Drawing / Clock Copying, Verbal Fluency Task, Trail Making Test A&B, and Boston Name Test). The MMSE can assess the severity of dementia; subjects scoring 24 points or less out of 30 are considered suspected of having dementia, while scores between 0 and 10 points are considered to have severe dementia. The CDR (Chronic Dementia Ratio) is categorized as follows: a score of 0 indicates a healthy individual, 0.5 indicates suspected dementia, 1 indicates mild dementia, 2 indicates moderate dementia, and 3 indicates severe dementia. The GDS (Growth and Development Score) is evaluated on a scale of 0 to 15 points, with a score of 6 or higher indicating suspected depression. The ADAS-cog (Adaptive Aptitude Test) is evaluated on a scale of 0 to 70 points, with higher scores indicating more severe dementia. The neuropsychological battery allows for the appropriate modification of the test combination depending on the subject's dementia status.

[0032] Clinical information includes, for example, the subject's age, sex, height, weight, BMI, years of education, medical history (presence or absence of diabetes, etc.), family history, presence or absence of dementia in family members, and vital signs (blood pressure, pulse, body temperature, etc.).

[0033] Biochemical test information includes blood test results, cerebrospinal fluid test results (CSF-TAU (including T-TAU and P-TAU), CSF-Aβ), ApoE genotype, etc.

[0034] The scaling function 231 can acquire at least one of neuropsychological test information, clinical information, and biochemical test information. For example, it may acquire only clinical information, or it may acquire either neuropsychological test information or biochemical test information. Furthermore, the neuropsychological test information, clinical information, and biochemical test information acquired may consist of only a portion of the items described above.

[0035] The scaling function 231 scales and equalizes image features (including image acquisition parameters in this case) and subject information because they have different units and attributes. Scaling methods include, for example, standardization or normalization. Standardization is a scaling method in which the mean is 0 and the variance is 1. The scaled value x' can be calculated using the formula x' = (x - μ) / σ, where x represents the value before scaling, μ represents the mean, and σ represents the standard deviation.

[0036] Normalization is a scaling method that sets the minimum value to 0 and the maximum value to 1. The scaled value x' is x' = (xx min ) / (x max -x min It can be calculated using the formula ). Here, x represents the value before scaling, and x max This indicates the maximum value that x can take, and x min This indicates the minimum value that x can take. The scaling function 231 outputs the scaled subject data X=(x1,x2,…,xn) to the trained predictive model 232.

[0037] The trained prediction model 232 functions as a prediction unit and predicts brain diseases in a subject based on subject data. The prediction model of the trained prediction model 232 is represented by f(X,θ). When subject data X=(x1,x2,…,xn) for one subject is input to the trained prediction model 232, it outputs a prediction result y and a probability estimate p. The prediction result y is whether or not the subject will undergo AD conversion after a certain number of years (prediction period: T years). The trained prediction model 232 performs binary classification to predict whether or not AD conversion will occur. For example, if y=1, it can be assumed that AD conversion will occur after T years (within T years), and if y=0, it can be assumed that AD conversion will not occur after T years (within T years). The probability estimate p indicates, for example, the probability of AD conversion occurring when y=1, and the probability of AD conversion not occurring when y=0. The probability estimate p is a value in the range of 0 to 1. Furthermore, a pre-trained prediction model 232 may be provided for each prediction period T, or a single pre-trained prediction model 232 may be used to predict whether or not AD conversion will occur over multiple prediction periods T.

[0038] In the example shown in Figure 4, both image features and subject information are used as subject data, but this is not the only option; image features alone may be used as subject data. In this case, subject information is not required.

[0039] The pre-trained predictive model 232 can be any predictive model that performs binary classification, such as machine learning models like random forests, SVMs (Support Vector Machines), AdaBoost, gradient boosting, logistic regression, decision trees, neural networks, deep learning, or ensembles of these models.

[0040] Next, we will explain how to generate the trained predictive model 232. The training dataset consists of a large number of case data. The scaled subject data of the subjects is set as X′=(x′1,x′2,…x′n), and the presence or absence of AD conversion after T years is determined by observing the subjects over time, and this is denoted as y′. Here, y′=1 indicates that AD conversion occurred, and y′=0 indicates that AD conversion did not occur.

[0041] The learning processing function 26 uses subject data X' from a large number of subjects and y' (whether or not the subject underwent AD conversion after T years) as a training dataset, and estimates model parameters θ such that y' = f(X', θ) with X' as the explanatory variable and y' as the dependent variable. This enables the generation of a trained predictive model 232. The generated trained predictive model 232 can be stored in the trained model parameters 33.

[0042] The prediction basis calculation function 24 functions as a contribution calculation unit and calculates the contribution of subject data to the prediction result of the trained prediction model 232. For example, let y be the prediction result when subject data X=(x1,x2,x3,...,xn) is input to the trained prediction model 232. The prediction basis calculation function 24 calculates the contribution C=(c1,c2,c3,...,cn) for the prediction result y, using the subject data X=(x1,x2,x3,...,xn) as the prediction basis. For example, contribution c1 is the contribution of subject data x1 as the prediction basis.

[0043] The contribution can be calculated using techniques such as SHAP (Shapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), and Explainable Boosting Machine. SHAP is a method that allocates the increase or decrease from the mean of the predicted probability of the data to be explained, according to the influence of each variable in that data. LIME is a method that approximates a trained model with a locally simple model (a model that can only be applied to the data to be explained and its surroundings), and calculates the coefficients of that approximated model as the contribution. Explainable Boosting Machine is a method that combines GAM (Generated Additive Model) and gradient boosting to achieve both the accuracy of gradient boosting and the interpretability of GAM. SHAP is described below.

[0044] In the following, for convenience, we consider three data sets, x1, x2, and x3, as subject data. SHAP technology introduces the concept of marginal contribution. Marginal contribution indicates, for example, how much the prediction result y increases when subject data x1 is input into the trained prediction model 232. Also, when subject data x1 is input into the trained prediction model 232, the prediction result y changes depending on whether subject data x2 and x3 have already been input. When there are three data sets of subject data, there are six possible input orders (addition orders) for subject data x1, x2, and x3. The contribution c1 of subject data x1 can be calculated by calculating the marginal contribution for all possible orders and taking the average value.

[0045] Figure 6 shows an example of how to calculate the contribution. There are six possible orders for adding subject data x1, x2, and x3: x1→x2→x3, x1→x3→x2, x2→x1→x3, x2→x3→x1, x3→x1→x2, and x3→x2→x1. If the marginal contribution of x1 in each addition order is φ11~φ16, then the contribution c1 of x1 can be calculated as the average value of φ11~φ16. Similarly, if the marginal contribution of x2 in each addition order is φ21~φ26, then the contribution c2 of x2 can be calculated as the average value of φ21~φ26. And if the marginal contribution of x3 in each addition order is φ31~φ36, then the contribution c3 of x3 can be calculated as the average value of φ31~φ36.

[0046] The prior knowledge matching function 25 matches the prediction basis with the prior knowledge database 34.

[0047] Figure 7 shows an example of the configuration of the prior knowledge database 34. The prior knowledge database 34 shown in Figure 7 corresponds to the AD conversion prediction task, and similar prior knowledge databases 34 can be constructed for other prediction tasks. As shown in Figure 7, the prior knowledge database 34 registers prior knowledge (explanatory text) for each data item, and the strength of the prior knowledge (reliability as evidence) is associated with it. The strength of the prior knowledge can be classified as, for example, "high," "medium," and "low," but is not limited to this and may also be expressed numerically. Furthermore, URLs of references to detailed explanations related to the prior knowledge are associated with it. The data items include items related to AD conversion, such as hippocampal atrophy degree, whole brain atrophy degree, medial temporal lobe atrophy degree, amyloid PET, MMSE, CDR, logical memory, GDS, years of education, etc. Note that the data items are not limited to the example in Figure 7.

[0048] As shown in Figure 7, for the data item "Hippocampal Atrophy," the accompanying explanatory statement is "There is strong evidence that hippocampal atrophy increases risk." In this case, the strength of knowledge is "High." Similarly, for the data item "MMSE," the accompanying explanatory statement is "There is strong evidence that a decline in MMSE increases risk." In this case, the strength of knowledge is "High." Furthermore, for the data item "Logical Memory," the accompanying explanatory statement is "There is evidence that logical memory increases risk." In this case, the strength of knowledge is "Medium."

[0049] The prior knowledge matching function 25 functions as an identification unit and refers to the prior knowledge database 34 to identify data items from the subject data that correspond to the subject data that contribute to the prediction results of the trained prediction model 232. If the subject data X = (x1, x2, x3, ..., xn), then each item of data x1, x2, x3, ..., xn is a data item, and the value of each data becomes the value of the data item. For example, if the item of data x1 is "hippocampal atrophy degree", then the value of the data item becomes the hippocampal atrophy degree value.

[0050] More specifically, the prior knowledge matching function 25 identifies data items corresponding to subject data based on the contribution calculated by the prior knowledge matching function 25 and a predetermined contribution threshold, and can read prior knowledge corresponding to the identified data items from the prior knowledge database 34.

[0051] FIG. 8 is a diagram showing an example of a method for reading prior knowledge. Let the subject data (data items) be x1, x2, …, xi, …, xn, and let the contribution degrees of each subject data be c1, c2, …, ci, …, cn. Let the threshold values of each contribution degree be Th1, Th2, …, Thi, …, Thn. Note that the threshold values Th1, Th2, …, Thi, …, Thn of each contribution degree may be the same value. As shown in FIG. 8, if C1 > Th1, the prior knowledge corresponding to the subject data x1 is read from the prior knowledge database 34. Also, if C2 < Th2, the prior knowledge corresponding to the subject data x2 is not read. The same applies hereinafter. In this way, the prior knowledge verification function 25 can select the prior knowledge corresponding to the subject data whose contribution degree is greater than the threshold value from the subject data x1, x2, …, xi, …, xn.

[0052] With the above configuration, regarding the prediction result by the prediction function 23, the relationship (association) between the basis for prediction and the prior knowledge known conventionally is shown, and a user such as a doctor can know how much the known evidence supports the prediction result. For example, if the basis for prediction (reason for prediction) matches a lot with the known evidence, it can be determined that the reliability of the basis for prediction is high.

[0053] The prior knowledge verification function 25 has a function as an output unit, and can associate the specified data item with the prior knowledge regarding the brain disease and output it to the user interface unit 10. The prior knowledge verification function 25 can output the subject data corresponding to the specified data item. Also, the prior knowledge verification function 25 may output the degree of coincidence (degree of relevance) between the specified data item and the prior knowledge. The degree of coincidence indicates how much the basis for prediction (reason for prediction) matches the prior knowledge.

[0054] The prior knowledge matching function 25 functions as a relevance summing unit and can calculate the degree of agreement (degree of relevance) based on at least one of the contribution of subject data corresponding to a data item and the strength (confidence level) of the prior knowledge as evidence. For example, if the contribution is C and the strength of the prior knowledge is S, the degree of agreement E can be calculated using the formula E = α·C + β·S. α and β are weighting coefficients and may be 0, provided that α + β ≠ 0. The degree of agreement may be classified into three categories, for example, "high," "medium," and "low," based on the value of E.

[0055] The prediction result display function 13 can display prediction results that associate the identified data items with prior knowledge about brain diseases. The prediction results can also include information output by the prior knowledge matching function 25.

[0056] Figure 9 shows the first example of the prediction result display. The "Alzheimer's disease conversion prediction result" shows the probability that a subject (patient) will convert to Alzheimer's disease after a certain number of years, as predicted by the prediction function 23 after analyzing the subject data of a given subject. In the example in Figure 9, both the conversion rates for 2 years and 5 years are displayed, but either conversion rate or a prediction period of 1 year, 3 years, etc., may be shown. The conversion rate represents the probability estimate p output by the trained prediction model 232.

[0057] "Prediction Reason" indicates the basis of the subject data that led to the prediction function 23. "Item" is a data item of the subject data, "Measured Value" is the value of the data item, and "Contribution to Prediction" is the degree of contribution, indicating how much each item of the subject data contributes to the prediction. "Known Evidence" displays "Explanation" as prior knowledge, and this "Explanation" is associated with the "Prediction Reason". "Degree of Agreement" indicates how well the "Prediction Reason" matches the prior knowledge. The "Degree of Agreement" can be expressed as, for example, "High," "Medium," or "Low," but is not limited to this and may also be expressed numerically. "Details" is provided with an arrow icon for displaying a detailed explanation, and by operating this icon, more detailed information about the explanation can be displayed.

[0058] The prediction result display function 13 functions as a display unit and can display data items, measured values ​​(values ​​of the subject data), and related prior knowledge in the order of their contribution to the prediction (the degree to which the subject data contributes to the prediction result). In the example in Figure 9, the information is displayed in the order of contributions of 5, 4, 3.6, 2.2, 2,... This allows users such as doctors to visually see the reasons for the prediction in the order in which they contribute to the prediction result, and to easily read important prediction reasons and known evidence. In the example in Figure 9, it can also be seen that the magnitude of the measured value of "hippocampal atrophy" is the greatest influencing factor that acts on the positive side of the prediction result, and that the small measured value of "temporal lobe atrophy" is a factor that acts on the negative side (in a direction that does not contribute to AD conversion) of the prediction result.

[0059] By displaying the prediction results as shown in Figure 9, physicians can see how well known evidence supports the prediction results of the diagnostic support device 50. If the prediction basis (reason for prediction) of the diagnostic support device 50 largely matches known evidence, it indicates that the reliability of the prediction basis is high.

[0060] Furthermore, if the correlation with known evidence is low, it can be interpreted that the diagnostic support device 50 is suggesting a new hypothesis based on evidence, but it is also possible that this is a result caused by bias in the training data. Therefore, if the basis for the prediction contradicts common sense or logic, instead of making a judgment based solely on the prediction result, it may be advisable to conduct more detailed additional tests to further improve reliability.

[0061] As described above, according to this embodiment, the basis for predicting brain diseases and prior knowledge are linked, so it can be used as a reference for doctors' diagnoses and can support doctors' diagnoses.

[0062] (Second Embodiment) In the first embodiment described above, the ROI atrophy degree was used as the image feature, but the image feature is not limited to the ROI atrophy degree. In the second embodiment, a configuration in which a feature vector is used as the image feature will be described.

[0063] Figure 10 shows an example of the configuration of the image feature calculation function 22 in the second embodiment. In the second embodiment, the image feature calculation function 22 can be configured as a CNN (Convolutional Neural Network) and comprises an input layer 22a, multiple (e.g., 17 layers) convolutional layers 22b, a pooling layer 22c, fully connected layers 22d and 22e, and an output layer 22f. Note that the CNN configuration is just one example and is not limited to the example in Figure 10. For example, VGG16, ResNet, DenseNet, EfficientNet, AttResNet, etc., may be used. The input layer 22a receives standardized gray matter images from the image processing function 21. That is, from the tissue segmentation results of gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF), images of gray matter that have undergone anatomical standardization are input to the input layer 22a. The output layer 22f outputs the prediction result, whether or not AD conversion is performed. The prediction results from output layer 22f will be used in the prediction basis calculation process described later.

[0064] The image feature calculation function 22 calculates a feature vector as an image feature, with the values ​​of each node in the fully connected layer 22e immediately preceding the output layer 22f as its elements. If the number of nodes in the fully connected layer 22e is N, then N image features can be obtained. The image feature calculation function 22 outputs the calculated feature vector to the prediction function 23.

[0065] The prediction function 23 acquires the feature vector output by the image feature calculation function 22 as an image feature and performs the same processing as in the first embodiment.

[0066] To generate a pre-trained CNN model, one should use a training dataset and update the CNN parameters to minimize the prediction error (e.g., cross-entropy error) of the CNN's prediction results (with or without AD conversion). The generation of a pre-trained CNN model can be performed using the training processing function 26.

[0067] Figure 11 shows an example of the configuration of the prediction basis calculation function 24 of the second embodiment. The prediction basis calculation function 24 includes a gradient calculation function 241, a weighting calculation function 242, an adder 243, and a ReLU 244. The feature maps of the output of the convolutional layer 22b of the image feature calculation function 22 (CNN) are A1, A2, A3, ..., Ak. k indicates the number of channels. By using k filters in the convolution operation, a k-channel feature map is generated. The feature map can be, for example, the final layer of the convolutional layer, which is on the input side of the fully connected layer. This is because the positional information of the image is lost in the fully connected layer, while the final layer of the convolutional layer can abstract the features of the image well. Note that it is not limited to the final layer, but any convolutional layer may be used.

[0068] The gradient calculation function 241 calculates the degree to which feature maps A1, A2, A3, ..., Ak influence the prediction results from the output layer 22f using gradients. The gradient calculation function 241 calculates the gradients α1, α2, α3, ..., αk for each of the feature maps A1, A2, A3, ..., Ak. Gradient calculation is an operation that calculates how much the prediction results change when each element of the feature map changes slightly, and then smooths the result within that feature map.

[0069] Next, the importance of the feature maps is calculated using the gradient calculation results. Specifically, the weighting calculation function 242 multiplies each of the feature maps A1, A2, A3, ..., Ak by the respective global average pooling values ​​of the gradients α1, α2, α3, ..., αk (M1, M2, M3, ..., Mk, which are the average values ​​of each gradient in the image) to perform a weighting calculation. The weighted feature maps α1·A1, α2·A2, α3·A3, ..., αk·Ak are added by the adder 243 and passed through the ReLU (activation function) 244 to generate a heatmap. The heatmap visualizes which parts of the image were used as the basis for the prediction, focusing on the feature maps extracted by the convolutional layer 22b. The image size of the heatmap is scaled to the size of the input standardized gray matter image, and the prediction basis can be visualized by superimposing the heatmap onto the standardized gray matter image.

[0070] A heatmap contains three-dimensional heatmap information that shows the basis (degree of influence) on the prediction result in numerical values ​​(also called heatmap values) corresponding to each coordinate position (x, y). The feature parts shown in the heatmap information can represent the height of the feature according to the magnitude of the heatmap value. The display method (e.g., color or density) can be changed according to the importance of the basis for the judgment. As a visualization method, in addition to Grad-CAM explained in Figure 11, Guided Grad-CAM and Guided Backprop techniques may also be used. Guided Backprop is a type of gradient-based highlighting method that considers the contribution to be higher the larger the amount of change when a value of a certain data is changed by a small amount.

[0071] Figure 12 shows a second example of the prediction result display. The difference from the first example shown in Figure 9 is that it displays the image region (the image region showing the basis for the prediction) and the name of the brain atlas corresponding to the image region. Each image region is displayed by overlaying the heatmap values, which have been clustered using a predetermined threshold to divide the region (regions 1 to 3), onto a standard brain image. In addition, the name of the atlas corresponding to the coordinates of regions 1 to 3 is displayed. In the example in Figure 12, regions 1 to 3 are "hippocampus, parahippocampal gyrus," "hippocampus, lingual gyrus," and "precuneus, calcarine sulcus," respectively.

[0072] In this way, doctors can not only see on images which specific brain regions strongly influenced the prediction, but also, by mapping the coordinates to the atlas, they can check for the presence or absence of evidence linked to the atlas.

[0073] Figure 13 shows an example of the configuration of the diagnostic support device 80 of this embodiment. The diagnostic support device 80 can use, for example, a personal computer. The diagnostic support device 80 may include, for example, a processing unit 20, which can consist of a CPU 81, ROM 82, RAM 83, GPU 84, video memory 85, and a recording medium reading unit 86. A computer program (program product) recorded on a recording medium 90 (for example, an optically readable disk storage medium such as a CD-ROM) can be read by the recording medium reading unit 86 (for example, an optical disk drive) and stored in the RAM 83. Here, the computer program includes the processing procedures described in Figures 15 and 16, which will be described later. The computer program may also be stored on a hard disk (not shown) and stored in the RAM 83 when the computer program is executed.

[0074] By executing the computer program stored in RAM83 on CPU81, the image processing function 21, image feature calculation function 22, prediction function 23, prediction basis calculation function 24, prior knowledge matching function 25, and learning processing function 26 can be executed. The video memory 85 can temporarily store data and processing results for various image processing tasks. In addition, the computer program can be downloaded from another computer or network device via a network such as the Internet, instead of being read by the recording medium reading unit 86.

[0075] In the example described above, the diagnostic support device 50 was configured to include a user interface unit 10, a processing unit 20, and a database unit 30, but it is not limited to this configuration. For example, the user interface unit 10, the processing unit 20, and the database unit 30 can be distributed as follows.

[0076] Figure 14 shows an example of the configuration of a diagnostic support system. The diagnostic support system comprises a terminal device 100, a diagnostic support server 200, and a data server 300. The terminal device 100, the diagnostic support server 200, and the data server 300 are connected via a communication network 1 such as the Internet. The terminal device 100 corresponds to the user interface unit 10 and is composed of a personal computer or the like. The diagnostic support server 200 corresponds to the processing unit 20, and the data server 300 corresponds to the database unit 30. The functions of the terminal device 100, the diagnostic support server 200, and the data server 300 are the same as those of the user interface unit 10, the processing unit 20, and the database unit 30, so their explanation is omitted.

[0077] Next, we will explain the processing performed by the diagnostic support device 50.

[0078] Figure 15 shows the procedure for the prediction process. The processing unit 20 acquires a medical image of the subject (S11) and acquires subject information of the subject (S12). The processing unit 20 performs image reconstruction on the acquired medical image (S13) and tissue segmentation (S14). Tissue segmentation is a process of separating and extracting, for example, gray matter, white matter, and cerebrospinal fluid.

[0079] The processing unit 20 performs anatomical standardization for each tissue segment (S15) and calculates image features from the anatomically standardized medical images (S16). The image features may be, for example, ROI atrophy or feature vectors.

[0080] The processing unit 20 scales the image features and subject information (S17), inputs the scaled subject data into the prediction function 23, and performs the prediction result calculation process (S18). The processing unit 20 performs the prediction basis calculation process (S19). The prediction basis calculation process includes the process of calculating the contribution of each subject data to the prediction result.

[0081] The processing unit 20 performs a comparison with prior knowledge based on the calculated contribution (S20). The comparison with prior knowledge identifies data items corresponding to the subject data based on the calculated contribution and a predetermined contribution threshold, and reads the prior knowledge corresponding to the identified data items from the prior knowledge database 34.

[0082] The processing unit 20 outputs the prediction result (S21) and terminates the process. The prediction result is as exemplified in Figure 9 or Figure 12 above.

[0083] As described above, the computer program instructs the computer to perform the following processes: acquire subject data about the subject's brain, predict the subject's brain disease based on that subject data, identify data items from the subject data that contribute to the prediction of the brain disease, and output the identified data items in association with prior knowledge about the brain disease.

[0084] Figure 16 shows the procedure for generating the trained predictive model 232. The processing unit 20 acquires medical images of subjects from a large number of case data (S31) and acquires subject information of the subject (S32). The processing unit 20 acquires training data collected through observation of the subject (S33). The training data is, for example, data indicating whether or not the subject underwent AD conversion.

[0085] The processing unit 20 performs image reconstruction on the acquired medical image (S34) and tissue segmentation (S35). Tissue segmentation is a process of separating and extracting, for example, gray matter, white matter, and cerebrospinal fluid. The processing unit 20 performs anatomical standardization for each segmented tissue (S36) and calculates image features from the anatomically standardized medical image (S37). Image features may be, for example, ROI atrophy degree or feature vectors.

[0086] The processing unit 20 determines whether there is other training data (S38), and if there is training data (YES in S38), it repeats the processing from step S31 onwards. If there is no training data (NO in S38), the processing unit 20 scales the image features and subject information (S39). If the subject data for training is input to the training model, the processing unit 20 updates the internal parameters of the training model so that the data output by the training model approaches the training data (S40).

[0087] The processing unit 20 determines whether the value of the loss function, which represents the error between the data output by the learning model and the training data, is within an acceptable range (S41). If the value of the loss function is not within an acceptable range (NO in S41), the processing from step S40 onward is repeated. If the value of the loss function is within an acceptable range (YES in S41), the processing unit 20 stores the generated trained prediction model 232 in the trained model parameters 33 (S42) and terminates the process.

[0088] In the above-described embodiment, AD conversion prediction was explained as the prediction task, but this embodiment can also be applied to other prediction tasks. Other prediction tasks will be described below.

[0089] Figure 17 shows other prediction tasks. Other prediction tasks include, for example, prediction of AD conversion period, prediction of amyloid-beta deposition, differentiation of AD / DLB disease, prediction of AD severity, prediction of tau abnormalities, and prediction of brain age. Below, the overview, evaluation method, dependent variable, and other details of each prediction task will be described.

[0090] AD conversion time prediction numerically predicts the time it will take for a subject to convert to AD in the future. Knowing when a subject is likely to convert to AD can be helpful in reviewing future lifestyle habits and developing long-term treatment plans. The evaluation is a regression, and the dependent variable is the length of time until AD conversion.

[0091] Amyloid-beta deposition prediction predicts amyloid-beta deposition from test results other than amyloid-beta (Aβ) (e.g., MRI images and subject information). For example, by estimating the distribution of amyloid-beta in a subject's brain, signs of amyloid-beta-related diseases can be estimated. Diseases related to amyloid-beta include, for example, mild cognitive impairment (MCI), mild cognitive impairment due to Alzheimer's disease, prodromal Alzheimer's disease, preclinical AD, Parkinson's disease, multiple sclerosis, cognitive decline, cognitive impairment, and neurodegenerative diseases such as amyloid-positive / negative diseases. The evaluation method is binary classification, and the dependent variable is positive or negative for amyloid-beta. The training data during learning can be determined by thresholds from the results of both amyloid PET (imaging) and cerebrospinal fluid (CSF) tests. Prediction result 1 can be positive or negative for amyloid-beta, and prediction result 2 can be the probability of positivity.

[0092] Differentiating between AD and DLB involves predicting whether a patient has Alzheimer's disease (AD) or Lewy body dementia (DLB). While DLB shares some similarities with AD in terms of brain atrophy, the treatment approaches differ, making differentiation crucial. The evaluation method is a binary classification, with the dependent variable being either AD or DLB. Prediction Result 1 can be either AD or DLB, and Prediction Result 2 can be the probability of AD and the probability of DLB.

[0093] AD severity prediction predicts the severity of Alzheimer's disease (mild, moderate, and severe) solely from brain imaging. By examining the difference between brain imaging and clinical progression, it can be used to improve treatment. The evaluation method is a numerical regression. The dependent variable is the Clinical Dementia Scale (CDR). A CDR of 0 indicates healthy individuals, a CDR of 0.5 indicates suspected dementia, a CDR of 1 indicates mild dementia, a CDR of 2 indicates moderate dementia, and a CDR of 3 indicates severe dementia. The independent variables are image features of the brain images.

[0094] Tau abnormality prediction predicts abnormal tau deposition from information other than tau PET. Tau is a protein expressed in nerve cells of the central and peripheral nervous systems, and tau abnormalities are thought to be a cause of neurodegenerative diseases such as Alzheimer's disease. The evaluation method is regression, and the dependent variable is tau PET SUVR (Standardized uptake value Ratio). One example of SUVR is to divide the sum of the SUV (tau accumulation) of four areas of cerebral gray matter (prefrontal cortex, anterior and posterior cingulate cortex, parietal lobe, and lateral temporal lobe) by the SUV of a specific reference area (e.g., cerebellum).

[0095] Brain age prediction predicts "brain age" from brain images. Even healthy individuals can use knowing their brain state (brain age) to review their lifestyle and perform other health checks. The evaluation method is regression, and the dependent variable is age. The training data during learning can be the actual age at the time the brain image was taken. The independent variables are the image features of the brain image.

[0096] According to this embodiment, the importance of the factors used as the basis for the prediction results of the prediction task can be visualized and presented, which can be used by physicians as a reference for diagnosing brain diseases and other conditions, thereby supporting physicians' diagnoses.

[0097] The diagnostic support device of this embodiment includes an acquisition unit that acquires subject data relating to the subject's brain, a prediction unit that predicts the subject's brain disease based on the subject data, an identification unit that identifies data items from the subject data that correspond to the subject data that contribute to the prediction result of the prediction unit, and an output unit that outputs the data items identified by the identification unit in association with prior knowledge regarding the brain disease.

[0098] In the diagnostic support device of this embodiment, the acquisition unit acquires image feature quantities calculated based on medical images of the subject's brain as subject data.

[0099] In the diagnostic support device of this embodiment, the image features include the degree of atrophy of a brain region.

[0100] In the diagnostic support device of this embodiment, the acquisition unit acquires subject information as subject data, which includes at least one of the examination information and clinical information related to the subject's brain.

[0101] The diagnostic support device of this embodiment includes a storage unit that stores prior knowledge as known evidence regarding brain diseases in association with data items related to the brain, and a contribution calculation unit that calculates the contribution of subject data that contributes to the prediction result. The identification unit identifies data items corresponding to the subject data based on the contribution and a predetermined contribution threshold, and the output unit reads the prior knowledge corresponding to the identified data items from the storage unit and outputs it.

[0102] In the diagnostic support device of this embodiment, the output unit outputs the degree of correlation between the data item and the prior knowledge.

[0103] The diagnostic support device of this embodiment includes a correlation calculation unit that calculates the correlation degree based on at least one of the contribution of subject data corresponding to the data item and the reliability of the prior knowledge as evidence.

[0104] In the diagnostic support device of this embodiment, the output unit outputs subject data corresponding to the data item.

[0105] The diagnostic support device of this embodiment includes a display unit that displays the data item, the subject data, and the prior knowledge corresponding to the data item in relation to each other, in order of the degree of contribution of the subject data corresponding to the data item to the prediction result.

[0106] The computer program of this embodiment causes the computer to perform the following processes: acquire subject data relating to the subject's brain; predict the subject's brain disease based on the subject data; identify data items from the subject data that contribute to the prediction of the brain disease; and output the identified data items in association with prior knowledge regarding the brain disease.

[0107] The diagnostic support method of this embodiment acquires subject data relating to the subject's brain, predicts the subject's brain disease based on the subject data, identifies data items from the subject data that contribute to the prediction result of the brain disease, and outputs the identified data items in association with prior knowledge regarding the brain disease. [Explanation of Symbols]

[0108] 1. Communication Network 10. User Interface Section 11. Image Input Function 12. Subject Information Input Function 13. Prediction Result Display Function 20 Processing Units 21 Image Processing Functions 211 Image reconstruction function 212 Organization division function 213 Anatomical Standardization Function 214 Smoothing function 215 Concentration value correction function 22 Image Feature Calculation Function 221 Atrophy Score Calculation Function 222 ROI identification function 223 Atrophy Degree Calculation Function 22a Input Layer 22b Convolutional layer 22c pooling layer 22d, 22e fully connected layer 22f output layer 23. Predictive Function 231 Scaling function 232 Pre-trained predictive models 24. Prediction basis calculation function 241 Gradient Calculation Function 242 Weighting calculation function 243 Adder 244 ReLU 25. Prior Knowledge Matching Function 26. Learning Processing Function 30 Database Department 31 ROI for Image Feature Calculation 32 Control Group Databases 33 Trained Model Parameters 34 Prior Knowledge Database 35 Brain Atlas Database 80 Diagnostic support device 81 CPU 82 ROM 83 RAM 84 GPU 85 Video Memory 86 Recording medium reading unit 90 Recording media 100 terminal devices 200 Diagnostic Support Servers 300 data servers

Claims

1. An acquisition unit that acquires subject data including subject information, which includes image features related to the subject's brain, as well as at least one of neuropsychological test information, clinical information, and biochemical test information. A prediction unit that predicts brain diseases of the subject based on the subject data, A memory unit that stores prior knowledge as known evidence regarding brain diseases, associated with data items of subject data related to the brain, A contribution calculation unit that calculates the degree to which the subject data contributes to the prediction result of the prediction unit, A selection unit selects whether or not to read prior knowledge corresponding to subject data from the storage unit based on the contribution calculated by the contribution calculation unit and a predetermined contribution threshold, A degree of agreement calculation unit calculates the degree of agreement by multiplying at least one of the contribution of the subject data and the confidence level indicating the strength of the prior knowledge as evidence by a weighting coefficient. An output unit reads the prior knowledge selected by the selection unit from the storage unit and outputs it. Equipped with, Diagnostic support device.

2. The acquisition unit is, Image features calculated based on medical images of the subject's brain are acquired as subject data. The diagnostic support device according to claim 1.

3. The aforementioned image features include the degree of atrophy in brain regions. The diagnostic support device according to claim 2.

4. The acquisition unit is, Subject information, including at least one of examination information and clinical information related to the subject's brain, is acquired as subject data. A diagnostic support device according to any one of claims 1 to 3.

5. The output unit is, Output the aforementioned degree of agreement. A diagnostic support device according to any one of claims 1 to 4.

6. The output unit is, Output the aforementioned subject data, A diagnostic support device according to any one of claims 1 to 5.

7. The system includes a display unit that displays the subject data and the corresponding prior knowledge in relation to the subject data, in order of the degree of contribution of the subject data to the prediction result. A diagnostic support device according to any one of claims 1 to 6.

8. On the computer, Subject data is obtained, which includes subject information that includes image features related to the subject's brain, as well as at least one of neuropsychological test information, clinical information, and biochemical test information. Based on the aforementioned subject data, the brain disease of the subject is predicted. Prior knowledge as known evidence regarding brain diseases is stored in the memory unit, corresponding to data items of subject data related to the brain. The degree of contribution to the prediction result of the aforementioned subject data is calculated, Based on the calculated contribution and a predetermined contribution threshold, it is selected whether or not to read prior knowledge corresponding to the subject data from the storage unit. The degree of agreement is calculated by multiplying at least one of the contribution of the subject data and the confidence level indicating the strength of the evidence of the prior knowledge by a weighting coefficient. The selected prior knowledge is read from the memory unit and output. A computer program that executes a process.

9. A computer, Subject data is obtained, which includes subject information that includes image features related to the subject's brain, as well as at least one of neuropsychological test information, clinical information, and biochemical test information. Based on the aforementioned subject data, the brain disease of the subject is predicted. Prior knowledge as known evidence regarding brain diseases is stored in the memory unit, corresponding to data items of subject data related to the brain. The degree of contribution to the prediction result of the aforementioned subject data is calculated, Based on the calculated contribution and a predetermined contribution threshold, it is selected whether or not to read prior knowledge corresponding to the subject data from the storage unit. The degree of agreement is calculated by multiplying at least one of the contribution of the subject data and the confidence level indicating the strength of the evidence of the prior knowledge by a weighting coefficient. The selected prior knowledge is read from the memory unit and output. How medical devices operate.

Images