Diagnostic support device and diagnostic support program

Abstract

Provided is a diagnostic assistance device or a diagnostic assistance program that is for identifying a disease with high accuracy without requiring special equipment or the knowledge and skills of a physician. This diagnostic assistance device is for providing assistance in identifying a disease on the basis of image data used for diagnosis, and comprises a calculation unit for calculating color tone-related values with respect to color tones of the image data on the basis of the luminances of the R, G, and B of each pixel forming said image data.

Description

[Technical Field]

[0001] The present invention relates to a diagnosis assistance apparatus and a diagnosis assistance program that assist a physician in identifying a disease based on image data. [Background Art]

[0002] As a method for a physician to identify a disease, for example, in the field of ophthalmology, an examination method for identifying a disease from the appearance of an affected area using a slit lamp microscope is widely used.

[0003] However, in an examination method using a slit lamp microscope, since a physician subjectively determines a disease based on the appearance of the affected area, there is a problem that the sensitivity and specificity of diagnosis depend on the knowledge and skill of the diagnosing physician. In particular, diagnosis becomes more difficult for rare diseases that are rarely encountered in actual clinical practice.

[0004] The aforementioned problem occurs not only in the field of ophthalmology as described above, but also in internal medicine, surgery, and other clinical departments where a physician identifies a disease from the appearance of an affected area.

[0005] As a method for solving this problem, for example, it has been considered to use an image analysis apparatus using optical coherence tomography as disclosed in Patent Document 1, but this method requires a special imaging device and has a problem of high cost. [Prior Art Documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2021-087762 [Summary of the Invention] [Problem to be Solved by the Invention]

[0007] This invention has been made in view of the aforementioned problems, and aims to provide a diagnostic support device or diagnostic support program for accurately identifying diseases without requiring the knowledge and skills of a diagnostic physician or special imaging equipment. [Means for solving the problem]

[0008] This invention was completed only after the inventors discovered that, from image data of an affected area to be diagnosed, if the color tone (more specifically, the balance of RGB brightness in the entire image data) is extracted as an objective numerical value from the appearance represented by the image data, diseases can be accurately identified based on this color tone-related value. In other words, the diagnostic assistance device according to the present invention is a diagnostic assistance device that assists in identifying a disease based on image data to be diagnosed, and is characterized by comprising a calculation unit that calculates a color-related value related to the color tone of the image data based on the R, G, and B luminances of each pixel forming the image data.

[0009] With the diagnostic support device configured in this way, a color-related value is calculated based on the R, G, and B luminances of each pixel forming the image data, so that the color tone of the image data to be diagnosed can be expressed as a numerical value. If color tones can be expressed numerically, diseases can be objectively identified based on these numerical values, regardless of the doctor's knowledge or skill.

[0010] Furthermore, since it only requires calculating the aforementioned color-related values ​​from image data, this can be achieved using inexpensive, general-purpose digital cameras or standard cameras equipped in devices for observing affected areas, such as endoscopes, and there is no need to prepare special and expensive imaging equipment.

[0011] The aforementioned color-related values ​​may be one or more of the mean, median, standard deviation, ratio of mean values, ratio of medians, and ratio of standard deviations of the R, G, and B luminances of each pixel forming the image data.

[0012] In order to quantify the values ​​while minimizing the influence of differences in brightness between images, it is useful for the calculation unit to calculate the color-related value by dividing the average value of one of the other two colors by the average value of the color with the largest average value among R, G, and B. For example, if the average value of R (Ra) is the largest, the color-related value will be calculated as Ga / Ra and / or Ba / Ra, which are the values ​​obtained by dividing the average values ​​of the other two colors (Ga, Ba) by Ra.

[0013] It would be even more convenient if the system further included a disease estimation unit that compares the aforementioned color-related values ​​with a predetermined threshold and estimates the disease name based on the comparison result.

[0014] The diagnostic support device may further include a threshold setting unit that sets a threshold based on the color-related values ​​calculated in advance for image data of the affected area in which the disease has been identified.

[0015] Furthermore, the diagnostic support device may also include a memory unit that stores the threshold values ​​obtained for each disease in association with each disease.

[0016] A specific embodiment of the present invention is one in which the threshold setting unit determines the threshold using an ROC curve.

[0017] This invention can be widely used to diagnose diseases from images of the affected area, but it is expected to be particularly effective in ophthalmic diseases, where lesions can often be observed and photographed optically. [Effects of the Invention]

[0018] According to the present invention, objective numerical values ​​called color-related values ​​can be extracted from the appearance of the affected area, which has conventionally been judged subjectively based on empirical rules, and diseases can be identified based on these values. As a result, it can be used as an aid for appropriate diagnosis by physicians, even for diseases that are difficult to differentiate even with the judgment of experienced physicians, and for rare diseases.

[0019] Among image diagnosis, particularly in telemedicine, when printing images captured by a camera or the like or projecting such images onto a screen, the fact that minute deviations from the actual tone and color exist for each piece of equipment used can sometimes become a critical problem in diagnosis. According to the present invention, since the tone of image data can be grasped as numerical values, the present invention is not affected by the performance of equipment used for printing or projection, and its effect can be expected even in telemedicine.

[0020] Furthermore, when a large amount of past diagnostic data is collected through combination with artificial intelligence, it is also possible to further improve the accuracy of diagnostic indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] [Figure 1] It is an overall schematic diagram of a diagnosis assistance apparatus according to an embodiment of the present invention. [Figure 2] It is a flowchart showing the procedure of threshold setting by the diagnosis assistance apparatus according to the present embodiment. [Figure 3] It is a flowchart showing the procedure of diagnosis assistance by the diagnosis assistance apparatus according to the present embodiment. [Figure 4] It is a photograph (original image data) of an affected area actually used in Example 1. [Figure 5] It is a diagram showing one of image data created from images used in Example 1. [Figure 6] It is a table showing numerical values calculated by a calculation unit for each piece of image data created in Example 1. [Figure 7] It is a graph showing an ROC curve when setting a threshold from the Ga / Ra values shown in FIG. 6 and a table showing the threshold. [Figure 8] It is a graph showing an ROC curve when setting a threshold from the Ba / Ra values shown in FIG. 6 and a table showing the threshold. [Figure 9] It is a graph showing ROC curves created for each of Ra, Ga and Ba shown in FIG. 6. [Figure 10]This table shows the numerical values ​​calculated by the calculation unit for each image data created in Example 2. [Figure 11] These graphs and tables show the results of a canonical plot created using three of the color-related values ​​calculated in Example 1. [Figure 12] This graph shows the ROC curve created for the discriminant function set in Example 3. [Figure 13] These graphs and tables show the results of a canonical plot created using two of the color-related values ​​calculated in Example 1. [Figure 14] This graph shows the ROC curve created for the discriminant function set in Example 3. [Figure 15] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 5. [Figure 16] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 15. [Figure 17] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 15. [Figure 18] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 15. [Figure 19] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 15. [Figure 20] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 15. [Figure 21] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 6. [Figure 22] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 21. [Figure 23] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 21. [Figure 24] This graph shows the ROC curve when setting a threshold value from the color-related values ​​shown in Figure 21. [Figure 25]This is an example of a photograph (original image data) of the affected area actually used in Example 7. [Figure 26] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 7. [Figure 27] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 7. [Figure 28] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 7. [Figure 29] This graph shows the ROC curve used when setting a threshold value from the color-related values ​​calculated in Example 7. [Figure 30] This graph shows the ROC curve used when setting a threshold value from the color-related values ​​calculated in Example 7. [Figure 31] This graph shows the ROC curve used when setting a threshold value from the color-related values ​​calculated in Example 7. [Figure 32] This graph shows the results of a canonical plot created using two of the color-related values ​​calculated in Example 7. [Figure 33] This graph shows the ROC curve created for the discriminant function set in Example 9. [Figure 34] This graph shows the results of a canonical plot created using two of the color-related values ​​calculated in Example 7. [Figure 35] This graph shows the ROC curve created for the discriminant function set in Example 9. [Figure 36] This graph shows the results of a canonical plot created using two of the color-related values ​​calculated in Example 7. [Figure 37] This graph shows the ROC curve created for the discriminant function set in Example 9. [Figure 38] This table shows the numerical values ​​calculated by the calculation unit for each image data used in Example 10. [Modes for carrying out the invention]

[0022] One embodiment of the present invention will be described below with reference to the drawings.

[0023] The diagnostic support device 100 according to this embodiment assists in identifying diseases based on image data of the affected area to be diagnosed (hereinafter also referred to as "diagnostic target image data") that is displayed in color using additive mixing with an RGB color model such as sRGB, which is an international standard set by the International Electrotechnical Commission (IEC). More specifically, the diagnostic support device 100 is structurally a dedicated or general-purpose computer equipped with, for example, a CPU, memory, input / output interface, etc.

[0024] The diagnostic support device 100 according to this embodiment includes, for example, as shown in Figure 1, an image processing unit 1 that receives raw image data of the affected area and creates image data from the raw image data; a calculation unit 2 that calculates a color tone related value representing the color tone of the image data from the brightness of each of the three colors R (Red), G (Green), and B (Blue) of each pixel forming the image data, based on the image data created by the image processing unit 1; and a disease estimation unit 3 that estimates a disease based on the color tone related value calculated by the calculation unit 2.

[0025] The image processing unit 1 creates image data by cropping a range containing only the lesion from the original image data obtained by capturing an image of the affected area using an imaging device such as a general-purpose digital camera. More specifically, the image processing unit 1 creates image data by cropping the range containing only the lesion from the original image data to be as large as possible. It is preferable that the size of the image data cropped from the original image data be as large as possible, because a larger number of pixels in the image data allows for more accurate extraction of the color characteristics of the image data as color-related values. The step of setting the range containing the lesion from the image of the affected area may be performed by a physician inputting instructions to the image processing unit 1 via an input interface, or the image processing unit 1 may be able to automatically determine the lesion using machine learning.

[0026] In order to more accurately reflect the tonal characteristics of the image data in the color-related values, in this embodiment, the image processing unit 1 deletes parts of the original image data that are overexposed due to reflections of, for example, the light source used during imaging, i.e., parts where the sum of the R, G, and B luminances exceeds a predetermined value.

[0027] The calculation unit 2, based on the R, G, and B luminances of each pixel forming the image data, calculates, for example, the average value of the luminance of each pixel for each of R, G, and B based on the image data to be diagnosed received from the image processing unit 1. While this average value itself may be used as the color-related value, in this embodiment, the calculation unit 2 performs further calculations using these average values ​​to calculate the ratio of the average value with the largest value to the average values ​​of the other colors, more specifically, the value obtained by dividing the average value of at least one of the other two colors by the average value of the color with the largest value, as the color-related value.

[0028] The disease estimation unit 3 estimates the disease name corresponding to the color-related value by comparing the color-related value calculated by the calculation unit 2 with a preset threshold. For example, if the calculated color-related value is smaller than a certain threshold, it is estimated that the color-related value corresponds to a normal finding. If the color-related value is greater than or equal to a certain threshold, it is estimated that the color-related value corresponds to disease A.

[0029] The threshold is set, for example, by the threshold setting unit 4 provided in the diagnostic support device 100. The threshold setting unit 4 receives image data of the affected area for which a disease has already been identified (also called past diagnostic image data) and disease data representing the name of the identified disease, calculates the color-related value, and sets a threshold for determining whether or not the disease is present based on the calculated color-related value.

[0030] More specifically, the threshold setting unit 4 calculates the color tone-related values ​​from the luminance of each of the three colors R, G, and B of each pixel forming the past diagnostic image data, in the same manner as the calculation unit 2 described above. Multiple combinations of these calculated color tone-related values ​​and disease data (disease name) for the image data from which the color tone-related values ​​were obtained are acquired, and a threshold for the color tone-related values ​​is statistically set from these combinations to distinguish between cases with a certain disease and cases without a certain disease.

[0031] In order to set a threshold, the threshold setting unit 4 needs to acquire multiple color-related value and disease data combinations for at least two types of disease data (for example, disease A and normal findings in this case). The more combinations of disease data and color-related values ​​that can be obtained for each of the two types of disease data, the more accurately the threshold for distinguishing between the two types of diseases can be determined. A more specific method by which the threshold setting unit 4 determines the threshold is to create ROC curves for various cases where the threshold is changed using the color-related values ​​of the two disease data to be distinguished, and then select the threshold that maximizes the AUC (Area under the curve) and 95% confidence interval of the ROC curve.

[0032] The diagnostic support device 100 further includes a storage unit 5 that stores, in association with the threshold set by the threshold setting unit 4 as described above, and disease data, which is the name of a disease that can be distinguished by this threshold (for example, disease A and normal findings).

[0033] An example of the operation of the diagnostic support device 100 configured in this way will be described below with reference to Figures 2 and 3. In this embodiment, we will describe the case in which the affected area to be diagnosed is identified as either disease A or normal.

[0034] First, let's explain the procedure for setting thresholds. Multiple raw image data of the affected area that has already been diagnosed as disease A, and multiple raw image data of the affected area that has already been diagnosed as having normal findings, are acquired and these raw image data are input into the diagnostic support device 100.

[0035] The image processing unit 1, upon receiving the raw image data, creates image data (past diagnostic image data) from the raw image data to be input to the threshold setting unit 4.

[0036] The threshold setting unit 4, having received multiple past diagnostic image data, calculates a color-related value for each past diagnostic image data (P1). For example, the threshold setting unit 4 obtains the color-related value by dividing the average value of at least one of the other two colors (e.g., Ga) by the average value of the average values ​​of the R luminance (Ra), G luminance (Ga), and B luminance (Ba) that has the largest value (e.g., Ra), and (e.g., Ga) as the color-related value (Ga / Ra).

[0037] The threshold setting unit 4 receives disease data corresponding to each past diagnostic image data (P2), and repeats this process until it obtains the necessary number of combinations of the calculated color-related value and the disease data corresponding to this color-related value to set a threshold (P3). After that, the threshold setting unit 4 sets a threshold to distinguish whether it is disease A or a normal finding based on the multiple combinations of color-related values ​​and disease data that have been collected (P4).

[0038] The thresholds set in this manner are output to the storage unit 5 and stored in the storage unit 5, linked to disease data that can be distinguished by these thresholds (in this case, disease A and normal findings).

[0039] Next, raw image data of the affected area, for example, to be diagnosed as either disease A or normal, is input into the diagnostic support device 100. At this time, the physician may also input information indicating that disease A is suspected for the raw image data of the affected area to be diagnosed.

[0040] As described above, when the raw image data to be diagnosed is input, the image processing unit 1 processes this raw image data in the same way as in the case of past diagnostic image data to create image data (image data to be diagnosed) (S1).

[0041] The diagnostic image data is sent to the calculation unit 2, which calculates the color tone related values ​​for this diagnostic image data (S2). The calculation unit 2 determines Ga / Ra as the color tone related values ​​in the same way as when the threshold setting unit 4 calculates the color tone related values. In this example, the physician has also input information that disease A is suspected for the original image data, so the calculation unit 2 calculates the color tone related values ​​for which a threshold has been set for disease A. When the same disease is suspected, the balance of R, G, and B, which represent the color tone of the image data, shows a very similar trend. Therefore, in this case, it is predicted that the color with the highest value among the average values ​​of R, G, and B luminances calculated by the calculation unit 2 will be R, the same as in past diagnostic image data, and the calculation unit 2 can determine Ga / Ra for the diagnostic image data in the same way as the threshold setting unit 4.

[0042] When the color-related values ​​calculated by the calculation unit 2 are sent to the disease estimation unit 3, the disease estimation unit 3 reads out thresholds (thresholds set for Ga / Ra) from the thresholds pre-stored in the memory unit 5 to distinguish between disease A and normal findings. The disease estimation unit 3 compares the color-related values ​​calculated for the diagnostic target image data with the thresholds read out from the memory unit 5 to estimate whether the affected area to be diagnosed is disease A or normal (S3).

[0043] The estimated results are displayed on the screen. This display helps the doctor determine whether the affected area being diagnosed is disease A or shows normal findings.

[0044] With the diagnostic support device 100 configured in this way, the calculation unit 2 calculates a color-related value that is related to the color tone of the image data, so that diseases can be identified based on the color-related value, which is an objective numerical value.

[0045] Furthermore, since the aforementioned color-related values ​​are the average value or ratio of the average values ​​of the R, G, and B luminances of each pixel forming the image data, they can be easily calculated using a general-purpose computer, and there is no need to prepare any special equipment.

[0046] As a color-related value, the average value of the color with the highest average brightness among the three colors R, G, and B is calculated by dividing the average value of at least one of the other two colors by the average value of that color. Therefore, it is possible to obtain a color-related value that is only related to color tone, while minimizing the influence of differences in brightness between image data caused by the shooting environment and other factors from the image data to be diagnosed.

[0047] Since the threshold is set using an ROC curve, the user (physician) can understand the degree of accuracy estimated by the disease estimation unit 3 from the AUC and 95% confidence interval values ​​of the ROC curve. As a result, the physician can make a diagnosis that takes the estimation accuracy into account.

[0048] The present invention is not limited to the embodiments described above. For example, the color-related value can be any value that can numerically represent the color characteristics of the image data. It may be the average value of the luminance of any one of the RGB colors of each pixel forming the image data, or it may be the median luminance of any one color or the standard deviation of any one color. Furthermore, the ratio of the median luminance or the standard deviation of the luminance calculated for R, G, and B may be used as the color-related value. Furthermore, in the embodiments described above, the threshold setting unit was described as calculating a threshold for the color tone-related values ​​based on the color tone-related values. However, the threshold setting unit may also set a function for the color tone-related values ​​or a discriminant expression (also called a discriminant function) that includes this function. As mentioned above, if the threshold setting unit sets a function or discriminant formula for color-related values, the disease estimation unit may use the function or discriminant formula set by the threshold setting unit to calculate which disease the diagnostic image data is for.

[0049] In the embodiments described above, we explained a case where the findings are assisted in determining whether the condition is disease A or normal. However, the invention is not limited to this case, and may, for example, be used to assist in determining whether the condition is disease A or disease B.

[0050] Furthermore, the diagnostic support device according to the present invention can be applied not only when diagnosing an affected area that is predicted to have a specific disease (disease A in the above embodiment), but also when diagnosing an affected area where the type of disease is not predicted. An example of the operation of the diagnostic support device when diagnosing an affected area where the type of disease is not predicted is described below.

[0051] If it is completely unknown what disease the affected area to be diagnosed has or whether it is normal, first prepare raw image data of the affected area where a disease has been previously identified, along with disease data corresponding to that raw image data, for multiple types of diseases, and input them into the diagnostic support device.

[0052] The threshold setting unit calculates two or more of several types of color-related values ​​for the image data created from these multiple original image data, such as the average value of the luminance of any one of the RGB colors of each pixel forming the aforementioned image data, the median value of the luminance of any one of the colors, the standard deviation of any one of the colors, and their ratios.

[0053] Then, the threshold setting unit sets a first threshold for these various color-related values, for example, a first threshold for distinguishing between disease A and disease B, which is Ra, the average value of R luminance. Furthermore, it sets a second threshold for distinguishing between disease A and disease C, which is the value obtained by dividing the average value of B luminance (Ba) by the average value of R luminance (Ra) (Ba / Ra). In this way, different threshold values ​​for different types of color-related values ​​are set for each of the multiple types of disease data obtained.

[0054] After these multiple thresholds are pre-stored in the memory unit, the original image data of the affected area to be diagnosed is input to the diagnostic support device. The calculation unit, upon receiving the image data created from the original image data, calculates several types of numerical values ​​as color-related values, such as two or more of the following: the average value of the luminance of any one of the RGB colors of each pixel forming the aforementioned image data, the median value of the luminance of any one of the colors, the standard deviation of any one of the colors, and their ratios.

[0055] The disease estimation unit compares each of the multiple color-related values ​​calculated by the calculation unit with threshold values ​​for the same type of color-related value stored in the memory unit, selects the most likely disease data from the comparison results with multiple thresholds, and outputs it as the estimation result.

[0056] As mentioned above, disease data may be the disease name, or it may be an abbreviation, symbol, or numerical value representing the disease name.

[0057] Since many ophthalmic diseases are relatively easy to image, it is considered that such diagnostic support devices are easily applicable. Specifically, examples include, but are not limited to, conjunctival amyloidosis, MALT lymphoma, subconjunctival hemorrhage (SCH), pterygium (PTG), scleritis, age-related macular degeneration (AMD), and central serous chorioretinopathy (CSC). Furthermore, it is not limited to the aforementioned ophthalmic diseases, but can be broadly applied to diseases in internal medicine, surgery, and other fields where, conventionally, doctors have diagnosed by observing the appearance of the affected area with the naked eye, a magnifying glass, or an endoscope and diagnoses the condition based on the color of the affected area.

[0058] In the embodiments described above, the case in which the disease estimation unit outputs only disease data was explained. However, the disease estimation unit may also output one or more of the following in addition to disease data: color-related values, thresholds, AUC, 95% confidence intervals, etc., for the diagnostic image data.

[0059] The diagnostic support device according to the present invention only needs to include a calculation unit, and does not necessarily need to include all of the aforementioned image processing unit, disease estimation unit, threshold setting unit, storage unit, etc. Some or all of the functions of these image processing unit, disease estimation unit, threshold setting unit, and storage unit, etc., may be performed by the user, such as a doctor or nurse.

[0060] The effects of the present invention can be achieved not only when using the aforementioned sRGB color model, but also when using other RGB color models.

[0061] Furthermore, it goes without saying that the present invention is not limited to these embodiments, and various modifications are possible without departing from its spirit. [Examples]

[0062] The present invention will be described in more detail below based on specific examples. However, the following examples are merely examples of the present invention, and the present invention is not limited to these examples.

[0063] <Example 1> In this first example, we investigated whether the diagnostic support device according to the present invention could identify two ophthalmic diseases (conjunctival amyloidosis and MALT lymphoma) that are considered difficult to distinguish even for experienced physicians.

[0064] First, as shown in Figure 4, multiple images of the affected area that had been definitively diagnosed as having one of these two diseases were prepared as raw image data. Figure 4(a) shows the actual images, and Figure 4(b) shows the diagnostic results for the image in Figure 4(a). The images shown in Figure 4 are some of the images used in Example 1. These original image data were input into a general-purpose PC, and by cropping the images, the image data shown in Figure 5 was created.

[0065] Next, each created image data and the corresponding disease data (in this case, the disease name) were input into the diagnostic support device. Then, the threshold setting unit, which received each image data and the corresponding disease data, calculated the average R, G, and B luminance values ​​of each pixel forming each image data, and instructed the calculation unit to calculate a value (Ga / Ra and Ba / Ra) by dividing the average value of the color with the highest average luminance (Ra) in each image data by the average value of the other colors in the same image data (Ga or Ba). The results of this calculation are shown in Figure 6.

[0066] Figure 7 shows the ROC curve created by the threshold setting unit using the relationship between Ga / Ra and actual diseases shown in Figure 6. The ROC curve shown in Figure 7 is used to determine the threshold for distinguishing between conjunctival amyloidosis and MALT lymphoma, with conjunctival amyloidosis set to 0 and MALT lymphoma set to 1. The ROC curve created for Ga / Ra had an AUC of 0.833 and a 95% confidence interval of 0.68-0.986. The data used to create the ROC curve and the threshold set using the ROC curve are shown in the table on the right side of Figure 7.

[0067] Figure 8 shows the ROC curve created similarly for Ba / Ra shown in Figure 6. The ROC curve for Ba / Ra shown in Figure 8 had an AUC of 0.948 and a 95% confidence interval of 0.865-1. Similar to Figure 7, the data used to create the ROC curve in Figure 8 and the threshold set using the ROC curve are shown in the table on the right side of Figure 8.

[0068] Furthermore, Figure 9 shows the results of creating ROC curves based on Ra, Ga, or Ba itself, as shown in Figure 6 above. These ROC curves in Figure 9 also showed high AUC and wide 95% confidence intervals. In particular, for Ba, the AUC was 0.857 and the 95% confidence interval was 0.717 - 0.997, indicating that a threshold with the same accuracy as the Ga / Ra and Ba / Ra cases could be set.

[0069] From the above results, it can be seen that by using one or more of the set thresholds for Ra, Ga, Ba, Ga / Ra, or Ba / Ra, which are color-related values ​​related to the color tone of image data, a highly reliable diagnostic index can be obtained for the image data being diagnosed.

[0070] Furthermore, using the threshold values ​​for Ga / Ra or Ba / Ra set by the ROC curve described above, 36 image data points were used as the diagnostic target image data, created by randomly dividing the past diagnostic image data used in this embodiment into four parts using image segmentation software. The accuracy rate for identifying diseases from these image data was confirmed to be very high, at 80% when Ga / Ra was used as the color-related value and 100% when Ba / Ra was used as the color-related value. The two diseases used as models in this Example 1 (conjunctival amyloidosis and MALT lymphoma) are diseases that are difficult for even experienced physicians to distinguish from their appearance, but it was confirmed that the diagnostic support device according to the present invention enables diagnosis with much higher accuracy than conventional methods.

[0071] <Example 2> The image data used in Example 1 was randomly divided into four parts, and using image data that was approximately 10% to 30% the size of the original image, the color tone association value for each image data was calculated in the same manner as in Example 1, and the difference between this value and the color tone association value calculated for the image data before and after division was examined.

[0072] For each randomly divided image data segment, the diagnostic support device was used to calculate Ga / Ra and Ba / Ra in the same manner as in Example 1. The calculation results are shown in Figure 10. The Ga / Ra color-related value calculated for each segmented image data was compared with the Ga / Ra of the unsegmented image data calculated in Example 1. Furthermore, the Ba / Ra color-related value calculated for each segmented image was compared with the Ba / Ra threshold set in Example 1. The diagnostic results from this comparison are shown in the leftmost column of Figure 10. As can be seen in Figure 10, the color tone association values ​​of each segmented image data were found to vary by approximately ±7% from the color tone association values ​​of the original image data from which each segmented image data was derived. The accuracy of the disease estimation results predicted from the magnitude of this variation was found to be very high, at 71% when using Ga / Ra and 79% when using Ba / Ra. These results show that, according to the diagnostic support device of the present invention, diseases can be identified with very high accuracy even when the diagnostic image data does not necessarily include the entire affected area. Furthermore, the results of this example showed that a larger image data size leads to higher accuracy in disease estimation.

[0073] <Example 3> In this example, discriminant analysis was performed using the three color-related values ​​Ra, Ga, and Ba from each past diagnostic image data in Example 1. In this discriminant analysis, a general discriminant analysis program was executed by the threshold setting unit to set one discriminant function that includes three functions showing the relationship between the three color-related values ​​and the disease data. Figure 11 shows a canonical plot visually representing the discriminant function set by the threshold setting unit, and Figure 12 shows the ROC curve created for the set discriminant function. The canonical plot shown in Figure 11 indicates that Ra and Ba showed large values ​​in differentiating conjunctival amyloidosis from MALT lymphoma, and the presence of opposing vectors for each group suggests that these values ​​were useful in the differential diagnosis. Figure 11 shows the ROC curves for conjunctival amyloidosis and MALT lymphoma, respectively. According to the ROC curves in Figure 11, the AUC is very high at 0.9905 for both diseases, suggesting that by setting a discriminant function using the three color-related values ​​Ra, Ga, and Ba in combination, conjunctival amyloidosis and MALT lymphoma can be distinguished with almost 100% accuracy. This third example demonstrates that, instead of setting thresholds for each color-related value, discriminant analysis can be performed by combining multiple types of color-related values ​​to accurately estimate the disease. Furthermore, the canonical plot allows for an understanding of the degree of contribution of each color in the differential diagnosis, making it easier to objectively differentiate between differences in disease color tones.

[0074] <Example 4> To verify the results obtained in Example 3 using other color-related values, Ga / Ra and Ba / Ra were calculated in the same manner as in Example 1. Discriminant analysis was then performed using these two color-related values ​​in the same manner as in Example 3, and a canonical plot and ROC curve were created based on the obtained discriminant function. The results are shown in Figures 13 and 14.

[0075] The results in Figure 13 show that discriminant functions using Ga / Ra and Ba / Ra can clearly differentiate conjunctival amyloidosis from MALT lymphoma. Furthermore, as shown in Figure 14, which illustrates the ROC curves for conjunctival amyloidosis and MALT lymphoma, the AUC was very high for both diseases, at 0.9286 for conjunctival amyloidosis and 0.9476 for MALT lymphoma. In other words, it was found that by setting a discriminant function using both Ga / Ra and Ba / Ra, conjunctival amyloidosis and MALT lymphoma can be distinguished with a very high probability.

[0076] <Example 5> As image data, we used past diagnostic image data that showed either subconjunctival hemorrhage (SCH), pterygium (PTG), or scleritis, which are anterior segment diseases, or normal findings. Using the same method as in Example 1, we set thresholds from these past diagnostic images and identified the diseases in these image data.

[0077] Figure 15 shows the calculated Ga / Ra and Ba / Ra ratios and their respective values ​​for this Example 5. Based on the values ​​in Figure 15, ROC curves were created to distinguish between PTG and scleritis, between scleritis and normal findings, and between scleritis and SCH, and thresholds were set for each to determine the thresholds for determining whether the findings were due to one of the three diseases or normal findings. These ROC curves are shown in Figures 16-20.

[0078] As can be seen from the shape of the graphs, the ROC curves shown in Figures 16-20 all exhibit very high specificity and sensitivity. Therefore, it is considered that by using thresholds set based on each of these ROC curves, it is possible to estimate with sufficiently high accuracy which of the three diseases and normal findings the target of diagnosis is.

[0079] Thus, if multiple combinations of threshold values ​​set for each of these color-related values ​​and disease data are available, the disease estimation unit can estimate which disease has the closest color tone among the diseases stored in the memory unit along with the threshold values ​​by comparing each color-related value obtained for diagnostic image data of unknown disease with these thresholds.

[0080] <Example 6> This embodiment is intended to confirm that the present invention can be applied to diseases other than those of the anterior region used in Examples 1 to 5. Specifically, past diagnostic image data of any of the following fundus diseases—age-related macular degeneration (AMD), central serous chorioretinopathy (CSC), or normal findings—were input into the diagnostic support device, and thresholds were set from these past diagnostic images using the same method as in Example 1. For fundus diseases, it is difficult to isolate only the lesion because the color difference between the lesion and the background is small, unlike in anterior region diseases. Therefore, in this embodiment, the image processing unit crops the central part of the photograph of the affected area to a predetermined size of approximately 4 mm square, and then removes the blood vessels to create and use image data. As a result, the image data used in this embodiment includes some parts other than the lesion.

[0081] The calculated Ga / Ra and Ba / Ra ratios and their respective values ​​for this Example 6 are shown in Figure 21. Based on the values ​​in Figure 21, ROC curves were created to determine thresholds for distinguishing between AMD and CSC, between AMD and normal findings, and between CSC and normal findings. These ROC curves are shown in Figures 22-24.

[0082] As can be seen from these ROC curves, the AUC of each of these ROC curves is very high, so by using thresholds set based on these curves, it is possible to estimate with high accuracy which of these diseases or normal findings the subject of diagnosis is.

[0083] <Example 7> In Example 7, we investigated whether the diagnostic support device according to the present invention could be used to identify diseases in fields other than ophthalmology, which are diagnosed by physicians observing the appearance of the affected area visually, with a magnifying glass, or with an endoscope. The study examined melanoma, pigmented nevi, and dermatofibroma as examples of disease cases. As shown in Figure 25, the images of these skin lesions were taken using a digital camera attached to a dermatoscope, which is a magnifying glass with approximately 10x magnification and a light, to observe the affected area.

[0084] As described above, each photograph of the lesion was used as the original image data, and the original image data was processed so that the abnormal area remained by separating it at the boundary between the normal and abnormal areas. For each image data obtained through this process, the mean value and standard deviation of the entire image data were calculated by the calculation unit based on the R, G, and B brightness of each pixel, in the same manner as in Example 1. The results are shown in Figures 26-28.

[0085] Incidentally, in the anterior segment ophthalmic images described above, because the amount of light is not constant, errors due to differences in brightness between image data were removed by dividing the average value of the R, G, and B luminances in a given image data by the average value of the color with the highest average luminance. However, in cases like the skin disease dealt with in this Example 7, where there are many black lesions, or when dealing with raw image data acquired with a device that can keep the light intensity constant, such as a dermatoscope, there is almost no difference in brightness between image data. Therefore, even if the average value and standard deviation of the R, G, and B luminances are used directly as color-related values, it is considered that errors due to differences in brightness between image data do not need to be considered.

[0086] As described above, the ROC curve created by the threshold setting unit is shown in Figure 29, based on the average values ​​of the R, G, and B brightness of each image data (Ra, Ga, Ba) and the confirmed diagnostic information of the affected area corresponding to each image data. The ROC curve shown in Figure 29 is used to determine the threshold for distinguishing between melanoma and pigmented nevus, with melanoma being set to 0 and pigmented nevus to 1.

[0087] The ROC curve generated for Ra had an AUC of 0.817 and a 95% confidence interval of 0.689-0.944. The ROC curve created for Ga had an AUC of 0.777 and a 95% confidence interval of 0.642-0.913. The ROC curve generated for Ba had an AUC of 0.800 and a 95% confidence interval of 0.663–0.937. From these results, it can be seen that by using one or more of the set thresholds for Ra, Ga, and Ba, which are color-related values ​​related to the color tone of image data, a highly reliable diagnostic index can be established to distinguish between melanoma and pigmented nevi in ​​the image data being diagnosed.

[0088] Similarly, a threshold for distinguishing between pigmented nevi and dermatofibromas was set by creating an ROC curve, with pigmented nevi set to 0 and dermatofibromas to 1. The ROC curve in this case is shown in Figure 30. The respective indicators for these ROC curves are as follows: The ROC curve generated for Ra had an AUC of 0.879 and a 95% confidence interval of 0.755-1. The ROC curve created for Ga had an AUC of 0.864 and a 95% confidence interval of 0.748-0.98. The ROC curve created for Ba had an AUC of 0.986 and a 95% confidence interval of 0.96-1. From these results, it can be seen that by using one or more of the set thresholds for Ra, Ga, and Ba, which are color-related values ​​related to the color tone of image data, a highly reliable diagnostic indicator can be used to distinguish between pigmented nevi and dermatofibromas in the image data being diagnosed.

[0089] Similarly, a threshold for distinguishing between dermatofibroma and melanoma was set by creating an ROC curve, with dermatofibroma assigned a value of 1 and melanoma a value of 0. The ROC curve in this case is shown in Figure 31. The respective indicators for these ROC curves are as follows: The ROC curve generated for Ra had an AUC of 0.977 and a 95% confidence interval of 0.938-1. The ROC curve created for Ga had an AUC of 0.973 and a 95% confidence interval of 0.924-1. The ROC curve generated for Ba had an AUC of 0.950 and a 95% confidence interval of 0.872-1. From these results, it can be seen that by using one or more of the set thresholds for Ra, Ga, and Ba, which are color-related values ​​related to the color tone of image data, a highly reliable diagnostic index can be established to distinguish between dermatofibroma and melanoma in the image data being diagnosed.

[0090] <Example 8> In Example 8, logistic regression analysis was performed to determine whether there was a significant difference in the average brightness of melanoma and pigmented nevus, pigmented nevus and dermatofibroma, and dermatofibroma and melanoma, based on the average brightness values ​​(Ra, Ga, Ba) of the calculated R, G, and B luminances of each image data and the definitive diagnostic information of the affected area corresponding to each image data as described above. Specifically, each of the aforementioned image data was first classified into three groups—melanoma, pigmented nevus, and dermatofibroma—based on the definitive diagnostic information, and the average values ​​of Ra, Ga, and Ba for each of these groups were calculated.

[0091] Next, logistic regression analysis was performed to determine the significant difference between the average R, G, and B brightness values ​​of past diagnostic images that had received a definitive diagnosis of melanoma, and the average brightness value of pigmented nevi. The results of this regression analysis showed that the p-value for brightness between melanoma and pigmented nevi was 8.3 × 10⁻⁶. -4 It was found that there was a significant difference in brightness. Next, the p-value for the mean brightness of pigmented nevi and dermatofibromas was 2.2 × 10⁻⁶. -6 Therefore, the p-value for the mean brightness between dermatofibroma and melanoma is 4.0 × 10⁻⁶. -8 In all cases, we obtained p-values ​​that were significantly smaller than the significance level (0.05), indicating that we were able to demonstrate with sufficient statistical significance that there is a significant difference in the mean brightness in these diseases.

[0092] Furthermore, these p-values ​​were found to be significantly lower than the significance level (0.025) adjusted using the Bonferroni method, which assumes, for example, that logistic regression analysis was repeated twice to compare these three groups at once.

[0093] <Example 9> Based on the average values ​​of R, G, and B luminance (Ra, Ga, Ba) of each image data calculated in Example 7, and the definitive diagnostic information of the affected area corresponding to each image data as described above, discriminant analysis was performed in the same manner as in Example 3 to distinguish between melanoma and pigmented nevus, pigmented nevus and dermatofibroma, and dermatofibroma and melanoma. In this discriminant analysis, a general discriminant analysis program was executed in the threshold setting unit to set one discriminant function for each combination of diseases to be discriminated, which includes three functions that show the relationship between three color-related values ​​and disease data. For the case of distinguishing between melanoma and pigmented nevus, a canonical plot visually representing the discriminant function set by the threshold setting unit is shown in Figure 32, and the ROC curve created for the set discriminant function is shown in Figure 33.

[0094] The canonical plot shown in Figure 32 indicates that Ra is particularly closely related to the differentiation between melanoma and pigmented nevi, and that it was useful in the differential diagnosis. Figure 33 shows the ROC curves for melanoma and pigmented nevus, respectively. According to the ROC curves in Figure 33, the AUC is high (0.8 or higher) for both diseases. Therefore, it is thought that by setting a discriminant function using the three color-related values ​​Ra, Ga, and Ba in combination, melanoma and pigmented nevus can be differentiated with a high degree of accuracy, which can be helpful in diagnosis.

[0095] Similarly, Figure 34 shows the results of the discriminant analysis to distinguish between pigmented nevi and dermatofibromas, and Figure 35 shows the ROC curves created for the set discriminant function. The canonical plot shown in Figure 34 indicates that Ba is particularly closely involved in differentiating between pigmented nevi and dermatofibromas, and that it was useful in this differentiation. Figure 35 shows the ROC curves for pigmented nevi and dermatofibroma, respectively. According to the ROC curves in Figure 35, the AUC is very high, above 0.98, for both diseases. Therefore, it is thought that by setting a discriminant function using the three color-related values ​​Ra, Ga, and Ba in combination, melanoma and pigmented nevi can be differentiated with a high degree of accuracy, which will be helpful in diagnosis.

[0096] Similarly, Figure 36 shows the results of the discriminant analysis to distinguish between dermatofibroma and melanoma, and Figure 37 shows the ROC curve created for the set discriminant function. Figure 37 shows the ROC curves for pigmented nevi and dermatofibroma, respectively. According to the ROC curves in Figure 37, the AUC is very high, above 0.97, for both diseases. Therefore, it is thought that by setting a discriminant function using the three color-related values ​​Ra, Ga, and Ba in combination, melanoma and pigmented nevi can be differentiated with a high degree of accuracy, which can be helpful in diagnosis.

[0097] <Example 10> Finally, for melanoma and pigmented nevi, two of the skin diseases used in Examples 7-9, we prepared completely different raw image data from the image data previously used as diagnostic images in Examples 7-9. When these were fed to the diagnostic support device according to the present invention, we verified whether the diseases could be distinguished using the methods in Examples 7 or 8. Figure 38 shows the numerical values ​​calculated by the calculation unit for each image data obtained using these images as raw image data. As an example, an ROC curve was created in the same manner as in Example 7. The AUC obtained from the ROC curve using Ra was 0.938, for Ga it was 1, and for Ba it was 0.875, indicating that it was possible to obtain highly reliable indicators for each parameter. Furthermore, using the same logistic regression analysis formula as in Example 8, we examined whether there was a significant difference in the mean values ​​of R, G, and B brightness for each image data obtained from each original image data between melanoma and pigmented nevi. The p-value was 0.01125, clearly indicating a significant difference in color tone. In all cases, it was possible to distinguish between the two diseases with sufficient statistical significance.

[0098] As explained above, it has been confirmed that the diagnostic support device according to the present invention can provide highly reliable information not only for ophthalmic diseases used in Examples 1 to 6, but also for dermatological diseases as described in Examples 7 and 8. Furthermore, these experimental results show that highly accurate diagnostic support is possible even for dermatological diseases, which have significantly different color tones from ophthalmic diseases. Therefore, it is considered that similarly high-accuracy diagnostic support can be provided for diseases in internal medicine, surgery, and other fields where doctors diagnose based on the color tone of the affected area by observing its appearance with the naked eye, a magnifying glass, or an endoscope. [Explanation of Symbols]

[0099] 100... Diagnostic support device 1. Image Processing Unit 2. Calculation Section 3. Disease Estimation Department 4. Threshold setting section 5...Storage section

Claims

1. A diagnostic support device that assists in identifying a disease based on image data to be diagnosed, The system includes a calculation unit that calculates a color tone-related value related to the color tone of the image data based on the R, G, and B luminances of each pixel forming the image data. The calculation unit calculates one or more of the following as color-related values: the average value, median value, standard deviation, ratio of average values, ratio of medians, and ratio of standard deviations of the R, G, and B luminances of each pixel forming the image data. The diagnostic assistance device calculates a value as the color tone-related value by dividing the average value of one of the other two colors by the average value of the color with the largest average value among R, G, and B.

2. The diagnostic assistance device according to claim 1, further comprising a disease estimation unit that estimates a disease name based on the result of comparing the aforementioned color-related values ​​with a predetermined threshold.

3. The diagnostic assistance device according to claim 2, further comprising a threshold setting unit that sets the threshold based on the color-related values ​​calculated in advance for image data of the affected area in which a disease has been identified.

4. The diagnostic assistance device according to claim 3, wherein the threshold setting unit determines the threshold using an ROC curve.

5. The diagnostic assistance device according to any one of claims 2 to 4, further comprising a storage unit that stores the threshold values ​​obtained for each disease in association with each disease.

6. The diagnostic aid device according to claim 1, wherein the disease is an ophthalmic disease.

7. A diagnostic assistance device that assists in identifying a disease based on image data to be diagnosed, comprising: a calculation unit that calculates a color-related value related to the color tone of the image data based on the R, G, and B luminances of each pixel forming the image data, the calculation unit that calculates one or more of the mean, median, standard deviation, mean ratio, median ratio, and standard deviation ratio of the R, G, and B luminances of each pixel forming the image data as the color-related value, and a calculation unit that calculates a value obtained by dividing the mean of one of the other two colors by the mean of the color with the largest mean among R, G, and B as the color-related value, comprising a computer that performs the function of a diagnostic assistance device.

Images