Ophthalmologic image processing program and ophthalmologic image processing device

Abstract

To provide an ophthalmologic image processing program and an ophthalmologic image processing device capable of efficiently supporting a medical care made by a medical worker, by properly processing an ophthalmologic image.SOLUTION: A control unit of the ophthalmologic image processing device acquires analysis information concerning a disease of a tissue contained in an ophthalmologic image. The control unit causes a storage device to store the analysis information in association with the ophthalmologic image from which the analysis information has been acquired. The control unit causes a display unit to display the ophthalmologic image. The control unit determines at least one of a folder for storing the analysis information and the ophthalmologic image and a priority order when causing the display unit to display the plurality of ophthalmologic images on the basis of at least one of the analysis information acquired with respect to the ophthalmologic image and analysis adequateness of the ophthalmologic image for acquiring the analysis information.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic image processing program and an ophthalmic image processing apparatus used to process ophthalmic images of a subject's eye. [Background technology]

[0002] In recent years, technologies have been proposed for acquiring various information useful for medical treatment by medical professionals (e.g., doctors, nurses, and medical technicians) based on ophthalmic images of a subject's eye. For example, an ophthalmic image processing system described in Patent Document 1 detects abnormal conditions from ophthalmic images and transmits ophthalmic images detected as having abnormal conditions to a specified destination. Also, a device described in Patent Document 2 detects findings from images generated based on received light signals for each wavelength. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-61238 [Patent Document 2] International Publication No. 2019 / 172206 Summary of the Invention [Problem to be solved by the invention]

[0004] Medical professionals need to interpret a variety of ophthalmic images. For example, when interpreting multiple ophthalmic images, the multiple ophthalmic images often contain a mixture of images of tissues that are likely to have disease and images of tissues that are unlikely to have disease. Furthermore, the multiple ophthalmic images may contain images that are not suitable for analyzing information about tissue diseases. Therefore, simply presenting a variety of ophthalmic images to medical professionals in a uniform manner often does not adequately assist the medical professionals in their medical treatment.

[0005] A typical object of the present disclosure is to provide an ophthalmic image processing program and an ophthalmic image processing device that can more efficiently assist medical professionals in performing medical examinations by appropriately processing ophthalmic images. [Means for solving the problem]

[0006] An ophthalmic image processing program provided by a typical embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing device that processes ophthalmic images, which are images of the tissues of a test eye. When the ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device, the ophthalmic image processing program causes the ophthalmic image processing device to execute an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic imaging device, an analytical information acquisition step of acquiring analytical information related to a disease of the tissue shown in the ophthalmic image acquired in the image acquisition step, an information storage step of storing the analytical information in a storage device in association with the ophthalmic image from which the analytical information was obtained, and a display step of displaying the ophthalmic image on a display unit. At least one of a folder that stores the analytical information and the ophthalmic images in the information storage step and a priority for displaying multiple ophthalmic images on the display unit in the display step is determined based on at least one of the analytical information acquired in the analytical information acquisition step and the analytical suitability of the ophthalmic images for acquiring the analytical information in the analytical information acquisition step.

[0007] An ophthalmic image processing device provided by a typical embodiment of the present disclosure is an ophthalmic image processing device that processes ophthalmic images, which are images of the tissues of a test eye, and a control unit of the ophthalmic image processing device executes an image acquisition step of acquiring ophthalmic images captured by an ophthalmic imaging device, an analytical information acquisition step of acquiring analytical information related to a disease of the tissue shown in the ophthalmic image acquired in the image acquisition step, an information storage step of storing the analytical information in a storage device in association with the ophthalmic image from which the analytical information was obtained, and a display step of displaying the ophthalmic image on a display unit, and determines at least one of a folder for storing the analytical information and the ophthalmic images in the information storage step and a priority for displaying multiple ophthalmic images on the display unit in the display step based on at least one of the analytical information acquired in the analytical information acquisition step and the analytical suitability of the ophthalmic images for acquiring the analytical information in the analytical information acquisition step.

[0008] The ophthalmologic image processing program and ophthalmologic image processing device according to the present disclosure can more efficiently assist medical professionals in their medical treatment. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of a mathematical model construction device 1, an ophthalmologic image processing device 21, and ophthalmologic image capturing devices 11A and 11B. [Figure 2] FIG. 2 is an explanatory diagram for explaining analytical information (probability of existence of each disease) output by the mathematical model of this embodiment. [Figure 3] 4 is a flowchart of ophthalmologic image processing executed by the ophthalmologic image processing apparatus 21 of the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of an ophthalmologic image 30 from which analysis information has been acquired. [Figure 5] FIG. 10 is a diagram showing an example of the data structure of list data 35. [Figure 6] 4 is a flowchart of a display process executed during ophthalmologic image processing according to the first embodiment. [Figure 7] FIG. 4 is a diagram showing an example of an image interpretation screen 40 when an ophthalmologic image in a normal folder is displayed in the first embodiment. [Figure 8] FIG. 10 is a diagram showing an example of an image interpretation screen 40 when an ophthalmologic image in an inappropriate image folder is displayed in the first embodiment. [Figure 9] 10 is a flowchart of ophthalmologic image processing executed by an ophthalmologic image processing apparatus 21 according to a second embodiment. [Figure 10] FIG. 10 is an explanatory diagram for explaining an example of a display order when a plurality of ophthalmologic images are displayed by the ophthalmologic image processing apparatus 21 of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Summary> An ophthalmic image processing device according to the present disclosure processes ophthalmic images, which are images of tissues of a subject's eye. An ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device. The ophthalmic image processing device (control unit) executes an image acquisition step, an analysis information acquisition step, an information storage step, and a display step. In the image acquisition step, the control unit acquires an ophthalmic image captured by an ophthalmic imaging device. In the analysis information acquisition step, the control unit acquires analysis information related to a disease of the tissue depicted in the ophthalmic image acquired in the image acquisition step. In the information storage step, the control unit stores the analysis information in a storage device in association with the ophthalmic image from which the analysis information was obtained. In the display step, the control unit displays the ophthalmic image on a display unit. At least one of a folder for storing the analysis information and the ophthalmic image in the information storage step and a priority for displaying multiple ophthalmic images on the display unit in the display step is determined based on at least one of the analysis information acquired in the analysis information acquisition step and the analysis suitability of the ophthalmic image for acquiring the analysis information in the analysis information acquisition step.

[0011] According to the ophthalmic image processing program and ophthalmic image processing device exemplified in the present disclosure, at least one of the folder for storing information (analysis information and ophthalmic images) and the display priority of multiple ophthalmic images is determined based on at least one of the disease analysis information obtained by analyzing the ophthalmic images and the analysis suitability of the ophthalmic images. For example, when the folder for storing information is determined based on the analysis information, medical professionals can easily improve the efficiency of medical treatment by treating information in a specific folder determined based on the analysis information differently from information in other folders. When the folder for storing information is determined based on the analysis suitability, medical professionals can selectively check information on ophthalmic images with high analysis suitability or information on ophthalmic images with low analysis suitability by checking the information in a specific folder. When the display priority of multiple ophthalmic images is determined based on the analysis information, medical professionals can also check multiple ophthalmic images according to an appropriate priority corresponding to the analysis information. When the display priority of ophthalmic images is determined based on the analysis suitability, medical professionals can also check multiple ophthalmic images according to the priority corresponding to the analysis suitability. Therefore, the technology of the present disclosure makes it easier to efficiently assist medical professionals in their medical treatment.

[0012] The analytical information may include at least one of various pieces of information related to diseases of tissues shown in the ophthalmologic image. For example, information indicating the probability that each of a plurality of diseases (which may include cases of normal eyes without any disease) is predicted to be present in the tissues shown in the ophthalmologic image may be used as the analytical information. In this case, medical professionals may be able to determine in advance the type of disease predicted to be highly likely to be present in the tissues from the analytical information. Information on the probability that some disease is predicted to be present in the tissues shown in the ophthalmologic image may also be used as the analytical information. Prediction information on the severity of a disease present in the tissue may also be used as the analytical information. Note that the analytical information may include not only information indicating that a disease is predicted to be present, but also information indicating that the probability of the disease being present is low.

[0013] In the analytical information acquisition step, the control unit may acquire analytical information by inputting the ophthalmic image to a mathematical model trained by a machine learning algorithm. The mathematical model may be trained by the machine learning algorithm so that, when the ophthalmic image is input, the mathematical model outputs analytical information related to a disease of a tissue shown in the input ophthalmic image. In this case, by using the mathematical model trained with a plurality of training data, useful analytical results can be easily acquired even in cases where it is difficult to appropriately acquire analytical results using a function that does not use a machine learning algorithm.

[0014] However, the control unit may acquire the analysis result from the ophthalmological image without using a machine learning algorithm in the analysis information acquisition step. For example, even in the case where the analysis result is acquired by performing image processing on the ophthalmological image using a function without using a machine learning algorithm, at least a part of the techniques exemplified in the present disclosure can be applied.

[0015] The control unit may also output the ophthalmic image to another device (e.g., a server) that stores a program for realizing the mathematical model. The device to which the ophthalmic image is output may input the ophthalmic image input from the ophthalmic image processing device into the mathematical model to obtain an analysis result, and output the obtained analysis result to the ophthalmic image processing device. The control unit may obtain medical data input from the device to which the ophthalmic image is output.

[0016] An adequacy determination step may be further executed in which an analysis adequacy of the ophthalmic image acquired in the image acquisition step is acquired and whether the acquired analysis adequacy satisfies a standard. In the information storage step, information (ophthalmic image data and analysis information acquired about the ophthalmic image) about an ophthalmic image whose analysis adequacy is determined not to satisfy the standard in the adequacy determination step (hereinafter referred to as an "inappropriate ophthalmic image") may be stored in a folder (hereinafter referred to as an "inappropriate image folder") different from the folder storing information about an ophthalmic image whose analysis adequacy is determined to satisfy the standard.

[0017] In this case, medical personnel can efficiently check only the information about the inappropriate ophthalmic images by checking the information stored in the inappropriate image folder, which makes it easier to efficiently assist medical personnel in their medical treatment.

[0018] The specific method for storing the information of the inappropriate ophthalmic images in the inappropriate image folder can be selected as appropriate. For example, the control unit may automatically store the information of the inappropriate ophthalmic images in the inappropriate image folder. Furthermore, when a user inputs an instruction to store the information of the inappropriate ophthalmic images, the control unit may extract information of one or more inappropriate ophthalmic images from the information of the multiple ophthalmic images and store the extracted information in the inappropriate image folder.

[0019] The control unit may acquire analysis information in the analysis information acquisition step even for an ophthalmological image whose analysis suitability is determined not to satisfy the standard in the suitability determination step (inappropriate ophthalmological image). The control unit may output the analysis information acquired for the inappropriate ophthalmological image in association with the inappropriate ophthalmological image for which the analysis information was acquired.

[0020] In this case, medical personnel can easily check the validity of the analysis information by referring to the analysis information acquired for the inappropriate ophthalmologic image.

[0021] A specific method for outputting the inappropriate ophthalmic images in association with the analysis information can be selected as appropriate. For example, the control unit may output list data (e.g., CSV data) in which the acquired analysis information is associated with each of the multiple inappropriate ophthalmic images stored in the inappropriate image folder. In this case, by referring to the list data when checking the multiple inappropriate ophthalmic images stored in the inappropriate image folder, medical professionals can more easily confirm whether the analysis information for each of the multiple inappropriate ophthalmic images was appropriately acquired. When outputting the list data, the list data for the inappropriate ophthalmic images and the list data for the appropriate ophthalmic images (ophthalmic images whose analysis appropriateness meets the standard) may be output separately. In this case, various checks for each of the inappropriate ophthalmic images and the appropriate ophthalmic images can be performed more efficiently. However, one list data may also be output collectively regardless of whether the analysis appropriateness meets the standard. Furthermore, when displaying an inappropriate ophthalmic image, the control unit may display the analysis information acquired for the inappropriate ophthalmic image to be displayed.

[0022] In the suitability determination step, the control unit may acquire multiple types of analysis suitability for the ophthalmic image and determine whether at least one of the acquired multiple types of analysis suitability satisfies a standard corresponding to the type. The control unit may output, for the ophthalmic image for which at least one analysis suitability is determined not to satisfy the standard in the suitability determination step, basis information indicating the type of analysis suitability that did not satisfy the standard, in association with the ophthalmic image.

[0023] In this case, medical personnel can easily understand the basis for determining that an ophthalmic image is inappropriate (the type of analysis suitability that did not meet the criteria) based on the basis information, which makes it easier to, for example, confirm the validity of analysis information obtained for an inappropriate ophthalmic image.

[0024] A specific method for outputting inappropriate ophthalmic images in association with the basis information can also be selected as appropriate. For example, the control unit may output list data (e.g., CSV data, etc.) in which basis information is associated with each of the multiple inappropriate ophthalmic images stored in the inappropriate image folder. In this case, medical professionals can easily check the basis information for each of the multiple inappropriate ophthalmic images by referring to the list data when checking the multiple inappropriate ophthalmic images stored in the inappropriate image folder. Furthermore, the control unit may display the basis information for the inappropriate ophthalmic images when displaying the inappropriate ophthalmic images.

[0025] The specific mode of the analysis suitability acquired for an ophthalmic image can be selected as appropriate. For example, the control unit may acquire, as the analysis suitability of the ophthalmic image, a pixel value histogram (e.g., a density histogram) that graphs the number of pixels having the same pixel value (e.g., density value) in the ophthalmic image. The control unit may determine whether the ophthalmic image is suitable for acquiring analysis information (whether the ophthalmic image is suitable) based on whether the pixel value histogram satisfies a criterion. In this case, whether the ophthalmic image is suitable is appropriately determined based on the pixel values ​​of the ophthalmic image. The control unit may also acquire, as the analysis suitability of the ophthalmic image, at least one of the degree of flare in the ophthalmic image, the amount of exposure, the suitability of a mask applied to the ophthalmic image, and the like. Furthermore, the signal strength of the ophthalmic image or an index indicating the quality of the signal (e.g., a signal strength index (SSI) or a quality index (QI)) may be acquired as the analysis suitability of the ophthalmic image. In addition, at least one of the ratio of the noise level to the signal level of the image (SNR (Signal to Noise Ratio)), the background noise level, the image contrast, etc. may be acquired as the analysis suitability of the ophthalmic image. In addition, the shooting conditions when the ophthalmic image was captured by the ophthalmic image capturing device may be acquired as the analysis suitability of the ophthalmic image.

[0026] Furthermore, the criteria (e.g., thresholds for determining whether or not the criteria are met) corresponding to each of the multiple types of analysis suitability may be predetermined fixed values ​​or may be set (changed) by the user.

[0027] In the information storing step, the control unit may store the information about the ophthalmic image in a folder among the plurality of folders corresponding to the analysis information acquired about the ophthalmic image. In this case, a medical professional can efficiently identify the ophthalmic image for which specific analysis information was acquired by checking the information in the folder corresponding to the analysis information.

[0028] A specific method for storing information in a folder corresponding to the analysis information can be selected as appropriate. For example, information indicating a distribution of the probability that each of a plurality of diseases is predicted to be present in tissues captured in an ophthalmologic image may be acquired as the analysis information. In this case, the control unit may store information about the ophthalmologic image in a folder corresponding to one or more diseases whose presence probability indicated by the analysis information is high, among a plurality of folders provided for each of the plurality of diseases. In this case, a medical professional can efficiently identify ophthalmologic images in which a particular disease is likely to be present by checking the information in the folder for the particular disease.

[0029] Furthermore, predicted information on the severity of a disease present in tissue may be acquired as the analysis information. In this case, the control unit may store information on ophthalmic images whose severity indicated by the analysis information is higher than a standard in a specific folder. In this case, a medical professional can efficiently identify ophthalmic images whose severity is predicted to be high by checking the information in the specific folder. Note that when the control unit stores information on ophthalmic images whose severity indicated by the analysis information is higher than a standard in the specific folder, the control unit may execute a notification process (e.g., a notification display process on a display unit, a notification email transmission process, etc.) to prompt the medical professional to check the ophthalmic images in the specific folder.

[0030] The control unit may determine a priority order for displaying the plurality of ophthalmologic images on the display unit in the display step based on the analysis information of the ophthalmologic images acquired in the analysis information acquisition step. In this case, the medical professional can check the plurality of ophthalmologic images according to an appropriate priority order based on the analysis information. This makes it easier to efficiently assist the medical professional in medical treatment.

[0031] The display priority of the ophthalmologic images can be selected as appropriate. For example, the control unit may switch between multiple ophthalmologic images and sequentially display them on the display unit. In this case, the display priority of the ophthalmologic images may be the priority when the multiple ophthalmologic images are switched and displayed on the display unit.

[0032] The control unit may also display information related to a plurality of ophthalmological images side by side. In this case, the priority of displaying the ophthalmological images may be the priority when arranging the plurality of ophthalmological images. The control unit may also sort the information related to the plurality of ophthalmological images to be displayed side by side according to the priority.

[0033] The control unit may also issue a notification (for example, by displaying a notification on the display unit or by sending an email prompting the medical staff to check the ophthalmologic image) in order of priority, starting with the ophthalmologic image with the highest priority.

[0034] The analysis information acquired in the analysis information acquisition step may include predicted information on the probability of disease presence in tissues shown in the ophthalmologic images. The control unit may prioritize display of ophthalmologic images for which the analysis information indicates a high probability of disease presence over display of ophthalmologic images for which the analysis information indicates a low probability of disease presence. In this case, medical personnel can prioritize review of ophthalmologic images predicted to have a high probability of disease presence. This further facilitates improvement in the efficiency of medical treatment.

[0035] The analysis information acquired in the analysis information acquisition step may include predicted information on the severity of a disease of the tissue shown in the ophthalmologic image. The control unit may prioritize display of ophthalmologic images in which the analysis information indicates a high degree of disease severity over priority display of ophthalmologic images in which the analysis information indicates a low degree of disease severity. In this case, medical personnel can prioritize review of ophthalmologic images in which the disease severity is predicted to be high. This further facilitates improvement in the efficiency of medical treatment.

[0036] The analysis information acquired in the analysis information acquisition step may include information indicating the distribution of the probability that each of a plurality of types of disease is predicted to exist in the tissues shown in the ophthalmologic image. An attention ranking may be assigned to each of the plurality of types of disease. The control unit may set a display priority for the ophthalmologic image according to the attention ranking of the disease with a high probability of existence indicated by the analysis information. In this case, medical personnel can prioritize viewing ophthalmologic images of tissues predicted to contain a disease with a high attention ranking. This further facilitates improving the efficiency of medical treatment.

[0037] The control unit may switch between determining whether or not to determine the priority of the ophthalmic images to be displayed on the display unit in the display step based on the analysis information of the ophthalmic images acquired in the analysis information acquisition step. The analysis information acquired for the ophthalmic images is not always accurate. Therefore, even among ophthalmic images with a low priority set based on the analysis information, there may be ophthalmic images that require review by a medical professional. As described above, in the first mode in which the display priority is determined based on the analysis information, the medical professional can efficiently review the multiple ophthalmic images according to an appropriate priority based on the analysis information. On the other hand, in the second mode in which the priority is not determined based on the analysis information, the medical professional can carefully review each ophthalmic image regardless of the priority determined based on the analysis information. Therefore, by switching between the first mode and the second mode, the medical professional can review the ophthalmic images in an appropriate procedure according to various situations.

[0038] The specific method for switching between the first mode and the second mode can be selected as appropriate. For example, the control unit may switch between the first mode and the second mode in response to an instruction input by a user. In this case, a medical professional can check an ophthalmologic image in a more appropriate procedure by switching between the first mode and the second mode to an appropriate mode depending on various situations.

[0039] The control unit may also determine, as an image for which the priority order according to the analysis information is determined, an ophthalmic image that has been displayed on the display unit in the display step, and exclude, as an image for which the priority order according to the analysis information is determined, an ophthalmic image that has not yet been displayed on the display unit in the display step. In this case, the medical staff can carefully check ophthalmic images that have not been displayed on the display unit in the past, and can efficiently check, in order of priority, ophthalmic images that have been displayed on the display unit in the past.

[0040] The control unit may also determine, as the images for which the priority order is to be determined according to the analysis information, the ophthalmological images for which the user has given an instruction to set a priority order, and exclude, as the images for which the priority order is to be determined according to the analysis information, the ophthalmological images for which the user has not given an instruction to set a priority order. In this case, the medical staff can efficiently check the ophthalmological images for which the priority order is to be set and can carefully check the other ophthalmological images.

[0041] In the display step, the control unit may randomly display an ophthalmic image whose priority is different from the next priority while sequentially displaying the multiple ophthalmic images according to the priority determined based on the analysis information. In this case, the ophthalmic image whose priority is different from the next priority is randomly displayed among the multiple ophthalmic images sequentially displayed based on the priority. This makes it easier for medical professionals to check the multiple ophthalmic images displayed sequentially while maintaining high concentration.

[0042] In the display step, when the control unit sequentially displays the multiple ophthalmic images on each of the multiple display units, the control unit may change the display order of the multiple ophthalmic images for each display unit. When reviewing multiple ophthalmic images, medical professionals' concentration tends to decrease over time. In response to this, by changing the display order of the multiple ophthalmic images for each display unit that displays the images, it becomes easier to prevent the multiple medical professionals from focusing on each ophthalmic image when reviewing the multiple ophthalmic images.

[0043] <Embodiment> (Device configuration) A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, this embodiment uses a mathematical model construction device 1, an ophthalmic image processing device 21, and an ophthalmic image capturing device 11 (11A, 11B). The mathematical model construction device 1 constructs a mathematical model by training the mathematical model using a machine learning algorithm. A program for realizing the constructed mathematical model is stored in a storage device 24 of the ophthalmic image processing device 21. The ophthalmic image processing device 21 inputs an ophthalmic image to the mathematical model to obtain an analysis result of the input ophthalmic image. The ophthalmic image capturing device 11 (11A, 11B) captures an ophthalmic image, which is an image of the tissue of the subject's eye.

[0044] As an example, a personal computer (hereinafter referred to as "PC") is used as the ophthalmic image processing device 21 of this embodiment. However, a device other than a PC may function as the ophthalmic image processing device. For example, the ophthalmic image capturing device 11B that captures ophthalmic images may function as the ophthalmic image processing device. In this case, the ophthalmic image capturing device 11B captures ophthalmic images and processes the captured ophthalmic images itself, thereby assisting medical professionals in performing medical examinations. Furthermore, control units of multiple devices may work together to process ophthalmic images. For example, the CPU of the PC and the CPU 13B of the ophthalmic image capturing device 11B may work together to process ophthalmic images. Furthermore, control units of multiple PCs may work together to process ophthalmic images. The multiple devices may include at least one of a smartphone, a mobile terminal, etc.

[0045] A PC is used as the mathematical model construction device 1 of this embodiment. As will be described in detail later, the mathematical model construction device 1 constructs a mathematical model by training the mathematical model using ophthalmic images (hereinafter referred to as "training ophthalmic images") acquired from the ophthalmic image capturing device 11A and disease information corresponding to the training ophthalmic images. However, the device that can function as the mathematical model construction device 1 is not limited to a PC. For example, the ophthalmic image capturing device 11A may function as the mathematical model construction device 1. Furthermore, control units of multiple devices (for example, the CPU of a PC and the CPU 13A of the ophthalmic image capturing device 11A) may cooperate to construct a mathematical model.

[0046] The ophthalmic imaging device 11 (11A, 11B) of this embodiment uses a fundus camera that captures a two-dimensional front image of the fundus of the subject's eye. However, devices that can function as an ophthalmic imaging device are not limited to fundus cameras. For example, at least one of an OCT device that captures images of the fundus tissue of the subject's eye, a scanning laser ophthalmoscope (SLO), an infrared camera, etc. may be used as the ophthalmic imaging device. It is also possible to use an ophthalmic imaging device that captures images of the tissue of the subject's eye other than the fundus tissue (for example, tissue of the anterior segment of the subject's eye, etc.).

[0047] In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that a controller other than a CPU may be used for at least some of the various devices. For example, a GPU may be used as a controller to speed up processing.

[0048] The mathematical model construction device 1 will now be described. The mathematical model construction device 1 is installed, for example, at a manufacturer that provides an ophthalmic image processing device 21 or an ophthalmic image processing program to users. The mathematical model construction device 1 includes a control unit 2 that performs various control processes and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller responsible for control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a mathematical model construction program for executing the mathematical model construction process. The communication I / F 5 also connects the mathematical model construction device 1 to other devices (for example, an ophthalmic image capturing device 11A and an ophthalmic image processing device 21, etc.).

[0049] The mathematical model construction device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by a user to input various instructions to the mathematical model construction device 1. The operation unit 7 can be, for example, at least one of a keyboard, a mouse, a touch panel, etc. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 7. The display device 8 displays various images. The display device 8 can be any of various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.).

[0050] The mathematical model construction device 1 can acquire data of ophthalmic images (hereinafter, sometimes simply referred to as "ophthalmic images") from the ophthalmic image capturing device 11A. The mathematical model construction device 1 may acquire the data of ophthalmic images from the ophthalmic image capturing device 11A by, for example, at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), etc.

[0051] The ophthalmic image processing device 21 will now be described. The ophthalmic image processing device 21 is disposed, for example, in a facility (e.g., a hospital or health checkup facility) where a diagnosis or examination is performed on a subject. The ophthalmic image processing device 21 includes a control unit 22 that performs various control processes and a communication I / F 25. The control unit 22 includes a CPU 23 that is a controller responsible for control, and a storage device 24 that can store programs, data, and the like. The storage device 24 stores an ophthalmic image processing program for executing the ophthalmic image processing described below. The ophthalmic image processing program includes a program for realizing the mathematical model constructed by the mathematical model construction device 1. The communication I / F 25 connects the ophthalmic image processing device 21 to other devices (e.g., the ophthalmic image capturing device 11B and the mathematical model construction device 1, etc.).

[0052] The ophthalmologic image processing device 21 is connected to an operation unit 27 and a display device 28. As with the operation unit 7 and display device 8 described above, various devices can be used for the operation unit 27 and the display device 28.

[0053] The ophthalmological image processing device 21 can acquire ophthalmological images from the ophthalmological image capturing device 11B. The ophthalmological image processing device 21 may acquire ophthalmological images from the ophthalmological image capturing device 11B by at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), etc. The ophthalmological image processing device 21 may also acquire a program or the like that realizes the mathematical model constructed by the mathematical model construction device 1 via communication or the like.

[0054] The ophthalmic image capturing devices 11A and 11B will be described. As an example, in this embodiment, a case will be described in which an ophthalmic image capturing device 11A that provides ophthalmic images to the mathematical model construction device 1 and an ophthalmic image capturing device 11B that provides ophthalmic images to the ophthalmic image processing device 21 are used. However, the number of ophthalmic image capturing devices used is not limited to two. For example, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from multiple ophthalmic image capturing devices. Furthermore, the mathematical model construction device 1 and the ophthalmic image processing device 21 may acquire ophthalmic images from a single common ophthalmic image capturing device.

[0055] The ophthalmic imaging device 11 (11A, 11B) includes a control unit 12 (12A, 12B) that performs various control processes, and an ophthalmic imaging section 16 (16A, 16B). The control unit 12 includes a CPU 13 (13A, 13B) that is a controller responsible for control, and a storage device 14 (14A, 14B) that can store programs, data, etc. When the ophthalmic imaging device 11 performs at least a part of the ophthalmic image processing described below, it goes without saying that at least a part of the ophthalmic image processing program for performing the ophthalmic image processing is stored in the storage device 14. The ophthalmic imaging section 16 includes various components necessary for capturing ophthalmic images of the subject's eye.

[0056] In this disclosure, the term "processor" refers to one or more hardware processors configured to execute program code (i.e., one or more instructions of a program) included in a program. In other words, a "processor" is a hardware device capable of executing one or more programmed processes. For example, a "processor" may be a general-purpose or special-purpose processor, such as a CPU, a microprocessor, a GPU, or a Data Flow Processor (DFP), etc.

[0057] In this disclosure, the term "memory" refers to one or more hardware memories that are non-transitory tangible recording media configured to store computer program code and / or data in a manner accessible to a processor. The "memory" may be implemented using memory technologies such as SRAM, SDRAM, non-volatile / flash-type memory, or other types of memory. Computer program code constituting a program is stored in the memory and executed by the processor to cause a device (e.g., the ophthalmic image processing device 21) to perform various functions.

[0058] In this disclosure, the term "circuit" refers to one or more hardware logic circuits configured to cause a device to perform a function. In other words, a "circuit" refers to one or more non-programmable devices. For example, a "circuit" may be a custom IC designed for a specific application in a non-programmable manner.

[0059] In the present disclosure, a circuit and / or a processor having a memory storing computer program code causes a device (such as the ophthalmic image processing device 21) to implement a function. The expression "a circuit and / or a processor" should be interpreted as a disjunction (logical OR), and not as at least one circuit and at least one processor.

[0060] (Mathematical model construction process) The mathematical model construction process executed by the mathematical model construction device 1 will be described. The mathematical model construction process is executed by the CPU 3 in accordance with a mathematical model construction program stored in the storage device 4. In the mathematical model construction process, a mathematical model is trained using a training dataset, thereby constructing a mathematical model that outputs analytical information based on an ophthalmic image. An ophthalmic image is input to the constructed mathematical model, and analytical information related to a disease of the tissue shown in the input ophthalmic image is output by the mathematical model.

[0061] An example of analytical information output by the mathematical model of this embodiment will be described. As shown in FIG. 4, the mathematical model of this embodiment outputs, as analytical information, a probability indicating whether the state of the tissue in the input ophthalmic image is predicted to be a disease-free, normal eye state or a state in which one of 10 types of diseases (diseases A to J) is present. In this embodiment, the sum of all probabilities is 100%. For example, in the example shown in FIG. 4, the probability that disease D is predicted to be present in the tissue in the ophthalmic image is the highest, followed by the probability that disease C is predicted to be present. Medical professionals can also use the analytical information to determine in advance the type of disease predicted to be present in the tissue. Furthermore, the highest probability of "normal eye" shown in FIG. 4 indicates that the state of the tissue in the ophthalmic image is predicted to be a disease-free, normal eye state. Note that, using the analytical information of this embodiment, medical professionals can also determine the probability that some disease is predicted to be present in the tissue in the ophthalmic image by comparing the sum of the probabilities of diseases A to J with the probability of a normal eye.

[0062] The mathematical model of this embodiment can also output, as analytical information, predicted information on the severity of disease in tissues depicted in the input ophthalmologic image, allowing medical professionals to appropriately check the ophthalmologic image after understanding the predicted information on the severity of disease.

[0063] A training dataset for training a mathematical model includes input data (input training data) and output data (output training data). The type of training dataset used to train the mathematical model is determined depending on the type of analytical information to be output by the mathematical model.

[0064] In this embodiment, an ophthalmological image of the tissue of the subject's eye (as an example, in this embodiment, a two-dimensional fundus camera image of the fundus tissue of the subject's eye taken from the front) is used as input training data. In addition, disease data indicating the presence or absence of a disease in the tissue shown in the input training data (ophthalmological image) and severity data indicating the severity of the disease in the tissue are used as output training data. The disease data and severity data may be generated, for example, by an operator (e.g., a medical professional) who has checked the input training data. In this embodiment, the disease data indicates whether the state of the tissue shown in the input training data (ophthalmological image) is a normal eye state or a state in which one of 10 types of diseases is present.

[0065] The flow of the mathematical model construction process will be described. The CPU 3 of the mathematical model construction device 1 acquires data of ophthalmic images (training ophthalmic images) captured by the ophthalmic image capturing device 11A as input training data. Next, the CPU 3 acquires disease data indicating the presence or absence of disease in the tissues captured in the training ophthalmic images acquired in S1, and severity data indicating the severity of the disease in the tissues, as output training data. An example of the correspondence between the input training data and the output training data has been described above.

[0066] The CPU 3 uses a training dataset to train a mathematical model using a machine learning algorithm, such as a neural network, a random forest, boosting, or a support vector machine (SVM).

[0067] Neural networks are a method of imitating the behavior of biological neuronal networks. Examples of neural networks include feedforward neural networks, RBF networks (radial basis functions), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

[0068] Random forest is a method to generate a large number of decision trees by learning from randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained as classifiers are traced, and the results obtained from each decision tree are averaged (or voted by majority vote).

[0069] Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

[0070] SVM is a method for constructing a two-class pattern classifier using linear input elements. SVM learns the parameters of the linear input elements based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance from each data point from the training data (hyperplane separation theorem).

[0071] A mathematical model refers to, for example, a data structure for predicting the relationship between input data and output data. A mathematical model is constructed by training using a training dataset. As described above, a training dataset is a set of input training data and output training data. For example, the correlation data (e.g., weights) between each input and output is updated through training.

[0072] In this embodiment, a multi-layer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also referred to as units) are arranged in each layer. In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layer neural network, is used. However, other machine learning algorithms may also be used. For example, generative adversarial networks (GAN), which utilize two competing neural networks, may be adopted as the machine learning algorithm.

[0073] The above-described process is repeated until the construction of the mathematical model is completed. When the construction of the mathematical model is completed, the mathematical model construction process ends. The program and data for realizing the constructed mathematical model are installed in the ophthalmologic image processing device 21.

[0074] (First embodiment) 3 to 8, the ophthalmic image processing performed by the ophthalmic image processing device 21 of the first embodiment will be described. In the ophthalmic image processing of the first embodiment, analysis information related to a disease is acquired for each of a plurality of ophthalmic images. The analysis suitability of the ophthalmic image for acquiring the analysis information is acquired for each of the plurality of ophthalmic images. Information about ophthalmic images whose analysis suitability is determined to not satisfy the standard is stored in a folder (an inappropriate image folder) different from the folder (a normal folder in the first embodiment) that stores information about ophthalmic images whose analysis suitability is determined to satisfy the standard. In the first embodiment, the priority order for displaying a plurality of ophthalmic images on the display device 28 is determined based on the analysis results acquired for each ophthalmic image. The ophthalmic image processing is executed by the CPU 23 of the ophthalmic image processing device 21 in accordance with an ophthalmic image processing program stored in the storage device 24.

[0075] As shown in FIG. 3, when the CPU 23 starts ophthalmic image processing, it acquires one ophthalmic image (or a set of ophthalmic images) from a folder (a folder for unprocessed images) storing ophthalmic images for which analysis information has not yet been acquired (S1). The CPU 23 acquires analysis information about the ophthalmic image acquired in S1 (S2). As described above, the mathematical model of this embodiment is trained according to a machine learning algorithm so as to output analysis information about diseases of tissues shown in the input ophthalmic image. The CPU 23 inputs the ophthalmic image acquired in S1 into the mathematical model, thereby acquiring the analysis information output by the mathematical model. Therefore, by using a mathematical model trained with multiple training data, useful analysis results can be easily acquired even in cases where it is difficult to appropriately acquire analysis results using functions that do not use machine learning algorithms.

[0076] As described above, in this embodiment, the analysis information acquired is a probability indicating whether the state of the tissue shown in the ophthalmologic image is predicted to be a disease-free normal eye state or a state in which one of 10 types of diseases (Diseases A to J) is present. In this embodiment, prediction information on the severity of the disease of the tissue shown in the ophthalmologic image is also acquired as the analysis information.

[0077] 4, the CPU 23 assigns an analysis region display frame 31 indicating a region (a circular region in this embodiment) from which analysis information is to be acquired to the ophthalmologic image 30 from which analysis information is acquired in S2. In the example shown in Fig. 4, the region inside the analysis region display frame 31 is the region to be analyzed. The region outside the analysis region display frame 31 is a masked region that is not to be analyzed.

[0078] The CPU 23 acquires the analysis suitability of the ophthalmic image acquired in S1 (S3). The analysis suitability is the suitability of the analysis performed by the mathematical model in S2. The CPU 23 determines whether the analysis suitability of the ophthalmic image acquired in S3 satisfies a standard (S5). An image whose analysis suitability satisfies the standard is determined to be an appropriate ophthalmic image. An image whose analysis suitability does not satisfy the standard is determined to be an inappropriate ophthalmic image.

[0079] In this embodiment, in S3, multiple types of analysis suitability are acquired for each ophthalmological image. In S5, it is determined whether at least one of the multiple types of analysis suitability acquired in S3 satisfies a standard corresponding to the type. An ophthalmological image for which at least one of the multiple types of analysis suitability does not satisfy the standard is determined to be an inappropriate ophthalmological image.

[0080] As an example, in this embodiment, the CPU 23 acquires a pixel value histogram (e.g., a density histogram) that graphs the number of pixels having the same pixel value (e.g., density value) in the ophthalmic image as the analysis suitability of the ophthalmic image. The CPU 23 determines whether the ophthalmic image is suitable for acquiring analysis information (whether it is an appropriate ophthalmic image) depending on whether the pixel value histogram satisfies a criterion. As a result, whether the ophthalmic image is an appropriate ophthalmic image is appropriately determined based on the pixel values ​​of the ophthalmic image. The CPU 23 also acquires the degree of flare in the ophthalmic image, the exposure amount, and the suitability of the mask portion (the area outside the analysis area display frame 31 shown in FIG. 4) applied to the ophthalmic image as the analysis suitability of the ophthalmic image. In S5, it is determined whether each of the multiple types of analysis suitability acquired in S3 satisfies a criterion corresponding to the type.

[0081] It is also possible to change the specific aspect of the analysis suitability. For example, the signal strength of the ophthalmic image or an index indicating the quality of the signal (e.g., SSI (Signal Strength Index) or QI (Quality Index)) may be acquired as the analysis suitability of the ophthalmic image. Also, at least one of the ratio of the noise level to the signal level of the image (SNR (Signal to Noise Ratio)), the background noise level, the image contrast, etc. may be acquired as the analysis suitability of the ophthalmic image. Also, the shooting conditions when the ophthalmic image was captured by the ophthalmic imaging device may be acquired as the analysis suitability of the ophthalmic image.

[0082] If the analysis suitability of the ophthalmic image acquired in S1 does not satisfy the standard (i.e., if the ophthalmic image acquired in S1 is determined to be an inappropriate ophthalmic image) (S5: YES), the CPU 23 associates the analysis information acquired in S2 and the inappropriateness grounds for determining the ophthalmic image as inappropriate in S5 (i.e., the type of analysis suitability that did not satisfy the standard) with the ophthalmic image acquired in S1 (i.e., the ophthalmic image from which the analysis information was acquired), and stores them in a folder for inappropriate images in the storage device 24 (S6). On the other hand, if the analysis suitability of the ophthalmic image acquired in S1 satisfies the standard (i.e., if the ophthalmic image acquired in S1 is determined to be an appropriate ophthalmic image) (S5: NO), the CPU 23 associates the analysis information acquired in S2 with the ophthalmic image acquired in S1, and stores them in a normal folder in the storage device 24 (S7).

[0083] As described above, the ophthalmic image processing device 21 of the first embodiment stores information about inappropriate ophthalmic images in an inappropriate image folder that is different from the folder (the normal folder in the first embodiment) that stores information about appropriate ophthalmic images. Therefore, medical professionals can efficiently check only the information about the inappropriate ophthalmic images by checking the information stored in the inappropriate image folder. This makes it easier to efficiently assist medical professionals in their medical treatment. Note that in this embodiment, information about the inappropriate ophthalmic images is automatically stored in the inappropriate image folder. However, when a user inputs an instruction to store information about the inappropriate ophthalmic images in the inappropriate image folder, the CPU 23 may extract information about the inappropriate ophthalmic images from information about the multiple ophthalmic images and store the information in the inappropriate image folder.

[0084] Furthermore, in the first embodiment, the CPU 23 acquires analysis information of the ophthalmological image in S2 regardless of whether the ophthalmological image acquired in S1 is an inappropriate ophthalmological image. That is, in the first embodiment, analysis information is acquired even for an ophthalmological image whose analysis suitability is determined not to satisfy the standard in S5. As will be described in detail later, the analysis information acquired for the inappropriate ophthalmological image is output in association with the inappropriate ophthalmological image. Therefore, medical personnel can easily confirm the validity of the analysis information by referring to the analysis information acquired for the inappropriate ophthalmological image.

[0085] In this embodiment, the process of acquiring analysis information of the ophthalmological image (S2) is executed before the process of determining whether the ophthalmological image is inappropriate (S5). However, the process of S2 may be executed after the process of S5.

[0086] Returning to the explanation of Fig. 3, when the process of acquiring and storing analysis information for one (or one set of) ophthalmic images (S1 to S7) is completed, the CPU 23 determines whether or not any unprocessed ophthalmic images remain in the folder for unprocessed images (S8). If any unprocessed ophthalmic images remain (S8: YES), the process returns to S1, and the process of S1 to S7 is repeated for the next ophthalmic image.

[0087] If there are no unprocessed ophthalmic images remaining (S8: NO), the CPU 23 outputs list data 35 (see FIG. 5) in which analysis information and the reason for inappropriateness are associated with each of one or more ophthalmic images for which the processes of S1 to S7 have been completed. Thereafter, a display process (S10) is executed to display the ophthalmic images for which the processes of S1 to S7 have been completed on the image interpretation screen 40 (see FIGS. 7 and 8), and the process ends.

[0088] As an example, in this embodiment, CSV data is used as the list data 35. However, data other than CSV data may be used as the list data. The method for outputting the list data 35 can be selected as appropriate. For example, the CPU 23 may output the list data 35 by storing it in the storage device 24, or may output the list data 35 by displaying it on the display device 28.

[0089] In the list data 35 shown in Figure 5, each ophthalmological image is associated with an examination number indicating the order in which the ophthalmological image was taken, the file name of the ophthalmological image, the date the ophthalmological image was taken, the analysis information obtained for the ophthalmological image, and the reason for the inappropriateness if the ophthalmological image is an inappropriate ophthalmological image (i.e., the type of analysis appropriateness that did not meet the criteria).

[0090] In this embodiment, each piece of analysis information includes information predicting the presence of a disease in the ophthalmological image ("X" in FIG. 5) or information predicting the absence of a disease in the ophthalmological image ("◯" in FIG. 5). Furthermore, when a disease is predicted to be present, the analysis information also includes information indicating the type of disease predicted to be present in the tissue (for example, the type of disease predicted to be present in the tissue with the highest probability). As shown in FIG. 5, in the first embodiment, analysis information is also acquired for an ophthalmological image whose analysis suitability is determined not to satisfy the standard in S5 (in FIG. 5, an ophthalmological image showing a "histogram" and a "mask" as the basis for inappropriateness). The analysis information acquired for an inappropriate ophthalmological image is output in association with the inappropriate ophthalmological image. Therefore, medical personnel can easily confirm the validity of the analysis information by referring to the analysis information acquired for the inappropriate ophthalmological image.

[0091] As shown in Fig. 5, the CPU 23 outputs, in association with the ophthalmic image, basis information indicating the type of analysis suitability that did not satisfy the criteria among the multiple types of analysis suitability acquired for the ophthalmic image ("basis for inappropriateness" in Fig. 5). Therefore, medical personnel can easily understand the basis for determining that the ophthalmic image is inappropriate (the type of analysis suitability that did not satisfy the criteria) based on the basis information. Therefore, for example, it becomes easier to confirm the validity of the analysis information acquired for the inappropriate ophthalmic image.

[0092] 5 includes both information about inappropriate ophthalmic images and information about appropriate ophthalmic images. However, the CPU 23 may output the list data of information about inappropriate ophthalmic images and the list data of information about appropriate ophthalmic images separately.

[0093] Furthermore, the method of outputting the analysis information in association with the ophthalmological image for which the analysis information has been acquired is not limited to the method of outputting the list data 35. For example, when displaying an ophthalmological image on an interpretation screen 40 (see FIGS. 7 and 8) described later, the CPU 23 of the present embodiment outputs the analysis information by also displaying the analysis information acquired for the ophthalmological image to be displayed.

[0094] Similarly, the method of outputting the inappropriate ophthalmological image in association with the basis for inappropriateness (basis information) is not limited to the method of outputting the list data 35. For example, when displaying the inappropriate ophthalmological image on the image interpretation screen 40 (see FIG. 8), the CPU 23 of the present embodiment outputs the basis for inappropriateness by also displaying the basis for inappropriateness of the inappropriate ophthalmological image to be displayed.

[0095] The display process (S10, see FIG. 3) in the first embodiment will be described in detail with reference to FIG. 6. First, the CPU 23 determines whether or not the user has input an instruction to select a folder of ophthalmologic images to be displayed on the image interpretation screen 40 from among a plurality of folders in which information about ophthalmologic images is stored (S11). If no instruction has been input (S11: NO), the determination of S11 is repeated and the system enters a standby state.

[0096] When an instruction to select a folder is input (S11: YES), the CPU 23 determines whether the first mode or the second mode has been selected by the user (S14). The first mode is a mode in which the priority order when displaying multiple ophthalmic images sequentially on the image interpretation screen 40 is determined based on analysis information acquired for each ophthalmic image. The second mode is a mode in which the priority order when displaying multiple ophthalmic images sequentially on the image interpretation screen 40 is determined regardless of the analysis information acquired for each ophthalmic image. In this embodiment, the user can instruct the ophthalmic image processing device 21 to execute either the first mode or the second mode by operating the operation unit 27.

[0097] When the first mode is selected (S14: YES), the CPU 23 determines, based on the analysis information acquired for each ophthalmological image, the priority order for displaying information about the multiple ophthalmological images included in the folder selected in S11 on the image interpretation screen 40. The CPU 23 generates an image list 41 of the multiple ophthalmological images included in the folder selected in S11 according to the determined priority order, and displays the image list 41 on the image interpretation screen 40 (S15).

[0098] 7 and 8, in the image list 41 of this embodiment, information on a plurality of ophthalmological images to be displayed in order on the image interpretation screen 40 is arranged from top to bottom according to priority. Similar to the list data 35 (see FIG. 5), the information on each ophthalmological image includes an examination number indicating the order in which the ophthalmological image was captured, the file name of the ophthalmological image, the capture date of the ophthalmological image, analysis information acquired about the ophthalmological image, and the reason for inadequacy if the ophthalmological image is an inadequacy image (i.e., the type of analysis suitability that did not meet the criteria).

[0099] The image interpretation screen 40 of this embodiment also displays a selected folder display area 42, a folder change button 43, a previous image button 44, and a next image button 45. The selected folder display area 42 displays a folder selected by the user from among multiple folders (i.e., a folder of ophthalmic images to be displayed on the image interpretation screen 40). The folder change button 43 is operated when a user (e.g., a medical professional) inputs an instruction to change the folder of ophthalmic images to be displayed on the image interpretation screen 40. The previous image button 44 is operated when a user inputs an instruction to switch an ophthalmic image to be displayed on the image interpretation screen 40 from among the multiple ophthalmic images displayed in the image list 41 to an ophthalmic image that has a higher priority on the image list 41. The next image button 45 is operated when a user inputs an instruction to switch an ophthalmic image to be displayed on the image interpretation screen 40 from among the multiple ophthalmic images displayed in the image list 41 to an ophthalmic image that has a lower priority on the image list 41. That is, when the next image button 45 is repeatedly operated, the CPU 23 switches the ophthalmologic image to be displayed on the image interpretation screen 40 in accordance with the set priority order.

[0100] 7 and 8, information about an ophthalmological image currently being displayed on the image interpretation screen 40 is indicated by a bold frame among information about a plurality of ophthalmological images included in the image list 41. The user can also display a desired ophthalmological image on the image interpretation screen 40 regardless of priority by directly selecting information about the desired ophthalmological image from the image list 41.

[0101] In this embodiment, when any one of the examination number, the photographing date, and the analysis information is selected from the image list 41, the CPU 23 sorts the priority (display order) of the multiple ophthalmologic images in the image list 41 according to the selected information. Therefore, the user can change the priority according to various situations.

[0102] 7 is a diagram showing an example of the image interpretation screen 40 when the normal folder (i.e., the folder storing information about appropriate ophthalmic images) is selected in the first mode. As shown in FIG. 7, by selecting the normal folder, the medical professional can check only the appropriate ophthalmic images in order.

[0103] Fig. 8 is a diagram showing an example of the image interpretation screen 40 when the inappropriate image folder (i.e., the folder storing information about inappropriate ophthalmic images) is selected in the first mode. As shown in Fig. 7, by selecting the inappropriate image folder, the medical staff can sequentially check only the inappropriate ophthalmic images.

[0104] 7 and 8, in the first mode, the CPU 23 determines, based on the analysis information of the ophthalmic images, the priority order for displaying the multiple ophthalmic images on the image interpretation screen 40. Therefore, the medical staff can check the multiple ophthalmic images according to the appropriate priority order based on the analysis information.

[0105] As described above, the analysis information acquired in this embodiment includes information indicating the distribution of the probability that each of multiple types of diseases (Diseases A to J) is predicted to be present in tissues depicted in ophthalmologic images. Each of the multiple types of diseases is also assigned an attention rank. The attention rank may be assigned in advance or in response to an instruction input by a user. In the example shown in FIGS. 7 and 8, the attention ranks are assigned in the following order: Disease A, Disease B, Disease C, Disease D, Disease E, Disease F, Disease G, Disease H, Disease I, Disease J, and normal eyes. As shown in FIGS. 7 and 8, the CPU 23 sets the display priority of ophthalmologic images according to the attention rank of diseases with a high probability of presence indicated by the analysis information. Therefore, medical professionals can preferentially view ophthalmologic images of tissues predicted to contain diseases with a high attention rank.

[0106] The analysis information acquired in this embodiment also includes predicted information on the probability of disease in the tissues depicted in the ophthalmologic images. The CPU 23 can also prioritize display of ophthalmologic images for which the analysis information indicates a high probability of disease over display of ophthalmologic images for which the analysis information indicates a low probability of disease. In this case, medical professionals can prioritize viewing ophthalmologic images predicted to have a high probability of disease.

[0107] The analysis information acquired in this embodiment also includes predicted information on the severity of disease in the tissues depicted in the ophthalmologic images. The CPU 23 may prioritize displaying ophthalmologic images for which the analysis information indicates a high degree of disease severity over displaying ophthalmologic images for which the analysis information indicates a low degree of disease severity. In this case, medical personnel can prioritize viewing ophthalmologic images for which the analysis information indicates a high degree of disease severity.

[0108] Returning to the description of FIG. 6 , if the second mode is selected (S14: NO), the CPU 23 determines the priority order for displaying information on the multiple ophthalmic images included in the folder selected in S11 on the image interpretation screen 40, regardless of the analysis information acquired for each ophthalmic image. For example, the CPU 23 may determine the priority order according to the order of capture, or may determine the priority order randomly. The CPU 23 generates an image list 41 of the multiple ophthalmic images included in the folder selected in S11 according to the determined priority order, and displays the image list 41 on the image interpretation screen 40 (S16).

[0109] As described above, in the first mode, medical professionals can review multiple ophthalmic images according to an appropriate priority determined based on the analysis information. This makes it easier for medical professionals to provide efficient medical assistance. In addition, in the second mode, medical professionals can carefully review each ophthalmic image regardless of the priority determined based on the analysis information. By switching between the first mode and the second mode, medical professionals can review ophthalmic images in an appropriate procedure depending on various situations.

[0110] When the priorities of the multiple ophthalmic images are determined (S15, S16), the CPU 23 displays the first ophthalmic image in the image list 41 (i.e., the ophthalmic image with the highest priority) on the image interpretation screen 40 (S17). Thereafter, the processes of S19 to S25 are repeated. When the next image button 45 is operated (S19: YES), the CPU 23 switches the ophthalmic image to be displayed on the image interpretation screen 40 from the multiple ophthalmic images displayed in the image list 41 to the ophthalmic image with the next lower priority on the image list 41 (S20). When the previous image button 44 is operated (S21: YES), the CPU 23 switches the ophthalmic image to be displayed on the image interpretation screen 40 from the multiple ophthalmic images displayed in the image list 41 to the ophthalmic image with the next higher priority on the image list 41 (S22). When the folder change button 43 is operated (S25: YES), the CPU 23 changes the folder of the ophthalmologic images to be displayed on the image interpretation screen 40 to the folder selected by the user, and repeats the processes of S14 to S25. When an instruction to end the display of the ophthalmologic images is input (S24: YES), the process ends.

[0111] (Second embodiment) The ophthalmic image processing executed by the ophthalmic image processing device 21 of the second embodiment will be described with reference to Fig. 9. Note that the same processing as the ophthalmic image processing of the first embodiment can be adopted as part of the ophthalmic image processing of the second embodiment. Therefore, among the multiple steps in the ophthalmic image processing of the second embodiment, steps that can adopt the same processing as the ophthalmic image processing of the first embodiment are assigned the same step numbers as in the first embodiment, and their description will be omitted.

[0112] 9, in the second embodiment, when the CPU 23 completes the process of acquiring analysis information of the ophthalmic image acquired in S1 (S2) and the process of acquiring the analysis adequacy (S3), the CPU 23 associates the analysis information and the reason for inadequacy with the ophthalmic image acquired in S1 and stores them in a folder corresponding to the analysis result (S31). Therefore, by checking the information in the folder corresponding to the analysis information, a medical professional can efficiently understand the ophthalmic image for which specific analysis information was obtained.

[0113] Specifically, the analysis information acquired in this embodiment includes information indicating the distribution of the predicted probability of a disease being present in tissues captured in an ophthalmological image (including the probability of a normal eye being predicted) for each of multiple types of diseases (diseases A to J) and normal eyes. In the second embodiment, folders corresponding to each of the multiple types of diseases (diseases A to J) and a folder corresponding to normal eyes are provided. The CPU 23 stores information about the ophthalmological image in a folder corresponding to one or more diseases whose presence probability indicated by the analysis information is high, among the multiple folders provided for each of the multiple diseases. Furthermore, when the analysis information indicates that the predicted probability of a normal eye is highest, the CPU 23 stores information about the ophthalmological image in a folder corresponding to a normal eye. Therefore, by checking the information in the folder for a specific disease, medical professionals can efficiently identify ophthalmological images in which a specific disease is likely to be present.

[0114] The analysis information acquired in this embodiment also includes predicted information on the severity of disease of tissues depicted in the ophthalmic images. The CPU 23 may store, in a specific folder, information on ophthalmic images for which the analysis information indicates a severity level higher than a reference level. In this case, medical personnel can efficiently identify ophthalmic images predicted to have high severity by checking the information in the specific folder. When the CPU 23 stores, in a specific folder, information on ophthalmic images for which the analysis information indicates a severity level higher than a reference level, the CPU 23 may execute a notification process (e.g., a notification display process on the display device 28 or a notification email transmission process) to prompt the medical personnel to check the ophthalmic images in the specific folder.

[0115] (Third embodiment) With reference to FIG. 10, the ophthalmic image processing executed by the ophthalmic image processing device 21 of the second embodiment will be described. FIG. 10 is an explanatory diagram illustrating an example of a display order when multiple ophthalmic images are displayed by the ophthalmic image processing device 21 of the third embodiment. As shown in FIG. 10, the ophthalmic image processing device 21 of the third embodiment randomly displays on the image interpretation screen 40 an ophthalmic image whose priority is different from the next priority while sequentially displaying multiple ophthalmic images according to the priority determined based on the analysis information. As a result, the ophthalmic image whose priority is out of the priority is randomly displayed among the multiple ophthalmic images sequentially displayed based on the priority. This makes it easier for medical professionals to check the multiple ophthalmic images displayed sequentially while maintaining high concentration.

[0116] The techniques disclosed in the above embodiments are merely examples. Therefore, the techniques exemplified in the above embodiments can be modified. For example, the ophthalmological image processing device 21 may selectively execute some of the processes exemplified in the above embodiments. Furthermore, the ophthalmological image processing device 21 can execute a combination of the processes exemplified in the first embodiment and the processes exemplified in the second embodiment.

[0117] The ophthalmic image processing device 21 can also sequentially display multiple ophthalmic images on multiple display units (e.g., display units provided in each of multiple terminal devices). In this case, the ophthalmic image processing device 21 may change the display order of multiple ophthalmic images for each display unit. When reviewing multiple ophthalmic images, medical professionals' concentration tends to decrease over time. In response to this, changing the display order of multiple ophthalmic images for each display unit that displays the images can help prevent multiple medical professionals from focusing on each ophthalmic image when reviewing multiple ophthalmic images.

[0118] Furthermore, the ophthalmic image processing device 21 may use an ophthalmic image that has been displayed on the image interpretation screen 40 as an image for which the priority order according to the analysis information is determined, and may exclude an ophthalmic image that has not yet been displayed on the image interpretation screen 40 from the images for which the priority order according to the analysis information is determined. In this case, the medical staff can carefully check ophthalmic images that have not been displayed on the image interpretation screen 40 in the past, and can efficiently check ophthalmic images that have been displayed on the image interpretation screen 40 in the past according to the priority order.

[0119] The ophthalmologic image processing device 21 may also set ophthalmologic images for which a user has instructed that they be prioritized as images for which priorities are to be determined according to the analysis information, and may exclude ophthalmologic images for which no instruction has been given that they be prioritized from images for which priorities are to be determined according to the analysis information. In this case, medical personnel can efficiently check the ophthalmologic images for which priorities are to be set and can carefully check other ophthalmologic images. [Explanation of symbols]

[0120] 11 (11A, 11B) Ophthalmic imaging device 21 Ophthalmological image processing device 23 CPU 24 Storage device 28 Display device 30 Ophthalmology Images 35 List Data 40 Reading Screen 41 Image List

Claims

1. An ophthalmic image processing program executed by an ophthalmic image processing device that processes an ophthalmic image, which is an image of tissue of a subject's eye, The ophthalmologic image processing program is executed by a control unit of the ophthalmologic image processing device, an image acquisition step of acquiring an ophthalmologic image captured by an ophthalmologic image capturing device; an analysis information acquisition step of acquiring analysis information related to a disease of a tissue shown in the ophthalmologic image acquired in the image acquisition step; an information storage step of storing the analysis information in a storage device in association with the ophthalmologic image from which the analysis information was obtained; a display step of displaying the ophthalmologic image on a display unit; causing the ophthalmologic image processing device to execute An ophthalmological image processing program characterized in that at least one of a folder for storing the analysis information and the ophthalmological images in the information storage step and a priority for displaying a plurality of the ophthalmological images on the display unit in the display step is determined based on at least one of the analysis information acquired in the analysis information acquisition step and the analysis suitability of the ophthalmological images for acquiring the analysis information in the analysis information acquisition step.

2. 2. The ophthalmologic image processing program according to claim 1, In the analysis information acquisition step, An ophthalmic image processing program characterized in that the ophthalmic image acquired in the image acquisition step is input into a mathematical model trained by a machine learning algorithm so as to output analytical information regarding a disease of the tissue depicted in the input ophthalmic image, thereby obtaining the analytical information.

3. 3. The ophthalmologic image processing program according to claim 1, an adequacy determination step of obtaining the analysis adequacy of the ophthalmologic image obtained in the image obtaining step and determining whether the obtained analysis adequacy satisfies a standard; An ophthalmological image processing program characterized in that, in the information storage step, information regarding the ophthalmological image for which the analysis suitability is judged not to meet the standard in the suitability judgment step is stored in a folder different from a folder for storing information regarding the ophthalmological image for which the analysis suitability is judged to meet the standard.

4. 4. The ophthalmologic image processing program according to claim 3, The analysis information is also acquired in the analysis information acquisition step for an inappropriate ophthalmological image, which is an ophthalmological image determined in the appropriateness determination step to have an analysis appropriateness that does not satisfy a standard; An ophthalmological image processing program, characterized in that the analysis information acquired for the inappropriate ophthalmological image is output in association with the inappropriate ophthalmological image for which the analysis information was acquired.

5. 5. The ophthalmologic image processing program according to claim 3, In the suitability determination step, a plurality of types of analysis suitability are acquired for the ophthalmologic image, and it is determined whether or not at least one of the acquired plurality of types of analysis suitability satisfies a criterion corresponding to the type; An ophthalmological image processing program characterized in that, for an ophthalmological image for which at least one of the analysis suitabilities is determined not to meet the standard in the suitability determination step, basis information indicating the type of analysis suitability that did not meet the standard is output in association with the image.

6. 6. An ophthalmologic image processing program according to claim 1, An ophthalmologic image processing program characterized in that, in the information storage step, information regarding the ophthalmologic image is stored in a folder among a plurality of folders corresponding to the analysis information obtained for the ophthalmologic image.

7. 7. An ophthalmologic image processing program according to claim 1, An ophthalmological image processing program characterized in that the priority order for displaying multiple ophthalmological images on the display unit in the display step is determined based on the analysis information of the ophthalmological images acquired in the analysis information acquisition step.

8. 8. An ophthalmologic image processing program according to claim 7, the analysis information acquired in the analysis information acquisition step includes prediction information of a probability of a disease existing in a tissue shown in the ophthalmologic image, An ophthalmological image processing program characterized by giving a higher priority to displaying an ophthalmological image in which the analysis information indicates a high probability of the presence of a disease than to displaying an ophthalmological image in which the analysis information indicates a low probability of the presence of a disease.

9. 9. An ophthalmologic image processing program according to claim 7, the analysis information acquired in the analysis information acquisition step includes prediction information of severity of a disease of a tissue shown in the ophthalmologic image, An ophthalmological image processing program characterized by giving a higher priority to the display of an ophthalmological image in which the analysis information indicates a high severity of the disease than to the display of an ophthalmological image in which the analysis information indicates a low severity of the disease.

10. 10. An ophthalmologic image processing program according to claim 7, the analysis information acquired in the analysis information acquisition step includes information indicating a distribution of a probability that each of a plurality of types of disease is predicted to be present in the tissue captured in the ophthalmologic image, Each of the multiple types of diseases is assigned a priority ranking, An ophthalmologic image processing program, characterized in that a priority order for displaying the ophthalmologic images is set according to a priority order of diseases with a high probability of existence indicated by the analysis information.

11. 11. An ophthalmologic image processing program according to claim 7, An ophthalmological image processing program characterized in that the priority of displaying multiple ophthalmological images on the display unit in the display step can be switched to or not determined based on the analysis information of the ophthalmological images acquired in the analysis information acquisition step.

12. 12. An ophthalmologic image processing program according to claim 7, An ophthalmological image processing program characterized in that, in the display step, while sequentially displaying a plurality of ophthalmological images according to a priority determined based on the analysis information, an ophthalmological image whose priority is different from the next priority is randomly displayed.

13. 13. An ophthalmologic image processing program according to claim 1, An ophthalmological image processing program characterized in that, in the display step, when a plurality of ophthalmological images are sequentially displayed on each of the plurality of display units, the display order of the plurality of ophthalmological images is changed for each of the display units.

14. An ophthalmic image processing device that processes an ophthalmic image that is an image of tissue of a subject's eye, The control unit of the ophthalmologic image processing device an image acquisition step of acquiring an ophthalmologic image captured by an ophthalmologic image capturing device; an analysis information acquisition step of acquiring analysis information related to a disease of a tissue shown in the ophthalmologic image acquired in the image acquisition step; an information storage step of storing the analysis information in a storage device in association with the ophthalmologic image from which the analysis information was obtained; a display step of displaying the ophthalmologic image on a display unit; Run an ophthalmological image processing device characterized in that at least one of a folder for storing the analysis information and the ophthalmological images in the information storage step and a priority for displaying a plurality of the ophthalmological images on the display unit in the display step is determined based on at least one of the analysis information acquired in the analysis information acquisition step and the analysis suitability of the ophthalmological images for acquiring the analysis information in the analysis information acquisition step.

Images