Diagnosis support device, learning device, diagnosis support method, learning method, and program

Abstract

To provide a diagnosis support device for screening a diabetic retinopathy from a fundus image.SOLUTION: A diagnosis support device includes: an identification section for identifying a blood circulation abnormality area in a fundus image with the use of a learning model for relation between the fundus image and a blood circulation abnormality area in the fundus image based on the fundus image being the image of an ocular fundus and the blood circulation abnormality area identified on the basis of a fluorescent fundus contrast image of the ocular fundus; and an output section for outputting information for indicating the blood circulation abnormality area in the fundus image of a patient, which is identified by the identification section with the use of the fundus image of the patient and the learning model.SELECTED DRAWING: None

Description

[Technical Field]

[0001] The present invention relates to a diagnostic support device, a learning device, a diagnostic support method, a learning method, and a program. This application claims priority based on Japanese Patent Application No. 2018-7585, filed on January 19, 2018, the contents of which are incorporated herein by reference. [Background technology]

[0002] Diabetic retinopathy is a serious disease that is the second leading cause of blindness in Japan, but subjective symptoms only appear after the disease has progressed considerably, so early detection and treatment through medical examinations, etc. To address this issue, a fundus image analysis system that highlights microaneurysms, an early change in diabetic retinopathy, has been proposed (see Patent Document 1), and an image analysis system that screens for diabetic retinopathy from fundus images (see Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-178802 [Patent Document 2] US2014 / 0314288A1 Summary of the Invention

[0004] One aspect of the present invention is a diagnostic support device that includes an identification unit that identifies an area of ​​abnormal blood circulation in a fundus image using a learning model that has learned the relationship between the fundus image and an area of ​​abnormal blood circulation in the fundus image based on the fundus image and an area of ​​abnormal blood circulation identified based on a fluorescent fundus angiography image of the fundus, and an output unit that outputs a fundus image of a patient and information indicating the area of ​​abnormal blood circulation in the fundus image of the patient identified by the identification unit using the learning model. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a configuration diagram of an information processing system including a server that is an embodiment of an information processing device of the present invention. [Figure 2] FIG. 2 is a block diagram showing the hardware configuration of a server in the information processing system of FIG. 1. [Figure 3] This is a functional block diagram showing an example of the functional configuration of the server in Figure 2 for realizing various processes, including NPA / NV existence probability map generation process, incidental finding existence probability map generation process, estimated NPA / NV identification process, and estimated NPA / NV display process. [Figure 4] This is a flowchart explaining the flow of various processes such as NPA / NV existence probability map generation process, incidental finding existence probability map generation process, estimated NPA / NV identification process, and estimated NPA / NV display process executed by the server in Figure 3. [Figure 5] 4 is a diagram showing the flow of various information used in the processing executed by the server of FIG. 3. [Figure 6] 4 is a diagram showing an example of fundus image information acquired in the process executed by the server in FIG. 3. FIG. [Figure 7] 4 is a diagram showing an example of fluorescein fundus angiography image information acquired in the process executed by the server of FIG. 3. FIG. [Figure 8] 4 is a diagram showing an example of an NPA·NV existence probability map output in the NPA·NV existence probability map generation process executed by the server of FIG. 3. FIG. [Figure 9] 4A to 4C are diagrams illustrating an example of an estimated NPA and an estimated NV identified in the estimated NPA / NV identification process executed by the server of FIG. 3. [Figure 10] FIG. 10 is a configuration diagram of an information processing system according to a modified example of an embodiment of the present invention. [Figure 11] FIG. 10 is a block diagram showing an example of a learning device according to a modified example of an embodiment of the present invention. [Figure 12] FIG. 2 is a diagram illustrating an example of a fundus image. [Figure 13] FIG. 10 is a diagram showing an example of a fluorescein fundus angiography image. [Figure 14] FIG. 10 is a diagram showing an example of an abnormal blood circulation region. [Figure 15] FIG. 1 is a diagram illustrating an example of a structure of a neural network. [Figure 16] FIG. 10 is a block diagram showing an example of a diagnosis support device according to a modified example of an embodiment of the present invention. [Figure 17] 10 is a flowchart showing an example of the operation of a learning device according to a modified example of an embodiment of the present invention. [Figure 18] 10 is a flowchart showing an example of the operation of a diagnosis support device according to a modified example of an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0006] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of an information processing system including a server 1 which is an embodiment of an information processing device of the present invention.

[0007] The information processing system shown in FIG. 1 is configured to include a server 1, an ophthalmologist terminal 2, and an examination device 3. The server 1, the ophthalmologist terminal 2, and the examination device 3 are connected to each other via a network N such as the Internet.

[0008] The server 1 is a server that manages the information processing system shown in Fig. 1, and executes various processes, such as a process for generating an NPA / NV existence probability map, a process for generating an incidental finding existence probability map, a process for identifying an estimated NPA / NV, etc. The specific content of the processes executed by the server 1 will be described later with reference to Fig. 3. "NPA·NV existence probability map generation process" refers to a series of processes executed by server 1 from generating NPA·NV teaching information to generating an NPA·NV existence probability map. "NPA / NV training information" refers to training information used to calculate the probability of the presence of retinal non-perfusion areas (NPA / non-perfusion areas) (hereinafter referred to as "retinal non-perfusion areas" or "NPA") (hereinafter referred to as "NPA presence probability") and the probability of the presence of neovascularization (NV / neovascularization) (hereinafter referred to as "NV presence probability") in a patient's fundus image information. Specifically, "NPA / NV training information" is generated based on fluorescein angiography image information and the NPA / NV annotation information attached to this information. "Retinal nonperfused area" refers to an area of ​​poor circulation in the retina that occurs as a result of retinal vascular occlusion in ocular ischemic disease.

[0009] The "NPA / NV existence probability map" is image information that displays the probability of NPA existence and the probability of NV existence in fundus image information. "Fundus image information" refers to image information based on a fundus image. "Fluorescein angiography image information" refers to information based on a fluorescein angiography image. "NPA / NV annotation information" refers to diagnostic notes by ophthalmologist D regarding at least one of retinal nonperfusion areas (NPA) and neovascularization (NV) attached to fluorescein fundus angiography image information. "Neovascularization" is a condition in which poor retinal circulation and ischemia in areas of retinal aperfusion progress further. If neovascularization causes bleeding or retinal detachment through the formation of a proliferative membrane, it can ultimately lead to blindness. Therefore, identifying areas of poor retinal circulation, such as areas of retinal aperfusion and neovascularization, is extremely important in clinical practice. The specific processing flow of the NPA / NV existence probability map generation process will be described later with reference to the flowchart of FIG.

[0010] The "incidental finding existence probability map generation process" refers to a series of processes executed by the server 1 from generating incidental finding teacher information to generating an incidental finding existence probability map. "Accompanying finding teacher information" refers to teacher information used when calculating the probability of the presence of an incidental finding in a patient's fundus image information. Specifically, "accompanying finding teacher information" refers to teacher information generated based on fluorescein angiography image information, fundus image information, and incidental finding annotation information attached to these image information. "Annotation information for incidental findings" refers to diagnostic notes made by Ophthalmologist D in response to fluorescein angiography image information and fundus image information in which he / she has made an incidental judgment that the image is "not normal," other than diagnostic notes regarding retinal nonperfusion areas (NPA) or neovascularization (NV). For example, information such as microaneurysms, fundus hemorrhages, hard exudates, soft exudates, venous abnormalities, intraretinal microvascular abnormalities, vitreous hemorrhages, proliferative membranes, and retinal detachment are all examples of "annotation information for incidental findings." The "accompanying finding existence probability map" is image information (not shown) in which the existence probability of an incidental finding is displayed in distinguishable form in fundus image information. The specific flow of the incidental finding presence probability map generation process will be described later with reference to the flowchart of FIG.

[0011] "Estimated NPA / NV identification processing" refers to a series of processes performed by server 1, from identifying an area in the fundus image information that is estimated to correspond to a retinal nonperfusion area (NPA) as an estimated NPA based on the NPA existence probability, to identifying an area in the fundus image information that is estimated to correspond to neovascularization (NV) as an estimated NV based on the NV existence probability. The specific flow of the estimated NPA / NV identification process will be described later with reference to the flowchart of FIG.

[0012] The ophthalmologist terminal 2 is an information processing device operated by the ophthalmologist D, and is configured, for example, by a personal computer. The ophthalmologist terminal 2 transmits NPA / NV annotation information and associated finding annotation information to the server 1, and acquires information on the estimated NPA / NV identified by the server 1. The various pieces of information acquired by the ophthalmologist terminal 2 are output from the ophthalmologist terminal 2 and used by the ophthalmologist D in his examination.

[0013] The examination device 3 is composed of various devices used in eye examinations of patients. The examination device 3 transmits to the server 1 fundus image information obtained by imaging in fundus examinations and fluorescent fundus angiography image information obtained by imaging in fluorescent fundus angiography examinations.

[0014] FIG. 2 is a block diagram showing the hardware configuration of the server 1 in the information processing system of FIG.

[0015] The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.

[0016] The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13 . The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.

[0017] The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

[0018] The output unit 16 is composed of various liquid crystal displays and the like, and outputs various information. The input unit 17 is configured with various hardware leads and the like, and inputs various pieces of information. The storage unit 18 is configured with a DRAM (Dynamic Random Access Memory) or the like, and stores various data. The communication unit 19 controls communications with other devices via a network N including the Internet.

[0019] The drive 20 is provided as needed. Removable media 30, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately attached to the drive 20. Programs read from the removable media 30 by the drive 20 are installed in the storage unit 18 as needed. The removable media 30 can also store various data stored in the storage unit 18 in the same way as the storage unit 18.

[0020] Next, the functional configuration of the server 1 having such a hardware configuration will be described with reference to FIG. FIG. 3 is a functional block diagram showing an example of the functional configuration of the server 1 in FIG. 2 in the information processing system of FIG. 1, for realizing the NPA / NV existence probability map generation process, the incidental finding existence probability map generation process, the estimated NPA / NV identification process, and the estimated NPA / NV display process.

[0021] As shown in Figure 3, when the NPA / NV existence probability map generation process is executed in the CPU 11 (Figure 2) of the server 1, an image acquisition unit 101, an annotation acquisition unit 102, a teaching information generation unit 103, and a calculation unit 104 function. When the incidental finding presence probability map generation process is executed, the image acquisition unit 101, the annotation acquisition unit 102, the teacher information generation unit 103, and the calculation unit 104 function. When the estimated NPA / NV identification process is executed, the estimated NPA / NV identification unit 106 functions. When the estimated NPA / NV display process is executed, the estimated NPA / NV display control unit 107 functions. An area of ​​the storage unit 18 (FIG. 2) is provided with an image DB 401, an annotation DB 402, and a teacher DB 403. The storage unit 18 may be arranged in the ophthalmologist terminal 2 instead of the server 1.

[0022] The image acquisition unit 101 acquires fluorescein angiography image information of a patient and fundus image information of the patient. The fluorescein angiography image information and fundus image information acquired by the image acquisition unit 101 are stored and managed in the image DB 401. Specifically, when a fluorescein angiography image of the patient is captured by the examination equipment 3 during a fluorescein angiography examination of the patient, fluorescein angiography image information based on the fluorescein angiography image is transmitted to the server 1. Furthermore, when a fundus image of the patient is captured by the examination equipment 3 during a fundus examination of the patient, fundus image information based on the fundus image is transmitted to the server 1. The image acquisition unit 101 of the server 1 acquires the fluorescein angiography image information and fundus image information transmitted from the examination equipment 3 to the server 1, and stores this image information in the image DB 401. This allows the server 1 to accurately manage all of the fluorescein fundus angiography image information and fundus image information of the patient without omission.

[0023] The annotation acquisition unit 102 acquires diagnostic notes by ophthalmologist D regarding at least one of retinal nonperfusion areas (NPA) and neovascularization (NV), which are attached to the patient's fluorescein angiography image information, as NPA-NV annotation information. Specifically, when ophthalmologist D attaches diagnostic notes regarding retinal nonperfusion areas (NPA) and neovascularization (NV) to the fluorescein angiography image information during a fluorescein angiography examination, the ophthalmologist terminal 2 transmits the diagnostic notes to the server 1 as NPA-NV annotation information based on the operation of ophthalmologist D. The annotation acquisition unit 102 of the server 1 acquires the NPA-NV annotation information transmitted from the ophthalmologist terminal 2 and stores this information in the annotation DB 402. Note that the fluorescein angiography image information and the NPA-NV annotation information attached to this image information are managed in association with each other. This allows server 1 to manage all diagnostic notes by ophthalmologist D regarding at least one of retinal nonperfusion areas (NPA) and neovascularization (NV) attached to fluorescein fundus angiography image information as NPA / NV annotation information.

[0024] Furthermore, the annotation acquisition unit 102 acquires diagnostic notes by ophthalmologist D regarding incidental findings that have been added to the fluorescein angiography image and the fundus image as incidental finding annotation information. Specifically, when diagnostic notes by ophthalmologist D regarding incidental findings are added to the fluorescein angiography image information and the fundus image information, the ophthalmologist terminal 2 transmits the diagnostic notes to the server 1 as incidental finding annotation information based on an operation by ophthalmologist D. The annotation acquisition unit 102 of the server 1 acquires the incidental finding annotation information transmitted from the ophthalmologist terminal 2 and stores this information in the annotation DB 402. The fluorescein angiography image information, the fundus image information, and the incidental finding annotation information added to these image information are managed in association with each other. This allows the server 1 to manage all of the fluorescent fundus angiography image information and the diagnostic notes of the ophthalmologist D regarding the incidental findings added to the fundus image information as incidental finding annotation information.

[0025] The teacher information generation unit 103 generates NPA / NV teacher information, which serves as teacher information for calculating the NPA existence probability and the NV existence probability, based on the fluorescent fundus angiography image information and the NPA / NV annotation information corresponding to this information. That is, the image DB 401 stores fluorescent fundus angiography image information obtained from multiple patients, and the annotation DB 402 stores NPA / NV annotation information. Based on the information stored in these databases, the training information generating unit 103 generates NPA / NV training information that serves as training information when calculating the probability of NPA / NV in fundus image information. This allows the server 1 to generate and store training information for calculating the probability of existence of NPA and the probability of existence of NV in the fundus image information of the patient. Furthermore, the teacher information generating unit 103 generates incidental finding teacher information based on the fluorescein fundus angiography image information, fundus image information, and incidental finding annotation information corresponding to these image information. That is, fundus image information and fluorescein angiography image information obtained from multiple patients are stored in the image DB 401, and associated finding annotation information is stored in the annotation DB 402. Based on the information stored in these databases, the training information generation unit 103 generates associated finding training information that serves as training information when calculating the probability of the presence of an associated finding in the fundus image information. This allows the server 1 to accumulate training information for calculating the probability of the presence of an incidental finding in the fundus image information of a patient.

[0026] The calculation unit 104 calculates the NPA existence probability and the NV existence probability based on at least the NPA / NV teacher information. Furthermore, when an incidental finding existence probability map (described later) has been generated, the calculation unit 104 calculates the NPA existence probability and the NV existence probability based on the incidental finding existence probability map and the NPA / NV teacher information. The specific method for calculating the NPA existence probability and the NV existence probability is not particularly limited. For example, the NPA existence probability and the NV existence probability may be calculated by extracting features common to fundus images having NPA and NV from the NPA / NV teacher information, and normalizing the degree of match with the features to determine whether the fundus image information of the patient has the features. The NPA existence probability and the NV existence probability may also be calculated using deep learning technology. This makes it possible to set criteria for identifying areas of fundus image information that are estimated to correspond to retinal nonperfusion areas (NPA) and areas that are estimated to correspond to neovascularization (NV).

[0027] Furthermore, the calculation unit 104 calculates the probability of the presence of an incidental finding in the fundus image information based on the incidental finding teacher information. The specific method for calculating the probability of the presence of an incidental finding is not particularly limited. For example, the calculation unit 104 may extract features common to fundus images having incidental findings from the incidental finding teacher information, and calculate the probability of the presence of an incidental finding by normalizing the degree of match with the features to determine whether the fundus image information of the patient has these features. The calculation may also use deep learning technology to calculate the probability of the presence of an incidental finding. This makes it possible to set criteria for identifying areas of fundus image information that are estimated to correspond to retinal nonperfusion areas (NPA) and areas that are estimated to correspond to neovascularization (NV).

[0028] The map generating unit 105 generates an NPA / NV existence probability map as image information in which the NPA existence probability and the NV existence probability are displayed in distinguishable fashion in the fundus image information. Specifically, the map generating unit 105 generates image information such as an NPA / NV existence probability map E shown in FIG. 8. This makes it possible to generate information that serves as a basis for estimating the presence of retinal nonperfusion areas (NPA) and neovascularization (NV) in fundus image information. The method for distinguishing and displaying the NPA existence probability and the NV existence probability in the fundus image information is not particularly limited. For example, the NPA existence probability and the NV existence probability may be distinguished and displayed by different colors, or by different shades of color. The map generating unit 105 also generates an incidental finding presence probability map (not shown) as image information in which the incidental finding presence probability is displayed in distinguishable form on the fundus image information. This makes it possible to generate information that serves as a basis for estimating the presence of retinal nonperfusion areas (NPA) and neovascularization (NV) in fundus image information. The method for distinguishably displaying the probability of the presence of an incidental finding in the fundus image information is not particularly limited. For example, the probability of the presence of an incidental finding may be distinguishably displayed by a different color or by a different shade of color.

[0029] The estimated NPA / NV identifying unit 106 identifies a region of the fundus image information that is estimated to correspond to a retinal nonperfusion region (NPA) as an estimated NPA, and identifies a region that is estimated to correspond to a neovascularization (NV) as an estimated NV, based on the NPA existence probability and the NV existence probability. Specifically, the unit 106 identifies a region of the patient's fundus image information where the NPA existence probability and the NV existence probability exceed a predetermined threshold as a region that is estimated to correspond to a retinal nonperfusion region (NPA) (estimated NPA) or a region that is estimated to correspond to a neovascularization (NV) (estimated NV). Note that the threshold can be changed at the discretion of the ophthalmologist D. This makes it possible to quickly and easily identify areas of image information based on fundus images where the presence of retinal nonperfusion areas (NPAs) is estimated (estimated NPAs) and areas where the presence of neovascularization (NV) is estimated (estimated NVs).

[0030] The estimated NPA / NV display control unit 107 executes control to display the estimated NPA region and the estimated NV region in the fundus image information. This allows areas where retinal nonperfusion is estimated (estimated NPA) and areas where neovascularization (NV) is estimated (estimated NV) to be superimposed on image information based on the fundus image.

[0031] Next, with reference to FIG. 4, a flow of a series of processes executed by the server 1 having the functional configuration of FIG. 3 will be described. FIG. 4 is a flowchart illustrating the flow of a series of processes executed by the server 1 of FIG.

[0032] As shown in FIG. 4, the server 1 executes the following series of processes. In step S1, the image acquisition unit 101 determines whether or not fluorescein fundus angiography image information has been transmitted from the inspection device 3. If fluorescent fundus angiography image information has been transmitted, the determination in step S1 is YES, and the process proceeds to step S2. On the other hand, if fluorescent fundus angiography image information has not been transmitted, the determination in step S1 is NO, and the process returns to step S1. That is, the determination process in step S1 is repeated until fluorescent fundus angiography image information is transmitted. Thereafter, if fluorescent fundus angiography image information has been transmitted, the determination in step S1 is YES, and the process proceeds to step S2. In step S2, the image acquisition unit 101 acquires the transmitted fluorescein fundus angiography image information.

[0033] In step S3, the annotation acquisition unit 102 determines whether or not the NPA / NV annotation information has been transmitted from the ophthalmologist terminal 2. If NPA / NV annotation information has been transmitted, the determination in step S3 is YES, and the process proceeds to step S4. On the other hand, if NPA / NV annotation information has not been transmitted, the determination in step S3 is NO, and the process returns to step S3. That is, the determination process in step S3 is repeated until NPA / NV annotation information is transmitted. Thereafter, if NPA / NV annotation information has been transmitted, the determination in step S3 is YES, and the process proceeds to step S4. In step S4, the annotation acquisition unit 102 acquires the transmitted NPA / NV annotation information.

[0034] In step S5, the training information generating unit 103 generates NPA / NV training information based on the fluorescent fundus angiography image information and the NPA / NV annotation information corresponding to the fluorescent fundus angiography image information. In step S6, the image acquisition unit 101 determines whether or not fundus image information has been transmitted from the examination equipment 3. If fundus image information has been transmitted, the determination in step S6 is YES, and the process proceeds to step S7. On the other hand, if fundus image information has not been transmitted, the determination in step S6 is NO, and the process returns to step S6. That is, the determination process in step S6 is repeated until fundus image information is transmitted. Thereafter, if fundus image information has been transmitted, the determination in step S6 is YES, and the process proceeds to step S7. In step S7, the image acquisition unit 101 acquires the transmitted fundus image information.

[0035] In step S8, the annotation acquisition unit 102 determines whether or not the incidental finding annotation information has been transmitted from the ophthalmologist terminal 2. If the accompanying finding annotation information has been transmitted, the determination in step S8 is YES, and the process proceeds to step S9. On the other hand, if the accompanying finding annotation information has not been transmitted, the determination in step S8 is NO, and the process skips steps S9 to S11 and proceeds to step S12. In step S9, the annotation acquisition unit 102 acquires the accompanying finding annotation information. In step S10, the training information generating unit 103 generates accompanying finding training information based on the fluorescent fundus angiography image information and the NPA / NV annotation information corresponding to the fluorescent fundus angiography image information. In step S11, the calculation unit 104 calculates the probability of the presence of an incidental finding in the fundus image information based on the incidental finding teacher information. In step S12, the map generating unit 105 generates an incidental finding presence probability map as image information in which the incidental finding presence probability is displayed in distinguishable form on the fundus image information.

[0036] In step S13, the calculation unit 104 calculates the NPA existence probability and the NV existence probability based on at least the NPA / NV teacher information. Also, if an incidental finding existence probability map has been generated, the calculation unit 104 calculates the NPA existence probability and the NV existence probability based on the NPA / NV teacher information and the incidental finding existence probability map. In step S14, the map generating unit 105 generates an NPA / NV existence probability map as image information in which the NPA existence probability and the NV existence probability are displayed in a distinguishable manner in the fundus image information. In step S15, the estimated NPA / NV identification unit 106 identifies areas of the fundus image information that are estimated to correspond to retinal nonperfusion areas (NPAs) as estimated NPAs, and identifies areas that are estimated to correspond to neovascularization (NV) as estimated NVs, based on the NPA existence probability and the NV existence probability. In step S16, the estimated NPA / NV display control unit 107 executes control to display the estimated NPA and NV area in the fundus image information.

[0037] In step S17, the server 1 determines whether or not an instruction to end the process has been issued. If an instruction to end the process has not been given, the determination in step S17 is NO, and the process returns to step S1. On the other hand, if an instruction to end the process has been given, the determination in step S17 is YES, and the process ends. The server 1 executes the above series of processes, whereby the estimated NPA region and the estimated NV region are displayed in the fundus image information.

[0038] Next, with reference to FIG. 5, the flow of various information used in various processes executed by the server 1 will be described. FIG. 5 is a diagram showing the flow of various information in the processing executed by the server 1.

[0039] As shown in Figure 5, when a fluorescein angiogram of a patient is taken in a fluorescein angiography examination, fluorescein angiogram information based on this fluorescein angiogram is acquired by server 1. This fluorescein angiogram information is accompanied by a diagnostic note from ophthalmologist D regarding at least one of retinal nonperfusion areas (NPA) and neovascularization (NV). This diagnostic note constitutes NPA / NV teaching information together with the fluorescein angiogram information as NPA / NV annotation information. When a fundus image of a patient is captured during an eye fundus examination, fundus image information based on this fundus image is acquired by the server 1. The fundus image information and NPA / NV teacher information are used for calculation processing by the NPA / NV annotation program of the calculation unit 104. As a result of the calculation processing by the NPA / NV annotation program, an NPA / NV existence probability map is generated. Based on this NPA / NV existence probability map, estimated NPA and estimated NV in the fundus image information are identified.

[0040] Ophthalmologist D may add diagnostic notes about incidental findings to the fluorescein angiography image information and fundus image information. In this case, the diagnostic notes constitute incidental finding annotation information together with the fluorescein angiography image information and fundus image information. The fundus image information and the NPA / NV teacher information are used in the calculation process by the incidental finding determination program of the calculation unit 104. As a result of the calculation process by the incidental finding determination program, an incidental finding presence probability map is generated. When the incidental finding presence probability map is generated in this way, estimated NPA and estimated NV in the fundus image information are identified based on the incidental finding presence probability map and the NPA / NV teacher information.

[0041] FIG. 6 is a diagram showing an example of fundus image information acquired in the process executed by the server 1. As shown in FIG. FIG. 7 is a diagram showing an example of fluorescein fundus angiography image information acquired in the process executed by the server 1. As shown in FIG.

[0042] During a patient's fundus examination, an image of the patient's fundus is taken using examination device 3, and fundus image information such as that shown in Figure 6 is obtained. Ophthalmologist D performs the examination while referring to the fundus image information shown in Figure 6. However, as shown in Figure 6, the fundus image information only reveals findings resulting from perfusion abnormalities, such as hemorrhage and exudates, and it is difficult to interpret the circulatory dynamics or the location of circulatory abnormalities. For this reason, a fluorescein fundus angiography is performed, and fluorescein fundus angiography image information such as that shown in Figure 7 is obtained. Fluorescein fundus angiography image information, as shown in Figure 7, can clearly determine the retinal circulatory dynamics. However, fluorescein fundus angiography is a burden on both patients and medical professionals due to its risk, invasiveness, and geographical constraints limited to large hospitals. Therefore, hesitation to take images of patients with good vision or reduced physical function, or to take repeated images, often leads to delayed understanding of the onset and progression of the disease, which delays treatment. Therefore, the server 1 performs the process of generating an NPA / NV existence probability map using accumulated fluorescein fundus angiography image information obtained from multiple patients and NPA / NV teacher information generated based on the NPA / NV annotation information attached to each of these pieces of information. This makes it possible to easily identify circulatory abnormalities from fundus images that can be easily taken at a nearby clinic or for medical checkups, without having to perform fluorescein fundus angiography, which is a burdensome test.

[0043] FIG. 8 is a diagram showing an example of an NPA·NV existence probability map output in the NPA·NV existence probability map generation process executed by the server 1. As shown in FIG.

[0044] The NPA / NV existence probability map E shown in Figure 8 is image information in which the NPA existence probability and the NV existence probability are displayed in fundus image information in a distinguishable manner. The NPA / NV existence probability map E can distinguish the NPA existence probability and the NV existence probability by using different colors or shades of color. For example, it is possible to distinguish the NPA existence probability and the NV existence probability in areas indicated by warm colors, and the NPA existence probability and the NV existence probability in areas indicated by cool colors, and to distinguish ... dark colors, and to distinguish the NPA existence probability and the NV existence probability in areas indicated by light colors. Furthermore, even in areas indicated by cool colors, where the NPA existence probability and the NV existence probability are low, it is also possible to distinguish the NPA existence probability and the NV existence probability in darker colors, and to distinguish the NPA existence probability and the NV existence probability in darker colors, and to distinguish the NPA existence probability and the NV existence probability in lighter colors. This makes it possible to generate information that serves as a basis for estimating the presence of retinal nonperfusion areas (NPA) and neovascularization (NV) in fundus image information.

[0045] FIG. 9 is a diagram showing an example of an estimated NPA and an estimated NV identified in the estimated NPA / NV identification process executed by the server 1. As shown in FIG.

[0046] Based on the contents of the NPA / NV existence probability map, the server 1 identifies, in the fundus image information, an area estimated to correspond to a retinal nonperfusion area (NPA) as an estimated NPA, and an area estimated to correspond to neovascularization (NV) as an estimated NV. For example, as shown in Figure 9, area A indicated by a dashed line can be determined as an estimated NPA, and area B can be determined as an estimated NV. The estimated NPA and estimated NV can be displayed superimposed on the fundus image information. This makes it possible to quickly and easily identify areas where retinal aperfusion or neovascularization is suspected from fundus images taken of a patient's fundus, without having to perform a fluorescein angiography test, which requires a special fundus camera or diagnostic device.

[0047] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope of achieving the object of the present invention are included in the present invention.

[0048] For example, in the above-described embodiment, the image acquiring unit 101 is configured to acquire various types of image information when the information is transmitted from the inspection device 3, but the image acquiring unit 101 may be configured to acquire various types of image information on its own initiative when an image is captured by the inspection device 3. Similarly, the annotation acquiring unit 102 is configured to acquire various types of annotation information when the information is transmitted from the ophthalmologist terminal 2, but the annotation acquiring unit 102 may be configured to acquire various types of annotation information on its own initiative when the various types of annotation information are input to the ophthalmologist terminal 2.

[0049] Furthermore, the hardware configurations shown in FIG. 2 are merely examples for achieving the object of the present invention, and are not particularly limited.

[0050] 3 is merely an example and is not particularly limited. That is, it is sufficient for the information processing system to have the function of executing the above-described series of processes as a whole, and the type of functional block used to realize this function is not particularly limited to the example in FIG.

[0051] Furthermore, the locations of the functional blocks are not limited to those shown in Fig. 3 and may be arbitrary. For example, at least some of the functional blocks of the server 1 may be provided in the ophthalmologist terminal 2 or the examination device 3. A single functional block may be configured by a single piece of hardware, or may be configured in combination with a single piece of software.

[0052] When the processing of each functional block is performed by software, the program that constitutes the software is installed into a computer or the like from a network or a recording medium. The computer may be a computer built into dedicated hardware, or may be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0053] The recording medium containing such a program may not only be composed of removable media that is distributed separately from the device itself in order to provide the program to each user, but may also be composed of recording media that are provided to each user in a state where they are pre-installed in the device itself.

[0054] In this specification, the steps describing the program to be recorded on the recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually. For example, in step S6 of FIG. 4, the image acquisition unit 101 determines whether fundus image information has been transmitted from the examination device 3, and acquires the fundus image information in step S7. However, when the incidental finding existence probability map generation process is executed, it is sufficient for the fundus image information to be appropriately managed at the time when the incidental finding existence probability map generation process is executed. Furthermore, when the incidental finding existence probability map generation process is not executed, it is sufficient for the fundus image information to be appropriately managed at the time when the NPA existence probability and the NV existence probability are calculated. Therefore, when the incidental finding existence probability map generation process is executed, it is sufficient for the fundus image information to be stored and managed in the image DB 401 from any time point before the incidental finding existence probability is generated in step S10. Furthermore, when the incidental finding existence probability map generation process is not executed, it is sufficient for the fundus image information to be stored and managed in the image DB 401 from any time point before the NPA existence probability and the NV existence probability are calculated in step S13.

[0055] In summary, the program to which the present invention is applied is sufficient if it has the following configuration, and can take on a variety of different embodiments. That is, the program to which the present invention is applied is A computer that controls an information processing device a fluorescent fundus angiography image acquisition step (e.g., step S2 in FIG. 4) for acquiring fluorescent fundus angiography image information (e.g., fluorescent fundus angiography image information C in FIG. 7); an NPA / NV annotation acquisition step (e.g., step S4 in FIG. 4 ) of acquiring, as NPA / NV annotation information, diagnostic notes by an ophthalmologist regarding at least one of retinal nonperfusion areas (NPA) and neovascularization (NV) attached to the fluorescein fundus angiography image information; an NPA / NV teacher information generation step (e.g., step S5 in FIG. 4 ) of generating NPA / NV teacher information serving as teacher information for calculating the probability of existence of the retinal nonperfusion area (NPA) and the probability of existence of neovascularization (NV) based on the fluorescein angiography image information and the NPA / NV annotation information corresponding to the fluorescein angiography image information; a fundus image acquisition step (e.g., step S7 in FIG. 4) for acquiring fundus image information (e.g., fundus image information F in FIG. 6); an NPA / NV existence probability calculation step (e.g., step S13 in FIG. 4) of calculating the existence probability of the retinal nonperfusion area (NPA) and the existence probability of neovascularization (NV) in the fundus image information based on the NPA / NV teacher information; an estimated NPA / NV identification step (e.g., step S15 in FIG. 4) of identifying a region of the fundus image information that is estimated to correspond to the retinal non-perfusion region (NPA) as an estimated NPA and identifying a region of the fundus image information that is estimated to correspond to the neovascularization (NV) as an estimated NV based on the probability of existence of the retinal non-perfusion region and the probability of existence of the neovascularization; Includes. This makes it possible to quickly and easily identify areas where retinal nonperfusion is suspected from fundus images taken of a patient's fundus, without having to perform a fluorescein angiography test, which requires a special fundus camera or diagnostic device.

[0056] In addition, a control process can be executed that further includes an NPA / NV existence probability map generation step (e.g., step S14 in Figure 4) that generates an NPA / NV existence probability map (e.g., NPA / NV existence probability map E in Figure 8) in which the NPA existence probability and the NV existence probability are distinguishably displayed in the fundus image information. This makes it possible to generate information that serves as a basis for estimating the presence of retinal nonperfusion areas (NPA) and neovascularization (NV) in fundus image information.

[0057] and an accompanying annotation acquisition step (e.g., step S9 in FIG. 4) of acquiring, as accompanying finding annotation information, diagnostic notes by the ophthalmologist regarding accompanying findings attached to the fluorescein fundus angiography image information and the fundus image information. an incidental finding teacher information generating step (e.g., step S10 in FIG. 4) of generating incidental finding teacher information serving as teacher information for calculating the presence probability of the incidental finding in the fundus image information based on the fluorescent fundus angiography image information, the fundus image information, and the incidental finding annotation information corresponding to the fluorescent fundus angiography image information and the fundus image information; an incidental finding existence probability calculation step (for example, step S11 in FIG. 4) of calculating the existence probability of the incidental finding in the fundus image information based on the incidental finding teacher information; further comprising In the NPA / NV existence probability calculation step, A control process can be executed to calculate the probability of presence of the retinal nonperfusion area (NPA) and the probability of presence of neovascularization (NV) in the fundus image information based on the probability of presence of the associated findings and the NPA / NV teaching information.

[0058] The method may further include an incidental finding existence probability map generating step (for example, step S12 in FIG. 4) of generating an incidental finding existence probability map in which the fundus image information is displayed with the probability of the presence of the incidental finding.

[0059] A modified example of one embodiment of the present invention will be described below with reference to the drawings. (Variation) 10 is a configuration diagram of an information processing system according to a modified example of an embodiment of the present invention. The information processing system according to a modified example of an embodiment of the present invention includes an ophthalmologist terminal 2, an examination device 3, a learning device 200, and a diagnosis support device 300. These devices are connected to each other via a network N.

[0060] The learning device 200 generates a learning model that represents the relationship between a fundus image, which is an image of the fundus, and an area of ​​abnormal blood circulation identified based on a fluorescein angiography image of the fundus, through learning. Here, the area of ​​abnormal blood circulation is an area of ​​abnormal blood circulation caused by a blood disorder on the retina that occurs in ocular ischemic diseases such as diabetic retinopathy. The learning device 200 acquires information indicating a fundus image of a patient and information indicating a fluorescein angiography image of the patient, associates the information indicating the acquired fundus image with the information indicating the fluorescein angiography image, and stores the information. Specifically, in a fundus examination of a patient, the examination device 3 captures the fundus image of the patient, creates fundus image notification information that includes the patient ID and information indicating the captured fundus image and is addressed to the learning device 200, and transmits the created fundus image notification information to the learning device 200. Also, in a fluorescein angiography examination of a patient, the examination device 3 captures the fluorescein angiography image of the patient, creates fluorescein angiography image notification information that includes the patient ID and information indicating the captured fluorescein angiography image and is addressed to the learning device 200, and transmits the created fluorescein angiography image notification information to the learning device 200.

[0061] The learning device 200 acquires the patient ID and information indicating the fundus image contained in the fundus image notification information sent from the inspection equipment 3 to the learning device 200, and the patient ID and information indicating the fundus image contained in the fluorescent fundus angiography image notification information, and stores the acquired patient ID, information indicating the fundus image, and information indicating the fluorescent fundus angiography image in association with each other.

[0062] The learning device 200 acquires diagnostic notes from ophthalmologist D regarding either or both of retinal nonperfusion areas (NPA) and neovascularization (NV) that are added to a patient's fluorescein angiography image as NPA / NV annotation information. Specifically, when ophthalmologist D's diagnostic notes regarding retinal nonperfusion areas (NPA) and neovascularization (NV) are added to a fluorescein angiography image during a fluorescein angiography examination, the ophthalmologist terminal 2 creates NPA / NV annotation notification information that includes the patient ID and the diagnostic notes and is addressed to the learning device 200 based on the operation of ophthalmologist D, and transmits the created NPA / NV annotation notification information to the learning device 200. The learning device 200 receives the NPA / NV annotation notification information sent by the ophthalmologist terminal 2 and stores the NPA / NV annotation information included in the received NPA / NV annotation notification information. Note that the information indicating the fluorescent fundus angiography image and the NPA / NV annotation information attached to this fluorescent fundus angiography image are stored in association with each other.

[0063] The learning device 200 acquires diagnostic notes by ophthalmologist D regarding incidental findings that are attached to the fundus image and the fluorescein angiography image as incidental finding annotation information. Specifically, when diagnostic notes by ophthalmologist D regarding incidental findings are attached to information indicating the fluorescein angiography image and information indicating the fundus image, the ophthalmologist terminal 2 creates incidental finding annotation notification information addressed to the learning device 200, including the patient ID and the diagnostic notes, based on the operation of ophthalmologist D, and transmits the created incidental finding annotation notification information to the learning device 200. The learning device 200 receives the incidental finding annotation notification information sent by the ophthalmologist terminal 2, acquires the patient ID and incidental finding annotation information included in the received incidental finding annotation notification information, and stores the acquired patient ID and incidental finding annotation information. Note that the information indicating the fundus image, the information indicating the fluorescein angiography image, and the incidental finding annotation information are stored in association with each other.

[0064] The learning device 200 acquires information representing a fundus image, information representing a fluorescein angiogram associated with the information representing the fundus image, NPA / NV annotation information, and associated finding annotation information, and identifies an area of ​​abnormal blood circulation based on the acquired information representing the fluorescein angiogram, the NPA / NV annotation information, and the associated finding annotation information. The learning device 200 generates NPA / NV learning information that associates the information representing the fundus image with an area of ​​abnormal blood circulation identified based on the fluorescein angiogram corresponding to the fundus image, and stores the generated NPA / NV learning information. The learning device 200 uses information indicating a fundus image included in the NPA / NV learning information as input information and areas of abnormal blood circulation identified based on a fluorescein angiogram corresponding to the fundus image as training information to generate a learning model that represents the relationship between the fundus image and the areas of abnormal blood circulation in the fundus image through learning. The specific method for generating the learning model is not particularly limited. For example, common features of fundus images containing areas of abnormal blood circulation may be extracted from the NPA / NV learning information, and the relationship between the extracted common features and the areas of abnormal blood circulation in the fundus image may be derived. The relationship between the fundus image and the areas of abnormal blood circulation in the fundus image may also be derived using neural network or deep learning techniques. The learning device 200 stores the generated learning model and generates learning model notification information that includes the generated learning model and is addressed to the diagnostic support device 300. The generated learning model notification information is then transmitted to the diagnostic support device 300.

[0065] The diagnosis support device 300 receives the learning model transmitted by the learning device 200 and stores the received learning model.

[0066] The ophthalmologist terminal 2 creates patient information including a patient ID and information indicating a fundus image of the patient, and addressed to the diagnosis support device 300, and transmits the created patient information to the diagnosis support device 300.

[0067] The diagnostic support device 300 receives the patient information transmitted from the ophthalmologist terminal 2 and acquires the patient ID and information indicating the patient's fundus image included in the received patient information. The diagnostic support device 300 uses the stored learning model to identify an area of ​​abnormal blood circulation in the fundus image based on the information indicating the acquired fundus image. The diagnostic support device 300 creates a diagnostic result addressed to the ophthalmologist terminal 2, which includes the patient's fundus image, information indicating the area of ​​abnormal blood circulation in the fundus image identified using the learning model, and the patient ID, and transmits the created diagnostic result to the ophthalmologist terminal 2. The learning device 200 and the diagnostic support device 300 included in the information processing system will be described below.

[0068] (Learning device 200) FIG. 11 is a block diagram showing an example of a learning device according to a modified example of the embodiment of the present invention. The learning device 200 includes a communication unit 205, a memory unit 210, an operation unit 220, an information processing unit 230, a display unit 240, and a bus line 250 such as an address bus or a data bus for electrically connecting each component as shown in FIG. 11.

[0069] The communication unit 205 is realized by a communication module. The communication unit 205 communicates with external communication devices such as the ophthalmologist terminal 2, the examination device 3, and the diagnosis support device 300 via the network N. Specifically, the communication unit 205 receives fundus image notification information transmitted by the examination device 3 and outputs the received fundus image notification information to the information processing unit 230. The communication unit 205 receives fluorescein angiography image notification information transmitted by the examination device 3 and outputs the received fluorescein angiography image notification information to the information processing unit 230. The communication unit 205 receives NPA / NV annotation notification information transmitted by the ophthalmologist terminal 2 and outputs the received NPA / NV annotation notification information to the information processing unit 230. The communication unit 205 receives associated finding annotation notification information transmitted by the ophthalmologist terminal 2 and outputs the received associated finding annotation notification information to the information processing unit 230. The communication unit 205 acquires the learning model notification information output by the information processing unit 230 and transmits the acquired learning model notification information to the diagnosis assistance device 300.

[0070] The storage unit 210 is realized by, for example, a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a flash memory, or a hybrid storage device that combines two or more of these. The storage unit 210 stores a program 211 executed by the information processing unit 230, an application 212, an image DB 213, an annotation DB 214, learning information 215, and a learning model 216.

[0071] The program 211 is, for example, an operating system, which is positioned between the user or application program and the hardware, and provides a standard interface to the user or application program, while at the same time efficiently managing each resource such as the hardware.

[0072] The application 212 causes the learning device 200 to receive fundus image notification information sent by the inspection device 3, and associates and stores the patient ID included in the received fundus image notification information with information indicating the fundus image. The application 212 causes the learning device 200 to receive fluorescein angiography image notification information sent by the inspection device 3, and associates and stores the patient ID included in the received fluorescein angiography image notification information with information indicating the fluorescein angiography image. The application 212 causes the learning device 200 to receive NPA / NV annotation notification information sent by the ophthalmologist terminal 2, and associates and stores the patient ID included in the received NPA / NV annotation notification information with the NPA / NV annotation information.

[0073] The application 212 causes the learning device 200 to receive incidental finding annotation notification information transmitted by the ophthalmologist terminal 2, and to store the patient ID included in the received incidental finding annotation notification information in association with the incidental finding annotation information. The application 212 causes the learning device 200 to acquire information indicating a fundus image associated with the patient ID, information indicating a fluorescein angiography image, NPA / NV annotation information, and incidental finding annotation information. The application 212 causes the learning device 200 to identify a region of abnormal blood circulation based on the information indicating the fluorescein angiography image, NPA / NV annotation information, and incidental finding annotation information that it has acquired. The application 212 causes the learning device 200 to generate NPA / NV learning information that associates information indicating a fundus image with an area of ​​abnormal blood circulation identified based on a fluorescein angiogram corresponding to the fundus image, and stores the generated NPA / NV learning information. The application 212 causes the learning device 200 to generate a learning model by learning, using information indicating the fundus image included in the NPA / NV learning information as input information and an area of ​​abnormal blood circulation identified based on a fluorescein angiogram corresponding to the fundus image as training information, to represent the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image. The application 212 causes the learning device 200 to store the generated learning model and transmit it to the diagnosis support device 300.

[0074] The image DB 213 stores patient IDs, information indicating fundus images, and information indicating fluorescein angiographic images in association with each other.

[0075] The annotation DB 214 stores patient IDs, NPA / NV annotation information, and accompanying finding annotation information in association with each other.

[0076] The learning information 215 stores NPA / NV learning information that associates information indicating a fundus image with an abnormal blood circulation region identified based on a fluorescein angiographic image corresponding to the fundus image.

[0077] The learning model 216 uses information indicating a fundus image included in the NPA / NV learning information as input information, and uses an area of ​​abnormal blood circulation identified based on a fluorescent fundus angiography image corresponding to the fundus image as teaching information, and stores a learning model that represents the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image.

[0078] The operation unit 220 is configured by, for example, a touch panel, detects a touch operation on a screen displayed on the display unit 240, and outputs the detection result of the touch operation to the information processing unit 230.

[0079] The display unit 240 is configured, for example, by a touch panel, and displays a screen for accepting information indicating a fundus image and information indicating a fluorescein angiography image received by the learning device 200. The display unit 240 also displays a screen for accepting operations for processing the NPA / NV annotation information and associated finding annotation information received by the learning device 200.

[0080] All or part of the information processing unit 230 is a functional unit (hereinafter referred to as a software functional unit) realized by, for example, a processor such as a CPU (Central Processing Unit) executing a program 211 and an application 212 stored in the storage unit 210. Note that all or part of the information processing unit 230 may be realized by hardware such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array), or may be realized by a combination of a software functional unit and hardware. The information processing unit 230 includes, for example, an image acquisition unit 231, an annotation acquisition unit 232, a learning information generation unit 233, and a learning unit 234.

[0081] The image acquisition unit 231 acquires the fundus image notification information output by the communication unit 205, and acquires the patient ID and information indicating the fundus image included in the acquired fundus image notification information. The image acquisition unit 231 associates the acquired patient ID with the information indicating the fundus image and stores them in the image DB 213. The image acquisition unit 231 acquires the fluorescein angiography image notification information output by the communication unit 205, and acquires the patient ID and information indicating the fluorescein angiography image included in the acquired fluorescein angiography image notification information. The image acquisition unit 231 associates the acquired patient ID with the information indicating the fluorescein angiography image and stores them in the image DB 213.

[0082] The annotation acquisition unit 232 acquires the NPA / NV annotation notification information output by the communication unit 205, and acquires the patient ID and NPA / NV annotation information included in the acquired NPA / NV annotation notification information. The annotation acquisition unit 232 associates the acquired patient ID and NPA / NV annotation information and stores them in the annotation DB 214. The annotation acquisition unit 232 acquires the incidental finding annotation notification information output by the communication unit 205, and acquires the patient ID and incidental finding annotation information included in the acquired incidental finding annotation notification information. The annotation acquisition unit 232 associates the acquired patient ID and incidental finding annotation information and stores them in the annotation DB 214.

[0083] The learning information generation unit 233 acquires a patient ID stored in the image DB 213 of the storage unit 210, information indicating a fundus image associated with the patient ID, and information indicating a fluorescein angiography image. The learning information generation unit 233 acquires a patient ID stored in the annotation DB 214 of the storage unit 210, NPA / NV annotation information associated with the patient ID, and associated finding annotation information. The learning information generation unit 233 identifies an area of ​​abnormal blood circulation based on the acquired information indicating the fluorescein angiography image, the NPA / NV annotation information, and the associated finding annotation information.

[0084] Each component of the learning device 200 will be specifically described below. FIG. 12 is a diagram showing an example of a fundus image, and FIG. 13 is a diagram showing an example of a fluorescein fundus angiography image. The learning information generation unit 233 extracts the green component of the fundus image and removes noise components from the green-component-extracted fundus image, which is a fundus image from which the green component has been extracted. The learning information generation unit 233 divides the green-component-extracted fundus image from which the noise component has been removed into rectangles. The learning information generation unit 233 resizes the green-component-extracted fundus image from which the noise component has been removed to a predetermined size based on the image divided into rectangles. When resizing to the predetermined size, interpolation is performed using an interpolation method such as bicubic interpolation. The learning information generation unit 233 extracts the green component from the fluorescent fundus angiography image and removes noise components from the green-component-extracted fluorescent fundus angiography image, which is the fluorescent fundus angiography image from which the green component has been extracted. The learning information generation unit 233 divides the green-component-extracted fluorescent fundus angiography image from which the noise component has been removed into rectangles. The learning information generation unit 233 resizes the green-component-extracted fluorescent fundus angiography image from which the noise component has been removed to a predetermined size based on the image divided into rectangles. When resizing to the predetermined size, interpolation is performed using an interpolation method such as bicubic interpolation.

[0085] The learning information generation unit 233 corrects the rotational components of the green component extracted fundus image from which the noise components have been removed and the green component extracted fluorescent fundus angiography image from which the noise components have been removed so that the positions of the eyeballs match, and resizes them to a predetermined size. The learning information generation unit 233 corrects the rotational components of the green component extracted fundus image from which the noise components have been removed and resized to a predetermined size so that the positions of the eyeballs match, and identifies an area of ​​abnormal blood circulation based on the green component extracted fundus image from which the noise components have been removed and the green component extracted fluorescent fundus angiography image from which the noise components have been removed.

[0086] FIG. 14 is a diagram showing an example of an abnormal blood circulation region. The learning information generating unit 233 generates NPA / NV learning information that associates information indicating a fundus image with an area of ​​abnormal blood circulation identified based on a fluorescein angiography image corresponding to the fundus image, and stores the generated NPA / NV learning information in learning information 215 of the storage unit 210. Returning to FIG. 11, the explanation will be continued.

[0087] The learning unit 234 acquires the NPA / NV learning information stored in the learning information 215 of the storage unit 210. The learning unit 234 acquires information indicating a fundus image and information indicating an area of ​​abnormal blood circulation included in the acquired NPA / NV learning information. The learning unit 234 uses the information indicating the acquired fundus image as input information and the area of ​​abnormal blood circulation identified based on the fluorescein angiography image corresponding to the fundus image as training information, and generates a learning model that represents the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image through learning.

[0088] Specifically, in this modified example of the present embodiment, it is assumed that it is difficult to learn and predict the entire image at once due to memory limitations of a GPU (graphics processing unit), so a method is used in which patches are extracted from an image and the extracted patches are trained by a neural network. Here, as an example, the patch size is set to 64px x 64px, and the stride (the interval at which the frame for patch extraction is moved) is set to 2px.

[0089] The generated patches are divided into two groups: those containing positive regions and those containing none. Patches are selected so that the proportion of each group used for training is equal. The phenomenon in which a neural network performs well only on the training data itself or images that are very similar to it, and performs significantly poorly on unknown images, is called over-fitting, and can be resolved by collecting more samples or applying geometric operations such as rotation to the training data. In a modification of this embodiment, after generating a patch, the rotation angle is determined based on a normal distribution of σ=3 degrees, and horizontal flipping is performed with a 50% probability and vertical flipping with a 20% probability.

[0090] FIG. 15 is a diagram illustrating an example of the structure of a neural network. An example of a neural network structure is based on U-Net. In Figure 15, b represents convolution (kernel_size=(3,3)), g represents max pooling (pool_size=(2,2)), and o represents up sampling (size=(2,2)). After each convolution layer, the ReLU activation function was used and batch normalization was performed. However, the last convolution layer used a sigmoid activation function and did not undergo batch normalization.

[0091] The arrows a1, a2, a3, and a4 represent skip connections due to concatenation, which are thought to contribute to restoring the positional information of the image.

[0092] The learning unit 234 stores the generated learning model in the learning model 216 of the storage unit 210. The learning unit 234 creates learning model notification information that includes the generated learning model and is addressed to the diagnostic support device 300, and outputs the created learning model notification information to the communication unit 205.

[0093] (Diagnosis support device 300) FIG. 16 is a block diagram showing an example of a diagnosis support device according to a modified example of the embodiment of the present invention. The diagnostic support device 300 includes a communication unit 305, a memory unit 310, an operation unit 320, an information processing unit 330, a display unit 340, and a bus line 350 such as an address bus or a data bus for electrically connecting each component as shown in FIG. 16.

[0094] The communication unit 305 is realized by a communication module. The communication unit 305 communicates with external communication devices such as the ophthalmologist terminal 2 and the learning device 200 via the network N. Specifically, the communication unit 305 receives learning model notification information transmitted by the learning device 200 and outputs the received learning model notification information to the information processing unit 330. The communication unit 305 receives patient information transmitted by the ophthalmologist terminal 2 and outputs the received patient information to the information processing unit 330. The communication unit 305 acquires diagnostic information output by the information processing unit 230 and transmits the acquired diagnostic information to the ophthalmologist terminal 2.

[0095] The storage unit 310 is realized by, for example, a RAM, a ROM, a HDD, a flash memory, or a hybrid storage device that combines two or more of these. The storage unit 310 stores a program 311, an application 312, and a learning model 216 that are executed by the information processing unit 330.

[0096] The program 311 is, for example, an operating system, which is positioned between the user or application program and the hardware, and provides a standard interface to the user or application program, while at the same time efficiently managing each resource such as the hardware.

[0097] The application 312 causes the diagnostic support device 300 to receive learning model notification information transmitted by the learning device 200 and store the learning model included in the received learning model notification information. The application 312 causes the diagnostic support device 300 to receive patient information transmitted by the ophthalmologist terminal 2 and acquire a patient ID and a fundus image included in the received patient information. The application 312 causes the diagnostic support device 300 to identify an area of ​​abnormal blood circulation in the acquired fundus image using the stored learning model. The application 312 causes the diagnostic support device 300 to create diagnostic information addressed to the ophthalmologist terminal 2, the diagnostic information including a fundus image of the patient, information indicating areas of abnormal blood circulation in the fundus image identified using a learning model, and a patient ID, and to transmit the created diagnostic information to the ophthalmologist terminal 2.

[0098] The operation unit 320 is configured by, for example, a touch panel, detects a touch operation on a screen displayed on the display unit 340, and outputs the detection result of the touch operation to the information processing unit 330.

[0099] The display unit 340 is configured with, for example, a touch panel, and displays a screen that receives information indicating a fundus image included in the patient information received by the diagnosis support device 300. The display unit 240 also displays the results of the diagnosis made by the diagnosis support device 300.

[0100] All or part of the information processing unit 330 is a software function unit realized by, for example, a processor such as a CPU executing a program 311 and an application 312 stored in the storage unit 310. Note that all or part of the information processing unit 330 may be realized by hardware such as an LSI, an ASIC, or an FPGA, or may be realized by a combination of a software function unit and hardware. The information processing unit 330 includes, for example, a receiving unit 331, an identifying unit 332, and a creating unit 333.

[0101] The receiving unit 331 acquires the learning model notification information output by the communication unit 305, and acquires the learning model included in the acquired learning model notification information. The receiving unit 331 accepts the acquired learning model, and stores the accepted learning model in the learning model 216 of the storage unit 310. The receiving unit 331 acquires the patient information output by the communication unit 305, and acquires the patient ID and information indicating the fundus image included in the acquired patient information. The receiving unit 331 accepts the acquired patient ID and information indicating the fundus image, and outputs the accepted patient ID and information indicating the fundus image to the identifying unit 332.

[0102] The identification unit 332 acquires the patient ID and information indicating the fundus image output by the reception unit 331. The identification unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310, and identifies an area of ​​abnormal blood circulation in the acquired fundus image using the acquired learning model. The identification unit 332 outputs the patient ID and information indicating the identified area of ​​abnormal blood circulation in the fundus image to the creation unit 333. Specifically, in this modification of the present embodiment, a case will be described in which patches are extracted from an image, and the extracted patches and a learning model are used to identify an area of ​​abnormal blood circulation in a fundus image, similar to the learning device 200. Here, as an example, the patch size is set to 64px x 64px, and the stride (the interval at which the frame for patch extraction is moved) is set to 2px. All of the generated patches are selected. The determination unit 332 acquires a 64x64x1 image based on the learning model. The determination unit 332 votes the acquired pixel values ​​with the corresponding pixels in the original image and averages them. Here, the determination unit 332 may convert the acquired image into a color display.

[0103] The creation unit 333 acquires the patient ID and information indicating the area of ​​abnormal blood circulation in the fundus image output by the identification unit 332. The creation unit 333 creates diagnostic information including the acquired patient ID and information indicating the area of ​​abnormal blood circulation in the fundus image, and addressed to the ophthalmologist terminal 2. The creation unit 333 outputs the created diagnostic information to the communication unit 305.

[0104] (Operation of information processing system) An example of the operation of the information processing system according to the modified example of this embodiment will be described with reference to FIGS. 17 is a flowchart showing an example of the operation of the learning device included in the information processing system of the modified example of this embodiment. Fig. 17 shows the operation after the ophthalmologist terminal 2 transmits NPA / NV annotation notification information and associated finding annotation notification information to the learning device 200, and the inspection device 3 transmits fundus image notification information and fluorescein angiography image notification information to the learning device 200.

[0105] (Step S201) The communication unit 205 of the learning device 200 receives the fundus image notification information sent by the inspection equipment 3, and outputs the received fundus image notification information to the information processing unit 230. The image acquisition unit 231 of the information processing unit 230 acquires the fundus image notification information output by the communication unit 205, associates the patient ID included in the acquired fundus image notification information with information indicating the fundus image, and stores the associated information in the image DB 213 of the storage unit 210.

[0106] (Step S202) The communication unit 205 of the learning device 200 receives the fluorescent fundus angiography image notification information transmitted by the inspection equipment 3, and outputs the received fluorescent fundus angiography image notification information to the information processing unit 230. The image acquisition unit 231 of the information processing unit 230 acquires the fluorescent fundus angiography image notification information output by the communication unit 205, associates the patient ID included in the acquired fluorescent fundus angiography image notification information with information indicating the fluorescent fundus angiography image, and stores the associated information in the image DB 213 of the storage unit 210.

[0107] (Step S203) The communication unit 205 of the learning device 200 receives the NPA / NV annotation notification information sent by the ophthalmologist terminal 2, and outputs the received NPA / NV annotation notification information to the information processing unit 230. The annotation acquisition unit 232 of the information processing unit 230 acquires the NPA / NV annotation notification information output by the communication unit 205, associates the patient ID included in the acquired NPA / NV annotation notification information with the NPA / NV annotation information, and stores the association in the annotation DB 214 of the storage unit 210.

[0108] The communication unit 205 of the learning device 200 receives the incidental finding annotation notification information transmitted by the ophthalmologist terminal 2, and outputs the received incidental finding annotation notification information to the information processing unit 230. The annotation acquisition unit 232 of the information processing unit 230 acquires the incidental finding annotation notification information output by the communication unit 205, associates the patient ID included in the acquired incidental finding annotation notification information with the incidental finding annotation information, and stores the associated information in the annotation DB 214 of the storage unit 210.

[0109] (Step S204) The learning information generation unit 233 of the learning device 200 acquires a patient ID stored in the image DB 213 of the storage unit 210, information indicating a fundus image associated with the patient ID, and information indicating a fluorescein angiogram. The learning information generation unit 233 acquires a patient ID stored in the annotation DB 214 of the storage unit 210, NPA / NV annotation information associated with the patient ID, and associated finding annotation information. The learning information generation unit 233 identifies an area of ​​abnormal blood circulation based on the information indicating the acquired fluorescein angiogram, the NPA / NV annotation information, and the associated finding annotation information. The learning information generation unit 233 generates NPA / NV learning information that associates information indicating the fundus image with an area of ​​abnormal blood circulation identified based on the fluorescein angiogram corresponding to the fundus image, and stores the generated NPA / NV learning information in the learning information 215 of the storage unit 210.

[0110] (Step S205) The learning unit 234 of the learning device 200 acquires the NPA / NV learning information stored in the learning information 215 of the storage unit 210. The learning unit 234 acquires information indicating a fundus image and information indicating an area of ​​abnormal blood circulation included in the acquired NPA / NV learning information. The learning unit 234 uses the information indicating the acquired fundus image as input information and the area of ​​abnormal blood circulation identified based on the fluorescein angiography image corresponding to the fundus image as training information, and generates a learning model by learning that represents the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image. The learning unit 234 stores the generated learning model in the learning model 216 of the storage unit 210.

[0111] In the flowchart shown in FIG. 17, the order of steps S201, S202, and S203 may be changed. 17, the learning device 200 can identify areas of abnormal blood circulation based on information representing a fluorescein angiogram, NPA / NV annotation information, and accompanying findings annotation information. The learning device 200 uses information representing a fundus image as input information and areas of abnormal blood circulation identified based on a fluorescein angiogram corresponding to the fundus image as training information, and can generate a learning model by learning that represents the relationship between the fundus image and areas of abnormal blood circulation in the fundus image.

[0112] 18 is a flowchart showing an example of the operation of the diagnostic support device included in the information processing system of the modified example of this embodiment. Fig. 18 shows the operation after the learning device 200 transmits learning model notification information to the diagnostic support device 300 and the ophthalmologist terminal 2 transmits patient information to the diagnostic support device 300.

[0113] (Step S301) The communication unit 305 of the diagnostic support device 300 receives the learning model notification information transmitted by the learning device 200 and outputs the received learning model notification information to the information processing unit 330. The reception unit 331 acquires the learning model notification information output by the communication unit 305 and acquires the learning model included in the acquired learning model notification information. The reception unit 331 accepts the acquired learning model and stores the accepted learning model in the learning model 216 of the storage unit 310.

[0114] (Step S302) The communication unit 305 receives the patient information transmitted by the ophthalmologist terminal 2 and outputs the received patient information to the information processing unit 330. The reception unit 331 acquires the patient information output by the communication unit 305 and acquires the patient ID and information indicating the fundus image included in the acquired patient information. The reception unit 331 accepts the acquired patient ID and information indicating the fundus image and outputs the accepted patient ID and information indicating the fundus image to the identification unit 332.

[0115] (Step S303) The identification unit 332 acquires the patient ID and information indicating the fundus image output by the reception unit 331. The identification unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310.

[0116] (Step S304) The identification unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310, and identifies the abnormal blood circulation region in the fundus image, which is the identification target, using the acquired learning model. The identification unit 332 outputs information indicating the identified abnormal blood circulation region in the fundus image and the patient ID to the creation unit 333.

[0117] (Step S305) The creation unit 333 acquires the patient ID and information indicating the area of ​​abnormal blood circulation in the fundus image output by the identification unit 332. The creation unit 333 creates diagnostic information including the patient ID and information indicating the area of ​​abnormal blood circulation in the acquired fundus image, and addressed to the ophthalmologist terminal 2. The creation unit 333 outputs the created diagnostic information to the communication unit 305.

[0118] The communication unit 305 acquires the diagnostic information output by the creation unit 333 and transmits the acquired diagnostic information to the ophthalmologist terminal 2 . According to the flowchart shown in Figure 18, the diagnostic support device 300 uses information indicating a fundus image as input information, and can identify an area of ​​abnormal blood circulation in the fundus image that is the target of identification, using a learning model that represents the relationship between the fundus image generated using an area of ​​abnormal blood circulation identified based on a fluorescent fundus angiography image corresponding to the fundus image as training information, and the area of ​​abnormal blood circulation in the fundus image.

[0119] In the above-described modified example, the diagnostic support device 300 receives learning model notification information transmitted by the learning device 200 and stores the learning model included in the received learning model notification information in the storage unit 310. The diagnostic support device 300 identifies an area of ​​abnormal blood circulation in the fundus image using a fundus image included in the patient information transmitted by the ophthalmologist terminal 2 and the stored learning model, and transmits a diagnostic result including information indicating the identified area of ​​abnormal blood circulation to the ophthalmologist terminal 2. However, this is not a limitation. For example, the diagnostic support device 300 may transmit the patient information to the learning device 200. The learning device 200 receives the patient information transmitted by the diagnostic support device 300 and identifies an area of ​​abnormal blood circulation in the fundus image using the fundus image included in the received patient information and the learning model. The learning device 200 creates a diagnostic result including information indicating the identified area of ​​abnormal blood circulation and transmits the created diagnostic result to the diagnostic support device 300. The diagnostic support device 300 may receive the diagnostic result transmitted by the learning device 200 and transmit the received diagnostic result to the ophthalmologist terminal 2.

[0120] According to a modification of this embodiment, the diagnosis support device 300 includes an identification unit that identifies an area of ​​abnormal blood circulation in a fundus image based on the fundus image and an area of ​​abnormal blood circulation identified based on a fluorescein angiography image of the fundus using a learning model that has learned the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image, and an output unit that outputs information indicating the area of ​​abnormal blood circulation in the fundus image of the patient identified by the identification unit using the fundus image and the learning model. With this configuration, the diagnosis support device 300 can accurately estimate an area of ​​abnormal blood circulation from a normal fundus image using a learning model obtained by machine learning using information on the area of ​​abnormal blood circulation identified by a doctor from a fluorescein angiography image and the corresponding fundus image as training information. The abnormal blood circulation region is generated based on the fluorescein angiographic image and an ophthalmologist's diagnostic note regarding either or both of the retinal nonperfusion region and neovascularization, which is added to the fluorescein angiographic image. By configuring in this way, the abnormal blood circulation region to be used as training information can be generated based on the fluorescein angiographic image and an ophthalmologist's diagnostic note regarding either or both of the retinal nonperfusion region and neovascularization, which is added to the fluorescein angiographic image.

[0121] The identifying unit identifies either or both of a retinal nonperfusion region and a region corresponding to neovascularization in the fundus image. By configuring in this way, it is possible to identify either or both of a retinal nonperfusion region and a region corresponding to neovascularization with high accuracy from a normal fundus image.

[0122] The output unit outputs an image in which the abnormal blood circulation area identified by the identifying unit is superimposed on the fundus image. By configuring in this way, it is possible to obtain an image in which the abnormal blood circulation area is superimposed on the normal fundus image.

[0123] According to a modification of this embodiment, the learning device 200 has a learning unit that generates, by machine learning, a learning model that represents the relationship between a fundus image and an area of ​​abnormal blood circulation in the fundus image, based on the fundus image, which is an image of the fundus, and an area of ​​abnormal blood circulation identified based on a fluorescein angiographic image of the fundus. By configuring in this way, the learning device 200 can generate, by machine learning, a learning model that represents the relationship between the fundus image and an area of ​​abnormal blood circulation in the fundus image.

[0124] The abnormal blood circulation region is generated based on the fluorescein angiographic image and an ophthalmologist's diagnostic note regarding either or both of the retinal nonperfusion region and neovascularization, which is added to the fluorescein angiographic image. By configuring in this way, the abnormal blood circulation region to be used as training information can be generated based on the fluorescein angiographic image and an ophthalmologist's diagnostic note regarding either or both of the retinal nonperfusion region and neovascularization, which is added to the fluorescein angiographic image.

[0125] Although the embodiments and their modifications of the present invention have been described above, these embodiments and their modifications are presented as examples and are not intended to limit the scope of the invention. These embodiments and their modifications can be embodied in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. The server 1, ophthalmologist terminal 2, testing device 3, learning device 200, and diagnosis support device 300 each have a computer built in. The processing steps of each of the aforementioned devices are stored in the form of a program on a computer-readable recording medium, and the computer reads and executes this program to perform the above processing. Here, computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc. Alternatively, the computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute the program. The program may also be for realizing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0126] In addition, the following supplementary notes are provided in relation to the above description. (Appendix 1) A computer that controls an information processing device, a fluorescein fundus angiography image acquisition step of acquiring fluorescein fundus angiography image information; an NPA / NV annotation acquisition step of acquiring, as NPA / NV annotation information, diagnostic notes by an ophthalmologist regarding at least one of a retinal nonperfusion area and neovascularization, which are added to the fluorescein fundus angiography image information; an NPA / NV teacher information generating step of generating NPA / NV teacher information serving as teacher information for calculating the probability of the presence of the retinal nonperfusion region and the probability of the presence of neovascularization based on the fluorescein angiography image information and the NPA / NV annotation information corresponding to the fluorescein angiography image information; a fundus image acquiring step of acquiring fundus image information; an NPA / NV existence probability calculation step of calculating the existence probability of the retinal non-perfusion region and the existence probability of the neovascularization in the fundus image information based on the NPA / NV teacher information; an estimated NPA / NV identification step of identifying a region of the fundus image information estimated to correspond to the retinal nonperfusion region as an estimated NPA and identifying a region of the fundus image information estimated to correspond to the neovascularization as an estimated NV based on the probability of existence of the retinal nonperfusion region and the probability of existence of the neovascularization; A program that executes control processing including: (Appendix 2) The method further includes a step of generating an NPA / NV existence probability map in which the NPA existence probability and the NV existence probability are displayed in the fundus image information in a distinguishable manner. The program described in Appendix 1. (Supplementary Note 3) An accompanying annotation acquisition step of acquiring, as accompanying finding annotation information, diagnostic notes by the ophthalmologist regarding accompanying findings attached to the fluorescein fundus angiography image information and the fundus image information; an incidental finding teacher information generating step of generating incidental finding teacher information serving as teacher information for calculating the probability of presence of the incidental finding in the fundus image information, based on the fluorescent fundus angiography image information, the fundus image information, and the incidental finding annotation information corresponding to the fluorescent fundus angiography image information and the fundus image information; an incidental finding existence probability calculation step of calculating an existence probability of the incidental finding in the fundus image information based on the incidental finding teacher information; further comprising In the NPA / NV existence probability calculation step, execute a control process for calculating the probability of presence of the retinal nonperfusion region and the probability of presence of neovascularization in the fundus image information based on the probability of presence of the associated findings and the NPA teacher information; 1. A program according to claim 1 or 2. (Appendix 4) The method further includes an incidental finding existence probability map generating step of generating an incidental finding existence probability map in which the presence probability of the incidental finding is discriminated and displayed on the fundus image information. The program described in Appendix 3.

[0127] (Supplementary Note 5) An information processing method executed by an information processing device, a fluorescein fundus angiography image acquisition step of acquiring fluorescein fundus angiography image information; an NPA / NV annotation acquisition step of acquiring, as NPA / NV annotation information, diagnostic notes by an ophthalmologist regarding at least one of a retinal nonperfusion area and neovascularization, which are added to the fluorescein fundus angiography image information; an NPA / NV teacher information generating step of generating NPA / NV teacher information serving as teacher information for calculating the probability of the presence of the retinal nonperfusion region and the probability of the presence of neovascularization based on the fluorescein angiography image information and the NPA / NV annotation information corresponding to the fluorescein angiography image information; a fundus image acquiring step of acquiring fundus image information; an NPA / NV existence probability calculation step of calculating the existence probability of the retinal non-perfusion region and the existence probability of the neovascularization in the fundus image information based on the NPA / NV teacher information; an estimated NPA / NV identification step of identifying a region of the fundus image information estimated to correspond to the retinal nonperfusion region as an estimated NPA and identifying a region of the fundus image information estimated to correspond to the neovascularization as an estimated NV based on the probability of existence of the retinal nonperfusion region and the probability of existence of the neovascularization; An information processing method including: (Appendix 6) A fluorescent fundus angiography image acquisition means for acquiring fluorescent fundus angiography image information; an NPA / NV annotation acquisition means for acquiring, as NPA / NV annotation information, diagnostic notes by an ophthalmologist regarding at least one of a retinal nonperfusion area and neovascularization, which are added to the fluorescein fundus angiography image information; an NPA / NV teacher information generating means for generating NPA / NV teacher information serving as teacher information for calculating the probability of the presence of the retinal non-perfusion region and the probability of the presence of neovascularization based on the fluorescein angiography image information and the NPA / NV annotation information corresponding to the fluorescein angiography image information; a fundus image acquiring means for acquiring fundus image information; an NPA / NV existence probability calculation means for calculating the existence probability of the retinal non-perfusion region and the existence probability of the neovascularization in the fundus image information based on the NPA / NV teacher information; an estimated NPA / NV identifying means for identifying a region of the fundus image information estimated to correspond to the retinal nonperfusion region as an estimated NPA and a region of the fundus image information estimated to correspond to the neovascularization region as an estimated NV based on the probability of the presence of the retinal nonperfusion region and the probability of the presence of the neovascularization region; An information processing device comprising: [Explanation of symbols]

[0128] 1: Server 2: Ophthalmologist terminal 3: Inspection equipment 11:CPU 12:ROM 13:RAM 14: Bus 15: Input / output interface 16: Display section 17: Input section 18: Storage part 19: Communications Department 20: Drive 30: Removable Media 101: Image acquisition unit 102: Annotation acquisition unit 103: Teacher information generation section 104: Arithmetic section 105: Map generation unit 106: Estimated NPA / NV identification department 107: Estimated NPA / NV display control unit 200: Learning device 205: Communications Department 210: Storage section 211: Program 212: App 213: Image DB 214:Annotation DB 215: Learning Information 216: Learning Model 220:Operation unit 230: Information Processing Department 231: Image acquisition unit 232: Annotation acquisition unit 233: Learning information generation unit 234: Learning Department 240: Display section 250: Bus line 300: Diagnostic support device 305: Communications Department 310: Storage section 311: Program 312: App 320:Operation unit 330: Information Processing Department 331: Reception 332: Specific part 333: Creation Department 240: Display section 350: Bus Line 401: Image DB 402:Annotation DB 403:Teacher DB A: Estimated NPA B: Estimated NV C: Fluorescein fundus angiography image information D: Ophthalmologist E: NPA·NV existence probability map F: Fundus image information

Claims

[Claim 1] an identifying unit that identifies an area of ​​abnormal blood circulation in the fundus image by using a learning model that has learned the relationship between the fundus image and the area of ​​abnormal blood circulation in the fundus image, based on the fundus image being an image of the fundus and the area of ​​abnormal blood circulation identified based on the fluorescein angiography image of the fundus; an output unit that outputs a fundus image of a patient and information indicating a region of abnormal blood circulation in the fundus image of the patient identified by the identification unit using the learning model; A diagnostic support device having the above configuration.

Images