Diagnostic support device, learning device, diagnostic support method, learning method, and program

Abstract

To provide an image analysis system which performs screening of diabetic retinopathy from a fundus image.SOLUTION: A diagnosis support device has: a specification unit for specifying a blood circulation abnormal region in a fundus image, which is an image of the fundus, by using a learning model obtained by learning a relation between the fundus image and the blood circulation abnormal region in the fundus image, on the basis of the fundus image and the blood circulation abnormal region specified on the basis of a fluorescence contrast image of the fundus; and an output unit for outputting information indicating the blood circulation abnormal region, in a patient's fundus image, that has been specified by the specification unit by using the patient's fundus image and the learning model.SELECTED DRAWING: None

Description

Technical Field

[0001] The present invention relates to a diagnostic support device, a learning device, a diagnostic support method, a learning method, and a program. This application claims priority based on Japanese Patent Application No. 2018-7585 filed in Japan on January 19, 2018, and incorporates the content herein by reference.

Background Art

[0002] Diabetic retinopathy is an important disease that ranks second among the causes of blindness in Japan. However, since the onset of symptoms occurs only after the disease has progressed considerably, early detection and early treatment through medical check-ups and the like are important. In response to this problem, a fundus image analysis system that emphasizes capillary hemangioma, which is an early change in diabetic retinopathy (see Patent Document 1), and an image analysis system that performs screening for diabetic retinopathy from fundus images (see Patent Document 2) have been proposed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

[0004] One aspect of the present invention is a diagnostic support device having a specifying unit that specifies an abnormal blood circulation region in a fundus image using a learning model that has learned the relationship between a fundus image, which is an image of the fundus of the eye, and the abnormal blood circulation region specified based on the fluorescein fundus angiogram image of the fundus, and an output unit that outputs information indicating the abnormal blood circulation region in the fundus image of the patient specified by the specifying unit using the fundus image of the patient and the learning model.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Embodiments for Carrying Out the Invention

[0006] Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of an information processing system including a server 1 which is an embodiment of an information processing apparatus of the present invention.

[0007] The information processing system shown in FIG. 1 is configured to include a server 1, an ophthalmologist terminal 2, and an inspection device 3. The server 1, the ophthalmologist terminal 2, and the inspection device 3 are mutually connected via a network N such as the Internet.

[0008] The server 1 is a server that manages the information processing system shown in FIG. 1, and executes various processes such as NPA·NV presence probability map generation processing, associated finding presence probability map generation processing, and estimated NPA·NV identification processing. Note that the specific content of the processes executed by the server 1 will be described later with reference to FIG. 3. The "NPA·NV presence probability map generation processing" refers to a series of processes from generating NPA·NV teacher information to generating an NPA·NV presence probability map among the processes executed by the server 1. "NPA·NV teacher information" refers to the teacher information used when calculating the probability of the existence of non-perfused areas (NPA / non-perfusion area) (hereinafter referred to as "non-perfused area of the retina" or "NPA") and the probability of the existence of neovascularization (NV / neovascularization) (hereinafter referred to as "NV existence probability") in the fundus image information of a patient. Specifically, "NPA·NV teacher information" is generated based on fundus fluorescein angiography image information and NPA·NV annotation information attached to this information. The "non-perfused area of the retina" refers to the area of poor retinal circulation resulting from retinal vascular occlusion in ocular ischemic diseases.

[0009] The "NPA·NV existence probability map" is image information in which the NPA existence probability and the NV existence probability are separately displayed in the fundus image information. "Fundus image information" refers to image information based on a fundus image. "Fundus fluorescein angiography image information" refers to information based on a fundus fluorescein angiography image. "NPA·NV annotation information" refers to the diagnostic notes of ophthalmologist D regarding at least one of the non-perfused area of the retina (NPA) and neovascularization (NV) attached to the fundus fluorescein angiography image information. "Neovascularization" is the further progression of poor retinal circulation and ischemia in the non-perfused area of the retina. Since bleeding and retinal detachment through the formation of a proliferative membrane occur from neovascularization, ultimately leading to blindness, it is very important in medical treatment to identify areas of poor retinal circulation such as non-perfused areas of the retina and neovascularization. Regarding the specific process flow of the NPA·NV existence probability map generation process, it will be described later with reference to the flowchart in Figure 4.

[0010] The "associated finding existence probability map generation process" refers to a series of processes among the processes executed by server 1, from generating associated finding teacher information to generating an associated finding existence probability map. "Accompanying finding teacher information" refers to the teacher information used when calculating the probability of the existence of accompanying findings in the fundus image information of a patient. Specifically, "accompanying finding teacher information" refers to the teacher information generated based on fundus fluorescein angiography image information and fundus image information, and the accompanying finding annotation information attached to these image information. "Accompanying finding annotation information" refers to the diagnostic annotations that ophthalmologist D has incidentally made a judgment of "abnormal" on the fundus fluorescein angiography image information and fundus image information, excluding the diagnostic annotations related to non-perfused retinal areas (NPA) or neovascularization (NV). For example, information such as capillary hemangioma, fundus hemorrhage, hard white spots, soft white spots, venous abnormalities, intraretinal microvascular abnormalities, vitreous hemorrhage, proliferative membranes, and retinal detachment are all examples of "accompanying finding annotation information". "Accompanying finding probability map" refers to the image information (not shown in the figure) that discriminately displays the probability of the existence of accompanying findings in the fundus image information. Regarding the specific process flow of the accompanying finding probability map generation process, it will be described later with reference to the flowchart in Figure 4.

[0011] "Estimated NPA·NV identification process" refers to a series of processes among the processes executed by the server 1, based on the NPA existence probability, identifying the area that is estimated to correspond to the non-perfused retinal area (NPA) in the fundus image information as the estimated NPA, and based on the NV existence probability, identifying the area that is estimated to correspond to the neovascularization (NV) in the fundus image information as the estimated NV. Regarding the specific process flow of the estimated NPA·NV identification process, it will be described later with reference to the flowchart in Figure 4.

[0012] The ophthalmologist terminal 2 is an information processing device operated by ophthalmologist D, and is composed of, for example, a personal computer or the like. In the ophthalmologist terminal 2, the transmission of NPA·NV annotation information and accompanying finding annotation information to the server 1, the acquisition of information related to the estimated NPA·NV identified by the server 1, etc. are performed. The various information acquired by the ophthalmologist terminal 2 is output from the ophthalmologist terminal 2 and used for the examination by ophthalmologist D.

[0013] The inspection device 3 is composed of various devices used in the eye examination of patients. The inspection device 3 transmits the fundus image information obtained by imaging in the fundus examination and the fundus fluorescein angiography image information obtained by imaging in the fundus fluorescein angiography examination to the server 1 respectively.

[0014] Figure 2 is a block diagram showing the hardware configuration of the server 1 among the information processing systems of Figure 1.

[0015] The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20.

[0016] The CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 18 to the RAM 13. In the RAM 13, data and the like necessary for the CPU 11 to execute various processes are also appropriately stored.

[0017] The CPU 11, ROM 12, and RAM 13 are interconnected via the bus 14. The input / output interface 15 is also connected to this bus 14. The output unit 16, input unit 17, storage unit 18, communication unit 19, and drive 20 are connected to the input / output interface 15.

[0018] The output unit 16 is composed of various liquid crystal displays and the like, and outputs various information. The input unit 17 is composed of various hardware leads and the like, and inputs various information. The storage unit 18 is composed of a DRAM (Dynamic Random Access Memory) and the like, and stores various data. The communication unit 19 controls communication with other devices via a network N including the Internet.

[0019] The drive 20 is provided as needed. A removable medium 30, which is composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is appropriately attached to the drive 20. The program read from the removable medium 30 by the drive 20 is installed in the storage unit 18 as needed. The removable medium 30 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

[0020] Next, the functional configuration of the server 1 having such a hardware configuration will be described with reference to FIG. 3. FIG. 3 is a functional block diagram showing an example of a functional configuration for realizing the NPA·NV presence probability map generation process, the associated finding presence probability map generation process, the estimated NPA·NV identification process, and the estimated NPA·NV display process among the functional configurations of the server 1 in FIG. 2 of the information processing system in FIG. 1.

[0021] As shown in FIG. 3, in the CPU 11 (FIG. 2) of the server 1, when the NPA·NV presence probability map generation process is executed, the image acquisition unit 101, the annotation acquisition unit 102, the teacher information generation unit 103, and the calculation unit 104 function. When the associated finding presence probability map generation process is executed, the image acquisition unit 101, the annotation acquisition unit 102, the teacher information generation unit 103, and the calculation unit 104 function. When the estimated NPA·NV identification process is executed, the estimated NPA·NV identification unit 106 functions. When the estimated NPA·NV display process is executed, the estimated NPA·NV display control unit 107 functions. An image DB 401, an annotation DB 402, and a teacher DB 403 are provided in one area of the storage unit 18 (FIG. 2). Note that the storage unit 18 may be arranged in the ophthalmologist terminal 2 instead of the server 1.

[0022] The image acquisition unit 101 acquires the fundus fluorescein angiography image information of the patient and the fundus image information of the patient. The fundus fluorescein angiography image information and the fundus image information acquired by the image acquisition unit 101 are respectively stored and managed in the image DB 401. Specifically, in the fundus fluorescein angiography examination of the patient, when the fundus fluorescein angiography image of the patient is captured by the examination device 3, the fundus fluorescein angiography image information based on the fundus fluorescein angiography image is transmitted to the server 1. Also, in the fundus examination of the patient, when the fundus image of the patient is captured by the examination device 3, the fundus image information based on the fundus image is transmitted to the server 1. The image acquisition unit 101 of the server 1 acquires the fundus fluorescein angiography image information and the fundus image information transmitted from the examination device 3 to the server 1, and stores these image information in the image DB 401. Thereby, the server 1 can accurately manage the fundus fluorescein angiography image information and the fundus image information of the patient without omission.

[0023] The annotation acquisition unit 102 acquires the diagnostic annotation of the ophthalmologist D regarding at least one of the non-perfused retinal area (NPA) and the neovascularization (NV) attached to the fundus fluorescein angiography image information of the patient as NPA·NV annotation information. Specifically, in the fundus fluorescein angiography examination, when the diagnostic annotation of the ophthalmologist D regarding the non-perfused retinal area (NPA) and the neovascularization (NV) is attached to the fundus fluorescein angiography image information, the ophthalmologist terminal 2 transmits the diagnostic annotation to the server 1 as NPA·NV annotation information based on the operation of the ophthalmologist D. The annotation acquisition unit 102 of the server 1 acquires the NPA·NV annotation information transmitted from the ophthalmologist terminal 2, and stores this information in the annotation DB 402. Note that the fundus fluorescein angiography image information and the NPA·NV annotation information attached to this image information are managed in association with each other. Thereby, the server 1 can accurately manage the diagnostic annotation of the ophthalmologist D regarding at least one of the non-perfused retinal area (NPA) and the neovascularization (NV) attached to the fundus fluorescein angiography image information as NPA·NV annotation information without omission.

[0024] In addition, the annotation acquisition unit 102 acquires the diagnostic notes of ophthalmologist D regarding the incidental findings attached to the fundus fluorescein angiography image and the fundus image as incidental finding annotation information. Specifically, for the fundus fluorescein angiography image information and the fundus image information, when the diagnostic notes of ophthalmologist D regarding the incidental findings are attached, the ophthalmologist terminal 2 transmits the diagnostic notes to the server 1 as incidental finding annotation information based on the operation of ophthalmologist D. The annotation acquisition unit 102 of the server 1 acquires the incidental finding annotation information transmitted from the ophthalmologist terminal 2 and stores this information in the annotation DB 402. Note that the fundus fluorescein angiography image information and the fundus image information, and the incidental finding annotation information attached to these image information are managed in association with each other. Thereby, the server 1 can manage without omission the diagnostic notes of ophthalmologist D regarding the incidental findings attached to the fundus fluorescein angiography image information and the fundus image information as incidental finding annotation information.

[0025] The teacher information generation unit 103 generates NPA·NV teacher information, which is teacher information for calculating the NPA presence probability and the NV presence probability, based on the fundus fluorescein angiography image information and the NPA·NV annotation information corresponding to this information. That is, a plurality of pieces of fundus fluorescein angiography image information obtained from a plurality of patients are stored in the image DB 401, and NPA·NV annotation information is stored in the annotation DB 402. The teacher information generation unit 103 generates NPA·NV teacher information, which is teacher information for calculating the presence probability of NPA·NV in the fundus image information, based on the information stored in these databases. Thereby, the server 1 can generate and store teacher information for calculating the NPA presence probability and the NV presence probability in the fundus image information of the patient. In addition, the teacher information generation unit 103 generates incidental finding teacher information based on the fundus fluorescein angiography image information and the fundus image information, and the incidental finding annotation information corresponding to these image information. That is, the image DB 401 stores fundus image information and fluorescein fundus angiography image information obtained from a plurality of patients, and the annotation DB 402 stores accompanying finding annotation information. Based on the information stored in these databases, the teacher information generation unit 103 generates accompanying finding teacher information, which serves as teacher information when calculating the probability of the presence of accompanying findings in the fundus image information. As a result, the server 1 can store teacher information for calculating the probability of the presence of accompanying findings in the fundus image information of the patient.

[0026] The calculation unit 104 calculates the NPA presence probability and the NV presence probability based on at least the NPA·NV teacher information. Also, when an accompanying finding presence probability map described later is generated, the calculation unit 104 calculates the NPA presence probability and the NV presence probability based on the accompanying finding presence probability map and the NPA·NV teacher information. Note that the specific method for calculating the NPA presence probability and the NV presence probability is not particularly limited. For example, features common to fundus images having NPA·NV are extracted from the NPA·NV teacher information, and the degree of coincidence with these features is normalized to calculate the NPA presence probability and the NV presence probability based on whether the fundus image information of the patient has these features. The NPA presence probability and the NV presence probability may be calculated using deep learning techniques. As a result, it becomes possible to set criteria for identifying the region estimated to correspond to the non-perfused area (NPA) of the retina and the region estimated to correspond to the neovascularization (NV) in the fundus image information.

[0027] In addition, the calculation unit 104 calculates the probability of the presence of accompanying findings in the fundus image information based on the accompanying finding teacher information. The specific method for calculating the probability of the presence of accompanying findings is not particularly limited. For example, features common to fundus images having accompanying findings are extracted from the accompanying finding teacher information, and the degree of coincidence with these features is normalized to calculate the probability of the presence of accompanying findings based on whether the fundus image information of the patient has these features. The probability of the presence of accompanying findings may be calculated using deep learning techniques. This makes it possible to set criteria for identifying a region in the fundus image information that is estimated to correspond to a non-perfused area (NPA) of the retina and a region that is estimated to correspond to neovascularization (NV).

[0028] The map generation unit 105 generates an NPA·NV presence probability map as image information in which the NPA presence probability and the NV presence probability are discriminatively displayed in the fundus image information. Specifically, it generates image information such as the NPA·NV presence probability map E illustrated in FIG. 8. This makes it possible to generate information serving as a basis for estimating the presence of a non-perfused area (NPA) of the retina and neovascularization (NV) in the fundus image information. Note that the method of discriminatively displaying the NPA presence probability and the NV presence probability in the fundus image information is not particularly limited. For example, the NPA presence probability and the NV presence probability may be discriminatively displayed by a difference in color, or may be discriminatively displayed by a difference in color shade. Further, the map generation unit 105 generates an incidental finding presence probability map (not shown) as image information in which the incidental finding presence probability is discriminatively displayed in the fundus image information. This makes it possible to generate information serving as a basis for estimating the presence of a non-perfused area (NPA) of the retina and neovascularization (NV) in the fundus image information. Note that the method of discriminatively displaying the incidental finding presence probability in the fundus image information is not particularly limited. For example, the incidental finding presence probability may be discriminatively displayed by a difference in color, or may be discriminatively displayed by a difference in color shade.

[0029] The estimated NPA·NV specifying unit 106 specifies, based on the NPA existence probability and the NV existence probability, an area estimated to correspond to the non-perfused area (NPA) of the fundus image information as the estimated NPA, and specifies an area estimated to correspond to the neovascularization (NV) as the estimated NV. Specifically, among the fundus image information of the patient, an area where the NPA existence probability and the NV existence probability exceed a predetermined threshold is specified as an area estimated to correspond to the non-perfused area (NPA) of the fundus (estimated NPA), or an area estimated to correspond to the neovascularization (NV) (estimated NV). Note that the threshold can be arbitrarily changed at the discretion of the ophthalmologist D. Thereby, among the image information based on the fundus image, the area (estimated NPA) where the existence of the non-perfused area (NPA) of the fundus is estimated and the area (estimated NV) where the existence of the neovascularization (NV) is estimated can be specified early and easily.

[0030] The estimated NPA·NV display control unit 107 executes control to display the estimated NPA area and the estimated NV area on the fundus image information. Thereby, on the image information based on the fundus image, the area (estimated NPA) where the existence of the non-perfused area of the fundus is estimated and the area (estimated NV) where the existence of the neovascularization (NV) is estimated can be superimposed and displayed.

[0031] Next, with reference to FIG. 4, a series of processing flows executed by the server 1 having the functional configuration of FIG. 3 will be described. FIG. 4 is a flowchart for explaining a series of processing flows executed by the server 1 of FIG. 3.

[0032] As shown in FIG. 4, in the server 1, a series of the following processing is executed. In step S1, the image acquisition unit 101 determines whether or not the fundus fluorescein angiography image information has been transmitted from the inspection device 3. When the fundus fluorescence angiography image information is transmitted, it is determined to be YES in step S1, and the process proceeds to step S2. On the other hand, when the fundus fluorescence angiography image information has not been transmitted, it is determined to be NO in step S1, and the process is returned to step S1. That is, until the fundus fluorescence angiography image information is transmitted, the determination process of step S1 is repeated. Thereafter, when the fundus fluorescence angiography image information is transmitted, it is determined to be YES in step S1, and the process proceeds to step S2. In step S2, the image acquisition unit 101 acquires the transmitted fundus fluorescence angiography image information.

[0033] In step S3, the annotation acquisition unit 102 determines whether NPA·NV annotation information has been transmitted from the ophthalmologist terminal 2. When the NPA·NV annotation information is transmitted, it is determined to be YES in step S3, and the process proceeds to step S4. On the other hand, when the NPA·NV annotation information has not been transmitted, it is determined to be NO in step S3, and the process is returned to step S3. That is, until the NPA·NV annotation information is transmitted, the determination process of step S3 is repeated. Thereafter, when the NPA·NV annotation information is transmitted, it is determined to be YES in step S3, and the process proceeds to step S4. In step S4, the annotation acquisition unit 102 acquires the transmitted NPA·NV annotation information.

[0034] In step S5, the teacher information generation unit 103 generates NPA·NV teacher information based on the fundus fluorescence angiography image information and the NPA·NV annotation information corresponding to the fundus fluorescence angiography image information. In step S6, the image acquisition unit 101 determines whether fundus image information has been transmitted from the inspection device 3. When fundus image information is transmitted, it is determined to be YES in step S6, and the process proceeds to step S7. On the other hand, when the fundus image information has not been transmitted, it is determined to be NO in step S6, and the process is returned to step S6. That is, until the fundus image information is transmitted, the determination process of step S6 is repeated. After that, when the fundus image information is transmitted, it is determined to be YES in step S6, and the process proceeds to step S7. In step S7, the image acquisition unit 101 acquires the transmitted fundus image information.

[0035] In step S8, the annotation acquisition unit 102 determines whether or not the accompanying finding annotation information has been transmitted from the ophthalmologist terminal 2. When the accompanying finding annotation information is transmitted, it is determined to be YES in step S8, and the process proceeds to step S9. On the other hand, when the accompanying finding annotation information has not been transmitted, it is determined to be NO in step S8, and the process skips steps S9 to S11 and proceeds to step S12. In step S9, the annotation acquisition unit 102 acquires the accompanying finding annotation information. In step S10, the teacher information generation unit 103 generates accompanying finding teacher information based on the fundus fluorescein angiography image information and the NPA·NV annotation information corresponding to the fundus fluorescein angiography image information. In step S11, the calculation unit 104 calculates the probability of the existence of the accompanying finding in the fundus image information based on the accompanying finding teacher information. In step S12, the map generation unit 105 generates an accompanying finding probability map as image information in which the accompanying finding probability is discriminatively displayed in the fundus image information.

[0036] In step S13, the arithmetic unit 104 calculates the NPA existence probability and the NV existence probability based on at least the NPA·NV teacher information. Also, when the accompanying finding existence probability map has been generated, the arithmetic unit 104 calculates the NPA existence probability and the NV existence probability based on the NPA·NV teacher information and the accompanying finding existence probability map. In step S14, the map generation unit 105 generates an NPA·NV existence probability map as image information in which the NPA existence probability and the NV existence probability are discriminatively displayed in the fundus image information. In step S15, the estimated NPA·NV identification unit 106 identifies, as the estimated NPA, the region in the fundus image information that is estimated to correspond to the non-perfused retinal area (NPA), and identifies, as the estimated NV, the region that is estimated to correspond to the neovascularization (NV) based on the NPA existence probability and the NV existence probability. In step S16, the estimated NPA·NV display control unit 107 executes control to display the estimated NPA and the NV region in the fundus image information.

[0037] In step S17, the server 1 determines whether there is an instruction to end the process. If there is no instruction to end the process, it is determined NO in step S17, and the process returns to step S1. On the other hand, if there is an instruction to end the process, it is determined YES in step S17, and the process ends. By the server 1 executing the series of processes as described above, the estimated NPA region and the estimated NV region are displayed in the fundus image information.

[0038] Next, with reference to FIG. 5, the flow of various information used in the various processes executed by the server 1 will be described. FIG. 5 is a diagram showing the flow of various information in the process executed by the server 1.

[0039] As shown in FIG. 5, in the fundus fluorescein angiography examination, when the fundus fluorescein angiography image of the patient is captured, the fundus fluorescein angiography image information based on this fundus fluorescein angiography image is acquired by the server 1. The diagnosis annotation of the ophthalmologist D regarding at least one of the non-perfused retinal area (NPA) and the neovascularization (NV) is attached to this fundus fluorescein angiography image information. This diagnosis annotation, as NPA·NV annotation information, together with the fundus fluorescein angiography image information, constitutes the NPA·NV teacher information. In the fundus examination, when the fundus image of the patient is captured, the fundus image information based on this fundus image is acquired by the server 1. The fundus image information and the NPA·NV teacher information are used for the arithmetic processing by the NPA·NV annotation program of the arithmetic unit 104. As a result of the arithmetic processing by the NPA·NV annotation program, an NPA·NV existence probability map is generated. Based on this NPA·NV existence probability map, the estimated NPA and the estimated NV in the fundus image information are specified.

[0040] There may be a case where the diagnosis annotation of the ophthalmologist D regarding the accompanying findings is attached to the fundus fluorescein angiography image information and the fundus image information. In this case, the diagnosis annotation, as the accompanying findings annotation information, together with the fundus fluorescein angiography image information and the fundus image information, constitutes the accompanying findings teacher information. The fundus image information and the NPA·NV teacher information are used for the arithmetic processing by the accompanying findings determination program of the arithmetic unit 104. As a result of the arithmetic processing by the accompanying findings determination program, an accompanying findings existence probability map is generated. Thus, when the accompanying findings existence probability map is generated, based on the accompanying findings existence probability map and the NPA·NV teacher information, the estimated NPA and the estimated NV in the fundus image information are specified.

[0041] FIG. 6 is a diagram showing an example of the fundus image information acquired in the process executed by the server 1. FIG. 7 is a diagram showing an example of the fundus fluorescein angiography image information acquired in the process executed by the server 1.

[0042] In the fundus examination of a patient, imaging of the patient's fundus is performed by the examination device 3, and fundus image information as shown in FIG. 6 is obtained. The ophthalmologist D conducts a diagnosis while referring to the fundus image information as shown in FIG. 6. However, as shown in FIG. 6, the fundus image information only shows findings resulting from perfusion abnormalities such as bleeding and white patches, and it is difficult to read the circulatory dynamics and the sites of circulatory abnormalities. Therefore, by performing a fluorescein fundus angiography examination, fluorescein fundus angiography image information as shown in FIG. 7 is obtained. As shown in FIG. 7, the fluorescein fundus angiography image information can clearly discriminate the retinal circulatory dynamics. However, the fluorescein fundus angiography examination is a test that places a heavy burden on both patients and medical staff due to the risks and invasiveness of the examination, geographical restrictions limited to large hospitals, etc., and there are issues such as hesitation in photographing cases with good vision or patients with reduced physical function, repeated photographing, etc., resulting in many cases where the onset and disease condition are not grasped in time and treatment is delayed. Therefore, the server 1 uses the fluorescein fundus angiography image information obtained from a plurality of accumulated patients and the NPA·NV teacher information generated based on the NPA·NV annotation information attached to each of these information to perform NPA·NV presence probability map generation processing. As a result, it becomes possible to easily identify circulatory abnormality findings from fundus images that can be easily photographed at nearby clinics or health checkups without performing the burdensome fluorescein fundus angiography examination.

[0043] FIG. 8 is a diagram showing an example of the NPA·NV presence probability map output in the NPA·NV presence probability map generation processing executed by the server 1.

[0044] The NPA·NV presence probability map E shown in FIG. 8 is image information in which the NPA presence probability and the NV presence probability are separately displayed in the fundus image information. The NPA·NV presence probability map E can distinguish and display the NPA presence probability and the NV presence probability by differences in color or color shading. For example, it is possible to perform a discriminative display such that areas shown in warm colors have a high NPA presence probability and a high NV presence probability, and areas shown in cool colors have a low NPA presence probability and a low NV presence probability. Also, even in areas shown in cool colors with a low NPA presence probability and a low NV presence probability, it is possible to perform a discriminative display such that areas with darker colors have an even lower NPA presence probability and an even lower NV presence probability than areas with lighter colors. Thereby, it is possible to generate information serving as a basis for estimating the presence of non-perfused areas (NPA) and neovascularization (NV) in the fundus image information.

[0045] FIG. 9 is a diagram showing an example of an estimated NPA and an estimated NV specified in the estimated NPA·NV specifying process executed by the server 1.

[0046] Based on the content of the NPA·NV presence probability map, the server 1 specifies, as an estimated NPA, an area in the fundus image information that is estimated to correspond to a non-perfused area (NPA) of the retina, and specifies, as an estimated NV, an area that is estimated to correspond to neovascularization (NV). For example, as shown in FIG. 9, the area A indicated by the broken line can be set as the estimated NPA, and the area B can be set as the estimated NV. The estimated NPA and the estimated NV can be superimposed and displayed on the fundus image information. Thereby, without performing a fluorescein fundus angiography examination that requires a special fundus camera or diagnostic device, it is possible to early and easily identify an area where the presence of non-perfused retina is estimated and an area where the presence of neovascularization is estimated from a fundus image obtained by imaging the fundus of a patient.

[0047] As described above, one embodiment of the present invention has been described. However, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the range that can achieve the object of the present invention are included in the present invention.

[0048] For example, in the above-described embodiment, the image acquisition unit 101 is configured to acquire various image information when it is transmitted from the inspection device 3. However, when imaging is performed by the inspection device 3, the image acquisition unit 101 may be configured to spontaneously go to acquire various image information. Similarly, the annotation acquisition unit 102 is configured to acquire various annotation information when it is transmitted from the ophthalmologist terminal 2. However, when various annotation information is input to the ophthalmologist terminal 2, the annotation acquisition unit 102 may be configured to spontaneously go to acquire various annotation information.

[0049] Also, each hardware configuration shown in FIG. 2 is merely an example for achieving the object of the present invention and is not particularly limited.

[0050] Also, the functional block diagram shown in FIG. 3 is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-described series of processes as a whole, and the functional blocks used to realize this function are not particularly limited to the example of FIG. 3.

[0051] Also, the location where the functional block exists is not limited to the location shown in FIG. 3 and may be arbitrary. For example, at least a part of the functional block of the server 1 may be provided in the ophthalmologist terminal 2 or the inspection device 3. And one functional block may be constituted by hardware alone or in combination with software alone.

[0052] When the processing of each functional block is executed by software, the program constituting the software is installed in a computer or the like from a network or a recording medium. The computer may be a computer incorporated in dedicated hardware. Also, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or personal computer in addition to a server.

[0053] A recording medium containing such a program is not only composed of a removable medium distributed separately from the apparatus main body to provide the program to each user, but also composed of a recording medium or the like provided to each user in a state pre-installed in the apparatus main body.

[0054] Note that in this specification, the steps of describing the program recorded on the recording medium include not only the processes performed in chronological order according to the order, but also the processes that are not necessarily processed in chronological order and are executed in parallel or individually. For example, in step S6 of FIG. 4, the image acquisition unit 101 determines whether fundus image information has been transmitted from the inspection device 3, and acquires the fundus image information in step S7. However, for the fundus image information, if the accompanying finding presence probability map generation process is executed, it is sufficient if it is appropriately managed at the time when the accompanying finding teacher information is generated. Also, if the accompanying finding presence probability map generation process is not executed, it is sufficient if it is appropriately managed at the time when the NPA presence probability and the NV presence probability are calculated. Therefore, for the fundus image information, if the accompanying finding teacher information is generated, it may be stored and managed in the image DB 401 from any time point before the accompanying finding teacher information is generated in step S10. Also, if the accompanying finding teacher information is not generated, it may be stored and managed in the image DB 401 from any time point before the NPA presence probability and the NV presence probability are calculated in step S13.

[0055] In summary, the program to which the present invention is applied may have the following configuration and can take various embodiments. That is, the program to which the present invention is applied is a fluorescence fundus angiography image acquisition step (for example, step S2 in FIG. 4) for acquiring fluorescence fundus angiography image information (for example, fluorescence fundus angiography image information C in FIG. 7) by a computer that controls an information processing apparatus, and ​An NPA·NV annotation acquisition step (for example, step S4 in FIG. 4) of acquiring, as NPA·NV annotation information, a diagnosis note of an ophthalmologist regarding at least one of a non-perfused area (NPA) and neovascularization (NV) attached to the fundus fluorescein angiography image information, An NPA·NV teacher information generation step (for example, step S5 in FIG. 4) of generating NPA·NV teacher information, which is teacher information for calculating the probability of existence of the non-perfused area (NPA) and the probability of existence of neovascularization (NV) based on the fundus fluorescein angiography image information and the NPA·NV annotation information corresponding to the fundus fluorescein angiography image information, A fundus image acquisition step (for example, step S7 in FIG. 4) of acquiring fundus image information (for example, fundus image information F in FIG. 6), An NPA·NV existence probability calculation step (for example, step S13 in FIG. 4) of calculating the probability of existence of the non-perfused area (NPA) and the probability of existence of neovascularization (NV) in the fundus image information based on the NPA·NV teacher information, An estimated NPA·NV identification step (for example, step S15 in FIG. 4) of identifying, as an estimated NPA, an area estimated to correspond to the non-perfused area (NPA) in the fundus image information and identifying, as an estimated NV, an area estimated to correspond to the neovascularization (NV) based on the probability of existence of the non-perfused area and the probability of existence of the neovascularization, including. Thereby, without performing a fundus fluorescein angiography examination that requires a special fundus camera or diagnostic device, it is possible to easily and early identify an area where the existence of non-perfused retina is estimated from a fundus image obtained by imaging the fundus of a patient.

[0056] Furthermore, it is possible to execute a control process further including an NPA·NV existence probability map generation step (for example, step S14 in FIG. 4) of generating an NPA·NV existence probability map (for example, NPA·NV existence probability map E in FIG. 8) in which the NPA existence probability and the NV existence probability are separately displayed in the fundus image information. As a result, it is possible to generate information serving as a basis for estimating the presence of non-perfused areas (NPA) and neovascularization (NV) in fundus image information.

[0057] Further, an accompanying annotation acquisition step (for example, step S9 in FIG. 4) of acquiring, as accompanying finding annotation information, the ophthalmologist's diagnostic notes regarding accompanying findings attached to the fundus fluorescein angiography image information and the fundus image information, an accompanying finding teacher information generation step (for example, step S10 in FIG. 4) of generating accompanying finding teacher information serving as teacher information for calculating the presence probability of the accompanying findings in the fundus image information based on the fundus fluorescein angiography image information, the fundus image information, and the accompanying finding annotation information corresponding to the fundus fluorescein angiography image information and the fundus image information, an accompanying finding presence probability calculation step (for example, step S11 in FIG. 4) of calculating the presence probability of the accompanying findings in the fundus image information based on the accompanying finding teacher information, further includes In the NPA·NV presence probability calculation step, further control processing for calculating the presence probability of the non-perfused area (NPA) and the presence probability of neovascularization (NV) in the fundus image information based on the presence probability of the accompanying findings and the NPA·NV teacher information can be executed.

[0058] Further, an accompanying finding presence probability map generation step (for example, step S12 in FIG. 4) of generating an accompanying finding presence probability map in which the presence probability of the accompanying findings is discriminatively displayed in the fundus image information can be included.

[0059] Hereinafter, a modification of an embodiment of the present invention will be described with reference to the drawings. (Modification) FIG. 10 is a configuration diagram of an information processing system according to a modification of an embodiment of the present invention. The information processing system according to a modification of an embodiment of the present invention includes an ophthalmologist terminal 2, an examination device 3, a learning device 200, and a diagnostic support device 300. These devices are connected to each other via a network N.

[0060] Based on a fundus image, which is an image of the fundus of the eye, and a blood circulation abnormal region identified based on a fluorescein fundus angiography image of the fundus of the eye, the learning device 200 generates, through learning, a learning model representing the relationship between the fundus image and the blood circulation abnormal region in the fundus image. Here, the blood circulation abnormal region is a region where blood circulation is abnormal due to a blood disorder on the retina that occurs in an ocular ischemic disease such as diabetic retinopathy. The learning device 200 acquires information indicating a fundus image of a patient and information indicating a fluorescein fundus angiography image of the patient, and associates and stores the acquired information indicating the fundus image and the information indicating the fluorescein fundus angiography image. Specifically, in a fundus examination of a patient, the examination device 3 captures a fundus image of the patient, creates fundus image notification information including the patient ID and information indicating the captured fundus image, and sends the created fundus image notification information to the learning device 200. Also, in a fluorescein fundus angiography examination of the patient, the examination device 3 captures a fluorescein fundus angiography image of the patient, creates fluorescein fundus angiography image notification information including the patient ID and information indicating the captured fluorescein fundus angiography image, and sends the created fluorescein fundus angiography image notification information to the learning device 200.

[0061] The learning device 200 acquires the patient ID included in the fundus image notification information sent from the examination device 3 to the learning device 200, the information indicating the fundus image, the patient ID included in the fluorescein fundus angiography image notification information, and the information indicating the fluorescein fundus angiography image, and associates and stores the acquired patient ID, the information indicating the fundus image, and the information indicating the fluorescein fundus angiography image.

[0062] The learning device 200 acquires the diagnosis notes of ophthalmologist D regarding either or both of the non-perfused area (NPA) and neovascularization (NV) attached to the fundus fluorescein angiography image of the patient as NPA·NV annotation information. Specifically, in the fundus fluorescein angiography examination, when the diagnosis notes of ophthalmologist D regarding the non-perfused area (NPA) and neovascularization (NV) are attached to the fundus fluorescein angiography image, the ophthalmologist terminal 2 creates NPA·NV annotation notification information including the patient ID and the diagnosis notes based on the operation of ophthalmologist D, and designates the learning device 200 as the destination, and transmits the created NPA·NV annotation notification information to the learning device 200. The learning device 200 receives the NPA·NV annotation notification information transmitted by the ophthalmologist terminal 2, and stores the NPA·NV annotation information included in the received NPA·NV annotation notification information. Note that the information indicating the fundus fluorescein angiography image and the NPA·NV annotation information attached to this fundus fluorescein angiography image are stored in association with each other.

[0063] The learning device 200 acquires the diagnosis notes of ophthalmologist D regarding the incidental findings attached to the fundus image and the fundus fluorescein angiography image as incidental finding annotation information. Specifically, when the diagnosis notes of ophthalmologist D regarding the incidental findings are attached to the information indicating the fundus fluorescein angiography image and the information indicating the fundus image, the ophthalmologist terminal 2 creates incidental finding annotation notification information including the patient ID and the diagnosis notes based on the operation of ophthalmologist D, and designates the learning device 200 as the destination, and transmits the created incidental finding annotation notification information to the learning device 200. The learning device 200 receives the incidental finding annotation notification information transmitted by the ophthalmologist terminal 2, acquires the patient ID and the incidental finding annotation information included in the received incidental finding annotation notification information, and stores the acquired patient ID and the incidental finding annotation information. Note that the information indicating the fundus image and the information indicating the fundus fluorescein angiography image, and the incidental finding annotation information are stored in association with each other.

[0064] The learning device 200 acquires information indicating a fundus image, information indicating a fluorescein fundus angiography image associated with the information indicating the fundus image, NPA·NV annotation information, and incidental finding annotation information, and identifies an abnormal blood circulation region based on the acquired information indicating the fluorescein fundus angiography image, NPA·NV annotation information, and incidental finding annotation information. The learning device 200 generates NPA·NV learning information associating the information indicating the fundus image with the abnormal blood circulation region specified based on the fluorescein fundus angiography image corresponding to the fundus image, and stores the generated NPA·NV learning information. The learning device 200 uses the information indicating the fundus image included in the NPA·NV learning information as input information, and uses the abnormal blood circulation region specified based on the fluorescein fundus angiography image corresponding to the fundus image as teacher information to generate a learning model representing the relationship between the fundus image and the abnormal blood circulation region in the fundus image by learning. The specific method for generating the learning model is not particularly limited. For example, common features shared by fundus images having an abnormal blood circulation region may be extracted from the NPA·NV learning information, and the relationship between the extracted common features and the abnormal blood circulation region in the fundus image may be derived. Techniques such as neural networks and deep learning may be used to derive the relationship between the fundus image and the abnormal blood circulation region in the fundus image. The learning device 200 stores the generated learning model, creates learning model notification information including the generated learning model and having the diagnostic support device 300 as the destination, and transmits the created learning model notification information to the diagnostic support device 300.

[0065] The diagnostic support device 300 receives the learning model transmitted by the learning device 200 and stores the received learning model.

[0066] The ophthalmologist terminal 2 includes a patient ID and information indicating the patient's fundus image, creates patient information having the diagnostic support device 300 as the destination, and transmits the created patient information to the diagnostic support device 300.

[0067] The diagnostic support device 300 receives the patient information transmitted by the ophthalmologist terminal 2, and acquires the patient ID included in the received patient information and the information indicating the fundus image of the patient. The diagnostic support device 300 uses the stored learning model to identify the blood circulation abnormal area in the fundus image based on the information indicating the acquired fundus image. The diagnostic support device 300 creates a diagnostic result including the fundus image of the patient, the information indicating the blood circulation abnormal area in the fundus image identified using the learning model, and the patient ID, with the ophthalmologist terminal 2 as the destination, and transmits the created diagnostic result to the ophthalmologist terminal 2. Hereinafter, the learning device 200 and the diagnostic support device 300 included in the information processing system will be described.

[0068] (Learning device 200) FIG. 11 is a block diagram showing an example of a learning device according to a modified example of an embodiment of the present invention. The learning device 200 includes a communication unit 205, a storage unit 210, an operation unit 220, an information processing unit 230, a display unit 240, and bus lines 250 such as an address bus and a data bus for electrically connecting each component as shown in FIG. 11.

[0069] The communication unit 205 is realized by a communication module. The communication unit 205 communicates with external communication devices such as the ophthalmologist terminal 2, the inspection device 3, and the diagnostic support device 300 via the network N. Specifically, the communication unit 205 receives the fundus image notification information transmitted by the inspection device 3, and outputs the received fundus image notification information to the information processing unit 230. The communication unit 205 receives the fluorescein fundus angiography image notification information transmitted by the inspection device 3, and outputs the received fluorescein fundus angiography image notification information to the information processing unit 230. The communication unit 205 receives the NPA·NV annotation notification information transmitted by the ophthalmologist terminal 2, and outputs the received NPA·NV annotation notification information to the information processing unit 230. The communication unit 205 receives the incidental finding annotation notification information transmitted by the ophthalmologist terminal 2, and outputs the received incidental finding annotation notification information to the information processing unit 230. The communication unit 205 acquires the learning model notification information output by the information processing unit 230, and transmits the acquired learning model notification information to the diagnostic support device 300.

[0070] The memory unit 210 is implemented by, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), a flash memory, or a hybrid memory device in which a plurality of these are combined. The memory unit 210 stores a program 211 executed by the information processing unit 230, an app 212, an image DB 213, an annotation DB 214, learning information 215, and a learning model 216.

[0071] The program 211 is, for example, an operating system, which is located between the user or application program and the hardware, provides a standard interface to the user or application program, and at the same time efficiently manages each resource such as the hardware.

[0072] The app 212 causes the learning device 200 to receive the fundus image notification information transmitted by the inspection device 3, and associates and stores the patient ID included in the received fundus image notification information with the information indicating the fundus image. The app 212 causes the learning device 200 to receive the fluorescein fundus angiography image notification information transmitted by the inspection device 3, and associates and stores the patient ID included in the received fluorescein fundus angiography image notification information with the information indicating the fluorescein fundus angiography image. The app 212 causes the learning device 200 to receive the NPA·NV annotation notification information transmitted by the ophthalmologist terminal 2, and associates and stores the patient ID included in the received NPA·NV annotation notification information with the NPA·NV annotation information.

[0073] The application 212 causes the learning device 200 to receive the accompanying finding annotation notification information transmitted by the ophthalmologist terminal 2, and associates and stores the patient ID included in the received accompanying finding annotation notification information with the accompanying finding annotation information. The application 212 causes the learning device 200 to acquire the information indicating the fundus image associated with the patient ID, the information indicating the fluorescein fundus angiography image, the NPA·NV annotation information, and the accompanying finding annotation information. The application 212 causes the learning device 200 to identify the blood circulation abnormal region based on the information indicating the acquired fluorescein fundus angiography image, the NPA·NV annotation information, and the accompanying finding annotation information. The application 212 causes the learning device 200 to generate NPA·NV learning information in which the information indicating the fundus image is associated with the blood circulation abnormal region identified based on the fluorescein fundus angiography image corresponding to the fundus image, and stores the generated NPA·NV learning information. The application 212 uses the information indicating the fundus image included in the NPA·NV learning information as input information, and uses the blood circulation abnormal region identified based on the fluorescein fundus angiography image corresponding to the fundus image as teacher information, and causes the learning device 200 to generate a learning model representing the relationship between the fundus image and the blood circulation abnormal region in the fundus image by learning. The application 212 stores the generated learning model in the learning device 200 and transmits it to the diagnostic support device 300.

[0074] The image database 213 stores by associating the patient ID, the information indicating the fundus image, and the information indicating the fluorescein fundus angiography image.

[0075] The annotation database 214 stores by associating the patient ID, the NPA·NV annotation information, and the accompanying finding annotation information.

[0076] The learning information 215 stores the NPA·NV learning information in which the information indicating the fundus image is associated with the blood circulation abnormal region identified based on the fluorescein fundus angiography image corresponding to the fundus image.

[0077] The learning model 216 uses the information indicating the fundus image included in the NPA·NV learning information as input information, and uses the blood circulation abnormal region specified based on the fundus fluorescein angiography image corresponding to the fundus image as teacher information, and stores a learning model representing the relationship between the fundus image and the blood circulation abnormal region in the fundus image.

[0078] The operation unit 220 is composed of, for example, a touch panel, detects a touch operation on the screen displayed on the display unit 240, and outputs the detection result of the touch operation to the information processing unit 230.

[0079] The display unit 240 is composed of, for example, a touch panel, and displays a screen for receiving information indicating the fundus image received by the learning device 200 and information indicating the fundus fluorescein angiography image. Further, the display unit 240 displays a screen for receiving an operation of processing the NPA·NV annotation information and the accompanying finding annotation information received by the learning device 200.

[0080] All or part of the information processing unit 230 is, for example, a functional unit (hereinafter referred to as a software functional unit) realized by a processor such as a CPU (Central Processing Unit) executing a program 211 and an application 212 stored in the storage unit 210. Note that all or part of the information processing unit 230 may be realized by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), or FPGA (Field-Programmable Gate Array), or may be realized by a combination of a software functional unit and hardware. The information processing unit 230 includes, for example, an image acquisition unit 231, an annotation acquisition unit 232, a learning information generation unit 233, and a learning unit 234.

[0081] The image acquisition unit 231 acquires the fundus image notification information output by the communication unit 205, and acquires the patient ID included in the acquired fundus image notification information and the information indicating the fundus image. The image acquisition unit 231 associates the acquired patient ID with the information indicating the fundus image and stores them in the image DB 213. The image acquisition unit 231 acquires the fluorescein fundus angiography image notification information output by the communication unit 205, and acquires the patient ID included in the acquired fluorescein fundus angiography image notification information and the information indicating the fluorescein fundus angiography image. The image acquisition unit 231 associates the acquired patient ID with the information indicating the fluorescein fundus angiography image and stores them in the image DB 213.

[0082] The annotation acquisition unit 232 acquires the NPA·NV annotation notification information output by the communication unit 205, and acquires the patient ID included in the acquired NPA·NV annotation notification information and the NPA·NV annotation information. The annotation acquisition unit 232 associates the acquired patient ID with the NPA·NV annotation information and stores them in the annotation DB 214. The annotation acquisition unit 232 acquires the associated finding annotation notification information output by the communication unit 205, and acquires the patient ID included in the acquired associated finding annotation notification information and the associated finding annotation information. The annotation acquisition unit 232 associates the acquired patient ID with the associated finding annotation information and stores them in the annotation DB 214.

[0083] The learning information generation unit 233 acquires the patient ID stored in the image DB 213 of the storage unit 210, the information indicating the fundus image associated with the patient ID, and the information indicating the fluorescein fundus angiography image. The learning information generation unit 233 acquires the patient ID stored in the annotation DB 214 of the storage unit 210, the NPA·NV annotation information associated with the patient ID, and the associated finding annotation information. The learning information generation unit 233 specifies the abnormal blood circulation region based on the acquired information indicating the fluorescein fundus angiography image, the NPA·NV annotation information, and the associated finding annotation information.

[0084] Hereinafter, each component of the learning device 200 will be specifically described. FIG. 12 is a diagram showing an example of a fundus image, and FIG. 13 is a diagram showing an example of a fluorescein fundus angiography image. The learning information generation unit 233 extracts the green component of the fundus image, and removes the noise component from the green component-extracted fundus image, which is the fundus image from which the green component has been extracted. The learning information generation unit 233 divides the green component-extracted fundus image with the noise component removed into rectangles. The learning information generation unit 233 resizes the green component-extracted fundus image with the noise component removed to a predetermined size based on the image divided into rectangles. When resizing to the predetermined size, interpolation is performed by an interpolation method such as bicubic interpolation. The learning information generation unit 233 extracts the green component of the fluorescein fundus angiography image, and removes the noise component from the green component-extracted fluorescein fundus angiography image, which is the fluorescein fundus angiography image from which the green component has been extracted. The learning information generation unit 233 divides the green component-extracted fluorescein fundus angiography image with the noise component removed into rectangles. The learning information generation unit 233 resizes the green component-extracted fluorescein fundus angiography image with the noise component removed to a predetermined size based on the image divided into rectangles. When resizing to the predetermined size, interpolation is performed by an interpolation method such as bicubic interpolation.

[0085] The learning information generation unit 233 corrects the rotation component so that the positions of the eyes match between the green component-extracted fundus image with the noise component removed and the green component-extracted fluorescein fundus angiography image with the noise component removed, and resizes them to a predetermined size. The learning information generation unit 233 identifies the blood circulation abnormal region based on the green component-extracted fundus image with the noise component removed and the green component-extracted fluorescein fundus angiography image with the noise component removed, which have been corrected for the rotation component so that the positions of the eyes match and resized to a predetermined size.

[0086] FIG. 14 is a diagram showing an example of the blood circulation abnormal region. The learning information generation unit 233 generates NPA·NV learning information in which information indicating a fundus image is associated with a blood circulation abnormality region specified based on a fluorescein fundus angiography image corresponding to the fundus image, and stores the generated NPA·NV learning information in the learning information 215 of the storage unit 210. Returning to FIG. 11, the description will be continued.

[0087] The learning unit 234 acquires the NPA·NV learning information stored in the learning information 215 of the storage unit 210. The learning unit 234 acquires information indicating the fundus image and information indicating the blood circulation abnormality region included in the acquired NPA·NV learning information. The learning unit 234 uses the information indicating the acquired fundus image as input information, and uses the blood circulation abnormality region specified based on the fluorescein fundus angiography image corresponding to the fundus image as teacher information, and generates a learning model representing the relationship between the fundus image and the blood circulation abnormality region in the fundus image by learning.

[0088] Specifically, in a modification of the present embodiment, assuming that it is difficult to learn and predict the entire image at once due to the memory limitation of the GPU (graphics processing unit), the case of using a method of extracting patches from the image and training the extracted patches with a neural network will be described. Here, as an example, the patch size is set to 64px×64px, and the stride (the interval for moving the frame of patch extraction) is set to 2px.

[0089] The generated patches are divided into two groups: those containing positive regions and those not containing any. Patches are selected so that the ratio of the two groups used for learning is equal. A phenomenon in which a neural network shows good performance only for the learning data itself or an image very similar to it, and shows significantly low performance for unknown images, is called over-fitting, and this can be solved by collecting more samples or by adding geometric operations such as rotation to the learning data. In a modification of this embodiment, after generating the patch, the rotation angle was determined based on a normal distribution with σ = 3 deg., and horizontal flipping was performed with a 50% probability and vertical flipping was performed with a 20% probability.

[0090] FIG. 15 is a diagram showing an example of the structure of a neural network. An example of the structure of the neural network is based on U-Net. In FIG. 15, b represents convolution (kernel_size=(3,3)), g represents max pooling (pool_size=(2,2)), and o represents up sampling (size=(2,2)). After each convolution layer, the activation function ReLU was used and batch normalization was performed. However, for the last convolution layer, sigmoid was used as the activation function and batch normalization was not performed.

[0091] Also, the arrows indicated by a1, a2, a3, and a4 represent skip connections by concatenation. This is considered to contribute to restoring the position information of the image.

[0092] The learning unit 234 stores the generated learning model in the learning model 216 of the storage unit 210. The learning unit 234 includes the generated learning model, creates learning model notification information addressed to the diagnostic support device 300, and outputs the created learning model notification information to the communication unit 205.

[0093] (Diagnostic support device 300) FIG. 16 is a block diagram showing an example of a diagnostic support device according to a modification of an embodiment of the present invention. The diagnostic support device 300 includes a communication unit 305, a storage unit 310, an operation unit 320, an information processing unit 330, a display unit 340, and bus lines 350 such as an address bus and a data bus for electrically connecting each component as shown in FIG. 16.

[0094] The communication unit 305 is implemented by a communication module. The communication unit 305 communicates with external communication devices such as the ophthalmologist terminal 2 and the learning device 200 via the network N. Specifically, the communication unit 305 receives the learning model notification information sent by the learning device 200 and outputs the received learning model notification information to the information processing unit 330. The communication unit 305 receives the patient information sent by the ophthalmologist terminal 2 and outputs the received patient information to the information processing unit 330. The communication unit 305 acquires the diagnosis information output by the information processing unit 230 and transmits the acquired diagnosis information to the ophthalmologist terminal 2.

[0095] The storage unit 310 is implemented by, for example, a RAM, a ROM, an HDD, a flash memory, or a hybrid storage device in which a plurality of these are combined. The storage unit 310 stores a program 311 executed by the information processing unit 330, an app 312, and a learning model 216.

[0096] The program 311 is, for example, an operating system, which is located between the user or application program and the hardware, provides a standard interface to the user or application program, and at the same time efficiently manages each resource such as the hardware.

[0097] The app 312 causes the diagnostic support device 300 to receive the learning model notification information sent by the learning device 200 and store the learning model included in the received learning model notification information. The app 312 causes the diagnostic support device 300 to receive the patient information sent by the ophthalmologist terminal 2 and acquire the patient ID and the fundus image included in the received patient information. The app 312 causes the diagnostic support device 300 to identify the blood circulation abnormal region in the acquired fundus image using the stored learning model. The app 312 causes the diagnostic support device 300 to create diagnostic information including the fundus image of the patient, the information indicating the blood circulation abnormal region in the fundus image identified using the learning model, and the patient ID, with the ophthalmologist terminal 2 as the destination, and transmit the created diagnostic information to the ophthalmologist terminal 2.

[0098] The operation unit 320 is configured by, for example, a touch panel or the like, detects a touch operation on the screen displayed on the display unit 340, and outputs the detection result of the touch operation to the information processing unit 330.

[0099] The display unit 340 is configured by, for example, a touch panel, and displays a screen for receiving information indicating a fundus image included in the patient information received by the diagnostic support device 300. Further, the display unit 240 displays the result diagnosed by the diagnostic support device 300.

[0100] All or part of the information processing unit 330 is, for example, a software functional unit realized by a processor such as a CPU executing a program 311 stored in the storage unit 310 and an application 312. Note that all or part of the information processing unit 330 may be realized by hardware such as an LSI, an ASIC, or an FPGA, or may be realized by a combination of a software functional unit and hardware. The information processing unit 330 includes, for example, a reception unit 331, a specification unit 332, and a creation unit 333.

[0101] The reception unit 331 acquires the learning model notification information output by the communication unit 305, and acquires the learning model included in the acquired learning model notification information. The reception unit 331 receives the acquired learning model, and stores the received learning model in the learning model 216 of the storage unit 310. The reception unit 331 acquires the patient information output by the communication unit 305, and acquires the patient ID included in the acquired patient information and the information indicating the fundus image. The reception unit 331 receives the acquired patient ID and the information indicating the fundus image, and outputs the received patient ID and the information indicating the fundus image to the specification unit 332.

[0102] The specifying unit 332 acquires the patient ID output by the reception unit 331 and the information indicating the fundus image. The specifying unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310, and uses the acquired learning model to specify the blood circulation abnormal region in the acquired fundus image. The specifying unit 332 outputs the information indicating the blood circulation abnormal region in the specified fundus image and the patient ID to the creation unit 333. Specifically, in a modification of the present embodiment, similar to the learning device 200, the case of extracting patches from an image and using the extracted patches and the learning model to specify the blood circulation abnormal region in the fundus image will be described. Here, as an example, the patch size is set to 64 px × 64 px, and the stride (the interval for moving the frame of patch extraction) is set to 2 px. All of the generated patches are selected. The specifying unit 332 acquires an image of 64×64×1 based on the learning model. The specifying unit 332 votes the acquired pixel values to the corresponding pixels of the original image and averages them. Here, the specifying unit 332 may convert the acquired image into a color display.

[0103] The creation unit 333 acquires the patient ID output by the specifying unit 332 and the information indicating the blood circulation abnormal region in the fundus image. The creation unit 333 creates diagnostic information including the acquired patient ID and the information indicating the blood circulation abnormal region in the fundus image and addressed to the ophthalmologist terminal 2. The creation unit 333 outputs the created diagnostic information to the communication unit 305.

[0104] (Operation of the information processing system) With reference to FIGS. 17 and 18, an example of the operation of the information processing system according to the modification of the present embodiment will be described. FIG. 17 is a flowchart showing an example of the operation of the learning device included in the information processing system according to the modification of the present embodiment. FIG. 17 shows the operation after the ophthalmologist terminal 2 transmits the NPA·NV annotation notification information and the accompanying finding annotation notification information to the learning device 200, and the inspection device 3 transmits the fundus image notification information and the fluorescein fundus angiography image notification information to the learning device 200.

[0105] (Step S201) The communication unit 205 of the learning device 200 receives the fundus image notification information transmitted by the inspection device 3, and outputs the received fundus image notification information to the information processing unit 230. The image acquisition unit 231 of the information processing unit 230 acquires the fundus image notification information output by the communication unit 205, associates the patient ID included in the acquired fundus image notification information with the information indicating the fundus image, and stores it in the image DB 213 of the storage unit 210.

[0106] (Step S202) The communication unit 205 of the learning device 200 receives the fluorescein fundus angiography image notification information transmitted by the inspection device 3, and outputs the received fluorescein fundus angiography image notification information to the information processing unit 230. The image acquisition unit 231 of the information processing unit 230 acquires the fluorescein fundus angiography image notification information output by the communication unit 205, associates the patient ID included in the acquired fluorescein fundus angiography image notification information with the information indicating the fluorescein fundus angiography image, and stores it in the image DB 213 of the storage unit 210.

[0107] (Step S203) The communication unit 205 of the learning device 200 receives the NPA·NV annotation notification information transmitted by the ophthalmologist terminal 2, and outputs the received NPA·NV annotation notification information to the information processing unit 230. The annotation acquisition unit 232 of the information processing unit 230 acquires the NPA·NV annotation notification information output by the communication unit 205, associates the patient ID included in the acquired NPA·NV annotation notification information with the NPA·NV annotation information, and stores it in the annotation DB 214 of the storage unit 210.

[0108] The communication unit 205 of the learning device 200 receives the associated finding annotation notification information transmitted by the ophthalmologist terminal 2, and outputs the received associated finding annotation notification information to the information processing unit 230. The annotation acquisition unit 232 of the information processing unit 230 acquires the associated finding annotation notification information output by the communication unit 205, associates the patient ID included in the acquired associated finding annotation notification information with the associated finding annotation information, and stores it in the annotation DB 214 of the storage unit 210.

[0109] (Step S204) The learning information generation unit 233 of the learning device 200 acquires the patient ID stored in the image DB 213 of the storage unit 210, the information indicating the fundus image associated with the patient ID, and the information indicating the fluorescein fundus angiography image. The learning information generation unit 233 acquires the patient ID stored in the annotation DB 214 of the storage unit 210, the NPA·NV annotation information associated with the patient ID, and the associated finding annotation information. The learning information generation unit 233 specifies the blood circulation abnormal region based on the acquired information indicating the fluorescein fundus angiography image, the NPA·NV annotation information, and the associated finding annotation information. The learning information generation unit 233 generates NPA·NV learning information in which the information indicating the fundus image is associated with the blood circulation abnormal region specified based on the fluorescein fundus angiography image corresponding to the fundus image, and stores the generated NPA·NV learning information in the learning information 215 of the storage unit 210.

[0110] (Step S205) The learning unit 234 of the learning device 200 acquires the NPA·NV learning information stored in the learning information 215 of the storage unit 210. The learning unit 234 acquires the information indicating the fundus image included in the acquired NPA·NV learning information and the information indicating the blood circulation abnormal region. The learning unit 234 uses the acquired information indicating the fundus image as input information, and uses the blood circulation abnormal region specified based on the fluorescein fundus angiography image corresponding to the fundus image as teacher information, and generates a learning model representing the relationship between the fundus image and the blood circulation abnormal region in the fundus image by learning. The learning unit 234 stores the generated learning model in the learning model 216 of the storage unit 210.

[0111] In the flowchart shown in FIG. 17, the order of steps S201, S202, and S203 may be changed. According to the flowchart shown in FIG. 17, the learning device 200 can identify the abnormal blood circulation region based on the information indicating the fundus fluorescein angiography image, the NPA·NV annotation information, and the associated finding annotation information. The learning device 200 uses the information indicating the fundus image as input information, and uses the abnormal blood circulation region identified based on the fundus fluorescein angiography image corresponding to the fundus image as teacher information, and can generate a learning model representing the relationship between the fundus image and the abnormal blood circulation region in the fundus image by learning.

[0112] FIG. 18 is a flowchart showing an example of the operation of the diagnostic support device included in the information processing system according to the modified example of the present embodiment. FIG. 18 shows the operation after the learning device 200 transmits the learning model notification information to the diagnostic support device 300 and the ophthalmologist terminal 2 transmits the patient information to the diagnostic support device 300.

[0113] (Step S301) The communication unit 305 of the diagnostic support device 300 receives the learning model notification information transmitted by the learning device 200, and outputs the received learning model notification information to the information processing unit 330. The reception unit 331 acquires the learning model notification information output by the communication unit 305, and acquires the learning model included in the acquired learning model notification information. The reception unit 331 accepts the acquired learning model, and stores the accepted learning model in the learning model 216 of the storage unit 310.

[0114] (Step S302) The communication unit 305 receives the patient information transmitted by the ophthalmologist terminal 2, and outputs the received patient information to the information processing unit 330. The reception unit 331 acquires the patient information output by the communication unit 305, and acquires the patient ID included in the acquired patient information and the information indicating the fundus image. The reception unit 331 accepts the acquired patient ID and the information indicating the fundus image, and outputs the accepted patient ID and the information indicating the fundus image to the specifying unit 332.

[0115] (Step S303) The specifying unit 332 acquires the patient ID output by the reception unit 331 and the information indicating the fundus image. The specifying unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310.

[0116] (Step S304) The specifying unit 332 acquires the learning model stored in the learning model 216 of the storage unit 310, and uses the acquired learning model to specify the blood circulation abnormal region in the fundus image to be specified. The specifying unit 332 outputs the information indicating the blood circulation abnormal region in the specified fundus image and the patient ID to the creation unit 333.

[0117] (Step S305) The creation unit 333 acquires the patient ID output by the specifying unit 332 and the information indicating the blood circulation abnormal region in the fundus image. The creation unit 333 creates diagnostic information including the patient ID and the information indicating the blood circulation abnormal region in the acquired fundus image, and designates the ophthalmologist terminal 2 as the destination. The creation unit 333 outputs the created diagnostic information to the communication unit 305.

[0118] The communication unit 305 acquires the diagnostic information output by the creation unit 333, and transmits the acquired diagnostic information to the ophthalmologist terminal 2. According to the flowchart shown in FIG. 18, the diagnostic support device 300 can specify the blood circulation abnormal region in the fundus image to be specified by using the relationship between the generated fundus image with the information indicating the fundus image as the input information and the blood circulation abnormal region specified based on the fluorescein fundus angiography image corresponding to the fundus image as the teacher information, and the blood circulation abnormal region in the fundus image.

[0119] In the above-described modification, the diagnostic support device 300 receives the learning model notification information transmitted by the learning device 200 and stores the learning model included in the received learning model notification information in the storage unit 310. The diagnostic support device 300 uses the fundus image included in the patient information transmitted by the ophthalmologist terminal 2 and the stored learning model to identify the blood circulation abnormal region in the fundus image, and transmits a diagnostic result including information indicating the identified blood circulation abnormal region to the ophthalmologist terminal 2. However, the present invention is not limited to this example. For example, the diagnostic support device 300 may transmit patient information to the learning device 200. The learning device 200 receives the patient information transmitted by the diagnostic support device 300, and uses the fundus image included in the received patient information and the learning model to identify the blood circulation abnormal region in the fundus image. The learning device 200 creates a diagnostic result including information indicating the identified blood circulation abnormal region, and transmits the created diagnostic result to the diagnostic support device 300. The diagnostic support device 300 may receive the diagnostic result transmitted by the learning device 200 and transmit the received diagnostic result to the ophthalmologist terminal 2.

[0120] According to a modification of the present embodiment, the diagnostic support device 300 includes an identification unit that identifies an abnormal blood circulation region in a fundus image using a learning model that has learned the relationship between the fundus image and the abnormal blood circulation region in the fundus image based on the fundus image, which is an image of the fundus, and the abnormal blood circulation region identified based on the fundus fluorescein angiography image of the fundus, and an output unit that outputs information indicating the abnormal blood circulation region in the fundus image of the patient identified by the identification unit using the fundus image of the patient and the learning model. By configuring in this way, the diagnostic support device 300 can accurately estimate the abnormal circulation region from a normal fundus image using a learning model obtained by machine learning with the abnormal circulation region information identified by a doctor from the fundus fluorescein angiography image and the corresponding fundus image as teacher information. The abnormal blood circulation area is generated based on a fundus fluorescence angiography image and a diagnosis note of an ophthalmologist regarding either or both of the retinal non-perfusion area and neovascularization attached to the fundus fluorescence angiography image. By configuring in this way, based on the fundus fluorescence angiography image and the diagnosis note of the ophthalmologist regarding either or both of the retinal non-perfusion area and neovascularization attached to the fundus fluorescence angiography image, an abnormal blood circulation area serving as teacher information can be generated.

[0121] The specific part identifies either or both of the area corresponding to the retinal non-perfusion area and neovascularization in the fundus image. By configuring in this way, either or both of the area corresponding to the retinal non-perfusion area and neovascularization can be accurately identified from a normal fundus image.

[0122] The output part outputs an image with the abnormal blood circulation area identified by the specific part superimposed on the fundus image. By configuring in this way, an image with the abnormal circulation area superimposed on the fundus image can be obtained from a normal fundus image.

[0123] According to a modification of this embodiment, the learning device 200 has a learning part that generates, by learning, a learning model representing the relationship between the fundus image and the abnormal blood circulation area in the fundus image based on the fundus image which is an image of the fundus and the abnormal blood circulation area identified based on the fundus fluorescence angiography image of the fundus. By configuring in this way, the learning device 200 can generate, by machine learning, a learning model representing the relationship between the fundus image and the abnormal blood circulation area in the fundus image.

[0124] The abnormal blood circulation area is generated based on a fundus fluorescence angiography image and a diagnosis note of an ophthalmologist regarding either or both of the retinal non-perfusion area and neovascularization attached to the fundus fluorescence angiography image. By configuring in this way, based on the fundus fluorescence angiography image and the diagnosis note of the ophthalmologist regarding either or both of the retinal non-perfusion area and neovascularization attached to the fundus fluorescence angiography image, an abnormal blood circulation area serving as teacher information can be generated.

[0125] As described above, the embodiments of the present invention and their modifications have been explained. However, these embodiments and their modifications are presented as examples and are not intended to limit the scope of the invention. These embodiments and their modifications can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and the equivalent scope thereof. Incidentally, the aforementioned server 1, ophthalmologist terminal 2, inspection device 3, learning device 200, and diagnostic support device 300 have a computer inside. And, the processes of each of the above-described devices are stored in a computer-readable recording medium in the form of a program, and the above processes are performed by the computer reading and executing this program. Here, the computer-readable recording medium refers to a magnetic disk, magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, and the like. Also, this computer program may be distributed to the computer via a communication line, and the computer that has received this distribution may execute the program. Also, the above program may be for realizing a part of the aforementioned functions. Furthermore, it may be a so-called differential file (differential program) that can realize the aforementioned functions in combination with a program already recorded in the computer system.

[0126] Incidentally, the following additional remarks are disclosed regarding the above description. (Supplementary Note 1) On the computer that controls the information processing device, a step of acquiring fundus fluorescein angiography image information for acquiring fundus fluorescein angiography image information, and a step of acquiring NPA·NV annotation information for acquiring a diagnosis annotation of an ophthalmologist regarding at least one of a non-perfused retinal area and neovascularization attached to the fundus fluorescein angiography image information as NPA·NV annotation information, and An NPA·NV teacher information generation step for generating NPA·NV teacher information, which is teacher information for calculating the probability of existence of the non-perfused retinal area and the probability of existence of neovascularization based on the fundus fluorescein angiography image information and the NPA·NV annotation information corresponding to the fundus fluorescein angiography image information. A fundus image acquisition step for acquiring fundus image information. An NPA·NV existence probability calculation step for calculating the probability of existence of the non-perfused retinal area and the probability of existence of neovascularization in the fundus image information based on the NPA·NV teacher information. An estimated NPA·NV identification step for identifying, as an estimated NPA, an area estimated to correspond to the non-perfused retinal area in the fundus image information and, as an estimated NV, an area estimated to correspond to the neovascularization based on the probability of existence of the non-perfused retinal area and the probability of existence of the neovascularization. A program for executing control processing including the above steps. (Appendix 2) Further including an NPA·NV existence probability map generation step for generating an NPA·NV existence probability map in which the NPA existence probability and the NV existence probability are discriminatively displayed in the fundus image information. The program according to Appendix 1. (Appendix 3) An accompanying annotation acquisition step for acquiring, as accompanying finding annotation information, the ophthalmologist's diagnostic notes regarding accompanying findings attached to the fundus fluorescein angiography image information and the fundus image information. An accompanying finding teacher information generation step for generating accompanying finding teacher information, which is teacher information for calculating the probability of existence of the accompanying findings in the fundus image information based on the fundus fluorescein angiography image information, the fundus image information, and the accompanying finding annotation information corresponding to the fundus fluorescein angiography image information and the fundus image information. An accompanying finding existence probability calculation step for calculating the probability of existence of the accompanying findings in the fundus image information based on the accompanying finding teacher information. Further including the above steps. In the NPA·NV existence probability calculation step, further Execute a control process for calculating the probability of the presence of the retinal non-perfused area and the probability of the presence of neovascularization in the fundus image information based on the probability of the presence of the accompanying findings and the NPA teacher information. The program according to Appendix 1 or 2. (Appendix 4) Further include an accompanying finding presence probability map generation step of generating an accompanying finding presence probability map in which the probability of the presence of the accompanying findings is discriminatively displayed in the fundus image information. The program according to Appendix 3.

[0127] (Appendix 5) An information processing method executed by an information processing apparatus, A fluorescent fundus angiography image acquisition step of acquiring fluorescent fundus angiography image information, An NPA·NV annotation acquisition step of acquiring, as NPA·NV annotation information, a diagnosis annotation of an ophthalmologist regarding at least one of a retinal non-perfused area and neovascularization attached to the fluorescent fundus angiography image information. An NPA·NV teacher information generation step of generating NPA·NV teacher information serving as teacher information for calculating the probability of the presence of the retinal non-perfused area and the probability of the presence of neovascularization based on the fluorescent fundus angiography image information and the NPA·NV annotation information corresponding to the fluorescent fundus angiography image information. A fundus image acquisition step of acquiring fundus image information, An NPA·NV presence probability calculation step of calculating the probability of the presence of the retinal non-perfused area and the probability of the presence of neovascularization in the fundus image information based on the NPA·NV teacher information. An estimated NPA·NV identification step of identifying, as an estimated NPA, an area estimated to correspond to the retinal non-perfused area in the fundus image information and identifying, as an estimated NV, an area estimated to correspond to the neovascularization based on the probability of the presence of the retinal non-perfused area and the probability of the presence of neovascularization. An information processing method including the above steps. (Appendix 6) A fluorescent fundus angiography image acquisition means for acquiring fluorescent fundus angiography image information, An NPA·NV annotation acquisition means for acquiring, as NPA·NV annotation information, a diagnosis note of an ophthalmologist regarding at least one of a non-perfused retinal area and neovascularization attached to the fundus fluorescence angiography image information, An NPA·NV teacher information generation means for generating NPA·NV teacher information, which is teacher information for calculating the probability of existence of the non-perfused retinal area and the probability of existence of neovascularization, based on the fundus fluorescence angiography image information and the NPA·NV annotation information corresponding to the fundus fluorescence angiography image information, A fundus image acquisition means for acquiring fundus image information, An NPA·NV existence probability calculation means for calculating the probability of existence of the non-perfused retinal area and the probability of existence of neovascularization in the fundus image information based on the NPA·NV teacher information, An estimated NPA·NV identification means for identifying, as an estimated NPA, an area estimated to correspond to the non-perfused retinal area in the fundus image information and, as an estimated NV, an area estimated to correspond to the neovascularization based on the probability of existence of the non-perfused retinal area and the probability of existence of the neovascularization, An information processing apparatus comprising the above.

Explanation of Signs

[0128] 1: Server 2: Ophthalmologist terminal 3: Examination equipment 11: CPU 12: ROM 13: RAM 14: Bus 15: Input / output interface 16: Display unit 17: Input unit 18: Storage unit 19: Communication unit 20: Drive 30: Removable media 101: Image acquisition unit 102: Annotation acquisition unit 103: Teacher information generation unit 104: Calculation unit 105: Map generation unit 106: Presumed NPA·NV Specific Part 107: Presumed NPA·NV Display Control Part 200: Learning Device 205: Communication Part 210: Memory Part 211: Program 212: Application 213: Image DB 214: Annotation DB 215: Learning Information 216: Learning Model 220: Operation Part 230: Information Processing Part 231: Image Acquisition Part 232: Annotation Acquisition Part 233: Learning Information Generation Part 234: Learning Part 240: Display Part 250: Bus Line 300: Diagnostic Support Device 305: Communication Part 310: Memory Part 311: Program 312: Application 320: Operation Part 330: Information Processing Part 331: Reception Part 332: Specific Part 333: Creation Part 240: Display Part 350: Bus Line 401: Image DB 402: Annotation DB 403: Teacher DB A: Presumed NPA B: Presumed NV C: Fluorescein Fundus Angiography Image Information D: Ophthalmologist E: NPA·NV Existence Probability Map F: Fundus Image Information

Claims

【Claim 1】 Using a learning model that has learned the relationship between a fundus image, which is an image of the fundus of the eye, and an abnormal blood circulation region identified based on the fluorescein fundus angiogram image of the fundus of the eye, a specifying unit that specifies an abnormal blood circulation region in the fundus image, and An output unit that outputs information indicating an abnormal blood circulation region in the fundus image of the patient specified by the specifying unit using the fundus image of the patient and the learning model A diagnostic support device having the above.

Images