Fundus image processing device and fundus image processing program

Abstract

The present invention provides a fundus image processing device and a fundus image processing program capable of appropriately presenting information on a wide range of blood vessels in a two-dimensional fundus image. [Solution] The control unit of the fundus image processing device performs a fundus image acquisition step, a vascular image acquisition step, and a probability map generation step. In the fundus image acquisition step, the control unit acquires multiple fundus images, including the blood vessels of the fundus of the eye under examination, taken by a fundus image acquisition device. In the vascular image acquisition step, the control unit acquires vascular images showing at least one of the arteries and veins included in each acquired fundus image. In the probability map generation step, the control unit generates a retinal vascular distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye under examination by adding the multiple vascular images acquired for each of the multiple fundus images in an aligned state.

Description

[Technical Field]

[0001] The present disclosure relates to a fundus image processing apparatus that processes a fundus image of an eye to be examined, and a fundus image processing program executed in the fundus image processing apparatus. [Background Art]

[0002] By observing the fundus, the state of blood vessels in a living body can be grasped non-invasively. Conventionally, information related to blood vessels (at least one of arteries and veins) obtained from fundus images has been used for various diagnoses and the like. For example, in the method for measuring an arteriovenous diameter ratio disclosed in Patent Document 1, a region R surrounded by two concentric circles with different radii is defined with the center of the optic papilla (hereinafter simply referred to as "papilla") as a reference n is set in plurality. The set region R n A plurality of blood vessels are extracted within. From the plurality of extracted blood vessels, two blood vessels with a small distance therebetween are selected as a blood vessel pair. An arteriovenous diameter ratio is calculated from the selected blood vessel pair. Further, methods for calculating the arteriovenous diameter ratio based on blood vessels after the second branching have also been proposed. [Prior Art Documents] [Non-Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2014-193225 [Summary of the Invention]

[0004] In the conventional fundus blood vessel analysis method described above, in order to facilitate comparison of the blood vessel states of a plurality of eyes, only one-dimensional information related to blood vessels in a specific region (for example, a region surrounded by two concentric circles centered on the papilla) is handled, and blood vessel information in other regions is not referenced. Therefore, if information related to a wide range of blood vessels in a two-dimensional fundus image can be appropriately presented, this will be very medically useful information.

[0005] A typical object of this disclosure is to provide a fundus image processing device and a fundus image processing program capable of appropriately presenting information about a wide range of blood vessels in a two-dimensional fundus image. [Means for solving the problem]

[0006] A first embodiment of a fundus image processing device provided by a typical embodiment of the present disclosure is a fundus image processing device for processing fundus images of an eye under examination, wherein the control unit of the fundus image processing device performs the following steps: a fundus image acquisition step of acquiring a plurality of fundus images including blood vessels of the fundus of the eye under examination, taken by a fundus image acquisition device; a blood vessel image acquisition step of acquiring a blood vessel image showing at least one of arteries and veins included in each of the acquired fundus images; and a probability map generation step of generating a retinal blood vessel distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye under examination by adding the plurality of blood vessel images acquired for each of the plurality of fundus images in an aligned state.

[0007] A second aspect of a fundus image processing device provided by a typical embodiment of the present disclosure is a fundus image processing device for processing a fundus image of an eye under examination, wherein the control unit of the fundus image processing device performs the following steps: a fundus image acquisition step of acquiring a fundus image to be analyzed, which includes blood vessels of the fundus of the eye under examination and is captured by a fundus image acquisition device; a blood vessel image acquisition step of acquiring a target blood vessel image, which is a blood vessel image showing at least one of arteries and veins included in the acquired fundus image; and a feature information generation step of generating blood vessel distribution feature information that shows the characteristics of the blood vessel distribution of the target blood vessel image by processing information of the region corresponding to the blood vessel region of the target blood vessel image from a retinal blood vessel distribution probability map, which is generated by adding a plurality of blood vessel images in an aligned state and shows the distribution of the probability of the presence of blood vessels present in the retina of the eye under examination.

[0008] A first aspect of a fundus image processing program provided by a typical embodiment of the present disclosure is a fundus image processing program executed by a fundus image processing device that processes fundus images of an eye under examination, wherein the fundus image processing program is executed by a control unit of the fundus image processing device, causing the fundus image processing device to execute: a fundus image acquisition step of acquiring a plurality of fundus images including blood vessels of the fundus of an eye under examination, taken by a fundus image acquisition device; a blood vessel image acquisition step of acquiring a blood vessel image showing at least one of arteries and veins included in each of the acquired fundus images; and a probability map generation step of generating a retinal blood vessel distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye under examination by adding the plurality of blood vessel images acquired for each of the plurality of fundus images in an aligned state.

[0009] A second aspect of the fundus image processing program provided by a typical embodiment of the present disclosure is a fundus image processing device that processes fundus images of an eye under examination, wherein the fundus image processing program is executed by a control unit of the fundus image processing device, causing the fundus image processing device to execute: a fundus image acquisition step of acquiring a fundus image to be analyzed, which includes blood vessels of the fundus of the eye under examination and is captured by a fundus image acquisition device; a blood vessel image acquisition step of acquiring a target blood vessel image, which is a blood vessel image showing at least one of arteries and veins included in the acquired fundus image; and a feature information generation step of generating blood vessel distribution feature information that shows the characteristics of the blood vessel distribution of the target blood vessel image by processing information of the region corresponding to the region of blood vessels in the target blood vessel image from a retinal blood vessel distribution probability map that shows the distribution of the probability of existence of blood vessels present in the retina of the eye under examination, which is generated by adding up a plurality of blood vessel images in an aligned state.

[0010] According to the fundus image processing device and fundus image processing program described herein, information regarding a wide range of blood vessels in a two-dimensional fundus image is appropriately presented. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the schematic configuration of the fundus image processing devices 1 and 21, and the fundus image acquisition devices 11A and 11B. [Figure 2] This figure shows an example of a fundus image 30 and vascular images 40A and 40B showing the blood vessels included in the fundus image 30. [Figure 3] This is a flowchart of the fundus image processing performed by the fundus image processing device 1 of the first embodiment. [Figure 4] This figure shows an example of the arterial retinal vascular distribution probability map 50A and the venous retinal vascular distribution probability map 50B. [Figure 5] This figure shows an example of the target vessel feature map 51A for arteries, the target vessel feature map 51B for veins, the vessel distribution histogram 52A for arteries, and the vessel distribution histogram 52B for veins. [Figure 6] This figure shows an example of an age-based histogram of arterial data. [Figure 7] This figure shows an example of a histogram of venous data broken down by age. [Figure 8] This is a difference histogram for arteries between the DM group and the control group. [Figure 9] This is a difference histogram for venous data between the DM group and the control group. [Figure 10] Figure 8 shows map 53A, which overlays the differential histogram data for the arteries of the DM group onto fundus image 30, and Figure 9 shows map 53B, which overlays the differential histogram data for the veins of the DM group onto fundus image 30. [Figure 11] This is a flowchart of the fundus image processing performed by the fundus image processing device 21 of the second embodiment. [Modes for carrying out the invention]

[0012] <Overview> The control unit of the fundus image processing apparatus of the first embodiment illustrated in this disclosure performs a fundus image acquisition step, a vascular image acquisition step, and a probability map generation step. In the fundus image acquisition step, the control unit acquires a plurality of fundus images, including the blood vessels of the fundus of the eye under examination, taken by a fundus image acquisition device. In the vascular image acquisition step, the control unit acquires vascular images showing at least one of the arteries and veins included in each acquired fundus image. In the probability map generation step, the control unit generates a retinal vascular distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye under examination by adding the plurality of vascular images acquired for each of the plurality of fundus images in an aligned state.

[0013] The vascular distribution probability map generated by the technology of this disclosure appropriately shows the two-dimensional distribution of the probability of the presence of retinal blood vessels within a set (population) of multiple eyes from which fundus images have been taken. Therefore, for example, by comparing the fundus image of the eye being examined (e.g., a vascular image obtained from the fundus image, or the fundus image itself) with the retinal vascular distribution probability map, the state of the blood vessels of the eye being examined relative to the blood vessels of the population can be appropriately understood over a wide range (e.g., for each region). By generating a vascular distribution probability map for each of several different populations, it is also possible to appropriately understand the characteristics of the state of blood vessels in each population over a wide range. In other words, the retinal vascular distribution probability map provides information indicating the state of retinal blood vessels for each region. Therefore, the technology of this disclosure appropriately presents information on a wide range of blood vessels in the fundus.

[0014] Furthermore, when focusing on multiple eyes of the same animal, the general structure of the retinal blood vessels in each eye tends to be uniform regardless of the eye being examined. Therefore, the correlation between the probability of blood vessels existing in each region on the blood vessel distribution probability map and the diameter of blood vessels in each region in the population becomes high. Thus, the blood vessel distribution probability map makes it easy to understand the state of blood vessel diameter for each region.

[0015] Various fundus images capable of acquiring blood vessel images can be used as the fundus image used to generate a retinal blood vessel distribution probability map. As an example, in the present disclosure, a two-dimensional color fundus image obtained by photographing the fundus from the front with a fundus camera is used as the fundus image. In this case, a blood vessel image can be appropriately acquired based on the color fundus image. Alternatively, a retinal blood vessel distribution probability map may be generated based on a two-dimensional OCT angiography image of the fundus captured by an OCT (Optical Coherence Tomography) apparatus. A two-dimensional fundus image obtained by photographing the fundus from the front with a scanning laser ophthalmoscope (SLO) may be input to a mathematical model.

[0016] In the probability map generation step, the control unit may generate a retinal blood vessel distribution probability map by adding and averaging a plurality of aligned blood vessel images. In this case, the retinal blood vessel distribution probability map can be handled in the same unit as the unit of the value (e.g., luminance value, etc.) of each pixel in each blood vessel image. However, it is also possible to omit the averaging after addition of a plurality of blood vessel images.

[0017] In the blood vessel image acquisition step, the control unit may acquire a blood vessel image indicating at least one of an artery and a vein included in the fundus image by inputting the fundus image to a mathematical model trained by a machine learning algorithm. In this case, it is easy to acquire a blood vessel image that indicates blood vessels with high accuracy. The mathematical model may be trained using fundus images of an eye to be examined captured in the past as input training data, and using a blood vessel image indicating at least one of an artery and a vein in the fundus image of the input training data as output training data. In this case, the trained mathematical model can appropriately output a blood vessel image based on the input fundus image.

[0018] However, it is also possible to change the method for acquiring a blood vessel image. For example, at least one of the plurality of blood vessel images may be generated in accordance with an instruction input by an operator via an operation unit (e.g., a mouse, etc.).

[0019] When the control unit displays the retinal vascular distribution probability map on the display unit, it may represent the brightness value of each pixel in the two-dimensional retinal vascular distribution probability map using color and shade (i.e., it may be used as a heat map). For example, in this disclosure, the higher the brightness value of a pixel, the darker the warm color, and the lower the brightness value of a pixel, the darker the cool color. By displaying a heat map of the retinal vascular distribution probability map, the two-dimensional distribution of the probability of the presence of blood vessels becomes easier to grasp more appropriately. However, it is also possible to change the specific display method of the retinal vascular distribution probability map. For example, the control unit may also display a monochrome retinal vascular distribution probability map on the display unit corresponding to the brightness value of each pixel.

[0020] The control unit may, in the vascular image acquisition step, acquire both arterial and venous vascular images included in each fundus image. In the probability map generation step, the control unit may generate an arterial retinal vascular distribution probability map by adding up multiple arterial vascular images, and may also generate a venous retinal vascular distribution probability map by adding up multiple venous vascular images. Depending on the patient's disease or other condition, different changes may appear between the arteries and veins in the fundus. Therefore, generating both an arterial retinal vascular distribution probability map (hereinafter sometimes referred to as the "arterial distribution probability map") and a venous retinal vascular distribution probability map (hereinafter sometimes referred to as the "vein distribution probability map") makes it easier to obtain more useful information.

[0021] However, the control unit may acquire multiple vascular images in which both arteries and veins are visible in the fundus, and generate a retinal vascular distribution probability map in which arteries and veins are grouped together by adding the acquired multiple vascular images. Alternatively, a retinal vascular distribution probability map in which arteries and veins are not classified in any way may be generated. Furthermore, the fundus image processing device can generate only one of the arterial distribution probability map or the vein distribution probability map.

[0022] The control unit may further perform a disc positioning step to identify the position of the optic nerve head in the acquired vascular image. In the probability map generation step, the control unit may generate a retinal vascular distribution probability map by adding together the positions of multiple vascular images with respect to the position of the optic nerve head. The structure of retinal blood vessels in the fundus of the eye is such that they spread outward from the disc. The general structure of retinal blood vessels spreading from the disc tends to be uniform regardless of the eye being examined. Therefore, when the retinal vascular distribution probability map is generated, aligning multiple vascular images with respect to the position of the disc further improves the accuracy of the distribution of the probability of the presence of blood vessels shown in the retinal vascular distribution probability map.

[0023] In detail, the control unit may, in the disc position identification step, identify the centroid position of the optic nerve head as seen in the vascular image. In the probability map generation step, the control unit may add together each of the multiple vascular images after they have been aligned with respect to the centroid position of the optic nerve head. In this case, since the multiple vascular images are aligned with respect to a single centroid position, the accuracy of the generated retinal vascular distribution probability map is further improved.

[0024] In the optic disc positioning step, the control unit may determine the position of the optic disc in the vascular image by inputting the fundus image into a mathematical model trained by a machine learning algorithm. In this case, the position of the optic disc is more likely to be determined with high accuracy. If a machine learning algorithm is also used to acquire the vascular image, the vascular image and the position of the optic disc may both be output by the same mathematical model, or they may be output separately by different mathematical models.

[0025] However, the method for determining the position of the optic disc can be changed. For example, an operator may determine the position of the optic disc in a fundus image or vascular image and input the determined position into the fundus image processing device. The control unit may then identify the position input via an operating unit or the like as the position of the optic disc. Alternatively, the control unit may determine the position of the optic disc by performing known image processing on the fundus image or vascular image.

[0026] The control unit may automatically align multiple vascular images. In this case, the amount of work required by the operator to generate the retinal vascular distribution probability map is appropriately reduced. Alternatively, the operator may input instructions for aligning multiple vascular images to the fundus image processing device via the control unit. The fundus image processing device may align multiple vascular images in response to the instructions input via the control unit. Even in this case, the retinal vascular distribution probability map is appropriately generated.

[0027] The control unit may perform processing to make the area of ​​the fundus captured in the fundus image and vascular image more uniform. For example, the control unit may make the width of the captured area in the image, which varies with the magnification at which the fundus image was taken, more uniform. Alternatively, the control unit may make the area of ​​the fundus captured in the image more uniform by extracting images within a specific range from the fundus image or vascular image. However, if multiple fundus images of the same area are taken by the fundus image acquisition device at the same magnification, the processing to make the area of ​​the fundus more uniform can be omitted.

[0028] The control unit may further perform a target vessel image acquisition step and a feature map generation step. In the target vessel image acquisition step, the control unit acquires a target vessel image, which is a vessel image to be analyzed. In the feature map generation step, the control unit generates a target vessel feature map from the retinal vessel distribution probability map, which corresponds to the region of the vessel in the target vessel image. The target vessel feature map shows the characteristics of the vessel distribution in the target vessel image. For example, according to the target vessel feature map, the probability of the presence of vessels in the region of the vessels shown in the target vessel image can be appropriately grasped on the image. Also, as mentioned above, the correlation between the probability of the presence of vessels in each region on the vessel distribution probability map and the thickness of the vessels in each region in the population is high. Therefore, according to the target vessel feature map, it is easy to understand, for example, when thick vessels are present in a region where thin vessels are likely to be present in the population, or when thin vessels are present in a region where thick vessels are likely to be present in the population. As described above, the characteristics of the vessels shown in the target vessel image can be easily grasped according to the target vessel feature map.

[0029] The target vascular images may be obtained based on the fundus images to be analyzed (hereinafter referred to as "target fundus images"). For example, the target vascular images may be obtained by inputting the target fundus images into a mathematical model trained by a machine learning algorithm. In addition, in the feature map generation step, the target vascular feature map may be generated after the retinal vascular distribution probability map and the target vascular images have been aligned. Various methods described above (for example, a method based on the position of the optic disc) can be used for alignment. Alignment may be performed automatically or manually by an operator.

[0030] The specific method for generating the target vascular feature map can also be selected as appropriate. For example, the control unit may generate the target vascular feature map by extracting regions corresponding to the vascular regions in the target vascular image from the retinal vascular distribution probability map. Alternatively, the control unit may generate the target vascular feature map by masking regions other than the vascular regions in the target vascular image from the retinal vascular distribution probability map.

[0031] The control unit may further perform a target vessel image acquisition step and a vessel distribution histogram generation step. In the target vessel image acquisition step, the control unit acquires the target vessel image, which is the vessel image to be analyzed. In the vessel distribution histogram generation step, the control unit generates a vessel distribution histogram based on information from the retinal vessel distribution probability map, specifically the region corresponding to the vessel region of the target vessel image. The vessel distribution histogram shows the number of pixels within the vessel region of the target vessel image according to the probability of existence of the vessel indicated by the retinal vessel distribution probability map. The vessel distribution histogram clearly shows the characteristics of the vessel distribution in the target vessel image. For example, consider a case where the range from the lowest to the highest probability of existence of a vessel in the retinal vessel distribution probability map is divided into N areas (256 in this disclosure as an example). In this case, each pixel within the vessel region of the target vessel image will belong to one of the N areas (hereinafter referred to as "probability area") divided in order of the probability of existence of the vessel. A vascular distribution histogram accurately captures the distribution of the number of pixels belonging to each of N probability areas for multiple pixels within the vascular region of a target vascular image. Therefore, vascular distribution histograms provide useful information that could not be obtained with previous analysis methods.

[0032] The target vascular images may be obtained using the same method as described in the method for generating the target vascular feature map. Furthermore, the information for the regions in the retinal vascular distribution probability map corresponding to the vascular regions in the target vascular images may be obtained after alignment between the retinal vascular distribution probability map and the target vascular images. Various methods described above can be used for this alignment.

[0033] The control unit may further perform an aggregated histogram generation step, which generates an aggregated histogram by aggregating multiple vascular distribution histograms generated for multiple target vascular images. In this case, the characteristics of the vascular distribution for the population targeted for aggregation can be appropriately grasped by the aggregated histogram.

[0034] The control unit may, in the aggregated histogram generation step, correct for the influence of changes in the distribution of the probability of blood vessel presence due to the age of each subject being analyzed, and then aggregate multiple blood vessel distribution histograms. In this case, it is possible to grasp the characteristics of blood vessel distribution using aggregated histograms in which the influence of age-related changes has been suppressed.

[0035] Furthermore, specific methods for correcting for the effects of changes in the distribution of blood vessel probability due to age can be selected as appropriate. For example, the control unit may correct for the effects of age-related changes by performing regression analysis using a regression equation with age as the explanatory variable and probability as the dependent variable for each probability area of ​​blood vessel existence, which is divided according to the probability of blood vessel existence.

[0036] The control unit may further perform a difference histogram generation step to generate a difference histogram between the vascular distribution histogram generated for each target vascular image and the aggregated histogram generated in the aggregated histogram generation step. In this case, the vascular distribution characteristics for each analysis target are shown in comparison with the vascular distribution characteristics of the population in the aggregated histogram. Therefore, the vascular distribution characteristics for each analysis target can be grasped even more easily.

[0037] The control unit may generate an age-specific aggregated histogram by aggregating the multiple vascular distribution histograms generated for each of the multiple target vascular images, according to the age group to which the subject being analyzed belongs. The age-specific aggregated histogram makes it possible to appropriately grasp the characteristics of vascular distribution by age group.

[0038] The control unit may further perform a population-specific aggregated histogram generation step and a difference histogram generation step. In the population-specific aggregated histogram generation step, the control unit generates a population-specific aggregated histogram by aggregating multiple vascular distribution histograms generated for multiple target vascular images belonging to each of the multiple populations to which the target vascular images belong. In the difference histogram generation step, the control unit generates a difference histogram between the multiple aggregated histograms generated for each population. In this case, the characteristics of the vascular distribution for each population can be appropriately grasped by the difference histogram. For example, by determining the population according to various conditions such as the presence or absence of disease in the subject, or the state of the disease, useful information that could not be obtained with past analysis methods can be obtained.

[0039] The multiple populations from which the aggregated histogram is generated may include a population of diabetic patients who have not developed diabetic retinopathy. In this case, the characteristics of diabetic patients who have not developed diabetic retinopathy will be clearly reflected in the difference histogram. Therefore, for example, by comparing information about the vascular distribution histogram for each individual eye with the difference histogram, it may be possible to predict the likelihood that a subject has diabetes or the likelihood that they will develop diabetic retinopathy.

[0040] However, the technologies illustrated in this disclosure are also likely to provide useful information on diseases other than those related to diabetes. For example, the technologies illustrated in this disclosure may provide useful information on at least one of the following: arteriosclerotic diseases, retinal vascular occlusive diseases (branch retinal vein occlusion (BRVO), branch retinal artery occlusion (BRAO), central retinal vein occlusion (CRVO), central retinal artery occlusion (CRAO)), retinal degenerative diseases such as retinitis pigmentosa, glaucoma, uveitis, refractive errors (myopia, hyperopia), and follow-up of retinopathy of prematurity.

[0041] Furthermore, when calculating the difference between the aggregated histogram of a population that meets specific conditions and the aggregated histogram of a comparison population, the comparison population can be set as appropriate. For example, the comparison population may include all subjects and eyes tested. Alternatively, the comparison population may be a population obtained by excluding the subjects and eyes of the comparison population from all subjects and eyes tested.

[0042] The control unit may further perform a feature information generation step and a mathematical model construction step. In the feature information generation step, the control unit processes information from the retinal vascular distribution probability map that corresponds to the vascular regions of a specific vascular image to generate vascular distribution feature information (for example, at least one of the vascular distribution histogram or target vascular feature map described above) that shows the characteristics of the vascular distribution of a specific vascular image. In the mathematical model construction step, the control unit uses the vascular distribution feature information as input training data and information indicating the presence or absence of a specific disease in subjects from whom the input training data was obtained as output training data to train and construct a mathematical model that outputs information about a specific disease using a machine learning algorithm.

[0043] Vascular distribution feature information reflects the characteristics of the vascular distribution in the target vascular image. Furthermore, depending on the type of disease, the characteristics of the vascular distribution in the target vascular image often change depending on the disease state of the subject (target subject) from whom the vascular image was acquired. Therefore, by inputting the vascular distribution feature information of the target subject into a mathematical model pre-trained with a training dataset containing vascular distribution feature information from multiple subjects, information regarding a specific disease can be appropriately obtained.

[0044] Furthermore, vascular distribution histograms may be used as the vascular distribution feature information (input training data) for training the mathematical model, and as the vascular distribution feature information input to the mathematical model. As mentioned above, vascular distribution histograms clearly show the characteristics of vascular distribution in the target vascular image. Therefore, using vascular distribution histograms makes it easier to obtain information about specific diseases with higher accuracy.

[0045] However, as mentioned above, the characteristics of the vascular distribution in the target vascular images are appropriately represented in the target vascular feature map. Therefore, it is possible to use the target vascular feature map as input training data and input data for mathematical models, either in place of or in conjunction with the vascular distribution histogram.

[0046] The mathematical model may be trained using vascular distribution feature information as input training data, and information indicating whether or not the subject has diabetes as output training data. The mathematical model may output information about the subject's diabetes (e.g., the probability of having diabetes) when vascular distribution feature information of the target vascular image is input. Research conducted by the inventor of the present invention has revealed that vascular distribution feature information changes depending on whether or not the subject has diabetes (regardless of whether or not diabetic retinopathy has developed). Therefore, by inputting the subject's vascular distribution information into the mathematical model, information about the subject's diabetes can be obtained with high accuracy.

[0047] However, information indicating the presence or absence of diseases other than diabetes-related diseases may be used as training data for output (for example, at least one of the following: arteriosclerotic diseases, retinal vascular occlusive diseases (branch retinal vein occlusion (BRVO), branch retinal artery occlusion (BRAO), central retinal vein occlusion (CRVO), central retinal artery occlusion (CRAO)), retinal degenerative diseases such as retinitis pigmentosa, glaucoma, uveitis, refractive errors (myopia, hyperopia), retinopathy of prematurity, etc.)). Even in this case, it may still be possible to obtain appropriate information about the diseases of the subjects.

[0048] Furthermore, the vascular distribution feature information for training the mathematical model (input training data), and the vascular distribution feature information input to the mathematical model, may include both arterial and venous vascular distribution feature information. In this case, it becomes easier to obtain disease-related information with higher accuracy. However, it is also possible to use combined vascular distribution feature information for arteries and veins. It is also possible to use vascular distribution feature information for either arteries or veins.

[0049] The training data used to train the mathematical model may include, in addition to vascular distribution feature information, information on at least one of the age and sex of the subject from whom the vascular distribution feature information was obtained. Vascular distribution feature information often changes depending on at least one of the subject's age and sex. Therefore, including information on at least one of age and sex in the training data makes it easier for the mathematical model to output appropriate information according to the age and sex of the target subject. When including information on at least one of age and sex in the training data, it is desirable to input information on at least one of the age and sex of the subject from whom the vascular distribution feature information was obtained, in addition to the vascular distribution feature information, into the constructed mathematical model.

[0050] The control unit of the fundus image processing apparatus of the second embodiment illustrated in this disclosure performs a fundus image acquisition step, a vascular image acquisition step, and a feature information generation step. In the fundus image acquisition step, the control unit acquires a fundus image of the eye under examination, including the blood vessels of the fundus, which is to be analyzed, taken by a fundus image acquisition device. In the vascular image acquisition step, the control unit acquires a target vascular image, which is a vascular image showing at least one of the arteries and veins included in the acquired fundus image. In the feature information generation step, the control unit generates vascular distribution feature information that shows the characteristics of the vascular distribution of the target vascular image by processing the information of the region corresponding to the vascular region of the target vascular image from a retinal vascular distribution probability map, which shows the distribution of the probability of the presence of blood vessels present in the retina of the eye under examination, generated by adding up multiple vascular images in an aligned state. According to the vascular distribution probability map, the two-dimensional distribution of the probability of the presence of retinal blood vessels in a set (population) of multiple eyes from which fundus images have been taken is appropriately shown. Furthermore, the information of the region corresponding to the vascular region of the target vascular image in the retinal vascular distribution probability map appropriately represents the characteristics of the vascular distribution of the target vascular image. Therefore, by generating vascular distribution feature information, the characteristics of the vascular distribution in the target vascular image can be easily understood.

[0051] In the second embodiment, the fundus image processing device may acquire information on a retinal vascular distribution probability map that has been pre-generated by a medical device manufacturer or research facility, and perform a feature information generation step based on the acquired information on the retinal vascular distribution probability map. Alternatively, the fundus image processing device may generate the retinal vascular distribution probability map. The method for generating the vascular distribution probability map can be the same as the method described in the first embodiment.

[0052] In the feature information generation step, the control unit may generate a target vascular feature map from the retinal vascular distribution probability map, specifically the map of the region corresponding to the vascular region in the target vascular image. As mentioned above, the target vascular feature map allows for a quick understanding of the vascular distribution characteristics in the target vascular image. Therefore, it appropriately assists in the diagnosis of the eye under examination.

[0053] The method for outputting the generated target blood vessel feature map can be selected as appropriate. For example, the control unit may output the target blood vessel feature map by displaying it on the display unit.

[0054] In the feature information generation step, the control unit may generate a vascular distribution histogram based on information from the retinal vascular distribution probability map, specifically the region corresponding to the vascular region in the target vascular image. This histogram shows the number of pixels within the vascular region of the target vascular image according to the probability of the presence of the vascular region indicated by the retinal vascular distribution probability map. As described above, the vascular distribution histogram clearly shows the characteristics of the vascular distribution in the target vascular image. Therefore, it appropriately assists in the diagnosis of the eye under examination.

[0055] The method for outputting the generated vascular distribution histogram can also be selected as appropriate. For example, the control unit may output the vascular distribution histogram by displaying it on the display unit. In this case, the control unit may display the vascular distribution histogram generated for the target vascular image and the aggregated histogram described in the first embodiment on the display unit in a comparable manner (for example, side by side). In this case, the user can easily compare the characteristics of the vascular distribution for the population of the aggregated histogram with the characteristics of the vascular distribution of the fundus being analyzed. In the second embodiment, the fundus image processing device may acquire an aggregated histogram that has been pre-generated by a medical device manufacturer or research facility. As mentioned above, the aggregated histogram is generated by aggregating multiple vascular distribution histograms generated for each of multiple vascular images. The vascular distribution histogram is generated based on the information of the region corresponding to the vascular region of each vascular image in the retinal vascular distribution probability map. The vascular distribution histogram shows the number of pixels within the vascular region of each vascular image according to the probability of existence of the vascular shown by the retinal vascular distribution probability map.

[0056] In the feature information generation step, the control unit may generate a vascular distribution histogram based on information from the retinal vascular distribution probability map, specifically the region corresponding to the vascular region in the target vascular image. The control unit may also generate a difference histogram, which is the difference between an aggregated histogram (a sum of multiple vascular distribution histograms generated for each of multiple vascular images) and the vascular distribution histogram generated for the target vascular image. In this case, the vascular distribution characteristics for each individual analysis target are shown in comparison with the vascular distribution characteristics in the population of the aggregated histogram. Therefore, the vascular distribution characteristics for each individual analysis target can be grasped even more easily. As mentioned above, the fundus image processing device may obtain aggregated histograms that have been pre-generated by a medical device manufacturer or research facility.

[0057] The population of the aggregated histogram used for comparison (comparative display or difference retrieval) with the vascular distribution histogram being analyzed can be selected as appropriate. For example, the comparison population may include all subjects and eyes examined. In this case, the user can compare the characteristics of the vascular distribution in the vascular image being analyzed with the characteristics of the average vascular distribution based on the output information. Alternatively, the comparison population may be a population of subjects and eyes examined that meet specific conditions (for example, a population of diabetic patients who have not developed diabetic retinopathy). In this case, the user can appropriately determine, based on the output information, whether the characteristics of the vascular distribution in the vascular image being analyzed approximate the characteristics of the vascular distribution in a population that meets the specific conditions.

[0058] Furthermore, the control unit may display the generated difference histogram directly on the display unit. The control unit may also output information based on the difference shown by the difference histogram (i.e., the difference between the characteristics of the vascular distribution of the target of analysis and the characteristics of the vascular distribution in the population of the aggregated histogram). For example, the control unit may output the difference shown by the difference histogram as a numerical value. The control unit may also output the degree of similarity between the characteristics of the vascular distribution of the target of analysis and the characteristics of the vascular distribution in the population of the aggregated histogram based on the difference shown by the difference histogram. In this case, the user can more easily determine whether the characteristics of the vascular distribution in the vascular image of the target of analysis are similar to the characteristics of the vascular distribution in a population that satisfies specific conditions.

[0059] A mathematical model that outputs information about a specific disease may be constructed by training a machine learning algorithm using multiple training datasets, each of which uses vascular distribution feature information as input training data and information indicating the presence or absence of a specific disease in the subject from whom the input training data was obtained as output training data. The control unit may further execute a disease information acquisition step by inputting the vascular distribution feature information of the target vascular image generated in the feature information generation step into the mathematical model, thereby acquiring information about a specific disease in the target subject (for example, the probability of having a specific disease) output by the mathematical model.

[0060] As mentioned above, vascular distribution feature information reflects the characteristics of the vascular distribution in the target vascular image. Furthermore, depending on the type of disease, the characteristics of the vascular distribution in the target vascular image often change depending on the disease state of the subject (target subject) from whom the target vascular image was acquired. Therefore, by inputting the vascular distribution feature information of the target subject into a mathematical model pre-trained with a training dataset containing vascular distribution feature information from multiple subjects, information regarding a specific disease can be appropriately obtained.

[0061] As mentioned above, the vascular distribution feature information (input training data) for training the mathematical model, and the vascular distribution feature information input to the mathematical model, can use at least one of a vascular distribution histogram and a target vascular feature map. Furthermore, the output training data may include information indicating the presence or absence of diabetes in the subjects from whom the input training data was obtained. The mathematical model may output information regarding diabetes in the target subjects (e.g., the probability of having diabetes) when the vascular distribution feature information of the target vascular images is input. In this case, information regarding diabetes in the target subjects can be obtained with high accuracy. However, as mentioned above, it is also possible to use information indicating the presence or absence of diseases other than those related to diabetes as output training data. Additionally, the vascular distribution feature information (input training data) for training the mathematical model, and the vascular distribution feature information input to the mathematical model, may include both arterial and venous vascular distribution feature information.

[0062] <Embodiment> (Device configuration) Hereinafter, one typical embodiment of the present disclosure will be described with reference to the drawings. As shown in Figure 1, in this embodiment, a fundus image processing device 1, a fundus image processing device 21, and fundus image acquisition devices 11A and 11B are used. The fundus image processing device 1 generates or constructs a retinal vascular distribution probability map, an aggregated histogram, and a mathematical model (details of which will be described later) that outputs information about a specific disease, based on a plurality of fundus images. The fundus image processing program for generating or constructing the retinal vascular distribution probability map, aggregated histogram, and mathematical model, etc., is stored, for example, in the storage device 4 of the fundus image processing device 1. Furthermore, the fundus image processing device 21 generates vascular distribution feature information that shows the characteristics of the vascular distribution of the vascular image to be analyzed (target vascular image) based on the retinal vascular distribution probability map and aggregated histogram, etc., generated by the fundus image processing device 1. In addition, the fundus image processing device 21 can also obtain information about a specific disease by inputting the vascular distribution information into the mathematical model constructed by the fundus image processing device 1. The fundus image processing program for generating vascular distribution feature information, etc., is stored, for example, in the storage device 24 of the fundus image processing device 21. The fundus image processing device 21 can also generate retinal vascular distribution probability maps and aggregated histograms. Furthermore, the fundus image processing device 1 can also generate vascular distribution feature information. Fundus image acquisition devices 11A and 11B capture fundus images of the eye under examination.

[0063] As an example, a personal computer (hereinafter referred to as "PC") is used as the fundus image processing device 1,21 in this embodiment. However, the device that can function as the fundus image processing device 1,21 is not limited to a PC. For example, a fundus image acquisition device 11A,11B or a server may function as the fundus image processing device 1,21. When the fundus image acquisition device 11A,11B functions as the fundus image processing device 1,21, the fundus image acquisition device 11A,11B can acquire fundus images and generate at least one of the following based on the acquired fundus images: a retinal vascular distribution probability map, an aggregated histogram, and vascular distribution feature information. Furthermore, the control units of multiple devices (for example, the CPU of the PC and the CPU of the fundus image acquisition device) may cooperate to perform the fundus image processing described later.

[0064] Furthermore, this embodiment illustrates the case where a CPU is used as an example of a controller that performs various processing tasks. However, it goes without saying that controllers other than the CPU may be used in at least some of the various devices. For example, processing speed may be increased by adopting a GPU as the controller.

[0065] The fundus image processing device 1 will now be described. The fundus image processing device 1 is installed, for example, by a manufacturer that provides the fundus image processing device 21 or fundus image processing program to a user, or in various research facilities (e.g., a university hospital). The fundus image processing device 1 is equipped with a control unit 2 that performs various control processing and a communication interface 5. The control unit 2 is equipped with a CPU 3, which is a controller that manages the control, and a storage device 4 that can store programs and data. The storage device 4 stores a fundus image processing program for executing fundus image processing (see Figure 3), which will be described later. The communication interface 5 connects the fundus image processing device 1 to other devices (e.g., a fundus image acquisition device 11A and a fundus image processing device 21, etc.).

[0066] The fundus image processing device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by the user to input various instructions to the fundus image processing device 1. The operation unit 7 can use at least one of the following: a keyboard, mouse, touch panel, etc. A microphone or the like may be used together with the operation unit 7, or in place of the operation unit 7, to input various instructions. The display device 8 displays various images. The display device 8 can use various devices capable of displaying images (for example, at least one of a monitor, display, projector, etc.).

[0067] The fundus image processing device 1 can acquire fundus image data (hereinafter sometimes simply referred to as "fundus image") from the fundus image acquisition device 11A. The fundus image processing device 1 may acquire fundus image data from the fundus image acquisition device 11A by, for example, wired communication, wireless communication, or a removable storage medium (e.g., a USB memory).

[0068] The fundus image processing device 21 will now be described. The fundus image processing device 21 is installed, for example, in a facility that performs diagnosis or examinations of a subject (e.g., a hospital or health checkup facility). The fundus image processing device 21 includes a control unit 22 that performs various control processing and a communication interface 25. The control unit 22 includes a CPU 23, which is a controller that manages the control, and a storage device 24 that can store programs and data. The storage device 24 stores a fundus image processing program for executing fundus image processing (see Figure 11), which will be described later. The communication interface 25 connects the fundus image processing device 21 to other devices (e.g., fundus image acquisition device 11B and fundus image processing device 1, etc.).

[0069] The fundus image processing device 21 is connected to the operation unit 27 and the display device 28. Various devices can be used in the operation unit 27 and the display device 28, as with the operation unit 7 and the display device 8 described above.

[0070] The fundus image processing device 21 can acquire fundus images from the fundus image acquisition device 11B. The fundus image processing device 21 can also acquire at least one of the retinal vascular distribution probability map and the aggregated histogram generated by the fundus image processing device 1. The fundus image processing device 21 may acquire the fundus images and the retinal vascular distribution probability map, etc., by at least one of the following methods: wired communication, wireless communication, or a removable storage medium (e.g., a USB memory stick).

[0071] The fundus imaging device 11 (11A, 11B) will now be described. Various devices for capturing images of the fundus of the eye under examination can be used as the fundus imaging device 11. As an example, the fundus imaging device 11 used in this embodiment is a fundus camera capable of capturing a two-dimensional color frontal image of the fundus using visible light. Therefore, the vascular image acquisition process described later can be performed appropriately based on the color fundus image. However, devices other than a fundus camera (for example, at least one of an OCT device, a laser scanning ophthalmoscope (SLO), etc.) may be used as the fundus imaging device. The fundus image may be a two-dimensional frontal image of the fundus taken from the front of the eye under examination, or a three-dimensional image of the fundus.

[0072] The fundus image acquisition device 11 comprises a control unit 12 (12A, 12B) that performs various control processing and a fundus image acquisition unit 16 (16A, 16B). The control unit 12 comprises a CPU 13 (13A, 13B) which is a controller that manages the operation, and a storage device 14 (14A, 14B) that can store programs and data. The fundus image acquisition unit 16 comprises optical elements for capturing fundus images of the eye under examination. When the fundus image acquisition device 11 performs at least a part of the fundus image processing described later (see Figures 3 and 11), it goes without saying that at least a part of the fundus image processing program for performing the fundus image processing is stored in the storage device 14.

[0073] Referring to Figure 2, an example of how the fundus image processing device 1,21 of this embodiment acquires a vascular image will be described. A vascular image is an image showing the blood vessels included in a fundus image. The fundus image processing device 1,21 of this embodiment acquires a vascular image showing the blood vessels in the input fundus image by inputting the fundus image into a mathematical model trained by a machine learning algorithm. Furthermore, the fundus image processing device 1,21 of this embodiment identifies the position of the optic nerve head (hereinafter sometimes simply referred to as "optic nerve head") (specifically, the centroid position of the optic nerve head) in the input fundus image by inputting the fundus image into a mathematical model trained by a machine learning algorithm. The mathematical model is pre-trained to output a vascular image and the position of the optic nerve head for the input fundus image when a fundus image is input. Note that the mathematical model that outputs the vascular image and the mathematical model that outputs the position of the optic nerve head may be constructed separately.

[0074] The mathematical model is built to output vascular images and papilla locations by being trained on a training dataset. The training dataset includes input data (input training data) and output data (output training data).

[0075] Figure 2 shows an example of a fundus image 30 and vascular images 40 (40A, 40B). In this embodiment, the fundus image 30, which is a two-dimensional color frontal image taken by a fundus image acquisition device (fundus camera in this embodiment) 11A, is used as input training data. In this embodiment, the image region of the fundus image 30 used as input training data includes both the optic disc 31 and the macula 32 of the eye under examination. In addition, the vascular images 40A and 40B, which are images showing at least one of the arteries and veins in the fundus image 30 used as input training data, and information indicating the position of the optic disc (in this embodiment, the centroid position G of the optic disc) are used as output training data. In this embodiment, the vascular images 40A of the arteries and 40B of the veins in the fundus image 30 are included in the output training data. Therefore, when the constructed mathematical model receives the fundus image 30 as input, it can output the vascular images 40A of the arteries and 40B of the veins contained in the input fundus image 30. The output training data (i.e., vascular images 40A, 40B and information indicating the position of the optic disc) may be generated, for example, in response to instructions entered by an operator who has reviewed the fundus image 30, which is the input training data.

[0076] (First Embodiment) Referring to Figures 3 to 10, the fundus image processing performed by the fundus image processing device 1 of the first embodiment will be described. As mentioned above, the fundus image processing device 1 generates a retinal vascular distribution probability map and an aggregated histogram, etc., based on a plurality of fundus images 30. Furthermore, the fundus image processing device 1 constructs a mathematical model that outputs information about a specific disease by training a mathematical model using a machine learning algorithm. The fundus image processing illustrated in Figure 3 is executed by the CPU 3 of the fundus image processing device 1 according to the fundus image processing program stored in the storage device 4.

[0077] As shown in Figure 3, the CPU 3 acquires multiple fundus images 30 (see Figure 2), which are two-dimensional color frontal images, captured by the fundus image acquisition device (fundus camera in this embodiment) 11A (S11).

[0078] Next, CPU3 acquires vascular images 40A and 40B showing at least one of the arteries and veins contained in each fundus image 30 acquired in S1 (S2). In this embodiment, CPU3 inputs the fundus image 30 into a mathematical model trained by a machine learning algorithm and acquires the vascular images 40A and 40B output by the mathematical model. Therefore, vascular images 40A and 40B showing the blood vessels in the fundus image 30 with high accuracy are easily acquired. In addition, in this embodiment, the vascular images 40A of the arteries and the vascular images 40B of the veins contained in each fundus image 30 are acquired separately.

[0079] Furthermore, processing may be performed to make the areas of the fundus visible in multiple fundus images 30 or multiple vascular images 40A, 40B more uniform. For example, depending on the magnification at which the fundus image 30 was taken, the width of the area of ​​the image, which changes with the magnification, may be made more uniform. Alternatively, the areas of the fundus visible in the images may be made more uniform by extracting images within a specific range from the fundus image 30 or vascular images 40A, 40B. The processing to make the areas of the fundus more uniform may be performed according to instructions entered by the operator (i.e., manually) or may be performed automatically by the CPU 3.

[0080] CPU3 identifies the position of the optic disc (in this embodiment, the centroid of the optic disc) in each of the vascular images 40A and 40B acquired in S2 (S3). In this embodiment, CPU3 inputs the fundus image 30 into a mathematical model trained by a machine learning algorithm, and obtains the position of the optic disc in the fundus image 30 output by the mathematical model. Therefore, the position of the optic disc in the fundus image 30 and the vascular images 40A and 40B can be identified with high accuracy.

[0081] CPU3 generates retinal vascular distribution probability maps 50A and 50B (see Figure 4) (S4) by adding together multiple vascular images 40A and 40B acquired in S2 after alignment, showing the distribution of the probability of the presence of blood vessels in the retina of the eye being examined. According to the retinal vascular distribution probability maps 50A and 50B, the two-dimensional distribution of the probability of the presence of retinal blood vessels within the set (population) of multiple eyes from which fundus images 30 were taken is appropriately shown. Therefore, according to the retinal vascular distribution probability maps 50A and 50B, information on a wide range of blood vessels in the fundus images is appropriately presented. Note that the general structure of retinal blood vessels tends to be uniform regardless of the eye being examined. Therefore, the correlation between the probability of the presence of blood vessels in each region on the retinal vascular distribution probability maps 50A and 50B and the thickness of the blood vessels in each region in the population is high. Therefore, according to the retinal vascular distribution probability maps 50A and 50B, it is easy to understand the state of blood vessel thickness for each region.

[0082] In S4, CPU3 generates retinal vascular distribution probability maps 50A and 50B by adding and averaging multiple aligned vascular images 40A and 40B. Therefore, the retinal vascular distribution probability maps 50A and 50B can be handled in the same units as the values ​​of each pixel (e.g., brightness values) in each of the vascular images 40A and 40B.

[0083] In S4, CPU3 generates an arterial retinal vascular distribution probability map 50A by adding vascular images 40A of multiple arteries, and also generates a venous retinal vascular distribution probability map 50B by adding vascular images 40B of multiple veins. Depending on the patient's disease or other condition, different changes may appear between the arteries and veins in the fundus. Therefore, generating both the arterial retinal vascular distribution probability map 50A and the venous retinal vascular distribution probability map 50B makes it easier to obtain more useful information.

[0084] In S4, the positions of the multiple vascular images 40A and 40B are aligned based on the position of the optic disc in each vascular image 40A and 40B identified in S3, and then the multiple vascular images 40A and 40B are added together. The structure of the retinal blood vessels at the fundus of the eye is such that they spread outward from the optic disc. The general structure of the retinal blood vessels spreading from the optic disc tends to be uniform regardless of the eye being examined. Therefore, when the retinal vascular distribution probability maps 50A and 50B are generated, the alignment of the multiple vascular images 40A and 40B based on the position of the optic disc further improves the accuracy of the distribution of the probability of the presence of blood vessels shown in the retinal vascular distribution probability maps 50A and 50B.

[0085] In detail, in S4, the multiple vascular images 40A and 40B are aligned based on the centroid position of the optic disc in each vascular image 40A and 40B identified in S3. As a result, the multiple vascular images 40A and 40B are aligned based on a single centroid position, further improving the accuracy of the generated retinal vascular distribution probability maps 50A and 50B.

[0086] Furthermore, in S4, the CPU 3 may automatically perform alignment of multiple vascular images 40A and 40B. In this case, the amount of work required by the operator to generate the retinal vascular distribution probability maps 50A and 50B is appropriately reduced. Alternatively, the operator may input instructions for aligning multiple vascular images 40A and 50B to the fundus image processing device 1 via the operation unit 7. The CPU 3 may perform alignment of multiple vascular images 40A and 40B in accordance with the instructions input via the operation unit 7. Even in this case, the retinal vascular distribution probability maps 50A and 50B are appropriately generated.

[0087] Figure 4 shows an example of an arterial retinal vascular distribution probability map 50A and a venous retinal vascular distribution probability map 50B displayed on the display device 8. In this embodiment, when the CPU 3 displays the retinal vascular distribution probability maps 50A and 50B on the display device 8, it displays the brightness value of each pixel in the two-dimensional retinal vascular distribution probability maps 50A and 50B (in this embodiment, 256 brightness levels from 0 to 255) represented by color and intensity (i.e., displayed as a heat map). In the example shown in Figure 4, the higher the brightness value of a pixel, the darker the warm color, and the lower the brightness value of a pixel, the darker the cool color. By displaying the heat-mapped retinal vascular distribution probability maps 50A and 50B, the two-dimensional distribution of the probability of blood vessel existence becomes easier to grasp more appropriately.

[0088] CPU3 generates target vascular feature maps 51A and 51B (see Figure 5), which are maps showing the characteristics of the vascular distribution in the vascular images to be analyzed (S5). Specifically, CPU3 acquires target vascular images, which are the vascular images 40A and 40B to be analyzed. In this embodiment, target vascular images of arteries and veins of the fundus to be analyzed are acquired. The target vascular images may be acquired in the same way as the method used in S2 described above, or in a different way than the method used in S2. CPU3 generates target vascular feature maps 51A and 51B from the retinal vascular distribution probability maps 50A and 50B, which correspond to the vascular regions of the target vascular images. In this embodiment, CPU3 generates the target vascular feature map 51A of arteries from the retinal vascular distribution probability map 50A, which corresponds to the vascular regions of the arterial target vascular images. Also, CPU3 generates the target vascular feature map 51B of veins from the retinal vascular distribution probability map 50B, which corresponds to the vascular regions of the vein target vascular images.

[0089] As shown in Figure 5, the target vascular feature maps 51A and 51B show the characteristics of the vascular distribution in the target vascular image. For example, according to the target vascular feature maps 51A and 51B, the probability of the presence of blood vessels in the region of the vascular area shown in the target vascular image can be appropriately grasped on the image. Furthermore, as mentioned above, the correlation between the probability of the presence of blood vessels in each region on the retinal vascular distribution probability maps 50A and 50B and the thickness of blood vessels in each region in the population is high. Therefore, according to the target vascular feature maps 51A and 51B, it is easy to understand, for example, cases where thick blood vessels are present in a region where thin blood vessels are likely to be present in the population, or where thin blood vessels are present in a region where thick blood vessels are likely to be present in the population. In summary, the characteristics of the blood vessels shown in the target vascular image can be easily grasped according to the target vascular feature maps 51A and 51B.

[0090] In S5, CPU3 generates target vessel feature maps 51A and 51B after aligning the retinal vessel distribution probability maps 50A and 50B with the target vessel image. Various methods described above (for example, a method based on the position of the optic disc) can be used for alignment. Alignment may be performed automatically or manually by an operator. CPU3 may also generate target vessel feature maps 51A and 51B by extracting regions corresponding to the vessel regions of the target vessel image from the retinal vessel distribution probability maps 50A and 50B. Alternatively, CPU3 may generate target vessel feature maps 51A and 51B by masking regions other than the vessel regions of the target vessel image from the retinal vessel distribution probability maps 50A and 50B.

[0091] CPU3 generates vascular distribution histograms 52A and 52B (see Figure 5), which are histograms showing the characteristics of the vascular distribution in the vascular images to be analyzed (S6). Specifically, CPU3 acquires target vascular images 40A and 40B, which are the vascular images to be analyzed, similar to S5. In this embodiment, target vascular images of arteries and target vascular images of veins in the fundus to be analyzed are acquired. Based on the information of the vascular regions in the target vascular images from the retinal vascular distribution probability maps 50A and 50B, CPU3 generates histograms as vascular distribution histograms 52A and 52B, which show the number of pixels within the vascular regions of the target vascular images according to the probability of existence of the blood vessels indicated by the retinal vascular distribution probability maps 50A and 50B. Specifically, in this embodiment, the range from the lowest to the highest probability of existence of blood vessels in the retinal vascular distribution probability maps 50A and 50B is divided into 256 steps according to the brightness value. In this case, each pixel within the blood vessel region of the target blood vessel image will belong to one of 256 areas (hereinafter referred to as "probability areas") that are divided in order of the probability of blood vessel existence. CPU3 creates a histogram of the number of pixels belonging to each probability area. In this embodiment, CPU3 generates an arterial blood vessel distribution histogram 52A from the arterial retinal blood vessel distribution probability map 50A based on the information of the blood vessel region in the arterial target blood vessel image. CPU3 also generates a venous blood vessel distribution histogram 52B from the venous retinal blood vessel distribution probability map 50B based on the information of the blood vessel region in the venous target blood vessel image.

[0092] In the vascular distribution histograms 52A and 52B shown in Figure 5, the horizontal axis represents the probability of blood vessels being present in the retinal vascular distribution probability maps 50A and 50B (i.e., multiple probability areas), and the vertical axis represents the number of pixels belonging to each probability area among the multiple pixels present within the vascular region of the target vascular image. The vascular distribution histograms 52A and 52B clearly show the characteristics of the vascular distribution in the target vascular image. In other words, the vascular distribution histograms 52A and 52B appropriately capture the distribution of the number of pixels belonging to each of the 256 probability areas for multiple pixels present within the vascular region of the target vascular image. Therefore, the vascular distribution histograms 52A and 52B also provide useful information that could not be obtained with previous analysis methods.

[0093] In S6, CPU3 generates vascular distribution histograms 52A and 52B after the retinal vascular distribution probability maps 50A and 50B have been aligned with the target vascular image. Various methods described above (for example, a method based on the position of the optic disc) can be used for alignment. Alignment may be performed automatically or manually by an operator.

[0094] CPU3 generates an aggregated histogram (S7) by aggregating the multiple vascular distribution histograms 52A and 52B that are selected for aggregation from the multiple vascular distribution histograms 52A and 52B generated for multiple target vascular images in S6. In this embodiment, CPU3 generates an arterial aggregated histogram by aggregating the vascular distribution histograms 52A of multiple arteries. CPU3 also generates a venous aggregated histogram by aggregating the vascular distribution histograms 52A of multiple veins. In this embodiment, CPU3 generates an aggregated histogram by averaging the aggregation results of the multiple vascular distribution histograms 52A and 52B. The age-based aggregated histogram (see Figures 6 and 7) described later is an example of an aggregated histogram. CPU3 can generate various aggregated histograms by appropriately setting the population to be aggregated. For example, as described later, the population may be set by age, or a population with specific conditions regarding diseases, etc., may be set. The aggregated histogram allows for an accurate understanding of the characteristics of the vascular distribution within the population being aggregated.

[0095] There are various ways to use the aggregated histogram. For example, the CPU 3 may display the generated aggregated histogram on the display device 8 to allow the user to understand the characteristics of the vascular distribution for the population being aggregated. The CPU 3 also generates a difference histogram, which is the difference between the vascular distribution histograms 52A and 52B generated for each target vascular image and the aggregated histogram. This process will be explained in detail in the fundus image processing of the second embodiment (see Figure 11).

[0096] CPU3 generates age-specific aggregated histograms (see Figures 6 and 7) (S8). Specifically, CPU3 aggregates the multiple vascular distribution histograms 52A and 52B generated in S6 for multiple target vascular images, according to the age group to which the subject being analyzed belongs (in this embodiment, 20s, 30s, 40s, 50s, 60s, and 70s), and generates age-specific aggregated histograms by averaging the aggregated results for each age group. In this embodiment, CPU3 generates age-specific aggregated histograms for arteries (see Figure 6) by aggregating the arterial vascular distribution histogram 52A by age group. CPU3 also generates age-specific aggregated histograms for veins (see Figure 7) by aggregating the venous vascular distribution histogram 52A by age group.

[0097] In the aggregated histograms (age-based aggregated histograms) shown in Figures 6 and 7, the horizontal axis represents the probability of blood vessels being present in the retinal blood vessel distribution probability maps 50A and 50B (i.e., multiple probability areas), and the vertical axis represents the average number of pixels belonging to each probability area among the multiple pixels present within the blood vessel region of the target blood vessel image. As shown in Figures 6 and 7, age-based aggregated histograms allow for an appropriate understanding of the characteristics of blood vessel distribution by age. The age-based aggregated histograms shown in Figures 6 and 7 were generated without specifying the conditions of the subjects being aggregated. Referring to Figures 6 and 7, it can be seen that as the age increases (as the age of the subject increases), the number of blood vessels decreases in almost all probability areas, both for arteries and veins. In other words, there is a high possibility that there is a high correlation between the age progression of the subject and the decrease in the retinal blood vessels of the subject.

[0098] Therefore, when CPU3 aggregates multiple vascular distribution histograms 52A and 52B generated for multiple subjects of different age groups (for example, when performing the S7 process described above and the S9 process described later), it aggregates the multiple vascular distribution histograms 52A and 52B after correcting for the effect of changes in the distribution of the probability of blood vessel existence due to the age of each subject being analyzed. As an example, in this embodiment, CPU3 corrects for the effect of changes due to age by performing regression analysis using a regression equation with age as the explanatory variable and probability of existence as the dependent variable for each probability area of ​​existence divided in order of the probability of blood vessel existence. As a result, it is possible to grasp the characteristics of the vascular distribution with aggregated histograms in which the effect of changes due to age has been suppressed. The process of correcting for the effect of changes in the distribution of the probability of blood vessel existence due to age is applied to both the aggregation process of the arterial vascular distribution histogram 52A and the aggregation process of the venous vascular distribution histogram 52B.

[0099] CPU3 generates multiple aggregated histograms for each population (S9). Specifically, for each of the multiple populations to which the target vascular images belong, CPU3 aggregates the multiple vascular distribution histograms 52A and 52B generated for the target vascular images in S6, and averages the aggregated results for each population. As a result, aggregated histograms are generated for each population. By generating aggregated histograms for each population, the characteristics of the vascular distribution in the fundus of each population can be appropriately grasped. Note that in S9, the aggregation process for the arterial vascular distribution histogram 52A and the aggregation process for the venous vascular distribution histogram 52B are executed.

[0100] CPU3 generates a difference histogram (see Figures 8 and 9) in S9, showing the difference between multiple aggregated histograms generated for each population (S10). In S10, difference histograms for arteries and veins are generated. The difference histograms generated in S10 allow for an appropriate understanding of the characteristics of the vascular distribution for each population. For example, by determining the population according to various conditions such as the presence or absence of disease in the subjects, or the state of the disease, useful information that could not be obtained with previous analysis methods can be obtained.

[0101] Figures 8 and 9 are examples of difference histograms for two different populations. The difference histograms shown in Figures 8 and 9 were generated by setting two populations, a DM group and a control group, and then performing the generation of aggregated histograms for each population (S9), and the generation of difference histograms for each of the two aggregated histograms (S10). The DM group is a population of diabetic patients who have not developed diabetic retinopathy (number of relevant individuals = 495). The control group is a population obtained by excluding the subjects and eyes of the DM group, which is the comparison target, from all subjects and eyes (number of relevant individuals = 10460) (number of relevant individuals = 9965). The aggregated histogram used as the comparison target for generating the difference histograms for the DM group and the control group is an aggregated histogram with all subjects and eyes (number of relevant individuals = 10460) as the population. Figure 8 is the difference histogram for arteries, and Figure 9 is the difference histogram for veins. Figure 10 shows map 53A, which overlays the differential histogram data for the arteries of the DM group shown in Figure 8 onto the fundus image 30, and map 53B, which overlays the differential histogram data for the veins of the DM group shown in Figure 9 onto the fundus image 30.

[0102] As shown in the difference histogram in Figure 8 and map 53A in Figure 10, focusing on the arteries of the DM group (the population of diabetic patients who have not developed diabetic retinopathy), the brightness (number of pixels) increased compared to the control group in the area around the main vessels where the probability of blood vessels is originally high (probability area 48-84), but decreased compared to the control group in the area where the probability of blood vessels is low (probability area 0-46). Furthermore, as shown in the difference histogram in Figure 9 and map 53B in Figure 10, the brightness (number of pixels) decreased in the veins of the DM group compared to the control group in all areas, but the decrease was particularly large in areas where the probability of retinal blood vessels is sparse, including around the fovea. The difference histogram between the vascular distribution histogram of the examined eye and the aggregated histogram of the comparison group (for example, an aggregated histogram with all examined subjects / eyes as the population) may be similar to the difference histogram of the DM group shown in Figures 8 and 9, potentially allowing for the prediction of the risk of developing diabetic retinopathy before its onset.

[0103] CPU3 constructs a mathematical model that outputs information about a specific disease by training the mathematical model using a machine learning algorithm (S11). In S11, the mathematical model is constructed by training the mathematical model with multiple training datasets. Each training dataset includes input data (input training data) and output data (output training data). Specifically, the input training data uses vascular distribution histograms 52A and 52B. The vascular distribution histograms 52A and 52B are examples of vascular distribution feature information that shows the characteristics of the vascular distribution in a specific vascular image. The vascular distribution feature information is generated by processing the information of the regions corresponding to the vascular regions in a specific vascular image from the retinal vascular distribution probability maps 50A and 50B. The output training data uses information indicating the presence or absence of a specific disease (in this embodiment, the presence or absence of diabetes) in the subject from whom the input training data was acquired. The information indicating the presence or absence of a specific disease may be generated, for example, by an operator operating the control unit 7. The mathematical model is trained to output information about a specific disease in the subject (in this embodiment, the probability that the subject has diabetes) when the subject's vascular distribution histograms 52A and 52B are input. The program that implements the mathematical model constructed by the fundus image processing device 1 is incorporated into the fundus image processing device 21.

[0104] The vascular distribution histograms 52A and 52B represent the characteristics of the vascular distribution in the target vascular images. Furthermore, the characteristics of the vascular distribution in the target vascular images change depending on whether or not the subject has diabetes. Therefore, by inputting the vascular distribution histograms 52A and 52B of the target subject into a mathematical model trained with a training dataset that includes vascular distribution histograms 52A and 52B, the probability that the target subject has diabetes can be appropriately obtained.

[0105] In this embodiment, both the arterial vascular distribution histogram 52A and the venous vascular distribution histogram 52B are used as input training data for training the mathematical model and as input to the constructed mathematical model. Therefore, it becomes easier to obtain disease information with higher accuracy.

[0106] Furthermore, in this embodiment, the training data for training the mathematical model includes, in addition to the vascular distribution histograms 52A and 52B, information on the age and sex of the subjects from whom the vascular distribution histograms 52A and 52B were obtained. Vascular distribution feature information such as vascular distribution histograms 52A and 52B often changes depending on at least one of the subject's age and sex. Therefore, by including information on at least one of age and sex in the input training data, it becomes easier for the mathematical model to output appropriate information according to the age and sex of the target subject. The constructed mathematical model is input not only to the vascular distribution histograms 52A and 52B, but also to the age and sex of the subjects from whom the vascular distribution histograms 52A and 52B were obtained.

[0107] As an example, in this embodiment, a neural network using five fully connected layers is trained on a training dataset using 5-fold cross-validation. However, it goes without saying that the algorithm can be selected as appropriate.

[0108] The inventor of the present invention conducted a trial to evaluate the usefulness of the constructed mathematical model. In this trial, the inventor obtained the probability that each subject had diabetes by inputting the vascular distribution histogram, age, and sex of several subjects with diabetes (diabetes group) into the constructed mathematical model. Similarly, the inventor obtained the probability that each subject had diabetes by inputting the vascular distribution histogram, age, and sex of several subjects without diabetes (normal group) into the constructed mathematical model. The inventor compared the difference between the probabilities obtained for the diabetes group and the probabilities obtained for the normal group using a t-test, one of the hypothesis testing methods.

[0109] As a result, the average probability obtained for the diabetes group was "0.57 ± 0.080", and the average probability obtained for the normal group was "0.53 ± 0.075", showing a statistically significant difference.

[0110] (Second Embodiment) Referring to Figure 11, the fundus image processing performed by the fundus image processing device 21 of the second embodiment will be described. As mentioned above, the fundus image processing device 21 of the second embodiment acquires the retinal vascular distribution probability maps 50A, 50B and aggregated histogram generated by the fundus image processing device 1 of the first embodiment. Based on the retinal vascular distribution probability maps 50A, 50B and aggregated histogram generated by the fundus image processing device 1, the fundus image processing device 21 of the second embodiment generates vascular distribution feature information that shows the characteristics of the vascular distribution of the vascular image to be analyzed (target vascular image). In the second embodiment, as vascular distribution feature information, for example, target vascular feature maps 51A, 51B (see Figure 5), vascular distribution histograms 52A, 52B (see Figure 5), and difference histograms (see Figures 8 and 9) are generated. Furthermore, a program for realizing the mathematical model constructed by the fundus image processing device 1 of the first embodiment is incorporated into the fundus image processing device 21 of the second embodiment. In the second embodiment, the vascular distribution histograms 52A and 52B generated based on the fundus image to be analyzed are input into a mathematical model to obtain the probability that the subject being analyzed has diabetes. The fundus image processing illustrated in Figure 11 is executed by the CPU 23 of the fundus image processing device 21 according to the fundus image processing program stored in the storage device 24.

[0111] First, the CPU 23 acquires a fundus image 30 (see Figure 2), which is a two-dimensional color frontal image, taken by the fundus image acquisition device (fundus camera in this embodiment) 11B, as the fundus image to be analyzed (S11). The CPU 23 acquires target vascular images (in this embodiment, target vascular images of arteries and target vascular images of veins) that show at least one of the arteries and veins included in the target vascular image acquired in S11 (S12). The CPU 23 identifies the position of the optic disc (in this embodiment, the centroid position of the optic disc) in the target vascular image (S13). Note that the processing in S11 to S13 can be the same as the processing in S1 to S3 in the first embodiment.

[0112] The CPU 23 generates target vascular feature maps 51A and 51B (see Figure 5), which show the characteristics of the vascular distribution in the target vascular image, and displays them on the display device 28 (S14). As described above, the target vascular feature maps 51A and 51B allow the characteristics of the vascular distribution in the target vascular image to be grasped at a glance. Therefore, the diagnosis of the eye under examination is appropriately assisted. The process of generating the target vascular feature maps 51A and 51B in S14 can be the same as the process in S5 in the first embodiment.

[0113] The CPU 23 generates vascular distribution histograms 52A and 52B (see Figure 5), which are histograms showing the characteristics of the vascular distribution in the target vascular image, and displays them on the display device 28 (S15). As mentioned above, the vascular distribution histograms 52A and 52B clearly show the characteristics of the vascular distribution in the target vascular image. Therefore, the diagnosis of the eye under examination is appropriately assisted. The process for generating the vascular distribution histograms 52A and 52B in S15 can be the same as the process in S6 in the first embodiment.

[0114] In S15, the CPU 23 may display the vascular distribution histograms 52A and 52B generated for the target vascular image and the aggregated histogram generated by the fundus image processing device 1 of the first embodiment (for example, an aggregated histogram of a population of subjects / eyes that meet specific conditions, or an aggregated histogram of a population of all subjects / eyes) on the display device 28 in a comparable manner (for example, side by side). In this case, the user can easily compare the characteristics of the vascular distribution for the population of the aggregated histogram with the characteristics of the vascular distribution of the fundus being analyzed.

[0115] The CPU 23 generates a difference histogram, which is the difference between the vascular distribution histograms 52A and 52B generated in S15 and the aggregated histogram generated by the fundus image processing device 1 of the first embodiment (for example, an aggregated histogram of a population of subjects / eyes that meet specific conditions, or an aggregated histogram of a population of all subjects / eyes), and displays it on the display device 28 (S16). According to the difference histogram generated in S16, the characteristics of the vascular distribution for each analysis target are shown in comparison with the characteristics of the vascular distribution in the population of the aggregated histogram. Therefore, the characteristics of the vascular distribution for each analysis target can be grasped even more easily.

[0116] CPU23 inputs the vascular distribution histograms 52A and 52B of the subject being analyzed (target subject) generated in S15 into the mathematical model constructed in S11 (see Figure 3), thereby obtaining information about a specific disease (in this embodiment, the probability that the target subject has diabetes) output by the mathematical model (S17). As mentioned above, the mathematical model is trained by a machine learning algorithm to output the probability that the subject from whom the vascular distribution histograms 52A and 52B were obtained has diabetes, upon input of the vascular distribution histograms 52A and 52B. The mathematical model is pre-trained using multiple training datasets, with the vascular distribution histograms 52A and 52B as input training data, and information indicating the presence or absence of diabetes in the subject from whom the input training data was obtained as output training data. As mentioned above, the vascular distribution histograms 52A and 52B represent the characteristics of the vascular distribution in the target vascular image. Furthermore, depending on the type of disease, the characteristics of the vascular distribution in the target vascular image often change depending on the diabetes status of the subject from whom the target vascular image was obtained. Therefore, by inputting the vascular distribution histograms 52A and 52B of the target subject into a mathematical model pre-trained with a training dataset containing vascular distribution histograms 52A and 52B of multiple subjects, the probability that the target subject has diabetes can be appropriately obtained.

[0117] In step S11 of this embodiment, both the arterial vascular distribution histogram 52A and the venous vascular distribution histogram 52B are input into the mathematical model. In addition, in step S11 of this embodiment, the age and sex information of the subject from whom the vascular distribution histograms 52A and 52B were obtained are also input into the mathematical model. As a result, the probability of having diabetes can be obtained with higher accuracy.

[0118] The technologies disclosed in the above embodiments are merely examples. Therefore, it is possible to modify the technologies exemplified in the above embodiments. It is also possible to perform only a portion of the multiple processes exemplified in the above embodiments. For example, in the second embodiment, only one or two of the target vascular feature maps 51A, 51B, vascular distribution histograms 52A, 52B, and difference histogram may be output as vascular distribution feature information. Furthermore, in the second embodiment, only the generation process of vascular distribution histograms 52A, 52B (S11-S13, S15) and the process of acquiring specific disease information using a mathematical model (S17) may be performed.

[0119] The process of acquiring a fundus image in S1 of Figure 3 is an example of the "fundus image acquisition step" of the first embodiment. The process of acquiring a vascular image in S2 of Figure 3 is an example of the "vascular image acquisition step" of the first embodiment. The process of generating a retinal vascular distribution probability map in S4 of Figure 3 is an example of the "probability map generation step" of the first embodiment. The process of identifying the position of the optic disc in S3 of Figure 3 is an example of the "optic disc position identification step" of the first embodiment. The processes of acquiring target vascular images in S5 and S6 of Figure 3 are examples of the "target vascular image acquisition step" of the first embodiment. The process of generating a target vascular feature map in S5 of Figure 3 is an example of the "feature map generation step" of the first embodiment. The process of generating a vascular distribution histogram in S6 of Figure 3 is an example of the "vascular distribution histogram generation step" of the first embodiment. The processes of generating aggregated histograms in S7 to S9 of Figure 3 are examples of the "aggregated histogram generation steps" of the first embodiment. The process of generating age-specific aggregated histograms in S8 of Figure 3 is an example of the "age-specific aggregated histogram generation step" of the first embodiment. The process of generating aggregated histograms for each population in S9 of Figure 3 is an example of the "population-specific aggregated histogram generation step" of the first embodiment. The process of generating difference histograms in S10 of Figure 3 is an example of the "difference histogram generation step" of the first embodiment. The processes of acquiring vascular distribution feature information in S5 and S6 of Figure 3 are an example of the "feature information generation step". The process of constructing a mathematical model in S11 of Figure 3 is an example of the "mathematical model construction step".

[0120] The process of acquiring the fundus image to be analyzed in S11 of Figure 11 is an example of the "fundus image acquisition step" of the second embodiment. The process of acquiring the target blood vessel image in S12 of Figure 11 is an example of the "blood vessel image acquisition step" of the second embodiment. The process of generating blood vessel distribution feature information in S14 to S16 of Figure 11 is an example of the "feature information generation step" of the second embodiment. The process of acquiring disease information (the probability of having diabetes in the above embodiment) in S17 of Figure 11 is an example of the "disease information acquisition step". [Explanation of Symbols]

[0121] 1. Fundus image processing device 3 CPU 4 Storage device 11 (11A, 11B) Fundus imaging device 21 Fundus Image Processing Device 23 CPU 24 Storage device 30 Fundus image 40A, 40B Vascular images 50A, 50B Retinal vascular distribution probability map 51A, 51B Target blood vessel characteristic map 52A, 52B Vascular distribution histogram 53A, 53B Map

Claims

1. A fundus image processing device for processing fundus images of an eye under examination, The control unit of the fundus image processing device is A fundus image acquisition step involves acquiring multiple fundus images, including the blood vessels of the fundus of the eye being examined, taken using a fundus imaging device. A vascular image acquisition step involves acquiring a vascular image showing at least one of the arteries and veins included in each of the acquired fundus images, A probability map generation step involves generating a retinal vascular distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye being examined by adding together multiple vascular images obtained for each of the multiple fundus images after alignment, A fundus image processing device characterized by performing the following:

2. A fundus image processing apparatus according to claim 1, The control unit, In the vascular image acquisition step, the arterial and venous vascular images included in each of the fundus images are acquired. The fundus image processing apparatus is characterized in that, in the probability map generation step, it generates a probability map of the retinal vascular distribution of arteries by adding up multiple arterial vascular images, and generates a probability map of the retinal vascular distribution of veins by adding up multiple vein vascular images.

3. A fundus image processing apparatus according to claim 1, The control unit, The optic disc position identification step is further performed to identify the position of the optic nerve head in the vascular image acquired in the vascular image acquisition step, The fundus image processing apparatus is characterized in that, in the probability map generation step, the retinal vascular distribution probability map is generated by adding each of the multiple vascular images in a state that is aligned with the position of the identified optic nerve head.

4. A fundus image processing apparatus according to claim 1, The control unit, A target vascular image acquisition step, which involves acquiring a target vascular image, which is the vascular image to be analyzed, A feature map generation step in which a map of the region corresponding to the region of the blood vessel in the target blood vessel image is generated as a target blood vessel feature map from the retinal blood vessel distribution probability map, A fundus image processing device characterized by further performing the following steps.

5. A fundus image processing apparatus according to claim 1, The control unit, A target vascular image acquisition step, which involves acquiring a target vascular image, which is the vascular image to be analyzed, A vascular distribution histogram generation step, in which, based on information of the region in the retinal vascular distribution probability map corresponding to the region of the blood vessel in the target vascular image, generates a vascular distribution histogram that shows the number of pixels within the region of the blood vessel in the target vascular image according to the probability of existence of the blood vessel indicated by the retinal vascular distribution probability map, A fundus image processing device characterized by further performing the following steps.

6. A fundus image processing apparatus according to claim 5, The control unit, A fundus image processing apparatus characterized by further performing an aggregated histogram generation step of generating an aggregated histogram by aggregating the multiple vascular distribution histograms generated for the multiple target vascular images.

7. A fundus image processing apparatus according to claim 6, The control unit, The fundus image processing apparatus is characterized in that, in the step of generating the aggregated histogram, it corrects for the effect of changes in the distribution of the probability of blood vessel presence due to the age of each subject being analyzed, and then aggregates the multiple blood vessel distribution histograms.

8. A fundus image processing apparatus according to claim 6, The control unit, A fundus image processing apparatus characterized by further performing a difference histogram generation step to generate a difference histogram between the vascular distribution histogram generated for each of the target vascular images and the aggregated histogram generated in the aggregated histogram generation step.

9. A fundus image processing apparatus according to claim 5, The control unit, A fundus image processing apparatus characterized by further performing an age-specific aggregated histogram generation step, which generates an age-specific aggregated histogram by aggregating the multiple vascular distribution histograms generated for multiple target vascular images according to the age group to which the subject being analyzed belongs.

10. A fundus image processing apparatus according to claim 5, The control unit, A population-specific aggregated histogram generation step involves aggregating multiple vascular distribution histograms generated for multiple target vascular images belonging to each of the multiple populations to which the target vascular images belong, thereby generating an aggregated histogram for each population. A difference histogram generation step that generates a difference histogram between multiple aggregated histograms generated for each population, A fundus image processing device characterized by further performing the following steps.

11. A fundus image processing apparatus according to claim 10, The fundus image processing apparatus is characterized in that the multiple populations from which the aggregated histogram is generated include a population of diabetic patients who have not developed diabetic retinopathy.

12. A fundus image processing apparatus according to claim 1, A feature information generation step involves generating feature information that indicates the characteristics of the vascular distribution of a specific vascular image by processing information from the retinal vascular distribution probability map that corresponds to the region of a specific vascular image. A mathematical model construction step involves constructing a mathematical model that outputs information about a specific disease by training it using a machine learning algorithm, with multiple training datasets in which the aforementioned vascular distribution feature information is used as input training data and information indicating the presence or absence of a specific disease in subjects from whom the input training data has been acquired is used as output training data. A fundus image processing device characterized by further performing the following steps.

13. A fundus image processing device for processing fundus images of an eye under examination, The control unit of the fundus image processing device is A fundus image acquisition step involves acquiring a fundus image of the eye under examination, including the blood vessels of the fundus, which is to be analyzed, using a fundus imaging device. A vascular image acquisition step involves acquiring a target vascular image which is a vascular image showing at least one of the arteries and veins included in the acquired fundus image, A feature information generation step involves generating a retinal vascular distribution probability map, which shows the distribution of the probability of the presence of blood vessels in the retina of the eye being examined, by processing the information of the region corresponding to the blood vessel region of the target vascular image, thereby generating vascular distribution feature information that shows the characteristics of the vascular distribution of the target vascular image. A fundus image processing device characterized by performing the following:

14. A fundus image processing apparatus according to claim 13, The fundus image processing apparatus is characterized in that, in the feature information generation step, the control unit generates a map of the region corresponding to the region of the blood vessel in the target blood vessel image from the retinal blood vessel distribution probability map as a target blood vessel feature map.

15. A fundus image processing apparatus according to claim 13, The fundus image processing apparatus is characterized in that, in the feature information generation step, the control unit generates a vascular distribution histogram that shows the number of pixels within the vascular region of the target vascular image according to the probability of existence of the vascular shown by the retinal vascular distribution probability map, based on information of the region in the retinal vascular distribution probability map that corresponds to the vascular region of the target vascular image.

16. A fundus image processing apparatus according to claim 13, In the feature information generation step, the control unit, Based on the information of the region corresponding to the blood vessel region in the target blood vessel image within the retinal blood vessel distribution probability map, a blood vessel distribution histogram is generated that shows the number of pixels within the blood vessel region of the target blood vessel image according to the probability of existence of the blood vessels indicated by the retinal blood vessel distribution probability map. A fundus image processing device characterized by generating an aggregated histogram obtained by aggregating multiple vascular distribution histograms generated for each of multiple vascular images, and a difference histogram which is the difference between the vascular distribution histogram generated for the target vascular image.

17. A fundus image processing apparatus according to claim 13, Multiple training datasets are used, where the aforementioned vascular distribution feature information is used as input training data, and information indicating the presence or absence of a specific disease in the subject from whom the input training data was obtained is used as output training data. A mathematical model that outputs information about the specific disease is then trained and constructed using a machine learning algorithm. The control unit, A fundus image processing apparatus characterized by further performing a disease information acquisition step in which information about a specific disease output by the mathematical model is obtained by inputting the vascular distribution feature information generated in the feature information generation step into the mathematical model.

18. A fundus image processing program executed by a fundus image processing device that processes fundus images of an eye under examination, The fundus image processing program is executed by the control unit of the fundus image processing device, A fundus image acquisition step involves acquiring multiple fundus images, including the blood vessels of the fundus of the eye being examined, taken using a fundus imaging device. A vascular image acquisition step involves acquiring a vascular image showing at least one of the arteries and veins included in each of the acquired fundus images, A probability map generation step involves generating a retinal vascular distribution probability map showing the distribution of the probability of the presence of blood vessels in the retina of the eye being examined by adding together multiple vascular images obtained for each of the multiple fundus images after alignment, A fundus image processing program characterized by causing the fundus image processing device to execute the following.

19. A fundus image processing program executed by a fundus image processing device that processes fundus images of an eye under examination, The fundus image processing program is executed by the control unit of the fundus image processing device, A fundus image acquisition step involves acquiring a fundus image of the eye under examination, including the blood vessels of the fundus, which is to be analyzed, using a fundus imaging device. A vascular image acquisition step involves acquiring a target vascular image which is a vascular image showing at least one of the arteries and veins included in the acquired fundus image, A feature information generation step involves generating a retinal vascular distribution probability map, which shows the distribution of the probability of the presence of blood vessels in the retina of the eye being examined, by processing the information of the region corresponding to the blood vessel region of the target vascular image, thereby generating vascular distribution feature information that shows the characteristics of the vascular distribution of the target vascular image. A fundus image processing program characterized by causing the fundus image processing device to execute the following.

Images