Ocular fundus image processing device and ocular fundus image processing program

Abstract

To provide on ocular fundus image processing device and an ocular fundus image processing program capable of more appropriately presenting, to a user, a degree of deviation when a site captured in an ocular fundus image is identified by a mathematical model.SOLUTION: In an image acquisition step, a control part acquires a three-dimensional image of the ocular fundus captured by an ocular fundus image capturing device. In a deviation degree acquisition step, the control part acquires a plurality of probability distributions for identifying each of a plurality of sites in an ocular fundus tissue captured in the three-dimensional image by inputting the three-dimensional image to a mathematical model trained by a machine learning algorithm, and acquires a deviation degree of the acquired probability distribution for the probability distribution when each of the plurality of sites is accurately identified. In a selection display step, the control part causes a display part to selectively display a map indicating a two-dimensional distribution of part of the plurality of acquired deviation degrees.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to a fundus image processing apparatus and a fundus image processing program used for processing fundus images of an eye to be examined.

Background Art

[0002] In recent years, techniques for obtaining medical information from fundus images of an eye to be examined using a mathematical model trained by a machine learning algorithm have been proposed. For example, the image processing apparatus described in Patent Document 1 inputs an ophthalmic image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying tissues in the ophthalmic image. The image processing apparatus obtains the degree of divergence of the obtained probability distribution from the probability distribution when the tissues are accurately identified as structural information indicating the degree of abnormality of the tissue structure.

Prior Art Documents

Non-Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order for a user (such as a doctor, etc.) to more effectively use the degree of divergence for diagnosis, etc., it is desirable to be able to present the degree of divergence to the user in an appropriate manner according to various situations.

[0005] A typical object of the present disclosure is to provide a fundus image processing apparatus and a fundus image processing program capable of more appropriately presenting the degree of divergence when a part shown in a fundus image is identified by a mathematical model to the user.

Means for Solving the Problems

[0006] A typical embodiment of the fundus image processing device provided in this disclosure is a fundus image processing device for processing fundus images of an eye under examination, wherein the control unit of the fundus image processing device performs the following steps: an image acquisition step of acquiring a three-dimensional image of the fundus captured by a fundus image acquisition device; a deviation degree acquisition step of acquiring a plurality of probability distributions for identifying each of a plurality of parts of the fundus tissue shown in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and acquiring the deviation degree of the acquired probability distribution to the probability distribution in which each of the plurality of parts is accurately identified; and a selection display step of selectively displaying on a display unit a map showing the two-dimensional distribution of some of the acquired plurality of deviation degrees.

[0007] A typical embodiment of the fundus image processing program provided in this disclosure is a fundus image processing program executed by a fundus image processing device that processes fundus images of an eye under examination, wherein the fundus image processing program is executed by a control unit of the fundus image processing device, causing the fundus image processing device to perform the following steps: an image acquisition step of acquiring a three-dimensional image of the fundus captured by a fundus image acquisition device; a deviation degree acquisition step of inputting the three-dimensional image to a mathematical model trained by a machine learning algorithm to acquire multiple probability distributions for identifying each of multiple parts of the fundus tissue shown in the three-dimensional image, and acquiring the deviation degree of the acquired probability distribution to the probability distribution in which each of the multiple parts is accurately identified; and a selection display step of selectively displaying a map showing the two-dimensional distribution of some of the acquired deviation degrees on a display unit.

[0008] According to the fundus image processing device and fundus image processing program described herein, the degree of discrepancy when the parts of the fundus image are identified by a mathematical model is presented to the user more appropriately. [Brief explanation of the drawing]

[0009] [Figure 1]This block diagram shows the schematic configuration of the mathematical model building device 101, the fundus image processing device 1, and the OCT devices 10A and 10B. [Figure 2] This is an explanatory diagram illustrating one example of a method for acquiring three-dimensional tomographic images. [Figure 3] This figure shows an example of a two-dimensional tomographic image 42. [Figure 4] This figure shows an example of a three-dimensional tomographic image 43. [Figure 5] This diagram schematically shows the structure of layers and boundaries in the fundus of the eye. [Figure 6] This is a flowchart of the mathematical model construction process performed by the mathematical model construction device 101. [Figure 7] This is a flowchart of the fundus image processing performed by the fundus image processing device 1. [Figure 8] This is a flowchart of the discrepancy acquisition process performed during fundus image processing. [Figure 9] This figure schematically shows the relationship between the two-dimensional tomographic image 42 input to the mathematical model and the one-dimensional regions A1 to AN within the two-dimensional tomographic image 42. [Figure 10] This figure shows an example of multiple deviation maps. [Figure 11] This figure shows an example of displaying a two-dimensional tomographic image and a deviation map. [Figure 12] This figure shows an example of displaying a thickness map and a deviation map. [Modes for carrying out the invention]

[0010] <Overview> The control unit of the fundus image processing device illustrated in this disclosure performs an image acquisition step, a deviation degree acquisition step, and a selection display step. In the image acquisition step, the control unit acquires a three-dimensional image of the fundus captured by a fundus image acquisition device. In the deviation degree acquisition step, the control unit inputs the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire multiple probability distributions for identifying each of multiple parts of the fundus tissue shown in the three-dimensional image, and acquires the deviation degree of the acquired probability distributions from the probability distribution in which each of the multiple parts is accurately identified. In the selection display step, the control unit selectively displays a map showing the two-dimensional distribution of some of the acquired deviation degrees (hereinafter referred to as the "deviation degree map") on the display unit.

[0011] According to the fundus image processing device illustrated in this disclosure, a two-dimensional distribution of some of the degrees of deviation for identifying each of multiple locations in the fundus tissue captured in a three-dimensional image (hereinafter referred to as "degree of deviation for identifying multiple locations") is selectively displayed on the display unit. Therefore, the degrees of deviation for identifying locations are presented to the user more appropriately.

[0012] Let's explain the degree of discrepancy. For example, if there is no abnormality in the tissue structure, the location can be easily identified accurately by the mathematical model, so the obtained probability distribution tends to be biased. On the other hand, if there is an abnormality in the tissue structure, the obtained probability distribution becomes less biased. Therefore, the degree of discrepancy between the probability distribution when the location is accurately identified and the probability distribution actually obtained often increases or decreases depending on the degree of structural abnormality. Thus, users can obtain information useful for diagnosis from the displayed degree of discrepancy map. For example, a user can identify the location on the fundus tissue where a high degree of structural abnormality is estimated to be based on the displayed degree of discrepancy map, and then improve the efficiency of diagnosis by carefully examining the image of the identified location (e.g., a two-dimensional tomographic image).

[0013] The degree of deviation may be output by a mathematical model. Alternatively, the control unit may calculate the degree of deviation based on the probability distribution output by the mathematical model.

[0014] The degree of deviation may include the entropy (average information amount) of the obtained probability distribution. Entropy represents the degree of uncertainty, randomness, and disorder. In the present disclosure, the entropy of the probability distribution output when a site is accurately identified is 0. Also, as the identification of the site becomes more difficult, the entropy increases. Therefore, by using the entropy of the probability distribution as the degree of deviation, the degree of abnormality of the structure of the tissue can be more appropriately quantified. However, a value other than entropy may be adopted as the degree of deviation. For example, at least any one of the standard deviation, coefficient of variation, variance, etc., which indicate the degree of dispersion of the obtained probability distribution, may be used as the degree of deviation. KL divergence, which is a measure for depicting the difference between probability distributions, may be used as the degree of deviation. Also, the maximum value of the obtained probability distribution may be used as the degree of deviation.

[0015] In the present disclosure, a three-dimensional image is composed of a plurality of two-dimensional images arranged side by side. The degree of deviation is obtained for each of the plurality of two-dimensional images constituting the three-dimensional image, or for each pixel, each column, or each row constituting the image. As a result, the degree of deviation when a specific site is identified by a mathematical model is obtained for the entire fundus tissue shown in the three-dimensional image. As an example, in the degree-of-deviation map of the present disclosure, a two-dimensional distribution of the degree of deviation is shown when the fundus tissue is viewed from the front (that is, in the direction along the optical axis of the light for taking the three-dimensional image). However, the direction in which the two-dimensional distribution of the degree of deviation is shown in the degree-of-deviation map can be changed as appropriate.

[0016] In addition, there are multiple layers in the fundus tissue. The degree of deviation obtained in the present disclosure is the degree of deviation when each of a plurality of specific layer·boundaries among the multiple layers in the fundus tissue and the boundaries between adjacent layers (hereinafter, may also be simply referred to as "layer·boundary") is identified by a mathematical model. However, the degree of deviation when parts other than layer·boundaries in the fundus tissue (for example, two or more parts among the optic disc, macula, fovea, fundus blood vessels, and diseased parts) are identified may also be obtained. Further, when displaying a deviation degree map of a specific layer, the control unit may display a two-dimensional distribution such as the average value of the deviation degrees of at least two boundaries included in the specific layer as the deviation degree map of the specific layer.

[0017] In the selection display step, the control unit may selectively display on the display unit the maps of a predetermined number of degrees of deviation with high ranks of the obtained multiple degrees of deviation. In this case, the maps of a predetermined number of degrees of deviation with high ranks are automatically displayed. Therefore, the user can easily confirm the deviation degree map of the part with a high degree of deviation in the identification by the mathematical model among the multiple parts. Thus, the diagnostic efficiency is improved.

[0018] Note that the specific method for displaying the maps of a predetermined number of degrees of deviation with high ranks can be appropriately selected. For example, the control unit may display the maps of a predetermined number of degrees of deviation in descending order of the average value of the degrees of deviation. Also, the control unit may display the maps of a predetermined number of degrees of deviation in descending order of the maximum value of the degrees of deviation in the two-dimensional map. Further, the control unit may display only one map of the degree of deviation with the highest rank, or may display multiple maps of the degrees of deviation in descending order of the ranks. Also, the control unit may display a composite map obtained by aligning and combining multiple maps of the degrees of deviation with high ranks.

[0019] The control unit may further perform a two-dimensional tomographic image display step and a site specification acceptance step. In the two-dimensional tomographic image display step, the control unit extracts a portion of the two-dimensional tomographic images from the three-dimensional image (i.e., the three-dimensional image input to the mathematical model) and displays them on the display unit. The two-dimensional tomographic image is a two-dimensional image that extends in the depth direction of the fundus tissue. In the site specification acceptance step, the control unit receives instruction input from the user to specify at least one of the sites shown in the displayed two-dimensional tomographic image. In the selection display step, if instruction input for specifying a site is received, the control unit may selectively display on the display unit a map of the degree of deviation of the specified site from among multiple sites.

[0020] In this case, the user can view the internal structure of the fundus tissue as seen in the two-dimensional tomographic image and then specify a region of interest to display a divergence map for that region. Therefore, based on the internal state of the fundus tissue as seen in the two-dimensional tomographic image, the user can easily view the divergence map of the appropriate region from among multiple divergence maps. For example, by specifying a region where an abnormality is observed in the two-dimensional tomographic image and viewing the divergence map for that region, the user can easily understand the degree of diffusion and location of the abnormal region in two dimensions. The number of regions that the user can specify (i.e., the regions for which the divergence map will be displayed) can be one or multiple.

[0021] The control unit may further perform a corresponding area display step, which displays the area on which the deviation map is displayed (hereinafter sometimes referred to as the "corresponding area") on the two-dimensional tomographic image. In this case, the user can easily and appropriately identify the area on which the deviation map is displayed on the two-dimensional tomographic image that extends in the depth direction. Therefore, the efficiency of diagnosis is further improved.

[0022] The method for displaying the corresponding area on the two-dimensional tomographic image can be selected as appropriate. For example, the control unit may indicate the corresponding area by changing its color relative to the color of other areas on the two-dimensional tomographic image. The control unit may indicate the corresponding area by displaying a frame surrounding the corresponding area on the two-dimensional tomographic image. The control unit may indicate the corresponding area by displaying a cursor or the like that points to the corresponding area.

[0023] The control unit may further perform an extraction position display step, which displays the location from which the two-dimensional tomographic image displayed on the display unit was extracted on a deviation degree map. In this case, the user can diagnose the fundus tissue of the eye being examined while confirming the extraction position of the displayed two-dimensional tomographic image on the deviation degree map. This makes it easier to make a more accurate diagnosis.

[0024] The method for displaying the extraction position of the two-dimensional tomographic image on the deviation map can be selected as appropriate. For example, the control unit may display lines indicating the extraction position of the two-dimensional tomographic image on a two-dimensional deviation map of the fundus tissue viewed from the front.

[0025] The control unit may further perform an extraction location specification acceptance step and a specified tomographic image display step. In the extraction location specification acceptance step, the control unit receives instruction input from the user to specify the extraction location of the two-dimensional tomographic image on the displayed deviation degree map. In the specified tomographic image display step, when instruction input for specifying the extraction location is received, the control unit extracts a two-dimensional tomographic image extending in the depth direction of the fundus tissue from the specified extraction location and displays it on the display unit. In this case, the user can check the two-dimensional distribution of the degree of deviation for identification of a specific area using the deviation degree map and then directly specify the location of the two-dimensional tomographic image they want to display on the deviation degree map. This makes it easier to perform a more appropriate diagnosis.

[0026] Furthermore, a method can be appropriately selected in which the user specifies the extraction position of the two-dimensional tomography image on the deviation map. For example, the control unit may have the user specify a line on the two-dimensional deviation map that indicates the extraction position of the two-dimensional tomography image (i.e., the position where the extracted two-dimensional tomography image and the deviation map intersect). In this case, the shape of the line may be a straight line or a shape other than a straight line (e.g., curved or ring-shaped). The size and angle of the line may also be changed according to the instructions entered by the user.

[0027] For each of the multiple diseases that may occur in the subject, a corresponding site within the fundus tissue may be associated. In the selection display step, the control unit may selectively display on the display unit the map of the degree of deviation for the site associated with the disease specified by the user from among the multiple degree of deviation obtained. In this case, the user can efficiently use the degree of deviation map for diagnosis according to the disease. For example, the user can easily check the degree of deviation map for the site associated with the expected disease of the subject simply by specifying the expected disease. The user can also predict the subject's disease by switching between specified diseases and checking the degree of deviation map for the site associated with each disease.

[0028] Furthermore, the method for having the user specify at least one of several diseases can be appropriately selected. For example, the control unit may have the user specify at least one of several pre-prepared disease names, or may have the user input a disease name. In addition, each of the multiple analysis modes for performing disease-related analysis on fundus images may be associated with a relevant region within the fundus tissue. The control unit may selectively display the deviation map for the region associated with the analysis mode to be executed from among the multiple analysis modes. In this case, the deviation map for the region related to the analysis mode (e.g., the region to be analyzed) will be displayed appropriately. The control unit may also have the user specify a disease while the deviation maps for at least one of the multiple regions (or all deviation maps) are displayed on the display unit. Alternatively, the user may specify a disease while a composite map, obtained by aligning and combining multiple deviation maps, is displayed on the display unit. In this case, regions with high deviations can be efficiently identified regardless of the type of disease.

[0029] The control unit may further perform a thickness map display step, which involves displaying a thickness map on the display unit showing the two-dimensional distribution of the thickness of a specific layer among multiple layers contained in the fundus tissue captured in the three-dimensional image. In the selective display step, the control unit may selectively display on the display unit a deviation map for at least one of the layers and boundaries within the range of the layer whose thickness is indicated by the thickness map, among the multiple layers contained in the fundus tissue and the boundaries between adjacent layers. In this case, the user can easily confirm both the thickness map and the deviation map for the layers and boundaries within the range of the thickness indicated by the thickness map. This makes it easier to perform a more appropriate diagnosis.

[0030] The thickness of a particular layer indicated by a thickness map may be a single layer or multiple layers. If the thickness of multiple specific layers is indicated by a thickness map, the layers whose thickness is indicated may be stacked consecutively. In this case, the total thickness of the stacked consecutively may be indicated by the thickness map.

[0031] Furthermore, the specific method for selectively displaying deviation maps for layers and boundaries within the indicated thickness range can also be appropriately selected. For example, a deviation map may be displayed for a specific layer identified by the thickness map. Alternatively, at least one of the deviation maps for the superficial retinal boundary and the deep retinal boundary of a specific layer may be displayed. The deviation maps for the superficial retinal boundary and the deep retinal boundary of a specific layer may be displayed side by side, or a composite map combining the two deviation maps may be displayed. Alternatively, deviation maps for all layers and boundaries within the indicated thickness range may be displayed. In this case as well, multiple deviation maps may be displayed separately, or a composite map may be displayed.

[0032] <Embodiment> (Device configuration) Hereinafter, one typical embodiment of the present disclosure will be described with reference to the drawings. As shown in Figure 1, in this embodiment, a mathematical model construction device 101, a fundus image processing device 1, and OCT devices (fundus image acquisition devices) 10A and 10B are used. The mathematical model construction device 101 constructs a mathematical model by training the mathematical model using a machine learning algorithm. The constructed mathematical model identifies (detects) specific parts (in this embodiment, multiple layers and boundaries in the fundus tissue) that are visible in the fundus image based on the input fundus image. The fundus image processing device 1 performs various processes using the results output by the mathematical model. The OCT devices 10A and 10B function as fundus image acquisition devices that capture fundus images (in this embodiment, tomographic images of the fundus) of the eye under examination.

[0033] As an example, a personal computer (hereinafter referred to as "PC") is used in the mathematical model construction device 101 of this embodiment. As will be described in detail later, the mathematical model construction device 101 trains a mathematical model using data of the fundus image of the eye under examination (hereinafter referred to as "training fundus image") acquired from the OCT device 10A and data indicating multiple parts of the eye under examination from which the training fundus image was taken (in this embodiment, multiple layers / boundaries). As a result, a mathematical model is constructed. However, the device that can function as the mathematical model construction device 101 is not limited to a PC. For example, the OCT device 10A may function as the mathematical model construction device 101. In addition, the control units of multiple devices (for example, the CPU of the PC and the CPU 13A of the OCT device 10A) may cooperate to construct the mathematical model.

[0034] Furthermore, a PC is used as the fundus image processing device 1 in this embodiment. However, the device that can function as the fundus image processing device 1 is not limited to a PC. For example, an OCT device 10B or a server may function as the fundus image processing device 1. When the OCT device 10B functions as the fundus image processing device 1, the OCT device 10B can capture fundus images and process the captured fundus images at the same time. Alternatively, a mobile device such as a tablet or smartphone may function as the fundus image processing device 1. The control units of multiple devices (for example, the CPU of the PC and the CPU 13B of the OCT device 10B) may cooperate to perform various processes.

[0035] Furthermore, this embodiment illustrates the case where a CPU is used as an example of a controller that performs various processing tasks. However, it goes without saying that controllers other than the CPU may be used in at least some of the various devices. For example, processing speed may be increased by adopting a GPU as the controller.

[0036] The mathematical model construction device 101 will now be described. The mathematical model construction device 101 is installed, for example, at a manufacturer that provides a fundus image processing device 1 or a fundus image processing program to a user. The mathematical model construction device 101 includes a control unit 102 that performs various control processing and a communication interface 105. The control unit 102 includes a CPU 103, which is a controller that manages the control, and a storage device 104 that can store programs and data. The storage device 104 stores a mathematical model construction program for executing the mathematical model construction processing (see Figure 6) which will be described later. The communication interface 105 connects the mathematical model construction device 101 to other devices (for example, an OCT device 10A and a fundus image processing device 1, etc.).

[0037] The mathematical model building device 101 is connected to an operation unit 107 and a display device 108. The operation unit 107 is operated by the user to input various instructions to the mathematical model building device 101. The operation unit 107 can use at least one of the following: a keyboard, mouse, touch panel, etc. A microphone or the like may be used with or in place of the operation unit 107 to input various instructions. The display device 108 displays various images. The display device 108 can use various devices capable of displaying images (for example, at least one of a monitor, display, projector, etc.). In this disclosure, "image" includes both still images and moving images.

[0038] The mathematical model building device 101 can acquire fundus image data (hereinafter sometimes simply referred to as "fundus image") from the OCT device 10A. The mathematical model building device 101 may acquire fundus image data from the OCT device 10A by at least one of the following: wired communication, wireless communication, or a removable storage medium (e.g., a USB memory stick).

[0039] The fundus image processing device 1 will now be described. The fundus image processing device 1 is installed, for example, in a facility that performs diagnosis or examinations on a patient (e.g., a hospital or health checkup facility). The fundus image processing device 1 includes a control unit 2 that performs various control processing and a communication interface 5. The control unit 2 includes a CPU 3, which is a controller that manages the control, and a storage device 4 that can store programs and data. The storage device 4 stores a fundus image processing program for executing fundus image processing (see Figure 7), which will be described later. The fundus image processing program includes a program that realizes the mathematical model constructed by the mathematical model construction device 101. The communication interface 5 connects the fundus image processing device 1 to other devices (e.g., the OCT device 10B and the mathematical model construction device 101).

[0040] The fundus image processing device 1 is connected to an operation unit 7 and a display device (display unit) 8. Various devices can be used in the operation unit 7 and the display device 8, similar to the operation unit 107 and the display device 108 described above.

[0041] The fundus image processing device 1 can acquire fundus images (in this embodiment, three-dimensional tomographic images of fundus tissue) from the OCT device 10B. The fundus image processing device 1 may acquire fundus images from the OCT device 10B by at least one of the following: wired communication, wireless communication, or a removable storage medium (e.g., a USB memory). The fundus image processing device 1 may also acquire programs for realizing mathematical models constructed by the mathematical model construction device 101 via communication or the like.

[0042] The OCT device 10 (10A, 10B) will be described below. The OCT device 10 is an example of a fundus imaging device that captures tomographic images of the fundus. In this embodiment, the case in which the OCT device 10A, which provides fundus images to the mathematical model construction device 101, and the OCT device 10B, which provides fundus images to the fundus image processing device 1 are used will be described. However, the number of OCT devices used is not limited to two. For example, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from multiple OCT devices. Alternatively, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from a single common OCT device.

[0043] The OCT device 10 can capture two-dimensional and three-dimensional tomographic images of the fundus of the eye being examined. An example of the acquisition method will be described. As shown in Figure 2, the OCT device 10 of this embodiment sets up multiple linear scanning lines 41 at equal intervals within a two-dimensional measurement area 40 that extends in a direction intersecting the optical axis of the OCT measurement light. By scanning the spot of the measurement light on each scanning line 41, the OCT device 10 can capture two-dimensional tomographic images 42 (see Figure 3) of the cross-sections intersecting each scanning line 41. The two-dimensional tomographic image 42 may be an averaged image generated by performing an averaged calculation on multiple two-dimensional tomographic images of the same area. Furthermore, the OCT device 10 can acquire (capture) a three-dimensional image (three-dimensional tomographic image) 43 (see Figure 4) by arranging multiple two-dimensional tomographic images 42 captured for multiple scanning lines 41 in a direction perpendicular to the image area.

[0044] Returning to the explanation of Figure 1, the OCT device 10A connected to the mathematical model construction device 101 can capture at least a two-dimensional tomographic image 42 (see Figure 3) of the fundus of the eye being examined. In addition, the OCT device 10B connected to the fundus image processing device 1 can capture a three-dimensional image 43 (see Figure 4) of the fundus of the eye being examined, in addition to the two-dimensional tomographic image 42 mentioned above.

[0045] (Structure of the layers and boundaries of the fundus) Referring to Figure 5, the structure of the layers in the fundus of the eye under examination, and the structure of the boundaries between adjacent layers, will be explained. Figure 5 schematically shows the structure of layers and boundaries in the fundus. The upper part of Figure 5 is the surface side (superficial side) of the retina in the fundus. In other words, the depth of the layers and boundaries increases as you move towards the lower part of Figure 5. Also, in Figure 5, the names of the boundaries between adjacent layers are enclosed in parentheses.

[0046] Let's explain the layers of the fundus. In the fundus, from the surface side (upper side of Figure 5) in order, there are the ILM (internal limiting membrane), NFL (nerve fiber layer), GCL (ganglion cell layer), IPL (inner plexiform layer), INL (inner nuclear layer), OPL (outer plexiform layer), ONL (outer nuclear layer), ELM (external limiting membrane), IS / OS (junction between photoreceptor inner and outer segment), RPE (retinal pigment epithelium), BM (Bruch's membrane), and Choroid.

[0047] Furthermore, boundaries that are likely to appear in tomographic images include, for example, NFL / GCL (boundary between NFL and GCL), IPL / INL (boundary between IPL and INL), OPL / ONL (boundary between OPL and ONL), RPE / BM (boundary between RPE and BM), and BM / Choroid (boundary between BM and Choroid).

[0048] (Mathematical model building process) Referring to Figure 6, the mathematical model construction process performed by the mathematical model construction device 101 will be described. The mathematical model construction process is executed by the CPU 103 according to the mathematical model construction program stored in the storage device 104.

[0049] In the following example, we will describe how to construct a mathematical model that outputs the identification results of multiple specific layers and boundaries from among the multiple layers and boundaries visible in a fundus image by analyzing an input two-dimensional tomographic image. In this embodiment, the mathematical model identifies ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM as multiple specific layers and boundaries. However, it is also possible to construct a mathematical model that outputs results different from the identification results of layers and boundaries during the mathematical model construction process. For example, a mathematical model that outputs the identification results of two or more parts of the input fundus image, such as the optic nerve head, macula, fovea, fundus blood vessels, and diseased areas, may be constructed by the mathematical model construction process.

[0050] Furthermore, the mathematical model illustrated in this embodiment outputs a probability distribution for identifying each of the specific parts (specific layers / boundaries in this embodiment) visible in the input fundus image.

[0051] In the mathematical model building process, the mathematical model is built by training it using a training dataset. The training dataset includes input data (input training data) and output data (output training data).

[0052] As shown in Figure 6, the CPU 103 acquires data from a fundus image (a two-dimensional tomographic image in this embodiment) captured by the OCT device 10A as input training data (S1). Next, the CPU 103 acquires data indicating each of several specific parts of the eye from which the fundus image acquired in S1 was taken as output training data (S2). The output training data in this embodiment includes label data indicating the positions of several specific layers and boundaries visible in the fundus image. The label data may be generated, for example, by an operator operating the control unit 107 while viewing the layers and boundaries in the fundus image.

[0053] Next, CPU103 trains a mathematical model using a training dataset with a machine learning algorithm (S3). Commonly known machine learning algorithms include neural networks, random forests, boosting, and support vector machines (SVM).

[0054] Neural networks are techniques that mimic the behavior of biological nerve cell networks. Examples of neural networks include feedforward neural networks, RBF networks (radiating basis function networks), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Basian networks, etc.).

[0055] Random forests are a method for generating multiple decision trees by learning from randomly sampled training data. When using random forests, the system follows the branches of multiple decision trees that have been pre-trained as classifiers, and takes the average (or majority vote) of the results obtained from each decision tree.

[0056] Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. It involves sequentially training simple, weak classifiers to construct a strong classifier.

[0057] SVM is a method for constructing a two-class pattern classifier using linear input elements. SVM learns the parameters of linear input elements based on a criterion (hyperplane separation theorem) that finds the margin-maximizing hyperplane that maximizes the distance to each data point from the training data.

[0058] A mathematical model refers to a data structure for predicting the relationship between input data (in this embodiment, data of two-dimensional tomographic images similar to the input training data) and output data (in this embodiment, data of the identification results for a specific set of parts). The mathematical model is constructed by training it using a training dataset. As mentioned above, the training dataset is a set of input training data and output training data. For example, training updates the correlation data (e.g., weights) between each input and output.

[0059] In this embodiment, a multilayer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data for the analysis results to be predicted, and one or more hidden layers between the input and output layers. Each layer has multiple nodes (also called units). In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multilayer neural network, is used.

[0060] As an example, the mathematical model constructed in this embodiment outputs a probability distribution for identifying each of the following regions (one of the one-dimensional, two-dimensional, three-dimensional, and four-dimensional regions including the time axis): the coordinates (one of the one-dimensional, two-dimensional, three-dimensional, and four-dimensional coordinates) in which each of the following specific regions (in this embodiment, a specific set of layers / boundaries) exists. In this embodiment, the softmax function is applied to cause the mathematical model to output a probability distribution. Specifically, the mathematical model constructed in S3 outputs a probability distribution in which the coordinates of the specific layer / boundary exist in a one-dimensional region extending in a direction intersecting the specific layer / boundary in the two-dimensional tomographic image (in this embodiment, the depth direction, which is along the optical axis of the OCT measurement light, and the vertical direction in Figure 3).

[0061] However, the specific method by which the mathematical model outputs a probability distribution for identifying a particular part can be modified as appropriate. For example, the mathematical model may output a probability distribution for identifying a part, where the random variable is the two-dimensional or three-dimensional coordinates in which a particular part (e.g., a feature area) exists in a two-dimensional or three-dimensional domain. Alternatively, the mathematical model may output a probability distribution for each region (e.g., each pixel) of the input ophthalmic image, where the random variable is the type of multiple parts (e.g., multiple layers and boundaries) in the eye being examined. Furthermore, the ophthalmic image input to the mathematical model may be a moving image.

[0062] Other machine learning algorithms may also be used. For example, a Generative Adversarial Network (GAN), which utilizes two competing neural networks, may be adopted as the machine learning algorithm.

[0063] The processes S1 to S3 are repeated until the construction of the mathematical model is complete (S4:NO). Once the construction of the mathematical model is complete (S4:YES), the mathematical model construction process ends. In this embodiment, the program and data for realizing the constructed mathematical model are incorporated into the fundus image processing device 1.

[0064] (Fundus image processing) The fundus image processing performed by the fundus image processing device 1 will be described with reference to Figures 7 to 12. The fundus image processing is performed by the CPU 3 of the fundus image processing device 1 according to the fundus image processing program stored in the storage device 4.

[0065] First, the CPU 3 acquires a three-dimensional image (three-dimensional tomographic image) of the fundus of the eye being examined (S11). The three-dimensional image is captured by the OCT device 10B and acquired by the fundus image processing device 1. As mentioned above, the three-dimensional image is composed of multiple two-dimensional images (two-dimensional tomographic images) captured by scanning the measurement light on different scan lines. Alternatively, the CPU 3 may acquire a signal (e.g., an OCT signal) from the OCT device 10B that will serve as the basis for generating the three-dimensional image, and generate the three-dimensional image based on the acquired signal.

[0066] CPU3 executes the deviation degree acquisition process (S12). In the deviation degree acquisition process, the deviation degree for the three-dimensional image acquired in S11 is obtained. The deviation degree is the degree of difference between the probability distribution for which the mathematical model identifies each of the multiple parts of the fundus tissue shown in the three-dimensional image and the probability distribution when each of the multiple parts is accurately identified by the mathematical model. The deviation degree often increases or decreases depending on the degree of structural abnormality, etc. Therefore, by understanding the deviation degree, the efficiency of diagnosis can be easily improved.

[0067] As shown in Figure 8, the CPU 3 extracts the T-th (initial value of T is "1") two-dimensional tomographic image from among the multiple two-dimensional tomographic images that constitute the three-dimensional image acquired in S11 (S36). Figure 9 shows an example of the extracted two-dimensional tomographic image 42. The two-dimensional tomographic image 42 shows multiple layers and boundaries in the fundus of the eye being examined. In addition, multiple one-dimensional regions A1 to AN are set within the two-dimensional tomographic image 42. In this embodiment, the one-dimensional regions A1 to AN set within the two-dimensional tomographic image 42 extend along axes that intersect specific layers and boundaries. In detail, the one-dimensional regions A1 to AN in this embodiment correspond to each region of the multiple (N) A scans that constitute the two-dimensional tomographic image 42 taken by the OCT device 10.

[0068] CPU3 inputs the T-th two-dimensional tomographic image into a mathematical model to obtain the probability distribution of the coordinates where the M-th region (initial value of M is "1") exists in each of the multiple one-dimensional regions A1 to AN, as the probability distribution for identifying the M-th region (S37). CPU3 obtains the degree of deviation of the probability distribution for the M-th region (S38). The degree of deviation obtained in S38 is the difference between the probability distribution obtained in S37 and the probability distribution when the M-th region is accurately identified.

[0069] In this embodiment, the degree of deviation is calculated as the entropy of the probability distribution P. The entropy is given by (Equation 1) below. The entropy H(P) takes the value 0 ≤ H(P) ≤ log(number of events), and the more biased the probability distribution P is, the smaller the value becomes. In other words, the smaller the entropy H(P), the higher the identification accuracy of the M-th part tends to be. H(P)=-Σplog(p)...(Math. 1)

[0070] Next, CPU3 determines whether the degree of deviation has been obtained for all parts (layers and boundaries) to be identified in the T-th two-dimensional tomographic image (S40). If the degree of deviation has not yet been obtained for some parts (S40: NO), "1" is added to the order M of the parts to be identified (S41), and the process returns to S37, where the degree of deviation for the next part is obtained (S37, S38). Once the degree of deviation for all parts has been obtained (S40: YES), CPU3 stores the multiple degree of deviation obtained for each of the multiple parts shown in the T-th two-dimensional tomographic image in the memory device 4 (S42).

[0071] Next, CPU3 determines whether or not the deviation degrees of all two-dimensional tomographic images constituting the three-dimensional tomographic image have been obtained (S44). If the deviation degrees of some of the two-dimensional tomographic images have not yet been obtained (S44: NO), "1" is added to the order T of the two-dimensional tomographic images (S45), and the process returns to S36, where the deviation degree of the next two-dimensional tomographic image is obtained (S36~S42).

[0072] Once the degree of deviation of all two-dimensional tomographic images has been obtained (S44:YES), CPU3 obtains a degree of deviation map for each of the multiple areas targeted for identification (S47). After that, the process returns to fundus image processing (see Figure 7). The degree of deviation map shows the two-dimensional distribution of the magnitude of the degree of deviation when the fundus is viewed from the front (i.e., in the direction along the optical axis of the OCT measurement light). As mentioned above, in this embodiment, a degree of deviation map is obtained for each of ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM.

[0073] Figure 10 shows an example of a discrepancy map for some areas. In the discrepancy map shown in Figure 10, areas with a large degree of discrepancy are represented in a bright color. The discrepancy map shown in Figure 10 was obtained from a three-dimensional image near the optic nerve head (hereinafter simply referred to as "optic nerve head"). Where the optic nerve head is present, the ILM and NFL are present, while layers and boundaries deeper than the NFL are missing. Therefore, where the optic nerve head is present, the degree of discrepancy regarding the identification of layers and boundaries deeper than the NFL is larger than where the optic nerve head is absent. In the example shown in Figure 10, the discrepancy map for the ILM is small overall. Therefore, it can be seen that the ILM is identified with a high probability throughout the entire two-dimensional region. On the other hand, in the discrepancy maps for the IPL / INL and BM, the degree of discrepancy is large in the central area where the optic nerve head is present within the two-dimensional region.

[0074] Returning to the explanation of Figure 7, the CPU 3 selectively displays a predetermined number of deviation maps with high deviation rankings (i.e., rankings of deviation magnitude) from among multiple deviation degrees for each of the multiple parts targeted for identification on the display device 8 as the default (S13). Therefore, the user can easily check the deviation maps of the parts among the multiple parts targeted for identification where the deviation degree of identification by the mathematical model is high. In this embodiment, the CPU 3 displays a predetermined number of deviation maps in descending order of the average value of the deviation degree within the two-dimensional deviation map. However, the method for determining the ranking of the deviation degree is not limited to using the average value. For example, the CPU 3 may display a predetermined number of deviation maps in descending order of the maximum deviation value within the two-dimensional deviation map. Furthermore, CPU3 may display the single deviation map with the highest deviation ranking, or it may display multiple deviation maps in descending order of deviation ranking. Alternatively, CPU3 may display a composite map created by aligning and combining multiple deviation maps with high deviation rankings.

[0075] CPU3 extracts a portion of the two-dimensional tomographic image extending in the depth direction of the fundus from the default position of the three-dimensional image acquired in S11 and displays it on the display device 8 (S14). Figure 11 shows an example in which the two-dimensional tomographic image extracted from the three-dimensional image is displayed on the display device 8 along with the selected deviation map. In this embodiment, as shown in Figure 11, a two-dimensional frontal image of the fundus, taken when the acquisition range of the three-dimensional image acquired in S11 is viewed from the front (i.e., in the direction along the optical axis of the OCT measurement light), is also displayed along with the deviation map and the two-dimensional tomographic image. In this case, the frontal image of the fundus may be an image generated based on the three-dimensional image (a so-called Enface image), or it may be an image taken by a device different from the device that took the three-dimensional image (for example, an SLO device or a fundus camera).

[0076] The default position for extracting a two-dimensional tomographic image from a three-dimensional image can be set as appropriate. For example, on a two-dimensional frontal image viewed from the front of the three-dimensional image, a line-shaped position crossing the center position horizontally may be set as the default position for extracting the two-dimensional tomographic image in S14. Alternatively, a characteristic area in the fundus (e.g., the optic disc or macula) may be identified using image processing, and a line-shaped position passing through the identified area may be set as the default position for extracting the two-dimensional tomographic image in S14. Note that the line is not limited to a straight line.

[0077] CPU3 displays the areas on the two-dimensional tomographic image that have a deviation map (sometimes called "corresponding areas") among the multiple areas identified by the mathematical model (S15). Therefore, the user can easily and appropriately identify the area where the deviation map is displayed (in Figure 11, "Area A") on the two-dimensional tomographic image that extends in the depth direction. In this embodiment, as shown in Figure 11, Area A is indicated by changing the color of Area A (layer / boundary) where the deviation map is displayed among the multiple layers / boundaries visible in the two-dimensional tomographic image.

[0078] The CPU 3 displays the location from which the two-dimensional tomographic image displayed on the display device 8 was extracted on the divergence map (S16). Therefore, the user can perform diagnoses of fundus tissue, etc., while confirming the extraction location of the displayed two-dimensional tomographic image on the divergence map. In this embodiment, as shown in Figure 11, a line P indicating the extraction location of the two-dimensional tomographic image is displayed on the two-dimensional divergence map when the fundus tissue is viewed from the front. Similarly, in this embodiment, a line P indicating the extraction location of the two-dimensional tomographic image is also displayed on the two-dimensional frontal image of the fundus.

[0079] Next, the CPU 3 determines whether the user has entered an instruction to specify at least one of the parts of the two-dimensional tomographic image displayed on the display device 8 in which the deviation degree map should be displayed (S17). As an example, in this embodiment, the user specifies the parts of the two-dimensional image displayed on the display device 8 in which the deviation degree map should be displayed by operating the operation unit 7 (for example, moving the cursor and clicking, or specifying a position using the touch panel). The number of parts specified by the user may be one or more. If part A is not specified (S17: NO), the process proceeds directly to S21.

[0080] When a region to display the deviation map is specified (S17:YES), the CPU3 displays region A (corresponding region), which is one of the multiple regions identified by the mathematical model, on the two-dimensional tomographic image (S18). Therefore, the user can easily and appropriately grasp region A, where the deviation map is displayed, on the two-dimensional tomographic image that extends in the depth direction. In the example shown in Figure 11, IPL / INL is specified on the two-dimensional image as region A to display the deviation map. In this embodiment, the CPU3 indicates the specified region (corresponding region) on the two-dimensional image by changing the color of IPL / INL relative to the colors of other regions.

[0081] Furthermore, CPU3 selectively displays the deviation map of the area specified by the user (corresponding area) from among the multiple areas identified by the mathematical model on the display device 8 (S19). Therefore, the user can easily confirm the deviation map of the appropriate area from among the deviation maps of multiple areas, based on the internal state of the fundus tissue shown in the two-dimensional tomographic image. The process then proceeds to S21. In the example shown in Figure 11, CPU3 displays the IPL / INL deviation map specified by the user on the display device 8. CPU3 also displays the location from which the two-dimensional tomographic image was extracted on the IPL / INL deviation map displayed on the display device 8.

[0082] Next, the CPU 3 determines whether the user has entered an instruction to specify the position P from which to extract the two-dimensional tomographic image on the deviation map displayed on the display device 8 (S21). If position P is not specified (S21: NO), the process proceeds directly to S25. As an example, in this embodiment, the user specifies the position P from which to extract the two-dimensional tomographic image on the deviation map displayed on the display device 8 by operating the operation unit 7 (for example, moving the cursor and clicking, or specifying a position using the touch panel). In this embodiment, the CPU 3 can also similarly accept instructions to specify the position P from which to extract the two-dimensional tomographic image on the two-dimensional frontal fundus image displayed on the display device 8. In the example shown in Figure 11, the position P from which to extract the two-dimensional tomographic image is specified by specifying the position of a straight line. However, it is also possible to change the method for specifying the extraction position P. For example, the line used to specify position P is not limited to a straight line, but may be ring-shaped or curved, etc.

[0083] When the position P for extracting the two-dimensional tomography image is specified on the divergence map (S21:YES), the CPU3 displays the specified position P on the divergence map (S22). Therefore, the user can diagnose the fundus tissue of the eye being examined while confirming the extraction position P of the displayed two-dimensional tomography image on the divergence map. As mentioned above, in the example shown in Figure 11, a line P indicating the extraction position of the two-dimensional tomography image is displayed on the two-dimensional divergence map when the fundus tissue is viewed from the front. In this embodiment, a line P indicating the extraction position of the two-dimensional tomography image is also displayed on the two-dimensional frontal fundus image. Similarly, when the extraction position P of the two-dimensional tomography image is specified on the frontal fundus image, a line P indicating the extraction position of the two-dimensional tomography image is displayed on both the divergence map and the frontal fundus image.

[0084] Furthermore, the CPU 3 extracts a two-dimensional tomographic image from the three-dimensional image that extends in the depth direction of the fundus tissue from the specified position P (i.e., a two-dimensional tomographic image that passes through the specified position P and extends in the depth direction) and displays it on the display device 8 (S23). Therefore, the user can check the two-dimensional distribution of the degree of discrepancy in identification for a specific part (in this embodiment, a specific layer / boundary) using the discrepancy map, and then directly specify the position of the two-dimensional tomographic image they want to display on the discrepancy map. After that, the process moves on to S25.

[0085] Next, CPU3 determines whether the user has entered an instruction to specify one of several diseases (S25). If no disease is specified (S25:NO), the process proceeds directly to S28. In this embodiment, the user can specify one of several diseases to display a discrepancy map for the area associated with the specified disease. Specifically, each of the multiple diseases that may occur in the subject is associated with at least one of the related areas in the fundus tissue. For example, in this embodiment, ERM (epiretinum) is associated with an area in the superficial layer of the retina, diabetic retinopathy with an area in the inner layer of the retina, and AMD (age-related macular degeneration) and CNV (choroidal neovascularization) with an area in the outer layer of the retina. When a disease is specified by the user (S25:YES), CPU3 selectively displays the discrepancy map for the area associated with the specified disease from among the multiple areas identified by the mathematical model on the display device 8 (S26). Therefore, the user can easily check the discrepancy map for the area related to the expected disease of the subject simply by specifying the expected disease. Furthermore, users can predict the patient's disease by switching between specified diseases and checking the degree of discrepancy map of the area associated with each disease. In addition, in S26, the area on which the degree of discrepancy map is displayed may also be displayed on the two-dimensional tomographic image.

[0086] Next, CPU3 determines whether or not an instruction to display the thickness map has been received (S28). The thickness map shows the two-dimensional distribution of the thickness of a specific layer among the multiple layers contained in the fundus tissue. The thickness map exemplified in Figure 12 is the thickness map of the layers from ILM to NFL / GCL among the multiple layers contained in the fundus tissue. However, it is also possible to display the thickness map of other layers. It is also possible to have the user specify which layer's thickness map to display.

[0087] When an instruction to display a thickness map is received (S28: YES), the CPU 3 displays the thickness map of a specific layer on the display device 8 (S29). Furthermore, the CPU 3 displays the deviation map on the display device 8 for the layers and boundaries within the range where the thickness map is shown, from among the layers and boundaries captured in the three-dimensional image acquired in S11 (S30). Therefore, the user can easily check both the thickness map and the deviation map for the layers and boundaries within the range where the thickness is shown by the thickness map. In the example shown in Figure 12, the thickness map of the layer from ILM to NFL / GCL is displayed. Also, the deviation maps of ILM and NFL / GCL that exist within the range from ILM to NFL / GCL are displayed. Note that in S230, the area where the deviation map is displayed may also be displayed on the two-dimensional tomographic image.

[0088] CPU3 determines whether or not a termination command has been entered (S32). If no termination command has been entered (S32: NO), the process returns to S17, and the processes from S17 to S32 are repeated. If a termination command is entered (S32: YES), the fundus image processing is terminated.

[0089] The technologies disclosed in the above embodiments are merely examples. Therefore, it is possible to modify the technologies exemplified in the above embodiments. First, it is possible to perform only some of the technologies exemplified in the above embodiments. For example, it is possible to omit some of the processes in S17, S21, S25, and S28.

[0090] Note that the process of acquiring a three-dimensional image in S11 of Figure 7 is an example of the "image acquisition step". The process of acquiring the degree of deviation in S12 (Figure 8) of Figure 7 is an example of the "degree of deviation acquisition step". The processes of selectively displaying some degree of deviation maps in S13, S19, S26, and S30 of Figure 7 are examples of the "selective display step". The processes of extracting and displaying two-dimensional tomographic images in S14 and S23 of Figure 7 are examples of the "two-dimensional tomographic image display step". The process of receiving input for specifying part A in S17 of Figure 7 is an example of the "part specification acceptance step". The processes of displaying the corresponding part on the two-dimensional tomographic image in S15 and S18 of Figure 7 are examples of the "corresponding part display step". The processes of displaying the extraction position of the two-dimensional tomographic image on the degree of deviation map in S16 and S22 of Figure 7 are examples of the "extraction position display step". The process in S21 of Figure 7, which accepts input for the extraction location of the two-dimensional tomography image, is an example of the "extraction location specification acceptance step". The process in S23 of Figure 7, which extracts and displays the two-dimensional tomography image, is an example of the "specified tomography image display step". The process in S29 of Figure 7, which displays the thickness map, is an example of the "thickness map display step". [Explanation of symbols]

[0091] 1. Fundus image processing device 3 CPU 4 Storage device 8 Display device 10A,10B OCT device 42 Two-dimensional tomography 43 Three-dimensional images

Claims

1. A fundus image processing device that processes a fundus image of a subject's eye, The control unit of the fundus image processing device includes: an image acquiring step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device; a deviation acquisition step of acquiring a plurality of probability distributions for identifying each of a plurality of sites in the fundus tissue shown in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and acquiring a deviation of the acquired probability distribution from a probability distribution when each of the plurality of sites is accurately identified; a selective display step of selectively displaying on a display unit a map showing a two-dimensional distribution of some of the deviation degrees among the acquired plurality of deviation degrees; A fundus image processing apparatus comprising:

2. The fundus image processing device according to claim 1 , A fundus image processing device characterized in that in the selective display step, maps of a predetermined number of deviation degrees having the highest ranking among the multiple deviation degrees acquired are selectively displayed on a display unit.

3. 3. The fundus image processing device according to claim 1, a two-dimensional tomographic image display step of extracting a part of a two-dimensional tomographic image extending in a depth direction of the fundus tissue from the three-dimensional image and displaying the extracted part on a display unit; a region designation receiving step of receiving an instruction input from a user for designating at least one of the regions appearing in the displayed two-dimensional tomographic image; Further execute A fundus image processing device characterized in that, in the selective display step, when an instruction input for specifying a region is accepted, a map of the deviation degree of the specified region among the multiple regions is selectively displayed on a display unit.

4. The fundus image processing device according to claim 3, A fundus image processing apparatus further comprising a corresponding region display step of displaying, on the two-dimensional tomographic image, a region among the plurality of regions for which a map of the degree of deviation is displayed.

5. 5. The fundus image processing device according to claim 1, A thickness map display step is further performed in which a thickness map showing a two-dimensional distribution of thicknesses of a specific layer among a plurality of layers included in the fundus tissue shown in the three-dimensional image is displayed on a display unit; The selective display step is characterized in that a map of the deviation degree for at least any of the layers and boundaries within the range whose thickness is indicated by the thickness map, among the multiple layers contained in the fundus tissue and the boundaries between adjacent layers, is selectively displayed on a display unit.

6. A fundus image processing program executed by a fundus image processing device for processing a fundus image of a subject's eye, The fundus image processing program is executed by a control unit of the fundus image processing device, an image acquiring step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device; a deviation acquisition step of acquiring a plurality of probability distributions for identifying each of a plurality of sites in the fundus tissue shown in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and acquiring a deviation of the acquired probability distribution from a probability distribution when each of the plurality of sites is accurately identified; a selective display step of selectively displaying on a display unit a map showing a two-dimensional distribution of some of the deviation degrees among the acquired plurality of deviation degrees; A fundus image processing program that causes the fundus image processing device to execute the above steps.

Images