Fundus image processing device and fundus image processing program

Abstract

To provide a fundus image processing device and a fundus image processing program capable of appropriately presenting, to a user, varieties of medical information obtained by processing an ophthalmologic image according to a situation.SOLUTION: A divergence map indicates a two-dimensional distribution of a degree of deviation of an acquired probability distribution with respect to the probability distribution where a layer or a boundary of an identification target in a fundus is accurately identified. A thickness analysis map indicates a two-dimensional distribution of an analysis result of a thickness of at least one layer in a fundus tissue. A control part includes the divergence map in subject's eye medical information that is initially displayed on a display part when a macular disease mode for performing diagnosis of a macular disease is selected as a diagnosis mode for performing diagnosis of the subject's eye. When a glaucoma mode for performing medical care for glaucoma is selected as the medical care mode, the control part includes the thickness analysis map in the subject's eye medical information that is initially displayed on the display unit.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a fundus image processing device and a fundus image processing program used to process a fundus image of a subject's eye. [Background technology]

[0002] In recent years, technologies have been proposed for acquiring various medical information to assist a user (e.g., a doctor) in medical treatment (diagnosis, examination, treatment, etc.) of the subject's eye by processing a fundus image of the subject's eye. For example, an image processing device described in Patent Document 1 acquires a probability distribution for identifying tissue in an ophthalmic image by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm. The image processing device acquires the degree of deviation of the acquired probability distribution from the probability distribution when the tissue is accurately identified as structural information indicating the degree of abnormality in the tissue structure. [Prior art documents] [Non-patent literature]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-18794 Summary of the Invention [Problem to be solved by the invention]

[0004] When an abnormality such as a disease occurs in a subject's eye, changes may occur in multiple layers in the fundus tissue and / or in at least one of the boundaries between adjacent layers (hereinafter, sometimes referred to as "layer boundaries"). Therefore, if users could properly understand the state of layers and boundaries using various medical information such as the degree of discrepancy, the efficiency of their medical treatment could be improved. However, the manner in which changes appear in layers and boundaries often differs depending on the condition of the subject's eye (e.g., the type of abnormality, such as a disease, occurring in the subject's eye). Therefore, simply mechanically presenting various types of medical information obtained by processing ophthalmic images to users is not sufficient to fully assist users in their medical treatment.

[0005] A typical object of the present disclosure is to provide a fundus image processing device and a fundus image processing program that can appropriately present various medical information obtained by processing ophthalmologic images to a user depending on the situation. [Means for solving the problem]

[0006] A fundus image processing device provided by a typical embodiment of the present disclosure is a fundus image processing device that processes fundus images in which a plurality of layers in the fundus of an eye to be examined and boundaries between layers are included in an imaging range, and a control unit of the fundus image processing device includes an image acquisition step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device, a discrepancy map generation step of inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of layers and boundaries in the fundus tissue depicted in the three-dimensional image, and generating a discrepancy map showing a two-dimensional distribution of discrepancies of the acquired probability distribution with respect to a probability distribution when the layer or boundary to be identified is accurately identified, and The method can execute a thickness analysis map generation step of generating a thickness analysis map showing a two-dimensional distribution of analysis results for the thickness of at least one layer of the fundus tissue shown in the image, and a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit, wherein in the display control step, when a macular disease mode in which macular disease treatment is performed is selected as the treatment mode in which treatment is performed for the test eye, the deviation map is included in the medical information of the test eye to be initially displayed on the display unit, and when a glaucoma mode in which glaucoma treatment is performed is selected as the treatment mode, the thickness analysis map is included in the medical information of the test eye to be initially displayed on the display unit.

[0007] A fundus image processing program provided by a typical embodiment of the present disclosure is a fundus image processing program executed by a fundus image processing device that processes fundus images in which a plurality of layers in the fundus of an eye to be examined and boundaries between layers are included in an imaging range, and the fundus image processing program is executed by a control unit of the fundus image processing device, thereby performing an image acquisition step of acquiring a three-dimensional image of the fundus photographed by the fundus image photographing device, and inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of layers and boundaries in the fundus tissue depicted in the three-dimensional image, and generating a discrepancy map that shows a two-dimensional distribution of the discrepancy of the acquired probability distribution with respect to the probability distribution when the layer or boundary to be identified is accurately identified. The fundus image processing device can be made to execute a generation step, a thickness analysis map generation step for generating a thickness analysis map showing a two-dimensional distribution of analysis results for the thickness of at least one layer in the fundus tissue shown in the three-dimensional image, and a display control step for displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit, wherein in the display control step, when a macular disease mode for treating macular disease is selected as the treatment mode for treating the subject's eye, the deviation map is included in the medical information of the subject's eye that is initially displayed on the display unit, and when a glaucoma mode for treating glaucoma is selected as the treatment mode, the thickness analysis map is included in the medical information of the subject's eye that is initially displayed on the display unit.

[0008] According to the fundus image processing device and fundus image processing program of the present disclosure, various medical information obtained by processing an ophthalmologic image is presented to a user appropriately according to the situation. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of a mathematical model construction device 101, a fundus image processing device 1, and OCT devices 10A and 10B. [Figure 2]FIG. 1 is an explanatory diagram for explaining an example of a method for capturing a three-dimensional tomographic image. [Figure 3] FIG. 4 is a diagram showing an example of a two-dimensional tomographic image 42. [Figure 4] FIG. 4 is a diagram showing an example of a three-dimensional tomographic image 43. [Figure 5] FIG. 1 is a diagram showing a schematic diagram of the structure of layers and boundaries in the fundus. [Figure 6] 1 is a flowchart of a mathematical model construction process executed by a mathematical model construction device 101. [Figure 7] 10 is a flowchart of fundus image processing executed by the fundus image processing device 1. [Figure 8] 10 is a flowchart of a layer / boundary identification process executed during fundus image processing. [Figure 9] 2 is a diagram showing a schematic diagram of a relationship between a two-dimensional tomographic image 42 input to a mathematical model and one-dimensional regions A1 to AN in the two-dimensional tomographic image 42. FIG. [Figure 10] 10 is a flowchart of a process for macular diseases executed during fundus image processing. [Figure 11] FIG. 10 is a diagram showing an example of an initial display screen in a macular disease mode. [Figure 12] FIG. 10 is a diagram showing an example of a display screen in which a thickness analysis map 70 is additionally displayed in a macular disease mode. [Figure 13] FIG. 10 is a diagram showing an example of a method for displaying a plurality of deviation maps 60. [Figure 14] 10 is a flowchart of a process for glaucoma executed during fundus image processing. [Figure 15] FIG. 10 is a diagram showing an example of an initial display screen in a glaucoma mode. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Summary> The fundus image processing device exemplified in the present disclosure processes a fundus image in which multiple layers and boundaries between layers in the fundus of the subject's eye are included in the imaging range. The control unit of the fundus image processing device can execute an image acquisition step, a deviation map generation step, a thickness analysis map generation step, and a display control step. In the image acquisition step, the control unit acquires a three-dimensional image of the fundus captured by the fundus image capturing device. In the deviation map generation step, the three-dimensional image is input to a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of the layers and boundaries in the fundus tissue captured in the three-dimensional image, and a deviation map is generated based on the acquired probability distribution. The deviation map shows a two-dimensional distribution of deviation of the acquired probability distribution from the probability distribution when the layer or boundary to be identified is accurately identified. In the thickness analysis map generation step, the control unit generates a thickness analysis map showing a two-dimensional distribution of analysis results for the thickness of at least one layer in the fundus tissue captured in the three-dimensional image. In the display control step, the control unit causes the display unit to display medical information including at least one of the deviation map and the thickness analysis map. Specifically, in the display control step, when a macular disease mode for treating macular diseases is selected as the examination mode for treating the subject's eye, the control unit causes the display unit to initially display the medical information of the subject's eye, including the deviation map. When a glaucoma mode for treating glaucoma is selected as the examination mode, the control unit causes the display unit to initially display the medical information of the subject's eye, including the thickness analysis map.

[0011] As a premise, when there are no abnormalities in the structure of layers and boundaries, the mathematical model is likely to accurately identify the layers and boundaries, and the probability distribution when identifying layers and boundaries using the mathematical model is likely to be biased. On the other hand, when there are abnormalities in the structure of layers and boundaries, the probability distribution is less likely to be biased. Therefore, the deviation map, which shows the two-dimensional distribution of the deviation between the probability distribution when layers and boundaries are accurately identified and the actually obtained probability distribution, is likely to show the degree of abnormality in the structure of layers and boundaries.

[0012] When a macular disease develops in a subject's eye, it is often accompanied by abnormalities in the structure of layers and boundaries. Therefore, in conventional macular disease treatments, a user needs to check two-dimensional tomographic images at various positions within the capture range of a three-dimensional tomographic image to determine whether or not a structural abnormality has occurred. In contrast, according to the technology disclosed herein, when a macular disease mode is selected, a discrepancy map is initially displayed, which allows the degree of abnormality in the structure of layers and boundaries to be easily grasped in a two-dimensional area. Therefore, the discrepancy map initially displayed on the display unit makes it easy for the user to appropriately grasp the positions where a structural abnormality is likely to occur.

[0013] Furthermore, when glaucoma develops in the subject's eye, the thickness of at least some layers often becomes thinner (i.e., thinning occurs). According to the technology of the present disclosure, when the glaucoma mode is selected, a thickness analysis map showing a two-dimensional distribution of the analysis results for the layer thickness is initially displayed. Therefore, the thickness analysis map initially displayed on the display unit makes it easier for the user to appropriately treat glaucoma based on the subject's layer thickness. Through the above processing, various medical information obtained by processing the ophthalmic image is appropriately presented to the user depending on the situation.

[0014] In the present disclosure, the initial display screen refers to a screen on which medical information including a map (at least one of a deviation map and a thickness analysis map) is first displayed when displaying medical information about the examinee's eye in any of the examination modes. Therefore, after the examination mode for the examinee's eye is started, some screen not including a map (e.g., a startup screen) may be displayed before the initial display screen including the map is displayed.

[0015] The specific aspect of the thickness analysis map initially displayed when the glaucoma mode is selected can be selected as appropriate. For example, in the thickness analysis map generating step, the control unit may generate a normal eye comparison map (e.g., at least one of a percentile map showing a two-dimensional distribution of the difference between the two layers and a deviation map showing a two-dimensional distribution of the deviation between the two layers) as the thickness analysis map, which shows the results of comparing the two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed with the two-dimensional distribution of the thickness of the same layer in a normal eye. Furthermore, in the thickness analysis map generating step, the control unit may generate a thickness map showing the two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed as the thickness analysis map. The control unit may display both the normal eye comparison map and the thickness map on the display unit as the thickness analysis map.

[0016] A macular disease is a disease that causes abnormalities in the central part of the retina (the macula and the area surrounding the macula) at the fundus. For example, at least one of diabetic retinopathy, age-related macular degeneration, retinitis pigmentosa, retinal vein occlusion, macular edema, premacular membrane, macular hole, submacular hematoma, etc. may be classified as a macular disease. As an example, in this embodiment, diabetic retinopathy, age-related macular degeneration, and retinitis pigmentosa are included in the macular diseases.

[0017] The deviation may be output by a mathematical model. Alternatively, the control unit may calculate the deviation based on the probability distribution output by the mathematical model.

[0018] The deviation may include the entropy (average information content) of the acquired probability distribution. Entropy represents the degree of uncertainty, disorder, and chaos. In the present disclosure, the entropy of the output probability distribution is 0 when layers and boundaries are accurately identified. Furthermore, the more difficult it is to identify layers and boundaries, the greater the entropy. Therefore, using the entropy of the probability distribution as the deviation more appropriately quantifies the degree of abnormality in the structure of layers and boundaries. However, values ​​other than entropy may also be used as the deviation. For example, at least one of the standard deviation, coefficient of variation, and variance, which indicate the degree of dispersion of the acquired probability distribution, may be used as the deviation. The KL divergence, which is a measure of the difference between probability distributions, may also be used as the deviation. Furthermore, the maximum value of the acquired probability distribution may also be used as the deviation.

[0019] In the present disclosure, a three-dimensional image is formed by arranging multiple two-dimensional images. The degree of deviation is obtained for each of the multiple two-dimensional images constituting the three-dimensional image, or for each pixel, column, or row constituting the image. As a result, the degree of deviation when layers and boundaries are identified by a mathematical model is obtained for the entire fundus tissue depicted in the three-dimensional image. As an example, the deviation map of the present disclosure shows a two-dimensional distribution of the degree of deviation when the fundus tissue is viewed from the front (i.e., along the optical axis of the light for capturing the three-dimensional image). However, the direction in which the two-dimensional distribution of the degree of deviation is shown in the deviation map can be changed as appropriate. Furthermore, when displaying a deviation map of a specific layer, the control unit may display a two-dimensional distribution, such as the average value of the deviations of at least two boundaries included in the specific layer, as the deviation map of the specific layer.

[0020] When the glaucoma mode is selected as the treatment mode, the control unit may exclude the discrepancy map from the medical information of the subject's eye to be initially displayed on the display unit. As described above, when glaucoma develops in the subject's eye, at least some layers often become thinned. However, layer thinning caused by glaucoma is often not accompanied by structural abnormalities in the layers and their boundaries. If no structural abnormalities in the layers and their boundaries occur, the discrepancy map is unlikely to show any significant features. Therefore, when the glaucoma mode is selected, excluding discrepancy maps that are not frequently used in glaucoma treatment from the initial display helps shorten the time required for glaucoma treatment.

[0021] When the glaucoma mode is selected as the diagnosis mode, the control unit may display the discrepancy map on the display unit together with the thickness analysis map or alternately with the thickness analysis map in response to an instruction input by the user. In this case, even when the glaucoma mode is selected, the user can appropriately grasp the degree of abnormality in the layer / boundary structure by displaying the discrepancy map as needed.

[0022] When the macular disease mode is selected as the diagnosis mode, the control unit may exclude the layer thickness analysis map from the medical information of the subject's eye that is initially displayed on the display unit. In diagnosis of macular disease, the layer thickness analysis map may be used to additionally confirm the presence or absence of large edema, but may not be used in diagnosis of macular disease itself. Therefore, when the macular disease mode is selected, excluding the layer thickness analysis map, which is not used very frequently, from the initial display targets can easily shorten the time required for diagnosis of macular disease.

[0023] When the macular disease mode is selected as the diagnosis mode, the control unit may display the thickness analysis map on the display unit together with the discrepancy map or alternately with the discrepancy map in response to an instruction input by the user. In this case, even when the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of layer / boundary thicknesses by displaying the thickness analysis map as needed.

[0024] The control unit may further execute an extraction position designation receiving step and a designated tomographic image display step. In the extraction position designation receiving step, the control unit receives an instruction input from a user to designate an extraction position of a part of a two-dimensional tomographic image extending in the depth direction of the fundus tissue from the image region of the three-dimensional image. In the designated tomographic image display step, when the instruction input for designating the extraction position is received, the control unit extracts a two-dimensional tomographic image extending in the depth direction of the fundus tissue from the designated extraction position and displays it on the display unit. In the extraction position designation receiving step, if a macular disease mode is selected as the diagnosis mode, the instruction input for the extraction position may be received on a discrepancy map displayed on the display unit. Also, if a glaucoma mode is selected as the diagnosis mode, the instruction input for the extraction position may be received on a thickness analysis map displayed on the display unit.

[0025] As mentioned above, when a macular disease develops in a subject's eye, abnormalities in the structure of layers and boundaries are often present. The discrepancy map allows the user to easily grasp the degree of abnormality in the structure of layers and boundaries in a two-dimensional area. Therefore, by specifying the extraction position of the two-dimensional tomographic image on the discrepancy map while selecting the macular disease mode, the user can appropriately specify the extraction position of the two-dimensional tomographic image according to the degree of abnormality in the structure that is highly related to macular disease.

[0026] Furthermore, when glaucoma develops in the subject's eye, the thickness of at least some layers often decreases. The thickness analysis map makes it easy to grasp the two-dimensional distribution of layer thicknesses. Therefore, by specifying the extraction position of the two-dimensional tomographic image on the thickness analysis map while selecting the glaucoma mode, the user can appropriately specify the extraction position of the two-dimensional tomographic image in accordance with the distribution of the thickness of layers that are highly related to glaucoma.

[0027] It should be noted that a method of having the user specify the extraction position of the two-dimensional tomographic image on a map (discrepancy map or thickness analysis map) can also be appropriately selected. For example, the control unit may have the user specify a line on the two-dimensional map indicating the extraction position of the two-dimensional tomographic image (i.e., the position where the extracted two-dimensional tomographic image intersects with the map). In this case, the shape of the line may be straight or may be a shape other than a straight line (e.g., curved or circular). The size and angle of the line may also be changed according to instructions input by the user.

[0028] A layer and a boundary in the fundus tissue associated with each of a plurality of macular diseases that may occur in the subject may be associated with the associated layer and boundary in the fundus tissue. In the deviation map generating step, the control unit may generate a deviation map for each of the plurality of regions by acquiring a plurality of probability distributions for identifying each of the plurality of regions among the plurality of layers and boundaries in the fundus tissue depicted in the three-dimensional image. In the display control step, when a macular disease mode is selected, the control unit may selectively display, on the display unit, a deviation map for a region associated with a macular disease specified by the user.

[0029] In this case, the user can efficiently use the deviation map according to the type of macular disease to be treated. For example, the user can easily check the deviation map for the region associated with the subject's predicted macular disease simply by specifying the predicted macular disease. The user can also predict the subject's macular disease by switching the specified macular disease and checking the deviation map for the region associated with each macular disease.

[0030] The method for prompting the user to specify at least one of the multiple macular diseases can also be selected as appropriate. For example, the control unit may prompt the user to specify at least one of multiple macular disease names prepared in advance, or may prompt the user to input the name of the macular disease. The type of macular disease may be specified before the display of the medical information begins. In this case, the control unit may set the discrepancy map initially displayed on the display unit in the macular disease mode to the discrepancy map for the region associated with the specified macular disease. The type of macular disease may also be specified after the initial display of the medical information begins. In this case, the control unit may switch the discrepancy map displayed on the display unit to the discrepancy map for the region associated with the specified macular disease. The control unit may prompt the user to specify the macular disease while the display unit is displaying a discrepancy map for at least one of the multiple regions (or all of the discrepancy maps). The control unit may prompt the user to specify the macular disease while the display unit is displaying a composite discrepancy map obtained by aligning and combining multiple discrepancy maps. In this case, areas with high discrepancies can be efficiently identified regardless of the type of macular disease.

[0031] The technology for selectively displaying a discrepancy map for a region associated with a macular disease specified by a user on a display unit can be implemented without being combined with other technologies in the present disclosure. In this case, the fundus image processing device can also be expressed as follows. a fundus image processing device that processes fundus images in which multiple layers in the fundus of a test eye and the boundaries between layers are included in the imaging range, wherein the control unit of the fundus image processing device is capable of executing the following steps: an image acquisition step that acquires a three-dimensional image of the fundus captured by the fundus image capturing device; a deviation map generation step that inputs the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire multiple probability distributions for identifying each of multiple sites among the multiple layers and boundaries in the fundus tissue captured in the three-dimensional image, and generates a deviation map for each of the multiple sites showing a two-dimensional distribution of the deviation of the acquired probability distribution from the probability distribution when the layer or boundary to be identified is accurately identified; and a display control step that displays medical information including the deviation map on a display unit, wherein in the display control step, the control unit selectively displays the deviation map for a site associated with a macular disease specified by the user on the display unit.

[0032] However, the method for selecting the discrepancy map to be displayed on the display unit can be changed as appropriate. For example, a user may input an instruction to the fundus image processing device via an operation unit or the like to select at least one layer / boundary (or all layers / boundaries) for which the user wishes to check the discrepancy map from among the multiple layers / boundaries. The control unit may cause the display unit to display the discrepancy map of the layer / boundary selected by the user from among the multiple layers / boundaries. The instruction to select a layer / boundary may be received before the display of medical information begins. In this case, the control unit may set the discrepancy map to be initially displayed on the display unit in the macular disease mode to the discrepancy map for the selected layer / boundary. Alternatively, the instruction to select a layer / boundary may be received after the initial display of medical information begins. In this case, the discrepancy map displayed on the display unit may be switched to the discrepancy map for the selected layer / boundary.

[0033] Furthermore, when the macular disease mode is selected as the diagnosis mode, the control unit may selectively initially display on the display unit a predetermined number of discrepancy maps with the highest discrepancy ranks among the plurality of discrepancy maps, allowing the user to easily check the discrepancy maps of the layers / boundaries with the highest discrepancy ranks in the classification by the mathematical model among the plurality of layers / boundaries.

[0034] The specific method for initially displaying a predetermined number of deviation maps with the highest deviation rankings can be selected as appropriate. For example, the control unit may initially display a predetermined number of deviation maps in descending order of average deviation values. Alternatively, the control unit may initially display a predetermined number of deviation maps in descending order of maximum deviation values ​​within the two-dimensional map. Alternatively, the control unit may initially display a single deviation map with the highest deviation ranking, or may initially display multiple deviation maps in descending order of deviation rankings.

[0035] The specific method for displaying the plurality of deviation maps on the display unit can also be selected as appropriate. For example, the control unit may cause the display unit to display the plurality of deviation maps separately. Alternatively, the control unit may cause the display unit to display a composite deviation map obtained by combining the plurality of deviation maps in a state where the plurality of deviation maps are aligned. The composite deviation map may be generated by superimposing the plurality of deviation maps after expressing each deviation map in a different color.

[0036] In the display control step, the control unit may cause the display unit to display a portion of a two-dimensional tomographic image of the fundus tissue extending in the depth direction, which is included in the image region of the three-dimensional image. The control unit may cause the color of a portion of the multiple layers and boundaries shown in the two-dimensional tomographic image displayed on the display unit, for which a discrepancy map is displayed, to match the display color of the discrepancy map on the display unit. In this case, the color makes it easy to determine which portion of the multiple layers and boundaries the displayed discrepancy map corresponds to. Therefore, the user can properly determine the portion of the two-dimensional tomographic image for which the discrepancy map is displayed, even when, for example, discrepancy maps for multiple portions are displayed simultaneously.

[0037] The technique for matching the color of the region on the two-dimensional tomographic image on which the deviation map is displayed with the display color of the deviation map can be implemented without being combined with other techniques in the present disclosure. In this case, the fundus image processing device can also be expressed as follows: a fundus image processing device that processes fundus images in which multiple layers in the fundus of a test eye and boundaries between layers are included in the imaging range, wherein the control unit of the fundus image processing device is capable of executing the following steps: an image acquisition step that acquires a three-dimensional image of the fundus captured by a fundus image capturing device; a discrepancy map generation step that inputs the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of the layers and boundaries in the fundus tissue captured in the three-dimensional image, and generates a discrepancy map that shows a two-dimensional distribution of the discrepancy of the acquired probability distribution with respect to the probability distribution when the layer or boundary to be identified is accurately identified; and a display control step that displays medical information including the discrepancy map on a display unit, wherein in the display control step, the control unit displays on the display unit a two-dimensional tomographic image of a portion of the fundus tissue that is included in the image area of ​​the three-dimensional image and extends in the depth direction of the fundus tissue, and matches the color of the portion of the multiple layers and boundaries captured in the two-dimensional tomographic image where the discrepancy map is displayed with the display color of the discrepancy map.

[0038] In the display control step, the control unit may cause the display unit to display a part of a two-dimensional tomographic image of the fundus tissue extending in the depth direction, which is included in the image region of the three-dimensional image. The control unit may execute an identification display that allows the user to identify the state of the degree of disparity in the region within the two-dimensional tomographic image. In this case, the user can easily check the state of the degree of disparity even in the two-dimensional tomographic image. Therefore, the user can appropriately understand the state of the layers and boundaries shown in the two-dimensional tomographic image according to the degree of disparity.

[0039] Note that a specific method for allowing a user to identify the state of the degree of deviation in a region within the two-dimensional tomographic image can be selected as appropriate. For example, the control unit may execute an identification display to indicate a region within the two-dimensional tomographic image where the degree of deviation is equal to or greater than a threshold. The identification display may be added either inside or outside the image region of the two-dimensional tomographic image. Furthermore, the control unit may allow a user to identify the state of the degree of deviation by assigning a color of a depth corresponding to the magnitude of the degree of deviation to each region within the two-dimensional tomographic image. Furthermore, the control unit may also display an additional display indicating the magnitude of the degree of deviation (for example, a color scale bar showing a color of a depth corresponding to the magnitude of the degree of deviation) outside the image region of the two-dimensional tomographic image.

[0040] The technique for allowing a user to identify the state of the degree of deviation in a region within a two-dimensional tomographic image can be implemented without being combined with other techniques in the present disclosure. In this case, the fundus image processing device can also be expressed as follows. A fundus image processing device processes fundus images in which multiple layers in the fundus of a test eye and boundaries between layers are included in the imaging range, and the control unit of the fundus image processing device executes the following steps: an image acquisition step for acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device; a deviation acquisition step for inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of the layers and boundaries in the fundus tissue depicted in the three-dimensional image, and acquiring a distribution of deviation degrees of the acquired probability distribution from the probability distribution when the layer or boundary to be identified is accurately identified; a two-dimensional tomographic image display step for displaying on a display unit a two-dimensional tomographic image of a portion of the fundus tissue extending in the depth direction, which is included in the image area of ​​the three-dimensional image; and a deviation degree identification display step for executing an identification display to allow a user to identify the state of the deviation degree in the area within the two-dimensional tomographic image.

[0041] <Embodiment> (Device configuration) A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, this embodiment uses a mathematical model construction device 101, a fundus image processing device 1, and OCT devices (fundus image capturing devices) 10A and 10B. The mathematical model construction device 101 constructs a mathematical model by training the mathematical model using a machine learning algorithm. A program realizing the constructed mathematical model is installed in the fundus image processing device 1. The constructed mathematical model identifies (detects) at least one of the layers and boundaries (in this embodiment, a specific number of layers and boundaries) shown in the fundus image based on the input fundus image. The fundus image processing device 1 performs various processes using the results output by the mathematical model. The OCT devices 10A and 10B function as fundus image capturing devices that capture fundus images of the subject's eye (in this embodiment, tomographic images of the fundus).

[0042] As an example, a personal computer (hereinafter referred to as a "PC") is used as the mathematical model construction device 101 of this embodiment. As will be described in detail later, the mathematical model construction device 101 trains a mathematical model using data on a fundus image of the subject's eye (hereinafter referred to as a "training fundus image") acquired from the OCT device 10A and data indicating multiple layers and boundaries of the subject's eye from which the training fundus image was captured. As a result, a mathematical model is constructed. However, the device that can function as the mathematical model construction device 101 is not limited to a PC. For example, the OCT device 10A may function as the mathematical model construction device 101. Furthermore, control units of multiple devices (for example, the CPU of the PC and the CPU of the OCT device 10A) may cooperate to construct a mathematical model.

[0043] Furthermore, a PC is used as the fundus image processing device 1 of this embodiment. However, the device that can function as the fundus image processing device 1 is not limited to a PC. For example, the OCT device 10B or a server may function as the fundus image processing device 1. When the OCT device 10B functions as the fundus image processing device 1, the OCT device 10B can capture fundus images while processing the captured fundus images. Furthermore, a mobile terminal such as a tablet terminal or a smartphone may function as the fundus image processing device 1. Control units of multiple devices (for example, the CPU of a PC and the CPU of the OCT device 10B) may cooperate to perform various processes.

[0044] In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that a controller other than a CPU may be used for at least some of the various devices. For example, a GPU may be used as a controller to speed up processing.

[0045] The mathematical model construction device 101 will now be described. The mathematical model construction device 101 is installed, for example, at a manufacturer that provides the fundus image processing device 1 or a fundus image processing program to users. The mathematical model construction device 101 includes a control unit 102 that performs various control processes, and a communication I / F 105. The control unit 102 includes a CPU 103 that is a controller responsible for control, and a storage device 104 that can store programs, data, and the like. The storage device 104 stores a mathematical model construction program for executing the mathematical model construction process (see FIG. 6 ), which will be described later. The communication I / F 105 also connects the mathematical model construction device 101 to other devices (for example, the OCT device 10A and the fundus image processing device 1, etc.).

[0046] The mathematical model construction device 101 is connected to an operation unit 107 and a display device 108. The operation unit 107 is operated by a user to input various instructions to the mathematical model construction device 101. The operation unit 107 can be, for example, at least one of a keyboard, a mouse, a touch panel, etc. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 107. The display device 108 displays various images. The display device 108 can be, for example, at least one of various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.). Note that, in the present disclosure, "image" includes both still images and moving images.

[0047] The mathematical model construction device 101 can acquire fundus image data (hereinafter, sometimes simply referred to as "fundus image") from the OCT device 10A. The mathematical model construction device 101 may acquire fundus image data from the OCT device 10A by, for example, at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), etc.

[0048] The fundus image processing device 1 will now be described. The fundus image processing device 1 is installed, for example, in a facility where a diagnosis or examination is performed on a subject (for example, a hospital or a health checkup facility). The fundus image processing device 1 includes a control unit 2 that performs various control processes, and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller responsible for control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a fundus image processing program for executing fundus image processing (see FIG. 7 ), which will be described later. The fundus image processing program includes a program for realizing a mathematical model constructed by a mathematical model construction device 101. The communication I / F 5 connects the fundus image processing device 1 to other devices (for example, the OCT device 10B and the mathematical model construction device 101).

[0049] The fundus image processing device 1 is connected to an operation unit 7 and a display device (display unit) 8. As with the operation unit 107 and display device 108 described above, various devices can be used for the operation unit 7 and the display device 8.

[0050] The fundus image processing device 1 can acquire a fundus image (in this embodiment, a three-dimensional tomographic image of fundus tissue) from the OCT device 10B. The fundus image processing device 1 may acquire a fundus image from the OCT device 10B by, for example, at least one of wired communication, wireless communication, a removable storage medium (e.g., a USB memory), etc. The fundus image processing device 1 may also acquire a program or the like that realizes the mathematical model constructed by the mathematical model construction device 101 via communication or the like.

[0051] The OCT device 10 (10A, 10B) will be described. The OCT device 10 is an example of a fundus image capturing device that captures tomographic images of the fundus. In this embodiment, a case will be described in which an OCT device 10A that provides fundus images to a mathematical model construction device 101 and an OCT device 10B that provides fundus images to a fundus image processing device 1 are used. However, the number of OCT devices used is not limited to two. For example, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from multiple OCT devices. Alternatively, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from a single shared OCT device.

[0052] The OCT device 10 can capture two-dimensional and three-dimensional tomographic images of the fundus of the subject's eye. An example of the capture method will be described. As shown in FIG. 2, the OCT device 10 of this embodiment sets a plurality of linear scanning lines (scan lines) 41 at equal intervals along which a spot is scanned within a two-dimensional measurement region 40 extending in a direction intersecting the optical axis of the OCT measurement light. The OCT device 10 can capture two-dimensional tomographic images 42 (see FIG. 3) of a cross section passing through each scanning line 41 by scanning a spot of measurement light along each scanning line 41. The two-dimensional tomographic image 42 may be an averaged image generated by performing an average process on multiple two-dimensional tomographic images of the same region. Furthermore, the OCT device 10 can acquire (capture) a three-dimensional image (three-dimensional tomographic image) 43 (see FIG. 4) by arranging multiple two-dimensional tomographic images 42 captured along the multiple scanning lines 41 in a direction perpendicular to each two-dimensional image region.

[0053] Returning to the explanation of Figure 1, the OCT device 10A connected to the mathematical model construction device 101 can capture at least a two-dimensional tomographic image 42 (see Figure 3) of the fundus of the subject's eye. The OCT device 10B connected to the fundus image processing device 1 can capture a three-dimensional image 43 (see Figure 4) of the fundus of the subject's eye in addition to the above-mentioned two-dimensional tomographic image 42.

[0054] (Layer and boundary structure of the fundus) The structure of layers at the fundus of the test eye and the boundaries between adjacent layers will be described with reference to Figure 5. Figure 5 schematically shows the structure of layers and boundaries at the fundus. The upper side of Figure 5 is the surface side (superficial layer side) of the retina at the fundus. In other words, the depth of the layer boundaries increases toward the bottom of Figure 5. Also, in Figure 5, the names of boundaries between adjacent layers are enclosed in parentheses.

[0055] The layers of the fundus are as follows: From the surface (upper side of Figure 5), the fundus consists of the ILM (internal limiting membrane), NFL (nerve fiber layer), GCL (ganglion cell layer), IPL (inner plexiform layer), INL (inner nuclear layer), OPL (outer plexiform layer), ONL (outer nuclear layer), ELM (external limiting membrane), IS / OS (junction between photoreceptor inner and outer segments, sometimes called EZ (ellipsoid zone)), RPE (retinal pigment epithelium), BM (Bruch's membrane), and choroid (choroid).

[0056] In addition, there are boundaries that are likely to appear in tomographic images, such as NFL / GCL (the boundary between NFL and GCL), IPL / INL (the boundary between IPL and INL), OPL / ONL (the boundary between OPL and ONL), RPE / BM (the boundary between RPE and BM), and BM / Choroid (the boundary between BM and Choroid).

[0057] In this embodiment, when focusing on all layers of the retina, the layers from the ILM to the RPE / BM are treated as all layers. Furthermore, when diagnosing the subject's eye, attention may be paid to the GCC (Ganglion Cell Complex). The GCC includes the NFL, GCL, and IPL. As an example, in this embodiment, the layers from the ILM to the IPL / INL are treated as the GCC.

[0058] (Mathematical model construction process) 6, a description will be given of a mathematical model construction process executed by the mathematical model construction device 101. The mathematical model construction process is executed by the CPU 103 in accordance with a mathematical model construction program stored in the storage device 104.

[0059] As an example, the following describes a case where a mathematical model is constructed that analyzes an input two-dimensional tomographic image and outputs identification results for multiple specific layers / boundaries among multiple layers / boundaries shown in a fundus image. In this embodiment, the multiple specific layers / boundaries, ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM, are identified by the mathematical model. Furthermore, the mathematical model illustrated in this embodiment outputs a probability distribution for identifying specific layers / boundaries shown in the input fundus image.

[0060] In the mathematical model construction process, a mathematical model is constructed by training the mathematical model using a training data set, which includes input data (input training data) and output data (output training data).

[0061] As shown in FIG. 6, the CPU 103 acquires data of a fundus image (a two-dimensional tomographic image in this embodiment) captured by the OCT device 10A as input training data (S1). Next, the CPU 103 acquires data indicating each of a specific number of layers and boundaries of the subject's eye from which the fundus image captured in S1 was captured as output training data (S2). The output training data in this embodiment includes label data indicating the positions of a specific number of layers and boundaries captured in the fundus image. The label data may be generated, for example, by an operator operating the operation unit 107 while viewing the layers and boundaries in the fundus image.

[0062] Next, the CPU 103 executes training of the mathematical model using the training data set by a machine learning algorithm (S3). Commonly known machine learning algorithms include neural networks, random forests, boosting, and support vector machines (SVMs).

[0063] Neural networks are a method of imitating the behavior of biological neuronal networks. Examples of neural networks include feedforward neural networks, RBF networks (radial basis functions), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

[0064] Random forest is a method to generate a large number of decision trees by learning from randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained as classifiers are traced, and the results obtained from each decision tree are averaged (or voted by majority vote).

[0065] Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

[0066] SVM is a method for constructing a two-class pattern classifier using linear input elements. SVM learns the parameters of the linear input elements based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance from each data point from the training data (hyperplane separation theorem).

[0067] The mathematical model refers to a data structure for predicting the relationship between, for example, input data (in this embodiment, data of two-dimensional tomographic images similar to the input training data) and output data (in this embodiment, data of the identification results of a specific number of regions). The mathematical model is constructed by training using a training dataset. As described above, the training dataset is a set of input training data and output training data. For example, the correlation data (e.g., weights) between each input and output is updated through training.

[0068] In this embodiment, a multi-layer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data of the analysis result to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also called units) are arranged in each layer. In particular, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layer neural network, is used.

[0069] As an example, the mathematical model constructed in this embodiment outputs a probability distribution for identifying each region, in which the random variables are the coordinates (one-dimensional coordinates, two-dimensional coordinates, three-dimensional coordinates, or four-dimensional coordinates) of each of a plurality of specific layers and boundaries within a region in a fundus image (one-dimensional region, two-dimensional region, three-dimensional region, or four-dimensional region including a time axis). In this embodiment, a softmax function is applied to have the mathematical model output the probability distribution. In detail, the mathematical model constructed in S3 outputs a probability distribution in which the random variables are the coordinates of each specific layer and boundary within a one-dimensional region extending in a direction intersecting the specific layer and boundary in the two-dimensional tomographic image (in this embodiment, the depth direction, which is the direction along the optical axis of the OCT measurement light, or the up-down direction in FIG. 3). However, the specific method by which the mathematical model outputs the probability distribution for identifying a specific region can be modified as appropriate. For example, the mathematical model may output a probability distribution in which the type of specific region in the test eye is the random variable for each region (e.g., each pixel) of the input ophthalmic image. The ophthalmological image input to the mathematical model may also be a moving image.

[0070] It should be noted that other machine learning algorithms may also be used, such as generative adversarial networks (GANs) that use two competing neural networks.

[0071] The processes of S1 to S3 are repeated until the construction of the mathematical model is completed (S4: NO). When the construction of the mathematical model is completed (S4: YES), the mathematical model construction process ends. In this embodiment, a program and data for realizing the constructed mathematical model are incorporated into the fundus image processing device 1.

[0072] (Fundus image processing) 7 to 15, fundus image processing performed by the fundus image processing device 1 will be described. The fundus image processing is performed by the CPU 3 of the fundus image processing device 1 in accordance with a fundus image processing program stored in the storage device 4. The CPU 3 may perform fundus image processing, for example, when an instruction to analyze an image file of a fundus image of the subject's eye is input.

[0073] As shown in FIG. 7, the CPU 3 acquires a three-dimensional image (three-dimensional tomographic image) of the fundus of the subject's eye (S11). The three-dimensional image is captured by the OCT device 10B and acquired by the fundus image processing device 1. As described above, the three-dimensional image is composed of a combination of multiple two-dimensional images (two-dimensional tomographic images) captured by scanning the measurement light on different scan lines. The CPU 3 may also acquire a signal (e.g., an OCT signal) from the OCT device 10B that is the basis for generating the three-dimensional image, and generate the three-dimensional image based on the acquired signal.

[0074] The CPU 3 executes layer / boundary identification processing (S12). In the layer / boundary identification processing, an identification result of at least one of the layers / boundaries of the fundus tissues (in this embodiment, the ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM) shown in the three-dimensional image acquired in S11 is obtained using a mathematical model trained by a machine learning algorithm. In addition, in the layer / boundary identification processing, a discrepancy degree when at least one of the layers / boundaries is identified by the mathematical model is also obtained. The discrepancy degree is the discrepancy degree between the probability distribution for the mathematical model to identify the layers / boundaries of the fundus tissues shown in the three-dimensional image (in this embodiment, each of the multiple layers / boundaries) and the probability distribution when each layer / boundary is accurately identified by the mathematical model. The discrepancy degree often increases or decreases depending on the degree of abnormality in the layer / boundary structure, etc. Therefore, by allowing the user to understand the discrepancy degree, diagnostic efficiency can be improved.

[0075] As shown in FIG. 8, the CPU 3 extracts the Tth two-dimensional tomographic image (the initial value of T is "1") from among the multiple two-dimensional tomographic images constituting the three-dimensional image acquired in S11 (S21). FIG. 9 shows an example of the extracted two-dimensional tomographic image 42. The two-dimensional tomographic image 42 shows multiple layers and boundaries in the fundus of the subject's eye. Furthermore, multiple one-dimensional regions A1 to AN are set in the two-dimensional tomographic image 42. In this embodiment, the one-dimensional regions A1 to AN set in the two-dimensional tomographic image 42 extend along axes that intersect with the layers and boundaries to be identified. Specifically, the one-dimensional regions A1 to AN in this embodiment correspond to the respective regions of the multiple (N) A-scans constituting the two-dimensional tomographic image 42 captured by the OCT device 10.

[0076] By inputting the Tth two-dimensional tomographic image into the mathematical model, the CPU 3 acquires a probability distribution of coordinates where the Mth (initial value of M is "1") part (the Mth layer / boundary in this embodiment) exists in each of the multiple one-dimensional regions A1 to AN as a probability distribution for identifying the Mth layer / boundary (S22). The CPU 3 acquires an identification result of the Mth layer / boundary based on the probability distribution (S23). However, in S23, the identification result of the Mth layer / boundary output by the mathematical model based on the probability distribution may be acquired. The CPU 3 also acquires a deviation of the probability distribution for the Mth layer / boundary (S24). The deviation acquired in S24 is the difference between the probability distribution acquired in S22 and the probability distribution when the Mth layer / boundary is accurately identified.

[0077] In this embodiment, the entropy of the probability distribution P is calculated as the degree of deviation. The entropy is given by the following (Equation 1). The entropy H(P) takes a value in the range 0≦H(P)≦log(number of events), and the more biased the probability distribution P, the smaller the value becomes. In other words, the smaller the entropy H(P), the higher the classification accuracy of the Mth layer / boundary tends to be. H(P)=-Σplog(p)...(Math. 1)

[0078] Next, CPU 3 determines whether or not the degrees of deviation for all regions (layers and boundaries) to be classified in the Tth two-dimensional tomographic image have been acquired (S25). If the degrees of deviation for some regions have not yet been acquired (S25: NO), "1" is added to the order M of the regions to be classified (S26), and the process returns to S22, where the classification results and degrees of deviation for the next region are acquired (S22 to S24). Once the classification results and degrees of deviation for all regions have been acquired (S25: YES), CPU 3 stores the classification results and degrees of deviation acquired for each of the multiple layers and boundaries appearing in the Tth two-dimensional tomographic image in storage device 4 (S27).

[0079] Next, CPU 3 determines whether the layer / boundary identification results and deviation degrees have been acquired for all two-dimensional tomographic images that make up the three-dimensional tomographic image (S28). If the identification results and deviation degrees for some two-dimensional tomographic images have not yet been acquired (S28: NO), "1" is added to the order T of the two-dimensional tomographic image (S29), and the process returns to S21, where the identification results and deviation degrees for the next two-dimensional tomographic image are acquired (S21-S27). Once the layer / boundary identification results and deviation degrees for all two-dimensional tomographic images have been acquired (S28: YES), the process returns to fundus image processing (see FIG. 7).

[0080] Next, the CPU 3 determines whether the macular disease mode or the glaucoma mode is selected as the examination mode for examining the subject's eye (S13). In this embodiment, the user can pre-select (set) either the macular disease mode or the glaucoma mode by operating the operation unit 7. The examination mode may also be automatically selected (set) based on information such as an electronic medical record. The examination mode may also be pre-selected (set) when a three-dimensional image of the fundus of the subject's eye is captured. In this case, the examination mode may be automatically set depending on the image capturing method used by the OCT device 10B. For example, if the "macular map" is selected as the capturing method used by the OCT device 10B, the "macular disease mode" may be set as the examination mode. Alternatively, if the "glaucoma map" is selected as the capturing method, the "glaucoma mode" may be set as the examination mode. The examination mode may also be set by selecting one of the examination modes, or by selecting another option (e.g., the type of medical information to be initially displayed). If the macular disease mode is selected (S13: YES), the CPU 3 executes the macular disease process (see FIG. 10) (S14). If the macular disease mode is not selected and the glaucoma mode is selected (S13: NO), the CPU 3 executes the glaucoma process (see FIG. 14) (S15).

[0081] The macular disease processing will be described in detail with reference to FIGS. 10 to 13. First, with reference to FIG. 11, an example of an initial display screen of the analysis results for the subject's eye to be examined when the macular disease mode is selected will be described. In this embodiment, the medical information displayed on the initial display screen of the macular disease mode includes a deviation map 60. The deviation map 60 shows a probability distribution for identifying at least one of the layers and boundaries of the fundus tissue shown in the three-dimensional image acquired in S11, and a two-dimensional distribution of the deviation between the probability distribution when the layer and boundary are accurately identified (in this embodiment, the two-dimensional distribution as viewed from the direction of the optical axis of the light used to capture the three-dimensional image). As described above, the deviation map 60 is likely to show the degree of abnormality in the structure of the layer and boundary. In the deviation map 60 shown in FIGS. 11 to 13, areas with a large degree of deviation are displayed in bright colors.

[0082] When a macular disease develops in a subject's eye, it is often accompanied by abnormalities in the structure of layers and boundaries. Therefore, in conventional macular disease diagnosis, a user must check two-dimensional tomographic images 80 at various positions within the capture range of a three-dimensional tomographic image to determine whether or not a structural abnormality has occurred. In contrast, in this embodiment, when a macular disease mode is selected, a discrepancy map 60 is initially displayed, which allows the degree of abnormality in the structure of layers and boundaries to be easily determined in a two-dimensional area. Therefore, the discrepancy map 60, initially displayed on the display device 8, makes it easy for the user to appropriately determine the positions where a structural abnormality is likely to occur.

[0083] Furthermore, in this embodiment, when the macular disease mode is selected, the thickness analysis map 70 (see FIGS. 12 and 15 , details of which will be described later) is excluded from the medical information of the subject's eye to be analyzed that is initially displayed on the display device 8. In the diagnosis of macular disease, the thickness analysis map 70 may be used to additionally confirm the presence or absence of significant edema, but may not be used in the diagnosis of macular disease itself. Therefore, when the macular disease mode is selected, the thickness analysis map 70, which is not used very frequently, is excluded from the initial display, which facilitates shortening the time required for the diagnosis of macular disease.

[0084] However, in this embodiment, even when the macular disease mode is selected, the thickness analysis map 70 is displayed on the display device 8 together with the discrepancy map 60 in response to an instruction input by the user (see FIG. 12). Therefore, even when the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of layer / boundary thickness by displaying the thickness analysis map 70 as needed.

[0085] 11, the medical information displayed on the initial display screen in the macular disease mode includes a two-dimensional tomographic image 80 of a portion of the fundus tissue extending in the depth direction, which is included in the image area of ​​the three-dimensional image. Therefore, the user can appropriately grasp the state of the layers and boundaries of the subject's eye to be analyzed from the two-dimensional tomographic image 80. The method for displaying the two-dimensional tomographic image 80 will be described in detail later.

[0086] Furthermore, the display screen of the analysis results in this embodiment displays a two-dimensional fundus observation image 50 of the subject's eye being analyzed observed from the front, and a selected mode display section 55 showing the selected treatment mode.

[0087] 10, in the macular disease processing, the CPU 3 identifies a layer / boundary from among a plurality of layers / boundaries in the subject's eye to be analyzed, for which a deviation map 60 is to be initially displayed on the analysis result display screen (S31). The CPU 3 generates the deviation map 60 for the layer / boundary identified in S31 based on the deviation acquired in the layer / boundary identification processing (see FIG. 8) (S32). The CPU 3 initially displays the deviation map 60 generated in S32 on the analysis result display screen (S33).

[0088] As an example, in this embodiment, layers and boundaries within the relevant fundus tissues are associated with each of multiple macular diseases that may occur in a subject. For example, IPL / INL and OPL / ONL are associated as layers and boundaries associated with diabetic retinopathy. RPE / BM and BM are associated as layers and boundaries associated with age-related macular degeneration. RPE / BM and BM are associated as layers and boundaries associated with retinitis pigmentosa. In this embodiment, a user can pre-specify the type of macular disease they wish to treat from among multiple macular diseases. In S31, if a type of macular disease is specified, the CPU 3 identifies the region (layer / boundary) associated with the specified macular disease as the region for which the discrepancy map 60 is to be initially displayed. Therefore, the user can efficiently use the discrepancy map 60 according to the type of macular disease they wish to treat.

[0089] In this embodiment, the user can also directly select one or more layers / boundaries from among the multiple layers / boundaries for which the user wishes to check the deviation map 60 via the operation unit 7. In S31, if a layer / boundary is designated by the user, the CPU 3 identifies the designated layer / boundary as the layer / boundary for which the deviation map 60 is to be initially displayed.

[0090] Furthermore, the CPU 3 can selectively initially display on the display unit a predetermined number of deviation maps 60 with high rankings of deviation from among the plurality of deviation maps 60 for the plurality of layers / boundaries. In this case, the user can easily check the deviation maps 60 of the layers / boundaries with high deviations in the classification by the mathematical model.

[0091] 11 and 12, a discrepancy map 60 of a specific layer / boundary (RPE / BM in FIGS. 11 and 12) is displayed among multiple layer / boundaries in the three-dimensional image to be analyzed. However, as shown in FIG. 13, in steps S31 to S33, the CPU 3 can also display discrepancy maps 60 for multiple layer / boundaries. In this case, the CPU 3 may separately display discrepancy maps 60A, 60B, and 60C for multiple layer / boundaries. The CPU 3 may also display a combined discrepancy map 60X obtained by combining the discrepancy maps 60A, 60B, and 60C for multiple layer / boundaries in an aligned state. The composite deviation map 60X may be generated by superimposing the deviation maps 60A, 60B, and 60C for each layer / boundary, after expressing the deviation maps 60A, 60B, and 60C in different colors.

[0092] Next, the CPU 3 extracts a two-dimensional tomographic image 80 at a default position within the imaging range of the three-dimensional image, and initially displays it on the display device 8 (S34). The default position for displaying the two-dimensional tomographic image 80 can be set as appropriate. For example, the CPU 3 may specify, among any lines (e.g., straight lines) that can be set on the deviation map 60, the position of a line with the highest deviation as the default position for extracting the two-dimensional tomographic image 80. The CPU 3 may also initially display, on the display device 8, the two-dimensional tomographic image 80 that spreads in the depth direction of the fundus from the specified linear default position. The CPU 3 may also specify, as the default position for extracting the two-dimensional tomographic image 80, the position of a line that passes through the center of the imaging range of the three-dimensional image (e.g., a straight line that crosses the center position on the left and right of a two-dimensional frontal image obtained by viewing the three-dimensional image from the front). Furthermore, the CPU 3 may identify a characteristic part of the fundus (for example, the optic disc or the macula) by using image processing or the like, and set a linear position passing through the identified part as the default position for extracting the two-dimensional tomographic image 80 in S34. Note that the line is not limited to a straight line.

[0093] The CPU 3 displays, on the two-dimensional tomographic image, the portion (sometimes referred to as the "corresponding portion") for which the deviation map 60 is displayed among the plurality of layers and boundaries in the fundus to be analyzed (S35). Therefore, the user can easily and appropriately grasp, on the two-dimensional tomographic image 80 extending in the depth direction, the layer and boundary for which the deviation map 60 is displayed (the layer and boundary indicated by "VT" in FIG. 11).

[0094] 11 and 12, lines V indicating the positions of the plurality of layers / boundaries identified in the layer / boundary identification process (see FIG. 8) are displayed in different colors on the two-dimensional tomographic image 80. The CPU 3 indicates the layer / boundary for which the discrepancy map 60 is displayed by matching the color of the line VT of the layer / boundary for which the discrepancy map 60 is displayed on the analysis result display screen among the plurality of layers / boundaries shown in the two-dimensional tomographic image 80 with the display color of the discrepancy map 60 itself. Therefore, the user can easily determine which portion of the plurality of layers and boundaries the displayed discrepancy map 60 corresponds to by the color of the line V indicating each layer / boundary. Therefore, even when, for example, discrepancy maps 60 for each of a plurality of portions are displayed simultaneously (for example, as shown in FIG. 13), the user can appropriately determine the portion for which the discrepancy map 60 is displayed on the two-dimensional tomographic image 80.

[0095] The CPU 3 allows the user to identify the state of the degree of discrepancy in the region of the two-dimensional tomographic image 80 for the layer / boundary for which the discrepancy map 60 is displayed on the analysis result display screen (S36). Therefore, the user can easily check the state of the degree of discrepancy in the two-dimensional tomographic image 80. Therefore, the user can appropriately grasp the state of the layer / boundary shown in the two-dimensional tomographic image 80 according to the degree of discrepancy.

[0096] Note that a specific method for allowing the user to identify the state of the degree of deviation in a region within the two-dimensional tomographic image 80 can be selected as appropriate. In the examples shown in Figures 11 and 12, the CPU 3 allows the user to identify the state of the degree of deviation by executing an identification display that indicates a region 81 within the two-dimensional tomographic image 80 where the degree of deviation is equal to or greater than a threshold. Alternatively, the CPU 3 may allow the user to identify the state of the degree of deviation by applying a color of a depth corresponding to the magnitude of the degree of deviation to each region within the two-dimensional tomographic image 80.

[0097] The CPU 3 displays the position at which the two-dimensional tomographic image 80 displayed on the display device 8 was extracted on the discrepancy map 60 (S37). Therefore, the user can perform a diagnosis of the fundus tissue while checking the extraction position of the displayed two-dimensional tomographic image 80 on the discrepancy map 60. In this embodiment, as shown in FIGS. 11 and 12, a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional discrepancy map 60 when the fundus tissue is viewed from the front. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. With the above processing, the initial display of the analysis results in the macular disease mode is completed.

[0098] Next, the CPU 3 determines whether a command to display the thickness analysis map 70 has been input by the user (S39). If no command has been input (S39: NO), the process proceeds directly to the determination in S41. When a command to display the thickness analysis map 70 is input via the operation unit 7 or the like (S39: YES), the CPU 3 generates a thickness analysis map 70 showing a two-dimensional distribution of the analysis results for the thickness of a specific layer based on the layer / boundary identification results obtained in the layer / boundary identification process (see FIG. 8), and displays the thickness analysis map 70 together with the deviation map 60 on the analysis result display screen (S40). Therefore, even when the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of the thickness of the layer / boundary by displaying the thickness analysis map 70 as needed. Needless to say, the display and non-display of the thickness analysis map 70 may be switched appropriately according to a command input by the user.

[0099] In S40, the CPU 3 displays a thickness analysis map 70 showing a two-dimensional distribution of thickness analysis results for the layer / boundary for which the deviation map 60 is displayed, among the multiple layers / boundaries in the three-dimensional image to be analyzed. Therefore, the user can easily compare the deviation map 60 and the thickness analysis map 70 for the same layer / boundary, making it easier to provide effective medical treatment. Note that, as shown in FIG. 13 , when deviation maps 60 for multiple layer / boundaries are displayed on the analysis result display screen, the CPU 3 may display a thickness analysis map 70 for each of the same multiple layers / boundaries separately, or may display a thickness analysis map 70 that collectively shows all the thicknesses of the same multiple layers / boundaries.

[0100] The specific form of the thickness analysis map 70 can be selected as appropriate. For example, the CPU 3 may generate and display, as the thickness analysis map 70, a normal eye comparison map (e.g., at least one of a percentile map showing a two-dimensional distribution of the difference between the two layers and a deviation map showing a two-dimensional distribution of the deviation between the two layers) that shows the results of comparing the two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed with the two-dimensional distribution of the thickness of the same layer in a normal eye. The CPU 3 may also generate and display, as the thickness analysis map 70, a thickness map showing the two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed. The CPU 3 may display both the normal eye comparison map and the thickness map, etc., as the thickness analysis map 70.

[0101] Next, the CPU 3 determines whether the user has input an instruction to specify a position P from which the two-dimensional tomographic image 80 is to be extracted on the deviation map 60 displayed on the display device 8 (S41). If the position P has not been specified (S41: NO), the process proceeds directly to S44. As an example, in this embodiment, the user specifies the position P from which the two-dimensional tomographic image is to be extracted on the deviation map 60 displayed on the display device 8 by operating the operation unit 7 (e.g., by moving and clicking the cursor, or by specifying a position using a touch panel). Note that in this embodiment, the CPU 3 can also accept an instruction to specify the position P from which the two-dimensional tomographic image 80 is to be extracted on the two-dimensional fundus observation image 50 displayed on the display device 8. In the examples shown in FIGS. 11 and 12, the position P from which the two-dimensional tomographic image 80 is to be extracted is specified by specifying the position of a straight line. However, the method for specifying the extraction position P can also be changed. For example, the line used to specify the position P is not limited to a straight line, but may be a circular or curved line, etc.

[0102] When a position P for extracting the two-dimensional tomographic image 80 is specified on the disparity map 60 (S41: YES), the CPU 3 displays the specified position P on the disparity map 60 (S42). Therefore, the user can diagnose the fundus tissue of the subject's eye while checking the extraction position P of the displayed two-dimensional tomographic image 80 on the disparity map 60. As described above, in the example shown in FIG. 11 , a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional disparity map 60 when the fundus tissue is viewed from the front. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. Note that when the extraction position P of the two-dimensional tomographic image 80 is specified on the fundus observation image 50, the line P indicating the extraction position of the two-dimensional tomographic image 80 is similarly displayed on the disparity map 60 and the fundus observation image 50.

[0103] Furthermore, the CPU 3 extracts from the three-dimensional image a two-dimensional tomographic image 80 extending in the depth direction of the fundus tissue from the designated position P (i.e., a two-dimensional tomographic image 80 passing through the designated position P and extending in the depth direction), and displays it on the display device 8 (S43). Therefore, the user can confirm the two-dimensional distribution of the discrepancy of classification for a specific region (in this embodiment, a specific layer / boundary) using the discrepancy map 60, and then directly specify the position of the two-dimensional tomographic image 80 that the user wants to display on the discrepancy map 60. Thereafter, the process proceeds to S44.

[0104] Next, the CPU 3 determines whether the user has input an instruction to designate at least one of a plurality of macular diseases (S44). If no macular disease has been designated (S44: NO), the process proceeds directly to S46. As described above, in this embodiment, the user can display the discrepancy map 60 for the layer / boundary associated with the designated macular disease by designating at least one macular disease. Specifically, at least one of the layers / boundaries in the associated fundus tissue is associated with each of a plurality of macular diseases that may occur in the subject. When the user designates a macular disease (S44: YES), the CPU 3 generates the discrepancy map 60 for the layer / boundary associated with the designated macular disease among the plurality of layers / boundaries identified by the mathematical model, and displays it on the display device 8 (S45). Therefore, the user can easily check the discrepancy map 60 for the region associated with the predicted macular disease of the subject simply by designating the predicted macular disease. The user can also predict the macular disease of the subject by switching the designated macular disease and checking the layer-boundary discrepancy map 60 associated with each macular disease. Note that, in S45 as well, the area where the discrepancy map 60 is displayed may be displayed on the two-dimensional tomographic image 80.

[0105] Next, the CPU 3 determines whether the user has input an instruction to specify at least one of the multiple layers / boundaries for which the discrepancy map 60 is to be displayed (S46). If a layer / boundary has not been specified (S46: NO), the process proceeds directly to S48. If a layer / boundary has been specified by the user (S46: YES), the CPU 3 generates a discrepancy map 60 for the specified layer / boundary from among the multiple layers / boundaries identified by the mathematical model, and displays it on the display device 8 (S47). This allows the user to easily confirm the discrepancy map 60 for the desired layer / boundary. Note that, in S47 as well, the region for which the discrepancy map 60 is displayed may be displayed on the two-dimensional tomographic image 80.

[0106] The CPU 3 determines whether an instruction to end the display of the analysis result display screen or an instruction to change the medical treatment mode has been input (S48). If no instruction has been input (S48: NO), the process returns to S39, and the processes of S39 to S48 are repeated. If an instruction to end the display or an instruction to change the medical treatment mode has been input (S48: YES), the process returns to fundus image processing (see FIG. 7).

[0107] The glaucoma processing will be described in detail with reference to FIGS. 14 and 15. First, with reference to FIG. 15, an example of an initial display screen of the analysis results for the subject's eye to be treated when the glaucoma mode is selected will be described. In this embodiment, the medical information displayed on the initial display screen in the glaucoma mode includes a thickness analysis map 70. As described above, the thickness analysis map 70 indicates a two-dimensional distribution of the analysis results for the thickness of at least one layer in a three-dimensional image of the subject to be analyzed. When glaucoma develops in the subject's eye, the thickness of at least some layers often becomes thinner (i.e., thinning occurs). In this embodiment, the thickness analysis map 70 is initially displayed when the glaucoma mode is selected. Therefore, the thickness analysis map 70 initially displayed on the display device 8 makes it easier for the user to appropriately treat glaucoma based on the thickness of the subject's layers.

[0108] As described above, the specific aspect of the thickness analysis map 70 can be selected as appropriate. For example, the CPU 3 may generate and display a normal eye comparison map (for example, at least one of a percentile map showing a two-dimensional distribution of percentiles of the difference between the two, and a deviation map showing a two-dimensional distribution of deviation between the two) as the thickness analysis map 70. The CPU 3 may also generate and display a thickness map showing a two-dimensional distribution of layer thicknesses as the thickness analysis map 70. The CPU 3 may display both the normal eye comparison map and the thickness map as the thickness analysis map 70.

[0109] Furthermore, in this embodiment, when the glaucoma mode is selected, the deviation map 60 (see FIGS. 11 to 13) is excluded from the medical information of the subject's eye to be analyzed that is initially displayed on the display device 8. Layer thinning caused by glaucoma is often not accompanied by structural abnormalities at the layer boundaries. If no structural abnormalities at the layer boundaries occur, the deviation map 60 is unlikely to show any significant features. Therefore, when the glaucoma mode is selected, the deviation map 60, which is not often used in the diagnosis of glaucoma, is excluded from the initial display, which helps to shorten the time required for diagnosis of glaucoma.

[0110] However, in this embodiment, even when the glaucoma mode is selected, the discrepancy map 60 is displayed on the display device 8 together with the thickness analysis map 70 in response to an instruction input by the user. Therefore, even when the glaucoma mode is selected, the user can appropriately grasp the degree of abnormality in the layer / boundary structure by displaying the discrepancy map 60 as needed.

[0111] Furthermore, the medical information displayed on the initial display screen in the glaucoma mode also includes a two-dimensional tomographic image 80 of a portion of the fundus tissue extending in the depth direction, which is included in the image area of ​​the three-dimensional image. Therefore, the user can properly grasp the state of the layers and boundaries of the subject's eye to be analyzed from the two-dimensional tomographic image 80.

[0112] Furthermore, the display screen for the analysis results in the glaucoma mode also displays a two-dimensional fundus observation image 50 obtained by observing the subject's eye from the front, and a selected mode display section 55 that indicates the selected treatment mode.

[0113] 14, in the glaucoma processing, the CPU 3 identifies a layer / boundary from among a plurality of layers / boundaries in the subject's eye to be analyzed, for which a thickness analysis map 70 is to be initially displayed on the analysis result display screen (S51). The CPU 3 generates the thickness analysis map 70 for the layer / boundary identified in S51 based on the layer / boundary identification result acquired in the layer / boundary identification processing (see FIG. 8) (S52). The CPU 3 initially displays the thickness analysis map 70 generated in S52 on the analysis result display screen (S53).

[0114] As an example, in this embodiment, when the glaucoma mode is selected, the layer and boundary for initially displaying the thickness analysis map 70 are set in advance. For example, at least one of all layers of the retina (in this embodiment, layers from ILM to RPE / BM) and GCC (layers from ILM to IPL / INL) is set as the layer and boundary for initially displaying the thickness analysis map 70. Note that the layer and boundary for initially displaying the thickness analysis map 70 may be set according to an instruction input by the user.

[0115] In this embodiment, the user can also directly select one or more layer / boundaries from among the plurality of layer / boundaries for which the user wishes to check the thickness analysis map 70 via the operation unit 7. In S51, if a layer / boundary is designated by the user, the CPU 3 identifies the designated layer / boundary as the layer / boundary for which the thickness analysis map 70 is to be initially displayed.

[0116] Next, the CPU 3 extracts a two-dimensional tomographic image 80 at a default position within the imaging range of the three-dimensional image, and initially displays it on the display device 8 (S54). The method for setting the default position for displaying the two-dimensional tomographic image 80 can be selected appropriately. For example, the CPU 3 may specify the position of a line passing through the center of the imaging range of the three-dimensional image (for example, a straight line crossing the center position on the left and right on a two-dimensional front image obtained by viewing the three-dimensional image from the front) as the default position for extracting the two-dimensional tomographic image 80. The CPU 3 may also specify a characteristic site on the fundus (for example, the optic disc or macula) using image processing or the like, and set a linear position passing through the specified site as the default position for extracting the two-dimensional tomographic image 80 in S54. Note that the line is not limited to a straight line.

[0117] The CPU 3 displays, on the two-dimensional tomographic image, the portion (sometimes referred to as the "corresponding portion") for which the thickness analysis map 70 is displayed among the plurality of layers and boundaries in the fundus to be analyzed (S55). Therefore, the user can easily and appropriately grasp the layer and boundary for which the thickness analysis map 70 is displayed on the two-dimensional tomographic image 80 extending in the depth direction.

[0118] The CPU 3 displays the extracted position of the two-dimensional tomographic image 80 displayed on the display device 8 on the thickness analysis map 70 (S56). Therefore, in the glaucoma mode, the user can diagnose the fundus tissue while checking the extracted position of the displayed two-dimensional tomographic image 80 on the thickness analysis map 70. In this embodiment, as shown in FIG. 15, a line P indicating the extracted position of the two-dimensional tomographic image 80 is displayed on the two-dimensional thickness analysis map 70 when the fundus tissue is viewed from the front. In this embodiment, the line P indicating the extracted position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. With the above processing, the initial display of the analysis results in the glaucoma mode is completed.

[0119] Next, the CPU 3 determines whether an instruction to display the deviation map 60 has been input by the user (S58). If no instruction has been input (S58: NO), the process proceeds directly to the determination in S60. When an instruction to display the deviation map 60 is input via the operation unit 7 or the like (S58: YES), the CPU 3 generates a deviation map 60 showing a two-dimensional distribution of deviations for a specific layer based on the deviations acquired in the layer / boundary identification process (see FIG. 8), and displays the deviation map 60 together with the thickness analysis map 70 on the analysis result display screen (S59). Therefore, even when the glaucoma mode is selected, the user can appropriately grasp the two-dimensional distribution of deviations between layers and boundaries by displaying the deviation map 60 as needed. Needless to say, the display and non-display of the deviation map 60 may be switched appropriately according to an instruction input by the user.

[0120] In S59, the CPU 3 displays a discrepancy map 60 showing a two-dimensional distribution of discrepancies for layers / boundaries for which thickness analysis maps 70 are displayed, among multiple layers / boundaries in the three-dimensional image to be analyzed. Therefore, the user can easily compare the discrepancy map 60 and the thickness analysis map 70 for the same layer / boundary, making it easier to provide effective medical treatment. When thickness analysis maps 70 for multiple layer / boundaries are displayed on the analysis result display screen, the CPU 3 may display the discrepancy maps 60 for each of the same multiple layers / boundaries separately, or may display a composite discrepancy map 60X that combines all of the discrepancy maps 60 for the same multiple layers / boundaries.

[0121] Next, the CPU 3 determines whether the user has input an instruction to specify a position P from which a two-dimensional tomographic image 80 is extracted on the thickness analysis map 70 displayed on the display device 8 (S60). If the position P has not been specified (S60: NO), the process proceeds directly to S63. As an example, in this embodiment, the user specifies the position P from which a two-dimensional tomographic image is extracted on the thickness analysis map 70 displayed on the display device 8 by operating the operation unit 7 (e.g., by moving and clicking the cursor, or by specifying a position using a touch panel). Note that in this embodiment, the CPU 3 can also accept an instruction to specify the position P from which a two-dimensional tomographic image 80 is extracted on the two-dimensional fundus observation image 50 displayed on the display device 8. In the example shown in FIG. 15, the position P from which a two-dimensional tomographic image 80 is extracted is specified by specifying the position of a straight line. However, the method for specifying the extraction position P can also be changed. For example, the line used to specify the position P is not limited to a straight line, but may be a circular or curved line, etc.

[0122] When a position P for extracting the two-dimensional tomographic image 80 is specified on the thickness analysis map 70 (S60: YES), the CPU 3 displays the specified position P on the thickness analysis map 70 (S61). Therefore, the user can diagnose the fundus tissue of the subject's eye while checking the extraction position P of the displayed two-dimensional tomographic image 80 on the thickness analysis map 70. As described above, in the example shown in FIG. 15 , a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional thickness analysis map 70 when the fundus tissue is viewed from the front. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. Note that when the extraction position P of the two-dimensional tomographic image 80 is specified on the fundus observation image 50, the line P indicating the extraction position of the two-dimensional tomographic image 80 is similarly displayed on the thickness analysis map 70 and the fundus observation image 50.

[0123] Furthermore, the CPU 3 extracts from the three-dimensional image a two-dimensional tomographic image 80 extending in the depth direction of the fundus tissue from the specified position P (i.e., a two-dimensional tomographic image 80 passing through the specified position P and extending in the depth direction), and displays it on the display device 8 (S62). Therefore, the user can confirm the two-dimensional distribution of the thickness analysis results for a specific region (in this embodiment, a specific layer / boundary) on the thickness analysis map 70, and then directly specify the position of the two-dimensional tomographic image 80 that the user wants to display on the thickness analysis map 70. Thereafter, the process proceeds to S63.

[0124] Next, the CPU 3 determines whether the user has input an instruction to specify at least one of the multiple layers / boundaries for which a thickness analysis map 70 is to be displayed (S63). If a layer / boundary has not been specified (S63: NO), the process proceeds directly to S65. If a layer / boundary has been specified by the user (S63: YES), the CPU 3 generates a thickness analysis map 70 for the specified layer / boundary from among the multiple layers / boundaries identified by the mathematical model, and displays it on the display device 8 (S64). This allows the user to easily confirm the thickness analysis map 70 for the desired layer / boundary. Note that, in S64 as well, the region for which the thickness analysis map 70 is displayed may be displayed on the two-dimensional tomographic image 80.

[0125] The CPU 3 determines whether an instruction to end the display of the analysis result display screen or an instruction to change the medical treatment mode has been input (S65). If no instruction has been input (S65: NO), the process returns to S58, and the processes of S58 to S65 are repeated. If an instruction to end the display or an instruction to change the medical treatment mode has been input (S65: YES), the process returns to fundus image processing (see FIG. 7).

[0126] Returning to the explanation of Figure 7, when the macular disease processing (S14) or glaucoma table processing (S15) is completed, it is determined whether or not an instruction to change the treatment mode has been input (S17). If an instruction to change the treatment mode has been input (S17: YES), the process returns to S13, and display control processing of medical information (analysis results) for the changed treatment mode is executed (S14 or S15). If an instruction to change the treatment mode has not been input (S17: NO), an end instruction has been input, and the process ends.

[0127] The techniques disclosed in the above embodiments are merely examples. Therefore, the techniques exemplified in the above embodiments can be modified. First, it is also possible to execute only some of the techniques exemplified in the above embodiments. Furthermore, in the above embodiments, when an instruction to display the thickness analysis map 70 is input in the macular disease mode (S39: YES), the thickness analysis map 70 is displayed together with the discrepancy map 60 (i.e., the thickness analysis map 70 is additionally displayed). However, when an instruction to display the thickness analysis map 70 is input in the macular disease mode, the CPU 3 may switch between the thickness analysis map 70 and the discrepancy map 60 for display. In this case, the extraction position of the two-dimensional tomographic image may be specified on the switched and displayed discrepancy map 60 or on the fundus observation image 50. Furthermore, in the above embodiments, when an instruction to display the discrepancy map 60 is input in the glaucoma mode (S58: YES), the discrepancy map 60 is displayed together with the thickness analysis map 70 (i.e., the discrepancy map 60 is additionally displayed). However, when an instruction to display the deviation map 60 is input in the glaucoma mode, the CPU 3 may switch the display of the deviation map 60 with the thickness analysis map 70. In this case, the extraction position of the two-dimensional tomographic image may be specified on the thickness analysis map 70 that has been switched and displayed, or may be specified on the fundus observation image 50.

[0128] The process of acquiring a three-dimensional image in S11 of FIG. 7 is an example of an "image acquisition step." The processes of generating a deviation map in S32, S45, and S47 of FIG. 10 and S59 of FIG. 14 are an example of a "deviation map generation step." The processes of generating a thickness analysis map in S40 of FIG. 10 and S52 and S64 of FIG. 14 are an example of a "thickness analysis map generation step." The processes of displaying medical information in S14 and S15 of FIG. 7 are an example of a "display control step." The processes of accepting input of an instruction for an extraction position of the two-dimensional tomographic image 80 in S41 of FIG. 10 and S60 of FIG. 14 are an example of an "extraction position designation receiving step." The processes of extracting and displaying a two-dimensional tomographic image in S43 of FIG. 10 and S62 of FIG. 14 are an example of a "designated tomographic image display step." [Explanation of symbols]

[0129] 1 Fundus image processing device 3 CPU 4 Storage device 8 Display device 10(10A,10B) OCT device 42 Two-dimensional tomographic images 43 Three-dimensional tomographic images 60 Deviation Map 70 Thickness analysis map 80 Two-dimensional tomographic images

Claims

1. A fundus image processing device that processes a fundus image in which a plurality of layers in the fundus of an eye to be examined and boundaries between the layers are included in an imaging range, The control unit of the fundus image processing device an image acquisition step of acquiring a three-dimensional image of the fundus photographed by the fundus image photographing device; a deviation map generation step of inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue shown in the three-dimensional image, and generating a deviation map showing a two-dimensional distribution of deviation of the obtained probability distribution from a probability distribution when the layer or boundary to be identified is accurately identified; a thickness analysis map generating step of generating a thickness analysis map showing a two-dimensional distribution of an analysis result of the thickness of at least any layer in the fundus tissue shown in the three-dimensional image; a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit; is executable, In the display control step, When a macular disease mode for performing a macular disease diagnosis is selected as a diagnosis mode for the subject's eye, the discrepancy map is included in the medical information of the subject's eye that is initially displayed on the display unit, A fundus image processing device characterized in that, when a glaucoma mode for providing treatment for glaucoma is selected as the treatment mode, the thickness analysis map is included in the medical information of the subject's eye that is initially displayed on the display unit.

2. The fundus image processing device according to claim 1, In the display control step, the control unit A fundus image processing device characterized in that, when the glaucoma mode is selected as the diagnosis mode, the deviation map is excluded from the medical information of the subject's eye that is initially displayed on the display unit.

3. A fundus image processing device according to claim 1, In the display control step, the control unit A fundus image processing device characterized in that, when the macular disease mode is selected as the diagnostic mode, the thickness analysis map is excluded from the medical information of the subject's eye that is initially displayed on the display unit.

4. The fundus image processing device according to claim 1, The control unit an extraction position designation receiving step of receiving an instruction input from a user to designate an extraction position of a part of a two-dimensional tomographic image extending in a depth direction of fundus tissue from an image region of the three-dimensional image; a designated tomographic image display step of extracting a two-dimensional tomographic image extending in a depth direction of the fundus tissue from the designated extraction position when an instruction input for designating the extraction position is accepted and displaying the extracted two-dimensional tomographic image on the display unit; Further execute In the extraction position designation receiving step, When the macular disease mode is selected as the diagnosis mode, an instruction input of the extraction position is accepted on the deviation map displayed on the display unit, and A fundus image processing device characterized in that, when the glaucoma mode is selected as the diagnosis mode, it accepts input of instructions for the extraction position on the thickness analysis map displayed on the display unit.

5. The fundus image processing device according to claim 1, In the display control step, the control unit a part of a two-dimensional tomographic image of the fundus tissue extending in the depth direction included in an image region of the three-dimensional image is displayed on the display unit; A fundus image processing device characterized in that the color of the part of the plurality of layers and boundaries shown in the two-dimensional tomographic image where the deviation map is displayed is made to match the display color of the deviation map.

6. A fundus image processing program executed by a fundus image processing device that processes a fundus image in which a plurality of layers in the fundus of an eye to be examined and boundaries between the layers are included in an imaging range, The fundus image processing program is executed by a control unit of the fundus image processing device, an image acquisition step of acquiring a three-dimensional image of the fundus photographed by the fundus image photographing device; a deviation map generation step of inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue shown in the three-dimensional image, and generating a deviation map showing a two-dimensional distribution of deviation of the obtained probability distribution from a probability distribution when the layer or boundary to be identified is accurately identified; a thickness analysis map generating step of generating a thickness analysis map showing a two-dimensional distribution of an analysis result of the thickness of at least any layer in the fundus tissue shown in the three-dimensional image; a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit; The fundus image processing device can be caused to execute the above. In the display control step, When a macular disease mode for performing a macular disease diagnosis is selected as a diagnosis mode for treating the subject's eye, the discrepancy map is included in the medical information of the subject's eye that is initially displayed on the display unit, A fundus image processing program characterized in that, when a glaucoma mode for treating glaucoma is selected as the treatment mode, the thickness analysis map is included in the medical information of the test eye that is initially displayed on the display unit.

Images