Fundus image processing device and fundus image processing program

Abstract

To provide a fundus image processing device and a fundus image processing program capable of appropriately presenting, to a user, varieties of medical information obtained by processing an ophthalmologic image according to a situation.SOLUTION: A divergence map indicates a two-dimensional distribution of a degree of deviation of an acquired probability distribution with respect to the probability distribution where a layer or a boundary of an identification target in a fundus is accurately identified. A thickness analysis map indicates a two-dimensional distribution of an analysis result of a thickness of at least one layer in a fundus tissue. A control part includes the divergence map in subject's eye medical information that is initially displayed on a display part when a macular disease mode for performing diagnosis of a macular disease is selected as a diagnosis mode for performing diagnosis of the subject's eye. When a glaucoma mode for performing medical care for glaucoma is selected as the medical care mode, the control part includes the thickness analysis map in the subject's eye medical information that is initially displayed on the display unit.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a fundus image processing device and a fundus image processing program used to process a fundus image of a subject's eye. [Background technology]

[0002] In recent years, a technology has been proposed for acquiring various medical information to assist a user (e.g., a doctor) in examining the subject's eye (diagnosis, examination, treatment, etc.) by processing a fundus image of the subject's eye. For example, an image processing device described in Patent Document 1 acquires a probability distribution for identifying tissue in an ophthalmic image by inputting the ophthalmic image into a mathematical model trained by a machine learning algorithm. The image processing device acquires the degree of deviation of the acquired probability distribution from the probability distribution when the tissue is accurately identified as structural information indicating the degree of abnormality in the structure of the tissue. [Prior art documents] [Non-patent literature]

[0003] [Patent Document 1] JP 2020-18794 A Summary of the Invention [Problem to be solved by the invention]

[0004] When an abnormality such as a disease occurs in the subject's eye, changes may appear in at least one of the layers in the fundus tissue and the boundaries between adjacent layers (hereinafter, sometimes referred to as "layers / boundaries"). Therefore, if the user can properly understand the state of the layers / boundaries using various medical information such as the degree of deviation, the efficiency of the user's medical treatment may be improved. However, the manner of changes appearing in the layers / boundaries often differs depending on the condition of the subject's eye (for example, the contents of the abnormality such as a disease occurring in the subject's eye). Therefore, it is difficult to sufficiently assist the user in medical treatment by simply mechanically presenting various types of medical information obtained by processing ophthalmic images to the user.

[0005] A typical object of the present disclosure is to provide a fundus image processing device and a fundus image processing program capable of appropriately presenting various medical information obtained by processing ophthalmologic images to a user depending on the situation. [Means for solving the problem]

[0006] A fundus image processing device provided by a typical embodiment of the present disclosure is a fundus image processing device that processes a fundus image in which a plurality of layers in the fundus of a test eye and boundaries between layers are included in an imaging range, and a control unit of the fundus image processing device includes an image acquisition step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device, a discrepancy map generation step of acquiring a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue depicted in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and generating a discrepancy map showing a two-dimensional distribution of the discrepancy of the acquired probability distribution with respect to the probability distribution when the layer or boundary to be identified is accurately identified, and It is possible to execute a thickness analysis map generating step of generating a thickness analysis map showing a two-dimensional distribution of analysis results for the thickness of at least any layer in the fundus tissue shown in the image, and a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit, in which, in the display control step, when a macular disease mode in which macular disease is treated is selected as the treatment mode in which treatment is performed for the test eye, the deviation map is included in the medical information of the test eye to be initially displayed on the display unit, and when a glaucoma mode in which glaucoma is treated is selected as the treatment mode, the thickness analysis map is included in the medical information of the test eye to be initially displayed on the display unit.

[0007] A fundus image processing program provided by an exemplary embodiment of the present disclosure is a fundus image processing program executed by a fundus image processing device that processes a fundus image in which a plurality of layers in the fundus of a test eye and boundaries between layers are included in an imaging range, and the fundus image processing program is executed by a control unit of the fundus image processing device, thereby performing an image acquisition step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device, and inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue depicted in the three-dimensional image, and generating a deviation map showing a two-dimensional distribution of the deviation of the acquired probability distribution from the probability distribution when the layer or boundary to be identified is accurately identified. The fundus image processing device can be caused to execute a generation step, a thickness analysis map generation step of generating a thickness analysis map showing a two-dimensional distribution of analysis results for the thickness of at least any layer in the fundus tissue shown in the three-dimensional image, and a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit, in which, in the display control step, when a macular disease mode for treating macular disease is selected as the treatment mode for treating the test eye, the deviation map is included in the medical information of the test eye to be initially displayed on the display unit, and when a glaucoma mode for treating glaucoma is selected as the treatment mode, the thickness analysis map is included in the medical information of the test eye to be initially displayed on the display unit.

[0008] According to the fundus image processing device and fundus image processing program of the present disclosure, various medical information obtained by processing an ophthalmologic image is presented to a user appropriately according to the situation. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of a mathematical model construction device 101, a fundus image processing device 1, and OCT devices 10A and 10B. [Diagram 2]FIG. 2 is an explanatory diagram for explaining an example of a method for capturing a three-dimensional tomographic image. [Diagram 3] FIG. 2 is a diagram showing an example of a two-dimensional tomographic image 42. [Figure 4] FIG. 4 is a diagram showing an example of a three-dimensional tomographic image 43. [Diagram 5] FIG. 1 is a diagram showing a schematic diagram of the layer / boundary structure of the fundus. [Figure 6] 1 is a flowchart of a mathematical model construction process executed by a mathematical model construction device 101. [Figure 7] 4 is a flowchart of fundus image processing executed by the fundus image processing device 1. [Figure 8] 13 is a flowchart of a layer / boundary identification process executed during fundus image processing. [Figure 9] 1 is a diagram showing a schematic diagram of a relationship between a two-dimensional tomographic image 42 input to a mathematical model and one-dimensional regions A1 to AN in the two-dimensional tomographic image 42. FIG. [Figure 10] 13 is a flowchart of a process for macular diseases executed during fundus image processing. [Figure 11] FIG. 13 is a diagram showing an example of an initial display screen in a macular disease mode. [Figure 12] FIG. 13 is a diagram showing an example of a display screen in which a thickness analysis map 70 is additionally displayed in a macular disease mode. [Figure 13] FIG. 13 is a diagram showing an example of a method for displaying a plurality of deviation maps 60. [Figure 14] 13 is a flowchart of a process for glaucoma executed during fundus image processing. [Figure 15] FIG. 13 is a diagram showing an example of an initial display screen in a glaucoma mode. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Summary> The fundus image processing device exemplified in the present disclosure processes a fundus image in which a plurality of layers in the fundus of the subject eye and boundaries between layers are included in the photographing range. The control unit of the fundus image processing device can execute an image acquisition step, a deviation map generation step, a thickness analysis map generation step, and a display control step. In the image acquisition step, the control unit acquires a three-dimensional image of the fundus photographed by the fundus image photographing device. In the deviation map generation step, the three-dimensional image is input to a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue depicted in the three-dimensional image, and a deviation map is generated based on the acquired probability distribution. The deviation map shows a two-dimensional distribution of deviation of the acquired probability distribution with respect to a probability distribution when the layer or boundary to be identified is accurately identified. In the thickness analysis map generation step, the control unit generates a thickness analysis map showing a two-dimensional distribution of an analysis result for the thickness of at least one layer in the fundus tissue depicted in the three-dimensional image. In the display control step, the control unit causes the display unit to display medical information including at least one of the deviation map and the thickness analysis map. In detail, in the display control step, when a macular disease mode for treating macular disease is selected as a treatment mode for treating the subject's eye, the control unit causes the display unit to initially display the medical information of the subject's eye including the deviation map. When a glaucoma mode for treating glaucoma is selected as a treatment mode, the control unit causes the display unit to initially display the medical information of the subject's eye including the thickness analysis map.

[0011] As a premise, when there are no abnormalities in the structure of layers and boundaries, the layers and boundaries are easily identified accurately by the mathematical model, and therefore the probability distribution when identifying layers and boundaries using the mathematical model is easily biased. On the other hand, when there is an abnormality in the structure of layers and boundaries, the probability distribution is less likely to be biased. Therefore, the degree of abnormality in the structure of layers and boundaries is likely to appear in the deviation map, which shows the two-dimensional distribution of the deviation between the probability distribution when layers and boundaries are accurately identified and the actually obtained probability distribution.

[0012] Here, when a macular disease develops in the subject's eye, it is often accompanied by abnormalities in the structure of the layer / boundary. Therefore, in conventional macular disease treatment, the user needs to check two-dimensional tomographic images at various positions within the shooting range of the three-dimensional tomographic image to determine whether or not structural abnormalities have occurred. In contrast, according to the technology disclosed herein, when the macular disease mode is selected, a deviation map that makes it easy to grasp the degree of abnormality in the structure of the layer / boundary in a two-dimensional area is initially displayed. Therefore, the user can easily appropriately grasp the position where the structural abnormality is likely to occur by the deviation map initially displayed on the display unit.

[0013] Furthermore, when glaucoma develops in the subject's eye, the thickness of at least some layers often becomes thinner (i.e., thinning occurs). According to the technology of the present disclosure, when the glaucoma mode is selected, a thickness analysis map showing a two-dimensional distribution of the analysis results for the layer thickness is initially displayed. Therefore, the user can easily appropriately treat glaucoma based on the layer thickness of the subject by using the thickness analysis map initially displayed on the display unit. Through the above processing, various medical information obtained by processing the ophthalmic image is appropriately presented to the user according to the situation.

[0014] In the present disclosure, the initial display screen refers to a screen on which medical information including a map (at least one of a deviation map and a thickness analysis map) is first displayed when displaying medical information about the examinee's eye to be examined in any of the examination modes. Therefore, after the examination mode for the examinee's eye to be examined is started, some screen not including a map (e.g., a start-up screen, etc.) may be displayed before the initial display screen including the map is displayed.

[0015] A specific aspect of the thickness analysis map initially displayed when the glaucoma mode is selected can be appropriately selected. For example, in the thickness analysis map generating step, the control unit may generate a normal eye comparison map (for example, at least one of a percentile map showing a two-dimensional distribution of the difference between the two, and a deviation map showing a two-dimensional distribution of the deviation between the two) showing a comparison result between the two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed as the thickness analysis map. In addition, in the thickness analysis map generating step, the control unit may generate a thickness map showing a two-dimensional distribution of the thickness of at least one layer in the three-dimensional image to be analyzed as the thickness analysis map. The control unit may display both the normal eye comparison map and the thickness map on the display unit as the thickness analysis map.

[0016] A macular disease is a disease that causes abnormalities in the central part of the retina (the macula and the area surrounding the macula) at the fundus. For example, at least one of diabetic retinopathy, age-related macular degeneration, retinitis pigmentosa, retinal vein occlusion, macular edema, premacular membrane, macular hole, submacular hematoma, etc. may be classified as a macular disease. As an example, in this embodiment, diabetic retinopathy, age-related macular degeneration, and retinitis pigmentosa are included in the macular diseases.

[0017] The deviation degree may be output by a mathematical model. Also, the control unit may calculate the deviation degree based on the probability distribution output by the mathematical model.

[0018] The deviation may include the entropy (average information amount) of the acquired probability distribution. Entropy represents the degree of uncertainty, disorder, and chaos. In the present disclosure, the entropy of the probability distribution output when the layer / boundary is accurately identified is 0. Moreover, the more difficult it is to identify the layer / boundary, the greater the entropy. Therefore, by using the entropy of the probability distribution as the deviation, the degree of abnormality of the structure of the layer / boundary is more appropriately quantified. However, a value other than the entropy may be adopted as the deviation. For example, at least one of the standard deviation, the coefficient of variation, and the variance, which indicate the degree of dispersion of the acquired probability distribution, may be used as the deviation. The KL divergence, which is a measure of the difference between probability distributions, may be used as the deviation. Moreover, the maximum value of the acquired probability distribution may be used as the deviation.

[0019] In the present disclosure, a three-dimensional image is formed by arranging a plurality of two-dimensional images. The deviation is acquired for each of the plurality of two-dimensional images constituting the three-dimensional image, or for each pixel, each column, or each row constituting the image. As a result, for the entire fundus tissue shown in the three-dimensional image, the deviation when the layer / boundary is identified by the mathematical model is acquired. As an example, the deviation map of the present disclosure shows a two-dimensional distribution of the deviation when the fundus tissue is viewed from the front (that is, in the direction along the optical axis of the light for capturing the three-dimensional image). However, the direction in which the two-dimensional distribution of the deviation is shown in the deviation map can be changed as appropriate. In addition, when displaying the deviation map of a specific layer, the control unit may display a two-dimensional distribution such as the average value of the deviation of at least two boundaries included in the specific layer as the deviation map of the specific layer.

[0020] When the glaucoma mode is selected as the treatment mode, the control unit may exclude the deviation map from the medical information of the examinee's eye to be initially displayed on the display unit. As described above, when the examinee's eye develops glaucoma, at least some layers are often thinned. However, the thinning of layers caused by glaucoma is often not accompanied by structural abnormalities of the layers and boundaries. If no structural abnormalities of the layers and boundaries occur, the deviation map is unlikely to show any significant features. Therefore, when the glaucoma mode is selected, the deviation map that is not frequently used in the treatment of glaucoma is excluded from the initial display, which makes it easier to shorten the time required for the treatment of glaucoma.

[0021] When the glaucoma mode is selected as the diagnosis mode, the control unit may display the discrepancy map on the display unit together with the thickness analysis map or alternately with the thickness analysis map in response to an instruction input by the user. In this case, even when the glaucoma mode is selected, the user can appropriately grasp the degree of abnormality in the layer / boundary structure by displaying the discrepancy map as necessary.

[0022] When the macular disease mode is selected as the diagnosis mode, the control unit may exclude the layer thickness analysis map from the medical information of the subject's eye to be initially displayed on the display unit. In diagnosis of macular disease, the layer thickness analysis map may be used to additionally check the presence or absence of large edema, but may not be used in diagnosis of macular disease itself. Therefore, when the macular disease mode is selected, the layer thickness analysis map, which is not used very frequently, is excluded from the initial display, which makes it easier to shorten the time required for diagnosis of macular disease.

[0023] When the macular disease mode is selected as the diagnosis mode, the control unit may display the thickness analysis map on the display unit together with the deviation map or alternately with the deviation map in response to an instruction input by the user. In this case, even when the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of the thickness of the layer / boundary by displaying the thickness analysis map as necessary.

[0024] The control unit may further execute an extraction position designation receiving step and a designated tomographic image display step. In the extraction position designation receiving step, the control unit receives an instruction input from a user to designate an extraction position of a part of a two-dimensional tomographic image extending in the depth direction of the fundus tissue from the image area of ​​the three-dimensional image. In the designated tomographic image display step, when the instruction input to designate the extraction position is received, the control unit extracts a two-dimensional tomographic image extending in the depth direction of the fundus tissue from the designated extraction position and displays it on the display unit. In the extraction position designation receiving step, when the macular disease mode is selected as the medical treatment mode, an instruction input of the extraction position may be received on a deviation degree map displayed on the display unit. Also, when the glaucoma mode is selected as the medical treatment mode, an instruction input of the extraction position may be received on a thickness analysis map displayed on the display unit.

[0025] As described above, when a macular disease develops in a subject's eye, it is often accompanied by abnormalities in the structure of layers and boundaries. The discrepancy map makes it easy for the user to grasp the degree of abnormality in the structure of layers and boundaries in a two-dimensional area. Therefore, while selecting the macular disease mode, the user can specify the extraction position of the two-dimensional tomographic image on the discrepancy map, thereby appropriately specifying the extraction position of the two-dimensional tomographic image according to the degree of abnormality of the structure that is highly related to macular disease.

[0026] In addition, when glaucoma develops in the examinee's eye, the thickness of at least some layers often becomes thin. The thickness analysis map makes it easy to grasp the two-dimensional distribution of layer thicknesses. Therefore, while selecting the glaucoma mode, the user can specify the extraction position of the two-dimensional tomographic image on the thickness analysis map, thereby appropriately specifying the extraction position of the two-dimensional tomographic image according to the distribution of the thickness of the layer that is highly related to glaucoma.

[0027] In addition, a method of having the user specify the extraction position of the two-dimensional tomographic image on a map (discrepancy map or thickness analysis map) can also be appropriately selected. For example, the control unit may have the user specify a line on the two-dimensional map indicating the extraction position of the two-dimensional tomographic image (i.e., the position where the extracted two-dimensional tomographic image and the map intersect). In this case, the shape of the line may be straight or may be a shape other than a straight line (e.g., curved or circular). The size and angle of the line may also be changed according to an instruction input by the user.

[0028] A layer and a boundary in the fundus tissue may be associated with each of a plurality of macular diseases that may occur in the subject. In the deviation map generating step, the control unit may generate a deviation map for each of the plurality of sites by acquiring a plurality of probability distributions for identifying each of a plurality of sites among a plurality of layers and boundaries in the fundus tissue shown in the three-dimensional image. In the display control step, when a macular disease mode is selected, the control unit may selectively display on the display unit a deviation map for a site associated with a macular disease specified by the user.

[0029] In this case, the user can efficiently use the deviation map according to the type of macular disease to be treated. For example, the user can easily check the deviation map for the part related to the predicted macular disease of the subject by simply specifying the predicted macular disease. In addition, the user can predict the macular disease of the subject by switching the specified macular disease and checking the deviation map for the part associated with each macular disease.

[0030] In addition, a method for allowing the user to specify at least one of the multiple macular diseases can be appropriately selected. For example, the control unit may allow the user to specify at least one of multiple macular disease names prepared in advance, or may allow the user to input the name of the macular disease. The type of macular disease may be specified before the display of medical information is started. In this case, the control unit may set the deviation map to be initially displayed on the display unit in the macular disease mode as the deviation map for the site associated with the specified macular disease. Also, the type of macular disease may be specified after the initial display of medical information is started. In this case, the deviation map displayed on the display unit may be switched to the deviation map for the site associated with the specified macular disease. The control unit may allow the user to specify the macular disease in a state in which the display unit displays a deviation map for at least one of the multiple sites (which may be all deviation maps). Also, the control unit may allow the user to specify the macular disease in a state in which the display unit displays a composite deviation map obtained by aligning and combining multiple deviation maps. In this case, areas with high discrepancies can be efficiently identified regardless of the type of macular disease.

[0031] The technique for selectively displaying a deviation map for a region associated with a macular disease specified by a user on a display unit can be implemented without being combined with other techniques in the present disclosure. In this case, the fundus image processing device can be expressed as follows. A fundus image processing device that processes fundus images in which multiple layers in the fundus of a test eye and boundaries between layers are included in an imaging range, wherein a control unit of the fundus image processing device is capable of executing an image acquisition step of acquiring a three-dimensional image of the fundus captured by a fundus image capturing device, a deviation map generation step of acquiring multiple probability distributions for identifying each of multiple sites among the multiple layers and boundaries in the fundus tissue captured in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and generating a deviation map for each of the multiple sites showing a two-dimensional distribution of deviation of the acquired probability distribution relative to the probability distribution when the layer or boundary to be identified is accurately identified, and a display control step of displaying medical information including the deviation map on a display unit, wherein in the display control step, the control unit selectively displays the deviation map for a site associated with a macular disease specified by a user on the display unit.

[0032] However, the method of selecting the deviation map to be displayed on the display unit may be changed as appropriate. For example, the user may input an instruction to select at least one layer / boundary (or all layers / boundaries) for which the user wishes to check the deviation map from among the multiple layers / boundaries, to the fundus image processing device via an operation unit or the like. The control unit may display the deviation map of the layer / boundary selected by the user from among the multiple layers / boundaries on the display unit. The instruction to select the layer / boundary may be received before the display of medical information is started. In this case, the control unit may set the deviation map to be initially displayed on the display unit in the macular disease mode to the deviation map for the selected layer / boundary. Also, the instruction to select the layer / boundary may be received after the initial display of medical information is started. In this case, the deviation map displayed on the display unit may be switched to the deviation map for the selected layer / boundary.

[0033] Furthermore, when the macular disease mode is selected as the diagnosis mode, the control unit may selectively initially display on the display unit a predetermined number of discrepancy maps having high discrepancy ranks among the multiple discrepancy maps. In this case, the user can easily check the discrepancy maps of layers / boundaries having high discrepancies in the classification by the mathematical model among the multiple layers / boundaries.

[0034] A specific method for initially displaying a predetermined number of deviation maps with high deviation ranks can be selected as appropriate. For example, the control unit may initially display a predetermined number of deviation maps in descending order of average deviation values. The control unit may also initially display a predetermined number of deviation maps in descending order of maximum deviation values ​​in a two-dimensional map. The control unit may also initially display one deviation map with the highest deviation rank, or may initially display multiple deviation maps in descending order of deviation rank.

[0035] A specific method for displaying the multiple deviation maps on the display unit can also be selected as appropriate. For example, the control unit may display the multiple deviation maps separately on the display unit. The control unit may also display a composite deviation map on the display unit, which is a composite of the multiple deviation maps in a state where the multiple deviation maps are aligned. The composite deviation map may be generated by superimposing the multiple deviation maps after expressing the colors of the respective deviation maps in different colors.

[0036] In the display control step, the control unit may display on the display unit a part of a two-dimensional tomographic image that is included in the image area of ​​the three-dimensional image and extends in the depth direction of the fundus tissue. The control unit may match the color of the part where the deviation map is displayed among the multiple layers and boundaries shown in the two-dimensional tomographic image displayed on the display unit with the display color of the deviation map on the display unit. In this case, the color makes it easy to know which part of the multiple layers and boundaries the displayed deviation map is. Therefore, the user can properly know the part where the deviation map is displayed on the two-dimensional tomographic image even when, for example, the deviation maps of each of the multiple parts are displayed simultaneously.

[0037] The technique for matching the color of the region on the two-dimensional tomographic image on which the deviation map is displayed with the display color of the deviation map can be implemented without being combined with other techniques in the present disclosure. In this case, the fundus image processing device can be expressed as follows. A fundus image processing device processes fundus images in which multiple layers and boundaries between layers in the fundus of a test eye are included in an imaging range, and a control unit of the fundus image processing device is capable of executing an image acquisition step of acquiring a three-dimensional image of the fundus captured by a fundus image capturing device, a deviation map generation step of acquiring a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue captured in the three-dimensional image by inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, and generating a deviation map showing a two-dimensional distribution of deviation of the acquired probability distribution with respect to the probability distribution when the layer or boundary to be identified is accurately identified, and a display control step of displaying medical information including the deviation map on a display unit, in which the control unit displays on the display unit a part of a two-dimensional tomographic image extending in the depth direction of the fundus tissue, which is included in an image area of ​​the three-dimensional image, and matches the color of the part where the deviation map is displayed among the multiple layers and boundaries captured in the two-dimensional tomographic image with the display color of the deviation map.

[0038] In the display control step, the control unit may cause the display unit to display a part of a two-dimensional tomographic image extending in the depth direction of the fundus tissue, which is included in the image region of the three-dimensional image. The control unit may execute an identification display that allows the user to identify the state of the degree of deviation in the region in the two-dimensional tomographic image. In this case, the user can easily check the state of the degree of deviation even in the two-dimensional tomographic image. Therefore, the user can appropriately grasp the state of the layer / boundary shown in the two-dimensional tomographic image according to the degree of deviation.

[0039] A specific method for allowing a user to identify the state of the degree of deviation in an area in the two-dimensional tomographic image can be appropriately selected. For example, the control unit may execute an identification display showing an area in the two-dimensional tomographic image where the degree of deviation is equal to or greater than a threshold. The identification display may be added either inside or outside the image area of ​​the two-dimensional tomographic image. The control unit may also allow a user to identify the state of the degree of deviation by applying a color of a darkness corresponding to the magnitude of the degree of deviation to each area in the two-dimensional tomographic image. The control unit may also perform an additional display showing the magnitude of the degree of deviation (for example, a color scale bar showing a color of a darkness corresponding to the magnitude of the degree of deviation, etc.) outside the image area of ​​the two-dimensional tomographic image.

[0040] The technique for allowing a user to identify the state of the degree of deviation in an area within a two-dimensional tomographic image may be implemented without being combined with other techniques in the present disclosure. In this case, the fundus image processing device may be expressed as follows. A fundus image processing device processes fundus images in which multiple layers in the fundus of a test eye and boundaries between layers are included in an imaging range, and a control unit of the fundus image processing device executes an image acquisition step of acquiring a three-dimensional image of the fundus photographed by a fundus image photographing device, a deviation acquisition step of inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to acquire a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue depicted in the three-dimensional image, and acquiring a distribution of deviation degrees of the acquired probability distribution relative to the probability distribution when the layer or boundary to be identified is accurately identified, a two-dimensional tomographic image display step of displaying on a display unit a part of a two-dimensional tomographic image extending in the depth direction of the fundus tissue included in the image area of ​​the three-dimensional image, and a deviation degree identification display step of executing an identification display to allow a user to identify the state of the deviation degree in the area within the two-dimensional tomographic image.

[0041] <Embodiment> (Device configuration) A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, in this embodiment, a mathematical model construction device 101, a fundus image processing device 1, and OCT devices (fundus image photographing devices) 10A and 10B are used. The mathematical model construction device 101 constructs a mathematical model by training the mathematical model using a machine learning algorithm. A program for realizing the constructed mathematical model is incorporated into the fundus image processing device 1. The constructed mathematical model identifies (detects) at least one of layers and boundaries (specific layers and boundaries in this embodiment) appearing in the fundus image based on the input fundus image. The fundus image processing device 1 executes various processes using the results output by the mathematical model. The OCT devices 10A and 10B function as fundus image photographing devices for photographing a fundus image of a subject's eye (a tomographic image of the fundus in this embodiment).

[0042] As an example, a personal computer (hereinafter, referred to as "PC") is used as the mathematical model construction device 101 of this embodiment. Although details will be described later, the mathematical model construction device 101 trains a mathematical model using data of a fundus image of the subject's eye (hereinafter, referred to as "training fundus image") acquired from the OCT device 10A and data indicating a plurality of layers and boundaries of the subject's eye from which the training fundus image was taken. As a result, a mathematical model is constructed. However, a device that can function as the mathematical model construction device 101 is not limited to a PC. For example, the OCT device 10A may function as the mathematical model construction device 101. Furthermore, control units of a plurality of devices (for example, a CPU of the PC and a CPU of the OCT device 10A) may cooperate to construct a mathematical model.

[0043] In addition, a PC is used as the fundus image processing device 1 of this embodiment. However, the device capable of functioning as the fundus image processing device 1 is not limited to a PC. For example, the OCT device 10B or a server or the like may function as the fundus image processing device 1. When the OCT device 10B functions as the fundus image processing device 1, the OCT device 10B can process the fundus image while photographing it. In addition, a mobile terminal such as a tablet terminal or a smartphone may function as the fundus image processing device 1. Control units of multiple devices (for example, the CPU of the PC and the CPU of the OCT device 10B) may cooperate to perform various processes.

[0044] In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that a controller other than a CPU may be used for at least a part of the various devices. For example, a GPU may be used as a controller to speed up the processing.

[0045] The mathematical model construction device 101 will be described. The mathematical model construction device 101 is disposed, for example, at a manufacturer that provides the fundus image processing device 1 or a fundus image processing program to a user. The mathematical model construction device 101 includes a control unit 102 that performs various control processes, and a communication I / F 105. The control unit 102 includes a CPU 103 that is a controller that manages control, and a storage device 104 that can store programs, data, and the like. The storage device 104 stores a mathematical model construction program for executing a mathematical model construction process (see FIG. 6) described later. The communication I / F 105 also connects the mathematical model construction device 101 to other devices (for example, the OCT device 10A and the fundus image processing device 1, etc.).

[0046] The mathematical model construction device 101 is connected to an operation unit 107 and a display device 108. The operation unit 107 is operated by a user to input various instructions to the mathematical model construction device 101. For example, at least one of a keyboard, a mouse, a touch panel, etc. can be used for the operation unit 107. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 107. The display device 108 displays various images. For the display device 108, various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.) can be used. Note that, in the present disclosure, "image" includes both still images and moving images.

[0047] The mathematical model construction device 101 can acquire fundus image data (hereinafter, may be simply referred to as "fundus image") from the OCT device 10A. The mathematical model construction device 101 may acquire fundus image data from the OCT device 10A by at least one of wired communication, wireless communication, a removable storage medium (e.g., USB memory), and the like.

[0048] The fundus image processing device 1 will be described. The fundus image processing device 1 is disposed in, for example, a facility (such as a hospital or a medical examination facility) where a diagnosis or examination is performed on a subject. The fundus image processing device 1 includes a control unit 2 that performs various control processes, and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller that manages control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a fundus image processing program for executing fundus image processing (see FIG. 7) described later. The fundus image processing program includes a program for realizing a mathematical model constructed by a mathematical model construction device 101. The communication I / F 5 connects the fundus image processing device 1 to other devices (such as an OCT device 10B and the mathematical model construction device 101).

[0049] The fundus image processing device 1 is connected to an operation unit 7 and a display device (display unit) 8. As the operation unit 7 and the display device 8, various devices can be used, similar to the operation unit 107 and the display device 108 described above.

[0050] The fundus image processing device 1 can acquire a fundus image (in this embodiment, a three-dimensional tomographic image of fundus tissue) from the OCT device 10B. The fundus image processing device 1 may acquire a fundus image from the OCT device 10B by at least one of wired communication, wireless communication, a removable storage medium (e.g., a USB memory), etc. The fundus image processing device 1 may also acquire a program for realizing the mathematical model constructed by the mathematical model construction device 101 via communication, etc.

[0051] The OCT device 10 (10A, 10B) will be described. The OCT device 10 is an example of a fundus image capturing device that captures a tomographic image of the fundus. In this embodiment, a case will be described in which an OCT device 10A that provides a fundus image to a mathematical model construction device 101 and an OCT device 10B that provides a fundus image to a fundus image processing device 1 are used. However, the number of OCT devices used is not limited to two. For example, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from multiple OCT devices. Also, the mathematical model construction device 101 and the fundus image processing device 1 may acquire fundus images from one common OCT device.

[0052] The OCT device 10 can capture two-dimensional and three-dimensional tomographic images of the fundus of the subject's eye. An example of a capturing method will be described. As shown in FIG. 2, the OCT device 10 of this embodiment sets a plurality of linear scanning lines (scan lines) 41 for scanning spots at equal intervals within a two-dimensional measurement region 40 that spreads in a direction intersecting with the optical axis of the OCT measurement light. The OCT device 10 can capture two-dimensional tomographic images 42 (see FIG. 3) of a cross section passing through each scanning line 41 by scanning a spot of measurement light on each scanning line 41. The two-dimensional tomographic image 42 may be an average image generated by performing an average process on a plurality of two-dimensional tomographic images of the same site. In addition, the OCT device 10 can acquire (capture) a three-dimensional image (three-dimensional tomographic image) 43 (see FIG. 4) by arranging a plurality of two-dimensional tomographic images 42 captured for the plurality of scanning lines 41 in a direction perpendicularly intersecting each two-dimensional image region.

[0053] Returning to the explanation of Fig. 1, the OCT device 10A connected to the mathematical model construction device 101 can capture at least a two-dimensional tomographic image 42 (see Fig. 3) of the fundus of the subject's eye. The OCT device 10B connected to the fundus image processing device 1 can capture a three-dimensional image 43 (see Fig. 4) of the fundus of the subject's eye in addition to the above-mentioned two-dimensional tomographic image 42.

[0054] (Layer and boundary structure of the fundus) The structure of layers at the fundus of the test eye and the boundaries between adjacent layers will be described with reference to FIG. 5. FIG. 5 shows a schematic diagram of the layer-boundary structure at the fundus. The upper side of FIG. 5 is the surface side (superficial layer side) of the retina at the fundus. In other words, the depth of the layer-boundary increases toward the bottom of FIG. 5. Also, in FIG. 5, the names of the boundaries between adjacent layers are enclosed in parentheses.

[0055] The layers of the fundus are as follows: From the surface side (upper side of Fig. 5), the fundus consists of the ILM (internal limiting membrane), NFL (nerve fiber layer), GCL (ganglion cell layer), IPL (inner plexiform layer), INL (inner nuclear layer), OPL (outer plexiform layer), ONL (outer nuclear layer), ELM (external limiting membrane), IS / OS (junction between photoreceptor inner and outer segment, sometimes called EZ (Ellipsoid zone)), RPE (retinal pigment epithelium), BM (Bruch's membrane), and Choroid (choroid).

[0056] In addition, examples of boundaries that tend to appear in cross-sectional images include NFL / GCL (the boundary between the NFL and GCL), IPL / INL (the boundary between the IPL and INL), OPL / ONL (the boundary between the OPL and ONL), RPE / BM (the boundary between the RPE and BM), and BM / Choroid (the boundary between the BM and Choroid).

[0057] In this embodiment, when focusing on all layers of the retina, the layers from the ILM to the RPE / BM are treated as all layers. When diagnosing the subject's eye, attention may be paid to GCC (Ganglion Cell Complex). GCC includes NFL, GCL, and IPL. As an example, in this embodiment, the layers from the ILM to the IPL / INL are treated as GCC.

[0058] (Mathematical model construction process) 6, a description will be given of a mathematical model construction process executed by the mathematical model construction device 101. The mathematical model construction process is executed by the CPU 103 in accordance with a mathematical model construction program stored in the storage device 104.

[0059] In the following, as an example, a case is illustrated in which a mathematical model is constructed that outputs identification results of a plurality of specific layers / boundaries among a plurality of layers / boundaries shown in a fundus image by analyzing an input two-dimensional tomographic image. In this embodiment, each of the plurality of specific layers / boundaries, ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM, is identified by the mathematical model. In addition, the mathematical model illustrated in this embodiment outputs a probability distribution for identifying a specific layer / boundary shown in the input fundus image.

[0060] In the mathematical model construction process, a mathematical model is constructed by training the mathematical model using a training data set. The training data set includes input data (input training data) and output data (output training data).

[0061] As shown in FIG. 6, the CPU 103 acquires data of a fundus image (a two-dimensional tomographic image in this embodiment) captured by the OCT device 10A as input training data (S1). Next, the CPU 103 acquires data indicating each of a specific number of layers / boundaries of the subject's eye from which the fundus image captured in S1 was captured as output training data (S2). The output training data in this embodiment includes label data indicating the positions of a specific number of layers / boundaries appearing in the fundus image. The label data may be generated, for example, by an operator operating the operation unit 107 while viewing the layer / boundaries in the fundus image.

[0062] Next, the CPU 103 executes training of the mathematical model using the training data set by a machine learning algorithm (S3). Commonly known machine learning algorithms include, for example, neural networks, random forests, boosting, and support vector machines (SVMs).

[0063] Neural networks are a method to mimic the behavior of biological neural networks. Examples of neural networks include feedforward neural networks, radial basis function (RBF) networks, spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

[0064] Random forest is a method to generate a large number of decision trees by learning based on randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained in advance as classifiers are traced, and the average (or majority vote) of the results obtained from each decision tree is taken.

[0065] Boosting is a method to generate a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

[0066] SVM is a method for constructing a two-class pattern classifier using linear input elements. For example, SVM learns the parameters of the linear input elements based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance from each data point from the training data (hyperplane separation theorem).

[0067] The mathematical model refers to, for example, a data structure for predicting the relationship between input data (in this embodiment, data of two-dimensional tomographic images similar to the input training data) and output data (in this embodiment, data of the identification results of a specific number of parts). The mathematical model is constructed by training using a training dataset. As described above, the training dataset is a set of input training data and output training data. For example, correlation data (e.g., weights) between each input and output is updated through training.

[0068] In this embodiment, a multi-layered neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data of the analysis result to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also called units) are arranged in each layer. In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layered neural network, is used.

[0069] As an example, the mathematical model constructed in this embodiment outputs a probability distribution in which the coordinates (one-dimensional coordinates, two-dimensional coordinates, three-dimensional coordinates, or four-dimensional coordinates) at which each of a specific number of layers / boundaries exists in a region (one-dimensional region, two-dimensional region, three-dimensional region, or four-dimensional region including a time axis) in a fundus image are random variables, as a probability distribution for identifying each site. In this embodiment, a softmax function is applied to make the mathematical model output a probability distribution. In detail, the mathematical model constructed in S3 outputs a probability distribution in which the coordinates at which a specific layer / boundary exists in a one-dimensional region extending in a direction intersecting a specific layer / boundary in a two-dimensional tomographic image (in this embodiment, the depth direction that is the direction along the optical axis of the measurement light of the OCT, and the up-down direction in FIG. 3) are random variables. However, a specific method in which the mathematical model outputs a probability distribution for identifying a specific site can be changed as appropriate. For example, the mathematical model may output a probability distribution in which the type of a specific site in the test eye is a random variable for each region (for example, for each pixel) of the input ophthalmic image. Additionally, the ophthalmic image input to the mathematical model may be a video image.

[0070] However, other machine learning algorithms may be used. For example, generative adversarial networks (GAN) that utilize two competing neural networks may be employed as the machine learning algorithm.

[0071] The processes of S1 to S3 are repeated until the construction of the mathematical model is completed (S4: NO). When the construction of the mathematical model is completed (S4: YES), the mathematical model construction process ends. In this embodiment, a program and data for realizing the constructed mathematical model are incorporated into the fundus image processing device 1.

[0072] (Fundus image processing) 7 to 15, fundus image processing performed by the fundus image processing device 1 will be described. The fundus image processing is performed by the CPU 3 of the fundus image processing device 1 in accordance with a fundus image processing program stored in the storage device 4. The CPU 3 may perform the fundus image processing, for example, when an instruction to analyze an image file of a fundus image of a subject's eye is input.

[0073] As shown in FIG. 7, the CPU 3 acquires a three-dimensional image (three-dimensional tomographic image) of the fundus of the subject eye to be examined (S11). The three-dimensional image is photographed by the OCT device 10B and acquired by the fundus image processing device 1. As described above, the three-dimensional image is composed of a combination of a plurality of two-dimensional images (two-dimensional tomographic images) photographed by scanning the measurement light on different scan lines. The CPU 3 may acquire a signal (e.g., an OCT signal) from the OCT device 10B that is the basis for generating the three-dimensional image, and generate the three-dimensional image based on the acquired signal.

[0074] The CPU 3 executes a layer / boundary identification process (S12). In the layer / boundary identification process, an identification result of at least one of the layers / boundaries of the fundus tissue captured in the three-dimensional image acquired in S11 (ILM, NFL / GCL, IPL / INL, OPL / ONL, IS / OS, RPE / BM, and BM in this embodiment) is acquired by a mathematical model trained by a machine learning algorithm. In addition, in the layer / boundary identification process, a deviation degree when at least one of the layers / boundaries is identified by the mathematical model is also acquired. The deviation degree is a deviation degree between a probability distribution for identifying the layers / boundaries of the fundus tissue captured in the three-dimensional image (each of a plurality of layers / boundaries in this embodiment) by the mathematical model and a probability distribution when each layer / boundary is accurately identified by the mathematical model. The deviation degree often increases or decreases depending on the degree of abnormality of the structure of the layer / boundary. Therefore, by a user understanding the deviation degree, the efficiency of diagnosis can be easily improved.

[0075] As shown in FIG. 8, the CPU 3 extracts a T-th (initial value of T is "1") two-dimensional tomographic image from among the multiple two-dimensional tomographic images constituting the three-dimensional image acquired in S11 (S21). FIG. 9 shows an example of the extracted two-dimensional tomographic image 42. The two-dimensional tomographic image 42 shows multiple layers and boundaries in the fundus of the subject's eye. Furthermore, multiple one-dimensional regions A1 to AN are set in the two-dimensional tomographic image 42. In this embodiment, the one-dimensional regions A1 to AN set in the two-dimensional tomographic image 42 extend along an axis intersecting the layer and boundary to be identified. In detail, the one-dimensional regions A1 to AN in this embodiment correspond to each of the multiple (N) A-scans constituting the two-dimensional tomographic image 42 captured by the OCT device 10.

[0076] CPU3 inputs the Tth two-dimensional tomographic image into the mathematical model, and acquires a probability distribution of coordinates where an Mth (initial value of M is "1") part (Mth layer / boundary in this embodiment) exists in each of a plurality of one-dimensional regions A1 to AN as a probability distribution for identifying the Mth layer / boundary (S22). CPU3 acquires an identification result of the Mth layer / boundary based on the probability distribution (S23). However, in S23, an identification result of the Mth layer / boundary output by the mathematical model based on the probability distribution may be acquired. CPU3 also acquires a deviation of the probability distribution for the Mth layer / boundary (S24). The deviation acquired in S24 is a difference between the probability distribution acquired in S22 and the probability distribution when the Mth layer / boundary is accurately identified.

[0077] In this embodiment, the entropy of the probability distribution P is calculated as the deviation. The entropy is given by the following (Equation 1). The entropy H(P) takes a value of 0≦H(P)≦log(number of events), and the more biased the probability distribution P is, the smaller the value becomes. In other words, the smaller the entropy H(P), the higher the classification accuracy of the Mth layer / boundary tends to be. H(P)=-Σplog(p)...(Math. 1)

[0078] Next, the CPU 3 judges whether or not the deviation degrees of all parts (layers / boundaries) to be classified in the Tth two-dimensional tomographic image have been acquired (S25). If the deviation degrees for some parts have not yet been acquired (S25: NO), "1" is added to the order M of the parts to be classified (S26), and the process returns to S22, where the classification results and deviation degrees of the next part are acquired (S22 to S24). When the classification results and deviation degrees of all parts have been acquired (S25: YES), the CPU 3 stores the classification results and deviation degrees acquired for each of the layers / boundaries shown in the Tth two-dimensional tomographic image in the storage device 4 (S27).

[0079] Next, the CPU 3 determines whether the layer / boundary identification results and deviation degrees for all two-dimensional tomographic images constituting the three-dimensional tomographic image have been acquired (S28). If the identification results and deviation degrees for some two-dimensional tomographic images have not yet been acquired (S28: NO), "1" is added to the order T of the two-dimensional tomographic image (S29), and the process returns to S21, where the identification results and deviation degrees for the next two-dimensional tomographic image are acquired (S21 to S27). When the layer / boundary identification results and deviation degrees for all two-dimensional tomographic images have been acquired (S28: YES), the process returns to fundus image processing (see FIG. 7).

[0080] Next, the CPU 3 determines whether the macular disease mode or the glaucoma mode is selected as the examination mode for examining the subject's eye (S13). In this embodiment, the user can select (set) in advance either the macular disease mode or the glaucoma mode by operating the operation unit 7. The examination mode may be automatically selected (set) based on information such as an electronic medical record. The examination mode may be selected (set) in advance when a three-dimensional image of the fundus of the subject's eye is captured. In this case, the examination mode may be automatically set according to the image capturing method in the OCT device 10B. For example, if the "macular map" is selected as the capturing method by the OCT device 10B, the "macular disease mode" may be set as the examination mode, and if the "glaucoma map" is selected as the capturing method, the "glaucoma mode" may be set as the examination mode. The examination mode may be set by selecting any one of the examination modes, or may be set by selecting another option (for example, the type of medical information to be initially displayed, etc.). If the macular disease mode is selected (S13: YES), the CPU 3 executes a macular disease process (see FIG. 10) (S14). If the macular disease mode is not selected and the glaucoma mode is selected (S13: NO), the CPU 3 executes a glaucoma process (see FIG. 14) (S15).

[0081] The macular disease processing will be described in detail with reference to Figs. 10 to 13. First, with reference to Fig. 11, an example of an initial display screen of the analysis result of the examined eye to be examined when the macular disease mode is selected will be described. In this embodiment, the medical information displayed on the initial display screen of the macular disease mode includes a deviation map 60. The deviation map 60 shows a probability distribution for identifying at least one of the layers and the boundaries of the fundus tissue shown in the three-dimensional image acquired in S11, and a two-dimensional distribution of deviations of the probability distribution when the layers and the boundaries are accurately identified (in this embodiment, the two-dimensional distribution when viewed from the direction of the optical axis of the light for capturing the three-dimensional image). As described above, the deviation map 60 is likely to show the degree of abnormality in the structure of the layers and the boundaries. In the deviation map 60 shown in Figs. 11 to 13, areas with large deviations are expressed in bright colors.

[0082] Here, when a macular disease develops in the examinee's eye, it is often accompanied by an abnormality in the structure of the layer / boundary. Therefore, in conventional medical treatments for macular disease, the user needs to check the two-dimensional tomographic images 80 at various positions within the shooting range of the three-dimensional tomographic image to determine whether or not an abnormality in the structure occurs. In contrast, in this embodiment, when the macular disease mode is selected, a deviation map 60 that allows the degree of abnormality in the structure of the layer / boundary to be easily grasped in a two-dimensional area is initially displayed. Therefore, the deviation map 60 that is initially displayed on the display device 8 makes it easier for the user to appropriately grasp the position where the structural abnormality is likely to occur.

[0083] Furthermore, in this embodiment, when the macular disease mode is selected, the thickness analysis map 70 (see FIGS. 12 and 15, details of which will be described later) is excluded from the medical information of the subject's eye to be analyzed that is initially displayed on the display device 8. In the diagnosis of macular disease, the thickness analysis map 70 may be used to additionally check the presence or absence of large edema, etc., but may not be used in the diagnosis of macular disease itself. Therefore, when the macular disease mode is selected, the thickness analysis map 70, which is not used very frequently, is excluded from the initial display, which makes it easier to shorten the time required for the diagnosis of macular disease.

[0084] However, in this embodiment, even when the macular disease mode is selected, the thickness analysis map 70 is displayed on the display device 8 together with the deviation map 60 in response to an instruction input by the user (see FIG. 12). Therefore, even when the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of layer / boundary thicknesses by displaying the thickness analysis map 70 as necessary.

[0085] 11, the medical information displayed on the initial display screen of the macular disease mode includes a two-dimensional tomographic image 80 of a part of the fundus tissue extending in the depth direction, which is included in the image area of ​​the three-dimensional image. Therefore, the user can appropriately grasp the state of the layer / boundary of the subject's eye to be analyzed from the two-dimensional tomographic image 80. A method of displaying the two-dimensional tomographic image 80 will be described later in detail.

[0086] Furthermore, in this embodiment, the display screen for the analysis results displays a two-dimensional fundus observation image 50 obtained by observing the subject's eye to be analyzed from the front, and a selected mode display section 55 that indicates the selected treatment mode.

[0087] As shown in Fig. 10, in the macular disease processing, the CPU 3 identifies a layer / boundary for initially displaying a deviation map 60 on the analysis result display screen among a plurality of layers / boundaries in the subject's eye to be analyzed (S31). The CPU 3 generates the deviation map 60 for the layer / boundary identified in S31 based on the deviation acquired in the layer / boundary identification processing (see Fig. 8) (S32). The CPU 3 initially displays the deviation map 60 generated in S32 on the analysis result display screen (S33).

[0088] As an example, in this embodiment, layers and boundaries in the relevant fundus tissues are associated with each of a plurality of macular diseases that may occur in a subject. For example, IPL / INL and OPL / ONL are associated as layers and boundaries associated with diabetic retinopathy. RPE / BM and BM are associated as layers and boundaries associated with age-related macular degeneration. RPE / BM and BM are associated as layers and boundaries associated with retinitis pigmentosa. In this embodiment, a user can specify in advance the type of macular disease that the user wishes to treat among a plurality of macular diseases. In S31, when the type of macular disease is specified, the CPU 3 specifies the portion (layer / boundary) associated with the specified macular disease as the portion for initially displaying the deviation map 60. Therefore, the user can efficiently use the deviation map 60 according to the type of macular disease that the user wishes to treat.

[0089] In this embodiment, the user can also directly select one or more layer / boundaries for which the user wishes to check the deviation map 60 from among the multiple layer / boundaries via the operation unit 7. In S31, when a layer / boundary is designated by the user, the CPU 3 identifies the designated layer / boundary as the layer / boundary for which the deviation map 60 is to be initially displayed.

[0090] Furthermore, the CPU 3 can selectively initially display on the display unit a predetermined number of deviation maps 60 with high rankings of deviation among the multiple deviation maps 60 for multiple layers / boundaries. In this case, the user can easily check the deviation maps 60 of the layer / boundaries with high deviations in the classification by the mathematical model among the multiple layers / boundaries.

[0091] In the example shown in Fig. 11 and Fig. 12, a discrepancy map 60 of a specific layer-boundary (RPE / BM in Fig. 11 and Fig. 12) among a plurality of layer-boundaries in a three-dimensional image to be analyzed is displayed. However, as shown in Fig. 13, in S31 to S33, the CPU 3 can also display the discrepancy map 60 for a plurality of layer-boundaries. In this case, the CPU 3 may separately display the discrepancy maps 60A, 60B, and 60C for a plurality of layer-boundaries. The CPU 3 may also display a composite discrepancy map 60X obtained by combining the discrepancy maps 60A, 60B, and 60C for a plurality of layer-boundaries in a state of being aligned. The composite deviation map 60X may be generated by superimposing the multiple deviation maps 60A, 60B, 60C on top of each other, after expressing the deviation maps 60A, 60B, 60C for each layer / boundary in different colors.

[0092] Next, the CPU 3 extracts the two-dimensional tomographic image 80 at a default position within the imaging range of the three-dimensional image, and initially displays it on the display device 8 (S34). The default position for displaying the two-dimensional tomographic image 80 can be set as appropriate. For example, the CPU 3 may specify the position of a line that has the highest degree of deviation among any lines (e.g., straight lines) that can be set on the deviation map 60, as the default position for extracting the two-dimensional tomographic image 80. The CPU 3 may initially display the two-dimensional tomographic image 80 that spreads in the depth direction of the fundus from the specified linear default position on the display device 8. The CPU 3 may also specify the position of a line that passes through the center of the imaging range of the three-dimensional image (e.g., a straight line that crosses the center position from left to right on a two-dimensional front image obtained by viewing the three-dimensional image from the front) as the default position for extracting the two-dimensional tomographic image 80. Furthermore, the CPU 3 may specify a characteristic part (e.g., the optic disc or the macula) on the fundus by using image processing or the like, and may set a line-like position passing through the specified part as a default position for extracting the two-dimensional tomographic image 80 in S34. Note that the line is not limited to a straight line.

[0093] The CPU 3 displays the portion (sometimes called the "corresponding portion") on which the deviation map 60 is displayed among the layers and boundaries in the fundus to be analyzed on the two-dimensional tomographic image (S35). Therefore, the user can easily and appropriately grasp the layer and boundary on which the deviation map 60 is displayed (the layer and boundary indicated by "VT" in FIG. 11) on the two-dimensional tomographic image 80 extending in the depth direction.

[0094] In detail, in the examples shown in Figs. 11 and 12, the lines V indicating the positions of the plurality of layer-boundaries identified in the layer-boundary identification process (see Fig. 8) are displayed in different colors on the two-dimensional tomographic image 80. The CPU 3 indicates the layer-boundary on which the deviation map 60 is displayed by matching the color of the line VT of the layer-boundary on which the deviation map 60 is displayed on the analysis result display screen among the plurality of layer-boundaries shown in the two-dimensional tomographic image 80 with the display color of the deviation map 60 itself. Therefore, the user can easily understand which part of the plurality of layers and boundaries the displayed deviation map 60 is, by the color of the line V indicating each layer-boundary. Therefore, the user can appropriately understand the part on the two-dimensional tomographic image 80 on which the deviation map 60 is displayed, even when, for example, the deviation maps 60 of each of the plurality of parts are displayed at the same time (for example, the case shown in Fig. 13).

[0095] The CPU 3 allows the user to identify the state of the degree of deviation in the region of the two-dimensional tomographic image 80 for the layer / boundary for which the deviation map 60 is displayed on the analysis result display screen (S36). Therefore, the user can easily check the state of the degree of deviation in the two-dimensional tomographic image 80. Therefore, the user can appropriately grasp the state of the layer / boundary shown in the two-dimensional tomographic image 80 according to the degree of deviation.

[0096] A specific method for allowing the user to identify the state of the degree of deviation in the region in the two-dimensional tomographic image 80 can be appropriately selected. In the example shown in Fig. 11 and Fig. 12, the CPU 3 allows the user to identify the state of the degree of deviation by executing an identification display showing the region 81 in the two-dimensional tomographic image 80 in which the degree of deviation is equal to or greater than a threshold value. The CPU 3 may also allow the user to identify the state of the degree of deviation by applying a color of a depth according to the magnitude of the degree of deviation to each region in the two-dimensional tomographic image 80.

[0097] The CPU 3 displays the position where the two-dimensional tomographic image 80 displayed on the display device 8 is extracted on the deviation map 60 (S37). Therefore, the user can perform diagnosis of the fundus tissue while checking the extraction position of the displayed two-dimensional tomographic image 80 on the deviation map 60. In this embodiment, as shown in Figs. 11 and 12, a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional deviation map 60 when the fundus tissue is viewed from the front direction. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. With the above processing, the initial display of the analysis result in the macular disease mode is completed.

[0098] Next, the CPU 3 judges whether or not an instruction to display the thickness analysis map 70 has been input by the user (S39). If no instruction has been input (S39: NO), the process proceeds directly to the judgment of S41. When an instruction to display the thickness analysis map 70 is input via the operation unit 7 or the like (S39: YES), the CPU 3 generates a thickness analysis map 70 showing a two-dimensional distribution of the analysis result for the thickness of a specific layer based on the layer / boundary identification result acquired in the layer / boundary identification process (see FIG. 8), and additionally displays the thickness analysis map 70 together with the deviation map 60 on the analysis result display screen (S40). Therefore, even if the macular disease mode is selected, the user can appropriately grasp the two-dimensional distribution of the thickness of the layer / boundary by displaying the thickness analysis map 70 as necessary. It goes without saying that the display and non-display of the thickness analysis map 70 may be appropriately switched according to an instruction input by the user.

[0099] In S40, the CPU 3 displays a thickness analysis map 70 showing a two-dimensional distribution of the thickness analysis results for the layer / boundary for which the deviation map 60 is displayed among the multiple layers / boundaries in the three-dimensional image to be analyzed. Therefore, the user can easily compare the deviation map 60 and the thickness analysis map 70 for the same layer / boundary, and can perform effective medical treatment. Note that, as shown in FIG. 13, when the deviation maps 60 for multiple layer / boundaries are displayed on the analysis result display screen, the CPU 3 may display the thickness analysis map 70 for each of the same multiple layer / boundaries separately, or may display the thickness analysis map 70 that compiles all the thicknesses of the same multiple layer / boundaries.

[0100] The specific aspect of the thickness analysis map 70 can be appropriately selected. For example, the CPU 3 may generate and display, as the thickness analysis map 70, a normal eye comparison map (for example, at least one of a percentile map showing a two-dimensional distribution of the difference between the two, and a deviation map showing a two-dimensional distribution of the deviation between the two, etc.) showing a comparison result between the two-dimensional distribution of the thickness of at least any layer in the three-dimensional image to be analyzed and the two-dimensional distribution of the thickness of the same layer in a normal eye. The CPU 3 may also generate and display, as the thickness analysis map 70, a thickness map showing a two-dimensional distribution of the thickness of at least any layer in the three-dimensional image to be analyzed. The CPU 3 may display both the normal eye comparison map and the thickness map, etc., as the thickness analysis map 70.

[0101] Next, the CPU 3 judges whether or not an instruction to specify a position P at which the two-dimensional tomographic image 80 is extracted on the deviation map 60 displayed on the display device 8 has been input by the user (S41). If the position P has not been specified (S41: NO), the process proceeds directly to S44. As an example, in this embodiment, the user specifies the position P at which the two-dimensional tomographic image is extracted on the deviation map 60 displayed on the display device 8 by operating the operation unit 7 (for example, by moving and clicking the cursor, or by specifying a position using a touch panel, etc.). Note that in this embodiment, the CPU 3 can also accept an instruction to specify the position P at which the two-dimensional tomographic image 80 is extracted on the two-dimensional fundus observation image 50 displayed on the display device 8 in the same way. In the examples shown in FIG. 11 and FIG. 12, the position of a straight line is specified, thereby specifying the position P at which the two-dimensional tomographic image 80 is extracted. However, it is also possible to change the method for specifying the extraction position P. For example, the line used to specify the position P is not limited to a straight line, and may be a ring or curved line, etc.

[0102] When the position P for extracting the two-dimensional tomographic image 80 is specified on the deviation map 60 (S41: YES), the CPU 3 displays the specified position P on the deviation map 60 (S42). Therefore, the user can diagnose the fundus tissue of the subject eye while checking the extraction position P of the displayed two-dimensional tomographic image 80 on the deviation map 60. As described above, in the example shown in FIG. 11, a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional deviation map 60 when the fundus tissue is viewed from the front direction. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. Note that when the extraction position P of the two-dimensional tomographic image 80 is specified on the fundus observation image 50, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the deviation map 60 and the fundus observation image 50.

[0103] Furthermore, the CPU 3 extracts a two-dimensional tomographic image 80 extending in the depth direction of the fundus tissue from the specified position P (i.e., a two-dimensional tomographic image 80 passing through the specified position P and extending in the depth direction) from the three-dimensional image, and displays it on the display device 8 (S43). Therefore, the user can check the two-dimensional distribution of the discrepancy of the classification for a specific portion (in this embodiment, a specific layer / boundary) using the discrepancy map 60, and then directly specify the position of the two-dimensional tomographic image 80 to be displayed on the discrepancy map 60. Then, the process proceeds to S44.

[0104] Next, the CPU 3 determines whether or not an instruction to specify at least one of a plurality of macular diseases has been input by the user (S44). If no macular disease has been specified (S44: NO), the process proceeds directly to S46. As described above, in this embodiment, the user can display the deviation map 60 for the layer / boundary associated with the specified macular disease by specifying at least one macular disease. In detail, at least one of the layers / boundaries in the relevant fundus tissue is associated with each of a plurality of macular diseases that may occur in the subject. When the user specifies a macular disease (S44: YES), the CPU 3 generates the deviation map 60 for the layer / boundary associated with the specified macular disease among the plurality of layers / boundaries that are identified by the mathematical model, and displays it on the display device 8 (S45). Therefore, the user can easily check the deviation map 60 for the site associated with the expected macular disease of the subject simply by specifying the expected macular disease. In addition, the user can predict the subject's macular disease by checking the layer-boundary discrepancy map 60 associated with each macular disease while switching the designated macular disease. Note that, in S45 as well, the area where the discrepancy map 60 is displayed may be displayed on the two-dimensional tomographic image 80.

[0105] Next, the CPU 3 judges whether or not the user has input an instruction to specify at least one of the layers / boundaries for which the deviation map 60 is to be displayed (S46). If the layer / boundary is not specified (S46: NO), the process proceeds directly to S48. If the layer / boundary is specified by the user (S46: YES), the CPU 3 generates the deviation map 60 for the specified layer / boundary among the layers / boundaries that are to be identified by the mathematical model, and displays it on the display device 8 (S47). Therefore, the user can easily check the deviation map 60 for the desired layer / boundary. Note that, in S47 as well, the area for which the deviation map 60 is displayed may be displayed on the two-dimensional tomographic image 80.

[0106] The CPU 3 judges whether an instruction to end the display of the analysis result display screen or an instruction to change the diagnosis mode has been input (S48). If no instruction has been input (S48: NO), the process returns to S39, and the processes of S39 to S48 are repeated. If an instruction to end the display or an instruction to change the diagnosis mode have been input (S48: YES), the process returns to fundus image processing (see FIG. 7).

[0107] The glaucoma processing will be described in detail with reference to Figs. 14 to 15. First, with reference to Fig. 15, an example of an initial display screen of the analysis result for the examinee's eye to be examined when the glaucoma mode is selected will be described. In this embodiment, the medical information displayed on the initial display screen of the glaucoma mode includes a thickness analysis map 70. As described above, the thickness analysis map 70 shows a two-dimensional distribution of the analysis result for the thickness of at least one layer in the three-dimensional image to be analyzed. When glaucoma develops in the examinee's eye, the thickness of at least some layers often becomes thin (i.e., thinning occurs). In this embodiment, the thickness analysis map 70 is initially displayed when the glaucoma mode is selected. Therefore, the user can more easily perform appropriate glaucoma treatment based on the thickness of the examinee's layers by using the thickness analysis map 70 initially displayed on the display device 8.

[0108] As described above, the specific aspect of the thickness analysis map 70 can be appropriately selected. For example, the CPU 3 may generate and display a normal eye comparison map (for example, at least one of a percentile map showing a two-dimensional distribution of the percentile of the difference between the two, and a deviation map showing a two-dimensional distribution of the deviation between the two, etc.) as the thickness analysis map 70. The CPU 3 may also generate and display a thickness map showing a two-dimensional distribution of the layer thickness as the thickness analysis map 70. The CPU 3 may display both the normal eye comparison map and the thickness map as the thickness analysis map 70.

[0109] Furthermore, in this embodiment, when the glaucoma mode is selected, the deviation map 60 (see FIGS. 11 to 13) is excluded from the medical information of the subject's eye to be analyzed that is initially displayed on the display device 8. Thinning of layers caused by glaucoma is often not accompanied by structural abnormalities of the layers and boundaries. If no structural abnormalities of the layers and boundaries occur, the deviation map 60 is unlikely to show any significant features. Therefore, when the glaucoma mode is selected, the deviation map 60, which is not often used in the diagnosis of glaucoma, is excluded from the initial display, which makes it easier to shorten the time required for diagnosis of glaucoma.

[0110] However, in this embodiment, even when the glaucoma mode is selected, the discrepancy map 60 is displayed on the display device 8 together with the thickness analysis map 70 in response to an instruction input by the user. Therefore, even when the glaucoma mode is selected, the user can appropriately grasp the degree of abnormality in the layer-boundary structure by displaying the discrepancy map 60 as necessary.

[0111] In addition, the medical information displayed on the initial display screen in the glaucoma mode also includes a part of a two-dimensional tomographic image 80 extending in the depth direction of the fundus tissue, which is included in the image area of ​​the three-dimensional image. Therefore, the user can properly grasp the state of the layer / boundary of the subject's eye to be analyzed from the two-dimensional tomographic image 80.

[0112] Furthermore, the display screen for the analysis results in the glaucoma mode also displays a two-dimensional fundus observation image 50 obtained by observing the subject's eye to be analyzed from the front, and a selected mode display section 55 indicating the selected examination mode.

[0113] As shown in Fig. 14, in the glaucoma processing, the CPU 3 identifies a layer / boundary for initially displaying a thickness analysis map 70 on the analysis result display screen among a plurality of layers / boundaries in the subject's eye to be analyzed (S51). The CPU 3 generates a thickness analysis map 70 for the layer / boundary identified in S51 based on the layer / boundary identification result acquired in the layer / boundary identification processing (see Fig. 8) (S52). The CPU 3 initially displays the thickness analysis map 70 generated in S52 on the analysis result display screen (S53).

[0114] As an example, in this embodiment, when the glaucoma mode is selected, the layer / boundary for initially displaying the thickness analysis map 70 is set in advance. For example, at least one of the entire layer of the retina (in this embodiment, the layer from the ILM to the RPE / BM) and the GCC (the layer from the ILM to the IPL / INL) is set as the layer / boundary for initially displaying the thickness analysis map 70. Note that the layer / boundary for initially displaying the thickness analysis map 70 may be set according to an instruction input by the user.

[0115] In this embodiment, the user can also directly select one or more layer / boundaries for which the user wishes to check the thickness analysis map 70 from among the multiple layer / boundaries via the operation unit 7. In S51, when a layer / boundary is designated by the user, the CPU 3 specifies the designated layer / boundary as the layer / boundary for which the thickness analysis map 70 is to be initially displayed.

[0116] Next, the CPU 3 extracts a two-dimensional tomographic image 80 at a default position within the imaging range of the three-dimensional image, and initially displays it on the display device 8 (S54). The method of setting the default position for displaying the two-dimensional tomographic image 80 can be appropriately selected. For example, the CPU 3 may specify the position of a line passing through the center of the imaging range of the three-dimensional image (e.g., a straight line crossing the center position from left to right on a two-dimensional front image obtained by viewing the three-dimensional image from the front) as the default position for extracting the two-dimensional tomographic image 80. The CPU 3 may also specify a characteristic site on the fundus (e.g., the optic disc or macula) using image processing or the like, and set a linear position passing through the specified site as the default position for extracting the two-dimensional tomographic image 80 in S54. The line is not limited to a straight line.

[0117] The CPU 3 displays the part (sometimes called the "corresponding part") on which the thickness analysis map 70 is displayed among the multiple layers and boundaries in the fundus to be analyzed on the two-dimensional tomographic image (S55). Therefore, the user can easily and appropriately grasp the layer and boundary on which the thickness analysis map 70 is displayed on the two-dimensional tomographic image 80 extending in the depth direction.

[0118] The CPU 3 displays the position where the two-dimensional tomographic image 80 displayed on the display device 8 is extracted on the thickness analysis map 70 (S56). Therefore, in the glaucoma mode, the user can perform diagnosis of the fundus tissue while checking the extraction position of the displayed two-dimensional tomographic image 80 on the thickness analysis map 70. In this embodiment, as shown in FIG. 15, a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional thickness analysis map 70 when the fundus tissue is viewed from the front. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. With the above processing, the initial display of the analysis result in the glaucoma mode is completed.

[0119] Next, the CPU 3 judges whether or not an instruction to display the deviation map 60 has been input by the user (S58). If no instruction has been input (S58: NO), the process proceeds directly to the judgment of S60. If an instruction to display the deviation map 60 is input via the operation unit 7 or the like (S58: YES), the CPU 3 generates a deviation map 60 showing a two-dimensional distribution of deviations for a specific layer based on the deviations acquired in the layer / boundary identification process (see FIG. 8), and additionally displays the map 60 together with the thickness analysis map 70 on the analysis result display screen (S59). Therefore, even if the glaucoma mode is selected, the user can appropriately grasp the two-dimensional distribution of deviations of the layer / boundary by displaying the deviation map 60 as necessary. It goes without saying that the display and non-display of the deviation map 60 may be appropriately switched according to an instruction input by the user.

[0120] In S59, the CPU 3 displays the deviation map 60 showing the two-dimensional distribution of deviations for the layer / boundary on which the thickness analysis map 70 is displayed among the multiple layers / boundaries in the three-dimensional image to be analyzed. Therefore, the user can easily compare the deviation map 60 and the thickness analysis map 70 for the same layer / boundary, and can perform effective medical treatment. When the thickness analysis maps 70 for multiple layer / boundaries are displayed on the analysis result display screen, the CPU 3 may display the deviation map 60 for each of the same multiple layers / boundaries separately, or may display a composite deviation map 60X obtained by combining all the deviation maps 60 for the same multiple layer / boundaries.

[0121] Next, the CPU 3 judges whether or not an instruction for designating a position P for extracting a two-dimensional tomographic image 80 on the thickness analysis map 70 displayed on the display device 8 has been input by the user (S60). If the position P has not been designated (S60: NO), the process proceeds directly to S63. As an example, in this embodiment, the user designates the position P for extracting a two-dimensional tomographic image on the thickness analysis map 70 displayed on the display device 8 by operating the operation unit 7 (for example, by moving and clicking the cursor, or by designating a position using a touch panel, etc.). Note that in this embodiment, the CPU 3 can also accept an instruction for designating the position P for extracting a two-dimensional tomographic image 80 on the two-dimensional fundus observation image 50 displayed on the display device 8 in the same manner. In the example shown in FIG. 15, the position of a straight line is designated to designate the position P for extracting a two-dimensional tomographic image 80. However, it is also possible to change the method for designating the extraction position P. For example, the line used for designating the position P is not limited to a straight line, and may be a ring or curved line, etc.

[0122] When the position P for extracting the two-dimensional tomographic image 80 is specified on the thickness analysis map 70 (S60: YES), the CPU 3 displays the specified position P on the thickness analysis map 70 (S61). Therefore, the user can diagnose the fundus tissue of the subject eye while checking the extraction position P of the displayed two-dimensional tomographic image 80 on the thickness analysis map 70. As described above, in the example shown in FIG. 15, a line P indicating the extraction position of the two-dimensional tomographic image 80 is displayed on the two-dimensional thickness analysis map 70 when the fundus tissue is viewed from the front direction. In this embodiment, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the two-dimensional fundus observation image 50. Note that when the extraction position P of the two-dimensional tomographic image 80 is specified on the fundus observation image 50, the line P indicating the extraction position of the two-dimensional tomographic image 80 is also displayed on the thickness analysis map 70 and the fundus observation image 50.

[0123] Furthermore, the CPU 3 extracts a two-dimensional tomographic image 80 extending in the depth direction of the fundus tissue from the specified position P (i.e., a two-dimensional tomographic image 80 passing through the specified position P and extending in the depth direction) from the three-dimensional image, and displays it on the display device 8 (S62). Therefore, the user can confirm the two-dimensional distribution of the thickness analysis results for a specific site (in this embodiment, a specific layer / boundary) using the thickness analysis map 70, and then directly specify the position of the two-dimensional tomographic image 80 to be displayed on the thickness analysis map 70. Then, the process proceeds to S63.

[0124] Next, the CPU 3 judges whether or not the user has input an instruction to specify at least one of the layers / boundaries for which the thickness analysis map 70 is to be displayed (S63). If the layer / boundary is not specified (S63: NO), the process proceeds directly to S65. If the layer / boundary is specified by the user (S63: YES), the CPU 3 generates a thickness analysis map 70 for the specified layer / boundary among the layers / boundaries that are to be identified by the mathematical model, and displays it on the display device 8 (S64). Therefore, the user can easily check the thickness analysis map 70 for the desired layer / boundary. Note that, in S64, the area on which the thickness analysis map 70 is displayed may also be displayed on the two-dimensional tomographic image 80.

[0125] The CPU 3 judges whether an instruction to end the display of the analysis result display screen or an instruction to change the diagnosis mode has been input (S65). If no instruction has been input (S65: NO), the process returns to S58, and the processes of S58 to S65 are repeated. If an instruction to end the display or an instruction to change the diagnosis mode have been input (S65: YES), the process returns to fundus image processing (see FIG. 7).

[0126] Returning to the explanation of FIG. 7, when the macular disease processing (S14) or the glaucoma table processing (S15) is completed, it is determined whether or not an instruction to change the treatment mode has been input (S17). If an instruction to change the treatment mode has been input (S17: YES), the process returns to S13, and display control processing of medical information (analysis results) for the changed treatment mode is executed (S14 or S15). If an instruction to change the treatment mode has not been input (S17: NO), an end instruction has been input, and the process ends.

[0127] The techniques disclosed in the above embodiment are merely examples. Therefore, the techniques exemplified in the above embodiment can be modified. First, it is possible to execute only a part of the techniques exemplified in the above embodiment. In the above embodiment, when an instruction to display the thickness analysis map 70 is input in the macular disease mode (S39: YES), the thickness analysis map 70 is displayed together with the deviation map 60 (i.e., the thickness analysis map 70 is additionally displayed). However, when an instruction to display the thickness analysis map 70 is input in the macular disease mode, the CPU 3 may switch the thickness analysis map 70 to the deviation map 60 and display it. In this case, the extraction position of the two-dimensional tomographic image may be specified on the deviation map 60 that is switched and displayed, or may be specified on the fundus observation image 50. In the above embodiment, when an instruction to display the deviation map 60 is input in the glaucoma mode (S58: YES), the deviation map 60 is displayed together with the thickness analysis map 70 (i.e., the deviation map 60 is additionally displayed). However, when an instruction to display the deviation map 60 is input in the glaucoma mode, the CPU 3 may switch the deviation map 60 to the thickness analysis map 70. In this case, the extraction position of the two-dimensional tomographic image may be specified on the thickness analysis map 70 or on the fundus observation image 50.

[0128] The process of acquiring a three-dimensional image in S11 of FIG. 7 is an example of an "image acquiring step". The processes of generating a deviation map in S32, S45, and S47 of FIG. 10 and S59 of FIG. 14 are an example of a "deviation map generating step". The processes of generating a thickness analysis map in S40 of FIG. 10 and S52 and S64 of FIG. 14 are an example of a "thickness analysis map generating step". The processes of displaying medical information in S14 and S15 of FIG. 7 are an example of a "display control step". The processes of accepting an instruction input of an extraction position of the two-dimensional tomographic image 80 in S41 of FIG. 10 and S60 of FIG. 14 are an example of an "extraction position designation accepting step". The processes of extracting and displaying a two-dimensional tomographic image in S43 of FIG. 10 and S62 of FIG. 14 are an example of a "designated tomographic image display step". [Explanation of symbols]

[0129] 1 Fundus image processing device 3 CPU 4 Storage device 8 Display device 10(10A,10B) OCT device 42 2D Tomographic Images 43 3D Tomographic Images 60 Deviation Map 70 Thickness Analysis Map 80 2D tomographic images

Claims

1. A fundus image processing device that processes a fundus image in which a plurality of layers in the fundus of an eye to be examined and boundaries between the layers are included in an imaging range, The control unit of the fundus image processing device an image acquisition step of acquiring a three-dimensional image of the fundus photographed by the fundus image photographing device; a deviation map generation step of inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying at least one of a layer and a boundary in the fundus tissue shown in the three-dimensional image, and generating a deviation map showing a two-dimensional distribution of deviation of the obtained probability distribution from a probability distribution when the layer or boundary to be identified is accurately identified; a thickness analysis map generating step of generating a thickness analysis map showing a two-dimensional distribution of an analysis result of the thickness of at least any layer in the fundus tissue shown in the three-dimensional image; a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit; is executable, In the display control step, When a macular disease mode for performing a macular disease diagnosis is selected as a diagnosis mode for treating the subject's eye, the discrepancy map is included in the medical information of the subject's eye that is initially displayed on the display unit, A fundus image processing device characterized in that, when a glaucoma mode for performing glaucoma treatment is selected as the treatment mode, the thickness analysis map is included in the medical information of the subject's eye that is initially displayed on the display unit.

2. The fundus image processing device according to claim 1, In the display control step, the control unit A fundus image processing device characterized in that, when the glaucoma mode is selected as the diagnosis mode, the deviation map is excluded from the medical information of the subject's eye that is initially displayed on the display unit.

3. A fundus image processing device according to claim 1, In the display control step, the control unit A fundus image processing device characterized in that, when the macular disease mode is selected as the diagnostic mode, the thickness analysis map is excluded from the medical information of the subject's eye that is initially displayed on the display unit.

4. The fundus image processing device according to claim 1, The control unit An extraction position specifying reception step of receiving an instruction input from a user for specifying an extraction position of a partial two-dimensional tomographic image that extends in the depth direction of the fundus tissue among the image regions of the three-dimensional image; A specified tomographic image display step of extracting a two-dimensional tomographic image that extends in the depth direction of the fundus tissue from the specified extraction position and displaying it on the display unit when an instruction input for specifying the extraction position is received; further executed, In the extraction position specifying reception step, when the macular disease mode is selected as the diagnosis mode, an instruction input of the extraction position is received on the deviation map displayed on the display unit, and when the glaucoma mode is selected as the diagnosis mode, an instruction input of the extraction position is received on the thickness analysis map displayed on the display unit. A fundus image processing apparatus characterized by this.

5. A fundus image processing apparatus according to claim 1, In the display control step, the control unit displays a partial two-dimensional tomographic image that extends in the depth direction of the fundus tissue included in the image region of the three-dimensional image on the display unit, and A fundus image processing apparatus characterized in that the color of a part where the deviation map is displayed among a plurality of layers and boundaries shown in the two-dimensional tomographic image is made to match the display color of the deviation map.

6. A fundus image processing program executed by a fundus image processing apparatus that processes a fundus image in which a plurality of layers in the fundus of the eye to be examined and the boundaries between the layers are included in the imaging range, By the fundus image processing program being executed by the control unit of the fundus image processing apparatus, An image acquisition step of acquiring a three-dimensional image of the fundus taken by a fundus image photographing apparatus; By inputting the three-dimensional image into a mathematical model trained by a machine learning algorithm, a probability distribution for identifying at least one of the layers and boundaries in the fundus tissue shown in the three-dimensional image is obtained, and the deviation of the obtained probability distribution from the probability distribution when the layer or boundary to be identified is accurately identified. A deviation map generation step of generating a deviation map showing a two-dimensional distribution of degrees; A thickness analysis map generation step of generating a thickness analysis map showing a two-dimensional distribution of the analysis results regarding the thickness of at least one of the layers in the fundus tissue shown in the three-dimensional image; a display control step of displaying medical information including at least one of the deviation map and the thickness analysis map on a display unit; The fundus image processing device can be caused to execute the above. In the display control step, When a macular disease mode for performing a macular disease diagnosis is selected as a diagnosis mode for treating the subject's eye, the discrepancy map is included in the medical information of the subject's eye that is initially displayed on the display unit, A fundus image processing program characterized in that, when a glaucoma mode for treating glaucoma is selected as the treatment mode, the thickness analysis map is included in the medical information of the test eye that is initially displayed on the display unit.

Images