Information processing device, information processing method, and program

Abstract

To provide means for easily grasping the relation between an abnormal region detected from information on the shape of a retinal layer and an abnormal region detected based on a tomographic image.SOLUTION: An information processing device comprises: a tomographic image acquisition unit which acquires a tomographic image of the eyeground of a subject eye; a shape abnormality information acquisition unit which acquires shape abnormality information being information on a portion having an abnormal shape in the retinal layer of the subject eye on the basis of the tomographic image; a tomographic abnormality information acquisition unit which based on a tomographic abnormal region being an abnormal region in the tomographic image, acquires tomographic abnormality information being information on a portion having the tomographic abnormal region in the retinal layer of the subject eye; and a display control unit which performs control to display at least a part of the shape abnormality information and at least a part of the tomographic abnormality information in a mutually distinguishable manner.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Fundus tomography imaging devices such as optical coherence tomography (OCT) devices can three-dimensionally observe the state inside the retinal layer. Tomography imaging devices such as OCT imaging devices have attracted attention in recent years because they are useful for more accurately diagnosing diseases. In addition, there is also a technology for detecting abnormal regions that may be related to diseases or the like from fundus tomography images. This technology has also attracted attention in recent years because it is useful for assisting in the diagnosis of diseases. Patent Document 1 discloses a technique for superimposing and displaying information on the shape of the retinal layer, such as layer thickness and curvature, in a tomography image on a fundus image. In Patent Document 1, a two-dimensional map (hereinafter also referred to as a shape map) projected onto a plane along the fundus (hereinafter also referred to as a fundus parallel plane) of information on the shape of the retinal layer is obtained, and a two-dimensional map obtained by imaging an abnormal region in the shape map is also obtained. In addition, Patent Document 2 discloses a technique for superimposing and displaying an abnormal region in a tomography image on a shape map.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] The abnormal regions detected from information on the shape of the retinal layer as disclosed in Patent Document 1 and the abnormal regions detected based on the tomographic image as disclosed in Patent Document 2 are abnormal regions detected through different processes. Therefore, means for easily grasping the relationship between these abnormal regions are required. The present invention has been made to solve the above problems.

Means for Solving the Problems

[0005] That is, an information processing apparatus according to an aspect of the present invention includes a tomographic image acquisition unit that acquires a tomographic image of the fundus of an eye to be examined, a shape abnormality information acquisition unit that acquires shape abnormality information, which is information about a portion having an abnormal shape in the retinal layer of the eye to be examined, based on the tomographic image, a tomographic abnormality information acquisition unit that acquires tomographic abnormality information, which is information about a portion having the tomographic abnormality region in the retinal layer of the eye to be examined, based on the tomographic abnormality region that is an abnormal region in the tomographic image, and a display control unit that performs control to display at least a part of the shape abnormality information and at least a part of the tomographic abnormality information so as to be distinguishable from each other.

Effects of the Invention

[0006] According to the present invention, means for easily grasping the relationship between the abnormal region detected from the information on the shape of the retinal layer and the abnormal region detected based on the tomographic image is provided.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Mode for Carrying Out the Invention

[0008] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. The present invention can be preferably applied to a workstation connected to an OCT image imaging device, a workstation for performing image analysis, and a reading terminal viewer. The OCT image imaging device includes, for example, a time domain OCT device, a spectral domain OCT device, and the like.

[0009] [First Embodiment] In the first embodiment, an information processing device that displays so as to distinguish a tomographic abnormal region, a tomographic abnormality map, a shape abnormality map, and a shape abnormality back-projection region will be described. In this embodiment, the tomographic abnormal region is an abnormal region of a specific tomographic image in OCT volume data obtained by photographing the fundus of an eye to be examined with an OCT image imaging device. Also, in this embodiment, the tomographic abnormality map is a two-dimensional map obtained by projecting the tomographic abnormal region in all tomographic images onto the fundus parallel plane. Further, in this embodiment, the shape abnormality map is a map showing an abnormal region in a shape map that is a two-dimensional map obtained by projecting information on the shape of a specific retinal layer in each tomographic image onto the fundus parallel plane in all tomographic images. Also, in this embodiment, the shape abnormality back-projection region is a region obtained by back-projecting the shape abnormality map onto the specific tomographic image.

[0010] Here, back-projection means, in this embodiment, a process of obtaining a region on a tomographic image corresponding to a predetermined region on a shape map or a shape abnormality map.

[0011] (Functional Configuration of Information Processing Apparatus) FIG. 1 is a block diagram showing a functional configuration of an information processing apparatus according to the first embodiment. The information processing apparatus 1 includes a tomographic image acquisition unit 101, a shape abnormality information acquisition unit 102, a tomographic abnormality information acquisition unit 103, a display control unit 104, and a storage unit 105. In this embodiment, the information processing apparatus 1 further includes a shape map acquisition unit 106 and an abnormality map acquisition unit 107. Further, the tomographic abnormality information acquisition unit 103 includes a tomographic abnormality region acquisition unit 108, and the abnormality map acquisition unit 107 includes a shape abnormality map acquisition unit 109, a tomographic abnormality map acquisition unit 110, and a shape abnormality back-projection region acquisition unit 111. The information processing apparatus 1 is connected to an input device 11 and a display device 12, and may also be connected to an external device (not shown) such as an OCT image imaging device via a network.

[0012] (Hardware Configuration of Information Processing Apparatus) The information processing apparatus 1 is composed of a computer including a processor, a memory, a storage, etc. In this case, by loading the program stored in the storage into the memory and having the processor execute the program, each function and each process provided in the information processing apparatus 1 are realized. Here, the functions are, for example, a shape abnormality information acquisition unit 102, a tomographic abnormality information acquisition unit 103, a shape map acquisition unit 106, a shape abnormality map acquisition unit 109, a tomographic abnormality region acquisition unit 108, a tomographic abnormality map acquisition unit 110, a shape abnormality back-projection region acquisition unit 111, a display control unit 104, etc. However, it is not limited to this configuration. For example, all or part of the functions listed above may be realized by a dedicatedly designed processor (such as an ASIC (Application Specific Integrated Circuit)) or an FPGA (Field Programmable Gate Array). Alternatively, part of the arithmetic processing may be executed by a processor such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor). Also, the information processing apparatus 1 may be composed of a single piece of hardware or a plurality of pieces of hardware. For example, using cloud computing or distributed computing, a plurality of computers may cooperate to realize the functions and processes of the information processing apparatus 1.

[0013] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus 1. In this example, the information processing apparatus 1 has a CPU (Central Processing Unit) 20, a GPU 21, a RAM (Random Access Memory) 22, a ROM (Read Only Memory) 23, an HDD (Hard Disk Drive) 24, and a communication I / F (interface) 25, and these are connected by a system bus 26. The CPU 20 and the GPU 21 are processors that read out and execute the programs stored in the ROM 23 and the HDD 24 in the RAM 22, and perform arithmetic processing, control of each part of the information processing apparatus 1, etc. The RAM 22 is a volatile memory medium and functions as a work memory when the CPU 21 executes a program. The ROM 23 is a non-volatile memory medium and stores firmware and the like necessary for the operation of the information processing apparatus 1. The HDD 24 is a non-volatile memory medium and stores tomographic images in the present embodiment and information regarding abnormalities in the retinal layers obtained from the tomographic images. The communication I / F 25 is a communication device based on standards such as Wi-Fi (registered trademark), Ethernet (registered trademark), and Bluetooth (registered trademark). The communication I / F 25 is used for communication with an OCT image capturing device, other computers, and the like. The display device 12 is a device for the information processing apparatus 1 to output information externally, and is typically a user interface for presenting information to the user. The display device 12 is, for example, a display attached to the information processing apparatus 1, or a mobile terminal of a hospital-related person via an external server. The display device 12 displays the display information generated by the display control unit 104. The input device 11 is a device for inputting information to the information processing apparatus 1, and is typically a user interface for the user to operate the information processing apparatus 1. Examples of the input device 11 include a keyboard, buttons, a mouse, a touch panel, and the like. Note that the configuration of the information processing apparatus 1 described above is an example and can be changed as appropriate. For example, examples of the processor that can be mounted on the information processing apparatus 1 include an ASIC, an FPGA, etc. in addition to the above-described CPU 21. Also, a plurality of these processors may be provided, and the plurality of processors may perform distributed processing. Further, the function of storing information such as image data in the HDD 24 may be provided not in the information processing apparatus 1 but in another data server. Also, the HDD 24 may be a storage medium such as an optical disk, a magneto-optical disk, or an SSD (Solid State Drive).

[0014] (Operation of the Information Processing Apparatus) Next, the information processing method in the information processing apparatus 1 according to the present embodiment will be described with reference to the flowchart of FIG. 3.

[0015] (Step S30: Acquisition of tomographic image) In the tomographic image acquisition step of Step S30, the tomographic image acquisition unit 101 acquires a tomographic image of the fundus of the eye to be examined held in the storage unit 105. In the present embodiment, as the tomographic image, a tomographic image captured as follows is used. That is, while performing one-dimensional scanning (hereinafter also referred to as A-scanning) in the depth direction along the optical axis with respect to the retina from the OCT image capturing device, continuous scanning (hereinafter also referred to as B-scanning) is performed while shifting the optical axis laterally. The tomographic image acquisition unit 101 acquires the two-dimensional tomographic image (hereinafter also referred to as B-scan image) captured thereby.

[0016] Note that the direction in which the optical axis is shifted to acquire the B-scan image is not particularly limited, and the B-scan image may be acquired by shifting in any manner as long as a tomographic image of the retinal layer can be obtained. Further, the three-dimensional volume data (hereinafter also referred to as OCT volume data) itself acquired from the OCT image capturing device may be used as the tomographic image. Here, the OCT volume data is obtained by continuously scanning while shifting the B-scan in a direction perpendicular to the B-scan image. Hereinafter, an example of acquiring and using the OCT volume data of the fundus of the eye to be examined captured by the OCT image capturing device as the tomographic image will be described. The OCT volume data acquired by the tomographic image acquisition unit 101 is held in the storage unit 105.

[0017] (Step S31: Acquisition of shape map) In the shape map acquisition step of Step S31, the shape map acquisition unit 106 acquires a shape map of a first predetermined retinal layer depicted in the OCT volume data acquired in Step S30. Here, the shape map acquired by the shape map acquisition unit 106 is a two-dimensional map obtained by projecting information on the shape of the first predetermined retinal layer obtained from all tomographic images onto a plane parallel to the fundus. Then, the shape map acquisition unit 106 transmits the acquired shape map to the shape abnormality information acquisition unit 102. Also, the information on the first predetermined retinal layer is transmitted to the shape abnormality information acquisition unit 102 as the layer information of the shape map (information indicating which layer the shape map focuses on).

[0018] Here, it is known that the retinal layers include, from the corneal side, ILM (inner limiting membrane), NFL (nerve fiber layer of the retina), GCL (ganglion cell layer), IPL (inner plexiform layer), INL (inner nuclear layer), OPL (outer plexiform layer), ONL (outer nuclear layer), OLM (outer limiting membrane), POS (outer segment of photoreceptor cells), RPE (retinal pigment epithelium), and Choroid (choroid).

[0019] In this embodiment, the information regarding the shape is the layer thickness of the first predetermined retinal layer, and it is acquired by the following method. First, for each tomographic image in the OCT volume data, segmentation of the retinal layer is performed. Then, among the regions of the segmented results, focusing on at least one or more predetermined layers (for example, ILM to RPE), the one-dimensional distribution of the thickness (information regarding the shape) in each tomographic image is acquired. And the one-dimensional distribution of the thickness (information regarding the shape) acquired in each tomographic image is projected (mapped onto the corresponding line) on the fundus plane. Subsequently, by arranging (integrating) all the one-dimensional distributions of the thickness (information regarding the shape) projected in each tomographic image, a two-dimensional shape map (layer thickness map) is acquired. Note that not only one shape map but also a configuration in which a plurality of different shape maps (for example, ILM to GCL, ILM to INL, etc.) regarding a plurality of layers are acquired as analysis targets may be used. Also, the information regarding the shape may be information other than the layer thickness, for example, the curvature of a specific retinal layer (the boundary between layers). Also, a configuration in which each of a plurality of information regarding the shape, such as the layer thickness and the curvature, is treated as information regarding the shape (that is, each shape map is used as an analysis target) may be used.

[0020] Note that the shape map is not limited to the method of projecting and integrating the information on the shape obtained from each of the aforementioned tomographic images onto the fundus parallel plane in all the tomographic images. For example, the shape map may be obtained by performing three-dimensional segmentation on OCT volume data to obtain information on the shape and then projecting it onto the fundus parallel plane.

[0021] Next, as the layer information (first predetermined retinal layer) of the shape map, information on at least one or more predetermined layers (for example, ILM to RPE) that are the focus of analysis among the segmented regions when obtaining the shape map is acquired. In the present embodiment, it is assumed that the user inputs the information using an input device 11 such as a mouse or a keyboard. Then, the information is transmitted to the shape abnormality information acquisition unit 102. Note that a configuration may be adopted in which a predetermined range previously held in the storage unit 105 is used as the first predetermined retinal layer.

[0022] (Step S32: Acquisition of shape abnormality information) In the shape abnormality information acquisition step of step S32, the shape abnormality information acquisition unit 102 acquires shape abnormality information, which is information on a portion having an abnormal shape in the retinal layer of the eye to be examined, based on the tomographic image. In the present embodiment, the shape abnormality information acquisition unit 102 performs an analysis process on the shape map of the eye to be examined acquired in step S31 to acquire information on an abnormal region of the shape that may have occurred based on the disease in the shape map. Then, the shape abnormality information and the layer information of the shape abnormality information are transmitted to the shape abnormality map acquisition unit 109 included in the abnormality map acquisition unit 107.

[0023] In the present embodiment, the shape abnormality information is acquired by the following method. First, a deep learning model trained to output a healthy-eye-like shape map (hereinafter also referred to as a healthy-eye-like shape map) with an arbitrary shape map as an input is acquired from the storage unit 105. The deep learning model is a learned model that has been trained using only the shape map obtained from the tomographic image of the fundus of a healthy eye, that is, a tomographic image having no abnormality, and outputs a shape map restored based on the input shape map. Next, the shape map obtained from the tomographic image of the fundus of the eye to be examined is input into the above-described deep learning model to output a healthy-eye-like shape map. In the present embodiment, a healthy-eye-like layer thickness map, which is a healthy-eye-like shape map regarding the layer thickness of the retinal layer, is obtained. Then, shape abnormality information is obtained based on the difference between the shape map of the eye to be examined (i.e., the measured value) and the output healthy-eye-like shape map (i.e., the estimated normal value). For example, a region where the difference value between the values of both at the corresponding positions (i.e., the measured value and the estimated normal value) exceeds a preset range is determined as a region having a shape abnormality.

[0024] Note that, in the above, shape abnormality information is obtained based on the difference value between the shape map of the eye to be examined and the healthy-eye-like shape map. However, as long as it is a method for quantifying the deviation from the healthy-eye-like shape map, shape abnormality information may be obtained based on other than the difference value. For example, shape abnormality information may be obtained based on the absolute value of the difference between the two. Also, shape abnormality information may be obtained based on the ratio between the two. For example, a region where the ratio value exceeds a predetermined range may be determined as an abnormal region.

[0025] Note that the method for obtaining shape abnormality information is not limited to the above-described method. For example, a deep learning segmentation model trained to input an arbitrary shape map and discriminate and output normal and abnormal regions in the map may be obtained from the storage unit 105 and used. Examples of such a learned model include an abnormality detection model learned using a shape map having no abnormal region as learning data. By applying the segmentation model to the shape map of the eye to be examined, shape abnormality information can be obtained. Also, shape abnormality information may be obtained based on the deviation of the shape map of the eye to be examined from a reference database after previously creating a reference database obtained by averaging the shape maps of a plurality of healthy eyes. That is, a configuration may be adopted in which a healthy-eye reference shape map obtained from the reference database is used instead of the healthy-eye-like shape map in the above processing.

[0026] Note that, in the present embodiment, as the layer information of the shape abnormality information, the layer information of the shape map (the first predetermined retinal layer) is obtained.

[0027] In addition, when a plurality of shape maps are acquired in step S31, shape abnormality information may be acquired from each shape map.

[0028] The abnormality map acquisition unit 107 has a function of acquiring an abnormality map in which the above-described shape abnormality information and the tomographic abnormality information described later are represented on at least one of a plane along the fundus of the eye to be examined and a plane in a direction perpendicular to the fundus of the eye to be examined. In the present embodiment, an example in which the abnormality map acquisition unit 107 includes a shape abnormality map acquisition unit 109, a tomographic abnormality map acquisition unit 110, and a shape abnormality back-projection region acquisition unit 111 will be described.

[0029] (Step S33: Acquisition of shape abnormality map) In the shape abnormality map acquisition step of step S33, the shape abnormality map acquisition unit 109 acquires a shape abnormality map, which is a map representing the shape abnormality information acquired in step S32 on a plane along the fundus of the eye to be examined. Then, the shape abnormality map and the layer information of the shape abnormality map are transmitted to the display control unit 104.

[0030] Specifically, the shape abnormality map acquisition unit 109 generates a shape abnormality map in which a value of 1 is assigned to a region determined to have a shape abnormality and a value of 0 is assigned to other regions in the shape abnormality information acquired in step S32. The threshold value for determining the shape abnormality may be determined in advance and held in the storage unit 105, or may be determined by input from the user via the input device 11.

[0031] Further, the shape abnormality map may represent the degree of abnormality (abnormality degree) not as a binary value but as a continuous value. For example, a function for calculating an abnormality degree from 0 to 1 by inputting the absolute value of the difference between the two for each pixel may be defined, and the shape abnormality map may be generated using the function. As the function, an increasing function that approaches 0 when the input value is small and approaches 1 when the input value is large may be used. Also, the absolute value of the difference between the two and the ratio to the healthy-eye-like shape map may be calculated as the abnormality degree of each pixel, and the distribution thereof may be used as the shape abnormality map.

[0032] In addition, in the present embodiment, as the layer information of the shape abnormality map (information indicating which layer the shape abnormality map focuses on), the layer information of the shape map (the first predetermined retinal layer) is acquired.

[0033] In addition, when a plurality of shape abnormality information is acquired in step S32, a shape abnormality map may be generated for each shape abnormality information.

[0034] In the present embodiment, a method for acquiring a shape abnormality map based on a shape map has been described, but the present invention is not limited thereto. As long as the shape abnormality map acquisition unit 109 can obtain a map representing the abnormal part of the shape in the fundus parallel plane from the shape abnormality information, the shape abnormality map may be acquired by any other method.

[0035] (Step S34: Acquisition of tomographic abnormality information) In the tomographic abnormality information acquisition step of step S34, the tomographic abnormality information acquisition unit 103 acquires tomographic abnormality information, which is information about a part including the abnormal region in the retinal layer of the eye to be examined, based on the tomographic abnormal region, which is the abnormal region in the tomographic image. In the present embodiment, the tomographic abnormal region acquisition unit 108 acquires an abnormal region (tomographic abnormal region) that may have occurred based on the disease in the tomographic image of the OCT volume data acquired in step S30, and transmits it to the display control unit 104. In addition, the tomographic abnormal region acquisition unit 108 acquires the layer information of the tomographic abnormal region (information specifying the layer where the tomographic abnormal region exists), and transmits it to the display control unit 104 together with the position of the tomographic image. Here, which tomographic image in the OCT volume data is selected may be specified by the user using an input device 11 such as a mouse or a keyboard. In the present embodiment, an example will be described in which the position of the tomographic image (hereinafter referred to as the display tomographic image) that the user wants to display using a mouse is input (clicked) on the shape abnormality map display window 402 described later.

[0036] In this embodiment, the tomographic abnormality region acquisition unit 108 acquires the tomographic abnormality region by the following method. First, a deep learning segmentation model trained to input a tomographic image and output the degree of abnormality (degree of likelihood of an abnormal region) of each pixel in the image is acquired from the storage unit 105. Examples of the deep learning segmentation model include an abnormality detection model trained using tomographic images without abnormalities as learning data. Subsequently, the above-described segmentation model is applied to the tomographic image of the eye to be examined, and the output (abnormality degree image) of the segmentation model is acquired. Then, by performing binarization processing using a preset threshold value on the image, the tomographic abnormality region is acquired in the form of a binary image (tomographic abnormality region image) representing whether each pixel of the tomographic image is normal (0) or abnormal (1). Note that the tomographic abnormality region may be acquired in a form that represents the degree of likelihood of an abnormal region as a continuous value, such as the abnormality degree image output by the above-described segmentation model.

[0037] Note that the tomographic abnormality region acquisition unit 108 may acquire the tomographic abnormality region for each of a plurality of tomographic images (B-scan images) constituting the OCT volume data. Further, the tomographic abnormality region acquisition unit 108 may perform three-dimensional segmentation on the OCT volume data and acquire the tomographic abnormality region as three-dimensional information. The tomographic abnormality information acquisition unit 103 acquires tomographic abnormality information obtained by integrating the tomographic abnormality region obtained for the tomographic image and information about the retinal layer having the tomographic abnormality region.

[0038] In this embodiment, information about the retinal layer having the tomographic abnormality region is acquired by the following method. That is, by comparing the tomographic abnormality region of the tomographic image of the eye to be examined with the segmentation result of the retinal layer of the tomographic image, information (for example, ILM~RPE) indicating which layer in the tomographic image has the tomographic abnormality region is acquired.

[0039] Note that the tomographic abnormality information acquisition unit 103 may acquire, for example, the following information instead of the information about the retinal layer having a tomographic abnormality region (for example, ILM to RPE). That is, the tomographic abnormality information acquisition unit 103 may acquire information as to whether or not the tomographic abnormality region is included within the range represented by the layer information of the shape map described above (for example, ILM to GCL) (that is, the first predetermined retinal layer). That is, it is not always essential for the tomographic abnormality information acquisition unit 103 to acquire information about all the retinal layers having a tomographic abnormality region, and it may acquire information only about some of the retinal layers having a tomographic abnormality region.

[0040] (Step S35: Acquisition of tomographic abnormality map) In the tomographic abnormality map acquisition step of step S35, the tomographic abnormality map acquisition unit 110 acquires a tomographic abnormality map, which is a map obtained by projecting the tomographic abnormality information about the second predetermined retinal layer onto a plane along the fundus of the eye to be examined. In the present embodiment, the tomographic abnormality map acquisition unit 110 acquires a tomographic abnormality map and layer information of the tomographic abnormality map by the following method. First, for each tomographic image in which the tomographic abnormality region was acquired in step S34, the tomographic abnormality map acquisition unit 110 projects the information of the tomographic abnormality region existing in the specific region (the second predetermined retinal layer) in a direction perpendicular to the fundus parallel plane to acquire a one-dimensional distribution of the abnormal region. Then, the tomographic abnormality map is acquired by arranging (integrating) the one-dimensional distributions of the abnormal regions in each tomographic image into a two-dimensional map. Here, the information about the second predetermined retinal layer may be input by the user using an input device 11 such as a mouse or a keyboard.

[0041] In the case of a configuration in which the tomographic abnormal regions of each tomographic image are represented by continuous values (degree of abnormality), a tomographic abnormal map may be obtained as a two-dimensional map of the degree of abnormality by projecting the degree of abnormality in a direction perpendicular to the fundus parallel plane. Here, the method of projecting the degree of abnormality is not limited to the maximum value projection, and any method may be used as long as it calculates a representative value from the set of degrees of abnormality in the projection direction. For example, the 95th percentile value may be used. Further, for the obtained tomographic abnormal map, a region equal to or greater than a preset threshold value may be set to 1 (abnormal), and a region less than the threshold value may be set to 0 (normal), thereby obtaining a tomographic abnormal map as a binary two-dimensional map.

[0042] Next, the tomographic abnormal map acquisition unit 110 acquires the above-described specific region (the second predetermined retinal layer) as layer information of the tomographic abnormal map.

[0043] Then, the tomographic abnormal map acquisition unit 110 transmits the acquired tomographic abnormal map and the layer information of the tomographic abnormal map to the display control unit 104. Here, in the present embodiment, the second predetermined retinal layer is a specific retinal layer in the tomographic image, but is not limited to including the entire range of the specific retinal layer, and may be any region in the tomographic image. Further, the tomographic abnormal map acquisition unit 110 may acquire information on the entire tomographic image or the entire retinal layer instead of the second predetermined retinal layer.

[0044] (Step S36: Acquisition of shape abnormal back-projection region) In the step of acquiring the shape abnormal back-projection region in step S36, the shape abnormal back-projection region acquisition unit 111 acquires a shape abnormal back-projection region indicating a region corresponding to the shape abnormal map in the tomographic image.

[0045] In this embodiment, the shape anomaly back-projection region acquisition unit 111 acquires a shape anomaly back-projection region by back-projecting a shape anomaly map onto a specific tomographic image, and transmits it to the display control unit 104 together with the layer information of the shape anomaly back-projection region. Here, which tomographic image in the OCT volume data is selected may be input by the user using an input device 11 such as a mouse or a keyboard. In this embodiment, it is determined by the user clicking on the position of the tomographic image to be displayed using a mouse on a shape anomaly map display window 402 described later. The position of this tomographic image is also used as the position for displaying the tomographic anomaly region.

[0046] In this embodiment, the back-projection is performed in the following procedure. First, the position of the tomographic image onto which the shape anomaly map is to be back-projected is acquired. The position of the tomographic image is input by the user using an input device 11 such as a mouse or a keyboard. Next, the anomaly region (one-dimensional information) on the shape anomaly map corresponding to the position of the tomographic image is acquired. Then, the acquired one-dimensional anomaly region is projected onto the layer region indicated by the layer information of the shape anomaly map on the tomographic image to acquire the shape anomaly back-projection region.

[0047] Note that in this embodiment, as the layer information of the shape anomaly back-projection region, the layer information of the shape map (the first predetermined retinal layer) is acquired.

[0048] (Step S37: Display control of anomaly information) In the display control process of step S37, the display control unit 104 performs control to display at least a part of the shape anomaly information and at least a part of the tomographic anomaly information so that they can be distinguished from each other. In this embodiment, the display control unit 104 generates display information that represents the shape anomaly map, the tomographic anomaly map, the shape anomaly back-projection region, and the tomographic anomaly region in a distinguishable manner. Then, the control is performed to display the generated display information, the acquired layer information, and the position information about the tomographic anomaly region to be displayed on the display device 12.

[0049] Hereinafter, as the first predetermined retinal layer and the second predetermined retinal layer, ILM~PRE is set, and an example of performing control to integrate and display the shape abnormality information and the tomographic abnormality information obtained for each retinal layer will be described.

[0050] FIG. 4 shows an example of a screen (display screen 401) on which the display information generated by the display control unit 104 is displayed on the display device 12. The display screen 401 includes a shape abnormality map display window 402, a shape abnormality back-projection region display window 403, a tomographic abnormality map display window 404, a tomographic abnormality region display window 405, and a detailed information window 406. The display screen 401 further includes a shape abnormality map / tomographic abnormality map display window 408 and a shape abnormality back-projection region / tomographic abnormality region display window 409. In FIG. 4, the positions 407 of the tomographic images shown in the shape abnormality back-projection region display window 403, the tomographic abnormality region display window 405, and the shape abnormality back-projection region / tomographic abnormality region display window 409 are also depicted.

[0051] The information displayed by the shape abnormality map display window 402, the tomographic abnormality map display window 404, and the shape abnormality map / tomographic abnormality map display window 408 includes a shape map. Here, for the sake of simplicity of explanation, the shape maps displayed in the windows 402, 404, 408 are assumed to be images binarized based on a predetermined threshold value for the layer thickness. That is, each of the windows 402, 404, 408 includes a shape region 411 where the thickness of the retinal layer is thicker than the predetermined threshold value. Note that the region 421 indicates a region where a masking process is performed to fill the optic disc where the thickness of the retinal layer cannot be measured with black pixels.

[0052] In addition, the information displayed in the shape abnormality inverse projection region display window 403, the tomographic abnormality region display window 405, and the shape abnormality inverse projection region / tomographic abnormality region display window 409 includes the tomographic image (hereinafter, also simply referred to as the display tomographic image) at the position (dotted line) 407. Here, for the sake of simplicity of explanation, the display tomographic images displayed in the respective windows 403, 405, 409 are assumed to be images that have been processed and binarized so that the difference between the region indicating the retinal layer and the other regions is clear. That is, each of the windows 403, 405, 409 includes the retinal layer region 413 in the display tomographic image.

[0053] The position (dotted line) 407 may be specified by, for example, the information processing apparatus 1 further having a target position specifying unit. That is, for example, the information processing apparatus 1 may further include a target position specifying unit that specifies the position (dotted line) 407. And the display control unit 104 may be configured to be able to perform control to display the tomographic abnormality region and the shape abnormality inverse projection region in the tomographic image including the target position. Information about the target position specified by the target position specifying unit may be input by the user from, for example, the input device 11. Also, not limited to this, the target position specifying unit may specify the target position on the shape map, or may specify the target position on the tomographic abnormality map. Also, the display control unit 104 may be configured to be able to perform control to display either one of the tomographic abnormality region and the shape abnormality inverse projection region in the tomographic image including the target position.

[0054] In the shape abnormality map display window 402, the shape abnormality map acquired in step S32 and the position (dotted line) 407 of the display tomographic image are superimposed and displayed on the shape map of the first predetermined retinal layer acquired in step S31. That is, the shape abnormality map display window 402 includes a shape abnormality projection region 412 indicating a region determined to have a shape abnormality in the shape abnormality map. Here, when a plurality of shape abnormality maps are acquired in step S32, it may be controlled to sequentially switch and display the shape map and the shape abnormality map in response to a user operation, or may be controlled to display them side by side.

[0055] In the shape abnormality inverse projection area display window 403, the shape abnormality inverse projection area 416 acquired in step S35 is superimposed and displayed on the displayed tomographic image.

[0056] In the shape abnormality inverse projection area / tomographic abnormality area display window 409, the tomographic abnormality area 414 acquired in step S34 and the shape abnormality inverse projection area 416 acquired in step S36 are superimposed and displayed so as to be distinguishable from each other on the displayed tomographic image. The tomographic abnormality area 414 and the shape abnormality inverse projection area 416 can be distinguished by emphasizing and displaying their respective areas and their boundaries in different display modes.

[0057] Here, the shape abnormality inverse projection area 416 is different from the tomographic abnormality map in which the abnormal area is specified on the tomographic image. It is obtained by specifying the abnormal area on the shape map and then inverse projecting it onto the tomographic image. Therefore, diseases that do not appear as abnormalities in the shape map are not reflected in the shape abnormality inverse projection area 416. Thus, there may be cases where different areas are displayed between the shape abnormality inverse projection area 416 and the tomographic abnormality area 414. Specifically, the following four types exist. · Type 1: An area that is a tomographic abnormality area but not a shape abnormality inverse projection area · Type 2: An area that is both a tomographic abnormality area and a shape abnormality inverse projection area · Type 3: An area that is not a tomographic abnormality area but is a shape abnormality inverse projection area · Type 4: An area that is neither a tomographic abnormality area nor a shape abnormality inverse projection area

[0058] Therefore, the display control unit 104 can perform control to further emphasize and display the areas with differences between the shape abnormality inverse projection area 416 and the tomographic abnormality area 414 so that the above four types can be distinguished. FIG. 5 shows an example in which in the shape abnormality inverse projection area / tomographic abnormality area display window 409 in the display screen 401, the type 1 area 51, the type 2 area 52, and the type 3 area 53 are further emphasized and displayed from FIG. 4.

[0059] Specifically, the Type 1 region 51 (outside the Shape Anomaly Inverse Projection Region 416 within the Fault Anomaly Region 414) may have a disease that does not appear as a thickness anomaly in the shape map, so it is displayed in Display Mode 1 (for example, shading the region with Color 1). The Type 2 region 52 (the region where the Fault Anomaly Region 414 and the Shape Anomaly Inverse Projection Region 416 overlap) is highly reliable as an abnormal region, so it is displayed in Display Mode 2 (for example, shading the region with Color 2). The Type 3 region 53 (outside the Fault Anomaly Region 414 within the Shape Anomaly Inverse Projection Region 416) may have a segmentation error in the shape map, so it is displayed in Display Mode 3 (for example, shading the region with Color 3). And the Type 4 region (outside the Shape Anomaly Inverse Projection Region 416 and outside the Fault Anomaly Region 414, and regions other than the Type 1 region 51, the Type 2 region 52, and the Type 3 region 53) is highly reliable as a normal region. Therefore, the Type 4 region is displayed in Display Mode 4 (for example, displaying the region without color). Also, details of which display mode corresponds to which region may be described in a legend or a pop-up window. And by displaying in these four display modes, the difference between the Shape Anomaly Inverse Projection Region 416 and the Fault Anomaly Region 414 can be distinguished and displayed. The Type 1 region 51 to the Type 4 region 54 may be switched between emphasized display and non-display on each display window according to the user's operation. Also, the Display Modes 1 to 4 may not only shade the regions with colors, but may also display only display modes such as dotted lines that can distinguish the boundaries, or any other method may be used.

[0060] On the tomographic anomaly map display window 404, the tomographic anomaly map acquired in step S35 and the position (dotted line) 407 of the displayed tomographic image are superimposed on the shape map of the first predetermined retinal layer acquired in step S31. That is, the tomographic anomaly map display window 404 includes a tomographic anomaly projection region 415 indicating the region where the tomographic anomaly region is projected in the tomographic anomaly map. Here, when a plurality of tomographic anomaly maps are acquired in step S35, the tomographic images and the tomographic anomaly maps may be sequentially switched and displayed or arranged and displayed according to the user's operation.

[0061] In the tomographic abnormality region display window 405, the tomographic abnormality region 414 acquired in step S34 is displayed superimposed on the displayed tomographic image. Here, when a tomographic abnormality region is acquired for each B-scan image constituting the OCT volume data in step S34, the displayed tomographic image (B-scan image) and the tomographic abnormality region 414 may be sequentially switched and displayed in response to a user operation. Note that it is preferable that the displayed tomographic images displayed in the shape abnormality back projection region display window 403, the shape abnormality back projection region / tomographic abnormality region display window 409, and the tomographic abnormality region display window 405 are the same B-scan image. In this case, when switching the B-scan image, it is preferable to switch the displayed tomographic images in conjunction with each other.

[0062] In the shape anomaly map / fault anomaly map display window 408, the fault anomaly map acquired in step S35 and the shape anomaly map acquired in step S33 are superimposed on the shape map acquired in step S31 so that the differences between the regions can be distinguished. Also, the position (dotted line) 407 of the displayed tomographic image is superimposed. Distinguishing the differences between the regions means that each region and its boundary are displayed in a different display mode with emphasis.

[0063] Moreover, the shape abnormality map is different from the cross-sectional abnormality map obtained by projecting the abnormal area specified on the cross-sectional image onto the fundus-parallel plane, and is a map in which the abnormal area is specified based on the shape abnormality information. Therefore, diseases that are not specified as abnormal in the shape abnormality information are not reflected in the shape abnormality map. In other words, different areas may be displayed on the shape abnormality map and the cross-sectional abnormality map. Specifically, there are the following four types. Type 5: Areas that are shown as anomalous regions on the fault anomaly map but not on the shape anomaly map Type 6: Areas that are shown as anomalous regions on both fault anomaly maps and shape anomaly maps Type 7: Areas that are not shown as anomalous regions in the fault anomaly map but are shown as anomalous regions in the shape anomaly map · Type 8: Regions not shown as abnormal regions in either the fault abnormality map or the shape abnormality map

[0064] Therefore, the display control unit 104 can perform control to further emphasize and display regions with differences between the shape abnormality map and the fault abnormality map so that the above four types can be distinguished. FIG. 6 shows an example in which the type 5 region 61 and the type 6 region 62 are displayed in the shape abnormality map - fault abnormality map display window 408 in the display screen 401.

[0065] Specifically, the type 5 region 61 (the region obtained by dividing the fault abnormality projection region 415 by excluding the shape abnormality projection region 412) may have a disease that does not appear as a thickness abnormality in the shape map, so it is displayed in display mode 1 (for example, the region is shaded with color 1). The type 6 region 62 (the region where the fault abnormality projection region 415 and the shape abnormality projection region 412 overlap) is highly reliable as an abnormal region, so it is displayed in display mode 2 (for example, the region is shaded with color 2). The type 3 region (no corresponding region in FIG. 6) may have a segmentation error in the shape map, so it is displayed in display mode 3 (for example, the region is shaded with color 3). And the type 4 region (the region of the complement of the fault abnormality projection region 415) is highly reliable as a normal region, so it is displayed in display mode 4 (for example, the region is displayed without color). Also, details of which display mode corresponds to which region may be described in a legend or a pop-up window. By displaying in these four display modes, the differences between the shape abnormality map and the fault abnormality map can be distinguished and displayed. Here, the regions of type 5 to type 8 may be switched between emphasized display and non-display on each display window according to the user's operation.

[0066] As described above, on the display screen 401, a shape abnormality map and a tomographic abnormality map related to a two-dimensional map obtained by projecting an abnormal region onto the fundus parallel plane are displayed in the upper part of the display screen 401. That is, a shape abnormality map display window 402, a tomographic abnormality map display window 404, and a shape abnormality map - tomographic abnormality map display window 408 are displayed in the upper part of the display screen 401. Also, a shape abnormality back-projection region 416 and a tomographic abnormality region 414 related to the abnormal region in the tomographic image are displayed in the lower part of the display screen 401. That is, a shape abnormality back-projection region display window 403, a tomographic abnormality region display window 405, and a shape abnormality back-projection region - tomographic abnormality region display window 409 are displayed in the lower part of the display screen 401. By doing so, it is possible to display the respective images in a distinguishable manner in each window according to the difference in the vertical position of the window positions displayed on the display screen 401.

[0067] Furthermore, information related to the shape abnormality is displayed on the left side of the display screen 401, that is, in the shape abnormality map display window 402 and the shape abnormality back-projection region display window 403. On the other hand, information related to the tomographic abnormality is displayed on the right side of the display screen 401, that is, in the tomographic abnormality map display window 404 and the tomographic abnormality region display window 405. And information combining the shape abnormality and the tomographic abnormality is displayed in the center of the display screen 401, that is, in the shape abnormality map - tomographic abnormality map display window 408 and the shape abnormality back-projection region - tomographic abnormality region display window 409. By doing so, it is possible to display the respective images in a distinguishable manner in each window according to the difference in the horizontal position of the window positions displayed on the display screen 401. And the details of those images are displayed in the detailed information window 406.

[0068] Here, the details of the image refer to the layer information of multiple shape abnormality maps, the layer information of the tomographic abnormality region, the position of the tomographic image, etc., but any other relevant information may be displayed. For example, the shape abnormality map displays the ILM~RPE, and it is assumed that an abnormality region also occurs in the ILM~RPE in the tomographic abnormality region. In that case, as the layer information in the shape abnormality map, details such as "The ILM~RPE layer is being displayed." can be displayed. Also, as the layer information of the tomographic abnormality region, for example, details such as "An abnormality has occurred in the ILM~RPE layer. The tomographic image is displaying the tomographic image at the central position coordinates." can be displayed. Also, regarding the shape map, it may be displayed that abnormalities also occur in other shape abnormality maps (ILM~GCL, ILM~INL).

[0069] Note that in FIG. 4, for the sake of simplicity of explanation, an example is shown in which the shape map, tomographic image, shape abnormality map, and tomographic abnormality map are each binarized and displayed, but it is not limited to this. For example, as the displayed tomographic image, the captured image may be used as it is, or each process may be performed so as to emphasize the information to be displayed and then displayed. Also, continuous values related to the shape information of the retinal layer such as the layer thickness and values indicating the degree of abnormality may be displayed by a heat map or the like. Also, regarding the tomographic abnormality region 414 and the shape abnormality back-projection region 416, continuous values indicating the degree may be displayed by a heat map or the like.

[0070] According to the information processing apparatus according to the present embodiment, due to the differences in the displayed windows and the display modes of each region in each window, the differences between the abnormality region based on the shape abnormality information and the abnormality region detected by the tomographic image can be clearly displayed to the user.

[0071] Note that the order of the steps S31 to S36 in the present embodiment can be appropriately changed. That is, after acquiring the tomographic abnormality information, the tomographic abnormality map may be acquired, after acquiring the shape abnormality information, the shape abnormality map may be acquired, and then the shape abnormality back-projection region may be acquired, and the detailed process order may be appropriately changed.

[0072] (Modification Example 1 of the First Embodiment) In the above-described embodiment, in step S37, the display control unit 104 displays the shape abnormality map, the shape abnormality back-projection region, the tomography abnormality map, and the tomography abnormality region so as to be distinguishable from each other. However, the implementation of the present invention is not limited to this, and as long as the shape abnormality information and the tomography abnormality information can be displayed separately from each other, the windows may be displayed in any combination. For example, a configuration in which only the shape abnormality map / tomography abnormality map display window 408 and / or the shape abnormality back-projection region / tomography abnormality region display window 409 is displayed on the display screen 401 may be used. Further, a configuration in which the shape abnormality map / tomography abnormality map display window 408 and / or the shape abnormality back-projection region / tomography abnormality region display window 409 is not displayed may be used. Also, a configuration in which the lower display is omitted or a configuration in which the upper display is omitted may be used. Further, a configuration in which only the detailed information window 406 is displayed may be used. Also, any combination of the other shape abnormality map display window 42, the shape abnormality back-projection region display window 403, the tomography abnormality map display window 404, and the tomography abnormality region display window 405 may be displayed on the display screen 401. Also, for that combination, a specific acquisition unit may be omitted from the hardware configuration of the information processing device.

[0073] When only the shape abnormality map / tomography abnormality map display window 408 is displayed on the display screen 401, it is possible to display the regions where abnormalities have occurred in the shape abnormality map and the regions where abnormalities have occurred in the tomography abnormality map separately from each other.

[0074] When only the shape abnormality back-projection region / tomography abnormality region display window 409 is displayed on the display screen 401, it is possible to display the regions where abnormalities have occurred in the shape abnormality back-projection region and the regions where abnormalities have occurred in the tomography abnormality region separately from each other.

[0075] When only the detailed information window 406 is to be displayed on the display screen 401, for example, the display control unit 104 may perform control to display the layer information in which an abnormality has occurred. Specifically, for example, when the layer in which an abnormality has occurred in the shape abnormality map and the layer in which an abnormality has occurred in the fault abnormality region are the ILM to GCL, the display control unit 104 performs control to display "An abnormality has occurred in the ILM to GCL in both the shape abnormality map and the fault abnormality region". By doing so, it is possible to distinguish and display the layer in which an abnormality has occurred in the shape abnormality map and the layer in which an abnormality has occurred in the fault abnormality map.

[0076] Further, the display control unit 104 may be configured to store the generated display information as an image in the storage unit 105. Further, it may be configured to output the generated display information to an external server or the like. In these cases, the control to directly display the display information on the display device 12 does not necessarily have to be performed.

[0077] (Modification Example 2 of the First Embodiment) In the above-described embodiment, in step S37, the display control unit 104 displayed the shape abnormality map, the shape abnormality inverse projection region, the fault abnormality map, and the fault abnormality region so as to be distinguishable from each other. However, the implementation of the present invention is not limited to this, and when the layer information displayed in the shape abnormality map and the layer information displayed in the fault abnormality map are different, the display control unit 104 may be configured to perform control to display at least one of the layer information. That is, the display control unit 104 may be configured to be able to perform control to display information regarding at least one of the first predetermined retinal layer and the second predetermined retinal layer when the first predetermined retinal layer and the second predetermined retinal layer are not the same. Specifically, when the layer information displayed in the shape abnormality map is ILM to GCL and the layer information displayed in the tomogram abnormality map is ILM to INL, the display control unit 104 performs control to display one or more of the layer information of the shape abnormality map and the layer information of the tomogram abnormality map. By doing so, it is possible to display the layers in which abnormalities occur in the shape abnormality map and the layers in which abnormalities occur in the tomogram abnormality map in a distinguishable manner. Note that in this modification example, the display of the detailed information window 406 may be omitted.

[0078] (Modification Example 3 of the First Embodiment) In the above-described embodiment, in step S35, the tomogram abnormality map acquisition unit 110 projects the tomogram abnormality region onto the fundus parallel plane with the second predetermined retinal layer as the processing range to obtain a tomogram abnormality map, and transmits it to the display control unit 104. However, the implementation of the present invention is not limited to this, and the tomogram abnormality map acquisition unit 110 may be configured to obtain a tomogram abnormality map by setting the same retinal layer as the first predetermined retinal layer as the second predetermined retinal layer. That is, the tomogram abnormality map acquisition unit 110 may set the layer for obtaining the tomogram abnormality map (layer information of the tomogram abnormality map) based on the layer information of the shape map (that is, the first predetermined retinal layer) (that is, by limiting the layer for performing the projection process). Specifically, for example, when the layer information of the shape map is ILM to GCL, the tomogram abnormality map acquisition unit 110 may set the layer information of the tomogram abnormality map to the same ILM to GCL as the layer information of the shape map. By doing so, it is possible to display the shape abnormality map and the tomogram abnormality map having the same layer information in a distinguishable manner.

[0079] (Modification Example 4 of the First Embodiment) In the above-described embodiment, in step S34, the tomographic abnormality region acquisition unit 108 acquired a tomographic abnormality region that might have occurred based on the disease in the tomographic image. However, the implementation of the present invention is not limited to this, and a configuration may be adopted in which the analysis target range in the tomographic image is limited based on the layer information of the shape map (that is, the first predetermined retinal layer), and the tomographic abnormality region is acquired. That is, the tomographic abnormality information acquisition unit 103 may be configured to acquire a tomographic abnormality region in the same retinal layer as the first predetermined retinal layer. By doing so, it is possible to display the shape abnormality back-projection region and the tomographic abnormality region having the same layer information in a distinguishable manner.

[0080] (Modification Example 5 of the First Embodiment) In the above-described embodiment, the layer information of the shape map (the first predetermined retinal layer) was set by the user input. However, the implementation of the present invention is not limited to this, and the shape map acquisition unit 106 may be configured to set the layer information of the shape map (the first predetermined retinal layer) to be the same as the layer information of the tomographic abnormality map (the second predetermined retinal layer). That is, the shape abnormality map acquisition unit 109 may be configured to set the same retinal layer as the second predetermined retinal layer as the first predetermined retinal layer and acquire the shape abnormality map. Further, the information processing apparatus 1 may be configured to have a determination unit that determines the target retinal layer to be processed based on the tomographic abnormality region. In this case, after the processing of steps S34 and S35, the processing from step S31 to step S33 may be performed. Specifically, when the layer information of the tomographic abnormality map is ILM to INL, the shape map acquisition unit 106 sets the layer information of the shape map to the same ILM to INL as the layer information of the tomographic abnormality map. By doing so, it is possible to display the shape abnormality map and the tomographic abnormality map having the same layer information in a distinguishable manner.

[0081] (Modification Example 6 of the First Embodiment) In the above-described embodiment, the layer information (the first predetermined retinal layer) of the shape map was set by the user's input. However, the implementation of the present invention is not limited to this. The shape map acquisition unit 106 may be configured to set the layer information (the first predetermined retinal layer) of the shape map to be the same as the layer information of the tomographic abnormality region (information indicating in which layer the tomographic abnormality region was detected). That is, this Modification Example 6 is an example in the case where all of the retinal layers in which the tomographic abnormality region was detected in Modification Example 5 are set as the second predetermined retinal layer. Therefore, in this case, similar to Modification Example 5 of the first embodiment, after the processes of step S34 and step S35, the processes from step S31 to step S33 may be performed. Specifically, when the layer information of the tomographic abnormality region is ILM~INL, the shape map acquisition unit 106 sets the layer information of the shape map to the same ILM~INL as the layer information of the tomographic abnormality region. By doing so, it is possible to perform shape analysis focusing on the layer in which an abnormality was detected on the tomographic image. In addition, it is possible to display the shape abnormality map and the tomographic abnormality region having the same layer information in a distinguishable manner.

[0082] (Modification Example 7 of the First Embodiment) In the above-described embodiment, in step S37, the display control unit 104 specified the position (dotted line) 407 at which the tomographic abnormality region is displayed on the shape abnormality map display window 402. Then, a display image in which the tomographic image at the position (dotted line) 407 and the tomographic abnormality region 414 are superimposed was generated and displayed on the tomographic abnormality region display window 405. Then, a display image in which the tomographic image at the same position (dotted line) 407 and the shape abnormality back-projection region 416 are superimposed was generated and displayed on the shape abnormality back-projection region display window 403. Further, a display image in which the tomographic image at the same position (dotted line) 407, the tomographic abnormality region 414, and the shape abnormality back-projection region 416 are superimposed was generated and displayed on the shape abnormality back-projection region·tomographic abnormality region display window 409.

[0083] However, the implementation of the present invention is not limited to this. The tomographic images displayed in the shape anomaly back-projection region display window 403 and the tomographic anomaly region display window 405 do not have to be tomographic images at the same position. That is, the position where the shape anomaly back-projection region is displayed on the shape anomaly map display window 402 is obtained, and a display image in which the tomographic image at that position and the shape anomaly back-projection region are superimposed is displayed in the shape anomaly back-projection region display window 403. On the other hand, the position where the tomographic anomaly region is displayed on the tomographic anomaly map display window 404 is obtained, and a display image in which the tomographic image at that position and the tomographic anomaly region are superimposed may be displayed in the tomographic anomaly region display window 405. That is, the display control unit 104 may be configured to be able to perform control to display such that the position of the tomographic image displaying the shape anomaly back-projection region 416 and the position of the tomographic image displaying the tomographic anomaly region 414 are different from each other. At that time, it is preferable not to display their superimposed images in the shape anomaly back-projection region / tomographic anomaly region display window 409.

[0084] [Second Embodiment] In the second embodiment, an information processing apparatus that acquires a shape map based on the curvature of a specific retinal layer will be described. Hereinafter, differences from the first embodiment will be described regarding the details of each configuration of the information processing apparatus according to the present embodiment.

[0085] (Shape Map Acquisition Unit 106) The shape map acquisition unit 106 acquires a shape map, which is a two-dimensional map projected onto the fundus parallel plane with all tomographic images, of information regarding the shape of a specific retinal layer (first predetermined retinal layer) depicted in each tomographic image (B-scan image) in the OCT volume data. Then, the acquired shape map is transmitted to the shape anomaly information acquisition unit 102.

[0086] In this embodiment, the shape map is the curvature of the boundary of a specific retinal layer, and it is obtained by the following method. First, for each tomographic image in the OCT volume data, segmentation of the retinal layer is performed. Then, among the segmented result regions, a one-dimensional distribution of the curvature of the boundary of at least one or more predetermined layers (for example, ILM~RPE) is obtained. Then, the one-dimensional distribution of the curvature (shape) obtained for each tomographic image is projected (mapped onto the corresponding line) onto the fundus parallel plane, and by arranging (integrating) all the one-dimensional distributions of the curvature (shape) projected for each tomographic image, a two-dimensional shape map is obtained. Note that not just one shape map, but a plurality of different shape maps (for example, ILM~GCL, ILM~INL, etc.) regarding a plurality of layers may be obtained.

[0087] Note that the shape map is not limited to the method of projecting and integrating the information regarding the shape obtained for each of the aforementioned tomographic images onto the fundus parallel plane for all tomographic images. For example, the shape map may be obtained by performing three-dimensional segmentation on the OCT volume data to obtain information regarding the shape and then projecting it onto the fundus parallel plane.

[0088] (Shape abnormality information acquisition unit 102, shape abnormality map acquisition unit 109) The shape abnormality information acquisition unit 102 acquires an abnormal region (shape abnormality projection region) and layer information in the shape map, and acquires shape abnormality information. Subsequently, the shape abnormality map acquisition unit 109 generates a shape abnormality map, which is a two-dimensional map obtained by imaging the shape abnormality projection region based on the shape abnormality information, and transmits the shape abnormality map and the layer information to the display control unit 104.

[0089] In this embodiment, the shape abnormality map is obtained by the following method. First, using the shape map of the eye to be examined as an input, a region having a curvature exceeding a threshold set in advance is determined as a region having a shape abnormality. Then, a value of 1 is assigned to the region having a shape abnormality and a value of 0 is assigned to the other regions to generate a shape abnormality map.

[0090] In this manner, the geometric abnormality map based on the curvature of the retinal layer, the geometric abnormality backprojection region, the tomographic abnormality map, and the tomographic abnormality region can be displayed in a manner that allows them to be distinguished from one another.

[0091] In this embodiment, the shape map is obtained based on the curvature of the boundary of the retinal layers, but it may be obtained based on other indices. It should be noted that the above-mentioned embodiments are merely examples of the embodiment of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical idea or main features. For example, it should be understood that an embodiment in which a part of the configuration of any of the embodiments is added to another embodiment, or an embodiment in which a part of the configuration of any of the embodiments is replaced with a part of the configuration of another embodiment, is also an embodiment to which the present invention can be applied.

[0092] Disclosure according to an embodiment of the present invention includes the following configurations and methods. (Configuration 1) a tomographic image acquisition unit for acquiring a tomographic image of a fundus of a subject's eye; a shape abnormality information acquiring unit that acquires shape abnormality information, which is information about a portion having an abnormal shape in a retinal layer of the subject's eye, based on the tomographic image; a tomographic abnormality information acquiring unit that acquires tomographic abnormality information, which is information about a portion having the tomographic abnormality region in the retinal layer of the subject's eye, based on a tomographic abnormality region, which is an abnormal region in the tomographic image; a display control unit that controls displaying at least a part of the shape abnormality information and at least a part of the tomographic abnormality information in a manner that they can be distinguished from each other; An information processing device comprising: (Configuration 2) an abnormality map acquisition unit that acquires an abnormality map in which at least a part of the shape abnormality information and at least a part of the tomographic abnormality information are represented on at least one of a plane along the fundus and a plane perpendicular to the fundus; The display control unit is configured to be able to perform control to display the abnormality map, the information processing apparatus according to Configuration 1. (Configuration 3) The abnormality map acquisition unit includes a shape abnormality map acquisition unit and a tomography abnormality map acquisition unit, The shape abnormality map acquisition unit acquires a shape abnormality map, which is a map representing the shape abnormality information for a first predetermined retinal layer on a plane along the fundus oculi, The tomography abnormality map acquisition unit acquires a tomography abnormality map, which is a map obtained by projecting the tomography abnormality information for a second predetermined retinal layer onto a plane along the fundus oculi, The display control unit is configured to be able to perform control to display the shape abnormality map and the tomography abnormality map, the information processing apparatus according to Configuration 2. (Configuration 4) The abnormality map acquisition unit includes a shape abnormality map acquisition unit and a shape abnormality back-projection region acquisition unit, The shape abnormality map acquisition unit acquires a shape abnormality map, which is a map representing the shape abnormality information for a first predetermined retinal layer on a plane along the fundus oculi, The shape abnormality back-projection region acquisition unit acquires a shape abnormality back-projection region indicating a region corresponding to the shape abnormality map in the tomography image, The display control unit is configured to be able to perform control to display the shape abnormality back-projection region and the tomography abnormality region, the information processing apparatus according to Configuration 2. (Configuration 5) A tomography abnormality region acquisition unit that acquires a tomography abnormality region, which is an abnormal region in a tomography image of the fundus oculi of an eye to be examined, A determination unit that determines a target retinal layer to be processed based on the tomography abnormality region, Based on the tomography image, a shape abnormality map acquisition unit that acquires a shape abnormality map representing information about a portion having an abnormal shape for the target retinal layer on a plane along the fundus oculi, A display control unit that performs control to display the shape abnormality map, An information processing apparatus characterized by comprising (Configuration 6) When the first predetermined retinal layer and the second predetermined retinal layer are not the same, the display control unit is configured to be able to perform control to display information regarding at least one of the first predetermined retinal layer and the second predetermined retinal layer. The information processing apparatus according to Configuration 3. (Configuration 7) The tomographic abnormality map acquisition unit is configured to acquire the tomographic abnormality map by setting the same retinal layer as the first predetermined retinal layer as the second predetermined retinal layer. The information processing apparatus according to Configuration 3. (Configuration 8) The tomographic abnormality information acquisition unit is configured to acquire the tomographic abnormality region in the same retinal layer as the first predetermined retinal layer. The information processing apparatus according to Configuration 3 or 4. (Configuration 9) The shape abnormality map acquisition unit is configured to acquire the shape abnormality map by setting the same retinal layer as the second predetermined retinal layer as the first predetermined retinal layer. The information processing apparatus according to Configuration 3. (Configuration 10) Further includes a attention position designating unit for designating an attention position on the shape abnormality map, The display control unit is configured to be able to perform control to display the tomographic abnormality region and the shape abnormality back-projection region in the tomographic image including the attention position. The information processing apparatus according to Configuration 4. (Configuration 11) The display control unit is configured to be able to perform control to display such that the position of the tomographic image displaying the shape abnormality back-projection region is different from the position of the tomographic image displaying the tomographic abnormality region. The information processing apparatus according to Configuration 4 or 10. (Configuration 12) The shape abnormality map acquisition unit inputs information regarding the shape of the retinal layer acquired from the tomographic image of the fundus of the eye to be examined into a learned model, and based on the difference between the information regarding the shape of the healthy-eye-like retinal layer obtained thereby and the information regarding the shape of the retinal layer acquired from the tomographic image of the fundus of the eye to be examined, is configured to acquire the shape abnormality map The learned model is a learned model according to any one of Configurations 3 to 11, which takes, as input, information on the shape of the retinal layer obtained from a tomographic image of the fundus of the eye, and outputs information on the shape of the retinal layer in a healthy eye for the retinal layer related to the input. (Configuration 13) The information on the shape of the retinal layer is layer thickness or curvature, and the information processing apparatus according to Configuration 12. (Method 1) A tomographic image acquisition step of acquiring a tomographic image of the fundus of the eye to be examined; A shape abnormality information acquisition step of acquiring shape abnormality information, which is information on a portion having an abnormal shape in the retinal layer of the eye to be examined, based on the tomographic image; A tomographic abnormality information acquisition step of acquiring tomographic abnormality information, which is information on a portion having the tomographic abnormality region in the retinal layer of the eye to be examined, based on the tomographic abnormality region that is an abnormal region in the tomographic image; A display control step of performing control to display at least a part of the shape abnormality information and at least a part of the tomographic abnormality information so as to be distinguishable from each other; An information processing method, characterized by including the above. (Configuration 14) A program for causing a computer to execute the information processing method according to Method 1.

Explanation of Signs

[0093] 1 Information processing apparatus 11 Input device 12 Display device 101 Tomographic image acquisition unit 102 Shape abnormality information acquisition unit 103 Tomographic abnormality information acquisition unit 104 Display control unit 105 Storage unit 106 Shape map acquisition unit 107 Abnormality map acquisition unit 108 Tomographic abnormality region acquisition unit 110 Tomographic abnormality map acquisition unit 111 Shape abnormality back-projection region acquisition unit

Claims

1. An OCT image acquisition unit that acquires a tomographic image of the fundus of an eye to be examined; A shape abnormality information acquisition unit that acquires shape abnormality information, which is information about a portion having an abnormal shape in the retinal layer of the eye to be examined, based on the tomographic image; A tomographic abnormality information acquisition unit that acquires tomographic abnormality information, which is information about a portion having the tomographic abnormality region in the retinal layer of the eye to be examined, based on a tomographic abnormality region that is an abnormal region in the tomographic image; A display control unit that performs control to display at least a part of the shape abnormality information and at least a part of the tomographic abnormality information so as to be distinguishable from each other; An information processing apparatus comprising the above, characterized in that.

2. The information processing apparatus according to claim 1, further comprising an abnormal map acquisition unit that acquires an abnormal map representing at least a part of the shape abnormality information and at least a part of the tomographic abnormality information on at least one of a plane along the fundus and a plane in a direction perpendicular to the fundus, wherein the display control unit is configured to be able to perform control to display the abnormal map.

3. The abnormal map acquisition unit includes a shape abnormality map acquisition unit and a tomographic abnormality map acquisition unit, the shape abnormality map acquisition unit acquires a shape abnormality map, which is a map representing the shape abnormality information about a first predetermined retinal layer on a plane along the fundus, the tomographic abnormality map acquisition unit acquires a tomographic abnormality map, which is a map obtained by projecting the tomographic abnormality information about a second predetermined retinal layer onto a plane along the fundus, and the display control unit is configured to be able to perform control to display the shape abnormality map and the tomographic abnormality map. The information processing apparatus according to claim 2.

4. The abnormal map acquisition unit includes a shape abnormality map acquisition unit and a shape abnormality back-projection region acquisition unit, the shape abnormality map acquisition unit acquires a shape abnormality map, which is a map representing the shape abnormality information about a first predetermined retinal layer on a plane along the fundus, the shape abnormality back-projection region acquisition unit acquires a shape abnormality back-projection region indicating a region corresponding to the shape abnormality map in the tomographic image, and the display control unit is configured to be able to perform control to display the shape abnormality back-projection region and the tomographic abnormality region. The information processing apparatus according to claim 2.

5. A tomographic abnormality region acquisition unit that acquires a tomographic abnormality region, which is an abnormal region in a tomographic image of the fundus of an eye to be examined; A determination unit that determines a target retinal layer to be processed based on the tomographic abnormality region; A shape abnormality map acquisition unit that acquires a shape abnormality map representing, on a plane along the fundus oculi, information about a portion having an abnormal shape in the target retinal layer based on the tomographic image; A display control unit that performs control to display the shape abnormality map; An information processing apparatus, characterized by comprising the above.

6. The information processing apparatus according to claim 3, wherein when the first predetermined retinal layer and the second predetermined retinal layer are not the same, the display control unit is configured to be capable of performing control to display information about at least one of the first predetermined retinal layer and the second predetermined retinal layer.

7. The information processing apparatus according to claim 3, wherein the tomographic abnormality map acquisition unit is configured to set the same retinal layer as the first predetermined retinal layer as the second predetermined retinal layer and acquire the tomographic abnormality map.

8. The information processing apparatus according to claim 3 or 4, wherein the tomographic abnormality information acquisition unit is configured to acquire the tomographic abnormality region in the same retinal layer as the first predetermined retinal layer.

9. The information processing apparatus according to claim 3, wherein the shape abnormality map acquisition unit is configured to set the same retinal layer as the second predetermined retinal layer as the first predetermined retinal layer and acquire the shape abnormality map.

10. Further comprising a attention position designation unit that designates an attention position on the shape abnormality map, The information processing apparatus according to claim 4, wherein the display control unit is configured to be capable of performing control to display the tomographic abnormality region and the shape abnormality back-projection region in the tomographic image including the attention position.

11. The information processing apparatus according to claim 4, wherein the display control unit is configured to be capable of performing control to display the tomographic image displaying the shape abnormality back-projection region and the tomographic image displaying the tomographic abnormality region at different positions from each other.

12. The shape abnormality map acquisition unit is configured to acquire the shape abnormality map based on the difference between the information about the shape of the retinal layer of a healthy eye-like obtained by inputting the information about the shape of the retinal layer acquired from the tomographic image of the fundus oculi of the eye to be examined into a learned model and the information about the shape of the retinal layer acquired from the tomographic image of the fundus oculi of the eye to be examined The information processing apparatus according to any one of claims 3 to 5, wherein the learned model takes, as input, information regarding the shape of the retinal layer obtained from the tomographic image of the fundus oculi, and outputs information regarding the shape of the retinal layer in a healthy-eye-like manner for the retinal layer related to the input.

13. The information processing apparatus according to claim 12, wherein the information regarding the shape of the retinal layer is layer thickness or curvature.

14. A tomographic image acquisition step of acquiring a tomographic image of the fundus oculi of the eye to be examined; A shape abnormality information acquisition step of acquiring shape abnormality information, which is information regarding a portion having an abnormal shape in the retinal layer of the eye to be examined, based on the tomographic image; A tomographic abnormality information acquisition step of acquiring tomographic abnormality information, which is information regarding a portion having the tomographic abnormality region in the retinal layer of the eye to be examined, based on the tomographic abnormality region that is an abnormal region in the tomographic image; A display control step of performing control to display at least a part of the shape abnormality information and at least a part of the tomographic abnormality information so as to be distinguishable from each other; An information processing method, characterized by including:

15. A program for causing a computer to execute the information processing method according to claim 14.

Images